"Not everyone at the White House has agreed with me," said Hagel of his methodology for deciding on detainee releases.
The outgoing secretary also said the United States could find it necessary to send noncombat troops to the front lines in Iraq to help Iraqi troops in their fight against ISIS.
"We have to look at all the options, and I think it may require a forward deployment of some of our troops -- not doing the fighting, not doing the combat work that we did at one time for six years in Iraq and we did for many, many years in Afghanistan, but to help airstrike precision (locate targets)," he told Starr.
"I would say, though, we're not there yet. Whether we get there or not, I don't know," Hagel added, reiterating his position on the issue.
In November, Hagel acknowledged the possibility of ground forces in Iraq
but said these would not be combat troops.
That same month, the chairman of the Joint Chiefs of Staff,
Gen. Martin Dempsey, told a congressional committee that he would not rule out asking the President to send U.S. ground troops into Iraq. Throughout the hearing, Dempsey and Hagel described a mission in Iraq and Syria that will likely be long and fraught with setbacks.
Disagreements between Hagel and officials in the President's inner circle have been widely reported during his short tenure as secretary of defense. But in his interview with CNN, Hagel spoke with uncharacteristic candor about friction related to the prisoner release.
"We've had a lot of conversations," Hagel said.
"With the White House?" Starr pushed back.
"Yes," said Hagel. "And Congress. And the press."
But Hagel also said he isn't fazed by the pressure on the issue of detainee transfers.
"Because I have the responsibility and I play my own game here," he emphasized. "And that is because, by law, I am the one official in government charged with certification of release of detainees. I take that responsibility very seriously."
Debate about detainee transfers was rekindled Thursday when news emerged that one of the five former prisoners freed to Qatar in exchange for Bergdahl's release by the Taliban may have attempted to rejoin the fight by making contact with suspected Taliban associates in Afghanistan.
The administration insists that the released detainees are being monitored, and officials remain confident the potential risks are mitigated by the measures they have in place.
In his interview with CNN, Hagel echoed that view, and said he has no regrets about making the deal.
"Absolutely it was the right decision," he said. "It was the right decision ... because we don't leave our troops behind."
"It was a prisoner of war exchange and I'm absolutely as committed to that decision today as when the decision was made." he added. "It was the right decision."
And when the White House ramped up pressure, Hagel said he knew how to take the heat.
Said Hagel: "If you're not prepared to deal with pressure, every day, in the job you're in, coming from a lot of different directions, then you shouldn't be in the job. It's just part of the process, part of any job I think."
Hagel's candor as he prepares to leave the Pentagon was evident on other issues as well.
He was blunt in addressing the budget cuts the military has grappled with in recent years as the wars in Iraq and Afghanistan draw to a close.
"I am concerned about the future of our force, the quality of that force," said Hagel, adding that he hears frequently from soldiers who worry they won't have the same career prospects in the military they'd have in the private sector.
About half of those soldiers say they plan to leave the military as a result, he added.
"I don't think Washington understands the depth of that issue because that's a difficult quantifiable issue," Hagel said. "It's not a budget line."
The secretary also addressed the ongoing fight against ISIS in Syria and Iraq, where the United States is leading an international coalition to take on the terror group.
ISIS poses a threat to U.S. interests, Hagel said, but the problem cannot be solved using military power alone.
"Big societal problems -- historic injustices, conflicts -- don't get solved militarily," said Hagel. "There is a military component to it, of course. But it's going to require -- always does and always will -- political settlements."
And that military component comes with risks, Hagel acknowledged, even when the United States doesn't commit ground combat troops.
"Any time you introduce American military power anywhere in the region ... there's always risk," he said. "There's risk to our people in Iraq, to our people in Afghanistan, to our people who are conducting strikes in Syria."
It's a risk the secretary understands well, having been a soldier during the Vietnam War.
"I think it has helped me a tremendous way to connect with second- and third-level issues of the men and women and their families who make the sacrifices," Hagel said of his experience in the military. "I think I understand it."
"It doesn't mean I'm any better," he added. "It doesn't mean I'm any smarter. But I think I understand it maybe in ways that I might not otherwise unless I had served."
From soldier to secretary of defense -- it's a transition even Hagel could not have imagined for himself during his time in uniform.
"Had I been told 48 years ago when I joined the Army I would end up being secretary of defense," he said, "I just would have laughed."