Editor's note: Read a version of this story in Arabic.
(CNN) -- It was perhaps the most contentious moment during a House Armed Services Committee hearing filled with them.
Rep. Jeff Miller, R-Florida, appeared to strike a nerve Wednesday when he pressed Defense Secretary Chuck Hagel on why it was taking the U.S. military so long to interview Army Sgt. Bowe Bergdahl, the soldier being treated in Germany after the Taliban released him May 31.
Hagel already had fielded lawmakers' accusations that the White House overstepped presidential authority and negotiated with terrorists to gain Bergdahl's freedom.
The defense chief started to explain that doctors treating Bergdahl will decide when he is ready to talk, but Miller cut him off.
"Wait a minute. Wait a minute. Why hasn't he been returned to the United States?" Miller asked, noting that wounded soldiers from battle zones get flown back for treatment within days. "You're trying to tell me that he's being held at Landstuhl, Germany, because of his medical condition?"
Hagel responded, "Congressman, I hope you're not implying anything other than that," as the two continued to talk over each other.
"I don't like the implication of the question," Hagel said at one point, and when Miller persisted about injured combat troops coming back to the United States quickly, Hagel raised his voice.
"This isn't just about a physical situation," he said. "This guy was held for almost five years in God knows what kind of conditions. ... This is not just about can he get on his feet and walk and get to a plane."
It was that sort of day. The Bergdahl swap at the end of last month has ignited a controversy in the partisan frenzy permeating Washington in an election year.
Hagel often appeared frustrated and even angry at times but stayed on-message after an opening statement that laid out the legal and ideological foundation for the agreement that the Taliban claim as a victory.
"War, every part of war, like prisoner exchanges, is not some abstraction or theoretical exercise," Hagel said early on, presenting a theme he repeated throughout the hearing. "The hard choices and options don't fit neatly into clearly defined instructions in 'how to' manuals. All of these decisions are part of the brutal, imperfect realities we all deal with in war."
'We complied with the law'
In defending the deal approved by President Barack Obama, Hagel declared that "we complied with the law, and we did what we believed was in the best interests of our country, our military and Sgt. Bergdahl."
"The President has constitutional responsibilities and constitutional authorities to protect American citizens and members of our armed forces. That's what he did," Hagel said. "America does not leave its soldiers behind. We made the right decision, and we did it for the right reasons -- to bring home one of our own people."
Republican critics contended the cost was too high because it freed hardened terror suspects possibly to rejoin the fight against U.S. interests, especially in return for a soldier accused by some in his unit of deserting his post.
Legislators from both parties also complained that the Obama administration failed to notify Congress ahead of time, as required by the National Defense Authorization Act for transfers of Guantanamo detainees.
Rep. Buck McKeon, R-California, the committee chairman, labeled as "misleading and at times blatantly false" White House officials' explanations at a classified briefing Monday for making the exchange and not notifying Congress beforehand.
'We could have done a better job'
Hagel conceded the administration should have tried harder to let Congress know about what he called rapidly moving developments, saying he understood the frustration caused by the speed of the mission.
"We could have done a better job of keeping you informed," Hagel said, noting that only a few hours elapsed between making the final arrangements with the Taliban through the government of Qatar, which mediated the agreement.
"As the opportunity to obtain Sgt. Bergdahl's release became clearer, we grew increasingly concerned that any delay, or any leaks, could derail the deal and further endanger" him, Hagel said.
Under the swap, the released Taliban detainees were taken to Qatar and will remain there for a year, with the Qatari government saying it will keep an eye on them.
McKeon said he worried the newly freed Taliban figures "still pose a threat to Americans and Afghans alike" because "in a year, they will be free to return to Afghanistan."
Other GOP legislators accused the administration of acting on its own to evade opposition to the deal in Congress as well as trying to hide a shift in U.S. policy by negotiating with terrorists to secure Bergdahl's release. They seized on Hagel's confirmation that Bergdahl at times was held by the Haqqani network, a Taliban wing designated a terrorist group in 2012.
Negotiating with terrorists?
Hagel repeatedly said the on-off negotiations dating back to 2011 involved the broader Taliban that formerly governed Afghanistan, and that the government of Qatar was an intermediary in the final talks in April and May.
"We didn't negotiate with Haqqani," Hagel said, adding that Bergdahl was moved around during his five years of captivity. That brought accusations of deceptive reasoning, with Rep. Mike Turner, R-Ohio, sharply challenging Hagel.
"How is it the United States could've been in negotiations with the Haqqani network, a listed terrorist organization, and it not conflict with our policy that we do not negotiate with terrorists?" Turner asked, to which Hagel replied that "we dealt directly with the government of Qatar."
Turner shot back: "So now, the new policy of this administration is, 'we don't negotiate with terrorists directly.' "
The next legislator up, Rep. Jim Cooper, D-Tennessee, criticized Turner for being "prosecutorial" and wondered aloud if his Republican colleague might be running for House GOP leader after Rep. Eric Cantor's primary defeat.
Missing since 2009
Bergdahl went missing on June 30, 2009, in Afghanistan's Paktika province, where he was deployed with the 1st Battalion, 501st Infantry Regiment, 4th Brigade Combat Team, 25th Infantry Division.
An Army fact-finding investigation conducted in the months after his disappearance said he left his outpost deliberately and of his own free will, according to an official briefed on the report.
The Army has no definitive finding that Bergdahl deserted because doing so would require knowing his intent -- something Army officials couldn't learn without talking to the soldier, a U.S. military official told CNN.
On Wednesday, The Washington Post -- citing close friends of Bergdahl's -- reported he had been discharged from the U.S. Coast Guard before his Army enlistment because of psychological issues.
Bergdahl served for about a month in the Coast Guard before being released on an "uncharacterized" and "administrative" discharge, defense officials told CNN.
But CNN has been unable to confirm the reason for the discharge.
The Post reported Bergdahl had told longtime friend Kim Harrison that he had faked the problems but that she doubted his explanation.
"I said, 'You don't fake a psychological discharge, you have to become unfit.' I told him that," the Post quoted her as saying. "The reality was it wasn't okay. I saw it in the letters, the way the writing was changing, the anger."
Some of those emotions are on display in excerpts of Bergdahl's writings the Post published. Harrison said she received the writings -- contained in a journal and on a laptop computer -- in a package from Bergdahl before he disappeared.
She shared the writings to counter a growing perception of her friend as a deserter, the Post reported.
"These thoughts insist on trying to overwhelm my mind. ... I've spent a lot of my life thinking blackness was all I had in front of me, that it would be blackness to the very last instent (sic). I know this is not right. I know that there is light in this darkness, and that I can actuly (sic) reach it if I keep walking, keep moving to it," Bergdahl wrote, according to the Post.
Bergdahl is "continuing to improve every day" as he recovers at Landstuhl Regional Medical Center in Germany, Pentagon spokesman Col. Steve Warren said this week, but the soldier hasn't spoken with his family yet.
When he is ready, Bergdahl will be flown to the San Antonio Military Medical Center in Texas, where he may be able to reunite with his parents.
CNN's Michael Pearson, Mariano Castillo, Paul Steinhauser, Catherine Shoichet, Virginia Nicolaidis, Kevin Liptak, Barbara Starr, Elise Labott, Qadir Sediqi, Dana Bash and Deirdre Walsh contributed to this report.