(CNN) -- Army Sgt. Bowe Bergdahl is on a flight from Europe and is scheduled to arrive in San Antonio, Texas, within hours.
Bergdahl had been in a military hospital in Landstuhl, Germany, since he was released in Afghanistan after five years of Taliban captivity.
He is expected to be taken to a military hospital in the Texas city for continued rehabilitation.
"I can confirm that he is now en route to the United States aboard a U.S. military aircraft, having departed Ramstein Air Base earlier this afternoon," Pentagon spokesman Rear Adm. John Kirby said in a statement Thursday. "He will arrive in San Antonio early tomorrow morning, where he will continue the reintegration process at Brooke Army Medical Center."
"Our first priority is making sure that Sgt. Bergdahl continues to get the care and support he needs."
Previously, three U.S. officials had told CNN that the soldier would arrive in Texas at midnight local time (1 a.m. ET).
Bergdahl was exchanged on May 31 for five Taliban figures detained at the U.S. military prison at Guantanamo Bay, Cuba.
The swap sparked a controversy that touches on a range of volatile issues -- ending the Afghanistan war, releasing Guantanamo detainees, accusations by members of Bergdahl's unit that he deserted -- and it comes in an election year that has further widened the partisan divide in Washington.
Defense Secretary Chuck Hagel has defended the deal, saying it was Bergdahl's last and best chance to get out. Critics have said the cost of the swap was too high, and that Congress should have been notified in advance.
Bergdahl is "continuing to improve every day," Pentagon spokesman Col. Steve Warren said this week. The solider had not spoken with his family.
The "decision to speak with the family is a decision the returnee has to make when he or she is emotionally in the right place to make the phone call," Warren said.
Bergdahl went missing on June 30, 2009, in Afghanistan's Paktika province, where he was deployed with the 1st Battalion, 501st Infantry Regiment, 4th Brigade Combat Team, 25th Infantry Division.
An Army fact-finding investigation conducted in the months after his disappearance concluded he left his outpost deliberately and of his own free will, according to an official who was briefed on the report.
The Army has no definitive finding that Bergdahl deserted because that would require knowing his intent -- something officials couldn't learn without talking to the soldier, a U.S. military official told CNN.
As Bergdahl recovers, details are slowly emerging about the conditions he endured during five years of captivity.
Citing an American official, The New York Times has reported that Bergdahl told medical staff that the box he was kept in for weeks at a time was pitch black and like a shark cage.
CNN has reported that Bergdahl has said he was kept in a small box after trying to escape, according to a senior U.S. official, who spoke on the condition of not being identified. The official also told CNN that Bergdahl suffers from psychological trauma caused by physical abuse.
In his own words
The exact conditions of his captivity are not known, and Bergdahl has not spoken publicly since his release.
But on Thursday, The Daily Beast published two letters the solider reportedly wrote to his family while imprisoned by the Taliban. The website obtained the letters by "sources in contact with the Taliban." It confirmed their authenticity with U.S. and Western officials.
In the letters -- dated 2012 and 2013 -- Bergdahl discusses his life and partially explains why he disappeared in 2009.
"To my friends & family, in regards to the circomestance (sic) here, I am as well as can be here," he wrote. "I am given food and drink."
Bergdahl had harsh words for Army leadership.
"Leadership was lacking, if not non-existent. The conditions were bad and looked to be getting worse for the men that where actuly (sic) the ones risking thier (sic) lives from attack," he wrote.
"If this letter makes it to the U.S.A., tell those involved in the investigation that there are more sides to the cittuwation (sic)," Bergdahl said. "Please tell D.C. to wait for all evadince (sic) to come in."
Hagel on Bergdahl deal
Hagel appeared Wednesday at a House Armed Services Committee hearing to defend the swap, where Rep. Jeff Miller, R-Florida, appeared to strike a nerve when he pressed the defense chief on why it was taking the U.S. military so long to interview Bergdahl.
Hagel already had fielded lawmakers' accusations that the White House overstepped presidential authority and negotiated with terrorists to gain Bergdahl's freedom.
Hagel started to explain that doctors treating Bergdahl will decide when he is ready to talk, but Miller cut him off.
"Wait a minute. Wait a minute. Why hasn't he been returned to the United States?" Miller asked, noting that wounded soldiers from battle zones get flown back for treatment within days. "You're trying to tell me that he's being held at Landstuhl, Germany, because of his medical condition?"
Hagel responded, "Congressman, I hope you're not implying anything other than that," as the two continued to talk over each other.
"I don't like the implication of the question," Hagel said at one point, and when Miller persisted about injured combat troops coming back to the United States quickly, Hagel raised his voice.
"This isn't just about a physical situation," he said. "This guy was held for almost five years in God knows what kind of conditions. ... This is not just about can he get on his feet and walk and get to a plane."
CNN's Ed Payne and Mariano Castillo contributed to this report.