One of the bandits reportedly put a gun to Lochte's forehead.
Lochte and fellow swimmers Gunnar Bentz, Jack Conger and Jimmy Feigen had left a hospitality house hosted by France early Sunday in a taxi bound for the Olympic Village.
The vehicle was "stopped by individuals posing as armed police officers who demanded the athletes' money and other personal belongings," said USOC spokesman Patrick Sandusky.
Appearing on NBC, Lochte offered chilling details.
"We got pulled over, in the taxi, and these guys came out with a police badge, no lights, no nothing -- just a police badge and they pulled us over," Lochte told NBC
. "They pulled out their guns, they told the other swimmers to get down on the ground -- they got down on the ground. I refused, I was like we didn't do anything wrong, so -- I'm not getting down on the ground.
"And then the guy pulled out his gun, he cocked it, put it to my forehead and he said, 'Get down,' and I put my hands up, I was like 'whatever.' He took our money, he took my wallet -- he left my cell phone, he left my credentials."
"I think they're all shaken up," Lochte's mother, Ileana Lochte, told USA Today
Lochte released a statement thanking family, friends, and fans for their support.
"While it is true that my teammates and I were victims of a robbery early Sunday morning, what is most important is that we are safe and unharmed," he said.
Bentz tweeted, "We are all safe. Thank you for your love and support."
Rio de Janeiro civil police told CNN they have opened an investigation, a few hours after first learning about the robbery from the media, and had not been contacted by the USOC.
Fellow Team USA swimmer Nathan Adrian said he heard the news about the incident when he woke up, but said he wasn't worried.
"[I'm] certainly very happy that all those boys are safe, you know that's the number one priority on top of everything," he said. "Rio is an amazing city and there is going to be problems anywhere you go."
"We were given briefings in a couple of meetings coming into this," American swimmer Ryan Murphy added. "I think we all trust our security guys and they have done a great job throughout this whole trip with us."
Lochte's mugging comes after a handful of incidents that have brought Rio's safety into question.
A Paralympic athlete was mugged in June
, and a few days before the Opening Ceremonies, Australian Olympians were robbed
after their building was evacuated.
The window of a media van was hit by an unidentified object
-- which one reporter thought was a bullet -- and shattered
And two stray bullets were also found at the equestrian site.
With 12 Olympic medals, Lochte, 32, is the second-most decorated male Olympic swimmer of all time, behind Michael Phelps.