Phelps swam the butterfly third leg of a tight race, pushing the US from second behind Great Britain up to first before freestyle specialist Nathan Adrian saw it home for gold in an Olympic record time of three minutes 27.95 seconds.
The win guaranteed the "Baltimore Bullet" his 23rd gold medal in five Games appearances stretching over 16 years, not to mention a fifth win at Rio 26
after success in the 4x100m freestyle, 4x200m freestyle, 200m butterfly and 200m individual medley.
Phelps, who now has 28 Olympic medals in total, had earlier told fans in a Facebook Live chat
that Saturday would be the last time he graced the Olympic pool.
And the most decorated Olympian of all time was clearly out to leave a valedictory flourish.
Phelps entered the pool with the US trailing Great Britain after Ryan Murphy's opening backstroke and Cody Miller's breaststroke leg.
Two lengths later and the US was in the clear and heading for gold followed by Team GB which would eventually claim silver and Australia bronze.
It was the bookend on an outstanding career, the swimming meet at Rio 2016 and yet another historic night for the American swim team.
Less than an hour before, its women's 4x100m relay medley team made their country the first to win 1,000 gold medals at the Summer Olympics.
Simone Manuel comfortably anchored her team to victory, seeing off Australia in second and Denmark in third.
And although teammates Lilly King, Kathleen Baker and Dana Vollmer had all won Olympic medals before, they celebrated excitedly among themselves as Manuel moved further clear of the field.
It seemed only apt that Phelps and his teammates would extend that record to 1,001 golds shortly after.
Elsewhere in the pool Saturday, Pernille Blume of Denmark took gold in the women's 50m freestyle, edging out 100m freestyle winner Manuel -- who finished second.
Meanwhile, two Italians made the podium in the men's 1,500m freestyle.
Gregorio Paltrinieri claimed gold while his compatriot Gabriele Detti had to settle for bronze after being pipped to silver by Connor Jaeger of the US.