But they can look back to a remarkable showcase in 2016 where records tumbled and some sporting greats made history.
So what will we remember from Brazil? Here are a few moments to savor.
Jamaican sprinter, superstar and crowd pleaser, Usain Bolt, ran into the record books in Rio with his ninth Olympic gold medal.
His final glory came with victory in the 4x100-meter relay
after finishing first in the individual 100m and 200m. He has repeated this astonishing feat at three successive Olympic Games -- a unique "Triple-Triple."
Famous for his cheeky antics and archer's stance after every race, Bolt is an 11-time world champion and the 100/200m record holder -- a brisk 9.58 seconds and 191.19 respectively.
The fastest man on the planet has not retired yet but has declared that Rio will be his last OIympics.
... and Mo twice
Britain's Mo Farah made his own piece of history with a "double-double." Farah defended both his 5,000m and 10,000m Olympic titles in Rio -- with victory in the shorter distance coming on Saturday.
Farah's triumph means he becomes just the second man after Finland's Lasse Viren to win both long-distance events at successive Olympics.
"It's every athlete's dream," Farah told reporters. "I just want to go home now and see my beautiful kids and hang my medals around their necks."
Farah's double gold success helped GB pass the 65-medal total it won when it hosted the games in London in 2012.
Olympic swimming legend Michael Phelps won his 23rd gold medal in the 4x100m medley
relay in Rio. He swam the butterfly third leg that helped the US team break the Olympic record.
Nicknamed the "Baltimore Bullet," Phelps made his fifth Olympic appearance over 16 years, won his fifth medal of the Rio Games and cemented his place as the most decorated Olympian of all.
He now has 28 medals in total but told fans on Facebook that there would be no more.
He was beaten just once in Rio -- in the 100m butterfly -- by Singapore's Joseph Schooling, who had been inspired by Phelps and was photographed with him as a youngster.
Will anyone ever match such an incredible Olympic career?
Fiji won its first ever Olympic medal after an outstanding performance in the rugby sevens tournament -- which made its debut.
The Pacific Islanders were hotly-tipped to win gold and duly crushed Great Britain 43-7 in the final, sparking scenes of joy among the 900,000 population and heaps of praise on social media.
Fiji Times editor Elenoa Baselaia told CNN: "Rugby is like a religion
in Fiji, and we're already a very religious country."
Who can forget the beaming smile of swimmer Yusra Mardini, who won her heat in the 100m butterfly?
Aged 14, Mardini swam for Syria at a world championship in 2012 but her home was destroyed in the ongoing conflict and she fled the country with her sister. She had to swim for her life when a boat capsized during her escape.
"I thought it would be a real shame if I drowned in the sea, because I am a swimmer," is how she summed up the nightmare to reporters this year.
She joined the refugee team at the Rio Games and although she didn't swim fast enough to progress, she managed to embody the spirit of the Olympics with her effort.
10K world record smashed ...
Rio's athletic events had only just begun when Ethiopia's Almaz Ayana stepped on to the track and stunned everyone by obliterating the women's 10,000m record that had stood for nearly 23 years.
She finished in 29 minutes 17.45 seconds, slicing a clear 14 seconds off the old record set by China's Junxia Wang in September 1993.
It was a fast race for those behind her too. Seven set national records and the other medalists recorded the third and fourth fastest times ever in the event.
... and Johnson's 400 goes too
South Africa's Wayde van Niekerk delivered one of the standout performances of the Rio Games when he broke the 400m record -- and from the difficulty of lane eight.
The time of 43.03 seconds shaved a little off the 1999 time set by American track legend Michael Johnson.
Now a pundit for the BBC, Johnson looked a bit tearful in the moment but was very quick to congratulate the new champion, tweeting that it was "well done and deserved."
Van Niekerk revealed that he had been inspired by Usain Bolt during a training camp in Jamaica.
"He told me back in Jamaica that 'you will break the world record.' And he came to me after (tonight) and said 'I told you you can do it.'"
Van Niekerk switched to the 400m in 2012 -- but has yet to fall in love with the event.
"I hate the 400m but look how far it's brought me," he said. "Once you've been blessed with something you need to use it as best as possible. I'm not going to waste it."
Teenage sensation Simone Biles produced an almost flawless routine
to land her fourth Olympic gold at Rio.
The American youngster is now the dominant force in women's gymnastics and underlined her status by winning golds in the team competition
, all-round event
, and the vault
as well as the final contest in the floor exercise.
She told CNN that Tokyo is still a long way off and wants a long vacation before she returns to training.
"We have to still take in this moment and enjoy it," Biles said.
New US swim star takes over from Phelps
Biles wasn't the only American teenager to light up the Rio Games.
In the pool, 19-year-old Katie Ledecky won the 800m freestyle, leaving rivals in her wake. She broke her own world record with a time of 8:04.79, finishing a staggering 11 seconds ahead of Britain's Jazmin Carlin.
Ledecky took home five Olympic medals from Rio, four of them gold and all in freestyle -- also winning the 200, 400 and 4x200m relay, plus a silver in the 4x100m relay.
She set two world records in Rio. Will there be more in Tokyo in 2020?
Cycling couple make Olympic history
British cyclists Jason Kenny and fiancee Laura Trott have become the king and queen of the track -- winning five gold medals at Rio, notching up some national records in the process.
Trott became Britain's most successful female Olympian by adding two golds to the two she won at London 2012.
Kenny also made history -- equaling former teammate Chris Hoy's British record of six gold medals -- after winning a nerve-shredding keirin race in which he was almost disqualified, and bringing his Rio haul to three golds.
The couple, who get married next month, have been invincible on the Olympic track.
Fellow cyclist Bradley Wiggins also became the most decorated British Olympian at Rio when he won in the team pursuit
, bringing his total medal tally to eight -- with five of them being gold.
Sweden breaks Brazilian hearts ...
The Swedish women's football team spoiled the Rio party, silencing the Maracana Stadium, when they beat the hosts on penalties to deny them a shot at gold in the final.
Led by captain Marta, Brazil couldn't find a way past a Sweden team it had beaten 5-1 during the tournament's group stage.
"We feel horrible about the defeat," Brazil's coach Vadao said. "We dominated the game and the strategy of the Swedes was what we expected and they played well."
Sweden subsequently lost the final to Germany -- whose men's team lost to Brazil in Saturday's final.
... but gymnasts delight
Brazil's gymnasts Diego Hypolito and Arthur Mariano made up for disappointment in the football and were overcome with joy when they took silver and bronze in the floor routine.
The result of the contest provided one of the best picture moments of the Rio Games.
GB's Max Whitlock took the gold, but the event will be remembered for the delighted gymnasts' reaction, hugging each other draped in the national flag.
The stadium, of course, was also ecstatic.
Olympic spirit prevails...
Fans cheered and applauded a wonderful display of sportsmanship from competitors in the 5,000m.
New Zealand runner Nikki Hamblin stumbled and fell -- and accidentally brought down American Abbey D'Agostino.
D'Agostino helped Hamblin back to her feet but the American had injured her leg in the incident and she slumped to the track.
Hamblin helped her up and stayed with her and both completed the race.
Hamblin lavished praise on D'Agostino, telling reporters: "That girl is the Olympic spirit right there."
... as do gallant competitors
For every winner in sport there has to be a loser -- but some impress with their determination and refusal to give up in the face of adversity.
Argentina's Juan Martin del Potro had to settle for silver in the tennis final against GB's Andy Murray
but the tennis star put up an almighty four-hour struggle against the double Wimbledon and reigning Olympic champion, winning many admirers who watched the epic match.
The two men embraced at the net, hugging each other at the end of a marathon match.
"Any athlete will give his best to be here and having this (Olympic medal) around my neck is a dream for any sportsman. To have the chance to have two is much more than a dream," del Potro told reporters after the game.
"Tennis is my life. I suffered when I wasn't able to play and today tennis is making me very happy again."
Some notable sporting firsts ...
The Rio Olympics delivered some more debut medal moments to match Fiji's.
It was the first appearance of Kosovo at the Olympic Games and it straight away won gold when Majilnda Kelmendi defeated Italy's Odette Giuffrida in the final of the women's 52 kg judo contest.
Puerto Rico has its first Olympic gold medal after tennis player Monica Puig defeated world number two Angelique Kerber
of Germany in the women's singles final.
And Vietnam won its first Olympic gold when Hoang Xuan Vinh was victorious in the 10-meter air pistol event.
... and seconds and thirds
Three sets of sisters, including triplets, competed in the Olympic marathon.
Estonia's marathon triplets -- Lily, Leila and Liina Luik -- ran with fellow competitors, the Hahner sisters of Germany and the Kim Hye-gyong sisters of North Korea.
None of them managed to come close to a medal ... but it's the taking part that counts.