Carter, 31, has appealed to the Court of Arbitration for Sport
against the decision to strip him and the Jamaican team of their 2008 Olympic 4x100 meters relay gold and is awaiting a hearing.
But, as it stands, Bolt can no longer claim to have completed the "triple triple" of nine successive Olympic sprint titles.
Bolt, who once said returning one of his gold medals would be "heartbreaking," is philosophical about the matter -- though he says he is yet to speak with Carter.
"It's just one of those things that happens in life," the 100m and 200m world-record holder told CNN before the Laureus World Sports Awards in Monaco last month.
"Until I see him I can't really say he did it on purpose or it was a mistake or I should be angry. Until I actually sit down and look at him face-to-face and talk to him, (I) don't know how I'm going to react."
'I've done my part -- I've won three times'
In January, the International Olympic Committee announced it had disqualified Carter after a reanalysis of the sprinter's urine and blood samples from Beijing resulted in a positive test for the prohibited substance methylhexaneamine.
At last August's Rio Olympics, Bolt cemented his status as an all-time great by becoming the only man to win all three sprint events at three Olympic Games -- but the decision to punish the Caribbean island's relay team now leaves him with eight Olympic golds.
The loss of that 2008 medal has not, the 30-year-old says, changed his mind about retiring at the end of this season.
"If it had come before the Olympics, maybe it would have taken a little bit away from me and I would have thought about it," he said of competing at Tokyo 2020.
"But the fact that I got the chance to say 'the triple triple,' it kind of made me feel good."
Key to Bolt's retirement decision was the workload involved in preparing for major races and events.
"To do four more years of this, it's tough. It's not easy to motivate yourself year after year. I've accomplished all I've wanted to so it would be very hard to motivate myself."
He also remains phlegmatic about about losing more relay golds.
"When it comes to individual gold medals, I've done my part. I've won three times. If I lose all of my relay gold medals, for me, I did what I had to do with my personal goals and that's what counts for me.
"I did the work. My team came through and they helped, but if something goes wrong and I lose my relay medals, it's just one of those things."
Playing football and traveling the world
Bolt, who plans to hang up his running spikes after August's World Championships in London, says he is looking forward to traveling the world and watching sporting events he had been unable to attend because of his training schedule.
"There have been NBA games that I would love to go courtside to watch," Bolt said. "There have been so many things that I wanted to travel or to go and see. I haven't had a chance to see Formula One ever and that's something that I've always (wanted to do).
"These are the things I look forward to when I can travel and want to see big events, sporting events."
The Manchester United fan also said plans were "in the pipeline" for him to train with Bundesliga side Borussia Dortmund once the athletics season concluded.
If that plan comes to fruition, Bolt might be called upon for one final sprint test -- against the German football team's very own speedster Pierre-Emerick Aubameyang.
In November, the Gabon international threw down the gauntlet to Bolt.
"I'm waiting for you man," Aubameyang exclusively told CNN, challenging him to a 30-meter showdown. "I hope to see you one day, and let's do this challenge!"