(CNN) -- Dave Chappelle wants to clear up a misconception: He never quit "Chappelle's Show," his hit Comedy Central series.
"Technically, I never quit," he said in an appearance on "The Late Show with David Letterman." "I'm seven years late for work."
Chappelle's television appearance was a rarity. Since the last episodes of "Chappelle's Show" aired in 2006, he's done just a handful of TV interviews, the last with Conan O'Brien in 2008, when O'Brien was still hosting "Late Night" on NBC. Chappelle has talked about a variety of reasons for ending the program, including burnout and the inability to devote time to his standup career.
He hasn't been a hermit, however. Last year he participated in Funny Or Die's Oddball Comedy & Curiosity Festival, and he also has dropped by comedy clubs to perform.
With Letterman, Chappelle -- cracking wise and looking sharp in a gray windowpane suit -- talked about disappearing to South Africa during the height of the controversy about the show's future and the truckloads of money Comedy Central was rumored to have offered him to continue.
"It was like living in the corner of perception and reality. It's a weird place to be," he said of the rumors surrounding his Africa trip. Rumors have a way of taking on a life of their own, he added: "When everyone thought Wesley Snipes was in jail, I saw him at a party."
As for the Comedy Central deal, said to be as high as $50 million? Well, money can't buy everything, Chappelle told Letterman. Sometimes he wonders about the "what ifs"; other times he's fine with the way things have gone.
He said he was at a restaurant with his wife and saw a man across the room. "He has $100 million, and we're eating the same entree," Chappelle told Letterman. Even though he may have less money than the $100 million man, "the difference in lifestyle is miniscule."
He summed it up, literally.
"The only difference between having $10 million and $50 million," he started -- and took a long pause -- "is an astounding $40 million."