That was decades ago. But last week, news that a gunman spewing racist hatred attacked a church in his community took him right back to that day, when he had returned from fighting in Korea and was told he had no right to be inside a bus-station waiting area in his own country.
The 80-year-old longtime activist and founder of the Committee On Better Racial Assurance says he means it when he calls himself a troublemaker in Charleston.
"I've made a lot of enemies here," he says. "And I say I made them because I came after them before they came after me."
But the shooting at Mother Emanuel, he says, has pushed him toward a haunting realization.
"I've been crying a lot, a hell of a lot," he says.
"I keep playing back tapes in my head and realizing that we have not come anywhere."
His whole life -- ever since that day at the bus station -- Saunders says he's been fighting to be seen as a man. It's a goal that he says feels out of reach now, more than ever, in this state where the Confederate flag still flies proudly and a deranged racist can still slaughter innocent people in a brazen church attack.
"One of the things I wanted all my life is to be a man, not a black man, but a man. There's no way I can ever be a man in Charleston, South Carolina," Saunders says. "I will always be a boy to the system.
"I really wanted to die free. But that's not going to happen in my lifetime."
Bridging the divide
Speaking shortly after the shooting this week, Charleston Mayor Joseph Riley vowed that his city would remain resilient and come together.
"This hateful person came to this community with some crazy idea that he would be able to divide," he said. "And all he did was make us more united, and love each other even more."
But talk to some of the area's most battle-hardened civil rights activists and you'll hear hints of a more complicated story
The shooting, they say, has exposed wounds that were long lurking beneath the surface in a city where racial divides remain a reality -- hidden, but unhealed.
Catalyst for change
"History is sort of repeating itself. ... It's disturbing," says William Pugh.
For the soon-to-be high school senior, the fight against racism in his home state has become more than just a chapter in a history book.
Last year, the football coach at his North Charleston high school was fired after word spread that the team had a ritual it followed after defeating predominantly African-American schools: smashing watermelons with caricatures drawn on them while making monkey noises.
Students spoke out afterward -- but not in the way Pugh expected. A lot of them, he says, were quick to criticize the school administration for firing the coach.
That's when Pugh joined the fray, writing a letter to the editor of his school newspaper.
"No one has stood up to tell them that their actions were wrong, and it's time for that to change," he wrote.
After this week's shooting, Pugh says he sees hope in the tragedy.
"I just hope that this can be used as catalyst for change," he says, adding that it's time for young people to pick up where past generations left off.
"It's going to take their wisdom," he says, "but our energy."
'We are still being violated'
In a wooden pavilion, behind an orphanage that's helped young African-American children in the Charleston area for more than a century, gospel music plays as the faint smell of ocean water lingers in air that is heavy with history.
This warm Saturday afternoon was supposed to be a day for joyful celebration: the annual marking of Juneteenth -- the day a century and a half ago when the last slaves in the United States finally gained their freedom.
When the shooting at Charleston's Mother Emanuel
sent shock waves through their community, the organizers debated whether they should cancel.
Sara J. Nesbit knew that wasn't an option.
So in addition to a festival with traditional music and dancing, she booked guest speakers to talk about the impact of the shooting.
It was a natural fit, Nesbit says. Juneteenth, after all, celebrates the emancipation of a people whose rights were violated.
"It all ties in," she says. "We are still being violated."
A woman comes to the podium and says she's proud to be from Charleston, but adds that the area's residents from different cultural backgrounds need to take more steps to find common ground.
"We don't know each other if we don't sit down with each other and hear each other's stories," she says.
A few minutes later, Nation of Islam Minister DeAndre Muhammad warns the crowd not to be complacent.
"This city will not have a race war, they say. But we must realize that we were victims of war as soon as the soles of our forefathers' feet set foot on the United States of America," he says. "We are casualties of war."
The gunman in last week's attack clearly picked his target with history in mind, says Damon L. Fordham, and history should be part of how we process what happened.
The shooting at Mother Emanuel
, the historian says, harkens back to the high frequency of attacks on black churches decades ago.
But for Fordham, who's spent years studying African-Americans in South Carolina, the issues at play in this most recent attack go far beyond the state's borders.
"Let's not kid ourselves. The problems are not geographic. ... There are no utopias on this Earth. This is not strictly a race problem," he says. "This is also a human problem and a global problem."
'I saw it coming'
The shooting at Mother Emanuel
devastated Minerva Brown King. But somehow, she wasn't surprised.
A week ago, the 70-year-old high school librarian was telling her friends at a church book group that something serious was simmering beneath the surface in their city, and something terrible was bound to happen soon.
"I saw it coming. I heard it. I smelled it. I felt it coming," she says. "I've been in this community long enough to feel the rumblings, the lack of true communication."
She wishes she hadn't been right. She knew four of the nine people killed in Wednesday's attack.
"It's just so bad that we have to live through tragedies like this to get people's attention," she says. "It's right there in front of us. We've just ignored it for so long."
It's been more than five decades since King sat at the S.H. Kress & Co. five-and-dime store in Charleston with her high school classmates, watching a waitress pour ammonia over the counter
in an attempt to get out of serving them, in what became the city's first civil rights sit-in.
It's been decades since she saw crosses burning on her family's front lawn as nightriders targeted her father, the head of Charleston's local NAACP chapter.
It's been decades since she met in marathon sessions with groups of civil rights activists inside Mother Emanuel, making picket signs and planning their next move.
And until recently, it had been decades since she'd marched.
Her concerns about what she says are Charleston's increasingly segregated schools brought King back to the streets several months ago. Then the April shooting death of Walter Scott
, which drew national attention after a video surfaced showing a police officer gunning him down, further fueled her fervor.
Now, she frequently finds herself wearing a T-shirt that says, "DO YOU BELIEVE US NOW?!" in bold white block letters on a black background. The message, she says, refers to complaints North Charleston's African-American residents made for years about excessive police force
. It wasn't until the video of the police shooting emerged, she says, that anyone took them seriously.
You don't have to travel far to see reminders of the struggles in Charleston's past. The longtime site of slave auctions is now a city-run museum. And signs in some of the city's most popular tourist areas document civil rights landmarks.
On the surface, it might seem the city's racial divides are relics of the past. But it's not the obvious signs that you have to worry about, King says.
It's the more insidious ones, according to King -- like increasingly segregated schools
and gentrification that's drastically changed the city's demographics
, sending most African-Americans into neighboring North Charleston.
Politicians claim the Mother Emanuel
attack "is the act of one deranged person," she says, "and that is so untrue."
Officials, she says, are still fostering a climate that feeds hate in Charleston -- and beyond.
"I want the governors and the mayors to start taking some responsibility for how they have contributed to the situation," she says.
Until they do, and until people start being more honest when talking about race, King says, she'll keep marching.
Right now, she says, it makes sense that people are flocking to the historic church in mourning. But King is ready to shift gears.
"That can't be the end of the struggle," she says. "That should be the beginning."