PHILADELPHIA, Mississippi (CNN) -- James Young still remembers the Ku Klux Klan tormenting his neighborhood. He can still see his father holding a gun on the living room couch ready to shoot anyone who threatened his family.
James Young poses with one of his young supporters after winning this week's election.
Nothing about Young's childhood ever made him think he could be the mayor of Philadelphia, Mississippi, the town best known for the killings of three civil rights workers in 1964.
That's the way it was for black kids growing up in this crucible of racial hostility -- big dreams were often squelched. Sitting on a sprawling Southern front porch this week, Young broke down in tears about what it means to be elected the town's first black mayor.
"When you've been treated the way we've been treated," he told CNN, choking up and then pausing to wipe the tears from his face. Watch tearful Young describe victory »
For a moment, he couldn't speak. He then regrouped, "That's why it's so overwhelming to be a part of this history."
In May, the 53-year-old Young was elected the mayor of Philadelphia, a town of about 8,000 in the east-central part of the state. Despite a 55 percent white majority, Young defeated Rayburn Waddell, a white, three-term incumbent, by the slim margin of 46 votes. See where the town is located »
Young described the victory as "an atomic bomb of change." Another resident rejoiced, saying Young's win symbolized the scab finally falling off this town's wound.
"I couldn't even have wrote that in a fairy tale," Young said. "Who would have thought a little country boy like me would be mayor of Philadelphia, Mississippi?"
Philadelphia was the site of one of the most notorious killings of the civil rights era. On June 21, 1964, three civil rights workers -- James Chaney, a 21-year-old black man from Mississippi; Andrew Goodman, 20, and Michael Schwerner, 24, both activists from New York -- were shot to death at the edge of town. The killings inspired the 1988 movie "Mississippi Burning."
"Philadelphia will always be connected to what happened here in 1964," said Jim Prince, the publisher of the Neshoba Democrat newspaper.
"But the fact that Philadelphia, Mississippi, with its notorious past, could elect a black man as mayor, it might be time to quit picking on Philadelphia, Mississippi."
Young knows his slim margin of victory means he still has to earn the trust of many more voters here. He knows there are still some in town who won't vote for him because he's black, but he says that number gets smaller and smaller as time passes.
"We have some -- a very small pocket -- that will never change. That's what we've got to deal with," said Young.
The mayor-elect says his election symbolizes a dramatic shift away from his hometown's racist past. And for many black residents, it means they can finally call this place home.
"The places where we were locked out, I'm gonna have the key," he said. "The places we couldn't go, I've got the key. No better way to say it than that."
He takes special pride that his victory comes the same year the nation swore in its first African-American president in Barack Obama. "It inspired people," Young said. "There are times and seasons, I think, for everything. The season arrived and the people let me know it was my time."
The mayor-elect says he won by shaking hands and knocking on doors all over town. But the groundwork for Young's climb to the top of Philadelphia's political world started decades ago.
Young was one of the first black students to integrate Philadelphia's white schools. After graduating from high school, he worked in a motor factory and then as a hospital housekeeper.
A white boss noticed Young's charming people skills and recommended that he become a paramedic. He eventually worked his way up to become the director of the EMT unit, and that catapulted him to his first elected job as a county supervisor in 1991.
He is also a Pentecostal minister preaching on Sunday and organizing weekly Bible studies.
"I've been prepping for this. I felt like I knew enough people. I felt like they knew me and that if I could convince them to just give me the opportunity, things could happen," said Young.
Driving around Philadelphia in a 1981 Ford pickup truck, Young basked in the glow of victory. He calls it the "honeymoon" period. As we drove down the road, black and white residents cheered.
"We're so happy," screamed one lady.
Young shouted back, "We did it!"
Until he's sworn in as mayor, Young will work out of a makeshift transition office provided by a prominent attorney. His victory might seem unlikely but there's little time left to celebrate.
"It's an awesome feeling to have that kind of respect that people support you in this way," Young said near the end of our interview. "I'll never let the people down which called for that."
All About Civil Rights • Mississippi