(CNN) -- With his Southern accent, background as a two-term governor and varied political experience, Mississippi Gov. Haley Barbour thought he might be the perfect candidate to go up against President Barack Obama.
For the past few months he explored whether he might have enough support for a 2012 White House run, road-testing his message in key early-voting states with the kind of stops any presidential wannabe must make.
In New Hampshire, he went to a gun shop and met voters at a diner, answering questions ranging from his position on gun control to the budget deficit; he courted evangelical Christians at events in Iowa, touting his socially conservative positions; he visited South Carolina towns, hoping that the state could be a key base of support for a Southerner like himself; and he ventured to Florida to meet with state legislators and vowed that if he ran, the Sunshine State would be key to his strategy.
As he explored a campaign, the question was whether a Southern governor with experience ranging from serving in the Reagan White House to chairing the Republican National Committee to forming a high-powered Washington lobbying firm could be competitive in the fight for the GOP nomination -- especially as many Republican voters might be looking for a fresh face to confront Obama.
"I do think the American people are looking for plain-spoken and common sense. They are looking for somebody to tell them the truth when it is not popular," Barbour told reporters earlier this month in New Hampshire.
"I think the American people are sick of happy talk, and I also think they are sick of the idea that ... anybody that disagrees with the president has the wrong motives."
Known as one of the smartest political minds among Republicans, Barbour, with a large Rolodex, spearheaded a successful fundraising drive last year as head of the Republican Governors Association, helping the GOP capture 11 governorships held by Democrats.
But a major reason Barbour decided not to run in 2012 was concern among activists and donors being recruited that continuing controversy over race could come to dominate the campaign, a source close to the Mississippi governor told CNN.
In December, Barbour told the Weekly Standard magazine, "I just don't remember it being that bad," recalling his youth in the segregated South. He later issued a statement explaining he believed his hometown of Yazoo City, Mississippi, was less tolerant of the Ku Klux Klan than other parts of the state but also admitted town leaders were not saints.
In March, the governor announced he would support the creation of two museums for the state -- one focusing on Mississippi's history and the other on the civil rights movement.
Barbour had tried to address the controversy during his recent campaign-style appearances and get ahead of it. But to be able to defeat Obama, the right candidate has to make the election a referendum on the incumbent and not become a target himself, the source said.
"The issue of being a Southerner and race could have and would have dominated the general election" if Barbour won the nomination as well as the Republican primary, the source added. The source said those issues were deciding factors but not the only ones that led to Barbour not to run.
It came down to a "gut feeling," a another source close to Barbour said. "He just wasn't quite there," the source said Monday. "It wasn't one particular thing. To run for president obviously is a big decision."
Barbour said as much Monday when he announced that he wasn't running in 2012. "I will not be a candidate for president next year," he said in a statement. "This has been a difficult, personal decision, and I am very grateful to my family for their total support of my going forward, had that been what I decided."
By exploring a possible candidacy, Barbour hoped to gain some traction by attacking the White House on economic issues.
"What I learned as governor of Mississippi is that winning the future doesn't start in Washington, D.C.," he told the Chicago Chamber of Commerce in March, a reference to a White House initiative. "It won't be accomplished through government boondoggles like taxpayer-subsidized high-speed rail or other pet projects. It can't be achieved by having government take control of our automakers, financial sector, health care system and energy industry."
But his efforts did not seem to generate a lot of excitement. When Republicans were asked their choice for a nominee in a CNN/Opinion Research Corp. poll taken April 9-10, Barbour received less than one half of 1 percent. He did win one straw poll in Charleston County, South Carolina, but didn't do well in some others.
While he thought he might find some strength with the conservative base in Iowa and South Carolina, his lobbying career and some of his firm's previous clients were considered a hindrance by some political experts. He would deflect questions regarding that by saying his lobbying background would help him be a better advocate since that's what presidents do for a living.
As he tested the waters, he touted his executive experience and his record as governor -- passing a major tort reform bill, overhauling the Medicaid system and luring businesses to the state.
"We didn't raise taxes," he told the U.S. Chamber of Commerce. "While we didn't raise taxes, my first four years as governor, state revenue went up 40%."
He told CNN affiliate KRNV-TV, "I have been more involved in economic development in our state and more successful than almost any other governor."
Barbour acknowledged there was more to do in his state. Today, Mississippi is one of the poorest in the nation, with its health care and schools ranked as some of the worst.
His potential candidacy also did not exactly receive a ringing endorsement from his wife, Marsha.
"He is seriously looking at it. It horrifies me. It's a very overwhelming task, and he's going out and doing some things just to see if he gets that fire in his belly. If he's not passionate about it, there's no reason for him to run," she told CNN affiliate WLOX-TV this month.
"Why am I afraid? It's been a lot to be first lady of the state of Mississippi, and this would be 50 times bigger, and it's a huge sacrifice for a family to make."
CNN political reporter Peter Hamby contributed to this report.