Panama City, Florida (CNN) -- President Barack Obama told CNN Saturday that in defending the right of Muslims to build a community center and mosque near ground zero in a speech on Friday night, he was "not commenting on the wisdom" of the project.
Instead, Obama said he was trying to uphold the broader principle that the government should treat "everyone equal, regardless" of religion.
His comments were seen as step back from the support he appeared to give the controversial project during a White House dinner on Friday, though a spokesman for the administration quickly moved to clarify the president's remarks.
"Just to be clear, the President is not backing off in any way from the comments he made last night," White House spokesman Bill Burton said in a statement Saturday afternoon.
"It is not his role as President to pass judgment on every local project. But it is his responsibility to stand up for the Constitutional principle of religious freedom and equal treatment for all Americans," Burton added.
Obama, speaking on Saturday after giving a speech on the Gulf Coast oil disaster in Panama City, Florida, told CNN he "was not commenting and I will not comment on the wisdom of making the decision to put a mosque there," referring to the area near ground zero.
"I was commenting very specifically on the right people have that dates back to our founding," the president added. "In this country, we treat everybody equally and in accordance with the law, regardless of race, regardless of religion."
At a White House dinner celebrating the Islamic holy month of Ramadan, Obama seemed to throw his support behind a controversial proposal to build an Islamic center and mosque near New York's ground zero, saying Friday that "Muslims have the same right to practice their religion as anyone else in this country."
"That includes the right to build a place of worship and a community center on private property in lower Manhattan, in accordance with local laws and ordinances," Obama said.
The president's remarks drew praise from New York Mayor Michael Bloomberg, who announced his support for the Islamic center last week. Bloomberg compared Obama's speech to a letter former President George Washington wrote in support of a Jewish congregation in Newport, Rhode Island.
Rep. Jerrold Nadler (D-New York), whose district includes ground zero, praised Obama's remarks on Saturday.
"The United States was founded on the principle of religious liberty and tolerance, and it is equally important 234 years later that we uphold this principal. Hate should have no place in America," Nadler said in a statement.
Still, critics of the proposed Islamic center -- including Sarah Palin and Rep. Peter King (R-New York) -- quickly denounced Obama's comments.
"President Obama is wrong," King said in a statement Friday night. "It is insensitive and uncaring for the Muslim community to build a mosque in the shadow of Ground Zero. Unfortunately, the President caved into political correctness."
Sarah Palin, the former Republican candidate for vice president, also weighed in.
"We all know that they have the right to do it, but should they?" she tweeted on Saturday.
Obama, who on Friday said he was speaking both as a citizen and as president, invoked the attacks of Sept. 11, 2001, which critics of the Islamic center cite as the main reason for blocking its construction.
"We must all recognize and respect the sensitivities surrounding the development of lower Manhattan," he said. "The 9/11 attacks were a deeply traumatic event for our country."
"The pain and suffering experienced by those who lost loved ones is unimaginable," Obama continued. "So I understand the emotions that this issue engenders. Ground zero is, indeed, hallowed ground."
The proposed Islamic center has provoked vocal opposition from some families of 9/11 victims and other groups. Nearly 70 percent of Americans oppose the plan, according to a CNN/Opinion Research Corporation Poll released Wednesday.
"In a breathtakingly inappropriate setting, the president has chosen to declare our memories of 9/11 obsolete and the sanctity of Ground Zero finished," Debra Burlingame, co-founder of 9/11 Families for a Safe & Strong America, said in a statement.
The Islamic center's leaders say they plan to build the $100 million, 13-story facility called Cordoba House three blocks from the site of the 9/11 attacks. The developer, Sharif El-Gamal, describes the project as an "Islamic community center" that would include a 500-seat performing arts center, a lecture hall, a swimming pool, a gym, a culinary school, a restaurant and a prayer space for Muslims.
Other families of 9/11 victims said they support the proposed Islamic center, and the president's position.
"America, the concept and the people and the land thrive when we chose to trust in our principles rather than cave to our basest fears," Donna Marsh O'Connor, spokeswoman for September 11th Families for Peaceful Tomorrows, said in a statement.
"What better place for healing, reconciliation and understanding than Ground Zero?" she added.
On Wednesday, the project's developers declined an offer by New York Gov. David Paterson to relocate the project to a state-owned site.
Earlier this month, the New York City Landmarks Preservation Commission unanimously denied landmark status for the building where the proposed Islamic center would stand, allowing the project to move forward.
CNN's Senior White House Correspondent Ed Henry contributed to this report.