Editor's note: John P. Avlon is a CNN contributor and senior political columnist for The Daily Beast. He is the author of "Wingnuts: How the Lunatic Fringe Is Hijacking America."
(CNN) -- I lived through 9/11. Now I live two blocks from ground zero. This Sunday, I walked over to view two competing rallies - pro and con - over the Islamic cultural center (with mosque) that is to be built on Park Place, near the site of the former World Trade Center.
The rally against the center was predictably more crowded. Perhaps 2,000 people clustered on West Broadway -- off-duty firefighters, construction workers and family members of the fallen from 9/11 -- all watched over by New York City's finest. The cranes of ground zero were visible in the background.
Looking at the signs and talking with the people, it soon became clear that the battle over the center has become a proxy war for other issues. It is a powerful magnet for passionate feeling in a combustible political environment.
Many signs warned about sharia law, as if the center represents a first step toward the unconstitutional imposition of Islam's religious principles on Americans.
At the side of the speaker's stage, two mock missiles had slogans painted on their side referring to Iran's pursuit of a nuclear weapon. Beneath one of the "missiles" was written "Obama: With a middle name is Hussain (sic). We understand. Bloomberg: What is your excuse?" (Mayor Michael Bloomberg has strongly supported the building of the Islamic center.)
And there were flashes of wing-nuttery. A man in a Confederate flag shirt and a cowboy hat held a sign that read "I'm afraid the Mosquerade has ended." A man in a plastic Obama mask with devil's horns affixed to its side held a sign saying "Evil Mosque."
For some folks, the Islamic center near ground zero has become a metaphor for what they fear Obama wants to do to the nation. This helps to explain the troubling rise in the number of people who believe fear-mongering emails that claim that the president is Muslim or not born in the nation.
But the vast majority of the signs I saw were sobering: "It's Not About Freedom of Religion, its about Respect for the (3000) Dead;" "The mosque is salt in the wound of 9/11;" "It's my first amendment right to protest this travesty...just because you have the right doesn't mean you should."
These are not signs of so-called "Islamophobia," let alone hatred of all Muslims, as the wife of the Imam, Daisy Kahn, sweepingly asserted this week. They are instead common sense and easily anticipated concerns in a community that lost so many lives to Islamist (not Islamic) terrorism less than a decade ago. If the goal of the center is really to promote healing, the reaction so far is evidence of failure. A genuinely inter-faith center on that site would better accomplish those aims.
The counter protest was pitifully small by comparison; a few dozen people, no stage set or amplifiers, signs with earnest slogans like "grief is no excuse for bigotry or war" or self-righteous non-sequiturs like "America, when did it become O.K to be a bigot and a racist again."
Conflict between the two protests was kept to a minimum by the police, but there were angry debates and heated exchanges. One man walked by the center protestors and said "die peacefully." He said his name was Al. I asked Al what he meant by that. He backed away and said that he hoped the protestors would find peace before the hereafter. "This has nothing to do with the mosque. They are just racist haters. That's not the America we know. I've been living here 40 years."
He pointed to a 10-year old boy in a red baseball cap, who I assumed was his grandson. "His father is a United States Marine. Where are we going to go? This is our country....They have opinions, that's fine. But don't hate Islam. Those who did wrong, they should be punished. But the same thing is true for the group in the White House that killed 1.5 million Iraqis. They should be punished."
"Are you comparing President Bush to terrorists?," I asked.
"Oh, absolutely," he said. "Look at what happened in Abu Ghraib. You think Muslims are going to forget that?"
The truth is rarely pure and never simple.
Attempts to paint all opponents of the center's location as bigots are wrong. Attacks on the center's organizers as a fifth column flank for Osama bin Laden are ignorant. The majority of people I spoke to seemed clear on the constitutional right to build an Islamic cultural center/mosque, as President Obama asserted. The real issue is what the president later called "the wisdom" of the site's selection.
The fact of this debate -- however messy -- is evidence of America's freedom and pluralism.
We are debating whether or not we should override our own sensitivities in an attempt to teach the world a larger lesson, or whether our openness is being exploited by people who would undo it. The stakes are heightened because we are engaged in a non-optional war against Islamist terrorism. But as President Bush and President Obama have repeatedly emphasized, our enemy is most assuredly not Islam as a whole.
People who are trying to exploit this debate for political gain are playing a dangerous game. America is great because we are good, and the example of our magnificent pluralism is the ultimate repudiation of the intolerance of extremists and fundamentalists.
The opinions expressed in this commentary are solely those of John Avlon.