ARLINGTON, Virginia (CNN) -- Campaign paraphernalia is everywhere nowadays. People are sporting T-shirts, hats and pins touting their candidate of choice. But wearing your political allegiances can cause a problem at the polls.
In some states, what you wear to the voting booth could determine whether you are allowed to vote or be sent home to change.
Virginia's State Board of Elections has banned campaign clothing and other paraphernalia from inside all polling places on Election Day. In the past, it had been done only sporadically.
"We want to take this important step to make sure that the rules are applied uniformly throughout Virginia," said the board's James Alcorn.
In a two-to-one vote, the board said it was seeking to strike a balance between free speech and zones free of distractions and harassment.
The ruling prompted calls to CNN's Voter Hot Line.
"You can't wear [Barack] Obama shirts or hats or anything that pertains to the voter, you know, the nominee, and I want to know why," asked a caller from Norfolk, Virginia.
"If that's not a violation of the First Amendment rights, I don't know what is," another caller said.
Restrictions on what a voter may do or wear at a polling place and near one generally do not violate a voter's First Amendment right to free speech.
There are certain times when your political expression can be kept in check, according to Richard Hasen, a law professor at Loyola Law School in Los Angeles, California.
In 1992, the U.S. Supreme Court said that a law creating a "campaign-free zone" at a polling place and for 100 feet around it does not violate the First Amendment right to free speech.
Hasen said the ruling was not about campaign clothing and pins but could be applied that way.
In Pennsylvania, the secretary of state sent a letter to all 67 counties informing them that the state's opinion is that wearing a campaign T-shirt or pin is not reason enough to refuse a person the right to vote.
"The point is to allow people who are registered and eligible to vote to do so. You don't want to turn anyone away because of what they are wearing," said Leslie Amaros, a spokeswoman for the secretary of state.
But Amaros said the decision on how to enforce the rules is up to each county.
In California, the state plans to offer jackets to cover election paraphernalia that voters might be wearing, according Richard Hasen.
In Virginia, voter rights groups have protested. Jennifer Schiffer of Lawyers Committee for Civil Rights Under the Law said volunteers will be at the polls with extra shirts to help people cover up if needed.
"We just decided we're going to do whatever we can to insure everybody can vote and isn't hampered by this law," Schiffer said. "Our greatest concern is making sure that everyone has the right to vote, and we felt this was going to be another way to keep some people from being able to vote, because they would be wearing paraphernalia."
Virginia election officials say rule-breakers will be confronted but not turned away.
"In the event that a person decides that they want to wear a T-shirt to the polls, we'll still make sure they get to vote," said Jean Cunningham of the Virginia State Board of Elections.
Voters will be asked to change clothes, turn their clothing inside out or cover up in another way.
"Our advice is, don't wear your stuff to the polls. If you are going to, wear a pin or sticker or something that's easily removable. ... You can put back on when you leave the polling place," Schiffer said.
CNN's Adam Levine and Martina Stewart contributed to this report
All About Elections and Voting • Voting Methods
From the Blogs: 
Sit tight, we're getting to the good stuff
