When asked whether he'd be willing to meet with activists, Walker dismissed the possibility by saying, "Who knows who that is?"
"I meet with voters. Who knows who that is," Walker said, apparently referring to Black Lives Matter leaders.
"It's the same way as saying we meet with the tea party. Who is the tea party? There's hundreds of thousands of people," he said.
But when pressed specifically to say whether he would meet with organizers of the protest movement that has sprung up nationally in response to police shootings across the nation, Walker at first refused to answer.
"That is a ridiculous question. I'm going to talk to voters. It's a ridiculous question," Walker said.
Later in Derry, Walker again declined to say whether he would meet with protesters if they came to his events.
"I'd have them stand up and ask a question, just like the guy asked the question today. I don't block who comes in or out," Walker said. "I prefer to just talk to voters."
Walker was referring to an African-American audience member at the Politics and Eggs breakfast in Manchester earlier Friday, who asked what the Wisconsin governor would do to protect citizens from police shootings.
"I'm actually more afraid of the police than I am of ISIS," said the questioner, who was asking Walker why he focused so much on ISIS and not police relations.
Walker touted legislation he signed in Wisconsin mandating independent reviews of police shootings, but also said he believed most police were good, hard-working people.
"It's a horrible situation for them (the police) because they don't want to be in that situation, and independent review validates them if they did things appropriately," Walker said.
But he also cited the police shooting of Walter Scott in South Carolina earlier this year and said that police relations must be addressed.
Black Lives Matter protesters have disrupted several presidential candidates' events recently. Democrats Bernie Sanders and Martin O'Malley were unable to finish speeches
at the Netroots Nation conference in Phoenix last month, and Sanders was also interrupted
earlier this month during an event in Seattle. More recently, Hillary Clinton met with several members of the movement after a forum in Keene, New Hampshire
For his part, Walker said in Derry Friday that anyone can attend one of his town hall events, so long as they are respectful.
"The people who want to interrupt and disrupt a meeting of people, who want to ask a wide variety of questions, obviously we have an issue with that," Walker said. "We don't restrict who comes in. People can come in and ask questions, they just got to be respectful of other people asking questions too."