"There are continual reports of law enforcement officers and community members being followed home; of people sitting in cars outside their homes, observing their movements and those of their families; and of people following them and their families as they move around the community," Ward said on Monday.
"While not direct physical threats, these activities are clearly designed to try to intimidate," he said.
Some of the armed protesters have been able to leave and return from the refuge center during the occupation.
"Let me be clear: The law enforcement agencies -- those that are local as well as the sheriff's deputies from around the state, the Oregon State Police troopers and the FBI agents -- will not be intimidated from doing their jobs.
"Everyone on the law enforcement side is working together to bring a peaceful resolution to this situation, and the behavior of these folks from outside of our community only serves to escalate the situation unnecessarily."
Ammon Bundy, the son of anti-government Nevada rancher Cliven Bundy, and others have been hunkered down in the wildlife refuge since early January. The protest started out as a call against the conviction of Dwight Hammond and his son Steven -- two ranchers who were found guilty in 2012 of committing arson on federal lands in Oregon -- but soon morphed into a bigger movement, rallying against the role of the federal government when it comes to land rights.
Ward says that both law enforcement and employees for the U.S. Fish and Wildlife Service, which runs the refuge, have been harassed recently.
He says Fish and Wildlife workers have seen vehicles driving slowly or idling in front of their homes.
"Self-identified militia members" will approach them in public, according to Ward, trying engage in a debate about the federal government.
"The people on the refuge -- and those who they have called to our community -- obviously have no consideration for the wishes or needs of the people of Harney County," he said. "If they did, they, too, would work to bring this situation to a peaceful close."
Protesters arriving on both sides
Ward said after a recently issued "call to action," more armed protesters and "outside militia members" have begun to show up.
But some people are coming with a different message for those at the wildlife refuge, according to CNN affiliate KTVZ
They're livid, befuddled and want the armed protesters to stop doing what they're doing.
"These are my public lands, these are your public lands, and what I see is a lunatic fringe of radical extremists who have taken my land over," Garrett VeneKlasen, a protester with the New Mexico Wildlife Federation, said.
After Ammon Bundy and his group finished a news conference, VeneKlasen took to the same stage to protest the occupation.
"I came here from New Mexico to speak on behalf of sportsmen, to speak on behalf of people that love land and wildlife, to come to tell these people to get the hell off of our land," he said. "Let's not candy coat this thing -- they are terrorists. They are domestic terrorists."
Another protester, who was holding a sign that said "Get the flock outta my wildlife refuge," told KTVZ that "I never thought I'd have to say this, but I'm here to oppose the armed occupation of a wildlife refuge."
At a community meeting of a couple hundred people in Burns, Franki Gould told CNN that she wants the protesters gone.
"We were a really close town and now we're very divided," she said.
Ward also spoke at that meeting, stressing unity among those from the community
"We can't get things to normal until we unite as a community and ask these folks to go home so we can start working together," he said to raucous applause at a community meeting of a couple hundred people.
"The fact is there is there is an hourglass, and it's running. And time is going to run out."