Madison, Wisconsin (CNN) -- Tea Party activists supporting a bill that would slash collective bargaining rights of Wisconsin public employees flocked to Madison on Saturday, bringing a new dimension to a budget battle that shows few signs of compromise.
As tens of thousands of people marched in favor or against the bill, Republican Gov. Scott Walker indicated he has the votes to pass it.
"We're going to stay firm on it," he said on the fifth day of large protests and political wrangling in the capital, which saw the arrival of conservative groups.
"Wisconsin is ground zero," said Tim Phillips, president of Americans for Prosperity. "I think it is going to determine largely whether the pampered nature of these public employees is finally reined in."
Walker has called on 14 Democratic state senators, who fled the state rather than allow a quorum permitting a vote on the bill, to return to Madison.
Bill opponents say they won't allow a vote unless Walker negotiates on the plan to eliminate collective bargaining rights for everything but wages. The legislation also would require most employees to pay more for their pensions and health insurance benefits.
"I have been informed that all state and local public employees -- including teachers -- have agreed to the financial aspects of Gov. Walker's request," said Sen. Jon Erpenbach, a Democrat. "This includes Walker's requested concessions on public employee health care and pension. In return they ask only that the provisions that deny their right to collectively bargain are removed. This will solve the budget challenge."
Walker, who says the state is "broke," is asking legislators to pass his Budget Repair Bill to combat a $137 million shortfall through June 30. An upcoming two-year budget for 2011-13 must address a pending $3.6 billion deficit, he said.
Opponents say the proposed legislation is an attack on workers' rights.
Until Saturday's counter-demonstrators appeared, the growing protests since Monday were largely made up of those against Walker's plan. They remained the clear majority of those marching; many were drawn Saturday by the arrival of Walker's supporters.
There were no reports of major incidents or arrests by late Saturday afternoon, and authorities expressed their gratitude.
"On behalf of all the law enforcement agencies that helped keep the peace on the Capitol Square Saturday, a very sincere thank you to all of those who showed up to exercise their First Amendment rights," Madison police said in a statement. "You conducted yourselves with great decorum and civility, and if the eyes of the nation were upon Wisconsin, then you have shown how democracy can flourish even amongst those who passionately disagree."
Teachers and other public employees kept the pressure on the governor to go to the bargaining table.
Madison teacher Karen Kaminsky said she wouldn't have gone to college if her father had not been in a union. "This will break the backs of middle-class families," she told CNN affiliate WTMJ, adding Walker should be willing to negotiate.
The governor's press secretary, Cullen Werwie, released a statement Saturday calling on Erpenbach, Senate Minority Leader Mark Miller and other Democrats to return from Illinois.
"These are many of the same senators who, two years ago, rammed through a billion dollar tax hike in 24 hours with no public input," the statement said. "The quickest way to resolve the current situation is for the Democratic senators to stop shirking their responsibilities and debate the bill in Madison."
Saturday's protests inside and outside the Capitol were the largest of the week, numbering about 55,000, said Carla Vigue, spokeswoman for the Wisconsin Department of Administration. Some estimates were higher.
Friday saw about 40,000 demonstrators, Vigue said.
The conservative group American Majority helped organize the "I Stand With Walker" campaign evident this weekend in Madison.
"Buses will be coming from across the state, bringing citizens fed up with big union contracts and bloated government," the group said on its website. "We win here, we win everywhere."
The defecting Democratic lawmakers say they won't return until Walker agrees to negotiate with the teacher's union on the governor's proposals to reduce the state's budget deficit by cutting benefits for its members. They also want language removed from the bill that critics say threatens the existence of all public employee unions in the state.
White House Press Secretary Jay Carney addressed the continued protests Friday, saying President Barack Obama "is very understanding of the need for state governments, governors, state legislatures to reduce spending to make tough choices to be fiscally responsible."
But he added, "He also feels very strongly that we need not to make this an assault on the collective bargaining rights of workers in any given state.
"He understands public employees need to make sacrifices just like everyone else."
Walker's proposed legislation requires workers to cover more of their health care premiums and pension contributions, although supporters say local governments will decide on health care contributions for their employees
Walker contends the increases are "modest" compared to what workers in private industry pay.
It also requires collective bargaining units to conduct annual votes to maintain certification, a costly procedure, and eliminates the right of unions to have dues deducted from worker paychecks.
CNN's Casey Wian and Chris Welch contributed to this report.
