(CNN) -- Embattled Republican Gov. Scott Walker defended a budget bill that would slash public workers' benefits and most collective bargaining rights in an effort to address Wisconsin's impending $3.6 billion dollar deficit.
"We have bill collectors waiting," Walker told reporters Friday. "It's time we step up and pay the bills that we owe."
The move comes as crowds of protesters gathered in Madison and Democratic state senators defended their decision to leave the state in an effort to stymie a bill they say is anti-union.
"Hiding out in another state is not the way democracies operate," Walker said.
Walker, who says the state is "broke," is asking legislators to pass his Budget Repair Bill to combat a $137 million shortfall through June 30. An upcoming two-year budget for 2011-13 must address a pending $3.6 billion deficit, he said.
But opponents of the bill, most of them Democrats, say the proposed legislation is an attack on workers' rights.
"We will return and do our job, but the governor had to do his job," said state Senate Minority Leader Mark Miller, one of 14 Democrats and two Republicans who left Madison to stall a vote on the controversial bill.
Republican Gov. Scott Walker is looking to slash $131 million from the state budget. Walker called on the 16 lawmakers to return to the state Capitol to vote on the measure Friday, but that did not happen.
CNN affiliates WREX and WTVO reported that several of the AWOL lawmakers fled to Rockford, Illinois.
The defecting lawmakers say they won't return until Walker agrees to negotiate with the teacher's union on the governor's proposals to reduce the state's budget deficit by cutting benefits for its members. They also want language removed from the bill that critics say threatens the existence of all public employee unions in the state.
"He's going to get his money. We all know that," Democratic state Sen. Jon Erpenbach told CNN. "This issue is all about the governor's attempt to strip collective bargaining rights away from the unions and essentially bust them."
Wisconsin's other legislative body is the Assembly, akin to the House in many states. Brian Schimming, vice chairman of the state Republican Party, and state Democratic Party spokesman Graeme Zielinski told CNN that the body will not take up the debate until Tuesday, further delaying action on the measure.
Walker has delayed the unveiling of his next budget to March 1, Schimming said.
Majority Republicans convened the Assembly a few minutes before the announced 5 p.m. start time Friday. No Democrats were present in the chamber when Republicans started voting on minor amendments to the bill, CNN affiliate WISC reported. Democrats rushed into the chamber screaming, "No!"
Assembly Democratic Minority Leader Peter Barca yelled at Speaker Pro Tem Bill Kramer to stop voting, calling the Republicans' move an outrage, WISC said. Kramer canceled the vote.
Walker called on the absent senators to return to Madison "out of respect for the institution of the legislature and the democratic process."
"Their actions by leaving the state and hiding from voting are disrespectful to the hundreds of thousands of public employees who showed up to work today and the millions of taxpayers they represent," Walker said.
Bryan Kennedy, president of the Wisconsin chapter of the American Federation of Teachers, called the lawmakers' decision to skip town "courageous."
"The same could be said of the U.S. Senate," Kennedy told CNN. "A small minority continues to constantly filibuster every piece of legislation and bring that body to standstill. We don't have a filibuster here in Wisconsin.
"The only thing these brave senators could do was to leave the state so that they could not be forced or compelled to bring a quorum to the body," Kennedy said. "They stood up for worker rights."
Kennedy blamed Walker for refusing to meet with union representatives.
"We are willing to come to the table and negotiate," Kennedy said. "He is the one not willing to come to the table. He wants to strip our rights and then dictate exactly what the terms and conditions of employment are."
Wisconsin Assistant Senate Majority Leader Glenn Grothman, a Republican, said Walker shouldn't have to negotiate.
"We had elections last November," Grothman told CNN. "The idea that Gov. Walker or the state legislature can't have public employees contribute toward their pensions without Bryan Kennedy signing off is absurd."
While the legislators skipped town, thousands of people came to Madison to protest the bill for a fourth day, including many teachers, who stand to be strongly affected by the bill. On Thursday, demonstrators spilled into the state's Capitol building, chanting, "This is our house" and "Forward not backward," voicing their opposition to the bill.
White House Press Secretary Jay Carney addressed the continued protests Friday, saying President Barack Obama "is very understanding of the need for state governments, governors, state legislatures to reduce spending to make tough choices to be fiscally responsible."
But he added, "He also feels very strongly that we need not to make this an assault on the collective bargaining rights of workers in any given state.
"He understands public employees need to make sacrifices just like everyone else."
Schools were canceled for the third consecutive day in the state's capital, Madison, because of anticipated staff absences.
In Milwaukee, public schools also were closed due to a high number of absentee calls from teachers.
"With these staff shortages, we will not be able to provide safe and appropriate class settings," said schools Superintendent Gregory Thornton.
Walker's proposed legislation requires workers to cover more of their health care premiums and pension contributions, although supporters say local governments will decide on health care contributions for their employees.
The legislation also requires collective bargaining units to conduct annual votes to maintain certification, a costly procedure, and eliminates the right of unions to have dues deducted from worker paychecks.
CNN's David Ariosto, Chris Welch and Phil Gast contributed to this report.