(CNN) -- The Tea Party Express is predicting large crowds for five events Thursday and Friday aimed at unseating U.S. Rep. Bart Stupak, D-Michigan.
The events are being held in Michigan's Upper Peninsula, which is part of the Democrat's large and rural First Congressional District.
The rallies come as sources said the top two House Democrats called Stupak to urge him to stay in Congress. A Stupak spokeswoman didn't confirm or deny rumors that the lawmaker is considering retiring from Congress.
"Every two years, Congressman Stupak discusses with his family and his constituents whether to seek re-election. He believes that is what the people of the First District of Michigan deserve," said Stupak spokeswoman Michelle Begnoche.
But Begnoche did say that Stupak's campaign plans are under way, with the gathering of 1,000 signatures needed to file for re-election formally by Michigan's deadline of May 11.
The socially conservative, anti-abortion rights Democrat gained national attention last month by helping deliver the remaining votes House Democrats needed to pass health care legislation. Stupak cut a deal with the White House that he says ensures no taxpayer money will be used to fund abortions. President Obama signed the bill into law last month.
Three Democratic sources said that House Speaker Nancy Pelosi of California and House Majority Leader Steny Hoyer of Maryland have both called Stupak to urge him to stay in Congress.
One source said the two House leaders told Stupak he's a valued and important member of the Democratic caucus, that there's still a lot more work to be done that will benefit Michigan and that the caucus will be there for him.
Despite the rumors and a cryptic statement from Stupak's office, several Michigan Democrats who know Stupak said they do not think he will retire.
The Tea Party Express said it is targeting Stupak because of his health care vote. The group began what it called a $250,000 TV and radio campaign against the nine-term congressman.
Tea Party Express organizers said local coordinators are predicting large crowds, but cold weather and snow could keep the numbers attending down.
"People are angry. They feel that Congressman Stupak betrayed them, and they are determined to hold him accountable," Tea Party Express Chairman Mark Williams said.
Stupak also is facing criticism from abortion rights groups who think he forced too many abortion restrictions into the health care bill. They are trying to raise money for his Democratic primary challenger, Connie Saltonstall.
Still, several Michigan political experts said they believe it would be hard to unseat Stupak. He is a popular figure in his district, and the anti-abortion, anti-gun Democrat still seems to fit the largely Catholic, socially conservative constituency.
So far, Stupak's potential challengers are unknown and not well-funded.
"The idea that Stupak, who has been a congressman for 18 years, can be beaten by somebody with no money -- from left or right -- with no constituency is unlikely," said Bill Ballenger of Inside Politics Michigan, a political newsletter.
This week's five rallies in Michigan are part of the Tea Party Express' third national bus tour, which is scheduled to end on April 15. The Tax Day event is expected to draw the Tea Party and other conservative groups to the nation's capital.
The tour, with a slogan of "just vote them out," is highlighting the states and districts of lawmakers that Tea Party Express is working to defeat in this year's congressional elections.
The tour kicked off recently in Searchlight, Nevada, which is the hometown of Senate Majority Leader Harry Reid.
Tea Party Express organizers said Reid and Stupak are their top two targets.
CNN's Evan Glass contributed to this report.