(CNN) -- Israel began releasing more than two dozen Palestinian prisoners late Tuesday, a goodwill gesture on the eve of new peace talks that have been complicated by Israeli plans for new housing in East Jerusalem.
The vans began pulling away from the Israeli prison in Ramla shortly before 10 p.m. (3 p.m. ET), carrying the 26 freed inmates to flag-waving receptions in the West Bank and Gaza. One group was handed over the Palestinian officials at the Israeli prison at Ofer and they were driven to meet with Palestinian Authority President Mahmoud Abbas in nearby Ramallah.
As they were driven away, Palestinian youths outside started throwing stones at the prison, where guards responded by firing stun grenades and tear gas.
Some of the Palestinians had been held for more than 20 years. But as the vans rolled away, a group of demonstrators waved signs condemning the release, complaining that the prisoners had Israeli blood on their hands.
In approving the release, a committee of Israeli Cabinet ministers stressed "that if any of the released prisoners return to hostile activity against the state of Israel, they will be returned to continue serving their sentences."
Abbas announced the release of the prisoners was just the beginning.
"We say to them and the rest who will follow, this is the beginning and there are other brothers who will be returning and be amongst you soon," he said.
The prisoner release came after Israel said it would forge ahead with a plan to build 900 housing units in East Jerusalem, which the Palestinians hope will be the capital of their future state.
The issue of Israeli settlements derailed the last round of direct talks, in 2010, and critics of Israeli Prime Minister Benjamin Netanyahu say building on disputed territory could derail the new talks, slated to start Wednesday in Jerusalem.
"Netanyahu has to decide which government he is heading: a government that is trying to reach a peace agreement, or a government that is trying to undermine all possibilities of this agreement," said Shelly Yachimovich, head of the Labor Party and the Israeli opposition.
In January, the United Nations Human Rights Council said Israeli settlements amount to "creeping annexation" of Palestinian territories by Israel and have taken a "heavy toll" on the rights and sovereignty of Palestinians.
And in Washington, where American leaders have pushed both sides to return to the table, the State Department said it had "serious concerns" with the new announcement.
"We've encouraged both sides broadly speaking to refrain from taking steps that could undermine trust. We said that from the beginning," State Department spokeswoman Marie Harf said. But, she added, "We've also been clear that these are complicated issues, and there will be bumps in the road."
Daoud Kuttab, a columnist for the online Middle Eastern news site Al-Monitor, said the released prisoners are a bargaining chip to make sure the Palestinians don't back out of the talks despite their anger over new settlements.
"They're trapped, and Mahmoud Abbas has egg on his face today because he can't get out of the talks," Kuttab told CNN.
But Rami Khoury, a veteran Middle East analyst at the American University of Beirut, said both sides agreed to resume talks for nine months under American mediation.
"This is a significant gesture, so therefore you're going to hear a lot of noise from both sides -- complaints, grievances," Khoury said. "But you can pretty much discount most of what you hear in public, because they're both clearly committed to doing this for nine months. And we really need to give it time to see what's going to happen."
Both Abbas and Netanyahu are "hemmed in by domestic constraints," leaving neither side likely to make significant changes on its own, Khoury said. U.S. Secretary of State John Kerry said last week that "many difficult choices" lie ahead for both sides, but held out hope that "reasonable compromises" could be reached.
Khoury called that a "weak statement" that downplayed expectations for the talks.
As a result, most observers have little hope that the talks will produce a breakthrough in the decades-old Israeli-Palestinian conflict. But he said the U.S. role in the talks is a "wild card" that could defy those expectations.
"What is the United States going to do?" he asked. "Will it push both sides? Will it pressure them? Will it cajole them? Will it entice them, and will it make serious bridging and endgame proposals? We have no idea about any of that."
CNN's Vladimir Duthiers, Saad Abedine, Michael Schwartz, Kareem Khadder and Tom Cohen contributed to this report.