(CNN) -- Rebels in the eastern Democratic Republic of the Congo began withdrawing from the key eastern city of Goma on Saturday, witnesses said, more than a week after they seized it from government forces.
Fighters from the M23 rebel group boarded trucks and other vehicles to head north to outside of the 20 kilometer (12 mile) buffer zone stipulated by regional leaders, a reporter on the ground who was not named for his safety said.
Earlier they had begun leaving Sake, a town to the west of Goma.
M23 rebels also left their posts at the border crossing with Rwanda, but remained in some parts of Goma as of early afternoon.
Tensions mounted in the city as several disputes arose. The M23 wanted to take a large cache of ammunition left by fleeing Congolese soldiers, the reporter said, but MONUSCO, the U.N. peacekeeping force for DR Congo, prevented the rebels from doing so.
MONUSCO forces also stopped the rebels from taking vehicles donated to the government by the European Union.
The rebel group has been accused of widespread looting by the United Nations and Goma residents. Government forces also looted as they fled last week, witnesses said.
The United Nations welcomed M23's move, but said the group must cease military activity and stay outside of Goma.
"We welcome this withdrawal, but emphasize that this is an early step and that stability in the region remains very fragile," said Kieran Dwyer, spokesperson for U.N. peacekeepers.
Regional leaders and the African Union had called for the rebels' withdrawal from Goma, supposed to have been completed by Thursday, as a condition of starting negotiations.
The regional leaders' plan foresees an integrated force being deployed for an initial period of three months at Goma airport, Ugandan Gen. Aronda Nyakairima said earlier this week. It would include a 100-strong neutral force, which would be in command, 100 troops from the Congolese army, and 100 members of the M23, Nyakairima said.
Two military observers would also be deployed from each of the neighboring regional powers, while MONUSCO would be responsible for securing the buffer zone outside Goma, he said.
The regional leaders also called on the rebels to "stop all war activities," and "stop talk of overthrowing an elected government."
The latest violence has forced more than 140,000 people to flee their homes, according to the U.N. refugee agency, on top of those already forced from their homes by previous rounds of fighting in the volatile region. And aid groups warn the conflict risks triggering a broader humanitarian crisis.
The M23 group was named for a peace deal of March 23, 2009, which it accuses the government of violating. The soldiers, mostly Tutsis, became part of the national army through that accord.
However, they broke away from the Congolese army in April, complaining they weren't being promoted as promised, and because of a lack of pay and poor conditions.
The latest unrest continues a cycle of misery in eastern DR Congo, a mineral-rich region at the epicenter of political and ethnic conflict involving its neighbors to the east, Uganda and Rwanda.
The area has been embroiled in violence since 1994, when Hutu forces crossed the border from Rwanda fearing reprisals after the genocide in that country.