Colombo, Sri Lanka (CNN) -- The U.N. secretary-general is recalling an envoy from Sri Lanka and closing a U.N. office there because authorities in that country "failed to prevent" protests disrupting the "normal functioning" of the world body's offices in Colombo, the nation's capital.
The move is a rare rebuke of a member U.N. state by Secretary-General Ban Ki-moon, who on Thursday called the "unruly" demonstrations organized and led by a Cabinet minister "unacceptable."
Ban recalled U.N. Resident Coordinator Neil Buhne to New York for consultations and decided that the United Nations Development Program Regional Center in Colombo will be closed.
"The Secretary-General calls upon the government of Sri Lanka to live up to its responsibilities towards the United Nations as host country, so as to ensure continuation of the vital work of the organization to assist the people of Sri Lanka without any further hindrance," the United Nations said in a statement.
Wimal Weerawansa, the construction minister, began a "fast unto death" outside the United Nations compound in Colombo Thursday to demand that the organization stop its investigation into alleged war crimes.
He told reporters he would fast until Ban dissolves a panel made up of an Indonesian, a South African and an American.
Ban appointed the three-member panel to advise him on violation of human rights and related issues when Tamil Tiger rebels were militarily defeated in May last year. The move is prelude to a war crimes inquiry.
The United Nations has been concerned about accountability issues related to the rebels' defeat, including alleged war crimes by troops and rebels -- allegations that both parties deny.
"Ban's move is intended to bring President Mahinda Rajapaksa before a war crimes tribunal. We will not allow that to happen," Weerawansa told a news conference earlier in the week.
Nearly 100 Weerawansa supporters gathered Thursday morning outside the U.N. compound. The entrance to the complex was open, however, as some staff went about their work.
In New York, U.N. associate spokesman Farhan Haq said Ban's chief of staff, Vijay Nambiar, met with Sri Lanka's ambassador to the United Nations, Palitha Kohona, who gave "full and clear" assurances of U.N. staff safety and security.
He added that Weerawansa said U.N. staff would be allowed to move in and out of the compound.
"We trust that the government of Sri Lanka will honor the commitments made in ensuring the safety and security of our staff so that they can continue the vital work being carried out by the United Nations each day to help the people of Sri Lanka," Haq told reporters in New York.
The U.N. Country Team (UNCT) confirmed that essential staff will return to normal work starting Friday.
"However, as there are some indications of demonstrators remaining outside the compound, the UNCT will assess whether all staff could return soon," Haq said.
Opposition leader Ranil Wickremasing urged the government to make a statement on how a minister ended up staging demonstrations.
"Today, Sri Lanka is on the verge of being labelled as an international fugitive facing the risk of being hauled up before the International Criminal Court," Rajapaksa's one time foreign minister and now an opposition MP, Mangala Samaraweera, told parliament.
The two English morning national newspapers also criticized the move.
The Daily Mirror said in an editorial "after all this drama the government issued one of its silliest statements ever" justifying the siege as a "peaceful demonstration."
The Island newspaper said Weerawansa and his supporters "must be condemned unreservedly for their abortive bid to hold U.N. staff incommunicado."