(CNN) -- UNESCO on Saturday condemned the destruction of three sacred tombs in Timbuktu, Mali, by Islamic militants who say they will destroy more.
The tombs, a 15th century shrine to Muslim saints, are part of a World Heritage site in Timbuktu.
The destruction started Friday.
"The militants broke the doors and wooden gates to the grave and then used the cloth inside to set fire to the tomb," said Sankoum Sissoko, a resident in Timbuktu. "They started chasing people away, telling them they shouldn't worship a saint, a human being."
He witnessed the attack together with other Timbuktu residents who had come to the tombs to pray.
"The Islamists know a lot of people visit the grave on Fridays. That's why they attack on those days," Sissoko said.
UNESCO Director-General Irina Bokova called for the militants to stop the destruction immediately. The U.N. agency sponsors cultural, scientific, and educational programs to further peaceful relations between nations.
"Reports that the Mausoleums of Sidi Mahmoud, Sidi Moctar and Alpha Moya have been destroyed is extremely distressing," she said. "There is no justification for such wanton destruction and I call on all parties engaged in the conflict to stop these terrible and irreversible acts, to exercise their responsibility and protect this invaluable cultural heritage for future generations."
Timbuktu Muslims on their way to Friday worship at the tombs were stopped and threatened by armed men from Ansar Dine, a militant group that seeks to impose strict Sharia law, Timbuktu Mayor Ousmane Halle said.
"The Islamists have surrounded he tombs with their cars after chasing people away and at the moment they're destroying the mausoleum of Sidi El Moctar," Halle said.
According to several witnesses the militants said they wouldn't stop until the destruction was complete.
"Today we will destroy all the mausoleums," Ansar Dine representative Sanda Ould Boumama said.
Just on Thursday, the World Heritage Committee had accepted the request of the Malian government to place Timbuktu on UNESCO's sites in danger.
France, the former colonial power in Mali, also condemned the attack.
"We call for an end to this violence and this intolerance," France said in a statement.
Journalist Katarina Hoije contributed to this report