(CNN) -- Police in northern France moved in Wednesday on makeshift migrant camps near the port of Calais, prompting a standoff with the defiant residents -- many of whom have fled conflicts in Syria, Sudan and Eritrea.
By late afternoon, a CNN producer observed that the migrants' tents had all been destroyed. One activist told CNN police were responsible. About a dozen police officers remained, along with 50 to 60 migrants who didn't know where to go.
A local prefect reportedly told the migrants they can stay at the camp until Thursday. But Thursday is Ascension Day, a public holiday, so it's not clear if the migrants will be cleared out before Friday.
Hundreds of migrants had gathered in the ramshackle camps, some seeking to claim asylum in France and others hoping to find a way to reach British soil.
Mattheu Adt of international humanitarian organization Medecins du Monde, told CNN from one of the camps that police had asked the migrants to move to an undisclosed location, but that the migrants were refusing. Authorities also asked the migrants to shower and decontaminate their clothes, amid concern over an outbreak of the contagious skin condition, scabies, he said.
Migrants refused to do that, Adt said, because of concerns their tattered tents would be gone when they returned.
"They fear they will be arrested at the showers," said Cécile Bossy, an activist with Medecins du Monde.
She said migrants were given scabies medication Tuesday night but didn't understand what it was for.
Adt said the makeshift camp where he is located housed about 600 people, roughly half of them Syrian, and between 200 and 300 Eritrean and Sudanese.
A spokesman for the charity Secours Catholique in Calais confirmed that police had arrived at about 6:30 a.m. local time at a camp and asked the migrants there to board buses to go to "decontamination" areas.
Many of the migrants refused, he said.
Earlier, police destroyed another camp -- which housed about 300 people -- with bulldozers, he said. The migrants have been involved in discussions with authorities but don't know where to go, he said.
Official: Dismantling under way
An official in Calais told CNN that authorities planned to dismantle the migrants' makeshift camps by the end of Thursday.
Roughly 550 migrants lived in those camps, said Georges Bos, the associate chief of staff of the Pas-de-Calais prefecture -- the local branch of the French government.
"By tonight there will be no possibility left to go to these camps," he said.
Bos said authorities had offered to take the migrants by bus to places where they could shower, adding that dismantling the camps was necessary to prevent further spread of scabies.
He said that the prefect of the Pas-de-Calais region had told migrants their immigration status would not be checked and that no arrests would be made.
It's not the first time French authorities have sought to move on the migrants who congregate in the area around Calais, many hoping to smuggle themselves into Britain inside freight trucks going across the English Channel. A camp in Sangatte was dismantled in 2002 and another known as the Jungle was broken up in 2009.
But after each clearance effort, new makeshift camps spring up.
Medecins du Monde is one of a number of humanitarian and rights groups that signed an open letter to French Prime Minister Manuel Valls on Tuesday voicing concern over the plight of the migrants.
"The situation in Calais is worsening in a deafening silence," it said. "About 700 foreigners, for the most part of Syrian, Afghan and Eritrean origin, fleeing conflicts, violence and persecution, are installed in the town."
About 550 of those migrants are living in makeshift camps, the letter said, having claimed asylum in France or waiting to attempt the crossing to Britain. "They live in catastrophic sanitary conditions which have encouraged the development of a scabies epidemic."
The groups were "stunned" last week to learn that authorities planned to clear the camps and tackle the scabies outbreak on Wednesday, the letter said -- without coming up with any alternative place of shelter for the large majority of people concerned.
"We can anticipate the effects of this expulsion ... inappropriate medical care, people wandering on the streets of Calais, daily police checks, violence, despair and the taking of growing risks to attempt a passage to the United Kingdom, which, since the start of the year, have already caused the deaths of several exiles," it said.
The groups urge the French government to come up with a plan to tackle the sanitary situation in the camps while living up to its responsibility to protect the migrants on its soil.
The letter was also sent to the French interior minister and the minister for health and social affairs.
CNN's Stephanie Halasz and journalist Bastien Inzaurralde contributed to this report.