(CNN)The number of deaths in Immigration and Customs Enforcement custody grew dramatically this year.
In the 2020 fiscal year, which ends today, 21 people died in ICE custody, according to a CNN tally of data released by the agency. That's more than double the number of deaths in fiscal year 2019 -- and the highest annual death toll since 2005.
Immigrant rights advocates say the deaths are a sign of deteriorating conditions, serious problems with medical care and ICE's flawed approach to handling the pandemic. ICE said it takes the health and safety of detainees seriously -- and maintains that deaths in its custody are "exceedingly rare."
More than a third of the people who died in ICE custody this year had tested positive for Covid-19 -- including a 56-year-old man from the Marshall Islands, who died in a Louisiana hospital, and a 61-year-old man from Mexico, who died in a Georgia hospital last week.
But advocates said the novel coronavirus isn't the only factor to blame for the growing number of deaths in ICE custody.
"We're seeing the pandemic is playing a role -- but also the conditions of detention, and what it does both to your mental health and the really poor medical care that exists inside," said Silky Shah, executive director of Detention Watch Network.
"As we're looking at this death toll going up, what it tells us is ... it's a system that shouldn't exist. People should be with their loved ones, with their families, being able to social distance and quarantine at home going through their immigration proceedings," Shah said. "They shouldn't be locked up."
Far fewer people are being detained by ICE
ICE responded to CNN's questions about why the death toll had grown so much this year with a statement describing the agency's efforts to prevent the spread of Covid-19, including reducing the population of its detention facilities to 75% capacity or less.
The total number of people detained in ICE facilities has decreased significantly during the pandemic.
Last year, the average daily population in ICE custody was 50,165 people. According to the latest statistics provided by the agency, as of September 25 there were 19,791 people in ICE custody.
ICE said at the beginning of the pandemic, it released more 900 individuals from custody after evaluating their records and is using the same methodology to evaluate "other potentially vulnerable populations currently in custody and while making custody determinations for all new arrestees."
There have been more than 6,100 confirmed coronavirus cases among ICE detainees since the pandemic began.
Scrutiny of the agency's facilities is intensifying
ICE facilities have long faced criticism for how they handle even routine medical care.
The spotlight has intensified recently in light of incendiary allegations about gynecological procedures purportedly performed on detainees in Georgia without informed consent. The agency has said it's cooperating with investigators and taking those allegations, raised by a whistleblower who'd worked as a nurse at the Irwin County Detention Center, seriously.
Last week a congressional investigation of ICE facilities found widespread failures in medical care, including some issues that resulted in death. ICE decried the recent investigation by the House Oversight Committee as a "one-sided review...done to tarnish" the agency's reputation.
ICE said any death that happens in the agency's custody is cause for concern, but also that deaths in its custody are "exceedingly rare."
"Statistically, fatalities in ICE custody occur at a small fraction of the national average for detained populations in federal or state custody -- less than 1% of the rate at which fatalities occur in federal and state custody nationwide," ICE said.
Where the death toll was the highest
Three of the detainees who died this year had been held at the Stewart Detention Center in Lumpkin, Georgia, before they were hospitalized with coronavirus. That's the highest death toll tied to any one ICE facility this year.
Stewart, the focus of CNN's 2018 series "Inside America's Hidden Border," is one of the country's largest immigrant detention centers.
For months, immigrant advocates warned the pandemic would worsen already-dangerous conditions inside the facility.
Amilcar Valencia was one of them. Sadly, he said, the growing death toll is not a surprise.
"It raises lots of alarms when three people have died. ... Just from the public health perspective, the detention center is putting at risk a lot of people just by having people detained there -- all the population inside, all the workers, everyone in the county is at higher risk because we have this detention center in South Georgia," he said.
A spokesman for CoreCivic, the private prison company that operates Stewart, said the company is working closely with ICE to ensure the health and safety of everyone at the facility.
"We have responded to this unprecedented situation appropriately, thoroughly and with care for the safety and well-being of those entrusted to us and our communities," spokesman Ryan Gustin said.
Regarding the deaths of detainees who'd been held at Stewart, Gustin said the detention center's size is an important factor to consider.
"We extend our heartfelt sympathy to the family and loved ones of those individuals that passed away," Gustin said. "While we take any loss of life in our facilities very seriously, it's important for context to understand that our Stewart facility cares for a higher number of people than most other immigration facilities in the country."
Valencia, the executive director of El Refugio ministry, is regularly in touch with people detained at Stewart.
"They say how fearful they are for their safety and health inside of the facility ... That's a common thing that comes up in every letter that we get," he said.