The residences, just a few miles from San Francisco, are teetering on the edge of the cliffs and most have already lost their backyards to erosion blamed on storms from El Niño, authorities said.
Pacifica city officials have declared a state of emergency and have asked residents of three apartment complexes to evacuate. The most recent evacuation came Monday, with 20 residents being told to leave.
"El Niño is hitting the city's coastline very hard and creating almost daily reports of impacts to both public and private property," Pacifica City Manager Lorie Tinfow said. "We need state and federal assistance to respond to the growing list of failing public infrastructure including the Beach Blvd. sea wall failure."
Residents were asked to leave as soon as possible and were only permitted in their apartments to remove their belongings.
"Recent bluff failures have resulted in unsafe conditions for living space at 310 Esplanade Avenue," said Chief Building Official Mike Cully. "Cavities in the bluff are forming to the south, west and north of the building and these critically over-steepened slopes are anticipated to fall back to more stable profiles in the next several days."
But not everyone is leaving.
"I'd rather be safe, yes, but I'd also rather not be on the street all night," one resident told CNN affiliate KGO. "I have nowhere else to go. I just moved in two months ago. And I don't have it like other people have it. I can't just pick up and leave. I don't have money like that."
They city rates the severity of the damage to buildings by rating them either yellow or red tag, red being more serious, but both ratings mean the building is unsafe.
Despite the latest evacuated building getting a yellow tag, Pacifica Police Chief Dan Steidle told CNN affiliate KGO "it's not a suggestion, it's an order from the building official that they cannot be in there."
"The City of Pacifica has contacted the American Red Cross and the Pacifica Resource Center with this information so they can provide assistance to the residents impacted by this action," Steidle said.
Since December, city officials say storms have damaged the pier, the Milagra Watershed Outfall and caused a sea wall to fail.