(CNN) -- Smoldering embers that had been removed from a fireplace likely caused the blaze that killed three children and their grandparents in a Connecticut home Christmas morning, authorities said Tuesday.
"The fire appears to have been caused by hot fireplace ashes and embers, which had been discarded," said Barry Callahan, the chief fire marshal in Stamford, Connecticut.
He said the fire that engulfed the large waterfront Victorian home was accidental, according to preliminary findings, but an investigating is ongoing.
"We're still trying to put the pieces together," he said.
It was unclear whether working smoke detectors had been installed in the house, which was under renovation, officials said.
The city of Stamford determined the three-story structure was unsafe and razed it Monday.
All that stood was the mailbox, around which grieving neighbors in the wealthy Shippan Point neighborhood placed flowers.
The fire tore through the million-dollar home shortly before 5 a.m. Sunday. Sometime after 3 a.m., hot fireplace ashes were removed and taken to an area in the rear of the house, Callahan said Tuesday.
Intense flames and heat pushed back firefighters searching for the trapped family members, Acting Fire Chief Antonio Conte told reporters. When they made it inside, they found the bodies of the grandmother and the three young girls -- a 10-year-old and 7-year-old twins.
The grandfather, Lomer Johnson, fell through the rafters outside a window where crews found one of the girls, Conte said. It appeared he had been trying to get her out, Conte said.
The girls' mother, Madonna Badger, and a friend escaped the blaze.
Johnson worked as a Santa at Saks Fifth Avenue's Manhattan store this holiday season, the department store said in a statement. He and his wife would have celebrated their 49th wedding anniversary Monday, WFSB reported.
Badger, a New York advertising executive, purchased the Connecticut home for $1.7 million in December 2010, according to property records.
City officials said they planned to keep investigating the fire, but will need time.
"It's going to take a while," Conte said. "That poor woman lost her entire family in one fell swoop."
A recent amendment to the home's building permit application indicated plans to install a new security and smoke detection system, Stamford Mayor Michael Pavia said, but it was unclear whether working smoke detectors had been installed at the time of the blaze.
"Mrs. Badger lost her three children and her two parents. When we made the initial contact with Mrs. Badger, the last thing on our minds to talk to her about was whether or not her building permit was valid or her smoke detector was working or any of that. ... As the follow-up investigation continues ... those are questions that we will pursue, but not now," he said.