(CNN) -- Slightly more than 580,000 customers from North Carolina to Maine are still without power as a result of the damage caused by Hurricane Irene, according to estimates released Friday by the U.S. Department of Energy.
Over 100,000 customers have had electric power restored since Thursday afternoon.
Over 6.4 million customers lacked power at the height of the outages, the department noted.
The rate of recovery from the storm has varied sharply up and down the Eastern Seaboard. Irene killed 43 people from Florida to New England while dumping torrential rain. Some of the worst flooding struck Vermont, New Jersey and upstate New York.
One sign of an accelerating recovery: Amtrak announced Friday that full service on all of its East Coast rail lines will be restored by Sunday.
The pace of recovery operations also accelerated in hard-hit Vermont Friday, as road access was improved to previously isolated communities such Killington, Menden, Pittsfield and Bridgewater, according to Suzie Dundas, a communications official for Killington.
Further south, North Carolina Gov. Bev Perdue announced that four additional counties in her state -- Currituck, Pitt, Onslow and Washington -- had been approved for federal disaster relief. A total of 13 counties in North Carolina now qualify for individual assistance under the relief program.
Perdue also announced a plan to reopen all of NC 12 -- the highway connecting most of that state's barrier islands to the mainland. A temporary bridge will be constructed -- at the cost of $10 million -- to allow full traffic flow in less than a month, the governor's office announced in a written statement.
Residents of Hatteras Island, North Carolina, will finally start heading home Sunday morning -- over a week after most of them evacuated ahead of the hurricane.
The staged re-entry through Tuesday will include the villages of Buxton, Hatteras and Frisco, Dare County Energency Management announced Thursday.
"Conditions, utility service and other supporting infrastructure in those villages is the most conducive for re-entry at this time," the county said in a statement. "As conditions improve in the remaining villages, re-entry will be established and announced as soon as possible."
Visitors will not be able to go to Hatteras Island. Instead, they can stay north of Oregon Inlet, the county said.
About 2,500 residents of the island who did not take part in a mandatory evacuation have been receiving vital supplies and services.
Meanwhile, the extent of Irene's damage has become more evident in upstate New York, where the storm battered a cluster of communities 50 miles southwest of Albany.
"There is a lot of damage left to clean up. I know the town of Prattsville has been almost completely condemned," said Jacob Hubbell of neighboring Margaretville. "Fleischmanns isn't doing too well either, and Main Street (in) Margaretville has been closed."
"It's safe to say that we probably won't be back to normal in the Catskills for at least a month."
In northern New Jersey, the Passaic River has begun to settle back into its banks, according to the National Weather Service. Most locations are expected to drop below flood stage during the weekend as the high water works its way downstream.
The development will be welcome news in the towns of Wayne, Totowa, Little Falls, Paterson and Woodland Park, where about 1,700 residents were evacuated from their homes this week.
President Barack Obama will travel to Paterson on Sunday to view the damage, the White House has announced.
The economic toll from the storm is significant, with the federal government estimating that the cost from wind damage alone will exceed $1 billion. Analysts have put the total expected cost of Irene much higher.
Agriculture Secretary Tom Vilsack said Thursday the storm also took a toll on agricultural production.
"I had an opportunity to take a look at fields in North Carolina," he said. "I have never seen anything like it. The corn was just totally destroyed. Tobacco hit hard, cotton hit hard." It remains to be seen how some other crops, such as soybeans and tomatoes, fared, he said, but "it's very clear that farmers in North Carolina, Virginia, along the East Coast have suffered pretty significant losses."
But, he said, it's unlikely that higher prices will result, as "we have such a diverse agriculture in the United States and we have so many acres planted and so many different crops. I don't think this is going to affect much of anything."
The federal government's tab for the storm could exhaust the $800 million left in the Federal Emergency Management Agency's disaster relief fund before the fiscal year ends on September 30.
With conservative House Republicans, led by Rep. Eric Cantor, R-Virginia, calling for spending cuts to offset any increase in emergency funds -- a measure opposed by many Democrats -- the ability of Congress to act quickly on the issue remains uncertain.
Mayor Jeffery Jones of Paterson said he was "outraged" about the funding dispute. "Mother Nature has a mind of her own, a will of her own, and we can't have the petty wrangling going on when we have folks in dire need," he said.
New Jersey Gov. Chris Christie echoed those sentiments during a news conference Wednesday, saying, "We don't have time to wait for folks in Congress to figure out how they want to offset this stuff with the budget cuts. Our people are suffering now. And they need support now."
CNN's Ed Payne, Phil Gast, Amber Lyon, Nicole Saidi, Alan Silverleib, and Stephanie Gallman contributed to this report