Perez helped broker a tentative contract agreement on Friday between dockworkers and their employers after President Barack Obama dispatched him to San Francisco this week.
In an interview with CNN hours after the deal, Perez said he's spoken to many business owners whose livelihoods took a hit because of the protracted dispute.
"I'm ever mindful of the fact that there certainly has been some damage done," Perez said. "If you talk to the citrus farmers whose citrus rotted on the port, that's damage that has been done."
Perez added, however, that he believes the conflict was resolved before it could do more permanent damage to country's 2015 economic outlook.
"2014 was the best year since the late 90s in terms of job growth and we see a real hop in the step of this economy," he said. "I think in the grand scheme, I'm confident that we can recover quickly."
Contract negotiations between the Pacific Maritime Association, which represents shipping companies and port operators, and the International Longshore and Warehouse Union, which represents dock workers on the West Coast, started in May. The two parties were unable to come to an agreement and have been without a contract since July.
The standoff has been a nightmare for U.S. businesses that use the West Coast ports to import and export their goods. For weeks, vessels carrying thousands of containers have been sitting on the waterfront waiting to be loaded or unloaded, and the problem grew exponentially worse when the PMA temporarily stopped all loading and unloading of ships this month.
The heads of the PMA and ILWU hailed the agreement as "good for workers and for the industry" in a joint statement Friday.
"We are also pleased that our ports can now resume full operations," they said.
Analysts say the process of clearing the ports of the existing congestion could take weeks, if not months.
Perez, who has been labor secretary since 2013, told CNN that playing mediator in the West Coast ports dispute was "among the most difficult" challenges he has faced so far in office.
"Helping to facilitate resolution involves the development of mutual trust and so I had to do that and I had to do it fast because I didn't have the luxury of time," he said.