The U.N.-brokered agreement, which has strong international backing, was signed in Skhirat, Morocco, Thursday.
It calls for unity between the two rival governments -- a self-declared, Islamist-leaning one in the capital Tripoli, and an internationally-recognized one in the east, based in Tobruk.
U.N. officials call it a historic day -- as well as the beginning of a difficult journey for the North African nation.
"Today, participants in the Libyan political dialogue have turned a page in the history of Libya," said Martin Kobler, special representative for the secretary general, in a statement.
"After a period of political divisions and conflict, Libya is restarting its political transition. The agreement puts in place a single set of legitimate institutions -- essential building blocks towards a peaceful, secure and prosperous Libya."
But the deal faces resistance, and questions over how it will be implemented.
Libya has been the scene of chaos and fighting between city states, rival tribes and Islamist militias after the 2011 revolution that saw the fall of Moammar Gadhafi. The longtime leader was killed after being captured by rebel forces in his hometown of Sirte, Libya, in October 2011.
The lack of government control after Gadhafi's fall, led to a power vacuum, with various Islamist militia controlling parts of the country, including al Qaeda and most recently, ISIS. Up to 800 Libyans have recently returned from fighting with ISIS in Iraq and Syria according to a U.N. report.
"It is far from clear whether this new deal will be fully embraced by both sides," said CNN Terrorism Analyst Paul Cruickshank. "The stakes could not be higher. The consequences of failure could be catastrophic for Libyan and European security. As a United Nations monitoring group warned last month, ISIS is taking advantage of a simmering civil war between both sides to solidify and expand its presence over a stretch of Libyan coastline around Sirte just a few hundred miles away from the southern islands of the European Union."
A Libyan source calls the agreement "a Christmas present come too early".
He said it allows the international community to be in a position of having to deal with one entity inside the country, but does not address other fundamental issues.
He's worried about a violent backlash to the new government by the militias. He's also concerned the deal did not address who would oversee disarmament of the various militias across Libya.
But for many Libyans craving for peace, the deal is a step in the right direction.
"We have to rebuild the Libyan State," said Salah Al-Mahzoum, deputy speaker of Libya's General National Congress . "Today, we started."