The new King, clad in military uniform, promised to head a changed monarchy "for new times," in an address before the packed parliamentary chamber.
"We have a great country, we are a great nation -- let us trust in it," he told the packed parliamentary chamber and the millions watching nationwide.
After swearing an oath promising to uphold the constitution, the new monarch expressed his respect for the Parliament and the nation -- as well as his own emotion at becoming monarch.
Felipe paid tribute to his father's "extraordinary" legacy over nearly four decades, including his leadership as the country returned to democracy following the death of dictator Francisco Franco in 1975.
He also thanked his mother, Queen Sofia, for a lifetime of hard work and devotion to the Spanish people.
Felipe spoke of the need for a monarch to unify his country and to work with its political leaders, a message with added significance at a time when many in the country's Catalonia region want a referendum on independence.
He also vowed to carry out his duties, remain loyal to his nation and work for the interests of Spain. "I will honor the pledge and the oath I have just taken," he said.
The head of state must set an example of honest and transparent behavior, he said, perhaps a reflection of the scandals that have beset the monarchy in recent times.
Felipe reflected on the country's recent economic woes, which have fueled high unemployment rates. But he also spoke of the need for its people, at a time of adversity, to "look ahead to the Spain we are building together as I begin this reign."
This, he said, will mean modernizing to meet the challenges of a globalized world.
The proclamation ceremony is a more subdued affair than a coronation, without the pomp of foreign dignitaries and elaborate ceremonies.
But crowds cheered loudly as Felipe, his wife, now Queen Letizia, and their two daughters emerged onto the balcony of the royal palace, joined by Juan Carlos and Sofia, to salute the Spanish people.
Chants of "Long live the king" rang out in Spanish.
Felipe's elder daughter, Leonor, 8, becomes Princess of Asturias, first in line to the throne.
Stroke of pen
His father's 39-year reign had come to an end with the stroke of a pen and the stroke of midnight Wednesday.
Less than three weeks after he announced his surprise abdication, longtime leader Juan Carlos signed the step into law at the royal palace in Madrid, making his son the new king.
In another ceremony held Thursday morning to mark the transfer of power, Juan Carlos took off the red sash signifying his role as head of Spain's military, fastening it around his son's waist.
Crowds lined up waving the red and yellow Spanish flags as military bands marched in the streets of Madrid.
Juan Carlos has said Felipe, a former Olympic yachtsman, has "the maturity, the preparation and the sense of responsibility necessary" to serve as king and "to lead to a new stage of hope using his experience and the drive of a new generation."
The new King is regarded as being untouched by the accusations of corruption and excess that have plagued the royal family as many Spaniards continue to struggle to find jobs and pull themselves out of financial ruin.
At his side will be the new Queen Letizia, who before she married into the royal family worked as a TV presenter.
Princess Cristina, the new King's older sister, is embroiled in a tax fraud and money-laundering investigation. She and her husband, Inaki Urdangarin, have denied allegations that they diverted public funds from Urdangarin's foundation for private use.
Spaniards have long held Juan Carlos, 76, in high regard for shepherding the country into democracy.
But his popularity took a hit in 2012 over a controversial elephant-hunting trip to Africa while the nation was mired in a deep economic crisis.
Some Spaniards have called for the monarchy to be abandoned, favoring the establishment of a republic instead.
The anti-monarchy movement has been fueled by the scandals that have hit the royal family in recent years.
The biggest task for Felipe, 46, will be to make the dealings of the royal palace more transparent. His family has a relatively austere reputation compared with other European monarchies, but there has been increasing resentment in Spain over the cost of the royal family to the public.
CNN's Al Goodman reported from Madrid, while Laura Smith-Spark wrote in London and Steve Almasy in Atlanta.