Obama stressed the need to uphold human rights in his remarks to the Vietnamese people and were broadcast on television in a nation that has a dismal record on the issue.
Human rights "is not a threat to stability" but reinforces it, Obama said.
Freedom of speech and expression "fuels" the economy, the President continued. "That is how some of our greatest companies began."
Highlighting freedom of the press, assembly, and religion, Obama said that while the U.S. is not trying to "impose the American form of government on Vietnam," the country should be more open to scrutiny in order to grow "stronger and more prosperous."
The White House said the President's speech was broadcast on Vietnamese television. It is almost unheard of for criticism of the country's human rights' record to be aired on state-run TV.
Before addressing the Vietnamese people on human rights, Obama met with six leaders from Vietnamese civil society. The President noted that several activists had been prevented from attending the meeting.
"We are not going to be able to release any names of folks that could not attend, but as the President said, a small number of invitees faced obstacles in attending the meeting," White House spokesman Eric Schultz said.
"The unprecedented nature of a meeting between a head of state and independent civil society in Hanoi demonstrates that we will continue to raise issues related to human rights with Vietnam even as our relationship progresses and we have continued differences
, and the President expressed to both the government and civil society that he believes our deepening engagement will allow us to more productively support progress on legal reforms and respect for universal values going forward," he added.
Audience members said Obama raised "important" issues in his speech, and praised his support for human rights.
"We feel really motivated to later express ourselves freely," Ngoc Dao, 23, told CNN.
Obama also discussed recent problems in the South China sea.
"In the South China Sea, the U.S. is not a claimant in current disputes, but we will stand with our partners in upholding key principles like freedom of navigation," Obama said.
"Vietnam will have greater access to the equipment you need to improve your security," Obama said. "Nations are sovereign and no matter how large or small a nation may be, its territory should be respected."
Though he did not mention the country by name, China has been making aggressive moves in the South China Sea, where it has multiple territorial claims.
"Big nations should not bully smaller ones," Obama said. "Disputes should be resolved peacefully."
China's Foreign Ministry responded Tuesday to Obama's comments, with spokeswoman Hua Chunyin saying: "The freedom of navigation they are talking about, is it true freedom of navigation enjoyed by every nation under international law, or it is a privileged freedom of navigation for American naval vessels and military aircraft?
"If the former, of course we welcome it with open arms, we protect it and resolutely support it," she added. "But if it is the latter, I think the international community would not agree."
Vietnamese fishermen told CNN this month that they are repeatedly boarded by Chinese-flagged vessels
and their equipment stolen and crew beaten.
Speaking about the war
A day after lifting the ban on the sales of lethal weapons to Vietnam, Obama also addressed the progress the U.S. and Vietnam have made since the Vietnam war.
"War, no matter what our intentions may be, brings suffering and tragedy," he said.
The President remarked on the loss of life on both sides, saying that family and friends in the U.S. "still ache" for the loss of the more than 58,000 Americans who were killed during the war.
Obama said that his daughters, like the predominantly young Vietnamese generation, have lived their lives knowing only "peace and normalized relations between Vietnam and the U.S."
The President also stressed the post-war progress has led to improved relations between the U.S. and Vietnam, saying, "As Vietnam has transformed so has the relationship between our two countries."
Obama credited Vietnam war veterans, including Sen. John McCain, for making that progress possible.
McCain and fellow vets, Secretary of State John Kerry and former Sen. Bob Kerrey, published an oped in The New York Times Tuesday headlined, "Moving on in Vietnam but remembering its lessons."
With President Obama's visit this week with crowds that we saw along the street today, the remarkably warm and generous welcome, the unbelievable excitement of people that we are here with a President of the United States at this moment is absolutely palpable and I think it is a demarcation point," Kerry told reporters Tuesday in Hanoi.
"I think that clearly we will never fail to honor the sacrifice of those who fought here and what their dreams were for this country, but I think one can say genuinely, definitively without a failing to honor past service, that we have reached a very new point in our relationship now," he added.
'I strongly support TPP'
Obama also used his speech to promote the Trans-Pacific Partnership (TPP), a U.S.-led trade deal that has attracted considerable criticism in recent months, with both Democratic presidential candidates
and presumptive Republican nominee Donald Trump
distancing themselves from the agreement.
"I strongly support TPP," Obama said, adding that it had "important strategic benefits."
"We have to get it done for the sake of our economic prosperity," he said.
Audience member Tung Dong, 21, said, "(Vietnam) should cooperate with the Americans more."
"For thousands of years we have been under the influence of China," he said. "We might as well have some Western influence now, more Western influence now."