"You've lost very much. You've given so very much, but please know that he will always be with you in the deepest recess in your heart, in your every thought," Biden told family members and loved ones of the victims.
"He'll be the voice you hear in your ear telling you it's OK. He'll be that feeling in your chest that just calms you down. That look from the mirror that gives you confidence to move forward and that sunset that says, 'I see you. I see you,'" he added.
The service in downtown Chattanooga comes nearly a month after Mohammad Abdulazeez shot and killed four Marines and one Navy sailor at a Navy operations support center.
The victims were Marine Gunnery Sgt. Thomas Sullivan, Lance Cpl. Squire K. Wells, Marine Staff Sgt. David Wyatt, Marine Sgt. Carson Holmquist and Navy Petty Officer Randall Smith.
In late May, the vice president lost his son, Beau Biden, to brain cancer.
Since Beau Biden's death, the vice president has spoken of his son in public speeches and casual conversations. In July, Biden said his son's death was still an "open wound" while participating in a roundtable on community colleges in Colorado.
"The day will come, as hard as it is to believe, when his memory brings a smile to your lips and a tear to your eyes. It takes time," Biden told the audience at the service on Saturday.
"In my experience, it takes getting through every season at least once, but it will come. It will come. And that's when you know you'll be all right," he said.
Biden, who met with the families privately ahead of the public memorial, acknowledged that while he didn't have the privilege of knowing the victims personally, he knew they were "committed, determined and trustworthy."
"They were my son and so many other sons I know," Biden told the gathering, which included Secretary of Defense Ashton Carter, who also spoke.
In June, the vice president, along with President Barack Obama, attended the funeral services for the Rev. Clementa Pinckney, the state senator who was one of nine people killed in the racially motivated shootings in Charleston, South Carolina. Two days later, Biden attended mass at Rev. Pinckney's church, Emanuel AME, where he told the audience, "I know from experience no words can mend a broken heart."
Beau Biden is the second of Joe Biden's children to precede their father in death. His 1-year-old daughter Naomi was killed in a Christmastime car accident in 1972. The crash also took the life of Biden's first wife, Neilia, who was Beau's mother.