The 62-year-old producer died Friday at a Miami hospital after an illness, according to Leslie Castro, executive assistant at the Federal Correctional Institution in Miami where Pearlman had most recently been serving a 25-year sentence for running Ponzi and bank schemes. He was convicted in 2008 of fraud and swindling investors out of more than $300 million.
In the mid-1990s he formed the iconic boy bands responsible for hits like "I Want It That Way" and "Quit Playing Games (with my Heart)" from the Backstreet Boys and "Bye Bye Bye," from *NSYNC.
An official cause of death was not announced.
Pearlman boosted the careers of entertainers who later achieved solo fame, like Backstreet Boy Nick Carter and former *NSYNC member Justin Timberlake. Timberlake tweeted Sunday he hoped Pearlman had "found some peace."
"His passing has touched a lot of people," said Mark NeJame, who represented Pearlman in civil lawsuits for many years.
"He literally revolutionized the world of pop music and [he was] a creative genius in that regard -- but [he] had a horrible dark side that eventually overshadowed his talents."
Pearlman "had a way of telling people what they wanted to hear," NeJame added. "He was very seductive."
Aaron Carter, brother of Backstreet Boys' Nick Carter, tweeted Sunday about Pearlman, "karma is real."
With Pearlman at the helm, the Backstreet Boys shot to stardom in 1996 and became one of the top-selling boy bands, selling more than 130 million records, according to the band's website.
Shortly after the launch of the Backstreet Boys, Pearlman followed up with a second boy band called *NSYNC -- which sold more than 55 million records.
*NSYNC's website was down Sunday in the wake of the news of Pearlman's death.
Former *NSYNC band members Lance Bass and Chris Kirkpatrick also posted their reactions on Twitter.
"He might not have been a stand up businessman," Bass tweeted, "but I wouldn't be doing what I love today [without] his influence. RIP Lou"
Kirkpatrick posted that he had "mixed emotions" about the news.
The Backstreet Boys haven't released an album since 2013, and in recent years members took time to focus on individual projects. The group recorded a song with the country duo Florida Georgia Line that was released Friday.
The Backstreet Boys are also working on a new album and there are plans for a short residency for the group in Las Vegas.