Egan was pronounced dead of a heart attack at NYU Langone Medical Center at 2:20 p.m.
"Thank God he had a peaceful death, passing away right after lunch today," Cardinal Timothy Dolan, Egan's successor and current archbishop of New York, said in a statement.
Egan was appointed archbishop of New York in 2000 and later cardinal in 2001 by Pope John Paul II. Egan retired from the archdiocese in 2009.
As the leader of one of the largest Catholic communities in the country, Egan oversaw a growth of more than 200,000 registered parishioners and welcomed Pope Benedict XVI on a visit to New York City in 2008 to commemorate the 200th anniversary of the founding of the Diocese of New York.
"Cardinal Egan spread love and knowledge, and brought comfort to countless New Yorkers and others across the country and the world who sought his guidance and counsel -- especially in the aftermath of 9/11," said New York Mayor Bill de Blasio in a statement.
Egan served previously as the bishop of Bridgeport, Connecticut, from 1988 to 2000, when the diocese was at the center of a child sex abuse scandal.
The Bridgeport diocese in 2001 paid $15 million to 26 plaintiffs to settle sexual abuse claims against eight priests.
Egan apologized for his involvement in the scandal in a letter read across the New York archdiocese in 2002, though he later retracted that apology in an interview with Connecticut Magazine 10 years later.
"I should never have said that," Egan said. "I did say 'If we did anything wrong, I'm sorry,' but I don't think we did anything wrong."
Egan was born in Oak Park, Illinois, in 1932 and was ordained in Rome at the Pontifical North American College in 1957. Funeral arrangements are pending, the archdiocese said.