The group, one of 330 marching in the nation's oldest and world's largest Saint Patrick's Day parade, is the first Irish LGBT organization to do so in 25 years.
It was a groundbreaking day for many, and the crowd was notably festive.
"As soon as we turned onto the avenue, there was a great cheer for us," Fay said. "It was an extraordinary moment. It was very profound. There was a lot of tears, a lot of joy."
Fay, who had been arrested and lost his job in the decades that he spearheaded the protest against what many viewed as a discriminatory policy excluding LGBT groups from the march, said Thursday was "long, long waited for."
Parading along in the alliance's lavender and green sashes were gay rights activist Edie Windsor and Mayor Bill de Blasio, also breaking his boycott and participating in the event for the first time since his inauguration in 2014.
Mayor: 'Today, everyone is celebrating together'
De Blasio, in a shamrock green tie, told reporters before the parade on Thursday that it was a "very powerful" and "moving" day for the city of New York.
"Today, everyone is celebrating together," de Blasio said. "Today, the city is at peace, and the city is unified, and we all feel tremendous pride in all of the people who brought us together."
The group's inclusion was announced in September and came after months of internal negotiations that included the parade's organizers, Irish LGBT activists, the archbishop of New York and the head of the Irish Consulate in the city.
Dr. John Lahey, a longtime member of the board of directors of the New York City St. Patrick's Day Parade, authorized the negotiations to bring a prominent gay group into the parade after he was elected chairman in June 2015, according to Pat Smith, a spokesman for the group.
Out@NBCUniversal, an LGBT employee-group associated with the broadcaster of the parade, became the first gay group to march in the parade last year.
2015 also saw a similar ban on LGBT organizations lifted for Boston's Saint Patrick's Day Parade.
'I've never seen such positive feeling'
On Tuesday, Lahey was honored as the New York City Council Irish Man of the Year, in part because of the "principled stand" he took as chairman of the parade's board, Council Speaker Melissa Mark-Viverito said.
"I've never seen such a positive feeling," Lahey said Thursday from the parade grounds. "I think it's not only removing a cloud, I think it's a plus for the parade, strengthening it and reaching out to an even broader community."
The last gay organization to march in the parade, in 1991, was met with jeers and thrown objects from the crowd.
Former Mayor David Dinkins, who marched then in solidarity with the group, remembered police officers shielding him with umbrellas as people in the crowd threw full cans of beer.
"It was as ugly as anything in Mississippi in the '60s," Dinkins said on Thursday. This year, the former mayor said, "I'm delighted."
One group drops out
One group, the Catholic League, dropped out of the parade after the announcement to include the Lavender and Green Alliance.
Bill Donohue, president of the Catholic advocacy organization, said his group would sit out the parade because the event's organizers would not also allow a pro-life group to march.
"It's their parade. I wouldn't have had any group advocating their own cause," Donohue said. "If you want to let the gay group in, then you have to let the pro-life group in under its own banner."
New York's Saint Patrick's Day Parade has a tradition of not allowing signage for advocacy groups, with the exception of the "England out of Ireland" campaign, Smith said.
Some 200,000 revelers were expected to march in the parade -- the largest in memory, Smith said, and on one of the sunniest days in recent years.