The two men were shot in the head about 1:50 p.m. in Ozone Park, a working-class section of Queens with a burgeoning Bangladeshi population.
"There is nothing in the preliminary investigation that would indicate they were targeted because of their faith," NYPD Deputy Inspector Henry Sautner told reporters.
Imam Maulama Akonjee, 55, and his assistant, Thara Uddin, 64, were transported in critical condition to Jamaica Hospital, where they later died, according to Sautner.
The victims were wearing religious attire at the time of the shooting, according to a law enforcement source with direct knowledge of the investigation.
Al-Furqan Jame Masjid Mosque is about two blocks from where the men were gunned down. They were walking home after midday prayers.
Investigators were trying to determine whether an earlier dispute at the mosque led to the shooting, according to law enforcement officials.
Nazim Uddin, a neighbor of the imam, told CNN he ran outside from his home when he heard gunfire. He remembered the imam as a good man who only a day earlier had delivered a eulogy at the mosque.
On Saturday evening, dozens of residents, mostly men, gathered under elevated train tracks waving signs and chanting, "We want justice!"
Police said surveillance video captured the gunman leaving the scene with his weapon in hand.
"The perpetrator of these senseless killings must be swiftly apprehended and face the full force of the law," Afaf Nasher, executive director of the New York chapter of the Council on American-Islamic Relations (CAIR) said in a statement. "We ask anyone with information about this attack to contact appropriate law enforcement authorities."
Mayor Bill de Blasio said Muslim communities always are in the "crosshairs of bigotry."
"It remains critical that we work to bridge the divides that threaten to undermine the greatness of our city and country," he said in a statement Sunday. "Rest assured that our NYPD will bring this killer to justice."