(CNN) -- Beleaguered New York mayoral candidate Anthony Weiner got into a war of words Wednesday with a man who lashed out at him for being "married to an Arab," in addition to chastising the former congressman about his well-documented sexting scandal.
The incident began inside a bakery in Brooklyn's Borough Park neighborhood, where the man yelled out to Weiner, who had just bought cookies and an iced coffee, "You're a real scumbag."
Weiner offered a quick retort, then said to no one in particular, "Very nice, that's a charming guy right there." It's then that the man, later identified as Saul Kessler, responded: "Married to an Arab."
The comment is an apparent reference to Weiner's wife, Huma Abedin, who works for Hillary Clinton. Abedin's late father was an Islamic scholar from India and her mother, a sociologist, is from Pakistan. Weiner is Jewish, as were most of those inside the Brooklyn bakery.
The Democratic candidate responded with, "Very nice, in front of children... That is charming." Kessler didn't stop talking, even as Weiner was leaving the bakery, saying, "You are disgusting, disgusting."
"It takes one to know one, jackass," Weiner replied.
The two then went face-to-face, with Kessler accusing Weiner -- who resigned in disgrace after 12 years in Congress representing part of New York City after admitting to sending sexually suggestive images and carrying on inappropriate relationships with women over the Internet -- of doing "disgusting things, and you have the nerve to even walk around in public."
A visibly agitated Weiner responded,"And you're a perfect person? You're my judge? What rabbi taught you that?"
At one point, Kessler implored Weiner to "think about your wife, how could you take the person you're most closest to ... and betray her?"
The mayoral hopeful then accused Kessler of acting like he is superior, even though he doesn't "have the moral authority to judge me."
The back-and-forth went on for about a minute longer before Weiner left the bakery. He seemed to quickly brush off the confrontation, saying, "He has every right to (challenge me). It's America."
The Weiner campaign released a video of Wednesday's exchange, a version of which also was captured by blogger Jacob Kornbluh. Talking with reporters later in the day, Weiner described what happened as "just some enthusiastic exchange with a voter."
"Look, you're allowed to say stuff to me," said Weiner, who, according to a Quinnipiac University poll released Tuesday, trails three other Democratic candidates ahead of next week's primary. "But if you're going to say vile things about me and my family, you should expect that I'm going to go back at you.
"Hecklers don't get a chance to necessarily get the last word if they cross the line."
Reached by CNN later in the day, Kessler admitted calling out Weiner for being "married to an Arab."
"It's just a certain feeling I have as a Jew," said the 51-year-old real estate manager, explaining his remark. "And my attitude is that -- not all Arabs -- but, in general, ... they want to kill every Jew."
Kessler said of a Jewish person marrying into such a family: "It's like a betrayal."
Kessler also said Weiner has bothered him for some time. Kessler called him a "phony." But Kessler said he should have acted differently.
"(Given that I felt) a little bit of anger, I probably shouldn't have said it," Kessler said about the Arab comment. "I shouldn't have called him a scumbag."
"But it was in the heat of the moment."