Rep. Ruben Gallego confirmed the comment to CNN's Wolf Blitzer on "The Situation Room."
"Yes, we heard him say that," the Arizona Democrat said.
A source familiar with Kelly's comments in the closed-door meeting with Democrats and the Congressional Hispanic Caucus told CNN that Kelly said he has worked to educate the President and move him away from his campaign promises.
Another source in the room with access to notes from the meeting told CNN that Kelly was talking about the difference between campaign speech and actual policy, saying, "You make campaign promises but then you learn the reality of governing, which is difficult."
The wall comments, Kelly said, were just "not fully informed" during the campaign.
The "uninformed" comment was first reported by The Washington Post.
No new proposal from White House
Lawmakers described the meeting as "positive" but also mostly a "rehash" of where both sides stand on stalled immigration talks.
The packed meeting with Kelly, House Minority Whip Steny Hoyer and key Hispanic Democrats lasted an hour, and was "cordial," according to Sen. Catherine Cortez Masto, a Nevada Democrat.
Kelly did not bring a new proposal from the White House on immigration, Rep. Luis Guiterrez, D-Illinois, said, calling it "unfortunate."
Democrats have been pushing for a solution for the Deferred Action for Childhood Arrivals policy
, which protected young undocumented immigrants who came to the US as children and which President Donald Trump is ending, before they vote for government funding.
Last week, Trump rejected a bipartisan proposal from senators that would have done so, in a meeting where he used vulgar language to refer to certain countries.
The infamous "shithole countries" remark didn't come up in Wednesday's meeting, lawmakers said, as the conversation focused on the narrow DACA fix Democrats would like to see and the White House's priorities for a deal.
"He called the meeting to listen, is how he framed it," Rep. Pete Aguilar of California said of Kelly.
Hispanic Caucus members said they were surprised Kelly said he was unfamiliar with a bipartisan bill Aguilar worked on with Texas Republican Rep. Will Hurd, and Aguilar said White House legislative director Marc Short, who was also in the meeting, has been informed about the bill.
Kelly said he would look into it.
"Gen. Kelly indicated that the President is motivated to have a fix," CHC Chairwoman Michelle Lujan Grisham said. "He didn't say he'd sign the bill, but he definitely indicated there would be positive review."
Gutierrez said he took notes on what Kelly said, quoting Kelly: "If this is the best we can do, the President will sign it."
Still, lawmakers expressed frustration with the lack of progress.
Arizona Rep. Raul Grijalva said the meeting was a "rehash" of old meetings, with the same talking points and positions.
New Jersey Sen. Robert Menendez said it was a "regurgitation" of both sides and the administration is too fixated on the border.
"We had a cordial conversation but we don't need a cordial conversation, we need a substantive one," he said.
Part of the group that negotiated the Senate compromise, Menendez said the administration's position on a deal is a "continuously moving" target and "it continuously expands."
Both men said DACA recipients were being "held hostage" to extract concessions.
Two emotional moments
Two moments in the Kelly meeting that were more emotional than the rest of the cordial conversation, a source in the room said.
At one point, things got a bit tense when Kelly used the term "chain migration" and Rep. Linda Sanchez confronted him, saying that's a term considered offensive and the issue is actually family reunification.
"Please don't use that term," she said.
Kelly didn't apologize or discuss it further but took it in, and later in the meeting acknowledged the term is disputed when discussing the issue.
Apology from Gutierrez
The source also said Gutierrez took a moment to say, noting he wanted to say it in front of colleagues, that he wanted to acknowledge in the past he got heated and used words he shouldn't have, a reference to past comments he made about Kelly and his military service. Kelly made a joke about having done so as well and it was positive, the source said.
On the House floor in September, Gutierrez said: "General Kelly, when he was the head of Homeland Security, lied straight to the faces of the Congressional Hispanic Caucus about preventing the mass deportation of (DACA recipients). ... General Kelly is a hypocrite who is a disgrace to the uniform he used to wear."
This story has been updated to included additional developments.