"No, I don't. I don't regret it," Bush said sharply, growing testy with a questioner while talking to reporters.
Pressed further, a more agitated Bush fired off: "No, do you have a better term? OK, you give me, you give me a better term and I'll use it. I'm serious."
Interestingly, it was an answer that Trump gave almost verbatim the night before, also in New Hampshire, when a reporter asked the GOP front-runner if he was aware that it was considered offensive.
"You mean it's not politically correct, and yet everybody uses it?" Trump said. "You know what? Give me a different term."
While Trump declared this week that "anchor babies" are not American citizens, Bush used the term in a more couched fashion. Bush, in an interview with conservative radio host Bill Bennett, said "greater enforcement" was needed "so you don't have these, you know, anchor babies as they're described, coming into the country."
And Bush attempted to clarify his use of the word.
"What I said is that it's commonly referred to that. I didn't use it as my own language. You want to get to the policy for a second? I think that people born in this country ought to be American citizens," he said.
The response from Bush and Trump to criticism of the comments marked a rare moment where the two surprisingly lined up on immigration. Both candidates have been battling in recent weeks as Trump has surged in the polls, in large part due to his outspokenness on the issue.
But Bush, who speaks Spanish fluently and whose wife is from Mexico, has a more open policy on immigration and has made his connections to the Latino community a central part of his campaign. He's previously talked about how his kids were taunted growing up because of the color of their skin, and Bush has used his life's story as a way to try and connect with nontraditional Republican voters.
Democrats have been quick to pounce on Bush's "anchor babies" comment to paint him as out of step with the mainstream. Hillary Clinton, for example, tweeted a response Thursday afternoon on Twitter: "How about 'babies,' 'children,' or 'American citizens.'"
Bush has publicly disagreed with Trump's call to end birthright citizenship, and earlier Thursday at a restaurant in New London, New Hampshire, the former governor invoked two other candidates to make his point.
"Now if people ... have a visa, and they have a child who is born here, then I think they ought to be American citizens. People like Marco Rubio, by the way, that's how he came. You know, to suggest that we make it impossible for a talented person like that not to be a candidate for president. Or Ted Cruz. I mean, I think we're getting a little overboard here. We're listening to the emotion rather than to the reality of this."
Later in Keene, Bush described the immigration debate as "hurtful," adding that "we ought to tone down the rhetoric." He also made a not-so-subtle dig at Trump's larger-than-life style of campaigning.
"You don't win when you're the large, you know, dog in the room, where it's all about you. You win when you connect with people about their aspirations, not about how great you are, how rich you are, or how this you are or how that you are," he said. "That's not leadership."