Ivanka Trump and Jared Kushner attend a Cabinet meeting in the Cabinet Room of the White House on March 8, 2018 in Washington, DC. / AFP PHOTO / MANDEL NGAN        (Photo credit should read MANDEL NGAN/AFP/Getty Images)
MANDEL NGAN/AFP/Getty Images
Ivanka Trump and Jared Kushner attend a Cabinet meeting in the Cabinet Room of the White House on March 8, 2018 in Washington, DC. / AFP PHOTO / MANDEL NGAN (Photo credit should read MANDEL NGAN/AFP/Getty Images)
Now playing
02:40
Trump wanted Jared and Ivanka fired, new book claims
Rep. Marjorie Taylor Greene (R-GA) speaks during a press conference outside the U.S. Capitol on February 5, 2021 in Washington, DC. The House voted 230 to 199 on Friday evening to remove Rep. Marjorie Taylor Greene (R-GA) from committee assignments over her remarks about QAnon and other conspiracy theories.
Drew Angerer/Getty Images
Rep. Marjorie Taylor Greene (R-GA) speaks during a press conference outside the U.S. Capitol on February 5, 2021 in Washington, DC. The House voted 230 to 199 on Friday evening to remove Rep. Marjorie Taylor Greene (R-GA) from committee assignments over her remarks about QAnon and other conspiracy theories.
Now playing
03:20
Marjorie Taylor Greene lashes out at media after backlash over controversial caucus
AP
Now playing
03:16
Maxine Waters: Jim Jordan is a bully and I shut him down
US Representative Marjorie Taylor Greene, Republican of Georgia, leaves her office on Capitol Hill on February 4, 2021 in Washington, DC.
NICHOLAS KAMM/AFP/Getty Images
US Representative Marjorie Taylor Greene, Republican of Georgia, leaves her office on Capitol Hill on February 4, 2021 in Washington, DC.
Now playing
03:51
Marjorie Taylor Greene launching 'America First' caucus
US President Joe Biden delivers remarks on Russia at the White House in Washington, DC on April 15, 2021. - The United States announced sanctions and the expulsion of 10 Russian diplomats Thursday in retaliation for what Washington says is the Kremlin's US election interference, a massive cyberattack and other hostile activity.
JIM WATSON/AFP/Getty Images
US President Joe Biden delivers remarks on Russia at the White House in Washington, DC on April 15, 2021. - The United States announced sanctions and the expulsion of 10 Russian diplomats Thursday in retaliation for what Washington says is the Kremlin's US election interference, a massive cyberattack and other hostile activity.
Now playing
02:22
White House backtracks on refugees decision after criticism
Biden speaks from the Treaty Room in the White House on April 14, 2021 in Washington, DC.
Andrew Harnik/Pool/Getty Images
Biden speaks from the Treaty Room in the White House on April 14, 2021 in Washington, DC.
Now playing
02:44
'National embarrassment': Biden reacts to mass shootings
MOSCOW, RUSSIA - APRIL 15:  Russian President Vladimir Putin (L) talks to talks to Foreign Minister Sergey Lavrov during a meeting with Uzbekistan President Islam Karimov in the Kremlin on April 15, 2013 in in Moscow, Russia. Karimov is on a state visit to Russia. (Photo by Sasha Mordovets/Getty Images)
Sasha Mordovets/Getty Images
MOSCOW, RUSSIA - APRIL 15: Russian President Vladimir Putin (L) talks to talks to Foreign Minister Sergey Lavrov during a meeting with Uzbekistan President Islam Karimov in the Kremlin on April 15, 2013 in in Moscow, Russia. Karimov is on a state visit to Russia. (Photo by Sasha Mordovets/Getty Images)
Now playing
02:07
Russia to expel 10 US diplomats in 'tit-for-tat response' to Biden sanctions
Now playing
03:10
Avlon: Here's what we know 100 days since the Capitol riot
A Russian flag flies next to the US embassy building in Moscow on October 22, 2018. - US national security advisor John Bolton is in Moscow holding meetings with senior Russian officials following Washington's weekend announcement of withdrawal from the Cold War-era Intermediate-Range Nuclear Forces Treaty, known as the INF. (Photo by Mladen ANTONOV / AFP)        (Photo credit should read MLADEN ANTONOV/AFP via Getty Images)
Mladen Antonov/AFP/Getty Images
A Russian flag flies next to the US embassy building in Moscow on October 22, 2018. - US national security advisor John Bolton is in Moscow holding meetings with senior Russian officials following Washington's weekend announcement of withdrawal from the Cold War-era Intermediate-Range Nuclear Forces Treaty, known as the INF. (Photo by Mladen ANTONOV / AFP) (Photo credit should read MLADEN ANTONOV/AFP via Getty Images)
Now playing
02:17
Political scientist: US-Russia relations are in the toilet
WASHINGTON, DC - JUNE 13: Rep. Michelle Lujan Grisham (D-NM) speaks during a news conference on immigration to condemn the Trump Administration's "zero tolerance" immigration policy, outside the US Capitol on June 13, 2018 in Washington, DC. (Photo by Toya Sarno Jordan/Getty Images)
Toya Sarno Jordan/Getty Images
WASHINGTON, DC - JUNE 13: Rep. Michelle Lujan Grisham (D-NM) speaks during a news conference on immigration to condemn the Trump Administration's "zero tolerance" immigration policy, outside the US Capitol on June 13, 2018 in Washington, DC. (Photo by Toya Sarno Jordan/Getty Images)
Now playing
02:39
Governor settles with former campaign staffer who accused her of sexual mistreatment
pool/cnn
Now playing
01:56
Hear what Dr. Gupta said when Cruz went maskless before
Now playing
02:30
Biden's decision to withdraw from Afghanistan is personal for this lawmaker
President Joe Biden speaks from the Treaty Room in the White House on Wednesday, April 14, 2021, about the withdrawal of the remainder of U.S. troops from Afghanistan.=
Andrew Harnik/AP
President Joe Biden speaks from the Treaty Room in the White House on Wednesday, April 14, 2021, about the withdrawal of the remainder of U.S. troops from Afghanistan.=
Now playing
02:10
Why Biden made his Afghanistan announcement in this particular room
U.S. President Joe Biden speaks from the Treaty Room in the White House about the withdrawal of U.S. troops from Afghanistan on April 14, 2021 in Washington, DC. President Biden announced his plans to pull all remaining U.S. troops out of Afghanistan by September 11, 2021 in a final step towards ending America's longest war.
Andrew Harnik/Pool/Getty Images
U.S. President Joe Biden speaks from the Treaty Room in the White House about the withdrawal of U.S. troops from Afghanistan on April 14, 2021 in Washington, DC. President Biden announced his plans to pull all remaining U.S. troops out of Afghanistan by September 11, 2021 in a final step towards ending America's longest war.
Now playing
01:03
Biden: It's time to end the forever war
Kinzinger
CNN
Kinzinger
Now playing
05:56
What Republican lawmaker fears after US troops leave Afghanistan
Rep. Matt Gaetz, R-Fla., questions witnesses during a House Armed Services Committee hearing on Capitol Hill, Wednesday, April 14, 2021, in Washington.
Manuel Balce Ceneta/AP
Rep. Matt Gaetz, R-Fla., questions witnesses during a House Armed Services Committee hearing on Capitol Hill, Wednesday, April 14, 2021, in Washington.
Now playing
02:59
Women detail late-night parties with Gaetz
(CNN) —  

Members of President Donald Trump’s family who work in his administration are “an influence that has to be dealt with,” former White House chief of staff John Kelly said Tuesday.

Though Kelly did not mention by name Trump’s daughter and senior adviser Ivanka Trump or her husband Jared Kushner, who also serves as a senior adviser to the President, he regularly sparred with them during his tenure at the White House, which ended earlier this year.

He also told Bloomberg Televisions’ “The David Rubenstein Show” that he was not talking about first lady Melania Trump, whom he called “a wonderful person.”

CNN has reached out to the White House for comment on Kelly’s remarks.

The comments from the former chief of staff, who has kept a low profile since leaving his post in January, provide a fresh look at how he views the Trump family members working for the administration.

Ivanka Trump’s blurred line between staffer and daughter long irked Kelly, according to people familiar with the matter, who say his regimented sense of order was tested by the implicit access being a child of the President affords. Kelly remarked privately that Ivanka Trump is just “playing government,” a source told CNN last year, and he largely brushed aside her agenda, once disregarding her child tax credit as “a pet project.”

The former official’s relationship with Ivanka Trump and Kushner during his tenure was notably marked by his refusal to grant them security clearances while he faced pressure to do so from the President. In April, CNN reported that Trump granted the couple clearances after both Kelly and former White House counsel Don McGahn refused to do so.

According to “Kushner Inc.,” a book focused on the couple, the President wanted the two to leave their posts at the White House and sought help to push them out from Kelly, whom he told, “Get rid of my kids; get them back to New York.”

Instead of firing Trump’s daughter and son-in-law, the book said Kelly and the President “agreed that they would make life difficult enough to force the pair to offer their resignations, which the President would then accept.” The White House called the book “completely false.”

CNN’s Pamela Brown and Kaitlan Collins contributed to this report.