CNN  — 

Defense Secretary Mark Esper and Chairman of the Joint Chiefs of Staff Gen. Mark Milley have refused to testify next week before the House Armed Services Committee regarding the military’s role responding to protests triggered by the death of George Floyd, a Democratic congressional aide confirmed to CNN Friday.

An informal briefing for lawmakers by Army Secretary Ryan McCarthy was also canceled for Friday, the aide said.

Rep. Adam Smith, the Democratic chairman of the House panel, called on Esper and Milley to appear in person before the committee “to explain this domestic engagement to the American people” this week, emphasizing that “the fate of our democracy depends on how we navigate this time of crisis.”

“At a time when our nation is hurting the President continues to forgo any effort to calm tensions, as we had all hoped he would. Instead he used force to remove peaceful protesters to stage a photo opportunity. Instead of taking a moment of silence in front of the historic St Johns’ Episcopal Church to pray for the lives lost he gathered his cronies to take a picture. Instead of doing the right thing, he once again did the absolute wrong thing,” Smith said in a statement Wednesday.

“I have serious concerns about using military forces to respond to protestors. The role of the U.S. military in domestic U.S. law enforcement is limited by law. It must not be used in violation of those limits and I see little evidence that President Trump understands this fundamental premise,” he added. “I remain gravely concerned about the President Trump’s seemingly autocratic rule and how it affects the judgment of our military leadership.”

The Pentagon said later on Saturday that Esper and Milley had not refused to testify.

“Secretary of Defense Esper and Chairman of the Joint Chiefs of Staff Gen. Milley have not ‘refused’ to testify before the HASC as some are reporting,” Chief Pentagon spokesman Jonathan Hoffman said in a statement.

“The DOD legislative affairs team remains in discussion with the HASC on this request. In the meantime, DOD has committed to provide Army Secretary McCarthy, Army Chief of Staff Gen. McConville, and DC National Guard Commanding General Maj. Gen. Walker to brief the committee next week on the presence of the National Guard in Washington, D.C. this past week,” Hoffman said.

Politico was first to report Esper and Milley’s refusal to appear before the committee next week.

Smith also wrote to Esper and Milley on Wednesday requesting written answers to several questions about the Pentagons potential plans to use active duty forces for law enforcement purposes. The deadline for those responses is June 9.

The same House aide told CNN Friday that the committee intends to continue to request that both Esper and Milley testify before lawmakers in person but, as of now, have received no indication from the Pentagon that they are working to schedule a hearing for a later date.

Asked Tuesday if he had confidence in Esper, Smith, said he “can’t really answer that question at this point” until he speaks with the defense secretary, which he had not done yet despite requesting a conversation with him.

Esper on shaky ground

CNN reported Wednesday that Esper is on shaky ground with the White House after saying he does not support using active duty troops to quell the large-scale protests across the US and those forces should only be used in a law enforcement role as a last resort.

Speaking from the Pentagon briefing room podium, Esper noted that “we are not in one of those situations now,” distancing himself from President Donald Trump’s recent threat to deploy the military to enforce order.

“The option to use active duty forces in a law enforcement role should only be used as a matter of last resort, and only in the most urgent and dire of situations. We are not in one of those situations now. I do not support invoking the Insurrection Act,” he told reporters. Esper also distanced himself from a maligned photo-op outside St. John’s Church.

Wednesday’s press briefing by Esper went over poorly at the White House, where his standing was already viewed to be tenuous, multiple people familiar with the matter said.

Esper’s comments Wednesday came after defense officials told CNN this week that there was deep and growing discomfort among some in the Pentagon even before Trump announced Monday that he is ready to deploy active duty forces if local leaders fail to ramp up enforcement efforts.

As gas wafted through the air in Lafayette Park across from the White House, Trump announced from the Rose Garden that if state or city leaders refuse “to take the actions that are necessary to defend the life and property of their residents,” he will invoke the Insurrection Act, an 1807 law that allows a president to deploy the US military to suppress civil disorder.

On Friday, Esper ordered that active duty troops who were moved to the Washington DC area should return to their home base of Fort Drum in New York, Army Secretary Ryan McCarthy said.

On Thursday, Esper ordered about 700 of the 1,600 active duty troops who were brought to the Washington region on Monday back to Fort Bragg in North Carolina. Friday’s order to the New York-based troops removes almost all the active duty presence brought in to bolster law enforcement. The troops had been positioned around Washington on “heightened alert status” to allow for their quick deployment amid nationwide protests.

McCarthy said that a small active duty component of the “Old Guard” based in Arlington, Virginia, will remain on stand-by for assistance, but that he hopes they will be taken off alert status this weekend. That will be contingent on protests in the capital continuing to be peaceful, McCarthy added.

This story has been updated on Saturday with a statement from the Pentagon.

CNN’s Kaitlan Collins, Kevin Liptak, Vivian Salama and Jim Acosta contributed reporting