Washington CNN  — 

Rod Rosenstein took aim at James Comey Monday, calling him a “partisan pundit” in a speech that included the most public retelling yet of the twists and turns of the Russia investigation by the man who oversaw it.

Speaking to a group of business and civic leaders in Baltimore, the former deputy attorney general – just days removed from a tumultuous tenure at the Department of Justice – recounted how he had prepared a memo in 2017 that supported President Donald Trump’s firing of Comey, then the FBI director, and defended his decision to appoint Robert Mueller as a special counsel in the wake of that firing.

Rosenstein also responded directly to a barb from the former FBI director, who said at a CNN town hall last week that Rosenstein’s character wasn’t strong and that his soul had been “eaten” by his time in the Trump administration.

“Now the former director seems to be acting as a partisan pundit, selling books and earning speaking fees while speculating about the strength of my character and the fate of my immortal soul. I kid you not. That is disappointing,” Rosenstein said.

In his speech before a crowd of nearly 1,000 people at the annual Greater Baltimore Committee dinner, Rosenstein acknowledged the unusual role he played in the drama of Trump’s Washington – as a Republican held up by the left for stewarding the Mueller probe.

“People spend a lot of time debating whose side I was on, based on who seemed to benefit the most from any individual decision,” Rosenstein said. “But trying to infer partisan affiliation from law enforcement decisions is what you might call a category error. It uses the wrong frame of reference.”

On Monday, with the frame of hindsight, Rosenstein told the audience why he disagreed with Comey’s handling of the Hillary Clinton email investigation, and how he put those concerns in a memo at Trump’s request.

Rosenstein also remembered how he disobeyed Trump’s request to include in that memo that Comey had told Trump that he wasn’t under investigation – “because number one, I had no personal knowledge of what the director said to the President, and number two, in any event, it was not relevant to my memo – and criticized the way the President carried out the firing.

“If I had been the decisionmaker, the removal would have been handled differently, with far more respect and far less drama,” Rosenstein said.

Asked to responded Monday to Rosenstein’s rebuke, Comey told CNN, “I wish him the best.”

Rosenstein didn’t quote Mueller in his evening remarks – like he did in a separate appearance at the University of Baltimore Law School earlier Monday – but he did borrow a line from Attorney General Bill Barr, aligning himself with Barr’s views on the appointment of special counsels.

Rosenstein stood by his decision to appoint Mueller and challenged critics to “explain what they would have done with the details as we knew at the time.”

He also said that in 2017 he viewed the opening of the investigation into Russian election interference as “justified,” but took credit for spurring an internal probe by the Justice Department’s Office of Inspector General that has been looking into surveillance techniques used by the FBI in the Russia probe’s early days.

“As acting attorney general though it was my responsibility to make sure that the Department of Justice would conduct an investigation that was independent both in fact and in perception, complete it expeditiously, hold perpetrators accountable if warranted by the facts and the law, and work with partner agencies to counter foreign agents and deter crime,” Rosenstein said. “We achieved those goals.”

Still, he expressed his displeasure with the process, noting “I disfavor special counsels.”

“I am glad that I only needed to appoint one in 25 months,” Rosenstein said.

CNN’s Sam Fossum contributed to this report.