WASHINGTON (CNN) -- Former Defense Policy Board Chairman Richard Perle sharply criticized former CIA Director George Tenet Friday, calling him "sloppy by nature," and taking issue with many of the assertions in his new book.

In an interview with CNN's Wolf Blitzer, Perle directly refuted a conversation Tenet describes in his new book, "At the Center of the Storm," during which the former CIA Director claims Perle told him the day after the September 11 attacks that, "Iraq has to pay the price for what happened yesterday. They bear responsibility."

"[I] never said the things that he attributes to me," Perle said. "He's just got it completely wrong. I never believed that Saddam Hussein was responsible for 9/11, and by the time I returned to the U.S., it was pretty clear that it was bin Laden, indeed, bin Laden was claiming credit."

Perle added that Tenet is attempting to "shift responsibility" for the "failure not only to anticipate 9/11, specifically, but the failure to understand the extent of the terrorist threat."

Perle, one of the chief architects of the Iraq war, maintained that invading the country was not a mistake, though acknowledged the war's aftermath was mismanaged.

"I'm sorry that after removing Saddam we did not hand things back to the Iraqis," he said. "I'm sorry that we embarked on an occupation that became the basis for an insurgency against us. I think the right thing to have done -- and I said it at the time -- was to hand things to the Iraqis as soon as Saddam was removed."