WASHINGTON (CNN) -- President Bush said Tuesday he has "confidence" in Attorney General Alberto Gonzales, though he acknowledged "mistakes were made" and is "not happy about them."

"I talked to him this morning," Bush said at a press conference in Mexico with Mexican President Felipe Calderon. "And we talked about his need to go up to Capitol Hill and make it very clear to members in both political parties why the Justice Department made the decisions it made, make it very clear about the facts, and, he's right, mistakes were made. And I'm frankly not happy about them, because there is a lot of confusion over what really has been a customary practice by the president. U.S. Attorneys and others serve at the pleasure of the president."

"Past administrations have removed U.S. attorneys," he added. "It's their right to do so. The Justice Department recommended a list of U.S. attorneys, I believed the reasons why were entirely appropriate."

-- CNN Ticker Producer Alexander Mooney