Washington (CNN) -- Facing a trio of possible scandals that threaten to overwhelm his second term agenda, President Barack Obama on Thursday rejected calls for a special counsel to investigate Internal Revenue Service targeting of conservative groups and declared he would fix any problems in government.
"My concern is making sure that if there is a problem in the government that we fix it," Obama told a Rose Garden news conference with visiting Turkish Prime Minister Recep Tayyip Erdogan. "That's my responsibility, and that's what we're going to do."
A light rain suddenly turned heavier when Obama began speaking, symbolizing his political woes that prompted him to make a trio of moves this week intended to regain the momentum or at least halt the damage.
The steps Wednesday represented a counter-attack by the suddenly beleaguered president against Republican attacks that his administration defied accountability in the IRS targeting, as well as secret subpoena of journalist phone records and erroneous talking points in the immediate aftermath of last year's Benghazi terrorist attack.
At a hastily arranged televised statement Wednesday night, Obama announced the acting IRS commissioner was forced to resign over the extra scrutiny given conservative groups seeking tax exempt status.
Earlier, the White House announced its support for strengthening protections of journalists and confidential sources even as Attorney General Eric Holder evaded questions at a congressional hearing about how his Justice Department obtained phone records of The Associated Press from 2012 as part of an investigation of classified leaks.
In addition, the White House released more than 100 pages of e-mails sought by GOP critics about the talking points on the September attack in Benghazi, Libya, that killed the U.S. ambassador and three other Americans.
Despite those steps, U.S. journalists continued raising those issues at Thursday's news conference with Erdogan.
When asked if a special prosecutor should be appointed in the IRS case documented by the agency's inspector general, Obama responded that "I think that it's going to be sufficient for us to be working with Congress."
Between congressional committees looking into it, including the first hearing on Friday by a House panel, a criminal investigation launched by Attorney General Eric Holder and the inspector general's report and likely further investigation, "I think we're going to be able to figure out exactly what happened, who was involved, what went wrong, and we're going to be able to implement steps to fix it."
He repeated his statement from Wednesday night that he was outraged by what happened, adding that "we're all vulnerable" if such a powerful agency as the IRS is acting on political bias.
"It doesn't matter whether you're a Democrat or a Republican, you should be equally outraged at even the prospect that the IRS might not be acting with the kind of complete neutrality that we expect," Obama said, repeating again that "I think we're going to be able to fix it."
Later, the White House announced that Danny Werfel, the controller of the Office of Management and Budget, was Obama's pick to be the new acting IRS commissioner through September, the end of the current fiscal year.
In a White House statement, Obama noted the 42-year-old Werfel has served in Democratic and Republican administrations.
"As we work to get to the bottom of what happened and restore confidence in the IRS, Danny has the experience and management ability necessary to lead the agency at this important time," the president said.
Meanwhile, another top IRS official announced that he will also be leaving the agency. Joseph Grant, commissioner of Tax Exempt and Government Entities Division, will retire on June 3, according to an internal IRS memo provided to CNN.
Mounting questions and demands by Republican leaders, and some Democrats, over the controversies have threatened to overwhelm Obama's agenda less than four months into his second term.
A clearly coordinated campaign of GOP attacks, including accusations of criminal behavior in the IRS targeting case, sought to score political points against both the Obama administration and former Secretary of State Hillary Clinton, considered the leading potential Democratic presidential contender in 2016.
"These three events that have gotten so much attention over the last few days -- IRS, AP, Benghazi -- tend to confirm a lot of our worst fears about our government," GOP Sen. Mike Lee of Utah said Thursday. "They tend to tell us what we don't want to believe, but that sometimes might be true; that your government's targeting you, that your government's spying on you, and that your government is lying to you."
House Speaker John Boehner told reporters that the actions of the Obama administration threatened to break what he called "bonds of trust" between the American people and the government.
"Nothing dissolves the bonds between the people and their government like the arrogance of power here in Washington and that's what the American people are seeing today from the Obama administration; remarkable arrogance," the Ohio Republican said. "Those bonds, once broken, are very hard to repair."
However, House Democratic leader Nancy Pelosi said Republicans were looking for any possible way to undermine the presidency of what she called a "great" and "visionary" leader.
"This is as much about enforcing their anti-government ideology, of not a public role in the creation of jobs, as well as undermining the president of the United States," the California Democrat told reporters. "... Because he's strong and because he's effective, they make him be object of their political action. We know how that works."
At the same time, Pelosi acknowledged that some of the issues were legitimate, such as the IRS targeting. Earlier this week, Senate Majority Leader Harry Reid of Nevada strongly criticized the secret subpoena of AP phone records.
After days of such bludgeoning, Obama responded by announcing Wednesday night that acting IRS Commissioner Steven Miller had been forced to resign.
Miller was aware since a year ago, when he was deputy commissioner, that IRS employees were stonewalling requests from conservative groups seeking tax-exempt status, according to the agency.
However, Miller didn't tell Congress about it when he testified before an oversight committee in July -- despite being questioned on the issue. He was named acting commissioner in November.
Miller, who will leave the agency next month, is expected to testify Friday at the first congressional hearing on the controversy, which will be held by the GOP-led House Ways and Means Committee.
Miller assumed the role of acting head of the IRS last November. The previous commissioner, Douglas Shulman, was appointed by President George W. Bush.
Shulman testified at a March 2012 congressional hearing that his agency did not target conservative groups for political reasons.
He will testify before the House Oversight Committee next week, according to a House GOP aide. Shulman, who is no longer in the government, agreed to come voluntarily, the source said.
Holder, who has ordered a criminal investigation into the situation, told legislators that investigators will look at the conduct of IRS offices nationwide.
"The facts will take us where ever they take us," he said, adding that possible charges could include lying to Congress about what happened.
According to the report by the IRS inspector general, the agency developed and followed a faulty policy to determine whether the applicants were engaged in political activities, which would disqualify the groups from receiving tax-exempt status.
The controversial move began in early 2010 and continued for more than 18 months, the report said, declaring that "the IRS used inappropriate criteria that identified for review Tea Party and other organizations applying for tax-exempt status based upon their names or policy positions instead of indications of potential political campaign intervention."
IRS officials, according to the report, did not consult anyone beyond the agency about the development of the additional screening criteria. They believed that the criteria they came up with were a screening shortcut meant to help with the influx of applications, the report said.
The IRS scrutiny began after the 2010 Supreme Court ruling in the Citizens United case that greatly expanded the ability of corporations, unions and other organizations to participate in election spending, though not through direct contributions to candidates or parties.
Following the ruling, the number of politically oriented groups seeking tax exempt status as social welfare organizations under section 501 (c) (4) of the federal tax code increased greatly at a time when the federal government, including the IRS, was dealing with austerity measures that reduced or stagnated personnel and resources.
However, the IRS watchdog found that the criteria used to flag potential political applications resulted in substantial delays and the request of unnecessary information from the groups.
The investigation by the Treasury inspector general for tax administration was initiated after congressional complaints began to surface in the media in 2012 that the IRS was targeting conservative groups and holding up applications.
In a written response included in the report, the IRS commissioner of the Tax Exempt and Government Entities Division said there was no criminal behavior behind the actions of the agents, but rather inefficient management.
"We believe the front-line career employees that made the decisions acted out of a desire for efficiency and not out of any political and partisan viewpoint," the commissioner wrote.
Among the recommendations made by the Treasury inspector general: The IRS must better document reasons why applications are chosen for review, develop a process to track requests for assistance, develop and provide training to employees before each election cycle and immediately resolve outstanding cases.
The report also called on Treasury to develop guidelines to explain social welfare activity -- the primary factor in obtaining tax-exempt status.
In the case of subpoenaed phone records, Holder said Wednesday that his decision to remove himself from the Justice Department investigation into a leak that led it to the surreptitious move left him unable to respond to questions about it.
"I don't know what happened there with the intersection between the AP and the Justice Department," Holder told the House Judiciary Committee. "I was recused from the case."
Asked about the AP case Thursday, Obama said he was unable to discuss specifics because of an ongoing investigation.
He added that he doesn't apologize for cracking down on leaks of classified information that can jeopardize U.S. troops and intelligence officers, but emphasized support for consideration of a new "shield law" for the media. The president also expressed full confidence in Holder.
Democratic Sen. Charles Schumer of New York previously introduced the proposed shield law that passed the Senate Judiciary Committee in 2009 but never advanced.
Federal shield legislation would protect journalists from revealing their sources and beef up protections for reporters and their sources caught up in such probes.
According to AP, the investigation into its records appears related to a story revealing that the CIA had thwarted an al Qaeda plot to blow up a U.S.-bound jetliner in May 2012, around the anniversary of the killing of Osama bin Laden.
The seized phone records covered a two-month period beginning in May 2012 and included more than 20 AP lines, including personal phones and AP phone numbers in New York; Hartford, Connecticut; and Washington.
"A shield law would keep lazy prosecutors from going after reporters' notes and phone records and compel them to actually conduct investigations that do not step all over the First Amendment," Teri Hayt, the First Amendment chairwoman of the Associated Press Media Editors, said in a statement issued before the White House announcement.
In the past decade, Congress has come close to passing a federal shield law. But support for the measure shrank during the WikiLeaks scandal in which thousands of classified U.S. diplomatic cables were released.
Obama has pledged transparency and openness since taking office, but legal analysts contend his administration's actions suggest otherwise.
For example, the Obama administration has used the Espionage Act, which was passed in 1917, to target suspected leakers in six cases, twice the number undertaken by all previous administrations combined.
In an example of Washington politics, many Republicans now criticizing the crackdown on AP have been the loudest advocates for stronger federal action against classified leaks.
The biggest pushback by Obama and the White House has been against a relentless Republican effort to vilify the administration's response to the Benghazi attack last September 11, particularly talking points used by U.N. Ambassador Susan Rice in the days that followed that erroneously contended the assault resulted from a demonstration over an anti-Islam film.
Republicans, who accuse the administration of failing bolster security prior to the attack and botching the response to it, contend the talking points amounted to misleading the public for political gain less than two months before the November election by removing references to a planned terrorist assault.
On Wednesday, the White House released more than 100 pages of e-mails in a bid to quell critics who say Obama and his aides played politics with national security.
The e-mails detail the complex back and forth between the CIA, State Department, and the White House in developing the unclassified talking points used by Rice.
Obama has called Republican concentration on the talking points a political "side show," and senior administration officials contend the e-mails demonstrate the process of developing the talking points was not focused on politics but rather on events.
The White House and its allies in Congress have made the case that any confusion and conflicting information in the early hours and days after the attacks stemmed from the "fog of war" -- not any deliberate effort to mislead the American people about the source of the attacks.
Analysis: CIA role in Benghazi underreported For instance, some of the e-mails expressed caution about what should be said publicly during an FBI investigation, while others focused on the strength of intelligence at the time.
The e-mails indicate the CIA was likely the lead organization in developing the talking points with the State Department recommending significant changes.
Rep. Darrell Issa, the chairman of the House Oversight Committee which is investigating the matter, told CNN's "Situtation Room" that his staff wants to digest the e-mails. He stressed that they were a selected set of documents as released and the committee is still seeking a range of other information.
However, a letter to Issa by the co-chairmen of an independent review of the Benghazi attack expressed irritation over his portrayal of the panel's work and requested a public hearing at which they can testify.
"The public deserves to hear your questions and our answers," wrote former U.S. ambassador, Thomas Pickering and retired chairman of the Joint Chiefs of Staff, Adm. Mike Mullen, who led the Accountability Review Board convened by Clinton to investigate the attack.
Eight months after their report cited "systemic failures and leadership and management deficiencies" at the State Department," Issa continues to be a leading critic of the accountability board, calling its review "a failure" and asking for further investigations into the Obama administration's response during the attack and its aftermath.
CNN's Chelsea J. Carter. Drew Griffin, David Fitzpatrick, Dana Bash, Greg Botelho, Tom Cohen, Kevin Liptak, Kevin Bohn, Jake Tapper, Tom Watkins and Ted Barrett contributed to this report.