Washington (CNN) -- The man overseeing how billions of dollars are spent in Afghanistan is the wrong man for the job, a U.S. senator charged Thursday.
Sen. Claire McCaskill, D-Missouri, criticized former Marine Corps Maj. Gen. Arnold Fields, the special inspector general for Afghanistan reconstruction, as he appeared before her and other senators at a subcommittee hearing to examine the performance of his office.
That office oversees $56 billion in Afghanistan spent on projects including schools, roads and water plants. The U.S. plans to spend $16 billion more next year.
"I don't think you are the right person for this job," McCaskill told Fields after more than an hour of questioning.
McCaskill had the support of at least one person in the visitors' gallery.
"Fire that man, fire that man right away," called out an unidentified person who stalked out of the hearing.
Biden on Afghanistan: 'Making progress'
The ranking Republican on the Senate subcommittee on contracting oversight, Sen. Scott Brown, R-Massachusetts, told Fields he was worried that the Taliban was siphoning money from reconstruction projects.
"I just want you to follow the money," Brown said. "Zero in on how the Taliban, how they are getting our money."
Fields defended the efforts of himself and his staff.
"Our work is indeed making a difference," Fields said.
In the ornate Senate hearing room, with marble columns, gold-and-white carved ceiling and three large glass chandeliers, Fields spoke of his poor upbringing in South Carolina, and his 34 years in the Marine Corps. He moved into his position as special inspector general in 2008.
"I came up hard... In poverty myself," Fields said about how he felt a connection with impoverished people in Afghanistan. "I know what it is to live in poverty."
"I will do my best to live up to your expectations," Fields told the senators.
Fields said he had worked diligently to build up the agency since it was created in 2008 and only fully funded in 2009. He told the subcommittee that over the past 18 months, his office had issued 34 audit reports and made more than 100 recommendations.
"We have 89 investigations going on," Fields noted.
Fields defended himself from accusations that he was slow to hire qualified staff, pointing out the challenge of persuading people to take short-term employment and put themselves in harm's way in a war zone.
McCaskill has been vocal in her displeasure with Fields' office. She and three Republican senators wrote to President Barack Obama in September urging that he fire Fields.
"It has been clear for several months that SIGAR's mission is not being served effectively,: the letter to Obama said, referring to Fields' office by its acronym. "SIGAR would be better served with new leadership."
It was the third letter the senators had written to the president about their displeasure about the Afghan overseers.
"SIGAR didn't have a plan," McCaskill said in her opening statement. "They didn't have the right investigative team in place."
She said that while Fields was one of the nation's heroes he had, in her words, "fallen short of the mark."
"I do think he has to go" McCaskill told CNN after the hearing.
"I'm going to continue to put the most intense pressure that I know how on the White House. Frankly I think he should have been let go before today...In many ways I thought this hearing was cruel to him,"McCaskill said.
"Once people began questioning his leadership and ability to perfom this job I think human nature is you want to prove to people they are wrong and I think he's dug in and I think it is unfortunate."
A spokeswoman for Fields' office told CNN that Fields was disappointed by the hearing and would have liked more time to explain himself."
"He thinks he is the right man for the job," said Susan Phalen. "There is still $56 billion that needs oversight, and tomorrow we're going to come back into the office and continue overseeing that taxpayer money."
