WASHINGTON (CNN) -- The United States is rushing emergency aid to Pakistan -- an initial $5 million -- to help people uprooted by the fighting against extremists, according to the State Department.
Veteran diplomat Richard Holbrooke criticized Bush for committing a "pathetic" amount of money to Pakistan.
The United States urged the Pakistanis to launch the military action that in recent days drove hundreds of thousands of people from their villages.
Meanwhile, the Senate Foreign Relations Committee took a longer-term view Tuesday and discussed how the United States might triple aid to Pakistan over the next five years.
The State Department said Tuesday the U.S. Embassy in Pakistan and officials of the U.S. Agency for International Development are evaluating the needs of civilians fleeing the Swat valley and surrounding regions in northwest Pakistan.
"We are of course very concerned about the well-being of civilians who are fleeing the fighting in the area," State Department spokesman Ian Kelly said. "And we have personnel, USAID personnel, on the ground in Pakistan, not in the Swat valley, but in Pakistan, who are directing assistance to help them."
Kelly said the $5 million was just an initial payment. Watch Holbrooke being grilled by Congress »
On Capitol Hill, Senate Foreign Relations Committee Chairman, Sen. John Kerry, D-Massachusetts, praised the new military offensive by the Pakistan military.
"In recent days we have seen encouraging signs that Pakistan's army is finally taking the fight to enemy, but much remains to be done," Kerry said in his opening to a hearing on Pakistan and Afghanistan.
Kerry and the senior Republican on the committee, Sen. Richard Lugar of Indiana, have introduced legislation to triple non-military U.S. aid to Pakistan to $1.5 billion a year, for five years. A total of $7.5 billion in non-military aid would be sent Pakistan to foster economic growth and development. Another $7.5 billion would be sent for the following five years.
The $1.5 billion per year would triple U.S. aid levels, currently at $500 million per year. In the past, U.S. military aid has surpassed economic and other assistance. The legislation also would separate military from non-military aid, promising that economic aid "is no longer the poor cousin to military aid."
The Kerry-Lugar bill would take into account the fluid situation in Pakistan, leaving military aid to be determined on a year-by-year basis. But it requires that for aid to continue, Obama would have to certify that Pakistani forces are making progress in combating al Qaeda and the Taliban, and are not interfering with civilian rule. Both the Obama administration and the Pakistani government have opposed that condition.
The bill also calls for strict benchmarks to measure the effectiveness of the aid.
The Obama administration envoy on Pakistan and Afghanistan policy, long-time diplomat Richard Holbrooke, spoke to the Senate committee Tuesday about what the United States has at stake in Pakistan.
Holbrooke criticized the Bush administration for providing what he said was excessive support for former Pakistani President Pervez Musharraf, and for committing a "pathetic" amount of money to help stabilize Pakistan's critical tribal areas.
Holbrooke warned members of the Senate Foreign Relations Committee that military and political success in neighboring Afghanistan would not be possible if Pakistan's western areas remain a sanctuary for the Taliban and al Qaeda.
He said the goal of stabilizing Pakistan, a nuclear-armed state, was as "tough as anything" he encountered in his lengthy foreign service career.
"We are in Afghanistan and Pakistan because of 9/11, because al Qaeda and its allies are camped out in western Pakistan and have pledged and promised and predicted and threatened to do it again to us and other countries," he said.
"If it were not for this fact, we would not be sitting here today," Holbrooke said. "Pakistan would still be a huge issue [but] the reason we consider it one of, if not the highest, priority in this administration is because [extremists in western Pakistan] directly threaten us."
Kerry said the United States must mend what he called "a broken relationship" with the Pakistani people.
"Today an alarming number of Pakistanis actually view America as a greater threat than al Qaeda," Kerry said. "Until this changes, there's little chance of ending tolerance for terrorist groups, or persuading any Pakistani government to devote the political capital necessary to deny such groups sanctuary and covert material support."
But the aid package for Pakistan has its critics, both Democrats and Republicans. Sen. Bob Corker, R-Tennessee, said Congress should slow down consideration of new aid to Pakistan.
"We have not hashed out what's happening, and we are going to be engaged there for many, many, many years. Many men and women will lose their lives. We're doubling down. And we haven't debated this yet," Corker said.
Holbrooke said the proposed legislation already had attracted attention throughout the region.
"The only beneficiary of a delay in this bill is the enemies of our nation, the people who are trying to have the next 9/11," Holbrooke said.
And one Democratic senator, Robert Menendez of New Jersey, said he had reservations about sending more long-term aid to Pakistan.
Menendez said he remained troubled by the past -- that Pakistan received $12 billion in U.S. aid and spent much of it building up its military against India on its eastern border instead of fighting insurgents gaining strength on its western border.
Even Kerry, a co-sponsor of the bill, warned that more aid can only do so much.
"Even as we take bold steps, we should realize that our aid package to Pakistan is not a silver bullet," Kerry said.
Pakistan's government is also now wrestling with a growing refugee crisis sparked by the Pakistani military's assault on militants in Taliban-held areas.
The assault began after the Taliban seized territory in violation of an agreement, signed earlier this year by Pakistani President Asif Ali Zardari, that allowed the Taliban to implement Islamic law, or sharia, in the region where the Taliban is strongest, in exchange for an end to fighting.
Last week, Obama met with Zardari and Afghan President Hamid Karzai as part of a plan to more effectively coordinate the efforts of the governments of Afghanistan, Pakistan and the United States to combat Islamic extremists.
In rolling out his new strategy for Afghanistan and Pakistan in March, Obama called on Congress to pass a stronger regional aid bill, versions of which were introduced in previous sessions of Congress.
CNN's Charlie Keyes contributed to this report.