(CNN) -- USAID Administrator Rajiv Shah has addressed reports of threats to international aid workers helping with flood relief efforts in Pakistan, calling it "deeply saddening that others would choose to use these environments to propagate themselves or to threaten international aid workers."
Shah, who leads the U.S. agency that provides economic and humanitarian assistance worldwide, said Friday that he himself had been forced to leave one food distribution site early due to "suspicious individuals" in the area.
"I had hoped to spend more time talking to people in line, but within a few minutes of being there, our Diplomatic Security detail informed me that there were some suspicious individuals in the area and we needed to leave," he said.
Shah's comments came a day after the U.S. State Department announced it had obtained information about threats against foreign aid workers who are assisting with relief efforts in the flood-ravaged country, where monsoon rains have killed 1,639 people.
"We are concerned that extremist elements within Pakistan may decide to attack foreigners who are in Pakistan helping the people of Pakistan, or may choose at this time to attack government institutions in Pakistan that are responding on behalf of the Pakistani people," State Department spokesman P.J. Crowley told reporters Thursday.
Shah expressed disappointment at the threats, saying "I can't tell you how ... inappropriate that feels when you've been there and you've talked to people who have literally lost everything and are just trying to survive."
At least 17 million Pakistanis have been affected by the monsoon floods that began a month ago, and thousands remain at risk.
Crowley said the United States is doing everything it can to ensure that the U.S. and Pakistani disaster response can continue despite the threat.
"The fact that suspicious individuals had been in this site -- in part because they knew I was coming, and the day before, our security folks who had gone out there, of course, hadn't seen any signs of them -- is an indication these are open sites," Shah said. "People can get there. The whole point is to have people who are in urgent need come and get food. So it's just -- it's deeply saddening that others would choose to use these environments to propagate themselves or to threaten international aid workers."