Islamabad, Pakistan (CNN) -- Pakistan has temporarily stopped cooperating with American intelligence officials after the furor caused by the arrest and release of CIA contractor Raymond Davis, who fatally shot two Pakistanis earlier this year, a senior Pakistani security source told CNN.
While the two countries will continue to share vital intelligence about any imminent acts of terrorism, joint operations have been put on hold, the source said, and the ISI -- Pakistan's intelligence agency -- has asked the CIA to "give them visibility" over what the source says are an estimated 40 covert American intelligence operatives working in Pakistan.
A senior U.S. official disputed the report, saying "cooperation with the Pakistanis continues."
The two countries' are already on rocky ground diplomatically, as Washington tries to persuade Islamabad of the need to allow it to hunt al Qaeda targets autonomously inside Pakistani territory. This has largely been conducted through unmanned aerial drone strikes -- another sore spot in relations, after such strikes have killed dozens of civilians.
The strained relationship dominated what one official called a "frank discussion" -- diplomatic jargon for a talk in which disagreements are aired openly -- Monday in Washington between CIA Director Leon Panetta and the chief of Pakistan's Inter Services Intelligence (ISI), Lt. Gen. Ahmad Shuja Pasha.
CIA spokesman Preston Golson called the meeting between the two men "productive" and said the "CIA-ISI relationship remains on solid footing."
One U.S. official referred to Pakistan as "playing a public negotiations vs. private negotiations game," but the official said it is a "challenging situation," and the two nations are "working through differences."
"The bottom line is that joint cooperation is essential to the security of the two nations. The stakes are too high," said the official.
A U.S. official who is not authorized to speak on the record said, "Pakistan has asked for certain things and we're working it out," but the official would not elaborate.
The Pakistani security source said the presence of the CIA officers not disclosed to the Pakistani government "amounts to a lack of trust and respect that makes our job very difficult. After incidents like this we do have to take a pause -- is it that we are not being trusted? If we cannot be trusted to fight this war on terror on our own turf, then who can?"
"The Raymond Davis case brought to our attention that there are about 40-plus people in the country operating. We have asked the CIA to give us visibility -- to work with us and not behind our backs," said the source, who did not want to be identified discussing a sensitive issue. The ISI is aware of the identities of "most of" the 40 individuals, the source said, but they had not named them to the Americans.
Davis was arrested in late January after fatally shooting two men in a Lahore street on a busy afternoon. Davis said the two men were attacking him and he shot them in self-defense. Lahore Police Chief Aslam Tareen, however, said the case was "clear-cut murder."
The United States claimed Davis had diplomatic immunity in the case, although Pakistani officials did not confirm this claim. The contractor was charged with murder but was released from jail in March after a payment was made to the victims' families. Punjab province Law Minister Rana Sanaullah said that $2.34 million was paid to the legal heirs.
U.S. officials have said Davis was a security contractor assigned to protect CIA officers working in the sprawling city, although some Pakistani officials believe he was a full-fledged spy hunting militants in Lahore.
The incident fueled and seemed to partially confirm Pakistani conspiracy theories of shadowy American agents working covertly in the country under the guise of security companies. Davis was working for an American security contractor, Hyperion.
"Raymond Davis was engaged in low-level spy work and I can say he was also engaged in stupidity. There could be an estimated 40 to 60 people like Raymond Davis and we do not have visibility on what they are doing, but we may be trying to find out," the source said. "Having people like this here is a breach of trust and respect."
A U.S. official acknowledged Tuesday that Pakistan has requested that a number of American personnel leave the country, but said a New York Times report indicating 335 CIA and American special forces have been asked to depart is a "very inflated number."
The official also disputed the claim in the article that the CIA has withdrawn all of its contractors after the arrest of Davis in January.
"There has been no major withdrawal of contractors," said the official.
However, the senior Pakistani security source said that to his knowledge, Pakistani officials had not asked that any American officials leave.
"There have been incidents in which Americans have left on their own accord, and contractors may also have left," that source said.
Pakistani control over who enters the country has had an impact on the United States. The American official indicated that since the Davis incident, it has been "taking longer for American military and intelligence officials to gain entry" into Pakistan.
Asked about reports that Pakistan had asked for drone strikes to stop, the Pakistani security source declined to comment. The United States does not officially acknowledge the CIA's secret drone program that targets al Qaeda militants, but it is the only country operating in the region with the capability to carry out such strikes.
Those strikes have inflamed tensions between the two countries, especially after attacks in which civilians are killed. A drone strike March 17 hit a jirga meeting in North Waziristan, one of the seven districts of Pakistan's volatile tribal region bordering Afghanistan, killing 44 people -- mostly civilians, two Pakistani sources said.
Although there have been no drone strikes since then, the United States "is committed to pursuing aggressive counterterrorism operations," a U.S. official said.
The Pakistani source said the freezing of cooperation between the two countries was not something they wanted to sustain indefinitely. It also will not affect drone strikes, the source said, as they are carried out autonomously by the Americans.
"We want to go back to working with them. The stakes are too high. We have to work together with trust and respect. Treat us as allies," the source said.
CNN's Pam Benson contributed to this report