(CNN) -- Amid a growing debate on whether to arm Libyan rebels trying to topple Moammar Gadhafi, a U.S. intelligence source confirmed Wednesday that the Central Intelligence Agency has people on the ground in Libya to help the United States increase its "military and political understanding" of the situation.
"Yes, we are gathering intel firsthand and we are in contact with some opposition entities," the source told CNN.
The White House refused to comment on a Reuters report that President Barack Obama has signed a secret order authorizing covert U.S. government support for rebel troops.
"I will reiterate what the president said yesterday -- no decision has been made about providing arms to the opposition or to any group in Libya," White House press secretary Jay Carney said in a statement. "We're not ruling it out or ruling it in. We're assessing and reviewing options for all types of assistance that we could provide to the Libyan people, and have consulted directly with the opposition and our international partners about these matters."
The two ranking members of the House Permanent Select Committee on Intelligence -- Republican Chairman Mike Rogers of Michigan and Democrat C.A. "Dutch" Ruppersberger of Maryland -- both told CNN they oppose arming the Libyan rebels, as did Republican Sen. Lindsey Graham of South Carolina.
All spoke after classified briefings for Congress on the Libya situation by Secretary of State Hillary Clinton and Defense Secretary Robert Gates, who reiterated that no decision had been made on arming the rebels and made clear there were no plans to send U.S. ground troops to Libya, participants said.
Government forces have been pushing from Bin Jawad to Ras Lanuf, a critical eastern oil town that the opposition seized on Sunday. Gadhafi's military has also launched escalated strikes in the western town of Misrata.
The new government offensive came amid an international arms embargo and airstrike campaign designed to establish a no-fly zone and provide humanitarian relief for civilians threatened by the Libyan military.
British Prime Minister David Cameron told the House of Commons that the U.N. Security Council mandate for the Libya mission "allows all necessary measures to protect civilians and populated areas (and) this would not necessarily rule out provision of assistance to those protecting civilians in certain circumstances."
"As I've said before, we do not rule it out but we have not taken the decision to do so," Cameron said Wednesday.
Obama also has said that arming the rebels remained an option. One significant problem, however, is the need to train opposition forces on how to use advanced weaponry.
A second issue is the dispute over whether a decision to arm the rebels falls within the scope of the U.N. mandate. Clinton insisted Tuesday that it does, while French Foreign Minister Alain Juppe said it didn't, though he said France would discuss the issue with the other coalition partners.
Along with Rogers, Ruppersberger and Graham, other members of Congress from both parties also questioned the idea of arming the rebels due to apossible al Qaeda and Hezbollah presence within Libya's opposition movement.
U.S. intelligence has detected evidence of "flickers" of al Qaeda and Hezbollah elements among the rebels, according to Adm. James Stavridis, the U.S. NATO commander. Stavridis stressed Tuesday, however, that such a presence appears to be minimal.
A senior counterterrorism official, unnamed because he is not authorized to speak on the record, backed up Stavridis' assessment, downplaying the concern about al Qaeda among the Libyan opposition.
"It's hard to tell who all the leaders are in the opposition," the official said, but "the rebels do not appear to be adopting an al Qaeda bent or ideology in Libya."
Carney told reporters Wednesday that the values being expressed by the rebels are "antithetical to the purposes and ideals set forth by terrorist organizations."
Rogers earlier said in a statement that the United States should be careful before handing out weapons, referring to efforts to arm the Afghanistan mujahedeen against Soviet forces in the 1980s that eventually benefited the Taliban now being fought by U.S. forces.
"We don't have to look very far back in history to find examples of the unintended consequences of passing out advanced weapons to a group of fighters we didn't know as well as we should have," Rogers said in a statement. "We need to be very careful before rushing into a decision that could come back to haunt us."
CNN's Reza Sayah, Alan Silverleib, Tom Cohen, Arwa Damon, Nic Robertson, Amir Ahmed, Paula Newton and Yousuf Basil contributed to this report