Washington (CNN) -- A member of the U.S. military assigned to the White House Communications Agency is under investigation in connection with alleged misconduct in Colombia, bringing to 12 the total number of military personnel being reviewed, officials said Monday.
One Defense Department official said the military member admitted to his leadership that he was involved in misconduct "of some kind" while in Colombia for the recent Summit of the Americas attended by President Barack Obama.
The agency is a non-White House office that provides the president with secure communications while he travels. It is staffed by members of the military who report through the Defense Information Systems Agency.
A total of 24 people -- 12 Secret Service members and 12 U.S. military members -- are under investigation in the alleged prostitution scandal that occurred before Obama arrived in Cartagena on April 13. The controversy has embarrassed the nearly 150-year-old Secret Service, which protects the president and other top officials and investigates criminal activity, and raised questions about a possible security breach immediately preceding Obama's visit.
Six Secret Service members have left their jobs in the wake of the incident in Cartagena, and one employee "has been cleared of serious misconduct but will face administrative action," the Secret Service said. Five other Secret Service employees are on administrative leave and have had their security clearances temporarily revoked.
In addition, the U.S. military is investigating 12 of its own service members for alleged misconduct.
Defense Secretary Leon Panetta said Monday that security clearances have been suspended for all U.S. military personnel involved in the incident in Cartagena, and he repeated his vow that anyone found to have violated regulations would be punished.
"We expect our people wherever they are, whether they're in Colombia or any other country or in the United States, to behave at the highest standards of conduct," Panetta said during a visit to the Colombian capital Bogota. "That's what we expect. And so we will wait for the investigation that is currently taking place."
At the White House, Press Secretary Jay Carney said Monday that there was no evidence of any wrongdoing by White House staff or advance team members. When asked about the possibility of White House Communications Agency staff members being involved, Carney pointed out that they are members of the military.
"To make clear, the Secret Service is investigating specific allegations of misconduct by members of the Secret Service. The Defense Department is investigating specific allegations of misconduct by members of the military," he said. "There have been no specific, credible allegations of misconduct by anyone on the White House advance team or the White House staff.
"Nevertheless, out of due diligence, the White House counsel's office has conducted a review of the White House advance team, and in concluding that review, came to the conclusion that there's no indication that any member of the White House advance team engaged in any improper conduct or behavior."
That drew the attention of Sen. Charles Grassley, the ranking Republican on the Senate Judiciary Committee. In a letter to the counsel's office, Grassley noted that he had asked Secret Service Director Mark Sullivan whether any White House advance staff had been involved and hadn't had an answer back by Monday. Grassley asked for answers to 14 questions, including details of how the White House review was conducted, whether any White House staffers had "overnight guests" and whether any additional room charges were incurred.
Also Monday, a source familiar with the investigation said that one of the Secret Service agents linked to the prostitution scandal brought a woman back to the Hilton Cartagena, the same hotel where Obama later stayed, five days before the president's arrival.
The source was not certain whether money exchanged hands or whether the Secret Service member simply brought a woman he met -- a foreign national -- to the Hilton.
According to the source, the incident appeared to be separate from the one a few nights later that caused 11 other Secret Service members to be sent home for alleged heavy drinking and consorting with prostitutes.
Based on the investigation, there is no evidence that the women realized the Secret Service personnel were with the agency, the source said. Investigators have interviewed the prostitutes and other women involved.
"It doesn't appear these women knew who the heck they were," the source said.
Meanwhile, House Homeland Security Chairman Peter King said he expects more Secret Service members to be forced out soon.
"I am very certain that within the next day or so, you're going to see a number of people leaving the Secret Service," said King, R-New York.
King, as well as the other source familiar with the investigation, said polygraph tests administered to the Secret Service members helped get information to force out six of them.
According to King, the 11 Secret Service members originally under investigation were all given drug tests, which came back negative.
From the start, Sullivan told legislators how outraged he was about the scandal, promising a broad and comprehensive investigation.
Most legislators seemed comfortable with the way the Secret Service has reacted to the scandal. However, both the House and Senate Homeland Security Committee chairmen are now conducting their own investigations and will probably hold hearings in coming weeks.
The 11 Secret Service employees accused of consorting with prostitutes arrived earlier the same day as a part of the "jump team" that flies in on military transport planes with vehicles in the president's motorcade.
According to sources, the alleged prostitutes -- the youngest of whom were in their early 20s -- signed in at the Cartagena hotel where Secret Service members apparently stayed, flashing their local ID cards. One of these women allegedly was later involved in a dispute about how much she was to be paid for the night, which brought the entire incident to light and sparked controversy in the United States and Colombia.
That woman has been identified as Dania Suarez, whose neighbors described her as a 24-year-old single mother who studies English. Suarez hasn't been seen in the neighborhood since the controversy erupted, though a visitor took suitcases from the house recently, they said.
While soliciting prostitution is in most cases legal for adults in Colombia, military law bars service members from patronizing prostitutes, engaging in conduct unbecoming an officer or, for enlisted personnel, conduct "prejudicial to good order and discipline." It is also considered a breach of the Secret Service's conduct code, government sources said.
CNN's Drew Griffin and Tom Cohen contributed to this report.