Washington (CNN) -- Time after time Wednesday, Secret Service Director Mark Sullivan insisted last month's prostitution scandal in Colombia was an aberration -- just poor choices by a dozen agents under the influence of alcohol.
"This is not a cultural issue, this is not a systemic issue," Sullivan told the Senate Homeland Security Committee, arguing that similar misconduct hasn't occurred on thousands of other overseas trips by Secret Service agents. "On this particular trip, we had individuals who made very bad decisions."
He maintained that view when Sen. Susan Collins, R-Maine, noted the agents used their own names when signing in prostitutes as overnight guests in their hotel rooms. Doesn't that show the agents lacked any fear of disclosure or discipline, Collins asked.
Sullivan responded that "between the alcohol and I don't know, the environment, these individuals did some dumb things," adding that he did not believe they acted because they thought their behavior would be tolerated.
In his own Secret Service career spanning decades, Sullivan noted, he never witnessed such behavior.
To Collins and other senators on the panel investigating the night of heavy drinking and consorting with prostitutes in Cartagena, Sullivan's personal dedication and loyalty to the agency he heads might be preventing him from accepting what seems obvious to them.
"I continue to believe that the problem is broader than you believe it to be," Collins said to Sullivan at the end of Wednesday's hearing. She later told reporters: "I think he has a difficult time coming to grips with the fact that he has a broader problem than just this one" episode.
The hearing was the first by a congressional committee on the April incident that embarrassed the nearly 150-year-old agency and raised security concerns.
In Colombia as part of the advance details before President Barack Obama arrival to attend the Summit of the Americas, a dozen agents hit the clubs of Cartagena for a night of drinking that ended with them bringing women back to their hotel rooms.
A morning-after dispute between one agent and a woman over payment led to a dozen Secret Service members being sent home and the resulting media coverage and investigations.
Collins and committee chairman Sen. Joe Lieberman, an independent from Connecticut who caucuses with the Democrats, called an announcement at the hearing that the Department of Homeland Security's acting inspector general will launch an independent investigation of the Colombia incident "a big change" and "significant."
Previously, the acting inspector general, Charles Edwards, was going to review the internal investigation by the Secret Service. In addition, at least two other congressional panels are looking into the scandal, and the military is investigating 12 members also allegedly involved.
In his opening statement Wednesday, Lieberman said it was hard to believe that on one night, the agents involved "suddenly and spontaneously did something they and other agents never had done before."
Lieberman and Collins provided new details of the incident, describing how the Secret Service agents went out on the town in groups of two or three to four different strip clubs and night clubs and returned to their hotel with foreign women who were signed in as overnight guests.
Collins said the circumstances suggested that "different rules apply on the road."
Sullivan, however, repeatedly argued the incident reflected misconduct by a few bad actors in an otherwise professional and exemplary agency. He apologized for what happened and declared himself "dumbfounded" when word of what happened first reached him the next day.
"When many of these people were interviewed, I don't think they could explain why they exhibited the behavior they did," he said, adding that "there is no excuse for that type of behavior from a conduct perspective and a national security perspective. That type of behavior was reckless."
Sullivan also revealed that two of the Secret Service agents who initially said they would resign over the scandal now are seeking to challenge their ouster. He said the agency will seek to revoke the security clearances of the two, which would effectively prevent them from continuing to work for the Secret Service.
Under respectful but persistent questioning by the panel members, Sullivan said the misconduct never compromised the security plan for the Obama trip.
The incident involved roughly 20 alleged prostitutes and has resulted in the dismissal of nine Secret Service members. Three others were cleared of serious misconduct. The military is investigating the alleged involvement of 12 service members, but has yet to announce any results.
Collins noted that "it is basic 'counterintelligence 101' that Secret Service personnel and others holding sensitive positions of trust in the U.S. government should avoid any situation that could provide a foreign intelligence or security service or criminal gangs with the means of exerting coercion or blackmail. Yet two of the primary means of entrapment -- sexual lures and alcohol -- were both present here in abundance."
While preliminary findings are that no weapons or classified material was in the agents' rooms, those involved "could easily have been drugged or kidnapped, or had their liaisons with these foreign national used to blackmail them," she said. "They willingly made themselves potential targets not only for intelligence or security services, but also for groups like" drug cartels.
Sullivan said in his opening statement that "at the time the misconduct occurred, none of the individuals involved in misconduct had received any specific protective information, sensitive security documents, firearms, radios or other security related equipment in their hotel rooms."
In addition, Sullivan noted that allegations of similar misconduct by Secret Service personnel in El Salvador in March 2011 appeared to be untrue.
"After several days in San Salvador and conducting 28 interviews with hotel managers and employees, individuals from the U.S. Department of State, other government agencies and contract employees assigned to assist the Secret Service ... no evidence was found to substantiate the allegations," he said.
The alleged incident was reported by CNN affiliate KIRO in Seattle. The station cited an unnamed U.S. government contractor who worked extensively with the Secret Service advance team in San Salvador before an Obama visit.
The source said he was with about a dozen Secret Service agents and a few U.S. military specialists at a strip club in the city a few days before Obama arrived. The men drank heavily at the club, the source said, and most of them paid extra for access to a VIP section where they were provided sexual favors in return for cash.
Sullivan said Wednesday that the owner of the business was interviewed and provided a sworn written statement saying "he had no knowledge or any other information that any Secret Service personnel had been to his business or information about misconduct by Secret Service personnel."
He outlined previously reported steps being taken to prevent future instances, including enhanced supervision on such assignments and a reinforced code of conduct. Sullivan also encouraged agents to blow the whistle on any misconduct by peers that they encounter.
"I don't think that our men and women need these guidelines because we have men and women of character, we have men and women of integrity," he said.
Collins remained unpersuaded, telling reporters after the hearing she was disturbed by Sullivan's repeated insistence that the Colombia scandal was an isolated incident.
Meanwhile, three Drug Enforcement Administration agents are under investigation for allegedly soliciting sex in Cartagena. One of them had a long-term relationship with a prostitute, two government sources familiar with the investigation said.
The DEA agents were not involved in security for the president's trip, the government sources said.
The DEA agent's relationship with the prostitute came to light after a Secret Service agent voluntarily reported to his superiors that he was at a party at the agents' Cartagena apartment on April 13 where the three agents and several women were present, the government sources said.
The Secret Service agent is the 13th employee to be caught up in the Colombia scandal. He is on administrative leave, according to several sources, but is not expected to lose his job because he came forward on his own to report the incident.
CNN's Ted Barrett, Kate Bolduan, Ashley Hayes and Dana Bash contributed to this report.