(CNN) -- A largely minority day care center Monday refused an offer to return to a swim club in suburban Philadelphia, Pennsylvania, amid allegations of racism.
A small group of protesters gathers in front of the club last week.
Swimming privileges for about 65 children from Creative Steps were revoked after one visit to the club, on June 29. Some black and Hispanic children said white club members made racist comments to them, asking why black children were there and raising concerns that "they might steal from us."
"Parents, do you want your children to return to the Valley Swim Club?" Alethea Wright of Creative Steps asked parents who attended a news briefing with her Monday. When none of them raised their hands, Wright said, "As you can see, the vote is unanimous."
The swim club, in Huntingdon Valley, Pennsylvania, extended the invitation on Sunday to return.
The day care center had originally contracted to use the pool, but the club canceled the agreement and returned the day care center's $1,950 check without explanation. Watch how the controversy started »
Swim club director John Duesler said last week that safety and crowding -- not racism -- prompted the cancellation. The club canceled contracts with two other day care centers because of safety and crowding, he said.
Wright has disputed that the pool was overcrowded.
Attorneys for the day care center said they will sue the swim club in a few days.
"The children's best interests are not being served," lawyer Carolyn Nichols said. "Simple lip service does not amount to change."
The day care center also wants the resignation of all swim club board members and the removal of any club members who made the racist comments, Nichols said.
"Our goal is to ensure that this type of behavior never happens again," she said.
Wright has said the children in her care were "emotionally damaged" by the incident.
"These children are scarred. How can I take those children back there?" she said.
The club has been subpoenaed by the state Human Relations Commission, which has begun a fact-finding investigation, said Bernice Duesler, the club director's wife.
CNN's Janet M. DiGiacomo, Susan Candiotti and Ross Levitt contributed to this report.