(CNN) -- Los Angeles Clippers co-owner Shelly Sterling is "moving quickly" to sell the embattled NBA franchise, with a source familiar with the situation saying she and her representatives would like to make a deal this week.
The recent activity includes talks with several high-profile potential suitors for the team, and NBA Commissioner Adam Silver is "very much involved" in Shelly Sterling's dealings with potential buyers, according to the source.
But as bidders make their pitches for his team, Donald Sterling -- Shelly's estranged husband and the team's other owner -- accused the NBA of violating his constitutional rights in trying to end his ownership of the franchise based on what he says was an illegally recorded conversation during a lovers' quarrel, according to a document provided by Donald Sterling's attorney.
The document, Sterling's 26-page response to charges of violating the league's bylaws, adhered to the NBA's Tuesday deadline for the billionaire businessman's reply.
The NBA has said the next step will be a special Board of Governors meeting on June 3, when the league's 30 ownership groups -- someone else will represent the Clippers -- will meet to hear evidence on Donald Sterling's behavior, then vote on terminating the Sterlings' ownership rights.
Sterling has been prejudged and won't get a fair hearing, the document says, citing comments from 10 other teams.
About one month ago, Silver banned Sterling from NBA facilities and functions and ordered him to pay a record fine after racist remarks by the Clippers' owner to former companion V. Stiviano surfaced online.
A source close to the situation said roughly two weeks ago that Donald Sterling -- who repeatedly denied he's a racist in a CNN interview -- wouldn't pay the fine, would reject the ban and might sue the NBA.
The document says Sterling faces a never-before-issued and unjust punishment.
"In reality, Mr. Sterling is being banned for life, fined $2.5 million, and stripped of his ownership rights for a purely private conversation with his lover that he did not know was being recorded and that he never intended to see the light of day," the document says.
Other NBA figures have made bigoted or racially insensitive remarks and weren't punished as harshly, Sterling argues.
His remarks were made in the heat of an argument, and while they were not justified, they were not meant to harm the NBA, the response says.
And while Sterling's remarks were viewed as racist, he says in his response that he was "instrumental in fostering the diverse body of players, coaches, general managers, employees and fans on which the NBA prides itself."
The document also says Sterling was active in the African-American community and that there was only one court case -- which he won -- of discrimination brought against him as owner of the Clippers in more than 30 years.
Sterling asks that the NBA's charges be denied and that his ownership rights not be terminated.
The NBA acknowledged receiving a response from Sterling and reaffirmed the June 3 meeting, saying it will start at 1 p.m. ET to "hear and vote upon this matter."
Donald Sterling's attorney, Maxwell Blecher, had no comment.
Stiviano has said Sterling knew she was recording him and that the two were never intimate.
CNN senior legal analyst Jeffrey Toobin said it appeared the Sterlings were taking a two-track strategy. By filing the document, Donald Sterling is protecting his legal options but, at the same time, he realizes selling the team is the most likely outcome.
"I don't think it's a winning argument, but it is a very impressive document," Toobin said.
CNN's source on the potential sale said it was a busy Memorial Day weekend for Shelly Sterling.
The source indicated she met Sunday with a group led by former NBA star Grant Hill to discuss the sale of the Clippers.
In addition, the source familiar with the situation said she met over the weekend at her Malibu, California, home with ex-Microsoft CEO Steve Ballmer, who made an aggressive offer for the franchise.
Shelly Sterling is interested in fielding offers from billionaire businessman Patrick Soon-Shiong, Hill's group, entertainment mogul David Geffen and Oracle founder Larry Ellison, and entrepreneur Magic Johnson and his Guggenheim partners, who two years ago teamed up to buy baseball's Los Angeles Dodgers.
Johnson has been at the center of the story from the outset because of controversial banter between 80-year-old Donald Sterling and Stiviano. After TMZ first reported those comments, Donald Sterling lashed out at Johnson in a CNN interview, questioning whether someone who contracted HIV should be considered a role model and asking rhetorically "what has he done" for the African-American community.
Johnson called the whole situation sad in his own interview with CNN, in which he urged Sterling to sell the team. The former Los Angeles Laker said if that happens -- and if his Guggenheim partners are interested -- "We'll make a run for it."
One person who apparently won't be the NBA franchise's next owner is media tycoon Oprah Winfrey. Despite much buzz in recent weeks, the source familiar with the situation says that Winfrey is "out of the running."
The same source said Shelly Sterling will accept offers this week for the Clippers.
Shelly Sterling's attorney, Pierce O'Donnell, said no formal offers have been made for the team, which has risen in recent years from a last-place laughingstock to a playoff team with two superstars in forward Blake Griffin and guard Chris Paul.
"Shelly and the NBA are working cooperatively on the transaction," O'Donnell said.
The Sterlings have been publicly defiant since the story broke, challenging the NBA and its right to force the Clippers' sale.
But late last week, the Sterlings agreed that Shelly would try to sell the NBA franchise voluntarily even as the league prepares a vote on terminating their franchise rights. The couple own the team through a trust.
Shelly Sterling previously had said she wanted to own at least part of the franchise, but doesn't want to be a controlling partner. But if it sold -- likely for between $1.5 billion and $2 billion, according to the well-placed source -- the NBA doesn't want either Sterling to have any ownership stake.
Still, they could take solace in the dizzying profit they'd make from such a deal; Donald Sterling reportedly bought the franchise in 1981 for about $12 million.
His response, though, laments that a forced sale would mean the Sterlings would incur "several hundred million dollars" in taxes they could avoid if they were allowed to keep the franchise and leave the team to their heirs.
CNN's Rosalina Nieves, Jill Martin and Michael Martinez contributed to this report.