(CNN) -- One day after joining more than a score of other women accusing Bill Cosby, famed model Beverly Johnson told CNN that the legendary comedian's reputation for such acts was a secret in Hollywood -- one she only wished she knew about 30 years ago.
Johnson detailed her own story about Cosby in Vanity Fair on Thursday, claiming he drugged her during a meeting at his Manhattan residence in the 1980s. She said in that article -- and again Friday on CNN's "New Day" -- that she doesn't believe he sexually assaulted her then, but she yelled obscenities at him before he dragged her outside and put her into a taxi.
As more women have come out against Cosby in recent weeks, many of whom say the comedian did rape them, Johnson said she began talking to people in the entertainment business about her own story.
"The response was, 'Oh yeah, we know he has been doing that for quite a while.' It was like, everyone knew," Johnson said. "But people like myself, I most certainly wouldn't have gone to Bill Cosby's brownstone if I knew the reputation he had with assaulting women."
Including Johnson, at least 23 women have spoken out to various media outlets with accusations against Cosby. This number includes 20 who have spoken with CNN, spoken on camera about their allegations or been the subject of responses from Cosby's attorneys.
CNN has reached out to Cosby's attorney Martin Singer about Johnson's allegation, but has not heard back yet. But overall, the comedian's camp has strongly and persistently denied these allegations.
Johnson: All of American was a Cosby fan
Johnson burst onto the scene as a model in 1970s, becoming the first African-American woman to appear on the cover of American Vogue.
But she was in a rough patch in mid-'80s, coming off a rough divorce that included a difficult custody battle, when her agent called to say that Cosby wanted her to audition for "The Cosby Show."
Johnson said that -- as someone trying to get more roles as an actress -- she was "very excited" by the prospect of appearing on TV's No. 1 show headlined by one of the nation's most revered figures.
"There wasn't a bigger figure in the entertainment industry than Bill Cosby and I was a big fan," she said. "I don't know anyone in white America or black America that was not a fan of Bill Cosby and 'The Cosby Show.'"
Johnson met up with Cosby, including one meeting in which she took her young daughter to Cosby's brownstone.
"He was very charming, very nice to my daughter and I," she recalled. "He suggested that I come back in a couple of days to rehearse the scene in 'The Cosby Show.'"
Claims she cursed comedian
So Johnson did go back, this time alone. As she recalled to CNN, Cosby offered her a cappuccino and was "very insistent" she take it, promising that it would "be the best coffee that I've ever had."
"The first sensation was a little woozy, and so then I took another sip," Johnson said. "After that second sip, I knew I had been drugged. It was very powerful, it came on very quickly. The room started to spin. My speech was slurred."
Steadying herself on Cosby, Johnson claimed she "just kind of cocked my head, because at that point I knew that he had drugged me," and told him, "You are a (motherf*****), aren't you?"
"I knew that he had drugged me, and I wanted him to know that he had drugged me," she said. "And the only word I could get out was (that expletive), and I kept saying it to him louder and louder."
Johnson admits her recollection at this point were hazy, though she remembered Cosby becoming angry, grabbing her by one arm, dragging her outside and flagging down a taxi for her.
"All I remember after I got into that taxi cab, and somehow telling (the driver) my address (was) saying, 'Did I just call Bill Cosby an MF?' I was concerned about him, rather than me."
Cosby a 'powerful man' in the industry
In the Vanity Fair article, Johnson wrote that she tried a few days later to call Cosby on a private number so he could "come clean and explain why he'd done what he had."
Cosby's wife, Camille, answered the phone. When Johnson asked to speak to Cosby, Camille told her it was late and they were in bed. Johnson said she'd call back, but never did.
Talking to CNN about why she didn't go public then with her story, Johnson explained that she was drugged but didn't think she was sexually assaulted and also spoke of beinging ashamed of speaking out. The fact the man she'd be accusing was Bill Cosby also factored in.
"(He was a) powerful man in an industry that I would like to be a part of," she said. "That went through my mind."
"It's not easy doing the right thing sometimes!!! Thank you for your love & support Break the Silence of Sexual Abuse!— Beverly Johnson (@BeverlyJohnson1) December 11, 2014
In her live interview with CNN, Johnson said she didn't "want to see anything happen to Bill Cosby" by sharing her story. What she does want -- besides showing solidarity with other Cosby accusers, like her longtime friend Janice Dickinson -- is for her speaking out to encourage others who have been the victims of sexual assault by any perpetrator to speak up.
"This, to me, is not about Bill Cosby. This, to me, is about violence against women," Johnson said. "... What I want to see happen is that women come out and speak their truth."