(CNN)Steve Harvey has set boundaries and is standing by them.
Harvey came under fire Wednesday after a strongly worded memo sent to the staff of his daytime talk show was leaked by a blogger Robert Feder.
In the note, which seems to have been sent ahead of the current fifth season of his talk show "Steve Harvey," the host asks that there be no attempts to meet with him while he's in his dressing room, in the makeup chair, or in the hallway -- "unless I ask to speak with you directly."
He also insisted that all those attempting to communicate with him "schedule an appointment."
"IF YOU OPEN MY DOOR, EXPECT TO BE REMOVED," he wrote, in all caps. "My security team will stop everyone from standing at my door who have the intent to see or speak to me."
Harvey was asked about the memo, leaked yesterday, by Entertainment Tonight, who probably made an appointment.
"I could not find a way to walk from the stage to my dressing room, to sit in my makeup chair, to walk from my dressing room to the stage or to just sit and have lunch without somebody just walking in," he told the outlet. "I've always had a policy where, you know, you can come and talk to me -- so many people are great around here, but some of them just started taking advantage of it."
Harvey told ET the decision to write the memo was meant to ask "everyone to simply honor and respect" his privacy.
"I'm in the hallway, I'm getting ambushed by people with friends that come to the show and having me sign this and do this. I just said, 'Wait a minute,'" he said. "And in hindsight, I probably should've handled it a little bit differently."
He is not, however, sorry that he wrote it.
"I just didn't want to be in this prison anymore where I had to be in this little room, scared to go out and take a breath of fresh air without somebody approaching me, so I wrote the letter," he said. "I don't apologize about the letter."
"Steve Harvey" will end its run in Season 5. A new syndicated daytime series from Harvey, titled "Steve," will debut in September and be filmed in Los Angeles.