Perez, the billionaire CEO of Related Group, has maintained a 20-year friendship with Trump.
They've done business together.
The Perez and Trump families have vacationed together.
Perez said he likes Trump and that he's always been a good business partner.
But the relationship stalled in December when Perez received an email from The Trump Organization that included attachments showing a border wall and a handwritten note from Trump.
"The handwriting was his," Perez said.
CNN was shown the email. The White House said it is unaware of the conversation between the two, and a senior White House official told CNN Trump has been looking at various blueprints and sketches of the wall along with his top advisers.
The email Perez received gives a glimpse into Trump's preliminary thinking about how to design it.
"Inside the email was the wall, you know, the 2,000 mile, 30-foot wall, with some sketches, and he said, 'I would love to get you involved in this. Please give me a call,'" Perez said.
The picture actually shows two walls, a primary wall and a secondary wall, with a security road in between.
It's a design former border patrol agent Rowdy Adams believes would be most effective in an urban settings.
"(By) the time someone breaches that first barrier and gets across the roadway ... you might want a secondary fence to slow them down until a response vehicle shows up," Adams explained.
Trump has at times said he wants to build a wall all along the border. Other times, he's said that it's not necessary in places where there's a "natural barrier."
On Wednesday, Trump revealed the wheels are in motion.
"The wall is getting designed right now," he said.
But he won't have his friend's help.
Perez said he's a Democrat who, despite a request from Trump himself, did not support his candidacy.
"I am totally opposed to the wall because a) it's wrong and b) it's not going to work anyway ... It's an insult to all Hispanics and maybe all immigrants in this country."
Perez believes an improved economy in Mexico is a better solution. He said he has five projects in Mexico right now.
"I think my Mexican partners would shoot me if I'm in any way involved in the wall," he said.
Perez, who was born in Argentina to Cuban parents, smiled and said he told Trump he has one more concern.
"I said you need to remember also that my name is 'Perez,' so if you do build a wall, what side of the wall do I get to stay in?" he said.