The chain has since apologized.
Chili's has a policy of providing veterans a free meal every Veterans Day.
Walker said he was disrespected at the restaurant's Cedar Hill, Texas, location when a man wearing a Donald Trump shirt interrupted his meal and challenged his service record, the veteran posted on his Facebook page.
"He said he was in Germany, and that they did not let blacks serve over there," Walker said, detailing what sparked Friday's incident.
The man eventually got the restaurant manager involved. The manager, after an exchange, took the meal away. A video of the exchange was later posted online.
Walker told CNN's Don Lemon the Trump supporter made him feel like was a slave.
"He really made me feel like I was almost like a slave who had to show paperwork to show my freedom," he said.
"I do believe that I was singled out, as a black man, in that restaurant. And were it not for the past temperature of the nation -- of this election -- you know I don't think he would be as emboldened to approach me in the manner that he did and just outright label me, you know, as a black man trying to steal food."
Walker said he'd never encountered such treatment before.
"I've had racism, but just to be as bold as he was -- he felt like he had the right to do this to me, and the manager participated in that," he said.
Chili's issues apology
Chili's issued a statement on Tuesday.
"On November 11, one of our restaurants failed to live up to our expectations and in doing so, we let down a valued guest," the president of Chili's Grill & Bar, Kelli Valade, said in an open letter. "Unfortunately, Mr. Walker was not treated in a manner our veterans deserve."
Chili's "personally apologized" to Walker for the "experience" and thanked him for his service.
Since the incident, the restaurant has "removed" the manager, the letter said.
Walker's attorney, Kim T. Cole, said they were trying to reach a resolution with the parent company of Chili's.
"At this point we are still in discussions with Brinker international -- the parent company of Chili's -- and they have spoken directly with Mr. Walker via telephone and expressed their sincere apology for the incident that occurred," she said. "And we're working together to reach a resolution."
Cell phone captures altercation
Moments into the video, Walker is seen flashing an identification card to the Chili's manager.
"You've questioned my military service and now you're questioning my service dog?" Walker repeatedly asks and talks over the restaurant manager.
"Sir, you need to calm down," the manager replies.
"I'll leave as soon as you answer my two questions," Walker says.
"No, I'm not answering your questions, I asked you not to record," the manager responds.
"Did you see my military service?" Walker asks the manager again. The restaurant manager denies on camera that he saw Walker's military service credentials.
"I'll take your food away from you, sir," the manager says.
"Oh, now you're taking my food away from me?" Walker says in response.
"Sir, I'm so sorry, you didn't provide any documents for me, you have a great day," the manager replies. He takes Walker's food off the table and walks away.
"Yes, I did provide documents to you, and they saw you," Walker says.
Walker gets the last words in the video: "I want the police called at this point."
Walker's initial accuser remains unidentified.
"I was grossly offended, embarrassed, dehumanized," Walker posted.