(CNN) -- His eyes stinging with pepper spray, a developmentally disabled 21-year-old man was hit and forced to the ground before being taken into custody by California sheriff's deputies in an incident that left his family Thursday demanding justice.
Antonio Martinez was taken to a hospital and detained for possible obstruction of justice, but there was no citation or charge filed on that or other counts, said San Diego County Sheriff's Department spokeswoman Jan Caldwell.
His father is considering suing the department and the city of Vista, California, the young man's sister, Jessica Martinez, said.
"But really, all we want is for the sheriff's deputy who did this to my brother to get fired," Jessica Martinez, 20, said. "That's all we want. We want justice."
The incident occurred Tuesday evening in Vista, where sheriff's deputies were looking into a domestic violence incident, Caldwell said.
One of them saw Martinez cover his head with his hood and, believing he might be involved in the incident, tried to talk with him, said the spokeswoman. The 21-year-old -- who weighs 158 pounds and stands 4 feet, 11 inches tall, according to a Sheriff's Department document -- didn't respond to the commands, she added.
Martinez is well-known around the Vista neighborhood, where he lives and his family has a bakery, his sister said. He was out walking between his home and the bakery when the deputy called out to him, at which point neighborhood men explained that the young man had Down Syndrome and wasn't doing anything wrong, according to Jessica Martinez.
The deputy didn't back off. Instead, while trying "to gain compliance and prevent a possible escape," he used pepper spray on Antonio Martinez, said Caldwell from the Sheriff's Department.
The deputy began using a baton as an agitated crowd approached, the spokeswoman added. The deputy hit Antonio Martinez with it, forcing him to the ground, then levied "a couple more strikes to get his hands free," Sheriff's Department Capt. Joe Rodi told CNN affiliate KGTV.
Jessica Martinez said she joined the crowd after a nearby business owner had run into her bakery, telling them police were beating up her brother.
"I heard my brother screaming mine and my sister's name," she said. "He was screaming for help.
"My brother wasn't fighting. He was crying and screaming. He was scared."
Two deputies got on top of Antonio Martinez as he rubbed his eyes from the pepper spray, eventually handcuffing him and putting him in a police car, his sister said.
He was taken to Tri-City Medical Center in Oceanside, California, where Jessica Martinez and her father found him with scratches but no broken bones.
The family asked to take Antonio Martinez home from the hospital, but a sheriff's deputy refused, Jessica Martinez said. It wasn't until around 11:30 that night -- at the Sheriff's Department, after being brought back from the hospital -- that he was released, she said.
"When I asked my brother if he was sad, Antonio told me his heart hurt," his sister said. "He's just really sad right now, and he doesn't want to go to the bakery any more."
The San Diego County Sheriff's Department has opened an investigation, according to Caldwell.
Rodi met with the family Wednesday and acknowledged the situation should have been handled differently.
"We made a mistake here," Rodi told KGTV.