Details were not immediately available about why Brown, 24, was kicked out of the Malibu facility where he has been treated for the past four months.
Nor was it clear exactly why he was taken into immediate custody, but he had been serving a five-year probation sentence when he entered rehab. That sentence stemmed from a 2009 assault on then-girlfriend Rihanna.
He was "cooperative when taken into custody" as he was picked up from the facility and taken to the Los Angeles County men's central jail where he was being held without bail Friday afternoon, a sheriff's department statement said.
His lawyer and representatives did not immediately respond to CNN calls for comment.
Brown's legal troubles began five years ago when he beat Rihanna as the two were in a rented Lamborghini on a Hollywood street. He pleaded guilty to felony assault in June 2009, which resulted in a sentence of five years of probation and 1,400 hours of community labor.
The judge has revoked Brown's probation twice in the last year, most recently because of his arrest on a sidewalk near the White House after allegedly punching a man.
Brown voluntarily entered a rehab program a day after being released from a Washington jail in October, but he was kicked out a few days later for "throwing a rock through his mother's car window" after a family session at the center, a probation report said. Brown was upset because his mother said she wanted him to stay in treatment, the report said.
"Mr. Brown preceded to walk outside and pick up a rock and threw it through his mother's car window and it shattered," according to letter from the rehab center included in the probation report.
His probation was revoked last November, but the judge allowed him to stay out of jail by entering a 90-day anger management and drug rehab program. Although he completed that program last month, the judge ordered him to remain a resident at the Malibu treatment facility until another hearing on April 23.
Brown's probation officer reported at a hearing last month that the singer "continues to make great improvement" in dealing with anger, stress and drugs, but the judge decided he could not go free until after his trial for an assault charge in Washington on April 17. If he is convicted in that case, the judge would decide at an April 23 hearing if Brown should complete his probation in jail.
Brown has been working on a highway cleanup labor crew in Los Angeles three days a week to fulfill the 750 hours of service remaining in his probation requirements, his probation report said. At that rate, Brown could complete the labor in mid-October and possibly be free from probation requirements by the end of the year.