During a special City Council meeting he convened Wednesday, Emanuel said he was sorry about the circumstances surrounding the death of Laquan McDonald, including the fact it took 13 months before police dashboard camera video of McDonald's shooting became public and the officer who killed him was charged.
"I own it," the mayor said. "I take responsibility for what happened, because it happened on my watch.
"And if we're going to fix it, I want you to understand that it's my responsibility."
also talked more broadly about a "lack of mutual respect" between some Chicagoans and police. Some believe officers treat them differently and make unfounded assumptions because of how they look or where they live, according to the mayor, a perception that feeds their anger and despair.
Demonstrators, who called for Emanuel to step down, confronted a line of police before the officers stepped aside and let them through as they flooded the streets.
Sometimes the protesters were walking down Michigan Avenue, but they also would stop and sit down in the road.
Some people have had an impression that police put protecting their own ahead of protecting all citizens, which critics say was illustrated by how other police officers' accounts of McDonald's death were at odds
with what dashcam video showed. Emanuel referred to a "code of silence."
Putting it succinctly, the mayor said, "We have a trust problem."
One protester called for police to release all videos in controversial cases in which citizens have died in incidents with police officers.
Another told CNN the angst between Chicago residents and police is not new.
A woman who identified herself as Magda said: "We cannot allow this (lack of justice) to continue."
"Historically Chicago does not have a good name with the treatment of the people, treatment of citizens. And I'm really proud of the young people that are here today because they're standing up," she added.
Many of the protesters appeared to be in their early 20s. Many of them were shouting, "Rahm, resign!" They also called for Cook County State's Attorney Anita Alvarez to step down.
Protesters began on Michigan Avenue then hooked onto other streets. When they went by the famed Tavern on Rush, the wait staff peered out windows giving thumbs up and snapping photographs.
Marcher Jay Payne, a 41-year-old paralegal with a civil rights law firm, said Emanuel and Alvarez need to resign because they've lost the support of the community. "They're protecting criminal cops," he said.
At one point a woman shouted through a megaphone, "Show me what democracy looks like." She kept reinforcing to the crowd that this would be a peaceful protest. Police officers walked behind and beside demonstrators as they proceeded through the streets.
The protests broke up around sunset, then resumed at a meeting of the Chicago Police Board.
CNN affiliate WLS
reported dozens of protesters angrily spoke during the meeting. More than 100 demonstrators gathered outside police headquarters.
Several officer-involved shootings
The mayor had a lot of recent history to work with ahead of Wednesday's speech -- little of it good.
The most high-profile case relates to McDonald. The 13-month gap from the incident until the charge and video release was too long for some, who accused police and Emanuel of a cover-up.
It contributed to Garry McCarthy losing his job as Chicago's police superintendent
and spurred calls for Emanuel to resign
. The furor grew when the public learned about the other officers' accounts.
There's also Ronald Johnson, killed by police eight days before McDonald. On Monday, reporters first saw video of his death, with Alvarez saying the officer who killed Johnson won't face charges
There also has been renewed focus on 17-year-old Cedrick LaMont Chatman, whose 2013 death near a bus stop was captured by four video cameras.
The Chicago agency that investigates all police-involved shootings, the Independent Police Review Authority, deemed that shooting justified. But Lorenzo Davis, IPRA's original supervising investigator on the case, didn't agree
. Davis said he was fired in July when he refused to change his report.
On Wednesday, Emanuel said IPRA will take a fresh look at the circumstances surrounding Johnson's and Chatman's final moments. He was even more definitive and emphatic talking about McDonald's death, which he called "totally avoidable."
But fixing this isn't simply a matter of prosecuting Van Dyke or giving McDonald's family money (the Chicago City Council approved a $5 million settlement in April). The mayor outlined steps that have been taken so far or are in the works, like police's increased use of body cameras, reshaping the leadership and approach of IPRA, and a renewed focus on community policing.
More than anything, according to Emanuel, there needs to be a culture change in how police see and deal with citizens, and vice versa.
"It is one thing to train officers on fighting crime," he said. "It is a whole other thing to build friendships and relationships, which are integral to fighting crime."
Justice Department probing Chicago police
As he's done before, Emanuel noted Wednesday that most Chicago police officers, day in and day out, do a good job. And it's a very difficult one, given the dangers in parts of the city -- a fact that not all citizens grasp, according to the mayor.
Chicago began being called the murder capital of the United States in 2012, after it registered 503 homicides, more than any other city. The FBI's 2014 statistics showed 411 killings, more than the 333 in New York and 260 in Los Angeles, two cities with larger populations.
Emanuel lamented Wednesday that young people join gangs "in search of self-worth" and because they don't see hope for a better life, saying, "We as a city must and can do better."
This includes tackling issues related to poverty, opportunity and family, as well as law enforcement's relationship with its citizens, he said.
Emanuel said Chicago needs help and welcomed a U.S. Justice Department investigation
into its police department, announced on Monday.
The federal "pattern-and-practice" investigation, as it is called, will look into whether city officers have made a habit of violating the law or the Constitution while on the beat, according to Attorney General Loretta Lynch.
"We will be a better city for it," the mayor said Wednesday. "It is in our self-interest, because we need (federal) assistance to make the fundamental and necessary changes."
A Chicago-based state legislator, citing the public's lack of confidence in Emanuel filed a bill
that would allow for a recall election of the mayor. Rep. La Shawn Ford told CNN affiliate WBBM, "Until he can regain confidence, we have to have something in place that we can try to bring the city together." Rep. Mary E. Flowers is a co-sponsor to the bill.
The house next meets on January 13.