A group supporting the filing, Organization for Black Struggle
, a 35-year-old St. Louis-based activist group, said the five would try to collect enough signatures -- 15% of the city's registered voters -- in the next 60 days.
The five "initiated recall (attempt) due to Mayor Knowles' failure to adequately rein in an out-of-control police department during the protests following Mike Brown's death," the activist group said.
The city government acknowledged Saturday that it received the affidavit.
Earlier this week, the police chief and the city manager resigned
in the wake of a U.S. Justice Department report alleging institutionalized racism at just about every level of Ferguson's municipal government, including the police department.
Residents elect City Council members, who in turn appoint the city manager. The city manager directs and supervises all city departments, including police.
The top municipal court clerk was fired earlier in connection with racist emails.
This month's Justice Department report was initiated after one of Ferguson's police officers, Darren Wilson, shot and killed African-American teen Michael Brown in August, a shooting that spurred months of protests.
Wilson, a white officer who said he shot Brown in self-defense, will not be charged in the case -- a grand jury declared it wouldn't indict him in November, and the Justice Department said this month that it would not bring federal civil rights charges in the case. Wilson resigned from the department in November, citing security concerns.
Mayor won't step down
Knowles indicated Friday he won't be stepping down.
"I think it's important to recognize that there's a lot of people who may be angry at the situation; there's a lot of people who are frustrated in this community with the way things have gone down," the mayor, who is white, told CNN.
"But there's a lot of people who still -- and who have expressed this to me -- express confidence in both my willingness, and members of the (City) Council's willingness, to listen, to be responsive, and to make changes as necessary," he continued.
"People in the community recognize this, now, not everybody. I didn't win every time with 100% of the vote. But I can tell you there are ways to remove me if that is the will of the people," the mayor said.
Blake Ashby, a white resident of Ferguson, said Friday that he believed Knowles has "consciously tried to reach out to all parts" of the city of 21,000 people.
"If we lose Mayor Knowles, we lose a force for change, and it will be harder to make the changes that the DOJ (Department of Justice) is asking for," Ashby said.
Rasheen Aldridge, a black member of a commission charged with recommending reforms in Ferguson, said that Knowles needed to resign in the wake of the DOJ report.
"He knew what was going on during his watch," said Aldridge, a member of the Ferguson Commission, which Gov. Jay Nixon formed last year.
'Several leads' in Ferguson shooting of officers
Investigators still are seeking breaks in the case of two police officers who were shot Wednesday night during a protest in Ferguson.
While the demonstrators' focus was Ferguson, neither of the wounded officers works for that police department.
One is from Webster Groves, a St. Louis suburb 13 miles south of Ferguson. The officer -- a 32-year-old with seven years' experience -- was shot at the high point of his cheek, just under his right eye, police said.
The other was hit in the shoulder and the bullet came out the middle of his back. He is a 41-year-old officer with the St. Louis County Police who has been in law enforcement for 14 years.
Both were treated and released.
"I cannot tell you an arrest is imminent, and there's certainly no one in custody," St. Louis County Police Chief Jon Belmar told reporters Friday afternoon.
"The detectives are working this investigation around the clock, and they will not rest until we have a conclusion in this investigation," the chief said.
Police have "several leads," he said. "I think we have a pretty good general idea of where we think the shots came from."