Tripoli, Libya (CNN) -- Libya's new government said it has control of new areas in Bani Walid as the battle rages on for one of the last cities loyal to the ousted ruler.
The city center and the northern part of the city are now under government control, said Ali Daeki, a member of the executive crisis committee.
Government forces have surrounded the city from all sides, and arrested more than 20 loyalists of former ruler, he said Saturday.
Troops loyal to Moammar Gadhafi are now in Dahra, a residential area in the southern part of the city, according to Daeki.
National Transitional Council fighters also battling for control in Gadhafi's hometown of Sirte, and will boost Bani Walid troops as soon as that battle is over, said Abdurahman Bousin, a council spokesman.
As the battle for control continues, the U.N. human rights office expressed concern on the number of prisoners in Libya and their treatment.
"It could be up to 7,000," said Mona Rishmawi, a senior official with the group in Geneva, Switzerland. "At this stage, there is no police infrastructure, there is no prison authorities. ... Right now, the Justice Ministry is not fully functional."
"There is allegations and evidence of torture" in the prisons, she said, citing lawyers, clients and human rights groups.
CNN's Raja Razek and Ingrid Formanek contributed to this report.