"I'm surprised State didn't do that already," Judge Emmet Sullivan in Washington said at one point during the hearing after first admonishing the State Department's counsel for not being able to confirm whether the FBI was in possession of the server.
The conservative organization Judicial Watch is suing the State Department over failure to provide records related to the employment of top Clinton aide Huma Abedin.
The State Department contends it is already conducting a reasonable search for those records, and they should not be expected to search devices and documents that aren't in their possession.
"Nobody buys that," Judicial Watch's Director of Research and Investigation Chris Farrell told CNN. "The judge doesn't buy that, and we certainly don't accept it."
Farrell also said he was pleased with Sullivan's order.
"I think we're making further progress on finally getting the State Department to come to grips with its obligations under the law," he added.
A State Department spokesperson declined to comment on the judge's latest order.
Clinton directed her aides to turn over the server to the FBI last week, along with a thumb drive containing more than 55,000 of her emails as secretary of state, in response to an ongoing investigation into her use of a private email server while in office.
The move came after an inspector general for the intelligence community told Congress classified information had been found in several emails from that server.
The server was wiped clean earlier this year, so it is unclear how much information will be retrieved beyond the thousands of emails Clinton already provided to the State Department.
In Thursday's hearing, Sullivan also proposed the State Department reach out to Clinton and her lawyers to determine whether any entity, such as the company that provided or managed her server, might have access to backup records.
Because neither Clinton nor the FBI are parties in the Judicial Watch lawsuit, the judge did not issue the order directly to them, but rather asked the State Department to seek their cooperation.