Alabama had planned to put Vernon Madison, 67, to death on Thursday night, but less than a half hour before the execution was to take place, Justice Clarence Thomas issued a temporary stay.
About two and a half hours later, the high court announced that a stay had been granted while the court decides what to do with an appeal from the defense.
Madison has been convicted three times in the shooting of Mobile police Cpl. Julius Schulte, who was responding to a April 1985 domestic disturbance call. Madison, who was on parole, sneaked up behind Schulte and shot him twice in the head, according to court documents. He also shot his girlfriend, who survived her wounds.
At his first and second trials, Madison argued that he was not guilty because he was mentally ill. At his third trial, he argued self-defense.
His attorneys from the Equal Justice Initiative, based in Montgomery, filed a petition
Wednesday with the Supreme Court.
Madison's sentence was imposed in 1994 by a judge, after a jury recommended life without parole. His lawyers argue the death sentence is unfair because a 2017 Alabama law no longer permits judicial override and they say Madison's sentence should be commuted to life without parole.
"Given Alabama's rejection of judicial override, the death sentence in this case constitutes cruel and unusual punishment and violates Mr. Madison's rights to a jury, fair and reliable sentencing and to due process and equal protection of the laws as guaranteed by the Fifth, Sixth, Eighth and Fourteenth Amendments of the United States Constitution and Alabama law," they wrote in their petition.
In prior appeals, Madison's attorneys have argued their client doesn't fully understand why he is being punished because dementia has taken his ability to remember his crime. They also say his health is declining.
In November, the US Supreme Court agreed with a state court ruling that Madison was mentally competent.
"The state court did not unreasonably apply (two prior decisions) when it determined that Madison is competent to be executed because -- notwithstanding his memory loss -- he recognizes that he will be put to death as punishment for the murder he was found to have committed," the justices wrote.
Madison's lawyers asked Gov. Kay Ivey for clemency.
"Mr. Madison suffers from vascular dementia as a result of multiple serious strokes in the last several years, and no longer has a memory of the commission of the crime for which he is to be executed," attorneys wrote.
"He does not understand why the state of Alabama is attempting to execute him," they said.
They argue that executing someone with dementia is counter to how society treats vulnerable citizens.
CNN reached out to the governor's office and the state's Attorney General's office but didn't get a response.
There are 182 inmates on Alabama's death row, three of whom have been there longer than Madison.
Before his execution was stayed, Madison had two oranges for his last meal and did not made any statements, officials said.