The court ruled that the inmate, Duane Buck, will now be able to go back into a lower court and argue that he should have a new sentencing hearing.
In a 6-2 ruling, Chief Justice John Roberts wrote the majority opinion holding that Buck has "demonstrated both ineffective assistance of counsel" and has an "entitlement to relief."
Justices Clarence Thomas and Samuel Alito dissented.
"But our holding on prejudice makes clear that Buck may have been sentenced to death in part because of his race. As an initial matter, this is a disturbing departure from a basic premise of our criminal justice system: Our law punishes people for what they do, not who they are," Roberts wrote.
At another point the Chief wrote: "When a jury hears expert testimony that expressly makes a defendant's race directly pertinent on the question of life or death, the impact of that evidence cannot be measured simply by how much air time it received at trial or how many pages it occupies in the record. Some toxins can be deadly in small doses."
Roberts sent the case back down to the lower court for further proceedings.
The case comes at a time of racial unrest in the criminal justice system.
It also comes as two justices, Stephen Breyer and Ruth Bader Ginsburg, have argued the court should take another look at the constitutionality of the death penalty. They questioned, in part, whether the penalty was being applied arbitrarily throughout the country.
At oral arguments, several justices had expressed concern about the testimony introduced in the case.
"What occurred at the penalty phase of this trial is indefensible," said Justice Samuel Alito, who called the testimony "bizarre."
"What competent counsel would put that evidence before a jury?" Justice Ruth Bader Ginsburg asked.
Justice Stephen Breyer questioned whether there was "some good reason" Buck shouldn't be able to reopen his case.
Buck has been on death row for the 1995 murders of Debra Gardner and Kenneth Butler. He did not argue his innocence, but asked for a new sentencing hearing because his own trial counsel was ineffective.
His lawyers argued that "no constitutional rule is more important than the dictate that race must play no role in a criminal sentence, much less a capital sentence."
The legal issue before the court was whether Buck's case meets the "extraordinary circumstances" test justifying the reopening of his sentencing.
At the heart of the matter was the testimony provided by Dr. Walter Quijano, one of two psychologists retained by the defense. Quijano testified that the fact that Buck was black "increased the probability" that he would commit future acts of criminal violence. In Texas, so-called "future dangerousness" must be established before a death sentence is rendered.
"Put another way," Buck's current lawyers argued in court papers, "Mr. Buck's lawyers presented evidence that Mr. Buck was more deserving of a death sentence under Texas law because of his race."
They said such prejudicial evidence is the "epitome of ineffective assistance of counsel."
The state itself eventually conceded error in six other cases where Quijano's testimony had been elicited, they pointed out.
Texas Solicitor General Scott Keller drew a distinction in court between Buck's case and the others because it was the defense itself who called Quijano and elicited race-related testimony on direct examination.
In the opinion, Roberts emphasized the unusual circumstances of Buck's case suggesting that it might not impact other death row inmates.
In court, Keller addressed the circumstances surrounding Buck's crime, including the fact that he murdered his ex-girlfriend, Gardner, in front of her children.
Keller pointed to the fact that he also shot his stepsister at point-blank range and shot another man through the heart.
But he was cut off, as the justices circled back to the race-based testimony.
Keller also argued that Quijano's testimony played a limited role at trial, and that other evidence of his future dangerousness came from the brutality of the murders, Buck's lack of remorse after he was apprehended as well as the testimony of another ex-girlfriend.
A group called the National Black Law Students Association, however, filed a brief in support of Buck emphasizing how the group believes the case impacts race relations today.
"When an expert witness told the jury that Mr. Buck was dangerous because he is black, he dredged up into the open for all members of the jury to see the monstrous specter that is never far from the surface: the violent black brute, the single most fearful, dehumanizing and cruel stereotype black people have had to endure," wrote the group's lawyer, Deborah N. Archer.
This story has been updated.