Anthony Kennedy retirement watch at a fever pitch

Story highlights

Kennedy, 80, has been a consistent swing vote in key Supreme Court rulings

Dozens of his former clerks will be in Washington for a reunion this weekend

(CNN) —

Justice Anthony Kennedy, the man who so often determines the outcome of the most controversial Supreme Court cases, is himself the center of brewing speculation.

Will he stay or will he go?

The rumors have swirled for months and the 80-year-old justice has done nothing either personally or though intermediaries to set the record straight on whether he will step down.

Helping drive the speculation, dozens of Kennedy’s former law clerks traveled to Washington this weekend to participate in a private clerk reunion that occurs regularly – and many of them wondered if it will be their last chance to meet with him while he is still on the bench.

At the end of a dinner with the former clerks Saturday night, Kennedy addressed the crowd, saying he had heard some speculation about an announcement tonight, “and here it is,” he said: The “bar will be open after dinner.”

But sources close to Kennedy say that he is seriously considering retirement, although they are unclear if it could occur as early as this term.

His departure would cause a seismic shift and offer President Donald Trump a chance to continue reshaping the court. Trump’s first nominee – Justice Neil Gorsuch, himself a former Kennedy clerk – joined the court earlier this year.

President Reagan meeting with Judge Anthony Kennedy in the Oval Office

PHOTO: White House Photo

Like no other justice in recent history, Kennedy has cast the vital swing vote in cases that grab the countries’ attention.

Why is Kennedy so important?

To liberals he is a hero for Obergefell v. Hodges – a landmark opinion that cleared the way for same-sex marriage in 2015 and will likely be his most lasting legacy.

“They ask for equal dignity in the eyes of the law and the Constitution grants them that right,” Kennedy wrote.

To the delight of abortion rights supporters, Kennedy voted to reaffirm the core holding of Roe v. Wade in 1992.

“As the court’s most important Justice – at the center of the institution’s ideological balance – Justice Kennedy’s ability to bridge the divide between left and right on critical issues such as the right to access abortion cannot be overstated,” said Elizabeth Wydra, president of the Constitutional Accountability Center. “Replacing Justice Kennedy with a Trump nominee would almost certainly sound the death knell for Roe, just as candidate Trump promised during the 2016 campaign.”

But nine years later, he sided once again with the liberals on the court to strike down a Texas law that abortion rights supporters thought was the most strict nationwide. Without Kennedy’s vote, the law would have been allowed to go into effect, inspiring other states to pass similar legislation.

In the same term, Kennedy pivoted on the issue of affirmative action when he voted for the first time in favor of a race-conscious admissions plan at a public university.

After that term, former acting Solicitor General Neal Katyal said, “It is very much Justice Kennedy’s Court.”

“You can’t understand how important his affirmative action opinion is without understanding his earlier jurisprudence,” said Katyal. “For decades, he has been the court’s most eloquent voice on the need to be color blind – why he changed his mind is something historians will debate for decades.”