The documents, obtained by CNN from the Mitrokhin Archive at Churchill College at the University of Cambridge, say that Abbas, who completed graduate work in Moscow in 1982, was a KGB agent while he was a member of the Palestine Liberation Organization in Damascus.
Palestinian leaders decried the report as a "smear campaign" targeted at Abbas.
"The Israeli government and its surrogates are using smoke and mirrors to confuse the public and deceive the international community in order to derail any efforts to revive the peace process, while appropriating more Palestinian land and building new illegal settlements," said Jamal Dajani, director of strategic communications at the Palestinian Prime Minister Rami Hamdallah's office.
Abbas' office declined to issue a separate statement, instead relying on the response from Hamdallah's office.
Israeli Prime Minister Benjamin Netanyahu's office declined to comment.
The story was first reported by Israel's Channel 1 Foreign News Editor Oren Nahari, who learned about the documents from his predecessor, a researcher studying the Mitrokhin Archive.
The Mitrokhin Archive is a collection of documents from Maj. Vasiliy Mitrokhin, a senior archivist in the KGB's foreign intelligence archive from 1972 to 1984. He defected in 1992 to the United Kingdom after the fall of the Soviet Union.
While working as an archivist, Mitrokhin took secret handwritten notes of the material to which he had access: hundreds of thousands of documents from the KGB's global network of intelligence.
Mitrokhin's files, totaling 19 boxes and thousands of papers, were cataloged and archived at the Churchill Archives Center. The files were made public in July 2014.
One of the files, labeled "Envelope K-24" relates to the "Near and Middle East." Paragraph 244 of the section refers to Mahmoud Abbas.
"'Mole' - Abbas Mahmoud 1935. Native Palestinian. Executive Committee FATAH, PLO in Damascus. KGB Agent."
The claim that Abbas was a KGB agent may have been simply an interesting footnote if not for the timing of the story. In recent days, Russian President Vladimir Putin has been trying to arrange a meeting between Abbas and Netanyahu to restart the moribund peace process.
Mikhail Bogdanov, Putin's special envoy for the Middle East peace process, visited the region this week to meet with the two leaders, and Russia's Ministry of Foreign Affairs announced the two sides had agreed to a meeting "in principle."
Friday was floated as a possible meeting date, but both sides blamed each other for the meeting's postponement.