Speaking at a NATO conference in Brussels, Mattis accused Russia of violating international law, labeling its actions as "aggressive" and "destabilizing."
Mattis, who spoke of the need to negotiate with Russia "from a position of strength" on Wednesday,
was cool on any prospect of working with Moscow.
"We are not in a position right now to collaborate on the military level, but our political leaders will engage and try to find common ground," he said Thursday.
"Russia's aggressive actions have violated international law and are destabilizing."
Mattis also said there is "very little doubt" that Russia has interfered in a number of democratic elections.
It was one of the starkest comments to date on Moscow's alleged hacking made by a Trump administration official.
Asked whether the Kremlin had meddled in the US election, Mattis answered: "There's very little doubt that they have either interfered -- or they have attempted to interfere -- in a number of elections in the democracies."
Earlier Thursday, Russian President Vladimir Putin urged for communication between his country's and America's intelligence services to be restored.
Putin said "all relevant governments and international groups should work together."
On Wednesday, Mattis reiterated US support for NATO while highlighting the threat posed by Moscow.
Mattis labeled NATO a "fundamental bedrock for the US and all the transatlantic community," and urged it be wary.
"While the United States and the alliance seek to engage Russia, we must at the same time defend ourselves if Russia chooses to act contrary to international law," Mattis said.
"Just as we did throughout the darkest days of the Cold War, the United States remains willing to keep open political channels of cooperation and deescalate tensions.
"We remain open to opportunities to restore a cooperative relationship with Moscow, while being realistic in our expectations and ensuring our diplomats negotiate from a position of strength."
On Wednesday Mattis also warned that the US could "moderate" its commitment to NATO if other member states fail to meet the requisite spending targets.
Just five of the 28 member states currently meet the alliance's spending target of 2% of GDP -- leading Mattis to urge others to pay up.
"America will meet its responsibilities, but if your nations do not want to see America moderate its commitment to this alliance, each of your capitals needs to show support for our common defense," he said.
NATO's Secretary General Jens Stoltenberg called it a "fair demand" during a news conference Wednesday.
"The message from Secretary Mattis is a fair message, it is a fair demand, that we need fairer burden sharing."