The actor was promoting his new film "Terminator Genisys," although the five-time Mr. Universe spoke about much more than his famous robotic role.
When Stern asked whether the breakup was Schwarzenegger's first real personal setback, the actor said it was entirely his fault.
"That was, without any doubt, the biggest setback and the biggest failure," he said. "You can't point the finger at anyone else."
The actor and former governor of California was married to Maria Shriver for more than 25 years before the two split in 2011 after it was revealed that he had fathered a child
with Mildred Baena, a former housekeeper.
Though he and Shriver got marriage therapy, it was "nonsense talk," he said: "It was the biggest mistake I ever made."
Through it all, Schwarzenegger said, he's maintained a loving relationship with his and Shriver's four children. He said the split didn't harm his relationship with them because they kept open lines of communication.
On a recent trip to Germany, he said, he returned to his hotel room to find it full of balloons and loving cards from his children.
The former politician also spoke about son Patrick's relationship with singer and actress Miley Cyrus and whether he was concerned about Cyrus' polarizing personality.
"Under normal circumstances, maybe I would be nervous," Schwarzenegger said, calling Cyrus "a fantastic person."
The young couple have split, and he is now pursuing a career in acting. Schwarzenegger said he is proud of all his children and hopes they will pursue their dreams.
"I don't care if they're a plumber, if they're a writer or if they're a teacher, whatever they want to do, as long as they're passionate," he said.