Kerry's remarks came the same day the Navy released more information about what exactly happened when the sailors were taken into custody, including their interactions with Iranians.
Kerry has been negotiating with Iran on a number of fronts for well over a year, including on the President's nuclear deal and a prisoner swap completed over the weekend.
But after U.S. Navy vessels "regrettably, inadvertently" drifted into Iranian waters, per Kerry, and the 10 sailors were taken by Iranians last Tuesday, Iran's military released footage of the troops with their hands behind their heads at gunpoint.
Though the sailors were released within 24 hours, the footage was inflammatory.
"I was very angry. I was very, very frustrated and angry that that was released," Kerry told John Berman on CNN's "New Day." "I raised it immediately with the Iranians. It was not put out by the Ministry of Foreign Affairs or the government directly, it was put out I think by the military over there, the (Revolutionary Guard), who is opposed to what we are doing."
Also on Monday, the Navy released more information about how the incident, which lasted about 15 hours, played out.
As the footage showed, the soldiers were held at gunpoint on the ship by Iranian troops. The Navy said the troops were also under machine gunpoint from more Iranian troops on the responding ships.
No fire was exchanged during the incident, but there was what the Navy categorized as as "verbal exchange."
The Navy also said that everything from its ships was accounted for after the incident except for two SIM cards removed from satellite phones.
As was known, the two Navy vessels were traveling together through the Persian Gulf with a scheduled refueling stop along the way. The vessels ended up in Iranian waters and one began to report engine trouble, though it's still not clear how they came to be there.
At 9:10 a.m. ET, the Navy Central Command was informed the ships were being questioned by the Iranians. Roughly 20 minutes later, communications began to deteriorate and were lost altogether 15 minutes after that.
The U.S. began to put together a rescue mission and broadcast their plans in Farsi over open channels to signal to parties in the area.
After three-and-a-half hours, the Navy received word from the Iranians that the sailors were in custody and "safe and healthy."
They were released at 3:43 a.m. ET, when it was already daylight in Iran.
Kerry said Monday that only because of the recent negotiations with Iran was he able to rapidly secure the sailors' release.
"The challenge is, three or four years ago, we wouldn't have known who to call ... and it could have grown into a major kind of hostage confrontation the way it had previously," Kerry said. The progress "could not have happened were we not building on this path with respect to the nuclear program," he added.
He would not divulge what he said to his counterparts about the sailors but stressed that he was firm.
"I'm not going to discuss what I said or didn't say, but suffice it to say I made it crystal clear how serious this was, it was imperative to get it solved ... and within hours we had an agreement," Kerry said.
In a separate interview with Wolf Blitzer, Kerry dismissed concerns that Saudi Arabia might seek to buy a nuclear weapon to balance the deck with Iran.
"You can't just buy a bomb and transfer it," Kerry told Blitzer.
"There's all kinds of (nonproliferation treaty, or NPT) consequences," Kerry continued. "I mean, there are huge implications of that. And Saudi Arabia knows, I believe, that that is not going to make them safer, nor is it going to be easy, because the very things that Iran went through, they would then be subject to with respect to NPT and inspection and so forth."
Kerry also addressed the missing American Robert Levinson, a former FBI agent who disappeared in Iran in 2007.
The secretary repeatedly wouldn't answer a direct question from Blitzer about whether the U.S. believes Levinson is alive, but said officials have not stopped pursuing Iranians' help in locating his trail.
"Bob Levinson is very, very much a part of our negotiation, very much a part of every conversation we have had with the Iranians," Kerry said. "We do not have evidence at this point as to whether he is. ... We are tracing every lead."
Kerry said he knows how difficult the process must be for Levinson's family, especially now seeing other Americans held in Iran coming home.
"We are trying to find out where he is and what the circumstances are," Kerry said. "We are proceeding as if he is (alive), we want him to be, we hope he is. We don't have capacity at this point to draw any kinds of conclusions, but we are working on it and the Iranians are cooperating."