Kerry was negotiating with Iranian Foreign Minister Mohammad Javad Zarif through the weekend in an effort to reach an agreement by Tuesday.
The result of the talks this week -- the culmination of years of diplomacy -- could define relations between Iran and the West for years to come, as well as influencing the future of the Middle East.
"We have difficult issues to resolve," Kerry told reporters in Vienna on Sunday. He said that while negotiators have made progress, major points of contention remained.
"At this point negotiations could go either way," Kerry said. "If hard choices get made in the next couple of days and made quickly, we could get an agreement this week. But if they are not made, we will not."
The United States and its Western allies are trying to secure a deal that would prevent Iran from obtaining a nuclear weapon.
The Iranian government insists its nuclear program is for peaceful purposes only and is demanding relief from the punishing international sanctions that have crippled its economy.
The Obama administration is contending with criticism from skeptical Republicans and the Israeli government, which has fiercely campaigned against a potential deal. The final say in Iran, meanwhile, rests with the country's supreme leader, Ayatollah Ali Khamenei.
'All the cards are on the table'
Other foreign ministers from nations involved in the talks were expected to arrive in Vienna in hopes of completing a deal.
"All the cards are on the table," French Foreign Minister Laurent Fabius said after his arrival.
"These negotiations began 12 years ago, and now we are 72 hours away from the moment when they should conclude," he told reporters.
"The main question is to know whether the Iranians will accept making clear commitments on what until now has not been clarified," Fabius said.
Sign of progress over weekend
A hint of progress in the complex negotiations was reported over the weekend.
Experts reached a tentative agreement on the language dealing with sanctions relief, which has been a key sticking point in the talks, a source familiar with the talks told CNN on Saturday.
However, the source cautioned there was a lot of work to do, and this language still had to be presented to the foreign ministers of the countries involved in the talks.
A senior Obama administration official had previously told reporters that "even if and when issues get resolved at an experts level, there will remain some open issues that can only be decided by ministers."
Officials on both sides of the negotiating table have said they are hopeful they will be able to broker a final deal, but have emphasized in recent days that gaps remain over issues that could make or break the agreement.
'We want a good agreement, only a good agreement'
Kerry's team is under pressure domestically to conclude any deal by Thursday at the latest. If an agreement is not reached by then, a congressional review period for evaluating the deal doubles from 30 to 60 days.
But Kerry reiterated Sunday that the United States remains willing to walk away if Iran doesn't budge on key points.
"I've said from the moment I became involved in this we want a good agreement, only a good agreement, and we're not going to shave anywhere at the margins in order just to get an agreement," he said.
Negotiators already pushed the deadline back from the earlier limit of June 30.
If no deal is reached this week, the negotiators could keep going a few days longer, extend the talks by a few months or simply give up, pack up and go home. The sides have said that there won't be another formal extension to the negotiations.
Wrestling with contentious differences
Iran reached a preliminary agreement over its nuclear program in April after multiple rounds of negotiations with world powers: the United States, the UK, France, China, Russia and Germany. But negotiators must iron out the details and resolve contentious differences on critical technical issues, including inspections of nuclear sites.
Iran also needs to resolve outstanding issues with the UN's International Atomic Energy Agency, which is responsible for probing and inspecting Iran's nuclear program.
The agency is still trying to get Iran to hand over additional documents and information that would shed light on the past military aspects of Iran's nuclear program, which the country has insisted has always been peaceful.
After meeting with Iranian President Hassan Rouhani last week, IAEA chief Yukiya Amano said he believes "both sides have a better understanding on some ways forward," but added "more work will be needed."