Tillerson was speaking in Tokyo, ahead of stops in Seoul and Beijing, where Chinese leaders are expected to present their own plan to defuse escalating tensions on the Korean peninsula.
"The diplomatic and other efforts of the past 20 years to bring North Korea to a point of denuclearization have failed. So we have 20 years of a failed approach," Tillerson said at a news conference with Japanese Foreign Minister Fumio Kishida.
China will formally present its own proposal to Tillerson on Saturday, a senior Chinese diplomat told CNN Thursday.
The Chinese plan calls for North Korea to suspend its ballistic missile tests and nuclear activities, and for the United States and South Korea to halt large-scale military exercises that annually antagonize Pyongyang.
"(North Korea) would like to see the Americans move first and take some actions first to show their sincerity, and vice versa," Xiao Qian, the director general of the Asian Affairs Department of the Ministry of Foreign Affairs, told CNN in an exclusive interview.
"So China's proposal is, why not do these things at the same time?"
This is the second time in as many weeks that China has floated this plan. A similar proposal by Foreign Minister Wang Yi
was rejected by the US State Department last week.
"There's no equivalence between North Korea's illegal missile and nuclear activities and what is our lawful, longstanding joint security exercises with our allies in the region," said Mark Toner, the acting State Department spokesman, in response to Wang's initial pitch.
The North Koreans don't seem to be buying it either. In a rare and hastily called press conference Thursday, Pak Myong Ho, the chargé d'affaires at the North Korean embassy in Beijing, also rebuffed Wang's proposal.
"Our stance is that since the US and its followers continue their nuclear threats and intimidation -- in the form of their annual military drills -- on the doorstep of DPRK (Democratic People's Republic of Korea), we will continue to strengthen our defense and preemptive strike capabilities," he said.
The new path?
Speaking side by side in Tokyo, Tillerson and Japan's Kishida reiterated their countries would stand united against North Korea's provocations.
While Kishida said Japan would assume a larger responsibility in the alliance, Tillerson said it was time for the US to stop throwing money at a solution that's not working.
"The United States provided $1.35 billion dollars in assistance to North Korea as an encouragement to take a different pathway. That encouragement has been met with further development of nuclear capabilities, more missile launches, including those of the recent February 11th and March 5th," he said.
Between 1995 and 2008, the US sent $1.35 billion to North Korea, mostly for food aid and energy assistance, according to a Congressional Research Service report
Analysts say when it comes to North Korea policy, there are few new options to try.
"Secretary Tillerson's comments on North Korea sound like a simplistic version of the past without a clear vision for the future," John Delury, a professor at Yonsei University's Graduate School of International Affairs told CNN.
"The history of US policy toward North Korea is complicated. It cannot be reduced to a talking point like -- we spent $1.3 billion and got nothing for it."
"A little knowledge is a dangerous thing," he added.
The US, South Korea, Japan, China and Russia have tried repeatedly to negotiate a solution with Pyongyang during Six-Party Talks, which began in 2003.
The sanctions currently levied against Pyongyang are among the most stringent ever enacted, but the pace of both North Korea's nuclear and missile programs appear to be speeding up.
'Face to face'
In his interview with CNN, Xiao addressed criticisms that China can and should be doing more unilaterally to stop North Korea.
The Trump administration has argued the China should use its vast economic leverage over North Korea to force the Kim Jong Un regime to cease its weapons development program.
"This is a rather complicated situation. You cannot solve the problem in a simple way," Xiao said.
"The DPRK has been saying all along that it is the threat from the US to make them choose other options, including nuclearization of their military capabilities. So it is, to a large extent, for the US and the DPRK to face to face, sit down and talk, and try to find a way to work out the problem," he said, using the abbreviation for North Korea's formal name -- the Democratic People's Republic of Korea.
Xiao also denied claims that Beijing has been carrying out systematic economic retaliation against South Korean businesses as a result of its opposition to the THAAD missile defense system that the US and South Korea believe is necessary to protect against North Korean missiles.
"But if they continue to move on with this deployment, there will be consequences -- and those consequence will be borne by the Americans and ROK (Republic of Korea)," he warned, without elaborating.