(CNN) -- Secretary of Defense Robert Gates said Tuesday that North Korea is becoming a direct threat to the United States, asserting that the rogue Communist regime is within five years of developing intercontinental ballistic missiles.
"I think that North Korea will have developed an intercontinental ballistic missile within that time frame," Gates told reporters during a visit to China. But he said he has doubts that the North Koreans will be able to field many ICBMs. "I believe they will have a very limited capability," he said.
A spokesman for the Defense Intelligence Agency later supported Gates' remarks, saying, "North Korea's two recent attempts at 'space launches' indicate a continued trend toward development of ICBM capabilities. This trend of development, in addition to North Korea's stated goal of developing a nuclear warhead, supports the Secretary of Defense's recent statement regarding potential threats from North Korea."
If North Korea acts with urgency, it could have ICBMs in five years, said John Pike, founder of GlobalSecurity.org, a military analysis group.
Part of development is just a matter of trial and error.
"If you test enough times, you will eventually test out whatever fabrication and design flaws there are and you will have a workable missile," Pike told CNN.
North Korea's most recent test of it's longest-range ballistic missile, in April 2009, was a failure in that it did not put a satellite into space. But experts point out that it flew over Japan before crashing, farther than any other North Korean missile. The 2,000-mile flight proved North Korea is getting better at building long-range missiles.
The North Koreans are "pretty aggressive with its ballistic missile program," said one U.S. official. "It poses a "serious threat."
The Gates assessment reflects not new thinking but rather an intelligence estimate put out a decade ago, according to US officials. The 2001 National Intelligence Estimate reads "most Intelligence Community agencies project that before 2015 the United States most likely will face ICBM threats from North Korea."
The five-year estimate is "certainly within the realm of possibility," the U.S. official said Tuesday.
In China, Gates told reporters that North Korea's path, combined with the limited patience of the South Korean public over the North's provocations, has changed the status quo on the peninsula.
Gates cited last March's sinking of the South Korean warship Cheonan, in which 46 sailors died. Seoul and much of the international community blamed the North for the sinking, but Pyongyang denied involvement in the attack.
In November, the South's navy fired into disputed waters near Yeonpyeong Island and, in retaliation, the North shelled the island, killing four South Koreans.
"Clearly if there is another provocation, there will be pressure on the South Korean government to react," Gates said. "We consider this a situation of real concern and we think there is some urgency to proceeding down the track of negotiations and engagement."
But Gates said that U.S. patience with the North is measured and called for "concrete actions" by Pyongyang.
"We don't want to see the situation that we've seen so many times before, which is the North Koreans engage in a provocation and then everybody scrambles diplomatically to try and put Humpty Dumpty back together again," Gates said.
Gates applauded China -- North Korea's biggest trading partner -- for its moderating influence on tensions in the Korean peninsula. "They clearly have played a helpful role," he said.
What Gates did not discuss in his comments is if North Korea would be able to put a nuclear warhead on top of an ICBM capable of reaching the United States. The lighter the nuclear warhead, the farther the missile can fly and the greater its accuracy.
Bruce Klingner, a senior fellow at the Heritage Foundation and a fomer CIA analyst specilizing in North Korea, said the North Koreans have talked about making progress in building a nuclear ICBM warhead, but very few details have leaked out of the secretive country.
So what can the United States do to diminish the threat from North Korea? Gates said diplomacy hasn't worked, he wants action from Pyongyang.
"Rhetoric is not enough at this point. I think there need to be to concrete actions by the North to demonstrate that they're truly serious about negotiation and engagement at this point," Gates said. "They could have a moratorium on missile testing, a moratorium on nuclear testing. There are several areas where they could take concrete actions."
The U.S. Missile Defense Agency has nearly two dozen interceptor missiles deployed in Alaska and California to shoot down any missile coming towards the western United States. But even though those missiles are deployed, it's unclear is they would work if needed.
The two most recent tests of the interceptor missiles, one a year ago and one just one month ago, both failed to knock down the incoming target missile. Gen. James Cartwright, vice chairman of the Joint Chiefs of Staff, said that he was "not worried" about the test failures.
But Riki Ellison, founder of the Missile Defense Advocacy Alliance, a non-profit group that urges missile defense development, said last month, "If you're the American public, you've paid a lot for this system, you want to make sure that you are going to be protected. This doesn't give the confidence that you need."
The budget for America's missile defense system is about $10 billion.
Gates made his remarks Tuesday after meeting with Chinese President Hu Jintao and spoke on the same day that China made the first test flight of its J-20 stealth fighter.
Asked if he thought the flight may have been timed to coincide with his visit, Gates said Hu had told him "that the test had absolutely nothing to do with my visit and had been a pre-planned test."
Asked if he believed that, Gates said, "Coming from President Hu, yes."
CNN's Tom Watkins contributed to this report.