Editor's note: Neil Johnson heads the complex systems research group at the University of Miami. He holds degrees from Cambridge and Harvard Universities and was Professor of Physics at Oxford University. The opinions expressed in this commentary are solely those of the author.
(CNN) -- If you have ever tried to grab a bargain that appears online, you'll know you have to be quick. The business of high frequency trading -- using algorithms and superfast computers to conduct trades in a fraction of a second -- is a supercharged version of this, with the potential to execute millions of buy and sell orders electronically each day through the myriad exchanges currently in existence.
Advocates argue that high frequency trading reduces market volatility and lowers transaction costs for small investors, but others claim it is unfair on slower traders, and can lead to instability -- trading algorithms and high frequency trading were behind the "Flash Crash" of May 6 2010, when the Dow Jones briefly plummeted almost 1,000 points.
Irrespective of how the popularity of high frequency trading changes in the future, this need for speed will continue to drive a technological arms race where the weapons of choice include new types of computer circuits hard-wired with dedicated trading algorithms; new tunnels blasted through mountains to ensure more direct optic fiber connections; new cables being laid in straighter paths across the Atlantic seabed; and even new networks of microwave towers to profit from the speed of electromagnetic signals through air.
And as the search for an evolutionary edge continues, "genetic" trading programs will be created that develop specific mutant offspring in response to real-time changes in market conditions. Even long-term pension funds play the high frequency game, hence it involves us all -- whether we like it or not.
There is one factor that limits speed: According to Einstein, nothing can travel faster than the speed of light. However this leaves ample opportunity for future increases in trading speed.
Light can travel approximately one foot in a nanosecond (a billionth of a second) in free space, meaning that current processers, routers and switching devices have plenty of room for improvement. And just as faster predators in the animal kingdom catch more prey and will be less vulnerable to other predators, faster trading companies will emerge at the expense of slower ones.
In parallel, a new form of ultrafast options market may emerge with second-scale contract times in order to hedge high frequency risk.
However, there are two problems that make the future of high frequency trading of unique global concern, irrespective of how popular it becomes.
The first is a scientific one: Financial markets represent the largest-ever sociotechnical system in existence, with a mix of state-of-the-art communications and computational power operating at speeds approaching the natural speed limit of light. Yet nobody, including Einstein, has ever produced a theory that predicts what might go wrong in an ultrafast global network of interconnected machines that carry out millions of operations in the blink of an eye -- or what can be done to prevent or manage it.
This leads to the second problem. How can regulators and governments possibly decide how to manage this emerging ultrafast financial jungle if nobody yet fully understands it?
My fellow researchers and I recently uncovered glimpses of what is already going wrong in the form of escalating patterns of "sub-second tsunamis." These tsunamis are huge spikes and dips in the price of an individual stock. Although the Flash Crash was fast, lasting only a few minutes, these sub-second tsunamis are over in the blink of an eye -- and there are thousands of them. A 10% daily change in a major stock would guarantee breaking news coverage, but these tsunamis typically send the price plummeting to almost zero. However they go unnoticed since the price quickly recovers as other algorithms jump in for the kill.
Their existence reveals a remarkable difference between the human trading world above the typical human response time of one second, and the all-machine ecology of trading algorithms below one second.
Just like cracks propagating in a structure prior to mechanical failure, these sub-second tsunamis escalated in the lead up to the 2008 financial meltdown. Most importantly, the stock showing highest proliferation are the banks that are now associated with the crisis. Yet nobody knew at the time. Indeed our research predicts a growing zoo of such tsunamis in the future, with each species having its own characteristic twist and turns.
Governments need the financial equivalent of an air traffic control system in order to know how to manage this brave new world, and hence what rules (if any) to impose. This in turn will require a joint research program between trading houses, regulators and academics.
However, instead of using conventional economics, the methodological approach should be built around complex adaptive systems and dynamical networks. Generative market models must be tested in real-time against high resolution data, to see if they can reproduce the observed price exchange dynamics down to the sub-second scale.
Our own research predicts that these sub-second market movements will be neither completely unpredictable nor predictable, but will instead have pockets of predictability that come and go in particular ways. Estimates of the market share of different trading algorithms will enable real-time system management, while ensuring that the secrecy of individual trading entities remains intact.
Without such a financial Manhattan Project, regulatory bodies will effectively be flying blind and may end up doing more harm than good.
The opinions expressed in this commentary are solely those of Neil Johnson.