Friday, April 02, 2010

Predicting complex, dynamic systems or why Stalin, Hitler and stock market analysts have failed us

The ability to predict a future event lies at the heart of what it is to be human. A hunter predicting the presence of animals at a waterhole can drastically optimise the chances of a successful kill. Empathetically predicting a rival's or ally's actions can determine the outcome in hierarchical social competition. Predicting in advance the consequences of leaving a small fire unquenched can save the lives of your family. The need to predict the consequences of complex, dynamic systems has been a primary driver of human evolution, both biological and technical.

What does it mean to predict something? It is a simple question with a relatively straightforward answer but its misapplication can have profound and disturbing consequences. Mistaking the systems that can or cannot be usefully predicted is a core political and financial flaw of the last 100 years that has lead to spectacularly negative consequences, from the rise of totalitarianism, both right and left, to financial bubbles whose froth still fizzes through our financial institutions today.

Predicting an event is easy - all you need is a model. Simply ensure the model's initial state resembles reality and run it for as long as you need. When the model stops its final state is your prediction.

If making a "model whose initial state resembles reality" sounds a bit complicated then, fortunately for us, all humans come with some pretty good models right out of the box. As babies we all quickly boot up an hard-wired physics engine in our heads. As your hand collides with a glass of water and you see it tip over the edge of the table it is a matter of a few trillion, near instantaneous yet autonomous calculations for you to know, with crashing certainty, the fate of the glass. There may be the odd slightly surprising variation, the glass may not break, it may teeter then just recover from the edge of catastrophe, but as sure as apples it will not fall half way to the floor, stop and then float back to its original position. Philosophers may argue that you can't tell what the glass will do until you try but evolution makes the assumption that gravity is universal and so we have evolved an extremely powerful, ultra-fast physical modelling system that allows us to determine the likely environmental effect of a given cause. Without this internal modelling ability we would be sea anemones, which is perhaps a little cruel to anemones who, with their reflex withdrawal of their tentacles when touched are also making cause and effect assumptions with associated energy expenditure cost / benefit ratios all encoded in their simple neural nets. It is almost safe to say that if something is alive then it is in the business of modelling reality because, to remain alive, the future must, to some extent, be predicted.

Prediction is a fundamental property of life and so it is fortunate that making predictions for simple linear systems is really quite easy - the model must be able to run faster than reality. Imagine firing a cannon and then sitting down with pencil and paper to work out the distance the cannon ball will travel. If you know the equations (the model) and have practised the arithmetic beforehand then you could probably calculate the answer before the ball lands and so predict the future. If you have to first remember the equations from your school physics class, then labour over the arithmetic then the ball may land before you have finished your calculation and you may as well use a measuring tape to find the actual answer. What makes a dynamic prediction worthwhile is that it happens before the event itself and for that to occur the model must be able to run faster than reality it is modelling.

For a model to run faster than reality it must be different from reality. If you were to model a portion of reality with absolute fidelity then it would be running at real-time speed. Thus a predictive model must take short cuts, it must round numbers up or down, it must miss out extraneous details and ignore apparently unconnected variables. Thus a model is compressed, its is squeezed of its useless details to leave the essential, highly optimised set of relationships that describe a precise set of cause and effect relationships. The equations for the trajectory of a cannon ball work very well and give answers that are more than good enough yet they do not include any information on the type, weight or nutritional content of cannon manufacturer's favourite breakfast cereal. Not including this information saves valuable time and enables dynamic predictions about firing distances to be made before the actual event occurs.

This is a very seductive idea. It would seem that in order to predict something accurately then all that is required is a sufficiently accurate model. Certainly Stalin and Hitler thought so. Interestingly both Stalin and Hitler lived in Vienna at exactly the same time, just a few years before the outbreak of the first world war and whilst there they both wrote about walking in the same park and how they enjoyed watching the spectacle of Emperor Franz Josef I clattering past in his gilded carriage surrounded by his glittering horse guards. From this imperial demonstration, and many other life experiences, they both drew the same lessons concerning the absolute centralization of state power. They wanted to create a bureaucratic model of both a society and the economy upon which it depends then, using this model, a tiny elite could efficiently predict and so command the real society and economy thus satisfying the needs of the people.

What was behind this revolutionary notion and was it coincidence that the idea seemed to ripen at just that time? In those hopeful, pre-war days it seemed as though science had amply demonstrated that careful observation allowed for the creation of genuinely accurate models and these models then allowed for useful predictions and finally that those predictions allowed complicated systems, such as manufacturing processes, to be accurately and profitably controlled. Empires were forged on the back of this assumption and both Stalin and Hitler simply extended it to come to an identical conclusion, that with enough observations, card indexing and information, an entire economy and an entire society could be usefully modelled and so predicted and so controlled - scientifically - from the top down.

After leaving Vienna they both set about creating a compressed model of their societies. Vast armies of bureaucrats and technicians set about gathering and collating data to create a huge, hive-like interactive model of the economy and the society. The human cogs in the state calculating machine churned their way through mountains of detail. They gathered data from the secret police, the shop floor and factory gate to feed the model and so generate predictions for future consumption. These would then become production targets. The targets would be met or not, the differences observed and fed back into the model, future targets created - and so the whole system would hum along in a dynamic, cybernetic balance freeing up the workers to enjoy a völkisch idyll or a glorious dictatorship-of-the-proletariat depending on your taste in utopian fantasy.

Were the great political and economic modellers of the 20th Century a success? No, the goddess of the eternal court of history did not acquit them. Their models failed, their societies required repression to keep the elites in power and dystopia reigned - but why? The historically accurate answer is, as you would expect, complex and multi-stranded but at its heart lies the misapplication of predicative modelling, specifically a missing piece of the explanatory jigsaw, the notion of chaotic systems. It is not that the science of the early 20th Century got it fundamentally wrong. Modelling does allow predictions and predictions do enable the exertion of control however this process cannot be usefully applied to every complex system. There exists a certain type of complex, dynamic system that is not amenable to prediction and it is these systems that today we label as chaotic.

Consider a system that is not chaotic. Such as system can guarantee that for every observable cause there will be a equivalent observable effect and that cause and effect scale together in a linear fashion. These systems can certainly be modelled and therefore predicted. Firing a cannon ball at a certain velocity will cause it to travel a certain distance. The distance the ball travels scales well with the initial muzzle velocity, the faster it is travelling the further it will go.

Now imagine a cannon where this cause and effect linkage did not scale in a simple linear fashion. Firing the cannon with a little gunpowder causes the ball to dribble out of the muzzle (as expected), a grain or two more powder and the ball sails into the heavens (a shock), add a bucket load more and the ball lands just a few feet away (profoundly disturbing). In a non-linear system the results are not random but the primary cause, in this case the amount of gunpowder, cannot be used to usefully predict the primary effect, the distance travelled by the cannon ball. This is because non-linear systems are sensitive to much more than their primary cause. The results they produce are combinations of many subtle interactions that feedback into each other, amplifying some minuscule causes and attenuating other gross causes to create predictive confusion. In the case of the non-linear cannon it may be that the temperature of the barrel affects the trajectory and therefore, without realising it, the amount of gunpowder used in the previous firings is feeding back into the performance of the current firing causing complex results. Eliminate this variable and you discover that another subtle set of interactions takes its place, and another, and another. The system is perfectly rational and explicable its just that the amount of detail required to usefully model it is intractable.

Not all non-linear systems are chaotic, some may just be complex, and the recent rise of staggering computational power has allowed more of these complex systems to be usefully predicted. What defines a truly chaotic system is its extreme, perhaps infinite, sensitivity to any change in its state. Modelling such a system now becomes officially impossible because, as we have seen, all models are short cuts. A model must be compressed in order to be predictive and since it is compressed it is different from the reality it is modelling. Yet for a chaotic system any deviation from reality will cause an unpredictable outcome. Surely though, it is just a matter of the degree of precision? If the model is not working because it is imprecise then simply add more precision. This is the basic flaw ignored by all experts who purport to predict and control complex, dynamic systems such as economies and societies.

To give a fundamental example, a model cannot have the ability to represent a number to infinite precision, at some stage a model's numerical values must be rounded either up or down. Applied to a model of a chaotic system, this deviation from reality is deadly. The rounding error, no matter how infinitesimal, will be churned through internal feedback loops within the system and amplified and attenuated. It is like taking a nice round lump of pizza dough and drawing two spots on it side by side. Now stretch the dough (amplification) and then fold it back on itself (attenuation), stretch and fold. The dough never breaks, the system is never disjointed, but after a few cycles where are the two dots? Just about the only thing you can predict is that they are not still side by side.

Now imagine a number with two values after the decimal point, say 1.23 and multiply that number by ten and we get 12.3X. What is the new number X? Is it zero or some other random number? There is nothing special about zero, it is a just a guess at the true value and it has a 1 in ten chance of being correct. Now divide the number by 1000 to give 0.01 and them multiply is by 1000 to give 1.00 and now information is lost. This is the same as stretching and folding the dough.Positive and negative feedback loops in a model constrained by finite precision must lead to information generation and information destruction and since all models must enforce a finite precision then the stretching and folding of finite precision numbers must perturb a model's trajectory through the space of all possible results and cause it to veer away from reality.

It must be appreciated that upping the precision of a model cannot solve this problem. It simply does not matter how small the error is, the feedback loops will eventually inflate it and so cause the model to spiral away into an alternative, non-useful, reality. The more accurate you make the model the more useful time you can buy but always and eventually the model must deviate from reality simply because it is a finite model and not the infinitely detailed reality it pretends to represent - the map is not the terrain.

This then is the reason top-down, command economies always fail and why Stalin and Hitler are not revered today as our economic and social saviours. The reality of an economy or society is a web of interlinked interactions within which feedback loops, both positive and negative, nested and isolated, exists in rampant abundance. This means that models of such systems, with their inevitable short cuts, must ultimately fail in their predictions. A chaotic system cannot be controlled by commands from the top because the models required to generate the commands cannot be compressed without radically disconnecting their predictions from reality. This is the lesson that the totalitarians of the 20th Century have bequeathed to us. Have we taken it to heart?

The recent financial crash has shown that, at least in the realm of predicting the stock market, we have not. It is actually trivially simple to make money from the stock market, a sure-fire thing, and here is how you do it. Pick a global company with an instantly recognisable name whose shares are traded daily. Now send off 1024 spam emails, half predicting that the stock price will rise and half predicting it will fall. Wait a week and send out 512 mails to the people who received your previous correct prediction, as before half predict up, half predict down. Repeat the process for the 256 recipients who have now received two correct predictions, then again for the 128 who have now received three correct predictions, 64 for four correct predictions, 32 for five, 16 for six, 8 for seven, 4 for eight correct predictions. These last four will think either that you are a magician or that you have insider knowledge. They may even have already made themselves very rich, entirely legitimately, from your astonishing ability to predict the market. You now ask the last four to send you £1000 for your final prediction with the promise that, if you get it wrong, you will pay them back - the recipients simply cannot lose. You receive £4000 and send back £2000. You make £2000 and walk away. Rinse and repeat.

If that sounds far fetched then you have never worked in an investment bank. This is pretty much how they make their money. They simply employ enough traders and analysts so that some of them are right for long periods of time and they look like they can predict the system. There will always be a more than enough traders who are currently on a lucky run so that investors can be deluded into thinking they are magicians or, in our scientific times, have some esoteric, possibly algorithmic system that gives them the ability to predict the movements of the market. Of course they can't predict those movements no matter how often their predictions turn out to be correct. They can't predict the chaotic market for all the same reasons of finite precision models attempting to simulate an uncompressible, chaotic system we have just seen, yet they look like they are for the same reason the spam trick works.

The global economic system is the very definition of a chaotic, complex, dynamic system and to think that it can be modelled and so predicted is our 21st Century hubris that simply invites Nemesis to do her worst - and so she does - conjuring for our arrogance stock market analysts with their liquid promises just as she conjured, in the last century, Stalin and Hitler with their utopian visions.

In short - Beware the prophet bearing finite precision models.