Friday, April 02, 2010

Predicting complex, dynamic systems or why Stalin, Hitler and stock market analysts have failed us

The ability to predict a future event lies at the heart of what it is to be human. A hunter predicting the presence of animals at a waterhole can drastically optimise the chances of a successful kill. Empathetically predicting a rival's or ally's actions can determine the outcome in hierarchical social competition. Predicting in advance the consequences of leaving a small fire unquenched can save the lives of your family. The need to predict the consequences of complex, dynamic systems has been a primary driver of human evolution, both biological and technical.

What does it mean to predict something? It is a simple question with a relatively straightforward answer but its misapplication can have profound and disturbing consequences. Mistaking the systems that can or cannot be usefully predicted is a core political and financial flaw of the last 100 years that has lead to spectacularly negative consequences, from the rise of totalitarianism, both right and left, to financial bubbles whose froth still fizzes through our financial institutions today.