The conundrum of unknown unknowns

24 Jul, 2017 at 17:18 | Posted in Economics | 2 Comments

Short-term weather forecasting is possible because most of the factors that determine tomorrow’s weather are, in a sense, already there … But when you look further ahead you encounter the intractable problem that, in non-linear systems, small changes in initial conditions can lead to cumulatively larger and larger changes in outcomes over time. In these circumstances imperfect knowledge may be no more useful than no knowledge at all.

economic_forecastingMuch the same is true in economics and business. What gross domestic product will be tomorrow is, like tomorrow’s rain or the 1987 hurricane, more or less already there: tomorrow’s output is already in production, tomorrow’s sales are already on the shelves, tomorrow’s business appointments already made. Big data will help us analyse this. We will know more accurately and more quickly what GDP is, we will be more successful in predicting output in the next quarter, and our estimates will be subject to fewer revisions …

Big data can help us understand the past and the present but it can help us understand the future only to the extent that the future is, in some relevant way, contained in the present. That requires a constancy of underlying structure that is true of some physical processes but can never be true of a world that contains Hitler and Napoleon, Henry Ford and Steve Jobs; a world in which important decisions or discoveries are made by processes that are inherently unpredictable and not susceptible to quantitative description.

John Kay

The central problem with the present ‘machine learning’ and ‘big data’ hype is that so many — falsely — think that they can get away with analysing real world phenomena without any (commitment to) theory. But — data never speaks for itself. Without a prior statistical set-up there actually are no data at all to process. And — using a machine learning algorithm will only produce what you are looking for.

Theory matters.


  1. I don’t know much about the weather, but, yes, the economy cannot be accurately predicted very far in advance, for the simple and obvious reason that the future must in large part be the outcome of decisions not yet taken in the present. If we could predict the future, we would in effect be making those decisions now.
    Even if we could make those decisions now we shouldn’t want to do so, if knowledge to better make those decisions will be accumulated in the experience of the interim.
    This is a sadly underappreciated aspect of uncertainty: we learn. We do not know now what we will learn and know later.

  2. When you want to understand what caused WWII, a historian does not start with a theory of how wars happen. The theory comes after he has, as comprehensively has possible, examined the (quantitative and non-quantitative) primary and other evidence. The theory is developed AFTER he has done this. If not, we have the parable of the drunkard and lamp post problem. Macro-economics post Samuelson has for many decades religiously submitted to theory.

Sorry, the comment form is closed at this time.

Blog at
Entries and Comments feeds.