Models and forecasts

8 January, 2016 at 17:01 | Posted in Economics | 5 Comments

Yesterday John Kay had an interesting article about models and forecasting in Financial Times:

A bane of this economist’s life is the belief that economics is clairvoyance. I should, according to this view, be offering prognostications of what gross domestic product growth will be this year and when the central bank will raise interest rates …

What was the right answer on January 1 1989 to the question “will the Berlin Wall be pulled down in 1989?” A shrewd commentator would have said (though few did) something like “almost certainly the Wall will stand but you should understand the potentially destructive forces undermining the Soviet engine and the East German state”. That type of response combines probabilistic and narrative thinking.

But people long for certainties, though they know they cannot have them. I have learnt that few really want answers when they ask me to predict GDP growth or advise whether interest rates will rise in the third quarter. It is usually easy to move the subject on to something more interesting than macroeconomic forecasting.

Kay’s remarks — and Tony Yates comments on them — made me think about an article that Oxford macroeconomist Simon Wren-Lewis wrote on models and forecasts a couple of years ago, saying that “macroeconomic forecasts are always bad,” but, on the other hand, since they are “probably no worse than intelligent guesses” and anyway are “not obviously harmful,” we have no reason to complain.

The thing is that Wren-Lewis is wrong. These forecasting models and the organizations and persons around them do cost society billions of pounds, euros and dollars every year. And if they do not produce anything better than “intelligent guesswork,” I’m afraid most taxpayers would say that they are certainly not harmless at all!

Mainstream neoclassical economists often maintain – usually referring to the methodological individualism of Milton Friedman – that it doesn’t matter if the assumptions of the models they use are realistic or not. What matters is if the predictions are right or not. But, if so, then the only conclusion we can make is – throw away the garbage! Because, oh dear, oh dear, how wrong they have been!

When Simon Potter a couple of years ago analyzed the predictions that the Federal Reserve Bank of New York did on the development of real GDP and unemployment for the years 2007-2010, it turned out that the predictions were wrong with respectively 5.9% and 4.4% – which is equivalent to 6 millions of unemployed. In other words — the “rigorous” and “precise” macroeconomic mathematical-statistical forecasting models were wrong. And the rest of us have to pay.

Potter is not the only one who lately has criticized the forecasting business. John Mingers comes to essentially the same conclusion when scrutinizing it from a somewhat more theoretical angle.

The empirical and theoretical evidence is clear. Predictions and forecasts are inherently difficult to make in a socio-economic domain where genuine uncertainty and unknown unknowns often rule the roost. The real processes that underly the time series that economists use to make their predictions and forecasts do not confirm with the assumptions made in the applied statistical and econometric models. Much less is a fortiori predictable than standardly — and uncritically — assumed. The forecasting models fail to a large extent because the kind of uncertainty that faces humans and societies actually makes the models strictly seen inapplicable. The future is inherently unknowable — and using statistics, econometrics, decision theory or game theory, does not in the least overcome this ontological fact. The economic future is not something that we normally can predict in advance. Better then to accept that as a rule “we simply do not know.”

So, to say that this counterproductive forecasting activity is harmless, simply isn’t true. Spending billions after billions of hard-earned money on an activity that is no better than “intelligent guesswork,” is doing harm to our economies.

In New York State, Section 899 of the Code of Criminal Procedure provides that persons “Pretending to Forecast the Future” shall be considered disorderly under subdivision 3, Section 901 of the Code and liable to a fine of $250 and/or six months in prison. Although the law does not apply to “ecclesiastical bodies acting in good faith and without fees,” I’m not sure where that leaves macroeconomic model-builders and other forecasters …

In an interesting discussion on the hopelessness of accurately modeling what will happen in the real world, Nobel laureate Kenneth Arrow – in Eminent Economists: Their Life Philosophies (CUP 1992) – pretty well sums up what the forecasting business is all about:

It is my view that most individuals underestimate the uncertainty of the world. This is almost as true of economists and other specialists as it is of the lay public. To me our knowledge of the way things work, in society or in nature, comes trailing clouds of vagueness … Experience during World War II as a weather forecaster added the news that the natural world as also unpredictable. cloudsAn incident illustrates both uncertainty and the unwillingness to entertain it. Some of my colleagues had the responsi-bility of preparing long-range weather forecasts, i.e., for the following month. The statisticians among us subjected these forecasts to verification and found they differed in no way from chance. The forecasters themselves were convinced and requested that the forecasts be discontinued. The reply read approximately like this: ‘The Commanding General is well aware that the forecasts are no good. However, he needs them for planning purposes.’

5 Comments »

RSS feed for comments on this post. TrackBack URI

  1. “These forecasting models and the organizations and persons around them do cost society billions of pounds, euros and dollars every year. And if they do not produce anything better than “intelligent guesswork,” I’m afraid most taxpayers would say that they are certainly not harmless at all!”

    If you are talking about that part of the operational budgets of government and financial institutions that directly relate to forecasting, if these are in the billions (doubtful), I think this is an issue in terms of the bigger scheme of things that can be ignored. It is a drop in the ocean in terms of the problems that have emanated from the practices of mainstream economics. Inefficient industries abound everywhere; the forecasting industry is just one of them.

    The operational costs of the forecasts are insignificant. But NOT the costs of the misforecasts (and methodological practice) themselves and this is where where economics as it is done has a massive social cost.The costs of producing the forecasts are insignificant. But relying on forecasts that are based on spurious methodology (ie using them as a basis for macro-policy), is potentially socially very costly.

    For example the mainstream profession did not pick up the factors that were leading to the 2008 financial crisis and I think the consequences of this mistake were a big part of the loss of credibility of the profession as a whole and why the austerity camp gained traction. Causes of this failure of the profession were things like ignoring economic history, irrational behaviour and unquantifiable evidence or phenomenon that does not fit in well with neo-classical models, assuming you can view a complex social system as the sum of individual units, and the importance of the financial sector. I think this is where criticism of the mainstream economics profession needs to be directed.

  2. ‘The Commanding General is well aware that the forecasts are no good. However, he needs them for planning purposes.’

    I meant to add, that the broad thrust of your argument – that the forecasting industry does real harm to economies – I am in complete agreement with🙂

    The question is though how do you plan given that we cannot predict the future? It could be the reason why many systems (including admittedly some very successful ones) go for a long term rule-based approach ??

  3. I have no problem with economic predictions and projections, just as long as they come with error estimates. Otherwise, it’s GIGO, isn’t it?

  4. Prof. Syll writes: “The economic future is not something that we normally can predict in advance. Better then to accept that as a rule “we simply do not know.” ”
    How on earth can a government decide fiscal or monetary policy with such impractical advice?
    Of course, the future can never be certain. All governments and individuals have to take risks that their best forecasts/expectations/hopes may be wrong.

    Nanikore comments: “relying on forecasts that are based on spurious methodology (ie using them as a basis for macro-policy), is potentially socially very costly. ”
    This is a truism. Should we base policies on tossing coins or divine inspiration instead of forecasts?
    Of course, forecasts are sometimes seriously wrong and the unexpected can happen. That’s a risk which cannot be avoided.
    Fortunately, in 2008-9, in the light of new information about the severity of the financial crisis, forecasts were radically revised . This prompted government fiscal and monetary measures which substantially mitigated impending economic disaster compared with what would have happened in the absence of revised forecasts.

  5. Transparency and honesty about the shaky foundations of forecasting would of course be great. The problem, however, is that not that many forecasters live up to those demands (“don’t bite the hand that feeds you”). And even if they did, you still have the problem that it is difficult to give error estimates of “unknown unknowns” and that lulling people into a false sense of security can be dangerous and costly to society (“financial markets with all their rational actors work just fine, so we don’t need any regulations,” etc., etc.).


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Blog at WordPress.com.
Entries and comments feeds.