Does it — really — take a model to beat a model? No!

9 Feb, 2021 at 16:57 | Posted in Economics | 3 Comments

Many economists respond to criticism by saying that ‘all models are wrong’ … But the observation that ‘all models are wrong’ requires qualification by the second part of George Box’s famous aphorism — ‘but some are useful’ … The relevant  criticism of models in macroeconomics and finance is not that they are ‘wrong’ but that they have not proved useful in macroeconomics and have proved misleading in finance.

kaykingWhen we provide such a critique, we often hear another mantra to which many economists subscribe: ‘It takes a model to beat a model.’ On the contrary, we believe that it takes facts and observations to beat a model … If a model fails to answer the problem to which it is addressed, it should be put back in the toolbox … It is not necessary to have an alternative tool available to know that the plumber who arrives armed only with a screwdriver is not the tradesman we need.

A similar critique yours truly sometimes encounters is that as long as I cannot come up with some own alternative model to the failing mainstream models, I shouldn’t expect people to pay attention.

This is, however, not only wrong for the reasons given by Kay and King, but is also to utterly misunderstand the role of philosophy and methodology of economics!

Clearing obstacles to science by clarifying limits and consequences of choosing specific modelling strategies, assumptions, and ontologies — that’s what philosophy and methodology can contribute to economics.

unnameadIt takes a model to beat a model has to be one of the stupider things, in a pretty crowded field, to come out of economics … I don’t get it. If a model is demonstrably wrong, that should surely be sufficient for rejection. I’m thinking of bridge engineers: ‘look I know they keep falling down but I’m gonna keep building them like this until you come up with a better way, OK?’

Jo Michell

Commenting on my critique of mainstream economics some colleagues argue that if we are going be taken seriously we also need to come up with a new and better theory. This is what Joan Robinson has to say on the issue:

It is often said that one theory can be driven out only by another; the neoclassicals have a complete theory (though I maintain it is nothing but a circular argument) and we need a better theory to supplant them. I do not agree. I think any other ‘complete theory’ would be only another box of tricks. What we need is a different habit of mind — to eschew fudging, to respect facts and to admit ignorance of what we do not know.


RSS feed for comments on this post. TrackBack URI

  1. It’s been a long time since I read it, but I remember that Thomas Balogh delivered a powerful critique of naive mathematical modelling in his 1982 book, “The Irrelevance of Conventional Economics”.

  2. Economists frequently, even usually confuse “model” and “theory”. It emphatically IS the case that it takes a theory to beat a theory. For “beating” is a stage in the development, the bettering of theory. If you have beaten a theory, you have improved it, have a new theory. Model? Perhaps not.

    How can one “respect facts” as Robinson suggests, without a theory? One cannot even state facts without a theory. So she isn’t really using the word “theory” in its basic meaning. Empiricism, positivism that fantasizes that it can do magic like consider facts without theories – is the most fertile source of fudging and bad habits of mind, of ignoring facts and logical consistency in the modern world. Nothing demonstrates this better than the Augean stable of economics, which makes a particular habit of this vice. (British) Empiricism turned into its opposite, Pretend-Empiricism long long ago.

    Mainstream economics is a pathogen that has acquired its resistance to truth by long ago internally incorporating defenses against such would-be cures as some of what Robinson says. Kay & King’s critique is better.

  3. Bridge engineers use a safety factor of two. In effect they are saying: take the prediction of my model and assume it is 100% wrong, and build for that load-tolerance.
    Financial options price modeling is affording me right now an opportunity to perfectly hedge my GME long. Why aren’t we modeling these hedges, instead of assuming that financial models just don’t work? What if you tested your theory that finance doesn’t work by trying a few hedged trades in a toy portfolio?

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

Blog at
Entries and comments feeds.