Why all models are wrong

13 Dec, 2018 at 17:53 | Posted in Economics, Theory of Science & Methodology | 11 Comments

moModels share three common characteristics: First, they simplify, stripping away unnecessary details, abstracting from reality, or creating anew from whole cloth. Second, they formalize, making precise definitions. Models use mathematics, not words … Models create structures within which we can think logically … But the logic comes at a cost, which leads to their third characteristic: all models are wrong … Models are wrong because they simplify. They omit details. By considering many models, we can overcome the narrowing of rigor by crisscrossing the landscape of the possible.

To rely on a single  model is hubris. It invites disaster … We need many models to make sense of complex systems.

Yes indeed. To rely on a single mainstream economic theory and its models is hubris.  It certainly does invite disaster. To make sense of complex economic phenomena we need many theories and models. We need pluralism. Pluralism both in theories and methods.

Using ‘simplifying’ mathematical tractability assumptions — rational expectations, common knowledge, representative agents, linearity, additivity, ergodicity, etc — because otherwise they cannot ‘manipulate’ their models or come up with ‘rigorous ‘ and ‘precise’ predictions and explanations, does not exempt economists from having to justify their modelling choices. Being able to ‘manipulate’ things in models cannot per se be enough to warrant a methodological choice. If economists do not think their tractability assumptions make for good and realist models, it is certainly a just question to ask for clarification of the ultimate goal of the whole modelling endeavour.

The final court of appeal for models is not if we — once we have made our tractability assumptions — can ‘manipulate’ them, but the real world. And as long as no convincing justification is put forward for how the inferential bridging de facto is made, model building is little more than hand-waving that give us rather a little warrant for making inductive inferences from models to the real world.

Mainstream economists construct closed formalistic-mathematical theories and models for the purpose of being able to deliver purportedly rigorous deductions that may somehow by be exportable to the target system. By analyzing a few causal factors in their ‘laboratories’ they hope they can perform ‘thought experiments’ and observe how these factors operate on their own and without impediments or confounders.

Unfortunately, this is not so. The reason for this is that economic causes never act in a socio-economic vacuum. Causes have to be set in a contextual structure to be able to operate. This structure has to take some form or other, but instead of incorporating structures that are true to the target system, the settings made in mainstream economic models are rather based on formalistic mathematical tractability. In the models they often appear as unrealistic ‘tractability’ assumptions, usually playing a decisive role in getting the deductive machinery to deliver precise’ and ‘rigorous’ results. This, of course, makes exporting to real-world target systems problematic, since these models – as part of a deductivist covering-law tradition in economics – are thought to deliver general and far-reaching conclusions that are externally valid. But how can we be sure the lessons learned in these theories and models have external validity when based on highly specific unrealistic assumptions? As a rule, the more specific and concrete the structures, the less generalizable the results. Admitting that we in principle can move from (partial) falsehoods in theories and models to truth in real-world target systems do not take us very far unless a thorough explication of the relation between theory, model and the real world target system is made. To have a deductive warrant for things happening in a closed model is no guarantee for them being preserved when applied to an open real-world target system.

If the ultimate criteria for success of a deductivist system are to what extent it predicts and cohere with (parts of) reality, modern mainstream economics seems to be a hopeless misallocation of scientific resources. To focus scientific endeavours on proving things in models is a gross misapprehension of what an economic theory ought to be about. Real-world economic systems do not conform to the restricted closed-system structure the mainstream modelling strategy presupposes.

What is wrong with mainstream economics is not that it employs models per se. What is wrong is that it employs poor models. They — and the tractability assumptions on which they to a large extent build on — are poor because they do not bridge to the real world in which we live. And — as Page writes — “if a model cannot explain, predict, or help us reason, we must set it aside.”

11 Comments

  1. “What is wrong with mainstream economics is not that it employs models per se.”

    What is wrong with mainstream economics is that it employs models even when it is not appropriate to do so. Formal models almost always have the least interesting things to say about what we want to know.

  2. What is wrong with mainstream economics is that it confuses analytic modeling with operational modeling, the a priori study of what is logically possible with the a posteriori study of what actually is.
    .
    That moment when the affable teacher in Econ 101 lays out the theoretical model of perfect competition and then points out the window and says something to the effect that the market for wheat or some other commodity approaches perfect competition in some unspecified way because there are many farmers selling wheat — says in effect that perfect competition is descriptive — that is a moment of supreme idiocy from which economic thinking can never recover. A few moments later, laying out the model of monopoly, the same professor may intimate that “too few sellers” is the root of “monopoly power” and a bad thing: still more idiocy. That is where it all goes wrong.
    .
    Macroeconomics is the subdiscipline where this confusion is at its most pathologically intense. Analytic modeling can be useful and appropriate in a theoretical research program aimed at working out on a conceptual level the identification of the necessary and sufficient elements of a systematic relationship. Having worked out the logic of a system’s mechanisms is useful preparation for studying the actual institutions of the political economy, just as a study of geometry is useful preparation for the work of a surveyor or cartographer.
    .
    A student of analytic geometry does not imagine that doing geometry is making maps, even if doing geometry is necessary to making maps. Economists are not clear on this distinction. They seem to think that when they construct a DSGE model, some vague isomorphism between the model and stylized facts of the world, will mean something. It is the same Econ 101 craziness that thinks “perfect competition” corresponds with “many sellers” and economic power is in essence about one-seller “monopoly”.
    .
    If you are going to study the world, you have to look at the world. You do not use the world in a loose way to test your model; you build models to precisely test the world. Whether it is necessary for the model to be isomorphic to the world in order to the test the world, to see what the world in its being is like, to measure the world, that is an open question. Sometimes, perhaps.
    .
    What is clear is that the operational model we use to test the world has to be logical in its mechanics. The key presumption of the whole scientific enterprise is that the world is a logical, not an irrationally magical, place: that the world can be understood and measured as ordered by functional relations defining mechanisms and systems.
    .
    Unfortunately, we cannot discover the logic of the world merely by direct observation. The relations of cause and effect are not directly observable, or at least we can not sort out signal and noise without prior supposition about what the logic of that relation may be be. And, that is the other use of modeling: analytic models to work out a priori the logic of systems and mechanisms: aka, theoretical analysis. You need a geometry to do a survey to make a measured map. The student of geometry builds models of perfect circles and right-angle triangles, things that have no correspondence to nature; the cartographer’s map is also a model, but a model that has explicit, measured correspondence to the world.
    .
    Mainstream economics has an elaborate theory of how a system of markets could govern allocational efficiency under an assumption of complete information (or at least sufficient statistics). The actual economy, suffused with pervasive and radical uncertainty, is organized primarily by bureaucratic hierarchy and there are very few actual markets in operation and most of those are trading financial assets. Economists work very hard to avoid acknowledging the discrepancy or its implications for their ignorance of what is.

    • And financial firms work to eliminate pervasive uncertainty by using derivative hedges that pay off whether your bets win or lose … thus allocating the bulk of new money to themselves … but economists still don’t understand derivatives, and therefore ignore them.

    • “I have demonstrated in my fields that natural farming produces harvests
      comparable to those of modern scientific agriculture. If the results of a non-active agriculture are comparable to those of science, at a fraction of the investment in labour and resources, then where is the benefit of scientific technology?”
      .
      From Masanobu Fukuoka’s “The One Straw Revolution” http://library.uniteddiversity.coop/Permaculture/The_One_Straw_Revolution.pdf
      .
      You can think about the world magically and still produce.
      .
      (In response to your assertion: “The key presumption of the whole scientific enterprise is that the world is a logical, not an irrationally magical, place […]”)

  3. Economic models have a fourth characteristic: if the advice based on them is followed by influential enough people, they can become partially self-fulfilling. A model that could be sustained would be good. A model whose application would falsify it would not be so good, especially if it resulted in a crisis.

    I am not sure that there are any good economic models. Maybe what we need is not just better models but a better appreciation of the limitations of whatever models we use?

  4. May I ask a question? As economic models are pre-requisites for policy and this even on micro level, how is it possible in a changing world to evaluate a model? Could it be possible to look on economic models as social constructions and try them out in limited systems?

    • Keynes makes a distinction between ‘proper’ science and pseudo-science. In terms of the present discussion, the scientific method is to focus on what seem to be the most important factors, model them and then evaluate the impact of acting ‘as if’ the model were reliable. Pseudo-science omits this last stage, and – in my opinion – seems to replace it with a combination of denial of the impact and the spurious argument that they used sciency-looking methods to produce the models, so if the results aren’t right we need to double-down on their application.

      Science typically works through many models to get to one that is ‘good enough’ in some context for some purpose. Compared with economics their models may seem absolute, but ‘proper’ scientists are never absolutely certain about them.

      Your last sentence seems to me very much in line with my reading of Keynes, and also quite sensible. But I fear policy-makers themselves still need to understand the proper role of models, or else they will continue to get things wrong in a crisis.

  5. Keep it up Lars! Janne


Sorry, the comment form is closed at this time.

Blog at WordPress.com.
Entries and comments feeds.