The main reason why almost all econometric models are wrong

27 Jun, 2018 at 11:59 | Posted in Statistics & Econometrics | 3 Comments

Since econometrics doesn’t content itself with only making optimal predictions, but also aspires to explain things in terms of causes and effects, econometricians need loads of assumptions — most important of these are additivity and linearity. Important, simply because if they are not true, your model is invalid and descriptively incorrect.  And when the model is wrong — well, then it’s wrong.

The assumption of additivity and linearity means that the outcome variable is, in reality, linearly related to any predictors … and that if you have several predictors then their combined effect is best described by adding their effects together …

catdogThis assumption is the most important because if it is not true then even if all other assumptions are met, your model is invalid because you have described it incorrectly. It’s a bit like calling your pet cat a dog: you can try to get it to go in a kennel, or to fetch sticks, or to sit when you tell it to, but don’t be surprised when its behaviour isn’t what you expect because even though you’ve called it a dog, it is in fact a cat. Similarly, if you have described your statistical model inaccurately it won’t behave itself and there’s no point in interpreting its parameter estimates or worrying about significance tests of confidence intervals: the model is wrong.

Andy Field

Let me take the opportunity to elaborate a little on why I find these assumptions of such paramount importance and ought to be much more argued for — on both epistemological and ontological grounds — if at all being used.

Limiting model assumptions in economic science always have to be closely examined since if we are going to be able to show that the mechanisms or causes that we isolate and handle in our models are stable in the sense that they do not change when we “export” them to our “target systems”, we have to be able  to show that they do not only hold under ceteris paribus conditions and a fortiori only are of limited value to our understanding, explanations or predictions of real economic systems.

Our admiration for technical virtuosity should not blind us to the fact that we have to have a cautious attitude towards probabilistic inferences in economic contexts. We should look out for causal relations, but econometrics can never be more than a starting point in that endeavour since econometric (statistical) explanations are not explanations in terms of mechanisms, powers, capacities or causes. Firmly stuck in an empiricist tradition, econometrics is only concerned with the measurable aspects of reality. But there is always the possibility that there are other variables – of vital importance and although perhaps unobservable and non-additive, not necessarily epistemologically inaccessible – that were not considered for the model. Those who were can hence never be guaranteed to be more than potential causes, and not real causes. A rigorous application of econometric methods in economics really presupposes that the phenomena of our real world economies are ruled by stable causal relations between variables. A perusal of the leading econom(etr)ic journals shows that most econometricians still concentrate on fixed parameter models and that parameter-values estimated in specific spatio-temporal contexts are presupposed to be exportable to totally different contexts. To warrant this assumption one, however, has to convincingly establish that the targeted acting causes are stable and invariant so that they maintain their parametric status after the bridging. The endemic lack of predictive success of the econometric project indicates that this hope of finding fixed parameters is a hope for which there really is no other ground than hope itself.

Real-world social systems are not governed by stable causal mechanisms or capacities. The kinds of “laws” and relations that econometrics has established, are laws and relations about entities in models that presuppose causal mechanisms being atomistic and additive. When causal mechanisms operate in real-world systems they only do it in ever-changing and unstable combinations where the whole is more than a mechanical sum of parts. If economic regularities obtain they do it (as a rule) only because we engineered them for that purpose. Outside man-made ‘nomological machines’ they are rare, or even non-existent. Unfortunately, that also makes most of the achievements of econometrics – as most of the contemporary endeavours of mainstream economic theoretical modelling – rather useless. No matter how often you call your pet cat a dog, it is still nothing but a cat …

3 Comments

  1. One the builders of the Econometric society in London,Gunnar Myrdal already in 1928 i think it was said later “I have no illusions that it will ever be possible to fit a
    general theory into a neat econometric model. The relevant variables and the
    relevant relations between them are too many to permit that sort of heroic
    simplification. This does not mean, however, that particular problems could not
    with advantage be treated in this way – provided that the variables and
    assumptions were selected on the basis of such insight into essential facts and
    relations as only a general theory can furnish”
    Gunnar Myrdal, (1957), Economic Theory and Underdeveloped Regions, London: University Paperbacks, Methuen.

    Later on, according to Myrdal there was a proliferation of terminological
    and mathematical innovation, which presented general equilibrium and welfare theories in a new form, that is through a new analytical language. In spite of this kind of escapism, however, this body of neoclassical economic thought still
    incorporates as he wrote:
    “One version or another of the old, discredited rationalistic psychology and
    utilitarian moral philosophy. By implying them – as practical conclusions make
    evident that it does – it becomes unfounded and false”
    Gunnar Myrdal. (1970), Objectivity in Social Research, London: Duckworth

    • Great Myrdal quote 🙂

  2. To take the cat & dog metaphor one step further, assumptions are often based upon observables. Both critters have tails, 4 paws, and piss on the carpet. Combined, that is a lot of piss on the carpet. However, if what one is trying to predict is barking behavior or tail wagging, combining the two additively has little effect on valid predictions as dogs a more prolific waggers and cats cannot bark. Then if the modeler assumes a linear relationship between barking and wagging, the more cats in the equation, the less linear the results appear – even though catless, the relationship would be quite linear.


Sorry, the comment form is closed at this time.

Blog at WordPress.com.
Entries and comments feeds.