Econometrics and the problem of unjustified assumptions

4 Apr, 2021 at 19:11 | Posted in Statistics & Econometrics | 4 Comments

There seems to be a pervasive human aversion to uncertainty, and one way to reduce feelings of uncertainty is to invest faith in deduction as a sufficient guide to truth. Unfortunately, such faith is as logically unjustified as any religious creed, since a deduction produces certainty about the real world only when its assumptions about the real world are certain …

economUnfortunately, assumption uncertainty reduces the status of deductions and statistical computations to exercises in hypothetical reasoning – they provide best-case scenarios of what we could infer from specific data (which are assumed to have only specific, known problems). Even more unfortunate, however, is that this exercise is deceptive to the extent it ignores or misrepresents available information, and makes hidden assumptions that are unsupported by data …

Econometrics supplies dramatic cautionary examples in which complex modellin​g has failed miserably in important applications …

Sander Greenland

Yes, indeed, econometrics fails miserably over and over again.

One reason why it does, is that the error term in the regression models used is thought of as representing the effect of the variables that were omitted from the models. The error term is somehow thought to be a ‘cover-all’ term representing omitted content in the model and necessary to include to ‘save’ the assumed deterministic relation between the other random variables included in the model. Error terms are usually assumed to be orthogonal (uncorrelated) to the explanatory variables. But since they are unobservable, they are also impossible to empirically test. And without justification of the orthogonality assumption, there is, as a rule, nothing to ensure identifiability:

Distributional assumptions about error terms are a good place to bury things because hardly anyone pays attention to them. Moreover, if a critic does see that this is the identifying assumption, how can she win an argument about the true expected value the level of aether? If the author can make up an imaginary variable, “because I say so” seems like a pretty convincing answer to any question about its properties.

Paul Romer

Nowadays it has almost become a self-evident truism among economists that you cannot expect people to take your arguments seriously unless they are based on or backed up by advanced econometric modelling​. So legions of mathematical-statistical theorems are proved — and heaps of fiction are being produced, masquerading as science. The rigour​ of the econometric modelling and the far-reaching assumptions they are built on is frequently not supported by data.

Econometrics is basically a deductive method. Given the assumptions, it delivers deductive inferences. The problem, of course, is that we almost never know when the assumptions are right. Conclusions can only be as certain as their premises — and that also applies to econometrics.

Econometrics doesn’t establish the truth value of facts. Never has. Never will.


RSS feed for comments on this post. TrackBack URI

  1. Yes, the fruits of econometrics have been very disappointing. However, this failure doesn’t change the basic character of econometrics.

    Prof. Syll uses very confusing language in claiming that “Econometrics is basically a DEDUCTIVE method. Given the assumptions, it delivers DEDUCTIVE INFERENCES.” (my capitalisation).

    According tho Haavelmo -“The Probability Approach in Econometrics” , Econometrica 1944:
    “The method of econometric research aims, essentially, at a conjunction of economic theory and actual measurements, using the theory and technique of statistical inference as a bridge pier.”

    Thus econometrics is essentially an INDUCTIVE method.
    Given the available data (and minimum assumptions) it tries to make probabilistic EMPIRICAL ESTIMATES of economic relationships using methods of STATISTICAL INFERENCE.

  2. “There seems to be a pervasive human aversion to uncertainty, and one way to reduce feelings of uncertainty is to invest faith in deduction as a sufficient guide to truth. ”

    Giving precise numbers to things also proves irresistible to journalists and politicians, as it negates the need to explain complex issues which are unquantifiable and require nuance that can only be realistically attempted in written language.

    A good example of this was the quantification of migration numbers into Britain after the expansion of the European Union, and the quantification of the estimated cost by the BOE and Treasury, to the last pound, of Brexit before the Referendum.These estimates made newspaper headlines, but in fact only served to raise scepticism towards ‘experts’. The serious content of the arguments was undermined.

    The political consequences of this fetish with quantification has had disastrous political consequences. It is difficult enough to try and estimate the effects of non-tariff barriers such as customs space and procedural time now needed for traders – prohibitively expensive, especially for small businesses. But even more than that – to explain the consequences of Brexit also requires an understanding of geopolitics and security issues – and even these have potentially serious economic consequences. Quantifying such things should be considered a non-starter.

    In the end this has all played to Johnson and right wing nationalists.

    And still, nothing has been learned.

  3. Why doesn’t this criticism apply to ergodicity and MMT?
    Erogodicists assume “when our assets are gone, they’re gone” despite bankruptcy forgiveness and ample evidence of bailouts on a huge scale. MMT assumes inflation is a price signal of real scarcity despite clear real-world examples of markets not clearing for purely psychological reasons.
    Why are ergodic and MMT deductions exempt and certain, whereas other models must be uncertain?
    Also: finance has innovated volatility products. You can put your money where your mouth is and make bets on uncertainty using VXX, UVXY, SVXY, XPS, SPX options, etc. You should be able to profit from uncertainty by expressing a view about its pervasiveness using innovative financial products. Profit and loss is how you export model claims to the real world in finance.
    billcinsd said in another blog comment that not everyone shares my obsession with financialization. But I’m not obsessed, just observant …

    • When was the last time you made a comment that wasn’t related to financialization? I must admit I generally picture you as the guy in Animotion

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

Blog at
Entries and Comments feeds.