Why technical fixes will not rescue econometrics

27 Sep, 2021 at 21:35 | Posted in Statistics & Econometrics | 1 Comment

On the issue of the various shortcomings of regression analysis and econometrics, no one sums it up better than David Freedman in his Statistical Models and Causal Inference:

David A. Freedman - WikipediaIn my view, regression models are not a particularly good way of doing empirical work in the social sciences today, because the technique depends on knowledge that we do not have. Investigators who use the technique are not paying adequate attention to the connection — if any — between the models and the phenomena they are studying. Their conclusions may be valid for the computer code they have created, but the claims are hard to transfer from that microcosm to the larger world …

Regression models often seem to be used to compensate for problems in measurement, data collection, and study design. By the time the models are deployed, the scientific position is nearly hopeless. Reliance on models in such cases is Panglossian …

Given the limits to present knowledge, I doubt that models can be rescued by technical fixes. Arguments about the theoretical merit of regression or the asymptotic behavior of specification tests for picking one version of a model over another seem like the arguments about how to build desalination plants with cold fusion and the energy source. The concept may be admirable, the technical details may be fascinating, but thirsty people should look elsewhere …

Causal inference from observational data presents may difficulties, especially when underlying mechanisms are poorly understood. There is a natural desire to substitute intellectual capital for labor, and an equally natural preference for system and rigor over methods that seem more haphazard. These are possible explanations for the current popularity of statistical models.

Indeed, far-reaching claims have been made for the superiority of a quantitative template that depends on modeling — by those who manage to ignore the far-reaching assumptions behind the models. However, the assumptions often turn out to be unsupported by the data. If so, the rigor of advanced quantitative methods is a matter of appearance rather than substance.

1 Comment

  1. 》there is a strong tendency in looking at data to assume that when two events frequently happen together, one causes the other. There is an even stronger tendency to assume that the one that occurs first causes the one that occurs second. These tendencies are easy to resist in the simplest cases. But they seem to creep back in when econometric studies become more complex. Sometimes I wonder if we can draw any conclusions at all from the results of regression studies.
    Fischer Black, “Noise”, 1986
    (By the way, doesn’t Rudd’s critique (in the previous blog entry) of the very limited inflation expectation theory espoused by Friedman and Phelps try to use econometric regression models to disprove other economic regression models?)

Sorry, the comment form is closed at this time.

Blog at WordPress.com.
Entries and Comments feeds.