Garbage-can econometrics

7 Jan, 2021 at 22:23 | Posted in Economics | 2 Comments

When no formal theory is available, as is often the case, then the analyst needs to justify statistical specifications by showing that they fit the data. That means more than just “running things.” It means careful graphical and crosstabular analysis …

garbageWhen I present this argument … one or more scholars say, “But shouldn’t I control for every-thing I can? If not, aren’t my regression coefficients biased due to excluded variables?” But this argument is not as persuasive as it may seem initially.

First of all, if what you are doing is mis-specified already, then adding or excluding other variables has no tendency to make things consistently better or worse. The excluded variable argument only works if you are sure your specification is precisely correct with all variables included. But no one can know that with more than a handful of explanatory variables. 

Still more importantly, big, mushy regression and probit equations seem to need a great many control variables precisely because they are jamming together all sorts of observations that do not belong together. Countries, wars, religious preferences, education levels, and other variables that change people’s coefficients are “controlled” with dummy variables that are completely inadequate to modeling their effects. The result is a long list of independent variables, a jumbled bag of nearly unrelated observations, and often, a hopelessly bad specification with meaningless (but statistically significant with several asterisks!) results.

Christopher H. Achen

This article is one of my absolute favourites. Why? Because it reaffirms yours truly’s view that since there is no absolutely certain knowledge at hand in social sciences — including economics — explicit argumentation and justification ought to play an extremely important role if purported knowledge claims are to be sustainably warranted. As Achen puts it — without careful supporting arguments, “just dropping variables into SPSS, STATA, S or R programs accomplishes nothing.”

2 Comments

  1. I’m still waiting for the econometric hierography that factors in how the human tool user problem manifests through perverse incentives and some executive ramrods risk analysts to push VaR way beyond its *assumed* predictive powers – reports of 6 months to over year. Just to get the – desired – rating and the rewards that come with it.

    .

    All compounded by how the xerox machine completely and irrevocably changed how the big 4 rating agencies business model went from one of income based on rigorous analysis of financial risk and how that gave buyers a reasonable valuation to
    suppliers of a product, to one of expecting a preferred valuation before handing over payment. All because some associate could print off a fraudulent certificate and pass it on for social networking reasons or some other form of exchange. Not too dissimilar of kids at the time using such technology to print fake ID’s or attempt to print money.

    .

    Lars your Mainstream economics finally made it post… imagery is so apropos …

  2. “explicit argumentation and justification ought to play an extremely important role if purported knowledge claims are to be sustainably warranted”
    .
    How is “explicit argument and justification” different from a story, given that “there is no absolutely certain knowledge at hand in social sciences”?
    .
    Isn’t the argument for “explicit argument and justification” just another story that you tell yourself, based on assumptions i.e. “conclusions without proof”?
    .
    Dropping data into SPSS might help you tell a story you never would have thought up on your own.
    .
    I can tell a story about drowning and ice cream consumption: people are too fat to swim due to ice cream and other junk food. Without bringing more data to the table, how do you know your story that ice cream and drownings both go up because of the temperature is better than mine? Aren’t you relying on simply a feeling to support the drowning and ice cream story? You read the example in a book, and it feels right, so you don’t need any data to justify your argument?


Sorry, the comment form is closed at this time.

Blog at WordPress.com.
Entries and comments feeds.