Heckman on where causality resides

17 Jul, 2022 at 13:53 | Posted in Statistics & Econometrics | 11 Comments

James HeckmanI make two main points that are firmly anchored in the econometric tradition. The first is that causality is a property of a model of hypotheticals. A fully articulated model of the phenomena being studied precisely defines hypothetical or counterfactual states. A definition of causality drops out of a fully articulated model as an automatic by-product. A model is a set of possible counterfactual worlds constructed under some rules. The rules may be the laws of physics, the consequences of utility maximization, or the rules governing social interactions, to take only three of many possible examples. A model is in the mind. As a consequence, causality is in the mind.

James Heckman

So, according to this ‘Nobel prize’ winning econometrician, “causality is in the mind.” But is that a tenable view? Yours truly thinks not. If one as an economist or social scientist would subscribe to that view there would be pretty little reason to be interested in questions of causality at all.  And it sure doesn’t suffice just to say that all science is predicated on assumptions. To most of us, models are seen as ‘vehicles’ or ‘instruments’ by which we represent causal processes and structures that exist and operate in the real world. As we all know, models often do not succeed in representing or explaining these processes and structures, but if we didn’t consider them as anything but figments of our minds, well then maybe we ought to reconsider why we should be in the science business at all …

The world as we know it has limited scope for certainty and perfect knowledge. Its intrinsic and almost unlimited complexity and the interrelatedness of its parts prevent the possibility of treating it as constituted by atoms with discretely distinct, separable and stable causal relations. Our knowledge accordingly has to be of a rather fallible kind. To search for deductive precision and rigour in such a world is self-defeating. The only way to defend such an endeavour is to restrict oneself to prove things in closed model worlds. Why we should care about these and not ask questions of relevance is hard to see. As scientists, we have to get our priorities right. Ontological under-labouring has to precede epistemology.

The value of getting at precise and rigorous conclusions about causality based on ‘tractability’ conditions that are seldom met in real life, is difficult to assess. Testing and constructing models is one thing, but we do also need guidelines for how to evaluate in which situations and contexts they are applicable. Formalism may help us a bit down the road, but we have to make sure it somehow also fits the world if it is going to be really helpful in navigating that world. In all of science, conclusions are never more certain than the assumptions on which they are founded. But most epistemically convenient methods and models that work in ‘well-behaved’ systems do not come with warrants that they will work in other (real-world) contexts.

11 Comments

  1. If you really do engineering, do you find yourself using arbitrary, often inconsistent heuristics? If you engineer a tide power generator for Nome, Alaska can you rely on tide predictions that are often off by an hour and a half? (Search for “Tide Prediction Error for the United States Stations”, then if you seperately look up today’s high tide level prediction for Nome, Ak, will you see the error margin is close to 100%? (High tide height is predicted to be 1.43 feet at time of writing, but the average difference between observed and predicted tide heights is 1.18 feet so the actual high tide might be 2.61 feet, or 0.25 feet, on average?) So, is the inductive tide model’s prediction basically noise, for this particular station at least (cherry-picked to demonstrate the non-ergodicity of tides)?
    .
    If you engineer a bridge, do you multiply the spec by a safety factor of two, which can cover up a lot of model flaws?
    .
    When you convert sighted tree diameter measurements to actual diameter, do you have an arbitrary heuristic that says you can throw away half the predictions (i.e., the negative solutions) of the Biltmore formula math model?
    .
    If economists are trying to engineer affordable dentistry, might they start by seeing prices as arbitrary to within (most of the time) a factor of two, as Fischer Black wrote in “Noise”? (Notice how the factor of two appears in the other examples, too?)
    .
    Should the poverty level be multiplied by a safety factor of two, as bridge engineers do?

    Comment by rsm— 21 Jul, 2022 #

    • @rsm
      Only an idiot would suggest “a tide power generator” at a location with very small tides like Nome, Alaska.
      .
      Your interpretation of the tide data for Nome, Alaska is nonsense.
      You claim that the data shows that tides at Nome are “basically noise”, ie occurring unpredictably at random times with random heights.
      However, like the predicted tides, the actual data clearly shows that Nome AK has very small tides (average high water about 2 feet) with a regular twice daily pattern.
      The RMS (standard deviation) of high water prediction errors is only 1.18 feet.
      The RMS of prediction errors for the time of high water is about 90 minutes.
      .
      NB. The corresponding RMS for other locations are considerably smaller than for Nome.
      .
      Alfred Marshall suggested that economics was similar in kind to the “law if tides”, but he did not claim that economics could achieve similar accuracy.

      Comment by Kingsley Lewis— 23 Jul, 2022 #

      • Are either of you suggesting markets function on some normative time line in nature which has no agency – ????? – boggles the mind.

        Comment by Skippy— 24 Jul, 2022 #

    • Kingsley, Wikipedia’s tidal power page says: “Tides are more predictable than the wind and the sun.” How can this be true, when the following account of a death on account of mispredicted tidal flow near Anchorage, AK (with a much higher tide than Nome, and with lower error margins) seems to suggest wind can overpower all tidal model inputs? From an article easily searchable on the Anchorage Daily News website:
      .
      《It was about two hours after low tide on June 23 when a 911 call was placed reporting two men swimming for their lives in a tidal river off Kincaid […] “Because of the numerous uncertain and, in some cases, completely unknown factors of local control mentioned above,” NOAA notes on its website, “it is not feasible to predict tides purely from a knowledge of the positions and movements of the moon and sun obtained from astronomical tables.”
      .
      Was the inductive model cold comfort to the guy who died as Russell’s wide-eyed, Pollyanna-like turkey, the one trusting faithfully in tide table predictions, the other in people’s goodness? How useful were inductive tide model predictions for Fukushima’s engineers? When doing engineering, can one afford to assume Marshall’s “Ceteris Paribus” beliefs, or do you use (possibly irrational) heuristics to plug model failures?
      .
      Skippy: is Fischer Black, in “Noise”, on the same page with your citations of Lakatos?
      .
      《In the end, a theory is accepted not because it is confirmed by conventional empirical tests, but because researchers persuade one another that the theory is correct and relevant.》
      .
      May I note further that reading the page on Lakatos at Stanford Encyclopedia of Philosophy, I came upon an argument I have used myself? “if stellar parallax is not observed, we can try to refute this apparent refutation by refining our instruments and making more careful measurements and observations.” What if the Greeks had used that philosophy with Aristarchus’s 3rd century BC heliocentric theory?
      .
      Also, here is a Lakatos quotation from SEP’s ceteris paribus page (which also quotes Marshall defending CP):
      .
      《Some scientific theories forbid an event occurring […] only on the condition that no other factor […] has any influence on it. […] Another way of putting this is to say that some scientific theories are normally interpreted as containing a ceteris paribus clause (Lakatos 1970, 101).》
      .
      But does nature forbid it, and do engineers have to build for nature?

      Comment by rsm— 25 Jul, 2022 #

  2. As always a pleasure to read your comments, Sander!
    Re Heckman’s claim that causality is “in the mind,” it would if that was true, be difficult to understand why people (including most scientists and philosophers) usually are so interested in questions about causality. In economics, there are plenty of models that have very little to do with the real-world system we live in, and I guess that is one reason many non-economists find it so non-enlightening and non-ampliative to work through those models. If we can’t seriously believe that the models used are substantively related to causal relations existing in the real world — and not only “in our minds” — there certainly are better ways to allocate our time and intellectual endeavours.

    Comment by Lars Syll— 18 Jul, 2022 #

    • Yes indeed, we agree. Our skeptical observations do however raise tough questions like: What are those better ways to allocate our very constrained mental and analytic resources? As I see it, that involves demoting but not discarding the toy models that currently serve as the foundation of semi-formal statistical “inference”. Those models do serve two important functions in fields like ours: (1) as teaching tools, providing lessons in how our naive intuitions fail even in simple systems and what regularities emerge in those systems – much like basic physics labs display phenomena like refraction and diffraction; (2) as information summaries or filters for data sets too large to visualize adequately, as with almost all real data sets today.

      As some predicted, those models have proven destructive because of methods that treat model forms as if laws of nature and treat modeling outputs as if those were truly sufficient statistics or facts of nature – much like aerodynamic modeling has killed test pilots when the modeling failed to account for all major forces in actual flight. The problem could then be framed as statistical theorists taking as their idol theoretical physics rather than engineering (I see that mistake reflected in the focus of foundational discussions on figures like Neyman to the neglect of figures like Tukey). A lesson I see is that our fields should be treated more like engineering than theoretical physics, and the damage done to them by statistical theory is akin to the harms caused when modeling is substituted for observations and experiments. Perhaps engineering science holds lessons on how to exploit the wealth of available mathematical tools without allowing the theory behind the tools to overwhelm and become confused with the actual empirical content of the field.

      Comment by lesdomes— 18 Jul, 2022 #

      • The engineering analogy is not a bad one, Sander 🙂
        Or maybe we should follow the advice of a famous British economist​, J. M. Keynes, who once said: “If only economists could manage to get themselves thought of as humble, competent people, on a level with dentists, that would be splendid!”

        Comment by Lars Syll— 19 Jul, 2022 #

        • I found this instructive.

          .

          Lakatos provides us with an answer. Research programs are not refuted, as in Popper, nor are they dramatically overturned, as in Kuhn. They simply attract more new adherents than their competitors. In the language of Lakatos, research programs are progressive or degenerative.

          .

          Knew someone about 10 years that stated they were a old Keynesian/Tobin sort and not a fortnight ago started authoritatively using the word empiricism and then when asked how referred Neo-Paleo Keynesianism.

          .

          So just more “neoclassical synthesis” self-correcting with some short and long term views mangled together with a side of practitioner bias. All I know is Keynes would burn it all with fire …

          Comment by Skippy— 20 Jul, 2022 #

        • The analogy with engineering is indeed apt.
          However this analogy has been emphasised by mainstream economists for well over a century.
          Perhaps philosophers and epidemiologists should read or re-read the first four chapters of Alfred Marshall’s “Principles of economics”:

          “The laws of human action are not indeed as simple, as definite or as clearly ascertainable as the law of gravitation; but many of them may rank with the laws of those natural sciences which deal with complex subject-matter. “
          “The laws of economics are to be compared with the laws of the tides, rather than with the simple and exact law of gravitation. For the actions of men are so various and uncertain, that the best statement of tendencies, which we can make in a science of human conduct, must needs be inexact and faulty.
          …The harder the task, the greater the need for steady patient inquiry; for turning to account the experience, that has been reaped by the more advanced physical sciences; and for framing as best we can well thought-out estimates, or provisional laws, of the tendencies of human action.”
          Marshall: “Principles of economics” 8e 1920, Book 1 chapter 3

          Comment by Kingsley Lewis— 20 Jul, 2022 #

          • Laws of human action … right back to praxoelogy ….

            Comment by Skippy— 20 Jul, 2022 #

  3. Lars: Your comment is beautifully put and captures my own sentiments (and of course David Freedman’s) well. Just a minor point: Your “In all of science, conclusions are never more certain than the assumptions on which they are founded.” True, in strict deductive logics about uncertainty, conclusions (deductions) cannot be more certain than the set of assumptions used in their derivation. But, under the same starting assumptions, scientist like all humans give far more certainty to deductions about the real world they like than to those they don’t like. We might dislike those biases for deceiving us about reality. Nonetheless, we humans constantly generate new assertions imbued with far more certainty than any deductive justification from the assumptions we can explicate; and that seems to be an essential part of the process by which we learn. The question is then how this kind of ampliative inference can be disciplined to provide more accurate conclusions. This question seems to me in the scope of statistical epistemology, which is an unsettled topic to say the least. However we approach the question, the deep controversies in statistics warns us to be as wary of our models of reasoning as we should be of our models of the phenomena we study as “scientists”.

    Comment by lesdomes— 18 Jul, 2022 #


Sorry, the comment form is closed at this time.

Blog at WordPress.com.
Entries and Comments feeds.