On statistics and causality

28 Jul, 2022 at 15:56 | Posted in Statistics & Econometrics | 9 Comments

Ironically, the need for a theory of causation began to surface at the same time that statistics came into being … This was a critical moment in the history of science. The opportunity to equip causal questions with a language of their own came very close to being realized but was squandered. In the following years, these questions were declared unscientific and went underground. Despite heroic efforts by the geneticist Sewall Wright (1889-1988), causal vocabulary was virtually prohibited for more than half a century. And when you prohibit speech, you prohibit thought and stifle principles, methods, and tools.

Readers do not have to be scientists to witness this prohibition. In Statistics 101, every student learns to chant, “Correlation is not causation.” With good reason! The rooster’s crow is highly correlated with the sunrise; yet it does not cause the sunrise.

Unfortunately, statistics has fetishized this commonsense observation. It tells us that correlation is not causation, but it does not tell us what causation is. In vain will you search the index of a statistics textbook for an entry on “cause.” Students are not allowed to say that X is the cause of Y — only that X and Y are “related” or “associated.”

Statistical reasoning certainly seems paradoxical to most people.

Take for example the well-known Simpson’s paradox.

From a theoretical perspective, Simpson’s paradox importantly shows that causality can never be reduced to a question of statistics or probabilities unless you are — miraculously — able to keep constant all other factors that influence the probability of the outcome studied.

To understand causality we always have to relate it to a specific causal structure. Statistical correlations are never enough. No structure, no causality.

Simpson’s paradox is an interesting paradox in itself, but it can also highlight a deficiency in the traditional statistical/econometric approach toward causality. Say you have 1000 observations on men and an equal amount of observations on women applying for admission to university studies, and that 70% of men are admitted, but only 30% of women. Running a logistic regression to find out the odds ratios (and probabilities) for men and women on admission, females seem to be in a less favorable position (‘discriminated’ against) compared to males (male odds are 2.33, female odds are 0.43, giving an odds ratio of 5.44). But once we find out that males and females apply to different departments we may well get a Simpson’s paradox result where males turn out to be ‘discriminated’ against (say 800 males apply for economics studies (680 admitted) and 200 for physics studies (20 admitted), and 100 female apply for economics studies (90 admitted) and 900 for physics studies (210 admitted) — giving odds ratios of 0.62 and 0.37).

Statistical — and econometric — patterns should never be seen as anything else than possible clues to follow. Behind observable data, there are real structures and mechanisms operating, things that are  — if we really want to understand, explain and (possibly) predict things in the real world — more important to get hold of than to simply correlate and regress observable variables.

Statistics cannot establish the truth value of a fact. Never has. Never will.

Lars P. Syll

9 Comments

  1. How fortuitous Lars … did you see Krugmans mea culpa NYT opinion post on inflation. I’ll summarize. Post GFC orthodox models were doing their thang, then oops … no longer conforming to reality … oops forgot to input Covid and Russian sanctions … with a side of built in fragility through long lines of information in trading exchange of real goods … but hay not is all lost … some industries are seeing huge profits and will roll those back in to stock buy backs … were all saved …

    Comment by Skippy— 29 Jul, 2022 #

  2. Lars says:
    .
    《Behind observable data, there are real structures and mechanisms operating, things that are  — if we really want to understand, explain and (possibly) predict things in the real world — more important to get hold of than to simply correlate and regress observable variables.》
    .
    But if you still envy physics, isn’t dark energy an example of a theory with no causal mechanism, based purely on the stunning observational result that the universe is actually expanding faster and faster, when it should be slowing down its expansion or contracting, according to standard physics?
    .
    Maybe physics enviers should (re-)read Deirdre McCloskey’s 1983 article “The Rhetoric of Economics,” Journal of Economic Literature 21(2), pp. 481–517 (linked on the “McCloskey Critique” Wikipedia page)?
    .
    《Nothing is gained from clinging to the Scientific Method, or to any methodology except honesty, clarity, and tolerance. Nothing is gained because the methodology does not describe the sciences it was once thought to describe, such as physics or mathematics; and because physics and mathematics are not good models for economics anyway; […]》

    Comment by rsm— 29 Jul, 2022 #

    • Nah, dark energy is not a theory. It’s not even a hypothesis. It’s just a label for an unknown. Cause X.

      Comment by Crash Carson— 29 Jul, 2022 #

      • If physicists were like geologists, shouldn’t they simply deny dark energy because there is no causal mechanism, just as Wegener’s continental drift theory was mocked and dismissed as pseudoscience? Yet physics somehow accepts this unreal cause into its current orthodoxy, despite the contradictions it leads to?
        .
        See Wikipedia’s Cosmological Constant article:
        .
        《In cosmology, the cosmological constant (usually denoted by the Greek capital letter lambda: Λ), alternatively called Einstein’s cosmological constant, is the constant coefficient of a term Albert Einstein temporarily added to his field equations of general relativity. He later removed it. Much later it was revived and reinterpreted as the energy density of space, or vacuum energy, that arises in quantum mechanics. It is closely associated with the concept of dark energy.》
        .
        […]
        .
        《If the universe is described by an effective local quantum field theory down to the Planck scale, then we would expect a cosmological constant of the order of {\displaystyle M_{\rm {pl}}^{2}} ({\displaystyle 1} in reduced Planck units). As noted above, the measured cosmological constant is smaller than this by a factor of ~10^120. This discrepancy has been called “the worst theoretical prediction in the history of physics!”》

        Comment by rsm— 29 Jul, 2022 #

        • Check out Hossenfelder on Dark Energy:

          Comment by lesdomes— 29 Jul, 2022 #

    • Lesdomes, is McCloskey’s critique particularly apt here, because we are all practicing rhetoric?
      .
      McCloskey says (copying by hand because the pdf is in images):
      .
      《The credo of Scientific Method […] is that all sure knowledge is modeled on the early 20th century’s understanding of certain pieces of 19th century physics. […] it is best labeled simply “modernism”, that is, the notion (as Booth puts it) that we know only what we cannot doubt and cannot really know what we can merely assent to.》
      .
      (Would Kingsley Lewis and Alfred Marshall agree?)
      .
      Despite Solow’s warning, to get into arguing about cavalry tactics at the Battle of Austerlitz a little bit, isn’t Hossenfelder using rhetoric to try to persuade us that dark energy doesn’t exist? May I equally use rhetoric such as quoting Wikipedia as saying the “new” paper Hossenfelder cites was “quickly countered”, thus theories that assume the existence of dark energy (Wikipedia has a section on Theories of Dark Energy, contrary to Crash Carson’s rhetoric in another comment) still prevail, socially, among physicists?
      .
      When Lars says:
      .
      《Without causal information, this — and many other ‘statistical’ paradoxes — aren’t really possible to solve.》
      .
      Does that apply equally to physics (and mathematics)?
      .
      Is it possible that certain stories of causes in some charismatic researcher’s head (or math proofs …) can just get more popular, due to pure rhetorical tricks?
      .
      In other words, if we try to stick to a “modernist” credo, can we ever know any cause? Is there always reasonable doubt (and reasonable doubt of doubt, to an infinite regress), and aren’t we usually claiming to know what we can merely assent to (only a few technicians actually do the measurements, everyone else can merely assent)?
      .
      (Sorry for the long post, on which I’ve spent more time than I care to admit, but may I add one other piece of rhetoric: why did Hossenfelder not list Brian Schmidt, verbally, along with the two other Nobel Prize winners, in her videos? Is it because Schmidt, in the “Greatest Unsolved Mysteries of the Universe” MOOC I took for free when it was first offered, was quite allright as I remember with acknowledging that space was inconsistent, while Hossenfelder prefers to use rhetoric to sell us a causal and consistent story? Is omitting him from her verbal account a rhetorical trick? Is she annoyed with the view he’s selling, that “If energy is fundamentally part of the fabric of space, that energy pushes on itself through gravity and makes the universe speed up”? – Brian Schmidt: In Conversation at theconversation.com?)
      .
      Why doesn’t it all come down to our feelings about stories? (Can I ask Shiller, Lakatos, McCloskey, Black, Roger Farmer …)

      Comment by rsm— 30 Jul, 2022 #

      • I did not have the impression that Hossenfelder was arguing dark energy doesn’t exist – my take is rather she was expanding on the sentiment pithily expressed by Crash Carson above: “dark energy is not a theory. It’s not even a hypothesis. It’s just a label for an unknown. Cause X.” I would put it like this: the only “theory” or universe description broken by reported observations (combined with our implicit auxiliary assumptions) follows a certain simple pattern predicted by our existing theory alone. We hope or expect there must be a mechanism that explains this discrepancy. “Dark energy” is a placeholder for that, just as “dark matter” is a placeholder for the gross gravitational anomalies seen in cosmology.

        In these and other works in general, Hossenfelder argues (convincingly for me) that theorists can spin off theoretical explanations endlessly, until they go orders of magnitude beyond anything testable by current or foreseeable technologies. It’s what theorists do for a living. A hazard from doing so is that research communities get trapped in a box of theories that might exclude what is “really going on” (to the extent such realities exist) and may even cling to theories far past the point that data has refuted those (or at least some exalted authorities will do that; witness Sir Harold Jeffreys clinging to fixed continents well into my lifetime; recall “science progresses funeral by funeral”). Lars and others see that in econometrics, I see it in health-medical sciences, and Hossenfelder sees it in physics, which in my view has been inappropriately idolized by other scientists since it too can suffer from human and social factors, like promotion or suppression of theories for motives other than pursuit of knowledge (like glory and power). Watch if you will her commentaries such as those on

        multiverse theories: https://www.youtube.com/watch?v=-dSua_PUyfM
        cosmogony: https://www.youtube.com/watch?v=VHhUCav_Jrk
        doing math and P-hacking for publication (re the W-boson mass):

        imposing authority and nonsense arguments to dismiss undesirable possibilities (re superdeterminism):

        – from that I see how not only in econ and medicine but in physics no less (!) that human-social factors can dominate and even warp whole topics, especially when the theorists become exalted despite their theories far outrunning the data, even when they make no contribution to empirical knowledge (where by the latter I mean checkable information about how observations behave or the causes of that behavior – “causes” being the conveyors of information, which we “know” cannot propagate faster than c ~ 300,000 km/sec).

        On the far more broad topic of human epistemology that you raise in talking of assent, my reaction is yes of course all our “knowledge” is woven with assent. We are utterly dependent on testimonies of others, whether on where to find the cheapest gas, or whether we should get another covid shot, or whether to believe projections about the universe going beyond our farthest observations (the CMB at ~380,000 yrs) down to a picosecond (that’s 25 orders of magnitude!). What distinguishes “scientific knowledge” from other knowledge claims remains hotly debated. I’m among those who hold a large part of the demarcation (as developed in the last century) is a focus on finding where theories work and fail according to explicit criteria. Theorists try to build out from existing frameworks but function only as speculators if their extensions can’t be quickly calibrated by such criteria against empirical studies (no, not computer simulations, unless they are studying simulators). That calibration is a central function of applied statistics. Hossenfelder’s complaint is then about those who sell what may perhaps be perfectly plausible speculations as something more than that, such as instrumentally or predictively useful knowledge as displayed by empirical studies. In that view, the Standard Model and General Relativity are knowledge, while string theory and many-worlds quantum mechanics are interesting speculations.

        I think if there is any point from the above worth bearing in mind, it’s that, like religions, philosophies, and political parties, sciences are emergent products of social collectives imbued with sufficient and accurate enough mechanisms for information storage, retrieval, and processing available to those who become (or today, are accepted as) participants in their development. They don’t reside in individual knowledge. The way I view modern sciences, even the most exalted scientists lack sufficient and accurate enough mechanisms for me to believe just on the basis of their exaltation that they “know” anything beyond what is admitted into the social framework in which their science operates. Of course, that doesn’t stop me from speculating about topics beyond my expertise, like I just did!

        Comment by lesdomes— 30 Jul, 2022 #

  3. Well put: “Behind observable data, there are real structures and mechanisms operating, things that are — if we really want to understand, explain and (possibly) predict things in the real world — more important to get hold of than to simply correlate and regress observable variables.”
    For an expansion on the same sentiments see
    Greenland, S. (2022). The causal foundations of applied probability and statistics. Ch. 31 in: Dechter, R., Halpern, J., and Geffner, H., eds. Probabilistic and Causal Inference: The Works of Judea Pearl. ACM Books, no. 36, 605-624,
    https://dl.acm.org/doi/10.1145/3501714.3501747, corrected version at
    https://arxiv.org/abs/2011.02677

    As a secondary point, however, the comments on Simpson’s Paradox seem to miss the distinction between the causal phenomenon of confounding and the statistical phenomenon of noncollapsibility. The two are often confused and “Simpson’s Paradox” has been used to refer to one, the other, or some unclear amalgam of the two. See
    Greenland, S., Robins, J. M., and Pearl, J. (1999). Confounding and collapsibility in
    causal inference. Statistical Science, 14, 29-46.
    Hernán MA , Clayton D , Keiding N (2011). The Simpson’s paradox unraveled. Int J Epidemiol, 40, 780–785.
    Pearl’s main book, Causality (2nd ed. 2009) also discusses the distinction and confusion.

    Comment by lesdomes— 29 Jul, 2022 #

    • Thanks for the links, Sander. And yes, you’re probably right about Simpson’s paradox. I mainly wanted to give an example of something that most people find intriguing the first time they run into it, and that at heart has to do with the need to go beyond statistical numbers and look for the causal data-generating processes behind given data. As Pearl so well has shown is the paradox really a question of confounding and the long refusal of statisticians to discuss causality. Without causal information, this — and many other ‘statistical’ paradoxes — aren’t really possible to solve.

      Comment by Lars Syll— 29 Jul, 2022 #


Sorry, the comment form is closed at this time.

Blog at WordPress.com.
Entries and Comments feeds.