Bayes and the ‘old evidence’ problem

10 Jan, 2022 at 23:32 | Posted in Theory of Science & Methodology | 1 Comment

Among the many achievements of Newton’s theory of gravitation was its prediction of the tides and their relation to the lunar orbit. Presumably the success of this prediction confirmed Newton’s theory, or in Bayesian terms, the observable facts about the tides e raised the probability of Newton’s theory h.

bayes-theorem - Rens van de SchootBut the Bayesian it turns out can make no such claim. Because the facts about the tides were already known when Newton’s theory was formulated, the probability for e was equal to one. It follows immediately that both C (e ) and C (e |h ) are equal to one (the latter for any choice of h ). But then the Bayesian multiplier is also one, so Newton’s theory does not receive any probability boost from its prediction of the tides. As either a description of actual scientific practice, or a prescription for ideal scientific practice, this is surely wrong.

The problem generalizes to any case of “old evidence”: If the evidence e is received before a hypothesis h is formulated then e is incapable of boosting the probability of h by way of conditionalization. As is often remarked, the problem of old evidence might just as well be called the problem of new theories, since there would be no difficulty if there were no new theories, that is, if all theories were on the table before the evidence began to arrive. Whatever you call it, the problem is now considered by most Bayesians to be in urgent need of a solution. A number of approaches have been suggested, none of them entirely satisfactory.

A recap of the problem: If a new theory is discovered midway through an inquiry, a prior must be assigned to that theory. You would think that, having assigned a prior on non-empirical grounds, you would then proceed to conditionalize on all the evidence received up until that point. But because old evidence has probability one, such conditionalization will have no effect. The Bayesian machinery is silent on the significance of the old evidence for the new theory.

Michael Strevens

1 Comment

  1. As far as I am concerned, the Stevens passage is flat-out wrong as a general comment on “Bayesian machinery”. There are thousands of potential variants of Bayesian philosophy, machinery, and inference [1] (and scores of potential variants of frequentist philosophy, machinery, and inference). One class of variants (which Senn aptly labeled subjunctive Bayes) can take the old observations and treat them as if as yet unobserved, so that neither P(e) nor P(e|h) [Stevens’s C(e), C(e|h)] are 1. Serious use of this approach (of deleting old observations from the knowledge base) may involve examining the history of the field, but it can be done.
    [1] Good IJ (1971) “46,656 varieties of Bayesians.” Letter in American Statistician, 25: 62–63. Reprinted in Good Thinking, University of Minnesota Press, 1982, 20–21.

    Comment by lesdomes— 11 Jan, 2022 #


Sorry, the comment form is closed at this time.

Blog at WordPress.com.
Entries and Comments feeds.