On the use and misuse of randomisation

14 Jun, 2019 at 10:11 | Posted in Economics | Comments Off on On the use and misuse of randomisation

‘Ideally controlled experiments’ tell us with certainty what causes what effects — but only given the right ‘closures.’ Making appropriate extrapolations from (ideal, accidental, natural or quasi) experiments to different settings, populations or target systems, is not easy. “It works there” is no evidence for “it will work here”. Causes deduced in an experimental setting still have to show that they come with an export-warrant to the target system. The causal background assumptions made have to be justified, and without licenses to export, the value of ‘rigorous’ methods and ‘on-average-knowledge’ is despairingly small.

Randomisation — as does econometrics — promises more than it can deliver, basically because it requires assumptions that in practice are not possible to maintain. And randomisation is — as econometrics — basically a deductive method. Given the assumptions, these methods deliver deductive inferences. The problem, of course, is that we will never completely know when the assumptions are right. And although randomisation may contribute to controlling for confounding, it does not guarantee it, since genuine randomness​ presupposes infinite experimentation and we know all real experimentation is finite. And even if randomisation may help to establish average causal effects, it says nothing of individual effects unless homogeneity is added to the list of assumptions. Causal evidence generated by randomisation procedures may be valid in ‘closed’ models, but what we usually are interested in, is causal evidence in the real target system we happen to live in.

Yours truly is extremely fond of economists like Angus Deaton. With razor-sharp intellect, ​he immediately goes for the essentials. He has no time for bullshit. And neither should we.