Natural ‘natural experiments’

24 Oct, 2023 at 11:14 | Posted in Statistics & Econometrics | Comments Off on Natural ‘natural experiments’

Evidently, however, the potential for the strictly natural natural experimental approach, which relies exclusively on natural events as instruments, is constrained by the small number of random events provided by nature and by the fact that most outcomes of interest are the result of many factors associated with preferences, technologies, and markets. And the prospect of the discovery of new and useful natural events is limited …

It is clear that the number of natural instruments will never be sufficient to eliminate the necessity of imposing auxiliary assumptions or of obtaining supplementary empirical information relevant to the assumptions needed for identification … Measurement without theory, however, is not significantly more valuable than it ever was before the use of natural natural experiments.

Mark Rosenzweig & Kenneth Wolpin

Rosenzweig and Wolpin discuss several serious issues with studies based on natural ‘natural experiments.’ One noticeable and significant problem they do not address, however, is that researchers using these randomization-based research strategies consistently formulate problems to achieve ‘exact’ and ‘precise’ results that are not the ones we truly want answers to. The design becomes the main focus, and as long as one can set up more or less clever experiments, they believe they can draw far-reaching conclusions about both causality and generalizing experimental outcomes to larger populations. Unfortunately, this often leads to a shift in this type of research away from interesting and important issues towards prioritizing method choices. While design and research planning are important, the credibility of research ultimately comes down to being able to provide answers to relevant questions that both citizens and researchers want to know.