‘Rigorous’ evidence — often worse than useless

9 April, 2017 at 16:35 | Posted in Economics | 1 Comment

So far we have shown that for two prominent questions in the economics of education, experimental and non-experimental estimates appear to be in tension. Furthermore, experimental results across different contexts are often in tension with each other. The first tension presents policymakers with a trade-off between the internal validity of estimates from the “wrong” context, and the greater external validity of observational data analysis from the “right” context. The second tension, between equally well-identifed results across contexts, suggests that the resolution of this trade-off is not trivial. There appears to be genuine heterogeneity in the true causal parameter across contexts.

8407_2008_58931_lThese findings imply that the common practice of ranking evidence by its level of “rigor”,
without respect to context, may produce misleading policy recommendations …

Despite the fact that we have chosen to focus on extremely well-researched literatures,
it is plausible that a development practitioner confronting questions related to class size, private schooling, or the labor-market returns to education would confront a dearth of well-identified, experimental or quasi-experimental evidence from the country or context in which they are working. They would instead be forced to choose between less internally valid OLS estimates, and more internally valid experimental estimates produced in a very different setting. For all five of the examples explored here, the literature provides a compelling case that policymakers interested in minimizing the error of their parameter estimates would do well to prioritize careful thinking about local evidence over rigorously-estimated causal effects from the wrong context.

Lant Pritchett & Justin Sandefur


1 Comment

  1. How can economics be taught as a coherent subject when even the experts do not agree about it? They should attempt to make it into a science about which the various stages in its axioms, assumptions, hypotheses are agreed first. Today different schools of thought teach different theories, which is the best that can be done, but it is not good enough. It is too much like having different religious beliefs!

Sorry, the comment form is closed at this time.

Create a free website or blog at WordPress.com.
Entries and comments feeds.