What counts as evidence?

10 January, 2019 at 15:29 | Posted in Theory of Science & Methodology | Leave a comment

What counts as evidence? I suspect we tend to overweight some kinds of evidence, and underweight others.

Yeh’s paper is a lovely illustration of a general problem with randomized control trials – that they tell us how a treatment worked under particular circumstances, but are silent about its effects in other circumstances. They can lack external validity. Yeh shows that parachutes are useless for someone jumping from a plane when it is on the ground. But this tells us nothing about their value when the plane is in the air – which is an important omission.

evidenceWe should place this problem with RCTs alongside two other Big Facts in the social sciences. One is the replicability crisis … The other (related) is the fetishization of statistical significance despite the fact that, as Deirdre McCloskey has said, it “has little to do with a defensible notion of scientific inference, error analysis, or rational decision making” and “is neither necessary nor sufficient for proving discovery of a scientific or commercially relevant result.”

If we take all this together, it suggests that a lot of conventional evidence isn’t as compelling as it seems. Which suggests that maybe the converse is true.

Stumbling and Mumbling

Leave a Comment »

RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

Blog at WordPress.com.
Entries and comments feeds.