When odds ratios mislead (wonkish)

17 Oct, 2018 at 19:10 | Posted in Statistics & Econometrics | Comments Off on When odds ratios mislead (wonkish)

A few years ago, some researchers from Georgetown University published in the New England Journal of Medicine a study that demonstrated systematic race and sex bias in the behavior of America’s doctors. Needless to say, this finding was widely reported in the media:

Washington Post: “Physicians said they would refer blacks and women to heart specialists for cardiac catheterization tests only 60 percent as often as they would prescribe the procedure for white male patients.”

N.Y. Times: “Doctors are only 60% as likely to order cardiac catheterization for women and blacks as for men and whites.”

Now let’t try a little test of reading comprehension. The study found that the referral rate for white men was 90.6%. What was the referral rate for blacks and women?

If you’re like most literate and numerate people, you’ll calculate 60% of 90.6%, and come up with .6*.906 = .5436. So, you’ll reason, the referral rate for blacks and women was about 54.4 %.

But in fact, what the study found was a referral rate for blacks and women of 84.7%.

What’s going on?

It’s simple — the study reported an “odds ratio”. The journalists, being as ignorant as most people are about odds and odds ratios, reported these numbers as if they were ratios of rates rather than ratios of odds.

Let’s go through the numbers. If 90.6% of white males were referred, then 9.4% were not referred, and so a white male’s odds of being referred were 90.6/9.4, or about 9.6 to 1. Since 84.7% of blacks and women were referred, 13.3% were not referred, and so for these folks, the odds of referral were 84.7/15.3 ≅ 5.5 to 1. The ratio of odds was thus about 5.5/9.6, or about 0.6 to 1. Convert to a percentage, and you’ve got “60% as likely” or “60 per cent as often”.

The ratio of odds (rounded to the nearest tenth) was truly 0.6 to 1. But when you report this finding by saying that “doctors refer blacks and women to heart specialists 60% as often as they would white male patients”, normal readers will take “60% as often” to describe a ratio of rates — even though in this case the ratio of rates (the “relative risk”) was 84.7/90.6, or (in percentage terms) about 93.5%.

Mark Liberman