Standing Statistics Right Side Up

• Statistics
• Sensitivity and specificity
• Likelihood functions
• Predictive value of tests
• Probability
• Bayes theorem

During the years I taught students about diagnostic reasoning, I would begin by explaining that the sensitivity of a diagnostic test for disease X is found by measuring how often the test result is positive in a population of patients, all of whom are known (by some independent and definitive criterion, the “gold standard“) to have disease X: that is, by measuring the frequency of true-positive results in that population. A test that yields positive results in 95 of 100 diseased patients, for example, has a sensitivity of 0.95. We would then talk about test specificity—the likelihood that the same test would have a false-positive result in a population of patients known by the gold standard not to have the disease. A test that yields positive results in 10 of 100 nondiseased patients has a specificity of 0.90.

I would then ask the students to imagine that in working up a new patient, they have gotten back a positive result from a test with the above sensitivity and specificity. What would they tell the patient about his or her probability of having disease X? Their answer was almost always “95%.” On the face of it, that answer seems pretty reasonable: Isn't that what you'd expect if a test were capable of detecting 95% of diseased patients? The problem is, it's wrong; worse, it actually stands diagnostic reasoning on its head.

In fact, test sensitivity and specificity are deductive measurements; they reason down from hypothesis (we assume the truth of the hypothesis that the patient being tested does, or does not, have the disease) to data (the likelihood that we will get a positive test result). The students' reasoning is upside down because what clinicians and patients really need to know is exactly the inverse. In short, they need an inductive …