Toward Evidence-Based Statistics

IN RESPONSE:

I thank the correspondents for their thoughtful reactions to these articles. I agree wholeheartedly with Dr. Sulmasy that Bayes does not “give all the answers,” and I tried to communicate that in my article; perhaps I did not navigate deftly enough between the Scylla and Charybdis of frequentism and Bayesianism. I also support his criticism of labeling frequentist probabilities as “objective” and inductive ones as “subjective.” The clinical trial example in the first article showed how the frequentist P value could be affected by the mental states of the investigators. The deeper one delves, the more the line between objectivity and subjectivity appears blurred, as does the value of the distinction. Readers interested in further exploration of these issues should look at Greenland's superb recent review (1).

Drs. Sulmasy and Morgan both question the meaning of the probability of a hypothesis, saying that hypotheses are either true or untrue. We can believe that the probabilities of hypotheses are appropriate bases for rational thought without believing that they are actually properties of reality (2). Such probabilities are not logically different from the prognostic information physicians give to patients. Even if one believes that a given patient would suffer the same fate whenever a particular treatment were given (that is, the individual probability is 0 or 1), if that fate is unknown to us, it is still meaningful to say that there is a 30% chance of death (3). Neither statistical nor physical models of the world need to be literally true to be extraordinarily useful. All that said, I should emphasize that the main focus of my article was not on probabilities of hypotheses but rather on the Bayesian measure of evidence, which is less controversial. Einstein “disproved” Newton, but “ F = M × A ” still helps to keep buildings upright.

I appreciate Dr. Caubet's enthusiasm for the many possible applications of Bayesian methods, but it is not clear where he is drawing the line between Bayesian approaches, designed for inference, and decision analysis, which incorporate costs in evaluating the consequences of actions. I am also not sure that “keeping the technology inside” should be seen as an advantage of any method whose use can affect patients. The value of Bayesian and decision analysis is not their automaticity but rather that judgments are made explicit; hiding them can make the technological cure worse than a disease.

Finally, I am eternally indebted to Dr. Sulmasy for associating my work, if only referentially, with that of Kant.

• Copyright ©2004 by the American College of Physicians