Rare Outcomes, Common Treatments: Analytic Strategies Using Propensity Scores

When treated patients are compared to controls, differing outcomes may reflect either effects caused by the treatment or differences in prognosis before treatment. Random assignment of patients to treatment or control, as in a randomized, controlled clinical trial (1), ensures that the groups were comparable before treatment and the prognosis in treated and control groups was nearly the same, so that differing outcomes indicate treatment effects. Somewhat more precisely, random assignment ensures that the only differences in prognosis between groups are due to chance, the flip of a coin in assigning treatments. In an ideal randomized trial, if a common statistical test rejects the hypothesis that the difference in outcomes is due to chance, a treatment effect is demonstrated. Notice that randomization does nothing to make patients have individually similar prognoses; rather, it ensures that assignment to treatment or control is unrelated to prognosis.

When random assignment is not used—that is, in an observational study—treated and control groups may differ in prognosis, and differing outcomes may not be effects of the treatment. Measured and recorded differences in prognosis—overt biases—can often be controlled by analytical adjustments (2), whereas unmeasured differences—hidden biases—may exist and must be addressed by other means (2-4). A prognostic variable or covariate is a variable describing the condition of patients before treatment. Bias refers to systematic differences between treated and control groups with respect to one or more prognostic variables; the bias is overt if the variable is measured and hidden if it is not.

Analytical adjustments for overt biases are of two kinds: 1) those that focus on the relationship between prognostic variables and outcomes and 2) those that focus on the relationship between prognostic variables and assignment of patients to treatment or control. The first strategy models the response directly, for example, through …