Accuracy of Death Certificates for Coding Coronary Heart Disease as the Cause of Death

Methods

Study Sample

The Framingham Heart Study began in 1948 when 5209 residents of Framingham, Massachusetts, who were 28 to 62 years of age enrolled in a prospective epidemiologic study. The selection criteria and study design have been detailed elsewhere [9-11]. Every 2 years, members of this cohort receive follow-up evaluations that include medical histories, physical examinations, and selected laboratory tests. Ascertainment of the vital status of study participants has been essentially complete.

Determination of Cause of Death

Since the beginning of the study, each death has been reviewed and assigned an underlying cause by a panel of three physicians. As part of the review process, all available medical information about each death is collected. This information typically includes Framingham Heart Study records, hospitalization records, and, when available, autopsy results. In the case of an out-of-hospital, witnessed death, family members are interviewed by telephone to better ascertain the circumstances surrounding death. The death certificate is usually available to the panel, but it is not used to determine the underlying cause of death.

After discussing the case, the panel jointly assigns an underlying cause of death, which is then coded into one of the following six mutually exclusive categories: 1) coronary heart disease, 2) stroke, 3) other cardiovascular disease, 4) cancer, 5) other, or 6) unknown. The panel makes a conscious effort to determine the true underlying cause of death; however, when cause cannot be reliably determined from all available data (for example, in the case of a nursing home resident with progressive inanition), the panel assigns the death to the category of unknown cause. Sudden death, defined as death occurring within 1 hour of symptom onset, is attributed to coronary heart disease unless another cause is apparent. These criteria for assigning cause of death have not changed since the beginning of the study. Diagnostic tests have improved over time, but the panel has access to the same results that are available to the physician coding the death certificate.

We compared the underlying cause of death listed by each patient's physician on the death certificate with the cause assigned by the Framingham Heart Study physician panel. In 1988, a trained nosologist, blinded to the findings of the physician panel, coded each death certificate according to the International Classification of Diseases, Ninth Revision (ICD-9) [12]. For our study, we used the ICD-9 code assigned by the nosologist to determine the underlying cause of death on the death certificate. Coronary heart disease was considered the underlying cause of death if the cause of death was assigned an ICD-9 code of 410 to 414 (ischemic heart disease) by the nosologist. Although ICD-9 code 427 (cardiac dysrhythmia) has also been used to identify deaths from coronary heart disease [13], none of the deaths in our sample was assigned this code by the nosologist. Other cardiovascular disease was considered to be the underlying cause of death for ICD-9 codes of 390 to 404, 415 to 425, 428, 429, and 440 to 459. Death certificates of persons who died after 1988 have not been nosologically coded.

Study Design

By the end of 1988, 2888 participants from the original cohort had died. Death certificates were available and nosologically coded for 2719 of these participants (94.1%). Because few deaths occurred at young ages, we excluded participants whose age at death was younger than 45 years (n = 36).

We examined the utility of the death certificate for coding coronary heart disease (as opposed to any other cause) as the underlying cause of death. With the cause of death determined by the physician panel serving as the reference standard, the sensitivity, positive predictive value, specificity, and negative predictive value of the death certificate for coding coronary heart disease as the underlying cause of death were calculated according to their usual definitions [14]. In the context of vital statistics, sensitivity has also been called the detection rate and positive predictive value has been called the confirmation rate [15]. To assess the effect of such cases on the overall utility of the death certificate, these variables were calculated with inclusion and exclusion of cases with panel-assigned unknown cause of death to assess the effect of such cases on the overall utility of the death certificate.

For comparative purposes, the sensitivity, positive predictive value, specificity, and negative predictive value of the death certificate for cancer and for stroke as the underlying cause of death were also calculated. Cancer was considered the underlying cause of death if the ICD-9 code was 140 to 239; stroke was considered the underlying cause of death if the ICD-9 code was 430 to 438.

To determine whether there was a time trend in the accuracy of the death certificate during the study, we compared the sensitivity, positive predictive value, specificity, and negative predictive value of the death certificate for coronary heart disease during three consecutive decades from 1955 through 1984. These decades were chosen because too few deaths occurred before 1955 to allow meaningful comparisons with later periods. To account for aging of the cohort, we further restricted this time-trend analysis to decedents whose age at death was 50 to 84 years (n = 2033).

Statistical Analysis

All analyses were performed by using SAS software [16]. Ninety-five percent CIs for the estimates of sensitivity, specificity, and positive and negative predictive values were calculated by using asymptotic calculations of the normal distribution. The chi-square test statistic with five degrees of freedom was used to assess equality of the marginal rates of diagnosis of underlying cause of death between the death certificate and the physician panel [17, 18]. Statistics were adjusted for age, sex, calendar year in which death occurred, or prevalence of coronary heart disease by using logistic regression [19] where such adjustment was indicated. For analysis of time trends in these statistics, we did not use the aggregate sample method described by Coughlin and colleagues [20]. Instead, logistic models were fitted to various subsets of participants. We let X = 1 if the panel assigned coronary heart disease as the cause of death and let X = 0 otherwise; similarly, we let Y = 1 if the death certificate recorded coronary heart disease as the cause of death and let Y = 0 otherwise. For sensitivity, we modeled Pr(Y = 1 | X = 1); for specificity, Pr(Y = 0 | X = 0); for positive predictive value, Pr(X = 1 | Y = 1); and for negative predictive value, Pr(X = 0 | Y = 0). For example, only patients with panel-assigned coronary heart disease death were used for analysis of sensitivity. Co-variates included in the models to test for time trend in the diagnosis of death from coronary heart disease were sex of the decedent, indicators for age group at time of death (50 to 64 years of age, 65 to 74 years of age, and 75 to 84 years of age), and decade of death (0 = 1955 to 1964, 1 = 1965 to 1974, and 2 = 1975 to 1984). Hosmer-Lemeshow statistics were computed to assess goodness of fit for the trend model. Further checks were made by using models with unique coefficients for each period and by using models without any time-period variable. A P value less than 0.05 was considered statistically significant.

Results

We analyzed a total of 2683 decedents whose underlying causes of death were coded by the death certificate and determined by the physician panel. The distribution of this sample by age and sex is given in Table 1. Overall, 52.6% of decedents were male and 41.9% were at least 75 years of age.

Table 2 presents a cross-classification of the 2683 deaths by underlying cause as assigned by the physician panel and as taken from the death certificate. The proportions of participants assigned to different categories of cause of death differed (P < 0.001) between the physician panel and the death certificate. The physician panel assigned coronary heart disease as the cause of death in 758 cases (28.3%), a cause unrelated to coronary heart disease in 1683 cases (62.7%), and unknown cause in 242 cases (9.0%). Of the 242 deaths assigned to unknown cause by the physician panel, the death certificate coded coronary heart disease as the cause of death in 124 (51.2%). The proportion of deaths assigned to unknown cause by the physician panel increased from 4.8% (9 of 189) in participants who were between 45 and 54 years of age to 22.5% (69 of 307) in participants who were 85 years of age and older.

Table 3 shows the overall utility of the death certificate for diagnosing coronary heart disease as the underlying cause of death. The sensitivity and positive predictive value of the death certificate for coronary heart disease were less than those for cancer, which had the highest values, and greater than those for stroke, which had the lowest values. The specificities and negative predictive values were relatively high for all three of these causes of death. After deaths from unknown cause were excluded (Table 3), the positive predictive value and specificity of the death certificate for coronary heart disease increased, reflecting the tendency for deaths from unknown cause to be attributed to coronary heart disease on the death certificate.

Overall, the utility of the death certificate for determining cause of death was worse in women than in men and tended to decrease as the age of the decedents increased. The positive predictive value of the death certificate was particularly low (43.9%) in women who were at least 75 years of age at time of death.

Table 4 compares the number of deaths from coronary heart disease coded by the death certificate with the number of deaths from coronary heart disease determined by the physician panel. Among all decedents, the death certificate attributed 184 (24.3%) more deaths to coronary heart disease than did the physician panel. The tendency of the death certificate to assign more deaths to coronary heart disease than did the physician panel increased with the age of the participants and was particularly prominent in the oldest group (participants ≥ 85 years of age at time of death). In the oldest group, the number of deaths coded as coronary heart disease by the death certificate was more than twice the number assigned to coronary heart disease by the physician panel. After cases that were assigned an unknown cause of death were excluded, this difference was smaller; 7.9% more deaths were assigned to coronary heart disease by the death certificate than by the physician panel overall. However, in participants who were at least 85 years of age, 43.1% more deaths were assigned to coronary heart disease by the death certificate.

Using logistic regression, we found no evidence of a time trend in the four measures of utility of the death certificate after adjustment for age and sex, whether including or excluding cases with unknown cause of death assigned by the physician panel (P > 0.2). Goodness of fit for these models was acceptable (P > 0.15 for each model). Neither the time-trend models (P > 0.2) nor the more general time-period models (P > 0.2) fitted significantly better than models without any time effect. The utility of the death certificate was affected by the presence of an antemortem diagnosis of coronary heart disease. After we adjusted for age and sex, the positive predictive value improved (83% in decedents with preexisting coronary heart disease compared with 44% in those without; P < 0.001) and specificity (79% compared with 86%; P = 0.005) and negative predictive value (80% compared with 97%; P < 0.001) decreased.

Discussion

We found that compared with coronary heart disease assigned as cause of death by a panel of three trained physician-adjudicators, the death certificate detected 83.8% of cases. Conversely, the physician panel confirmed coronary heart disease as the cause of death in only 67.4% of cases attributed to coronary heart disease on the death certificate. The positive predictive value of the death certificate tended to be lower in women than in men, a finding that can be explained by the fact that fewer women died of coronary heart disease. In general, the utility of the death certificate decreased as the age of the decedents increased. This is presumably a consequence of the obfuscation of signs and symptoms of disease that occurs with advancing age and the higher prevalence of multiple comorbid disease processes in older persons [21-23].

When the physician panel assigned unknown cause of death, coronary heart disease was often listed as the underlying cause of death on the death certificate (51% of cases). When physicians complete a death certificate, they are required to assign an underlying cause of death; our data suggest that physicians may use coronary heart disease as a “default” cause. We found that compared with the physician panel, the death certificate was 24.3% more likely to assign coronary heart disease overall and more than twice as likely to assign it in the oldest age group. Because the cause of some deaths could not be established by the physician panel despite diligent investigation, the true rate of misclassification by the death certificate is likely to be between 7.9% and 24.3% overall and between 43.1% and 109% for decedents 85 years of age and older. The disparity we observed in assignment of underlying cause of death between death certificates and a trained panel of physicians suggests that physicians should receive more training in methods of death certificate completion.

The underlying cause of death listed on death certificates has previously been shown to be incorrect in as many as 30% to 40% of deaths [24-26]. Messite and Stellman [27] recently used six standardized, written case reports prepared by the National Center for Health Statistics to assess the accuracy of death certificate completion. They reported only 56% agreement between underlying cause of death as recorded by physicians and the correct standardized diagnoses published by the National Center for Health Statistics. Level of agreement varied from 15% to 99%, depending on the complexity of the case [27].

Although the autopsy must be considered definitive in determining the underlying cause of death, autopsy studies of the accuracy of death certificates have been limited by selection bias [22, 28, 29]. In the United States, autopsy is done for only 11.9% of deaths [30]. Therefore, the inaccuracy of the death certificate in an autopsy study may reflect the tendency to obtain autopsies selectively in cases of uncertainty [21, 28]. However, a recent autopsy study from a single German municipality [31] was population-based with nearly complete ascertainment and had findings similar to ours. For circulatory deaths, of which deaths from coronary heart disease were a subset, the study authors reported the death certificate to have a sensitivity of 83% and a positive predictive value of 69%. The authors also demonstrated a 21% higher net assignment of coronary heart disease deaths by the death certificate [31].

As in our study, other investigators [13, 32, 33] have relied on review of all available clinical information to determine the cause of death that served as the reference standard. In each of these studies, decedents were younger and were selected on the basis of having a high likelihood of death due to cardiovascular disease; this would result in a greater positive predictive value and a lower negative predictive value of the death certificate than those obtained by using nonselected decedents. Our study was community-based and therefore allowed a meaningful estimate of sensitivity and specificity.

National mortality statistics, which necessarily rely on death certificates, may overestimate the effect of coronary heart disease as an underlying cause of death, especially in older persons. Because coronary heart disease is the leading cause of death in the United States (accounting for approximately 500 000 deaths per year [1]), misclassification may cause a large absolute shift in cause-of-death statistics. As the U.S. population ages, the incidence of many chronic diseases will increase. Improvement in the accuracy of death certification by physicians may permit more accurate detection of trends in cause of death. Maximizing the accuracy of these statistics is desirable because national mortality data influence the allocation of resources by many government and medical institutions.

Stehbens [34] has suggested that because vital statistics are calculated on the basis of data obtained from death certificates, the observed time trends in mortality from coronary heart disease may be purely artifactual. Although changes in ICD coding of death certificates have had abrupt effects on cause-specific mortality rates [35, 36], we found that the accuracy of the death certificate did not change significantly over three decades of follow-up when all death certificates were coded according to ICD-9 standards.

Compared with coronary heart disease, the sensitivity and positive predictive value of the death certificate for cancer and stroke as causes of death provided interesting contrasts. Assignment of cancer as the underlying cause of death on the death certificate had the highest sensitivity and positive predictive value, a finding that may reflect the chronicity of most neoplastic diseases that allows antemortem diagnosis. In contrast, the sensitivity and positive predictive value of the death certificate were particularly poor with regard to stroke.

Our findings deserve particular attention for research that uses the death certificate as the sole means of establishing the diagnosis of coronary heart disease. The use of a diagnostic test (for example, the death certificate) with sensitivity and specificity less than 100% will affect the ability of a study to detect differences between groups of participants at risk. Assuming that the death certificate causes nondifferential misclassification of cause of death, imperfect sensitivity and specificity will cause an underestimate of the true relative risk and a bias toward the null. As an example, consider a follow-up study in which the risk for disease is compared between an exposed group and an unexposed group. If the true risk for disease is 10% among the exposed group and 5% among the unexposed group, and if disease is classified with 83.8% sensitivity and 84.1% specificity (as for death from coronary heart disease in our study), then the expected relative risk is calculated to be 1.18, considerably smaller than the true relative risk of 2.0 [37]. For rarer diseases, the null bias is even greater. Substituting true disease risks of 2% and 1% among the exposed and unexposed, respectively, in the above example yields a relative risk of 1.04 compared with a true relative risk of 2.0 [37]. This bias toward the null is likely to be more marked for coronary heart disease and stroke and less marked for cancer, for which the death certificate had a higher sensitivity and specificity. Nonetiologic studies of coronary heart disease frequency and trends should be less affected by this null bias.

Our study is limited by the lack of an external criterion, or gold standard, against which to measure the accuracy of the death certificate. However, we suggest that systematic review of all available clinical evidence by a panel of trained physicians may provide a more objective determination of underlying cause of death than that obtained from the death certificate. If the physician panel underassigned coronary heart disease, our estimates would have to be revised downward. An extreme scenario could be constructed in which all 124 decedents with unknown cause of death assigned by the physician panel and coronary heart disease assigned by the death certificate actually died of coronary heart disease. In such a case, the rate of overclassification by the death certificate would be 6.8% overall and 25.9% in the oldest age group. It is theoretically possible that attending physicians had information that was not available to the panel and that death certificates were therefore more accurate than the physician panel in diagnosing coronary heart disease. However, this seems unlikely given the similarity of our findings to those of other studies. We did not code the location of deaths (that is, in or out of the hospital) and therefore could not determine whether place of death affected the utility of the death certificate. Other potential limitations include limited representation of minority populations among decedents and location of the study in a single municipality. Finally, community physicians in Framingham may complete death certificates with greater precision than is typical elsewhere because of the degree of scrutiny to which records of participants in the Framingham Heart Study have been subjected.

The results of our study suggest that the death certificate overrepresents coronary heart disease as a cause of death, especially in older persons. Therefore, national mortality statistics, which are based on death certificate data, probably overestimate the frequency of coronary heart disease as an underlying cause of death. Our findings also suggest that researchers should exercise considerable caution when using the death certificate to determine cause of death in etiologic studies of coronary heart disease because of the potential for a significant null bias.

Dr. Martin: Duke University Medical Center, PO Box 3012, Durham, NC 27710.