Limitations of the 1990 American College of Rheumatology Classification Criteria in the Diagnosis of Vasculitis

Methods

Patients

The institutional review boards of Duke University Medical Center and the Durham Veterans Affairs Medical Center approved this study. For all patients, an antineutrophil cytoplasmic antibody (ANCA) test was performed by indirect immunofluorescence in the Clinical Immunology Laboratory at Duke University Medical Center using standard techniques [15].

The patient sample was described in a previous report [15]. Briefly, 346 adult inpatients and outpatients, referred for evaluation of a possible vasculitis by rheumatologists at both medical centers, were eligible for the original study, a prospective study of the operating characteristics of ANCA for the diagnosis of Wegener granulomatosis [15]. In that study, we excluded patients who had previously diagnosed Wegener granulomatosis (n = 29) and patients who were receiving either immunosuppressive therapy or disease-modifying antirheumatic drugs (n = 65). For the current study, we further excluded patients who had a previously established vasculitis (n = 14) at the time of evaluation by the rheumatologist.

Study Protocol

Medical records of consecutive eligible patients were reviewed by a rheumatologist who was blinded to the patient's diagnosis. A standard protocol form was completed to record baseline demographic information, comorbid conditions, symptoms, physical findings, and laboratory and radiographic results. A symptom or physical finding reported within the 6 months before the rheumatologic evaluation was considered present. If information on symptoms and physical findings was missing, it was considered absent. The same rheumatologist reviewed the medical records 2 to 8 months after the initial rheumatologic evaluation to collect information on laboratory, biopsy, or angiography results (where applicable) and final diagnosis.

Patient Measures

The clinician's recorded final diagnosis, made 2 to 8 months after the initial rheumatologic evaluation, along with histopathology or angiography results, was considered the gold standard. If a patient did not have follow-up visits at either medical center during this period, this information was obtained from the referring physician. If a patient died, available autopsy records were examined to determine the immediate cause of death.

Our primary interest was to compare vasculitis diagnosed according to ACR classification criteria with the clinician's final diagnosis (as the gold standard). All patients for whom complete follow-up information was available were classified according to the 1990 ACR criteria for Wegener granulomatosis [16], giant-cell arteritis [17], polyarteritis nodosa [18], and hypersensitivity vasculitis [19] on the basis of initial presenting symptoms and physical findings.

Wegener Granulomatosis

Two of the following four criteria were required to meet ACR classification criteria for Wegener granulomatosis: 1) nasal or oral inflammation; 2) nodules, fixed infiltrates, or cavities on a chest radiograph; 3) microscopic hematuria or more than 5 erythrocytes per high-power field; and 4) granulomatous inflammation on biopsy [16]. These criteria have a reported sensitivity of 88.2% and a reported specificity of 92% for the classification of Wegener granulomatosis compared with other vasculitides [16].

Giant-Cell Arteritis

Three of the following five criteria were required to meet ACR classification criteria for giant-cell arteritis: 1) age 50 years or older, 2) new-onset localized headache, 3) temporal artery tenderness or decreased temporal artery pulse, 4) erythrocyte sedimentation rate of at least 50 mm/h, and 5) abnormal artery biopsy specimen characterized by mononuclear infiltration or granulomatous inflammation [17]. These criteria have a reported sensitivity of 93.5% and a reported specificity of 91.2% for the classification of giant-cell arteritis compared with other vasculitides [17].

Polyarteritis Nodosa

Three of the following 10 criteria were required to meet ACR classification criteria for polyarteritis nodosa: 1) weight loss of at least 4 kg; 2) livedo reticularis; 3) testicular pain or tenderness; 4) myalgias, weakness, or tenderness; 5) mononeuropathy or polyneuropathy; 6) diastolic blood pressure greater than 90 mm Hg; 7) elevated blood urea nitrogen or creatinine level; 8) hepatitis B virus infection; 9) arteriographic abnormality; and 10) biopsy specimen of small or medium-sized artery containing polymorphonuclear leukocytes [18]. These criteria have a reported sensitivity of 82.2% and a reported specificity of 86.6% for the classification of polyarteritis nodosa compared with other vasculitides [18].

Hypersensitivity Vasculitis

Three of the following five criteria were required to meet ACR classification criteria for hypersensitivity vasculitis: 1) age at disease onset older than 16 years, 2) medication at disease onset as a precipitating factor, 3) palpable purpura, 4) maculopapular rash, and 5) biopsy specimen showing granulocytes around an arteriole and venule [19]. These criteria have a reported sensitivity of 71% and a reported specificity of 83.9% for the classification of hypersensitivity vasculitis compared with other vasculitides [19].

Statistical Analysis

Descriptive analyses were done by using PC-SAS for Windows, version 6.11 (SAS Institute, Cary, North Carolina). Separate 2 × 2 contingency tables were constructed for patients who did and did not have a final clinical diagnosis of Wegener granulomatosis, giant-cell arteritis, polyarteritis nodosa, and hypersensitivity vasculitis compared with patients who did and did not meet ACR criteria for these four vasculitides. We constructed contingency tables for the entire cohort and for those patients with a final clinical diagnosis of vasculitis. The sensitivity, specificity, positive predictive value, and negative predictive value of ACR criteria for Wegener granulomatosis, giant-cell arteritis, polyarteritis nodosa, and hypersensitivity vasculitis were calculated for each of these scenarios.

Results

Patients

One hundred ninety-eight patients who did not have a previously established diagnosis of vasculitis at presentation met the inclusion criteria for our study and at 2 to 8 months had complete follow-up information available for review. Sixty-eight percent of the patients were female, and 77% were white; the average age was 51.1 years. In 113 instances (57%), patients were referred for a rheumatology consultation to consider a possible vasculitis. Other patients were seen because they had an unknown diagnosis that was suspected to be a connective tissue disease other than vasculitis (n = 76) or because they were follow-up patients (n = 9) and were thought to have a possible vasculitis by the evaluating rheumatologist.

Baseline Clinical Features and Diagnostic Evaluation

Table 1 lists the clinical and laboratory features of patients in the cohort. The most frequently reported symptoms were fatigue, myalgia or weakness, weight loss, sinus symptoms, and headaches. Hemoptysis and jaw claudication were relatively infrequent. On physical examination, the most common features were neuropathy or foot drop, palpable purpura, and oral or nasal ulcers. Livedo reticularis and ocular findings were uncommon.

The Westergren erythrocyte sedimentation rate was greater than 50 mm/h in 43% of patients. Twelve percent of patients had fixed infiltrates or nodules on chest radiography. Fifty-six patients (28%) were positive for antinuclear antibody at baseline; vasculitis was ultimately diagnosed in 9 (18%) of these patients. Twenty-nine patients (15%) had positive results on the ANCA test at baseline; 14 of these were positive for cytoplasmic ANCA and 15 were positive for perinuclear ANCA. At follow-up, vasculitis was ultimately diagnosed in 12 (41%) of these 29 patients. The following conditions were diagnosed in the remaining 17 ANCA-positive patients: rheumatoid arthritis flare (3 patients), undifferentiated connective tissue disease (4 patients), pulmonary fibrosis (1 patient), fever of unknown origin (1 patient), cancer (1 patient), myasthenia gravis (1 patient), multiple sclerosis (1 patient), polyradiculopathy of unknown origin (1 patient), usual interstitial pneumonitis (1 patient), sinusitis (1 patient), and neuropathy (1 patient). No diagnosis was made in 1 patient.

One hundred eleven patients underwent angiography, biopsy, or both. Among the 18 patients who had angiography, vasculitis was diagnosed in 7 (39%). One hundred five patients (53%), including 43 of the 51 patients in whom a vasculitis was ultimately diagnosed, underwent at least one biopsy. Skin biopsies and temporal artery biopsies were most commonly performed, and biopsies that were more invasive (including kidney and lung biopsies) were less frequently performed.

Clinicians' Final Diagnoses and the American College of Rheumatology Classification Criteria

Vasculitis was diagnosed in 51 patients (26%) (Table 2). The distribution of diagnoses in these 51 patients ranged from systemic to limited forms of vasculitis: Six had Wegener granulomatosis, 8 had polyarteritis nodosa, 8 had giant-cell arteritis, 6 had hypersensitivity vasculitis, and 23 had other types of vasculitis. In the remaining 147 patients, the diagnosis was other clinical entities that are included in the differential diagnosis of vasculitis, such as cancer, infections, and other connective tissue diseases (for example, systemic lupus erythematosus and rheumatoid arthritis). Of the 19 patients who died during the follow-up interval, 7 had vasculitis.

Of the 198 patients in the cohort, 69 (35%) met one or more sets of ACR classification criteria on the basis of initial clinical presentation. Among these were 38 (75%) of the 51 patients in whom vasculitis was ultimately diagnosed (Table 3) and 31 (21%) of the 147 patients in whom a condition other than vasculitis was diagnosed (Table 4). Fifteen patients with vasculitis met two or more sets of ACR vasculitis criteria. Thirteen patients with a diagnosis of vasculitis did not meet any ACR criteria (Table 3; patients 21, 30, 31, 41 to 44, and 46 to 51); 3 of these patients had vasculitis of the central nervous system. Of the 28 patients who were given specific diagnoses of Wegener granulomatosis, polyarteritis nodosa, giant-cell arteritis, or hypersensitivity vasculitis, 14 met the relevant ACR criteria sets for these conditions alone (Table 5). Of the 31 patients who did not have vasculitis but met ACR criteria, 14 met ACR criteria for giant-cell arteritis and 18 met ACR criteria for Wegener granulomatosis, polyarteritis nodosa, or both.

Operating Characteristics of American College of Rheumatology Vasculitis Criteria in Diagnosis

When ACR criteria for Wegener granulomatosis, polyarteritis nodosa, giant-cell arteritis, and hypersensitivity vasculitis were applied to the entire cohort, the sensitivities ranged from 50% to 100% and the specificities were between 89% and 92% (Table 6). With respective prevalences in the study sample of 3%, 4%, 4%, and 3%, the positive predictive values for Wegener granulomatosis, polyarteritis nodosa, giant-cell arteritis, and hypersensitivity vasculitis were 25%, 17%, 29%, and 29%, respectively. When the ACR criteria were applied only to the 51 patients with a final diagnosis of vasculitis, the positive predictive values of the ACR criteria for Wegener granulomatosis, polyarteritis nodosa, giant-cell arteritis, and hypersensitivity vasculitis increased to 40%, 29%, 75%, and 32%, respectively, as the prevalence of these conditions increased to 12%, 16%, 16%, and 12%, respectively.

Discussion

We examined the diagnostic operating characteristics of the 1990 ACR criteria for Wegener granulomatosis, polyarteritis nodosa, giant-cell arteritis, and hypersensitivity vasculitis in a prospective cohort of patients evaluated by university-based rheumatologists for a possible vasculitis. Seventy-five percent of patients who had a final diagnosis of vasculitis and 21% of patients who did not have vasculitis met one or more sets of the four ACR vasculitis criteria that we studied. When the ACR criteria were applied only to the 51 patients who had a final diagnosis of vasculitis, thereby increasing the prevalence of physician-diagnosed Wegener granulomatosis, giant-cell arteritis, polyarteritis nodosa, and hypersensitivity vasculitis, the positive predictive values ranged from 29% to 75%. However, when these criteria were applied to the entire cohort, regardless of the patient's final diagnosis, the positive predictive values of these four ACR criteria were low, ranging from 17% to 29%.

These results offer a different perspective on ACR classification criteria: that is, their performance as diagnostic criteria for rare conditions in usual clinical practice. Although the ACR vasculitis classification criteria were never intended for diagnostic purposes, as pointed out by Hunder and colleagues [5], clinicians often use these criteria, as they use other ACR criteria, to diagnose vasculitis [20]. This approach has some potential problems because the capacity of any diagnostic test or criteria set to identify a patient with a disease is better when disease prevalence is high than when disease prevalence is low [21, 22]. In our analyses, as expected, the positive predictive value of the ACR criteria improved as the prevalence of specific types of vasculitis increased. However, if these four ACR criteria were applied in a setting in which the prevalence of vasculitis is very low, such as a primary care or a population-based setting, the likelihood of identifying persons who meet ACR vasculitis criteria but who do not have vasculitis would be even higher than seen in our cohort.

Our findings indicate that application of ACR criteria in clinical care for diagnosis of vasculitis is not a good practice. Not all vasculitides have pathognomic diagnostic features. Although a biopsy is often helpful for diagnosing vasculitis, some tissue specimens yield nonspecific findings because of sampling of focal and segmentally distributed vascular lesions [1]. Thus, rheumatologists rely on some combination of clinical, laboratory, and biopsy (or angiographic) data, as well as “clinical gestalt,” to diagnose vasculitis in patients, but they are comfortable with imprecise diagnoses that often evolve over time.

The ACR criteria do not require biopsy or other invasive tests for classification of vasculitis, and some clinicians may use these criteria sets, along with the results of certain laboratory studies (such as ANCA tests), to establish specific diagnoses of vasculitis. However, pitfalls are associated with the use of ANCA tests [15, 23], including the fact that more than 50% of the positive cytoplasmic ANCA and perinuclear ANCA test results in our study occurred in patients who did not have a vasculitis. Moreover, our results indicate that many patients who do not have a vasculitis may meet ACR vasculitis criteria; this is a particular problem for clinicians who are making diagnostic or treatment decisions.

These findings may also have important implications for researchers who may not have the opportunity to see patients with vasculitis and may use ACR criteria differently than clinicians. Clinical epidemiologists or immunologists, for example, might use ACR criteria as a gold standard to characterize patients with vasculitis for studies of new serologic markers. This practice is undesirable for two reasons. First, meeting ACR criteria might suggest homogeneity of clinical presentation for each diagnosis. However, we found that patients with specific types of vasculitis may meet more than one set of ACR criteria, highlighting the heterogeneity of presentation and clinical overlap among these syndromes. Second, meeting ACR criteria may imply homogeneity in course or prognosis, but heterogeneity in course and outcome of patients with each specific type of vasculitis is usual. This phenomenon has also been observed in studies of patients who meet ACR criteria for rheumatoid arthritis [24-26] or systemic lupus erythematosus [27, 28]. Thus, epidemiologic or clinical studies of patients with vasculitis that use ACR vasculitis criteria should provide additional information, such as biopsy results, laboratory studies, symptoms, or even patients' actual diagnoses, so that clinician-readers may place the results into the context of patients seen in their practices.

Our study had several limitations. First, we used the clinician's final diagnosis of the patient as the gold standard in our analyses. This practice is not unusual in studies of patients with rheumatologic diseases. Indeed, the referring rheumatologist's diagnosis was used as the gold standard comparison in the development of the ACR vasculitis criteria [7]. Although occult vasculitis may have been missed because not all patients had biopsy or angiography, it would have been unethical to require all patients to undergo invasive procedures simply for our study. The rheumatologists probably arranged for these studies in the settings in which the procedure seemed to contribute to a diagnosis of vasculitis.

Second, we studied patients who were seen by university-based rheumatologists who practiced at two sites. This referral bias is difficult to avoid in studies of rare medical conditions such as vasculitis, but this setting simulates conditions experienced by rheumatologists in usual clinical practice. Even in a setting in which a high prevalence of vasculitis would be expected, however, the ACR vasculitis criteria performed poorly. Finally, the number of patients with physician-diagnosed vasculitis was small, as reflected by the wide CIs around our estimates of sensitivity and positive predictive value. Strategies to increase sample size (such as extending the enrollment period or conducting a multicenter study) were beyond the scope of the original study. Although the number of patients with physician-diagnosed vasculitis would be increased under those conditions, more patients with diagnoses other than vasculitis would, presumably, also be identified.

We conclude that the ACR criteria should not be used for diagnosis of vasculitis. Patients who do not have a vasculitis may meet ACR vasculitis criteria, and patients who have a specific type of vasculitis may meet more than one set of ACR criteria. Thus, we encourage clinicians to be cautious in their use of these criteria and to refer patients who may have a vasculitis to clinicians with expertise in evaluating and managing these syndromes. We also recommend that the ACR reappraise these criteria. In introducing the 1990 ACR vasculitis classification criteria sets, Hunder and colleagues [5] stated that “criteria that would differentiate patients with vasculitis from patients without vasculitis would constitute a possible second study to be conducted at a later date.” Given that the ACR classification criteria reach an audience that extends beyond researchers and rheumatologists, perhaps the time has come to conduct that study.

Dr. Allen: Division of Rheumatology, Allergy and Immunology, Duke University Medical Center, Box 3440, Durham, NC 27710. Dr. Pincus: Division of Rheumatology and Immunology, Vanderbilt University School of Medicine, 203 Oxford House, Box 5, Nashville, TN 37232-4500.