Performance Characteristics of a Rapid HIV Antibody Assay in a Hospital with a High Prevalence of HIV Infection

Methods

Between 1992 and 1994, we did a serosurvey for HIV infection at the Bronx-Lebanon Hospital Center, a general hospital in New York City that has one of the highest prevalences of HIV-seropositive patients in the United States [8]. Patients were ineligible if they were younger than 18 and older than 44 years of age, were known to be infected with HIV, had sought HIV testing at the hospital, had been admitted for conditions related to the acquired immunodeficiency syndrome (AIDS), were too ill to complete an interview, had been admitted to services where interviews or phlebotomy were not approved or other serosurveys were underway (intensive care, hemodialysis, psychiatry, detoxification, obstetrics, pediatrics, or isolation inpatient services; urgent care and trauma emergency triage categories), had previously been offered HIV testing by nonstudy staff during admission, or did not speak English or Spanish. We approached 761 patients in the emergency department and 1298 inpatients. In these groups, 522 and 1227 patients were eligible, respectively. Forty-seven percent of the eligible patients in the emergency department (n = 246) and 48% of the eligible inpatients (n = 591) agreed to HIV testing through an approved consent process. Of all patients who agreed to testing, 98% (n = 822) also agreed to be interviewed about risk behaviors for HIV. No data were available on the prevalence of HIV, risk behaviors for HIV, and the medical characteristics of patients who were ineligible for the study or declined to participate.

A trained technician tested fresh serum specimens using the Genie HIV-1 and HIV-2 assay (Genetic Systems, Seattle, Washington), a rapid, synthetic peptide enzyme assay that is used to measure highly conserved regions of HIV-1 and HIV-2 transmembrane glycoproteins and has been approved for investigational use. This assay is done in 10 minutes and does not require special equipment. Serum specimens are mixed with diluent, pipetted into test tubes, agitated manually, poured into cartridges, and washed with reagents. Color intensity is rated by sight on a scale of 0 to 4+; specimens with values of 1+ to 4+ are considered reactive. After this assay was done, a technician who was unaware of the results of the rapid assay tested serum specimens twice more using a licensed, whole viral lysate nonrapid assay (Abbott Laboratories, Abbott Park, Illinois) for all viral epitopes. Samples that were reactive repeatedly were confirmed by Western blot assay for HIV-1 (Cambridge Biotech, Worcester, Massachusetts) using established criteria [9]. Patients were not notified of the results of the rapid test and were asked to return for counseling 2 to 3 weeks after testing, at which time they were notified only of the results of the nonrapid tests. Using 1) the Genie HIV-1 and HIV-2 assay, 2) a nonrapid whole viral lysate assay [Genetic Systems] for all viral epitopes of HIV-1 and HIV-2, and 3) the Cambridge blot assay, technicians at the Centers for Disease Control and Prevention (CDC) (who had had extensive experience with rapid assays and were unaware of hospital assay results) retested frozen serum specimens from patients whose rapid and nonrapid assay results at the hospital were discordant. The interpretive criteria used by the hospital for the blot assays were also used in these retests.

We calculated the sensitivity, specificity, positive and negative predictive values, and exact confidence intervals for the rapid test using the results of tests done at the hospital. The standard algorithm of nonrapid assay and blot results was the gold standard for diagnosis of HIV infection [10, 11]. Results were classified as true positive (reactive on rapid assay, reactive on nonrapid assay, and positive on blot), true negative (nonreactive on rapid assay and nonrapid assay), false positive (reactive on rapid assay and nonreactive on nonrapid assay or blot), or false negative (nonreactive on rapid assay, reactive on nonrapid assay, and positive on blot). Patients with indeterminate results on blots were excluded from calculations. Using a two-tailed significance level of P = 0.05, proportions were compared using uncorrected chi-square tests, and medians were compared using Wilcoxon rank-sum tests [10].

Results

Of 837 patients tested, 45 (5.4%) were infected with HIV-1 according to nonrapid assay and blot assay results from the hospital. Two patients had indeterminate blot results. About half of all patients (57.5% [481 of 837]) and fewer than two thirds of infected patients (64.4% [29 of 45]) returned for counseling. Results of serum testing were not significantly associated with rates of return for counseling (64.4% [29 of 45] of infected patients and 56.9% [450 of 790] of uninfected patients returned for counseling; P > 0.2) or timing of return (infected patients returned for counseling a median of 28 days later [range, 6 to 790 days]; uninfected patients returned a median of 20 days later [range, 3 to 634 days]; P = 0.1). Patients from the emergency department were more likely than inpatients to return for counseling (156 of 246 [63.4%] patients from the emergency department compared with 325 of 591 [55.0%] inpatients returned; P = 0.03). Patients from the emergency department also returned for counseling when initially scheduled more often than did inpatients (patients from the emergency department returned a median of 16 days [range, 3 to 597 days] after testing compared with a median of 22 days [range, 6 to 790] for inpatients; P < 0.05), although many inpatients received counseling before discharge.

Results of rapid and nonrapid assays and blots done at the hospital were discordant for nine patients: Seven had specimens that were weakly reactive on rapid testing (results of 1+ to 2+) and nonreactive on nonrapid assays below a borderline range of an optical density to cutoff ratio of 0.8 to 1.0 (Table 1). Results of rapid assays done at the hospital did not agree with those done at the CDC for six of these nine patients. One additional patient (patient 444) had a weakly reactive specimen on rapid assay, a reactive specimen on nonrapid assay, and an indeterminate blot result at the hospital; at the CDC, the patient had a nonreactive specimen on nonrapid assay and an indeterminate blot result (p24 band only by Cambridge blot assay). After the tests were repeated at the CDC, the result was positive by Cambridge blot assay (p24, gp41, p50, p66, gp120, and gp160 bands) but indeterminate by Organon Teknika assay (Durham, North Carolina) (gp120 and gp160 bands only) (Table 1). Patient 444 was retested with the Cambridge blot assay at the hospital 16 months later and was classified as infected according to standard criteria [9].

The rapid assay had a sensitivity of 1.00 and a specificity of 0.991 (Table 2). Neither variable differed appreciably among demographic and HIV risk characteristics, including birthplace (98.6% [355 of 360] of foreign-born persons were born in the Caribbean, Latin America, or Africa), injection drug use, and presence of a positive result on a serologic test for syphilis. Specificity varied from 0.946 among the first 100 persons tested to 0.997 among those tested later. The overall positive predictive value for this sample was 0.865; it varied from 0.583 among the first 100 persons tested to 0.95 among those tested later. This difference resulted largely from variations in specificity (Table 2). Patients who did not report risk behaviors for HIV had the lowest positive predictive value (0.667), largely because they had a lower prevalence of HIV infection. Overall negative predictive value was 1.00.

Discussion

The sensitivity, specificity, and positive and negative predictive values of the rapid assay were excellent for fresh serum specimens in a high-volume clinical laboratory of a hospital that sometimes operates under less-than-ideal conditions and has a high prevalence of patients with HIV infection. Peptide-based assays tend not to detect strains of HIV that diverge from common North American strains as well as whole viral lysate assays did [12]. However, this peptide-based rapid assay yielded no false-negative results among the 19% of infected patients who had serologic evidence of such divergent strains [13] or among foreign-born patients. The rapid assay also produced excellent results for patients with histories of injection drug use or reactive results on a serologic test for syphilis, which are two characteristics associated with relatively high rates of false-positive results for nonrapid assays [14, 15]. Rapid assays may be inaccurate for several reasons, including indistinct reaction end points that inexperienced technicians may find difficult to read [16]. Performance improved after the hospital technician had received further training on reading end points and dilution technique during testing of the first 100 specimens.

Performance of the rapid assay used in this setting was similar to that in evaluations involving more experienced technicians, reference laboratories, manufacturer sponsorship, frozen serum specimens, and populations with a lower seroprevalence of HIV: The assay's sensitivity was 0.995 to 1.00, its specificity was 0.995 to 1.00, its positive predictive value was 0.986, and its negative predictive value was 1.00 [16-18]. This assay did as well as the first rapid test licensed in the United States for diagnostic use—the Single Use Diagnostic Systems assay (SUDS; Murex Corp., Norcross, Georgia). A recent evaluation [19] of SUDS found a sensitivity of 1.00 (95% CI, 0.888 to 1.00), a specificity of 0.995 (CI, 0.989 to 0.998), a positive predictive value of 0.880 (CI, 0.677 to 0.968) in a clinic with 2.9% prevalence, and a negative predictive value of 1.00 (CI, 0.993 to 1.00). The fairly wide CIs seen in that evaluation and in our study emphasize the need to evaluate rapid assays in large populations that would benefit from this technology, such as that of pregnant women presenting for prenatal care during labor, patients in emergency departments, clients of drug treatment programs, clinic patients with sexually transmitted diseases, persons in street outreach programs, and organ donors.

Accurate rapid assays offer advantages to patients and providers that may improve the acceptability of counseling and testing programs. Because the sensitivity and specificity of licensed rapid assays are similar to those of licensed nonrapid assays, a negative rapid assay does not require confirmation with blot assays or other methods. Therefore, seronegative patients can learn their serostatus shortly after phlebotomy. Because rapid tests allow counseling to be done by the same provider before and after testing, the continuity, consistency, and confidentiality of counseling may be enhanced; this may encourage the adoption of risk-reducing behavior. Eliminating delayed counseling for seronegative patients may also yield important public health and economic benefits in the United States, where most tested persons are seronegative [20]. The results of reactive rapid assays, like those of reactive nonrapid assays, require confirmation by nonrapid methods. Nevertheless, notifying patients about a preliminary rapid assay result may encourage the early adoption of risk-reducing behaviors and increase rates of return for confirmatory test results. This, in turn, could hasten diagnosis and delivery of medical and social services for infected persons. Because of their excellent sensitivity, rapid assays can also be valuable when prompt preliminary serostatus information is needed for clinical management or for screening organ donors.

Appendix

Additional members of the CDC-Bronx-Lebanon HIV Serosurvey Team are Stephen Bekoe-Tabiri, Franklin Benjamin, Stere Carniciu, Carlos Chavaria, Vicsa Chiriboga, Maura Disla, Auria Fernandez, Camelia Ganea, Gilbert Martinez, Reineisse Parsons, Felipe Rosario, William Rothman, Wanda Santos, Rita Shulman, and Yvette Vasquez from Bronx-Lebanon Hospital Center and Karen Brown, Faye Cowart, Avis Daugharty, Marty Edeline, Ernest Lewis, Brent McRae, Vincent Raimondi, Larry Wells, Roy Winter, and Patrick Minor from the CDC.

From the Centers for Disease Control and Prevention, Atlanta, Georgia; and the Bronx-Lebanon Hospital Center, Bronx, New York.

Ms. Olivo and Dr. Ernst: Bronx-Lebanon Hospital Center, 1276 Fulton Avenue, Bronx, NY 10456.

Mr. Schable: Division of AIDS, STD, and TB Laboratory Research, Mailstop D-12, Centers for Disease Control and Prevention, 1600 Clifton Road, Atlanta, GA 30333.

Drs. Weber and Janssen: Division of HIV/AIDS Prevention, Mailstop E-46, Centers for Disease Control and Prevention, 1600 Clifton Road, Atlanta, GA 30333.