Internists' Attitudes about Clinical Practice Guidelines

Methods

Questionnaire

We designed a practice guidelines questionnaire that sought information about 1) demographic and professional characteristics of responders, including their year of graduation from medical school, board certification, academic affiliations, hours per week devoted to patient care, practice type, practice setting, and the principal method of clinical reimbursement; 2) responders' ratings of their familiarity with and confidence in practice guidelines issued by ACP, various medical specialty societies, and other major health care organizations; 3) responders' attitudes regarding guidelines and their effects on medical care; 4) any change in responders' clinical practice during the last year as a result of guidelines; and 5) responders' ratings of the importance of practice guidelines and other sources of information for clinical decision making. The questionnaire was approved by the ACP Clinical Efficacy Assessment Subcommittee (CEAS) after it was pilot-tested with 80 primary care physicians affiliated with the Johns Hopkins Health Plan [25] and 95 volunteers at the 1991 ACP Annual Meeting.

Familiarity with, confidence in, and attitudes about guidelines were assessed using 5-point ordinal scales, with anchors appropriate to the judgment requested (for example, 5 = “very familiar” to 1 = “not familiar”; 5 = “great confidence” to 1 = “no confidence”). Estimated effects of guidelines on various aspects of medical practice were scored as “likely to decrease,” “to have no effect,” and “likely to increase”

Sample

We drew a stratified random sample of 400 associates, 1000 general internists, and 1200 internist specialists from ACP membership records. Members with specialty certification were oversampled because their response rate to previous ACP surveys was lower than that of other members. (Johnson White L. Personal communication.) The total sample of 2600 physicians represented 3.5% of the ACP membership and included internists from every state.

The sample size for the survey was based on assumptions of a response rate of 65% and a 2-point standard deviation on the 5-point ordinal scales that comprised most items in our survey. Under these assumptions, a sample size of 2600 gave us greater than 90% power to detect a 1-point difference for comparisons involving the smallest subspecialty groups (for which we expected about 100 responders).

Survey Procedure

The survey was mailed in December 1991, accompanied by a letter from the executive vice-president of ACP encouraging participation in the survey. Follow-up mailings were sent to nonresponders 1 and 2 months later, and data collection was terminated 6 weeks after the third mailing. Each physician was assigned a number that was placed on the cover sheet of mailed questionnaires. The same number was used to access information in an ACP membership database about physician year of graduation, specialty certification, practice location, and membership status. Cover sheets were removed from returned questionnaires after recording the identification number. In this way, we were able to identify nonresponders, compare demographic characteristics of responders and nonresponders, and preserve the anonymity of physicians during data abstraction and analysis.

Analysis

Data from returned questionnaires were double-entered and audited. Self-reported year of graduation and subspecialty certification were compared with information obtained from ACP databases, with κ statistics of 0.98 and 0.53, respectively. The low κ for subspecialty certification resulted from responders reporting a qualification that was not in ACP files. Many physicians join ACP before completing their specialty training, and ACP has not previously updated its files for specialty status more often than once every 3 years. Therefore, we used self-reported professional characteristics from our survey in our analysis. To compensate for oversampling of subspecialists and undersampling of Associate ACP members, the ratio of the proportion of Associates, generalists, and specialists in the final sample to their true proportion in the ACP membership was used to generate adjustment weights that were applied to all analyses. Questions answered by fewer than 90% of responders were not considered in the analysis.

Frequency distributions of responses to questionnaire items were examined before statistical tests were selected and applied. Physician characteristics were coded as dichotomous variables except for year of graduation, which was recoded into four categories. We then tested the statistical significance of associations between physician characteristics and dichotomous question responses (for example, impact compared with no impact on clinical practice) using the chi-square test. For items with 5-point response scales, we tested the statistical significance of differences between ratings for different physician subgroups using analysis of variance. Stepwise logistic and multiple linear regression models were used to explore relations between sets of physician characteristics and responses to individual questions. Given the multiple comparisons, differences in response distributions between physician subgroups were considered statistically significant at the P ≤ 0.005 level.

We developed an overall measure of physician attitudes toward guidelines by summing ordinal scale ratings regarding four positive views of guidelines (strength of agreement with statements that guidelines generally are good educational tools, unbiased syntheses of expert opinion, a convenient source of advice, and intended to improve quality of care) and the inverse of ordinal ratings regarding four negative views of guidelines (oversimplified or “cookbook” medicine, a challenge to physician autonomy, too rigid to apply to individual patients, and intended to cut costs). Possible scores ranged from 8 to 40, with 40 representing the most “positive” attitudes about guidelines. The internal consistency coefficient (Chronbach α) for this scale was 0.76.

Results

Description of Responders and Nonresponders

Of the 2600 physicians in our original sample, 35% returned questionnaires after the first mailing, 18% after the second mailing, and 8% after the third mailing. Eighty-seven physicians from the original sample were determined to be ineligible because of death (n = 5), retirement (n = 48), survey damage (n = 1), or no forwarding address (n = 33). Thus, we received completed questionnaires from 1513 (60%) of 2513 eligible physicians.

Characteristics of those who did and did not respond to the survey were compared using data available from the ACP membership file. Responders and nonresponders were similar in terms of their year of graduation from medical school, geographic location of practice, and prevalence of generalist and subspecialty certification in internal medicine (Table 1). In addition, the characteristics of responders were similar to those of the target population of College members, after adjustment for intentional oversampling in some strata (Table 2).

Familiarity with Guidelines

The percentage of responders reporting that they were familiar (4 or 5 on a scale of 1 = “not at all familiar” to 5 = “very familiar”) with the content of selected guidelines varied from 11% for ACP exercise stress test guidelines to 59% for National Cholesterol Education Program (NCEP) guidelines (Table 3). Ratings of familiarity with guidelines that have actually been published were significantly greater than ratings for a fictitious ACP guideline about computed tomography of the head (mean score, 1.8 of 5; P < 0.001 for comparison with mean score for any of the other guidelines), which was included in the survey to provide a measure of the degree to which familiarity scores might be inflated because of a general desire to appear knowledgeable. The responders who reported familiarity with this nonexistent guideline (7% of total) provide a baseline for interpretation of the familiarity reported for the other guidelines.

Subspecialists were more likely to report familiarity with guidelines pertaining to their own subspecialty than with those pertaining to other subspecialties or to general medical practice. For example, board-certified cardiologists reported significantly higher awareness than did general internists of cardiology-related guidelines issued by ACP (mean, 2.7 compared with 2.0; P < 0.001), the American College of Cardiology (ACC; mean, 4.1 compared with 2.5; P < 0.001) and the NCEP (mean, 4.3 compared with 3.7; P < 0.001). Similarly, gastroenterologists reported greater familiarity than did general internists with colon cancer guidelines issued by ACP (mean, 4.1 compared with 3.0; P < 0.001) and the American Gastroenterological Association (AGA; mean, 4.6 compared with 2.9; P < 0.001). Compared with subspecialists, general internists were somewhat more familiar with ACP's Common Diagnostic Tests (mean, 2.8 compared with 2.5; P < 0.001), the United States Preventive Services Task Force's (USPSF) Guide to Clinical Preventive Services (2.6 compared with 2.0; P < 0.001), and the American Cancer Society's Cancer-Related Check-up (3.6 compared with 3.2; P < 0.001).

In our multivariate model, familiarity with the ACP guidelines included in our survey was independently associated with two physician characteristics: with familiarity being less for recent medical school graduates than for older graduates and less for subspecialists than for generalists. Recent graduates were more familiar with USPSF guidelines than were older graduates (P < 0.001) after adjusting for all other physician characteristics.

Confidence in Guidelines

Figure 1 shows the percentage of survey responders who indicated high or very high confidence in guidelines that various organizations might issue. Although 82% of College members gave high ratings to ACP for guidelines that it might produce (mean score, 4.2), only 6% gave such ratings to guidelines of the national Blue Cross and Blue Shield Association (mean score, 2.1). Confidence in guidelines from any of three specialty organizations (ACP, 4.2; ACC, 3.9; AGA, 3.7) was higher than in those that might come from the American Medical Association (AMA; 3.3) or selected federal agencies (National Institutes of Health, 3.5; USPSF, 3.0; Agency for Health Care Policy and Research, 2.5; P < 0.005 for ACP compared with any other organization). On average, specialists gave the highest confidence ratings to guidelines from their specialty organizations, and nonspecialists gave the highest ratings to the ACP guidelines. Among those unfamiliar with the specific guidelines included in our survey, confidence ratings were highest for guidelines from the American College of Physicians, followed by the American College of Cardiology, American Cancer Society, American Gastroenterological Association, National Institutes of Health, and USPSF.

In our multivariate model, guidelines from the National Institutes of Health received higher mean confidence ratings from subspecialists than from general internists. Higher confidence in USPSF guidelines was associated with Associate membership, recent graduation, general practice, and reimbursement by fixed salary. Confidence in guidelines that might be issued by the Agency for Health Care Policy and Research (AHCPR) was higher among Associates than among Members, Fellows, or Masters and was higher among internists who were not in private practice.

Attitudes about Guidelines

When asked to rate the strength of their agreement with various statements about guidelines (Table 4), most responders agreed or strongly agreed that guidelines are good educational tools (64%) and convenient sources of advice (67%). Most also felt that guidelines are developed to improve the quality of health care (70%) and are likely to have this effect (65%). However, a sizable minority viewed guidelines less favorably: for example, as oversimplified or “cookbook” medicine (25%), too rigid to apply to individual patients (24%), and a challenge to physician autonomy (21%). Less than one fifth of responders expected guidelines to reduce the number of malpractice suits brought against internists or the prevalence of defensive medical practices, whereas two thirds of responders expected guidelines to be used in disciplinary actions against physicians and one third felt that guidelines are likely to decrease physician satisfaction with the practice of medicine. Although many responders felt that guidelines are meant to decrease health care costs (61%), few (22%) expect them to do so. Indeed, 43% of ACP members predict that guidelines will actually increase total health care costs.

Several physician characteristics were independently associated with particular attitudes toward guidelines (Table 4). Salaried internists were more likely than fee-for-service clinicians to feel that guidelines are good educational tools and a convenient source of advice, whereas fee-for-service clinicians were more likely to feel that guidelines are intended to decrease health care costs, are too rigid to apply to individual patients, and are unlikely to improve the quality of health care. Physicians who saw patients for more than 20 hours per week were more likely than those who spent less time in clinical practice to consider guidelines to be too rigid for use with individual patients and a challenge to physician autonomy.

On the summary score for general attitudes toward guidelines, significantly higher ratings (positive attitudes) were scored for non-private practice than for private practice physicians, for salaried than for fee-for-service physicians, and for physicians who see patients for less than 20 hours per week than for those seeing patients for 20 hours or more per week.

Effect of Guidelines on Personal Practice

Figure 2 presents responder ratings of the impact that various sources of medical information have had on their own clinical decision making. Sixty percent of responders indicated that clinical practice guidelines had some influence on their decision making, and 16% indicated that guidelines had a major effect, which is less than the percentage for colleagues, continuing medical education (CME), or textbooks. Only 11% of responders said that their practice had changed during the last year as a result of an ACP guideline, and 18% reported that a change had occurred as a result of any guideline. In a multivariate analysis, ACP Associates were more likely than other ACP members to report that practice guidelines have an impact on their practice.

Discussion

Among responders to our survey, familiarity with published practice guidelines of major health organizations varied substantially. The greater familiarity of ACP members with National Cholesterol Education Project (NCEP) (59% of responders) and American Cancer Society guidelines (48%) may reflect the relevance of coronary heart disease prevention and cancer detection to internists, the promotion of NCEP and American Cancer Society guidelines in the scientific and lay press over many years, the effect of intensive federally funded physician education efforts, and the strong interest of the pharmaceutical industry in physician awareness of opportunities to use cholesterol-lowering agents [26-28]. As expected, subspecialists were more familiar with guidelines from their own subspecialty organization than with an ACP guideline on the same topic. Internists' familiarity with the USPSF Guide to Clinical Preventive Services was low, even though the content of these guidelines is relevant to all internists [29]. These guidelines have been available for fewer years than either NCEP or American Cancer Society guidelines.

Our findings suggest that physicians' confidence in guidelines issued by different organizations is strongly related to the physicians' affiliation with the organization. Cardiologists and gastroenterologists, in particular, reported greatest confidence in guidelines issued by their respective subspecialty organizations, whereas generalists rated ACP guidelines most highly. The striking contrast between confidence in ACP compared with Blue Cross/Blue Shield guidelines highlights the importance of the issuing organization on confidence, particularly because these two organizations have worked together closely on guideline development over the past decade, and the Blue Cross/Blue Shield guidelines for diagnostic testing are the same as ACP guidelines [30, 31]. This finding suggests that guidelines issued by an insurer may engender suspicion based on the presumption that they are intended to decrease costs and may have little to do with disagreement about their content or method of development.

Our results reveal relatively low levels of confidence in guidelines that may be issued by the AHCPR, particularly among more experienced clinicians. Because the first Agency practice guideline was released after this survey was done, the low confidence in AHCPR guidelines must be caused by factors other than the content of these guidelines, such as concerns about the potential uses of guidelines by the federal government. Responders may also have been unfamiliar with this organization. The efforts of AHCPR to collaborate with major professional organizations may therefore be important.

The reported high confidence in ACP guidelines by ACP members may reflect a reluctance to criticize the organization sponsoring the survey. However, we took measures to preserve the confidentiality of responders and to encourage frank responses to questions about ACP and other organizations. Further, other surveys have shown that ACP guidelines are viewed positively by physicians who are not ACP members [25] and by representatives of other professional organizations [32].

We found that most ACP members have positive attitudes about guidelines in general, perceiving them to be of educational value and likely to improve the quality of patient care. This finding appears to be at odds with the prevailing tone of commentaries, letters, and editorials appearing in many medical journals and newsletters [33-36]. It is possible that College members have developed a favorable view of guidelines through familiarity with the Clinical Efficacy Assessment Project or that guidelines have been positively presented to members through College publications. However, the attitudes of ACP members may not be representative of the attitudes of other physician groups. We found that salaried and non-private practice physicians, who may be over-represented in the ACP membership, had more positive attitudes toward guidelines.

Despite identifying many favorable views about guidelines, our survey also identified several misgivings. Many responders anticipate that guidelines may threaten physician autonomy, figure in disciplinary actions against physicians, and decrease satisfaction with the practice of medicine. Many also believe that guidelines may increase health care costs, and few expect defensive medical practices to decrease. These divergent views suggest that guidelines are perceived to have many different purposes and that their effects on a complex health care system are expected to be mixed [18, 37-40].

Internists' familiarity with and confidence in guidelines were affected by subspecialty status, whereas specific attitudes about guidelines did not differ between generalists and subspecialists. Attitudes about guidelines were influenced by how physicians are paid and whether or not they are in private practice. Physicians who seek fee-for-service environments may have particular concerns about loss of autonomy or the economic context of practice may affect physician reactions to guidelines [41].

Associate ACP members, more recent graduates from medical school, appeared less concerned about rigidity of guideline recommendations and were more likely to expect guidelines to reduce the number of malpractice suits brought against physicians and the prevalence of defensive medical practices. We do not know whether the more positive attitudes of recent graduates will persist over time.

Most survey responders reported that clinical practice guidelines had some effect on their decision making, although fewer than 20% rated this as major. This corroborates that guidelines to date have had but modest effect on clinical practice [12, 13, 42]. For guidelines to improve health care in North America, they must be both valid and widely followed [22, 43]. Our observation that internists report greatest confidence in, but not necessarily familiarity with, guidelines from professional organizations with which they are affiliated, suggests that factors other than validity may influence which guidelines are followed. Greater efforts are thus needed to build physicians' confidence in the guidelines issued by some organizations and to promote physicians' awareness of guidelines felt to be particularly important.