Process-Based Measures of Quality: The Need for Detailed Clinical Data in Large Health Care Databases

Computing Process-Based Measures

Process-based measures of health care quality can be constructed as follows. First, experts combine evidence from many research studies to create evidence-based guidelines on clinical practices. These guidelines summarize recommendations for the care of patients with a given illness. Patients who are eligible to receive medical care on the basis of a specific guideline are then identified. A criterion for quality is also created to determine which patients actually received medical care as recommended by the guideline. The number of patients who received guideline-related care is divided by the number of patients who are eligible to receive that care. Rates that describe the extent to which patients receive processes of health care as recommended by evidence-based guidelines are thus computed [3].

One problem is obvious: Patients do not always fit neatly into guidelines. For example, a patient may decline recommended health care or a provider may appropriately reject a guideline recommendation on the basis of circumstances or patient preferences. In both examples, the guideline-related criterion misclassifies the patient as receiving inappropriate care and therefore represents a measurement error. Although measurement errors cannot be eliminated, they can be minimized by including obvious exceptions to a guideline in the quality of care criteria [4]. Because some measurement errors are inevitable, rates that compare different providers or the same provider at different times must be used with appropriate caution when it is being determined whether the quality of health care differs.

Examples of Process-Based Measures

Many potentially useful process-based measures of quality exist because a sequence of many processes of health care must be successfully completed to improve health outcomes. Failure at any step in the sequence decreases the likelihood of improved outcomes. Therefore, processes of health care criteria are often constructed as a set, with each criterion relating to one of many steps in a sequence. Successive criteria in a set may apply to different subgroups of patients because the criteria closely follow a clinical pathway that incorporates a branching logic of sound clinical practice (for example, if a given clinical finding exists, then choose test or treatment A; if it does not exist, then choose test or treatment B). When the clinical pathway branches, different subgroups of patients are formed. Because each subgroup requires different therapeutic management, different quality criteria apply. Some examples of questions that can be answered by measuring the quality of various processes follow.

Accessibility of care. Do clinicians detect and treat treatable disease in their enrolled populations? For example, of patients with recent myocardial infarction and one to three diseased coronary arteries, what percentage has undergone percutaneous angioplasty or coronary artery bypass graft surgery? Measurement can be refined to minimize error by excluding patients who have common contraindications to either procedure.

Adequacy of collecting clinical data on patients. Do clinicians collect and document data that are critical to confirming the diagnosis and selecting appropriate treatment when taking the patient's history and examining the patient? For example, what percentage of patients who are scheduled for coronary artery bypass graft surgery has had a determination of left ventricular ejection fraction to assess risk status?

Accuracy of diagnosis. Are diagnoses missed? For example, of patients who attended primary care visits during the past year and cited probable depression on a screening questionnaire, what percentage was further evaluated or treated by the primary care physician or referred to a specialist? Are diagnoses supported by the information collected? For example, of patients in whom chronic obstructive pulmonary disease was diagnosed, what percentage had a productive cough that lasted 3 or more months for 2 consecutive years and dyspnea on exertion or a FEV₁ of less than 60%? Correct diagnosis is vital for both patient care and valid comparisons of data on health care outcomes. Valid comparisons of diagnosis-specific outcomes cannot be made if some providers misdiagnose or fail to diagnose illnesses.

Appropriate decision making on treatment regimens and self-care counseling. Do clinicians appropriately choose treatment regimens and counsel patients on self-care according to the presumed or provisional diagnosis and patient preferences? For example, what percentage of patients in whom peptic ulcer disease was diagnosed has been tested for Helicobacter pylori infection? What percentage of patients who tested positive for H. pylori infection is being treated with amoxicillin, tetracycline, metronidazole, or clarithromycin? This example demonstrates a sequence of two indicators that form a branching clinical pathway as previously described. Do physicians counsel patients on self-care when appropriate? For example, what percentage of persons with asthma who receive ongoing treatment has been instructed in the use of a peak flow meter for monitoring exacerbations of their disease?

Disease monitoring and therapy. Do clinicians adequately monitor the chronic illnesses of patients to permit adjustments of therapy that control symptoms, improve function, and prevent or detect and treat complications? For example, what percentage of insulin-treated diabetic patients who receive ongoing treatment is monitored for glycemia control at defined intervals with a glycosylated hemoglobin test or the equivalent?

Appropriate health promotion and disease prevention. Do persons in a target population have access to health promotion and disease prevention services that address each person's risk for disease? For example, of persons who are known tobacco smokers and are receiving ongoing treatment, what percentage has been advised to quit smoking in the past year?

Safe, timely, and effective implementation of tests, procedures, and therapy. Are tests, procedures, and drug protocols managed competently, safely, and in an appropriate time frame to improve outcomes? For example, of patients who arrive in the emergency department with chest pains of more than 30 minutes but less than 6 hours in duration and had confirmed acute myocardial infarction without specific contraindications to thrombolytic therapy, what percentage receives thrombolytic agents within 2 hours of arrival?

Communication with patients. Do patients obtain information and encouragement to mobilize their own resources to promote their health, and are they involved in decisions to the extent that they desire? For example, of patients who have had mastectomies for breast cancer, what percentage would answer “yes” to the question, “Were you involved less than you wanted to be in the decision to have your mastectomy?”

The foregoing list of health care processes is only a brief discussion of the number and complexity of processes that contribute to improvement of health and satisfaction of recipients and that can be measured. However, a practical problem hinders large-scale use of process-based measures because sufficient details on clinical data are difficult to obtain. The complexities of clinical practice cannot be captured because most existing large databases lack these details and therefore cannot create measures that are credible to clinicians. The value of process-based measures reinforces the urgency to continue enriching large health care databases by adding important clinical data.

Using Process-Based Measures

Various parties are interested in defining and measuring health care quality to serve their own purposes [5]. For example, provider groups need measurements to evaluate their own internal quality management. Regulators and accreditors may use comparisons of quality to stimulate internal quality management. Consumers and employers may compare quality and cost before choosing their plan or provider networks [6, 7]. These different purposes necessarily affect the choice of measurement method. Process-based methods serve better than outcome-based measures of quality when the following objectives are being considered (Table 1).

Determining how to improve care. When a provider group engages in internal quality management, outcome-based measures may indicate the need for improvement but do not reveal what is necessary to improve health care. Process-based measures identify specific processes of care that need to be changed and whether specific changes can improve those processes. For example, physicians in a group practice can measure how well they follow patients who had unexpected abnormal test results in order to improve their performance on early detection and treatment of disease [8].

Detecting opportunities to improve health care when provider comparisons show no differences in outcomes. When external organizations, such as accreditors and purchasers, compare quality among health care providers, their review may indicate that all providers achieve similar outcomes, thereby implying that none of them needs to improve. However, process-based measures may reveal specific areas in which all providers should improve because of failure to adopt effective new therapies. For example, the Health Care Financing Administration's Cooperative Cardiovascular Program revealed substantial deficiencies within study hospitals because of failures to treat patients with thrombolytic agents, β-blockers, and aspirin [9], although earlier comparisons had not revealed significant differences in the hospitals' adjusted mortality rates [10].

Requiring only short-term data collection. When short-term measures of quality are needed to make informed choices about providers or to intervene to improve health care, users cannot wait for years for data on important health outcomes to become evident. However, decision makers can obtain information in time to guide their choices if they use measures related to processes of care that are shown by previous research to improve long-term outcomes. For example, randomized, controlled trials involving persons with type 1 diabetes demonstrate that repeated monitoring of glycosylated hemoglobin levels improves the control of glycemia [11] and that improved control prevents microvascular complications [12]. Six years of patient follow-up was needed in this trial to demonstrate a difference in improved outcome. However, measurement of physicians' use of glycosylated hemoglobin tests for diabetics in the previous year could provide immediate information on the quality of medical care [13].

Attributing quality of care to specific providers. When external parties hold specific providers accountable for health care, particularly individual physicians, the specific contribution of the numerous professionals and organizations who provide health care is difficult to identify [14]. Tracking the providers of specific processes of health care is easier. Process-based methods can better identify a provider's unique contribution. For example, when RAND investigators compared patient mortality rates (computed at 30 and 180 days after hospital admission), any of the providers of hospital care, posthospital care in nursing homes, home care, or ambulatory care could have contributed to the observed differences in the rates. However, the investigators also used process-based methods in their study, and these methods specifically measured the performance of physicians and nurses in a given hospital [15].

Comparing providers who serve different kinds of patients. When providers are being compared (particularly on the basis of outcome-based measures), the varying risks for worse outcomes related to different characteristics of the patients being served should be considered. Unfortunately, the extent to which differences in outcomes can be attributed to various forms of clinical conditions, to different degrees of severity and complexity of illness, or to differences in patients' sociocultural attributes and personal preferences is often unknown. Rewarding providers by comparing inadequately adjusted outcomes gives the providers incentives to avoid sicker or more difficult patients because they have worse outcomes and could thus create the appearance of providing worse health care [16]. In addition, because patients are likely to leave providers who give worse care, such providers eventually lose patients who would have worse outcomes, thereby creating a false impression that outcomes for their patients are improving over time. Process-based measures match patients to specific health care processes that are indicated for given conditions. That is, different quality criteria apply to clinically different patient subgroups.

Comparing providers who serve few patients. When comparing performance among providers who serve few patients, a bad outcome among a few patients is more likely to be a chance event than evidence of poor health care. An insufficient number of patients poses a problem, especially for outcome-based measures, because differences in provider performance account for a small proportion of any difference in outcomes. Process-based measures compare low-volume providers better because smaller sample sizes are sufficient to distinguish differences in performance from those caused by chance. Mant and Hicks [17] computed the potential magnitude of required sample sizes by comparing the sensitivity of a measure on the basis of severity-adjusted hospital mortality rates for patients who had myocardial infarction (a popular measurement of the quality of care) with a measurement based on hospital use of thrombolytic agents, β-blockers, and aspirin for such patients. Using assumptions that favored the sensitivity of the mortality rates, they predicted deaths that would be averted by adopting processes of care. In a theoretical comparison of two community hospitals where all variables except the specific processes of care were assumed to be equal, Mant and Hicks estimated how many cases were needed to detect differences in the quality of health care. Even assuming perfect adjustment for differing patient risks for death, they estimated that 1 year of mortality data could detect with statistical significance a difference of 9% or higher in mortality rates, which is the equivalent of 40 lives saved per year in the better hospital. They estimated that the same difference in lives saved per year could be detected with 2 weeks' worth of process-based data.

Measuring health care to modify patient risk for disease. When health care is directed at reducing risk rather than treating established disease (for example, disease prevention), the goal is to reduce risks for worse outcomes in the long term. Therefore, no short-term change in outcomes is expected. Process-based measures can apply to services that change a patient's risk for a bad outcome (for example, administration of immunizations or cancer screening tests).

Creating Large Databases for Process-Based Measures of Quality

Process-based measures reveal short-term differences in quality among providers and the precise steps that must be taken to improve quality. The disadvantage of this method is that many process-based measures are needed for a comprehensive assessment. Furthermore, because they require detailed clinical data currently found only in medical records, the expense of abstracting records is a deterrent.

Fortunately, the increasing availability of clinical details in large health care databases is making comprehensive process-based measures of quality more possible, even for large patient populations. In fact, existing health care databases are better suited for process-based measurement than they are for outcome-based measurement because they capture details on services provided but little detail on outcomes achieved. The databases most commonly used in published studies of quality measurement are those derived from fee-for-service claims to the Medicare program, particularly the Medicare National Claims History File. This file captures billable services provided to a given beneficiary each quarter and associated diagnoses [18]. The capacity to merge this file with files that contain information on the providers of specific services and files that contain beneficiary characteristics and deaths makes this unique source of information valuable for measuring quality.

The Medicare fee-for-service databases have several advantages that would facilitate process-based measurements if similar databases were available for Medicare managed care and non-Medicare populations. These databases link services that are provided to a beneficiary in all settings of health care, regardless of whether the care is received in the beneficiary's state of residence. Services are linked to diagnoses and to identified providers and are available for analysis within months of the receipt of care.

Disadvantages of using the Medicare databases include the possible skewing of information to affect payments. Many important processes also are not captured in sufficient detail. For example, the results of diagnostic tests are not captured, although they may be implied in discharge diagnoses. Diagnosis coding often lacks sufficient detail or is inaccurate in distinguishing patient groups with different prognoses and indications for health care. Furthermore, information on drugs that are prescribed in ambulatory settings is lacking because fee-for-service Medicare has no drug benefit.

Patient-specific drug information is available in many managed care organizations that provide drug benefits. Databases that contain information on drugs are especially useful when based on the Universal Pharmacy Claims Form, which links patient, prescribing clinician, diagnosis, drug name and form, dose and frequency, and number and dates of refills. Managed care organizations are increasingly linking such data with traditional databases on service utilization to monitor the quality of health care. Innovators are beginning to include the results of laboratory tests and diagnostic procedures in merged health care databases. Such merged databases facilitate measuring the quality of many health care processes at a relatively low cost.

Other data that are valuable for process-based measures of quality cannot be easily added to health care databases and are best obtained by abstracting medical records until computerized patient records are widely used [19]. Key items of past and present medical and family history represent such data (for example, medical history and family history of colon cancer are important variables when measuring the appropriate use of colonoscopy). Data on the performance and results of physical tests are also useful (for example, blood pressure measurement or fundus examination through a dilated pupil).

Additional data that are valuable for process-based measures are currently obtained by patient surveys. Such data include patient acceptance of recommended tests and treatments; patient understanding of their medical condition and how they should participate in treatment programs; and patient opinions on the timeliness, convenience, and comfort of interactions with health care personnel [20] (as discussed by Lillard and Farmer in this issue). Some patient characteristics obtained by surveys (for example, a patient's preferred language, level of education, and social support mechanisms) are useful for stratifying patient groups for whom delivery of health care may be difficult. When comparing providers to stimulate improvements in the quality of health care, stratification on such patient characteristics is preferable because it reveals the specific patient groups for whom improvements are needed. In contrast, statistical adjustment for these factors conceals the varying results for different patient groups.

Conclusion

In summary, process-based measures of health care quality should rely on processes that have been shown by scientific evidence to produce improved outcomes and are congruent with patient preferences. Such measures are particularly useful when the objective is to improve the quality of health care, when all providers have equivalent outcomes that could be improved by adopting innovative processes of care, and when short-term measures are needed to compare low-volume providers and direct change in long-term outcomes. A barrier to wide-scale use of process-based measures is the difficulty in obtaining sufficiently detailed clinical data. Fortunately, large-scale, process-based measures are becoming more feasible because such clinical data are being included in large databases.