Search tips
Search criteria 


Logo of hsresearchLink to Publisher's site
Health Serv Res. 2007 February; 42(1 Pt 1): 1–6.
PMCID: PMC1955236

The Importance of Population-Based Performance Measures

Joseph S Ross, M.D., M.H.S. and Albert L Siu, M.D., M.S.P.H.

Crossing the Quality Chasm, the landmark report issued by the Institute of Medicine, identified six aims to guide quality improvement efforts in health care—safety, effectiveness, patient-centeredness, timeliness, efficiency, and equity—and recommended restructuring payment methods to align incentives to support these aims in the organization and delivery of health care services (Institute of Medicine 2001). The movement toward higher quality care has involved substantial leadership to create change, nationally and locally. Public and private stakeholders representing patients, physicians and other health care professionals, hospitals, health plans, employers, government, and regulators have collaborated in an effort to improve quality in the U.S. health care system. Many experts have argued that nationwide quality improvement will not take place without the establishment of national goals, common measures of performance, and transparency of information around quality and costs. As might be expected, early efforts of nationwide quality improvement have focused significant attention on the more reasonable and achievable of these tasks, the adoption of common measures of performance. Their adoption has been driven in part by the work of quality improvement organizations, such as the Institute for Healthcare Improvement, the National Quality Forum, and the National Committee for Quality Assurance, who have set performance measurement priorities and established consensus standards.

Performance measures typically evaluate the structure, process, or outcomes of care (Donabedian 1980, 1982, 1985) and are expected to be important, scientifically acceptable, usable, and feasible (McGlynn 1998, 2003b; Kerr et al. 2001). Moreover, performance measures should be actionable—outcome differences will result from altering processes of care under the control of health care professionals (Brook, McGlynn, and Cleary 1996)—and based on strong clinical evidence (McGlynn 2003a). Because the strongest clinical evidence is often only available for specific interventions in well-defined populations (e.g., aspirin for patients with cardiovascular disease, glaucoma screening for older adults), the earliest performance measures have evaluated arguably narrow aspects of care as an approach to improving quality, generally encouraging and monitoring adherence to evidence-based guidelines for specific disease management.

This more narrow approach to quality improvement is valuable for its ease, practicality, feasibility, reliability, and validity. However, there are limitations as well, and several papers in this issue of the journal illustrate some of these challenges. A paper by Weinberg et al. (2007) illustrates the complex course of an episode of illness, across health care settings. More narrow quality measures focus on performance of health care professionals in a single health care setting, “missing” the opportunity to examine overall care for an illness—with significant ramifications. Reporting on patients' experience with postsurgical care coordination across multiple health care delivery settings after undergoing unilateral knee replacement, Weinberg et al. (2007) found that patients reported problems with coordination across settings and between providers and themselves (the patients) which were associated with greater joint pain, lower functioning, and lower satisfaction with their care. More narrow process measures hold providers accountable for the quality of patients' care in an isolated health care setting and for transferring them to the next health care setting, but these providers are neither held accountable nor supported (with financial incentives) to coordinate care across settings.

A paper by FitzGerald et al. (2007) highlights the importance of not only coordinating care, but specifically aligning financial incentives across health care professionals and settings to improve quality of care. Aligning incentives is particularly important when the drive for higher quality and more efficient care in one health care setting, promoted by performance measures such as length of stay or procedure volume, might lead to lower quality and less efficient care in another setting. Reporting on the impact of the Short Stay Transfer Policy, a part of the Balanced Budget Act of 1997 that discounted hospital payments for 10 Diagnostic Related Groups (DRGs) for patients discharged early to a postacute care setting, FitzGerald et al. (2007) found the Short Stay Transfer Policy reduced the financial incentives to discharge patients early to postacute care. After enactment, there were lower rates of early discharge to postacute care settings for patients admitted for elective joint replacement or hip fracture surgery, without change in overall postacute care utilization. Essentially, the Short Stay Transfer Policy eliminated the financial incentive for hospitals to discharge patients early to postacute care and decrease length of stay, while receiving the full DRG reimbursement; thus ensuring that patients who would still benefit from hospital care were not “pushed out” early. This policy has the potential to improve quality of care for patients in both health care settings: in the hospital, patients are not transferred until it is clearly appropriate to do so; in postacute care, health care professionals are not over-burdened by caring for more severely ill or disabled patients who would continue to benefit from inpatient care.

A paper by Kahn et al. (2007) underscores the importance of examining quality of care more broadly. Establishing an association between improved processes of care and improved outcomes of care is a recognized challenge (Kerr et al. 2001; Leatherman et al. 2003; Williams et al. 2005), likely related to the specific, well-defined nature of measured processes of care. For instance, seven processes of acute myocardial infarction care within a hospital have been shown to be correlated, but process completion accounted for only a small amount of variation in outcomes (Bradley et al. 2006). Kahn et al. (2007) have taken a more holistic, albeit complicated, approach to quality measurement by considering 120 disease-specific process measures related to many different clinical conditions and recategorizing them to collectively represent six different domains of clinical care. Using instrumental variables, they demonstrated a significant relationship between better ambulatory processes of care and better health-related quality of life.

In addition to problems with current performance measures narrowly assessing only one or a few specific processes of care and failing to capture coordination of care or aligning financial incentives across health care settings, there are other limitations to consider. Because many are designed for specific clinical conditions, challenges arise in their application to more complicated patients with multiple co-morbid conditions (Tinetti, Bogardus, and Agostini 2004; Boyd et al. 2005) or to understudied populations of patients, such as children, the elderly, women, minorities, and patients with other chronic diseases (Heiat, Gross, and Krumholz 2002; Masoudi et al. 2003; Coca et al. 2006). More narrow measures are also limited in that they only measure what they are measuring and there is little evidence to suggest that improvements in measured processes of care correlates with improvements in unmeasured processes. Conceivably, health care professionals and settings may be “gaming the system,” focusing all of their attention on the measured processes of care to the detriment of other aspects of care. Finally, assigning responsibility for specific measures of care remains a challenge. For instance, currently, both primary care physicians and cardiologists are held accountable for patients being prescribed β-blockers postacute myocardial infarction (i.e., their care for this process is measured). However, which was responsible for providing this care? This same problem exists for health care settings—should the emergency department or inpatient ward be held responsible for providing antibiotics within four hours to a patient hospitalized for pneumonia?

Few process measures approach quality improvement broadly, using provider-specific population-based measures of quality care. Population-based measures have the advantage of aggregating patient-level quality to the provider- or system-level, allowing for the evaluation of provider- or community-level efficiency, socioeconomic, racial, and ethnic disparities, and coordination of care. One of the larger challenges in designing population-based measures is the assignment of patients to health care providers and providers to health care settings for process measurement. However, also in this issue of the journal, a paper by Bynum et al. (2007) demonstrates a successful method for assigning patients and providers for population-based performance measurement, allowing assessment of provider specific costs, continuity of care, and quality of care. Using Medicare claims and enrollment files to first assign enrollees to their predominant ambulatory physician and then to a hospital where that physician provided inpatient services (or where the plurality of that physician's patient panel received inpatient services), Bynum et al. (2007) assigned 96 percent of eligible enrollees to a specific physician and 94 percent of physicians to a specific hospital and found that, on average, two-thirds of medical admissions occurred at the patients' assigned hospitals and two-thirds of physician billings were by the assigned hospitals. Importantly, they found that estimates of risk-adjusted costs across physician groups in 1 year were highly predictive of costs in a subsequent year, suggesting not only the reliability of their measurements but also the potential to use these measurement to leverage more efficient health care delivery.

The choice of more narrow common process measures with a focus on individual patient care, targeted for specific clinical conditions, populations, and settings, was appropriate as an initial approach to improving quality and has improved care (Jencks, Huff, and Cuerdon 2003; Williams et al. 2005). However, there are important limitations to such narrowly defined quality measures. And despite their limitations, these measures are increasingly being tied to reimbursement and promoted by both quality improvement organizations (The Leapfrog Group 2006) and commercial health maintenance organizations and other health plans (Rosenthal et al. 2006) as part of incentive and reward, or pay for performance, programs. Moreover, the Center for Medicare and Medicaid Services is widely expected, and is being encouraged (Berwick et al. 2003), to adopt pay for performance for health care delivery settings and professionals. As we move forward, and before pay for performance “standardizes” the use of more narrow process measures, we need to develop quality measures which apply to all patients, capture the course of care across health care settings, and hold groups of providers accountable for health care quality, ideally organized to include a hospital and its affiliated postacute care and rehabilitation centers, ambulatory clinics, physicians, nurses and other health care professionals. The adoption of population-based measures is the next and a necessary challenge in our effort to improve health care safety, effectiveness, patient-centeredness, timeliness, efficiency, and equity.


  • Berwick DM, DeParle NA, Eddy DM, Ellwood PM, Enthoven AC, Halvorson GC, Kizer KW, McGlynn EA, Reinhardt UE, Reischauer RD, Roper WL, Rowe JW, Schaeffer LD, Wennberg JE, Wilensky GR. Paying for Performance: Medicare Should Lead. Health Affair (Millwood) 2003;22(6):8–10.
  • Boyd CM, Darer J, Boult C, Fried LP, Boult L, Wu AW. Clinical Practice Guidelines and Quality of Care for Older Patients with Multiple Comorbid Diseases: Implications for Pay for Performance. Journal of American Medical Association. 2005;294(6):716–24.
  • Bradley EH, Herrin J, Elbel B, McNamara RL, Magid DJ, Nallamothu BK, Wang Y, Normand SL, Spertus JA, Krumholz HM. Hospital Quality for Acute Myocardial Infarction: Correlation among Process Measures and Relationship with Short-Term Mortality. Journal of American Medical Association. 2006;296(1):72–8.
  • Brook RH, McGlynn EA, Cleary PD. Quality of Health Care. Part 2: Measuring Quality of Care. New England Journal of Medicine. 1996;335(13):966–70. [PubMed]
  • Bynum JPW, Bernal-Delgado E, Gottlieb D, Fisher E. Assigning Ambulatory Patients and Their Physicians to Hospitals: A Method for Obtaining Population-Based Provider Performance Measurements. Health Services Research. 2007;42(1, Part I):45–62. [PMC free article] [PubMed]
  • Coca SG, Krumholz HM, Garg AX, Parikh CR. Underrepresentation of Renal Disease in Randomized Controlled Trials of Cardiovascular Disease. Journal of American Medical Association. 2006;296(11):1377–84.
  • Donabedian A. Explorations in Quality Assessment and Monitoring. The Definition of Quality and Approaches to its Assessment. Vol. 1. Ann Arbor, MI: Health Administration Press; 1980.
  • Donabedian A. Explorations in Quality Assessment and Monitoring. The Criteria and Standards of Quality. Vol. 2. Ann Arbor, MI: Health Administration Press; 1982.
  • Donabedian A. Explorations in Quality Assessment and Monitoring. The Methods and Findings of Quality Assessment and Monitoring: As Illustrated Analysis. Vol. 3. Ann Arbor, MI: Health Administration Press; 1985.
  • Fitzgerald JD, Boscardin WJ, Hahn BH, Ettner SL. Impact of the Medicare Short Stay Transfer Policy on Patients undergoing Major Orthopedic Surgery. Health Services Research. 2007;42(1, Part I):25–44. [PMC free article] [PubMed]
  • Heiat A, Gross CP, Krumholz HM. Representation of the Elderly, Women, and Minorities in Heart Failure Clinical Trials. Archives of Internal Medicine. 2002;162(15):1682–8. [PubMed]
  • Institute of Medicine. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academy Press; 2001.
  • Jencks SF, Huff ED, Cuerdon T. Change in the Quality of Care Delivered to Medicare Beneficiaries, 1998–1999 to 2000–2001. Journal of American Medical Association. 2003;289(3):305–12.
  • Kahn KL, Tisnado DM, Adams JL, Liu H, Chen W-P, Hu FA, Mangione CM, Hays RD, Damberg CL. Does Ambulatory Process of Care Predict Health-Related Quality of Life Outcomes for Patients with Chronic Disease? Health Services Research. 2007;42(1, Part I):63–83. [PMC free article] [PubMed]
  • Kerr EA, Krein SL, Vijan S, Hofer TP, Hayward RA. Avoiding Pitfalls in Chronic Disease Quality Measurement: A Case for the Next Generation of Technical Quality Measures. American Journal of Managed Care. 2001;7(11):1033–43. [PubMed]
  • Leatherman S, Berwick D, Iles D, Lewin LS, Davidoff F, Nolan T, Bisognano M. The Business Case for Quality: Case Studies and an Analysis. Health Affair (Millwood) 2003;22(2):17–30.
  • Masoudi FA, Havranek EP, Wolfe P, Gross CP, Rathore SS, Steiner JF, Ordin DL, Krumholz HM. Most Hospitalized Older Persons Do Not Meet the Enrollment Criteria for Clinical Trials in Heart Failure. American Heart Journal. 2003;146(2):250–7. [PubMed]
  • McGlynn EA. Choosing and Evaluating Clinical Performance Measures. Joint Commission Journal on Quality Improvement. 1998;24(9):470–9. [PubMed]
  • McGlynn EA. An Evidence-Based National Quality Measurement and Reporting System. Medical Care. 2003a;41(1 suppl):I8–15. [PubMed]
  • McGlynn EA. Selecting Common Measures of Quality and System Performance. Medical Care. 2003b;41(1 suppl):I39–47. [PubMed]
  • Rosenthal MB, Landon BE, Normand SL, Frank RG, Epstein AM. Pay for Performance in Commercial HMOs. New England Journal of Medicine. 2006;355(18):1895–902. [PubMed]
  • The Leapfrog Group. 2006. “Incentive & Reward Compendium Guide & Glossary” [accessed on December 4, 2006]. Available at
  • Tinetti ME, Bogardus ST, Jr, Agostini JV. Potential Pitfalls of Disease-Specific Guidelines for Patients with Multiple Conditions. New England Journal of Medicine. 2004;351(27):2870–4. [PubMed]
  • Weinberg DB, Gittell JH, Lusenhop RW, Kautz CM, Wright J. Beyond Our Walls: Impact of Patient and Provider Coordination across the Continuum on Outcomes from Surgical Patients. Health Services Research. 2007;42(1, Part I):7–24. [PMC free article] [PubMed]
  • Williams SC, Schmaltz SP, Morton DJ, Koss RG, Loeb JM. Quality of Care in U.S. Hospitals as Reflected by Standardized Measures, 2002–2004. New England Journal of Medicine. 2005;353(3):255–64. [PubMed]

Articles from Health Services Research are provided here courtesy of Health Research & Educational Trust