Robust performance measurement will likely require multiple measures, including those of outcomes, processes and structures.30
For mortality to provide an adequate measure of performance, either to compare institutions or to track quality of care over time, a focus on single conditions or procedures will be required. Such a focus would facilitate the recognition of explanations for poor performance (e.g., delivery of established processes of care known to reduce mortality among patients with a particular condition) and permit more accurate risk adjustment than would otherwise be possible with overall hospital mortality. In some cases, administrative data alone may be sufficient for risk adjustment.31
In others, adequately adjusting for risk may require combining administrative data with select clinical information. In cardiac surgery, for instance, supplementing administrative data with the coronary disease pattern, angina class and left-ventricular function provides an acceptable measurement of risk-adjusted mortality.32
In other cases, more comprehensive collection of clinical data has been considered a worthwhile investment, as with the US Department of Veterans Affairs' National Surgical Quality Improvement Program.33
The mere avoidance of death conveys little about the quality of care related to most conditions and procedures. Thus, outcomes-based performance measures will have to include important measures of morbidity, such as surgical site infections, key nosocomial infections and functional status (e.g., 30 days after acute stroke), to name a few. Measuring functional status would require substantial investment, but it would serve 2 important goals. First, it would reduce the bias in purely hospital-based measurement owing to variations in lengths of stay (e.g., hospitals would not be rewarded for simply discharging patients to die elsewhere). Second, it would force hospitals to help coordinate follow-up care and decrease the fragmentation that plagues the management of so many patients with serious health problems.
The timely delivery of medicine and other processes of care with a strong relation to important outcomes (e.g., established therapies for acute myocardial infarction) have a clear role to play in performance measurement, but process measures must also be chosen with care. Variations in patient eligibility criteria (or in clinical documentation of such variation) can introduce spurious variations in performance. Overemphasis of process measures can induce “playing for the test” (i.e. hospitals respond to process-based performance measurement by focusing their efforts on the processes of care being measured, regardless of the impact on other aspects of quality), as has likely happened with the focus on time to administration of antibiotics for patients with pneumonia at hospitals in the United States.34
Finally, some structural measures, though relatively blunt, may provide valid performance measures. Patient volume represents a prominent structural measure of quality (since higher volumes tend to correlate with improved outcomes, especially for surgical procedures35
), but others might include nurse staffing ratios,36
the organization of intensive care units37
and possibly measures related to hospital governance.38
Advocates of hospital standardized mortality ratios acknowledge that hospitals should adopt a “dashboard” or “scorecard” approach, for which mortality is just one component. However, mortality will always overshadow other measures. The knowledge that a local hospital has exemplary rates of adherence to hand hygiene, for instance, will do little to assuage the public in the face of an apparent excess of deaths. Composite measures have found some application (e.g., a score that combines mortality with performance on several target processes of care). However, just as with composite outcomes in clinical trials, care must be taken to avoid large gradients in importance among components.39
For example, a performance measure for cardiac care that combined mortality after myocardial infarction with various process measures (e.g., rates of smoking-cessation counselling) would raise concerns that differences in performance could be perceived to be associated with mortality but would actually be driven by results for less important components.