Prior studies of variation in hospital resource utilization that focus only on expired patients have been interpreted to mean that hospitals with low resource utilization should be regarded as performance benchmarks. Our findings agree with previous investigators that such benchmarking based only on expired individuals should be viewed with caution,29
particularly when benchmarking is extrapolated beyond expired individuals.4–6
First, and most importantly, this study identified substantial variation among the 6 study hospitals on adjusted mortality rates in the Looking Forward cohort, particularly when observed for 180-days after the initial admission. This health outcome variation also was inversely correlated with resource utilization variation, meaning that hospitals with higher adjusted resource utilization within 180 days after an initial hospitalization also had lower adjusted mortality.
Second, the patterns of resource utilization across hospitals were not the same between the Looking Forward and Looking Back cohorts; the only consistent pattern was that site 6 had the highest level and site 4 had the lowest level of resource utilization in both cohorts, when measured by either utilization measure. However, changes in rank order occurred among sites that did not significantly differ from each other; simple reporting of means without accounting for significant differences, while simpler for general audiences, may result in incorrect assumptions that sites truly differ from each other on utilization measures. Further, the study hospitals varied considerably in the size of the difference between risk-adjusted utilization measures derived from all patients and measures derived only from expired patients.
The 1.5-fold difference across the six study hospitals in risk-adjusted 180-day mortality among elderly Medicare patients hospitalized for HF challenges the notion that studies of expired patients provide valid and useful information on hospital efficiency, which should be measured by both resource utilization and health outcomes. Although prior studies have demonstrated mortality variation across hospitals for HF patients,7
we also found negative correlations between measures of resource utilization and 180-day mortality across the study hospitals. While we do not intend to suggest that this correlation implies a causal relationship between more resources and better outcomes, it does suggest a need for further work to explore how care processes and resource utilization during an initial hospitalization and subsequent visits influence health outcomes. Although hospitals with excellent adherence to evidence-based process measures30
have slightly lower risk-adjusted mortality than hospitals with poorer adherence,7, 31
these widely accepted process measures are unlikely to drive the substantial differences in resource utilization that we observed across teaching hospitals in a single state.
The authors of prior studies of variations in hospital resource utilization have acknowledged that utilization must be weighed with outcomes to assess efficiency, the common practice of restricting analyses to expired individuals (which is represented with our “Looking Back” approach) ignores outcome differences and overlooks the real possibility that resource utilization influences outcomes. The relationship between hospital efficiency and quality of care is complex,32–36
and focusing on expired individuals is likely to be overly simplistic. Appropriate estimation of the value of health care spending requires assessment of potential outcome differences, and cannot be done with a “Looking Back” approach. We believe that future studies should use the “Looking Forward” approach to ensure that important outcomes are not missed. Furthermore, clinicians have very limited ability to identify patients who are destined to die within six months and selectively withhold health care resources from those patients.37–39
Although studying only expired patients is expedient due to human subject protection issues that apply only to living individuals,40
a better solution is to study databases that include all individuals and to not ignore health outcomes.
The methods we employed differed in several ways from the methods used in prior studies of variations in hospital resource utilization, but in most cases the changes in methods strengthened the study. Notably, we examined patients with a principal diagnosis of HF, whereas prior studies included patients with a principal or secondary diagnosis of HF. We chose to be more restrictive in order to enhance the clinical homogeneity of the study cohort, since resource utilization patterns are likely to be driven by the principal diagnosis. For instance, use of resources to care for a patient who is hospitalized for hip fracture will differ from the use of resources to care for a patient who is hospitalized for HF, even if the hip fracture patient receives some treatment for HF. Similarly, we excluded patients whose clinical characteristics were likely to skew utilization patterns. Transfer11–14
and transplant patients41–43
often have unmeasured severity of illness beyond what can be captured by diagnosis codes or comorbid conditions.44
Hospitalizations associated with surgery incur additional resource utilization and convalescence that occurs with surgical procedures. Future studies should exclude these types of patients, since these types of patients can vary substantially across hospitals. Of note, although the proportion of patients in the excluded categories varied substantially across hospitals, sensitivity analyses that included these patients also found substantial health outcome variation between sites that were inversely correlated with resource utilization variation.
We also expanded on the risk-adjustment methodology used by prior studies of variations in hospital resource utilization,1–3
which only adjusted for age, gender, ethnicity, and the presence of 12 chronic conditions. Our regression models adjusted for age; gender; ethnicity; 21 comorbid conditions; dual Medicaid eligibility (to partially account for socioeconomic status); and admission year (to account for secular trends in clinical practice). In addition, we performed sensitivity analyses that adjusted for selected clinical laboratory values as well. Risk-adjustment methods using administrative data are subject to potential biases from unmeasured risk factors and other differences in care.45
Although the risk-adjustment methods we used cannot capture all differences across HF patients at different hospitals, we use a comprehensive list of covariates that are similar to other validated risk adjustment models for heart failure, 45
and we also find similar results with our sensitivity analyses using clinical laboratory values that may capture some of these unmeasured risk factors.
Our study has additional limitations. First, excluding individuals with missing cost data could affect internal validity of this study if there was a systematic pattern of missingness, such as related to severity of illness. However, the underlying cause of missing cost data was due to a known variable (in this case, time), and inclusion of these individuals actually strengthens our findings of mortality differences between sites (Appendix 2).
Second, because we used administrative data from the six study hospitals, we were unable to identify hospitalizations at other hospitals or include them in our calculations of resource utilization. However, prior studies suggest very high “hospital loyalty” among patients hospitalized for chronic illnesses;46
specifically, these studies found that chronically ill patients who were hospitalized in any of our six study hospitals had 80–90% of their total hospital days in the same hospital.47
Third, due to lack of data, our study could not account for outpatient utilization. It is possible that the rank ordering of hospitals on resource utilization and the relationship between resource utilization and mortality would have changed if we had been able to include outpatient care.
Fourth, by counting hospital days and costs for all hospitalizations during the 180-day period of analysis for each patient, we included resource utilization that may not be directly attributable to the study condition, HF. However, we adopted this approach for comparability with prior studies, and analyses of days and costs for initial hospitalizations alone found similar variation across hospitals as our main analyses.
Fifth, even the direct cost values from one site may incorporate other costs (e.g., teaching costs) that would have been attributed differently at another site. However, the similar associations observed between 180-day mortality and both resource utilization measures, total direct costs and total hospital days, suggest that total direct costs are a reasonable representation of resource utilization.
Finally, our results may not generalize to smaller hospitals and nonteaching hospitals, which did not participate in our study. Nonetheless, our findings suggest that focusing only on expired patients may lead to different ranking of hospitals with regard to resource utilization. More importantly, these studies ignore potentially large differences in health outcomes among chronically ill patients. Further studies should be conducted that include these and other hospitals to determine whether similar findings occur.
Assessing hospital efficiency requires that we consider outputs as well as inputs, that is, health outcomes as well as resource utilization. Contrary to public discussion of variation,4–6
it is likely that not all variation is inefficient or wasteful. However, much more work is needed to truly distinguish inefficient from beneficial resource utilization. The six hospitals involved in our study are currently investigating the underlying processes and practices that contribute to the variation in resource utilization and outcomes for HF that we identified. Their goal is to improve the outcomes of patients with HF and to provide care to those patients as efficiently as possible.