In this analysis of 22750 Medicare beneficiaries hospitalized with heart failure at 150 US hospitals, we found substantial variation in hospital adherence to the 4 CMS process measures. Yet, with the exception of the positive association between hospital-level conformity to the assessment of left ventricular function and cardiovascular readmission, there were no associations between the CMS hospital performance measures or the composite measures and patient-level mortality or cardiovascular readmission rates at 1 year. However, we did find a significant association between hospital-level adherence to prescription of beta-blockers at discharge and lower mortality at 1 year. To explore these associations with risk-adjusted hospital-level outcomes, we conducted bootstrap analyses and found the results to be generally consistent with the primary analysis.
These findings are generally consistent with a previous analysis examining patient-level predictors and outcomes of 5791 patients from the 91 hospitals who participated in OPTIMIZE-HF. In that study, only conformity with a measure for prescription of a beta-blocker for left ventricular systolic dysfunction was significantly associated with a lower risk of 60-day to 90-day mortality after propensity adjustment and risk adjustment.5
The findings are also consistent with a study using an administrative data source to examine associations between hospital-level processes of care and hospital-level outcomes in 3657 acute care hospitals, which found that assessment of left ventricular function and prescription of an ACE inhibitor at discharge were not significantly associated with improved survival at 1 year.3
The absolute risk reduction in risk-adjusted mortality between hospitals performing in the 25th percentile compared with those performing in the 75th percentile was 0.002 (P
= .05) for assessment of left ventricular function, –0.003 (P
= .04) for ACE inhibitor use, and 0.002 (P
= .08) for 1-year mortality. In contrast, a study of 2958 patients drawn from a 20-hospital health care system in a single community reported an association between CMS process measures at discharge and 1-year survival, though multiple known confounders were not included in the multivariable models and nurse case managers continued to be involved in the care of patients after discharge.16
The present analysis expands upon findings from previous studies in two key ways. First, this study links Medicare administrative data to a detailed clinical source to allow for both longitudinal outcome assessment and rigorous risk adjustment. Thus, we were able to determine whether CMS process measures for heart failure had measurable effects up to 1 year after discharge in a broad cohort of patients from all regions of the United States. In addition, the analysis examines how overall hospital adherence levels are related to patient-level mortality and cardiovascular readmission, thereby addressing the question of whether patients who are treated at hospitals that score higher on process measures have better outcomes. This analytic approach addresses whether receiving care at a hospital with better conformity to recommended processes of care is associated with improvements in long-term outcomes for patients with heart failure. Previous research from CRUSADE (Can Rapid Risk Stratification of Unstable Angina Patients Suppress Adverse Outcomes with Early Implementation of the ACC/AHA Guidelines) has also addressed the associations between hospital-level predictors and patient level outcomes, but for patients hospitalized with acute coronary syndromes.17-19
Although hospital profit status17
and the presence of an inpatient cardiology service18
were not significantly associated with inpatient outcomes, hospital participation in clinical trials19
was significantly related to patient-level mortality.
There are several potential explanations for the lack of associations in this study. First, the processes of care selected for the performance measures may truly not be associated with outcomes. Evidence of associations between discharge instructions, assessment of left ventricular function, and smoking cessation counseling are based on expert opinion rather than randomized clinical trials. Furthermore, outcomes after hospital discharge likely reflect a combination of many domains of care and may be dominated by postdischarge care processes, frequency of follow-up, and the underlying disease process. For example, being discharged with an ACE inhibitor or ARB does not ensure that a patient will remain on therapy or that an effective dose has been prescribed, nor does it ensure that the clinical effects will be observable within 1 year. However, the significant relationship observed between beta-blockers at discharge and mortality at 1 year demonstrates that associations can be detected when they exist. Second, hospital documentation of process measures may not reflect actual care. For example, patient education may be documented in the medical record even if it was completed at discharge in a rushed or superficial manner. Conversely, physicians or nurses may instruct a patient about medications, diet, symptoms of worsening heart failure, and daily weight monitoring but may not record this in the patient's medical record. Third, the self-reported nature of the process measure forms carries the risk that hospitals purposely underreport eligible patients to inflate the process measure adherence score, a violation that was suspected but not confirmed in a study of process measure adherence in family practices in the United Kingdom.20
Finally, studies examining effects of system-level exposures on individual-level outcomes may be limited by the inability to control for unobserved system-level characteristics, which could result in null associations.
Other findings in this study warrant comment. First, we found a small but significant association between assessment of left ventricular function and greater risk of cardiovascular readmission. The reason for this finding is unclear; we suspect it may reflect residual confounding in which patients who are sicker in ways we did not measure may have been more likely to undergo assessment of left ventricular function and be hospitalized as compared with healthier patients. Second, the demographic characteristics of the sample are comparable to another study estimating trends in mortality among hospitalized Medicare beneficiaries with heart failure,21
providing some evidence of how the results of the current study are generalizable to Medicare fee-for-service beneficiaries. Third, the high mortality and cardiovascular readmission rates found in this patient population indicate that this is a high-risk population that would likely benefit from improved process measure conformity in measures with a strong process–outcome link.
Our study has some limitations. First, the process–outcome association may be confounded by socioeconomic factors or other unmeasured confounders related to both health status and hospital adherence level. Second, to the extent that Medicare beneficiaries enrolled in OPTIMIZE-HF are not representative of all Medicare beneficiaries with heart failure, the results may not be generalizable. Evidence suggests, however, that Medicare beneficiaries enrolled in OPTIMIZE-HF are similar to Medicare fee-for-service beneficiaries hospitalized with heart failure in terms of baseline characteristics, survival, and all-cause readmission.22
Third, the generalizability of the results may be further limited if participating hospitals differ from nonparticipating hospitals in ways not reflected patient demographic characteristics, core measures, or postdischarge outcomes. Fourth, patient eligibility for a performance measure was based on documentation in the medical record, which may not always be accurate. For example, some patients may have had undocumented contraindications or intolerances, leading to an overestimation of the number of patients eligible for the performance measure. Finally, the cross-sectional nature of the data did not allow us to assess changes in performance measure conformity and clinical outcomes over time.
Performance measures are used for public reporting of the quality of cardiovascular care at the hospital level, affecting financial payments to medical centers and individual physicians. Thus, it is essential that measures be prioritized to include those that are known to be closely associated with patient outcomes. Given the lack of associations between individual measures and a composite measure and postdischarge clinical outcomes, the use of the CMS heart failure performance measures in their current form in pay-for-performance programs may not be the most efficacious way to assess and reward quality. Although clearly stated methods have been used to develop and implement heart failure performance measures, these measures are not fulfilling their stated purpose. Consequently, additional measures with stronger process–outcome links after hospital discharge should be considered. If a documentation process at the hospital does not accurately capture the most important elements of care provided, it may be unreasonable to expect that incentives for these process measures would improve outcomes.
To our knowledge, this analysis is the first examine how overall hospital conformity to the 4 current CMS heart failure-specific process measures is associated with individual-level, long-term outcomes in a broad cohort of patients from all regions of the United States. To build upon these results, future research is needed to refine how performance measures are created and selected. Consideration should be given to prospective validation and testing of measures, rather than the selection of measures by expert panels. Before implementing pay-for-performance broadly across all systems, the limitations of current performance measures and the differences in measure reliability across disease types, provider settings, and patient populations need to be better recognized. In addition, a minimally important difference needs to be defined before policy makers decide to implement new process measures, especially given the small effect sizes.4
The small effect sizes may not be sufficient to justify broad policy changes, especially if the cost of such changes would not justify changes that were not clinically significant. It is essential that new process of care measures for heart failure be developed and implemented so that the quality of care can be more accurately measured and outcomes of this high-risk patient population can be improved.