This study demonstrates that the statistical models currently being used by CMS to measure and publicly report 30-day mortality rates after admission for AMI, HF, and pneumonia at non-Federal U.S. hospitals demonstrate good discrimination when applied to VHA hospitals using three years of data. Restricted to patients aged 65 years or older, performances of the statistical models were similar for VHA hospitals and non-Federal hospitals using administrative claims data for fee-for-service Medicare beneficiaries. For instance, c-statistics for the three models were 0.79, 0.73, and 0.72 as compared with 0.71, 0.70, and 0.72 (2
). Moreover, model estimates for the patient covariates were predominantly estimated as being in the same direction and of comparable magnitude in VHA and non-Federal hospitals (2
). These results suggest that these statistical models could be used by the VHA to measure the quality of care delivered by its hospitals and potentially to benchmark outcomes between the VHA and private sector.
As among non-Federal hospitals caring for fee-for-service Medicare beneficiaries, we observed modest heterogeneity in 30-day RSMRs among VHA hospitals for AMI, HF, and pneumonia. Using three years of data, it is possible to identify outlying performers both at the top and bottom, either for purposes of targeting performance improvement or identifying methods or strategies implemented at top performing facilities that could be widely adopted throughout VHA. Moreover, this might be accomplished with minimal additional cost because the necessary administrative data are already routinely collected within the VHA electronic medical record.
Over the past 15 years, VHA has been a leader in quality improvement, beginning with its nationally integrated healthcare system reform in 1995 that included information technology implementation, performance measurement and reporting, service integration, and realigned payment policies (12
). VHA has a sophisticated performance measurement system that is comprised of more than 120 performance measures and hundreds of additional process and outcome measures that are tracked on a routine basis (19
). Moreover, in creating the National Surgical Quality Improvement Program (NSQIP), VHA pioneered the use of surgical RSMRs to guide quality improvement efforts (20
). Similarly, VHA's Inpatient Evaluation Center measures RSMRs for intensive care units to monitor performance (21
). VHA has not, however, routinely measured hospital performance using 30-day RSMRs for common medical conditions. Adopting the statistical models currently being used by CMS to monitor AMI, HF, and pneumonia outcomes provides the VHA with another tool to assess performance, compare hospitals, and drive improvement.
There are several considerations in interpreting our results. First, we did not validate the statistical models’ performance against VHA medical record data. However, prior studies of mortality following AMI in VHA using claims data and clinical information have yielded similar estimates, suggesting that this is not likely an important source of bias (22
). For example, 30-day mortality following AMI was previously calculated as 16.3% using administrative claims data and 16.0% using detail clinical information (22
). Second, we only examined 30-day RSMRs after AMI, HF, and pneumonia hospitalizations for patients 65 years of age or older and our findings could potentially vary when these methods are applied to patient populations that include younger individuals. Third, our study focused only on 30-day RSMRs, not other important dimensions of quality, such as processes of care or patient experiences. Fourth, our data sources do not allow for identification of patients who are admitted for comfort care nor for identification of clinical characteristics diagnosed outside of VHA among dual users. Finally, although the models used for these measures were validated against models based on medical record data, we cannot exclude the possibility of important unmeasured factors influencing our findings.
In conclusion, our study demonstrates that the statistical models currently being used by CMS to measure and publicly report hospital 30-day mortality rates after admission for AMI, HF, and pneumonia at non-Federal hospitals demonstrate similar discrimination when applied to VHA hospitals. These models could thus be used by the VHA to measure the quality of care delivered by its hospitals. These findings provide a benchmark for national VHA hospital performance and suggest that administrative claims data should be used in conjunction with the large array of performance measures that VHA already has in place to inform and promote quality improvement in an effort to achieve better outcomes within the VHA.