Several studies have been conducted to compare the performance of the two most common risk adjustment methods, the Charlson and the Elixhauser methods [9
]. This article adds to the literature by applying the CMS-HCC risk adjustment to predict mortality outcomes and comparing its performance with the Charlson and Elixhauser methods. Our findings suggest that the CMS-HCC method outperforms the Charlson and Elixhauser methods in predicting the risk of in-hospital and six-month mortality among all Medicare beneficiaries with a hospital admission including subgroups with a principal diagnosis of AMI, CHF, DM, and stroke. Based on point estimates of c-statistic, we also found that the Elixhauser method was superior to the Charlson method in models including individual diagnosis indicators to predict mortality. This is consistent with the previous literature [15
There are several possible explanations for the superior performance of the CMS-HCC method. First, the CMS-HCC risk adjustment method captures more conditions than the Charlson and Elixhauser methods. The CMS-HCC aggregates 189 condition categories into 70 categories [3
], while the Charlson identifies 19 comorbidities [1
] and the Elixhauser identifies 30 comorbidities [2
]. Second, the CMS-HCC has more detailed information on the severity of a condition relative to the Elixhauser and Charlson methods. For example, under the CMS-HCC method, a patient with diabetes can be coded as HCC19 (diabetes without complications), HCC18 (diabetes with ophthalmologic or unspecified manifestation), HCC17 (diabetes with acute complications), HCC16 (diabetes with neurologic or other specified manifestation), or HCC15 (diabetes with renal manifestation) [3
] depending on the severity and complications associated with his/her diabetes. On the other hand, there are only two categories for diabetes under both the Charlson method (diabetes, mild to moderate vs. diabetes with chronic complications) [6
] and the Elixhauser method (diabetes, uncomplicated vs. diabetes, complicated) [2
]. Third, the CMS-HCC captures more complications that result from the process of care relative to the Charlson and Elixhauser methods. The Charlson and Elixhauser methods only include comorbidities and remove complications from the models given that one of the main purposes of these risk adjustment methods is to adjust for the baseline health status differences across patients before they were admitted to the hospital. Including complications due to the process of hospital care could overestimate severity of patient case mix among those who receive poorer inpatient quality of care. The CMS-HCC tries to capture all conditions associated with higher costs and hence includes complications because its original purpose was to predict Medicare expenditures. For example, pneumonia is included in CMS-HCC (as HCC111 and HCC112) but not in the Charlson and Elixhauser comorbidity lists because it is not distinguishable from a complication arising in the hospital [2
]. Some diagnoses (e.g. myocardial infarction) included in the Charlson index might be due to complications of procedures (e.g. lumbar spine surgery) [5
]. Therefore, the Charlson method includes them only if the condition occurred prior to the index hospitalization [5
Inpatient mortality rates are often used to measure quality of care inside hospitals [20
]. Risk adjustment is needed to ensure a fair comparison across hospitals by adjusting for patients' baseline clinical risk prior to hospital admission (e.g. comorbidities). However, if a risk adjustment method not only adjusts for pre-admission conditions, but also takes into account complications due to poor quality of inpatient care, it could potentially lead to a biased conclusion. The influence of complications due to the inpatient process of care can be eliminated by coding risk adjusters based on the pre-index date claims (i.e. the fourth analytical file in our study). However, as our results highlighted, removing the index date claim resulted in poorer predictions for both in-hospital and six-month mortality across all three methods. In-hospital mortality models had larger drops in c-statistic values than six-month mortality. The reason may be that in-hospital mortality is highly correlated with the conditions and complications during the inpatient stay. The CMS-HCC models had a larger drop in performance than the other two risk adjustment methods. In spite of this, the CMS-HCC still performed as well or better than the other two methods when we removed the influence of concurrent complications arising during the hospitalization by coding risk adjusters based only on the pre-index date claims.
In addition, the predictive power of the CMS-HCC models predicting in-hospital mortality decreased when additional diagnostic information from inpatient and outpatient claims in the 12-month prior to admission was included. This is in contrast to the Charlson and Elixhauser models whose performance in predicting in-hospital mortality increased with the inclusion of additional information. This difference may occur due to the fact that the clinical complications during the index hospitalization included only in the CMS-HCC model play an important role in its predictive performance for in-hospital mortality and adding prior diagnostic information dilutes its prediction power (Table and Table ).
The Charlson and Elixhauser methods were originally designed to serve as risk adjusters using only inpatient or hospital discharge data. Health services researchers have been increasingly using both inpatient and outpatient data for coding these two risk adjusters given the wide availability of longitudinal administrative claims datasets. However, these data also make it possible to exclude diagnoses codes related to complications due to inpatient processes of care by only including diagnoses from inpatient and outpatient claims identified before the index hospitalization date (i.e. exclude index hospitalization claim). In this case, our results indicate that the CMS-HCC is more favorable than the other two methods because it captures more comprehensive diagnosis information than the Charlson and Elixhauser methods and complications due to inpatient processes of care are not an issue. However, one should be careful when using CMS-HCC without removing diagnoses from index hospitalization particularly in studies evaluating inpatient processes of care.
There are several limitations of our study that should be addressed. First, some of our models using individual diagnosis indicators failed to converge. This may be due to our small sample sizes for some conditions which limited our ability to conduct a full comparison across all conditions and analytic files for the individual diagnosis indicator models across the three methods. However, all models converged for our primary sample of all patients with hospital admissions and the results were consistent with those from the models that converged in the disease-specific samples. Second, we limited our assessment of the outcomes to in-hospital and six-month mortality. Model performance may be different when examining a longer time horizon for mortality as well as for other health outcomes. Finally, our evaluation of the three risk-adjustment methods was conducted using samples of Medicare beneficiaries. Since the CMS-HCC risk score was originally developed and calibrated using data on Medicare beneficiaries whereas Charlson and Elixhauser methods were not, it is possible that our study results were more favorable for the CMS-HCC. However, one might argue that the CMS-HCC method was developed to predict Medicare expenditures and not mortality unlike the other two methods. Nevertheless, future evaluations of these risk adjustment methods in other patient populations are needed.