The SRTR center-specific outcomes for kidney transplant are used by patients and payers to judge quality of care and to select centers to perform transplants. It is critical, therefore, that these outcomes be accurate and unbiased. The SRTR adjusts center-specific outcomes for baseline differences in patient, donor, and transplant characteristics with data collected by the OPTN. However, their methods do not adjust for the differences in baseline comorbidity usually used in comparing outcomes between different health care providers. In our study, we recalculated adjusted graft survival outcomes used in the criteria recently adopted by CMS to identify underperforming kidney transplant centers, using comorbidity ascertained from CMS billing claims. Our results indicate that comorbidity is an important predictor of graft failure after kidney transplant, and that adjusting for differences in baseline comorbidity considerably alters which centers are identified as underperforming, according to CMS criteria.
The aggregate PPA of 80.8% could be interpreted to suggest that the effect of lack of comorbidity adjustment is relatively small. However, it implies that, on average, 3 to 4 centers would be incorrectly identified as performing or underperforming every 6 months. Because these may not be the same centers every 6 months, the cumulative effects could be greater. In addition, private payers may use more stringent criteria than CMS for determining their preferred providers. Hence, the number of affected centers could be substantially higher.
That adjusting for comorbidity would affect center-specific graft survival calculations should not be surprising. A large proportion of graft failures are due to death with a functioning graft, so factors that affect mortality could be expected to affect graft survival. However, few studies have examined the effects of comorbidity in kidney transplant patients. Jassal et al (9
) compared 4 different comorbidity indices for 6324 transplants in the Canadian Organ Replacement Registry. Specifically, they compared the Charlson Comorbidity Index (10
) to 3 indices adapted for use in non-transplant ESRD patients, the Khan Index (11
), the Davies Index (12
), and a modified Charlson Comorbidity Index (13
). They found that all 4 indices correlated with mortality after transplant, and that the Charlson Comorbidity Index was superior (14
). In a single-center study, Wu et al (15
) also found that the Charlson Comorbidity Index was associated with mortality in 715 kidney transplant patients.
We selected comorbid conditions from the Elixhauser (16
) Comorbidity Index, a set of 30 conditions which nearly subsumes the Charlson Comorbidity Index. While these indices have been well-validated for predicting mortality in non-transplant populations (17
), they have not been validated in administrative transplantation registries. In general, comorbidity indices tend to have high specificity and low sensitivity when applied to administrative data (24
). In a study of the National Registry of Atrial Fibrillation, the specificity of Medicare claims for cardiovascular comorbidity exceeded 90%, while sensitivity was much lower (25
). Thus, using Medicare claims in the present study likely results in under-identification of prognostically important comorbid conditions. Improved ascertainment of comorbid conditions is likely to strengthen our finding that an important component of differences in outcomes between transplant centers is heterogeneous prevalence of baseline comorbidity across centers. Additionally, diagnosis codes in Medicare claims carry no indication of disease severity. More detailed recording of comorbidity severity is also likely to strengthen our conclusions.
By necessity, we analyzed CMS criteria in a subset of transplant patients with Medicare coverage. Patients in this subset had longer duration of pretransplant ESRD, and were more likely to be older and African American or Hispanic than patients without Medicare coverage, and the subset included a disproportionate number of deceased-donor recipients. Comorbidity prevalence among excluded patients was likely lower than among included patients. Therefore, lack of comorbidity adjustment for excluded patients may plausibly be less important, and the PPA between the adjustment techniques may be modestly higher for all transplant patients than for the subset with pretransplant Medicare coverage.
Importantly, this study was not designed to determine the optimal method for adjusting center performance for comorbid conditions, but rather to demonstrate that such adjustment is likely to have an effect on performance results. In this study, more than 40% of patients carried private insurance, and excluding them from comorbidity-adjusted outcome calculations would likely be unacceptable. A strong argument could be made that the only accurate method of equitably adjusting outcomes for all transplant centers would be for the OPTN to prospectively collect comorbidity data at the time of transplant. In the mean time, strong consideration should be given to suspending the comparison of center-specific outcomes that are not adjusted for comorbidity, given the potential to harm programs and patients.
There is evidence that the level of mortality risk that centers are willing to tolerate in accepting transplant candidates varies, and that risk tolerance correlates with the chances that a center is an underperforming center by CMS and SRTR criteria. In a recent study, Schold et al calculated kidney transplant candidate mortality before transplant (26
). They found substantial variation from center to center in pretransplant mortality, indirectly suggesting variation in the threshold of risk that centers will accept. Importantly, of the 19% of centers that met CMS criteria for underperformance, 51% were in the highest quartile of pretransplant candidate mortality, and only 7% were in the lowest. These results are consistent with our results: baseline mortality risk strongly influences designation of underperformance.
Transplant centers do not accept all ESRD patients, and comorbidity influences access to transplants. For example, Gaylin et al reported that ESRD patients with more comorbidity, particularly cardiovascular comorbidity, were less likely to be accepted for kidney transplants than patients with less comorbidity (27
). Similarly, Winkelmayer et al (28
) reported that ESRD patients with comorbidity were less likely to undergo transplants. However, survival is better for patients accepted for transplants than for those on the waiting list (29
). Pressure on centers to improve published outcomes, leading them to deny transplants that could improve survival for higher risk patients, is of concern.
Optimal outcomes and equitable access are laudable goals in transplantation. However, efforts to optimize the performance of transplant centers should not be confused with efforts to optimize organ allocation. We suspect that it is not the intent of CMS to use Conditions of Participation to discourage centers from performing transplants in high risk patients who may benefit from them. Similarly, it may not be the intent of private payers (who select transplant centers for their insured patients) to use SRTR data to encourage centers to perform transplants in lower risk, and less costly, patients. Nevertheless, lack of comorbidity adjustment, along with use of these data by CMS and private payers to define center performance, could encourage centers to seek to improve their outcomes by accepting lower risk patients. This is not to say that centers should accept patients who are too high risk to benefit from transplants, but that the decision should be made in the best interests of patients and not to improve outcome statistics.
Finally, we note that centers have no reason to try to improve their published outcomes by selecting patients based on characteristics currently used to adjust those outcomes. For example, donation after cardiac death and extended criteria donor are currently used in the adjustment of deceased-donor kidney transplant outcomes. Thus, a center’s decision to use kidneys from donation after cardiac death or extended criteria donors should not be influenced by concern that these will negatively affect the center’s outcomes. Only characteristics not used in adjusting these outcomes represent a potential problem.
Based on this analysis, we suggest that any assessment of center performance include adjustment for baseline recipient comorbidity. This analysis indicated that cardiovascular comorbidity, neurological disease, and anemia and fluid disorders common to ESRD carry elevated prognostic significance. Conceptually, baseline comorbidity confounds the association between risk of graft failure and transplant center, so usual conventions for identifying important confounders (non-trivial disease prevalence and clinically meaningful risk of graft failure, given disease) should guide the selection of conditions for adjustment. We also suggest that comorbidity data would best be ascertained prospectively, rather than with administrative data. The associations reported in this analysis would likely be magnified in the presence of a data collection mechanism with superior sensitivity to Medicare claims. Prospective ascertainment would facilitate adjustment for comorbidity in non-Medicare patients. Finally, we note that all statistical assessments of center performance should be interpreted with caution. Comparisons of observed with expected events depend on the quality of the estimate of what is expected. This analysis showed that modest gains in model discrimination can have important consequences for individual centers. Moreover, differences between observations and expectations can arise by chance, and may not reflect transplant center practices.
In summary, the results of this analysis suggest that published SRTR center-specific outcomes, currently used by CMS and other payers to decide where transplants should be performed, do not adequately account for differences in baseline comorbidity across centers. Consequently, these center-specific outcomes may disadvantage centers that are willing to perform transplants in higher risk patients, compared with more restrictive centers. In the future, such centers may be encouraged to deny transplants to higher risk patients to improve SRTR-reported outcomes, even when survival may be improved by providing transplants to these higher risk patients. Payers should reassess how center-specific outcomes are calculated, and how decisions are made to determine which centers should perform transplants.