|Home | About | Journals | Submit | Contact Us | Français|
It is unknown whether high-risk cardiac surgical patients have less access to high-quality surgeons compared with lower-risk patients.
To determine whether high-quality surgeons are less likely to perform coronary artery bypass graft (CABG) surgery on high-risk patients compared with low-quality surgeons.
Retrospective cohort study using the New York State (NYS) CABG Surgery Reporting System (CSRS) of all patients undergoing CABG surgery in NYS who were discharged between 1997 and 1999 (51,750 patients; 2.20 percent mortality). Regression modeling was used to estimate the association between surgeon quality and patient risk of death. Surgeon quality was quantified using the observed-to-expected mortality ratio (O-to-E ratio).
Higher-risk patients are more likely to receive CABG surgery from higher-quality surgeons. For every 10 percentage point increase in patient risk of death (e.g., from 5 to 15 percent), there is an absolute reduction of 0.034 in the surgeon O-to-E ratio (p < .001).
This study suggests that high-risk CABG patients are significantly more likely to receive care from high-quality surgeons compared with lower risk patients.
Patient race, ethnicity, and socioeconomic status (SES) are known to be important determinants of access and quality in the American health care system in general (Agency for Healthcare Research and Quality 2005) and for coronary artery bypass graft (CABG) care in particular (Mukamel, Weimer, and Mushlin 2006; Mukamel et al. 2004). Until recently, the question of whether severely ill patients also encounter barriers to health care received very little attention. In the new era of “scorecard medicine” (Topol and Califf 1994), there is concern that public reporting of health care quality may lead some physicians to avoid sicker patients in order to improve or “protect” their performance rating (Werner, Asch, and Polsky 2005). While striving to decrease the disparities for traditional vulnerable populations such as nonwhite and low-income patients, it is possible that a potentially more vulnerable patient population of high-risk patients will be increasingly denied access to high-quality care because of the perception that caring for them may threaten a physician's or hospital's quality ranking.
Physician and hospital profiling is becoming increasingly pervasive (Marshall et al. 2000), driven by the widespread perception of quality deficits in the health care system (Schuster, McGlynn, and Brook 1998; Institute of Medicine 2000; McGlynn et al. 2003) and the demand for accountability and reform (Institute of Medicine 2001). Report cards are designed to allow patients and third-party payers to make informed choices, and to provide physicians and hospitals with benchmarking information to use for quality improvement. Report cards may also have unintended consequences. In New York State (NYS), which is widely believed to produce one of the most credible quality measures for CABG surgery, over 60 percent of cardiothoracic surgeons surveyed report that they refused to operate on high-risk patients on at least one occasion (Burack et al. 1999) and 83 percent of interventional cardiologists indicated that they were more reluctant to treat high-risk patients as a result of public reporting of physician outcomes (Narins et al. 2005). Although the evidence of the impact of public reporting on access to care is mixed (Omoigui et al. 1996; Peterson et al. 1998), recent work suggests that in the most severely ill patients with acute myocardial infarctions (AMI), performance profiling led surgeons to preferentially operate on lower-risk patients (Dranove et al. 2003). This resulted in higher rates of heart failure and recurrent AMI, and possibly higher mortality rates in patients admitted with AMI in states with public report cards compared with states without public reporting (Dranove et al. 2003).
We examined whether high-risk cardiac surgical patients are less likely to receive care from high-quality surgeons compared with lower-risk patients. This analysis was performed using patient-level data from the NYS CABG Surgery Reporting System (CSRS) database, which contains clinical data on mortality and preoperative risk factors for all patients undergoing isolated CABG surgery in NYS (Hannan et al. 1994). We hypothesized that high-risk patients would be less likely to receive CABG surgery from high-quality surgeons and more likely to be treated by low-quality surgeons. We note that due to limitations of these data (which do not include information about distance, zip code or sociodemographics), we are not estimating a choice model. Furthermore, our analysis cannot ascertain whether the sorting of patients and surgeons is due to patient or surgeon choice. Thus, this study answers the question of whether there is a sorting of patients (by risk) and surgeons (by quality), but does not address the question of what processes and mechanisms lead to this sorting.
This study uses data from the NYS Cardiac Surgery Reporting System and includes all patients undergoing isolated CABG surgery in NYS who were discharged between 1997 and 1999 (51,750 patients; 2.20 percent mortality). This database includes information on patient demographics, hospital and physician identifiers (encrypted), preoperative risk factors, and outcomes. These clinical data were collected prospectively at the hospital level and were then submitted to the NYS Department of Health (DOH). Audit mechanisms were in place to ensure the validity of the data (Hannan et al. 1991). This study was exempted from Institutional Review Board approval because it involved the secondary use of preexisting data.
Information on left ventricular function was missing for approximately 3 percent of the patients.1 We constructed an imputation model (Little and Rubin 2002) to predict the ejection fraction using the STATA (STATA Corp, College Station, TX) “impute” procedure which is based on best subset regression. Imputed values for patients with missing ejection fraction were calculated using the imputation model.
Based on our previous work, we chose to use fixed-effects regression as the basis for assessing surgeon quality (Glance et al. 2006). The question of whether to base quality assessment on conventional logistic regression versus hierarchical modeling has been the subject of extensive debate. Since our study hypothesis is predicated on the assumption that there is a correlation between surgeon quality and patient risk (we hypothesized that higher-quality surgeons selectively avoid higher-risk patients), we chose fixed-effects regression to quantify patient risk. Unlike conventional regression and hierarchical modeling, fixed-effect regression will result in consistent estimates of provider effect when provider effects are correlated with observable patient risk factors (Greene 2003).
We first developed a patient-level model to predict the probability of in-hospital death after CABG surgery using logistic regression with surgeon fixed-effects. Bivariate analyses were first performed to evaluate the association between in-hospital mortality and patient risk factors. Those risk factors with a p-value≤.20 were considered candidate variables for inclusion in the multivariate prediction model. Forward stepwise selection (p-value ≤.05) was used to identify the risk factors, which were independently associated with in-hospital mortality. The method of fractional polynomials (Royston and Altman 1994) was used to determine the optimal transformation for continuous covariates. Robust variance estimators (White 1980) were used to account for the fact that outcomes for patients treated by the same surgeon were likely to be correlated. We then added surgeon indicator variables in order to incorporate surgeon fixed-effects. The predicted probability of death for the ith patient is treated by jth surgeon, with risk factors xkij is given by
Using the approach described by DeLong et al. (1997), we set Pj = 1 when the surgeon is the jth surgeon, and 0 otherwise. For surgeon j = 1, Pj is set equal to −1 for j = 2 …J. Using this parameterization, pn (see equation ) corresponds to the unbiased estimate of the risk of mortality for patient “n” (n=1, …, N) if he or she is treated by the “average” surgeon:
Surgeon quality relative to his or her peers was quantified using the ratio of the observed to expected mortality rate (O-to-E ratio). Equation (2) was use to calculate the predicted probability of death for each patient. These probabilities were then averaged over all of the patients treated by surgeon “j” to calculate the expected mortality rate for surgeon “j.”
We then used multivariate linear regression to examine the association between surgeon quality and patient risk. The analysis was performed at the level of the individual CABG patient. The surgeon O-to-E ratio was the dependent variable. We used weighted least squares regression to account for heteroscedasticity and robust variance estimators to account for the intrasurgeon correlation. Weighted least squares accounted for the fact that the variance of the O-to-E ratio is determined, in part, by the surgeon volume. Robust variance estimators (Williams 2000) were used to account for the fact that patient outcomes for the same surgeons may not be independent. We included information on race and ethnicity in the model as potential confounders (information on SES, other than race and ethnicity, is not available in the CSRS data and NYS does not provide patient identifiers to allow linkage with administrative data to obtain more complete data on SES).
All statistical tests were two-tailed and p-values < .05 were considered significant. Data management, regression analysis, and regression diagnostics were performed using STATA SE/9.2 (STATA Corp, College Station, TX).
The study sample consisted of 57,150 patients treated by 189 surgeons at 33 hospitals. The final model (Table 1) included 30 risk factors divided into seven groups: demographics, hemodynamic status, comorbidities, severity of atherosclerotic disease, ventricular function, history of previous open heart surgery, and coronary anatomy. The C-statistic was 0.835 indicating good discrimination. The Hosmer–Lemeshow statistic was 11.0 with a p-value of .202, indicating acceptable fit.
Table 2 presents the results of the multivariate analyses examining the association between surgeon O-to-E ratio, a measure of surgeon quality, and patient predicted mortality. We constructed three different models: (1) the base model, which included only the patient predicted mortality (2) the base model plus race and ethnicity; and (3) the base model, plus race and ethnicity, and hospital fixed effects.
In all of the models, there is a significant inverse association between predicted patient risk of death and surgeon quality. We found no evidence that high-quality surgeons selectively avoid high-risk patients. Instead, we found that higher-risk patients are more likely to be treated by higher-quality surgeons, even after controlling for patient race and ethnicity. For every 10-percentage point increase in patient risk of death (e.g., from 5 to 15 percent), there is an associated absolute reduction of 0.034 in the surgeon O-to-E ratio. This effect persists, but is attenuated after controlling for hospital fixed effects. Adding hospital indicators to the model causes the absolute reduction in surgeon O-to-E ratio to drop to 0.01 for a 10-percentage point increase in patient risk of mortality. Thus, within the same hospital, higher-risk patients still tend to be treated by higher-quality surgeons. A substantively important amount of the correlation between quality and risk, however, is generated by the hospital patients are going to.
In this study, we find that high-risk patients undergoing CABG surgery are more likely to be treated by high-quality surgeons than by low-quality surgeons. We find no evidence, in this study, that high-quality surgeons are selectively avoiding high-risk patients. To our knowledge, this is the first study to examine whether high-risk patients are less likely to receive CABG surgery from high-quality surgeons compared with lower-risk patients.
From the perspective of optimizing overall outcomes, it is reassuring to see that high-quality surgeons are more likely to operate on the most severely ill patients. This study is not able to determine the mechanisms for these findings. Referring physicians may preferentially send the sickest patients to the highest quality surgeons while avoiding sending these high-risk patients to low-quality surgeons. It is possible that some low-quality surgeons are rejecting more difficult cases based on their perception of their own abilities. Further work is necessary to better understand the underlying mechanisms for these findings.
The findings of this study should be interpreted with caution because this study has several limitations. First, although we were able to control for the effects of patient race and ethnicity, we could not control for patient income level or insurance coverage. However, since patient race and ethnicity tend to be correlated with education level, income, and insurance coverage (Smedley, Stith, and Nelson 2003), the impact of omitting these SES variables will be attenuated in this analysis. Furthermore, prior work has demonstrated that patients with better insurance coverage (Mukamel, Murthy, and Weimer 2000) and higher income levels (Rothenberg et al. 2004) tend to be treated by higher-quality surgeons. Assuming that patients with better insurance coverage and higher income levels tend to be healthier, then it is likely that omitting these variables from our model would have led us to underestimate the likelihood that high-risk patients will be treated by high-quality surgeons.
Second, the inverse relation between surgeon quality and patient risk could appear to follow from the mathematical relationship between surgeon quality, measured by the ratio of the observed-to-expected mortality rate, and patient risk, quantified using patient expected mortality rate. On initial examination of this mathematical relationship, it would appear that a surgeon with a higher expected mortality rate would necessarily have a lower O-to-E ratio. However, a surgeon with a sicker case mix would also be expected to have a higher observed mortality rate, causing both the numerator (OMR) and the denominator (EMR) in the O-to-E ratio to increase. Furthermore, patient outcome, and therefore the surgeon's observed mortality rate, is assumed to be a function of both baseline patient risk factors and surgeon quality. To expect the surgeon's O-to-E ratio to simply decrease with increases in patient risk would imply that surgeon quality and patient severity-of-disease do not have a significant impact on patient outcome. Clearly, patient severity-of-disease is an important determinant of patient outcome. And, there is extensive literature documenting that there is significant variability in outcomes across surgeons for both cardiac (O'Connor et al. 1991, 1996; Hannan et al. 1994, 1995) and noncardiac procedures after controlling for patient case mix (Birkmeyer et al. 2003). Assuming an unbiased prediction model, the O-to-E ratio should only be a function of surgeon quality, and would not be expected to decrease (or increase) simply because of changes in surgeon case mix.
Furthermore, the patient-level expected mortality rate is a stochastic variable. Our analysis is based on a regression model in which the dependent variable is the surgeon O-to-E ratio and the independent variable is the patient-level expected mortality rate. Errors in the patient-level expected mortality rate would be associated with inverse changes in the surgeon O-to-E ratio. But, the contribution of the error term that is due to the estimation of the patient-level estimated mortality rate should be small as long as the physician volume is relatively large. Therefore, the mechanically induced bias resulting from having the “same” variable on both sides should be small. In fact, if a surgeon has “n” patients, then the error term on the right hand side is “1/n” times the error in the estimation of the patient-level expected mortality rate.
Third, measures of physician quality may be sensitive to the choice of risk adjustment model (Iezzoni 1997; Glance, Osler, and Dick 2002a, b, 2005), unobserved risk factors (Lilford et al. 2004), sample sizes (Dimick, Welch, and Birkmeyer 2004), the use of in-hospital mortality versus 30-day mortality, and the choice of modeling methodology (Glance et al. 2006). It is not possible to know with absolute certainty whether or not quality rankings used as the basis for our analysis accurately reflect true surgeon quality.
Fourth, it could be argued that a subset of surgeons designated as “high-quality” surgeons are identified as high-quality precisely because they are systematically upcoding patient risk factors. If extensive miscoding is occurring, then it is possible that a subset of surgeons is receiving credit for treating higher risk patients. In theory, this group of surgeons would be both more likely to be mislabeled as high quality (because their predicted mortality rate would increase) and to be found caring for higher-risk patients (due to upcoding of patient risk factors). Even though the NYS DOH conducts extensive audits, it is not possible to completely insure the accuracy of the data. We cannot rule out upcoding of patient risk factors as an explanation for our findings.
Fifth, our analysis may be biased if we have incorrectly estimated patient risk. In our conceptual model, we hypothesized that high-risk patients may be “similar” to racial and ethnic minorities with respect to access to high-quality physicians. In the case of racial/ethnic minorities, patient race is relatively fixed (of course, one can argue that patient race may differ depending on whether it is assigned by the provider or by the patient). In the case of “high-risk” patients, risk level is a function of observable and unobservable risk factors, and is subject to potential bias caused by the omission of important risk factors and model misspecification. Moreover, surgeon quality is also based on the same set of patient risk factors, and is also subject to misspecification and omitted variable bias.
Finally, this study does not consider the possibility that high-risk patients who would benefit from CABG surgery are unable to find a surgeon willing to operate on them. This study only considers patients who actually underwent CABG.
Despite these limitations, we believe that the findings of this study are relatively robust. Our analysis controlled for patient race and ethnicity, which are very important drivers of patient access. The use of clinical, as opposed to administrative data to identify patient risk groups and to rank surgeon quality, is a significant strength of this analysis. Furthermore, since NYS has one of the earliest and most credible CABG report cards, the use of data from NYS is ideal for exploring the impact of public release of outcome data on patient access to high-quality physicians.
In general, we find that high-risk CABG patients are more likely to receive care from high-quality surgeons compared with lower risk patients. This finding is important in light of prior evidence that some surgeons are more reluctant to care for the highest risk patients in the aftermath of publicly released outcomes report cards. Our study does not, however, exclude the possibility that some of the highest risk patients no longer have access to surgical revascularization.
This project was supported by a grant from the Agency for Healthcare and Quality Research (RO1 HS 13617).
The views presented in this manuscript are those of the authors and may not reflect those of Agency for Healthcare and Quality Research or of the New York State Department of Health or of the Cardiac Advisory Committee.
1For the purpose of this analysis, we defined an ejection fraction of zero as missing data.