|Home | About | Journals | Submit | Contact Us | Français|
Public reporting of patient outcomes is an important tool to improve quality of care, but some observers worry that such efforts will lead clinicians to avoid high-risk patients.
To determine whether public reporting for percutaneous coronary intervention (PCI) is associated with lower rates of PCI for patients with acute myocardial infarction (MI) or with higher mortality rates in this population.
Retrospective observational study conducted using data from fee-for-service Medicare patients (49,660 from reporting states and 48,142 from nonreporting states) admitted with acute Ml to US acute care hospitals between 2002 and 2010. Logistic regression was used to compare PCI and mortality rates between reporting states (New York, Massachusetts, and Pennsylvania) and regional nonreporting states (Maine, Vermont, New Hampshire, Connecticut, Rhode Island, Maryland, and Delaware). Changes in PCI rates over time in Massachusetts compared with nonreporting states were also examined.
Risk-adjusted PCI and mortality rates.
In 2010, patients with acute MI were less likely to receive PCI in public reporting states than in nonreporting states (unadjusted rates, 37.7% vs. 42.7%, respectively; risk-adjusted odds ratio [OR], 0.82, 95% CI, 0.71–0.93, P=.003). Differences were greatest among the 6708 patients with ST-segment elevation Ml (61.8% vs 68.0%; OR, 0.73 [95% CI, 0.59–0.89]; P=.002) and the 2194 patients with cardiogenic shock or cardiac arrest (41.5% vs 46.7%, OR 0.79 [0.64, 0.98], P=.030). In Massachusetts, odds of PCI for acute Ml were comparable with odds in nonreporting states prior to public reporting (40.6% versus 41.8%, OR 1.00 [0.71, 1.41]). However, after implementation of public reporting, odds of undergoing PCI in Massachusetts decreased compared with nonreporting states (41.1% versus 45.6%, OR 0.81 [0.47, 1.38], p=.030 for difference in differences). Differences were most pronounced for the 6081 patients with cardiogenic shock or cardiac arrest (pre-reporting, 44.2% versus 36.6%, OR 1.40 [0.85, 2.37] post-reporting, 43.9% versus 44.8%, OR 0.92 [0.38, 2.22], p=.028 for difference in differences). There were no differences in overall mortality among acute MI patients in reporting versus non-reporting states.
Among Medicare beneficiaries with acute Ml, the use of PCI was lower for patients treated in 3 states with public reporting of PCI outcomes compared with patients treated in 7 regional control states without public reporting. However, there was no difference in overall acute Ml mortality between states with and without public reporting.
Public reporting of patient outcomes is a key tool to drive improvements in healthcare delivery. Over the past 20 years, three states, New York (1991), Pennsylvania (2001), and Massachusetts (2005), have instituted mandatory public reporting of outcomes for percutaneous coronary intervention (PCI). The notion behind public reporting is simple: collecting and publicly reporting performance will enable patients to choose high quality hospitals, and will motivate clinicians to improve performance.
However, critics worry that public reporting may create disincentives for physicians and hospitals to care for the sickest patients, potentially leading clinicians to avoid offering lifesaving procedures to these patients.1–5 Prior research in this area, primarily focused on public reporting of coronary artery bypass grafting (CABG) outcomes, is nearly two decades old, and has yielded mixed results.6–8 Over the past two decades, PCI has overtaken CABG as a mainstay of therapy for ischemic heart disease.9 As a result, many states are now planning to implement public reporting for this procedure.10 Yet, we are unaware of prior national studies examining whether public reporting of PCI outcomes is associated with lower PCI rates, especially among patients who might benefit from this procedure.
Therefore, we sought to examine the association between public reporting and rates of PCI among patients with acute myocardial infarction (MI). We focused on MI patients because clinical trial data and guidelines support the use of PCI in this population.11,12 We sought to answer three questions: first, are patients with an acute MI less likely to receive a PCI in public reporting versus non-reporting states? Second, does this relationship vary with the risk profile of the patient (i.e. if the patient presented with shock or cardiac arrest versus not)? Finally, given that public reporting is meant to improve outcomes, is public reporting associated with lower mortality for patients with acute MI?
We used Medicare Provider Analysis and Review (MedPAR) files from 2002 through 2010 to identify patients over 65 years of age with a primary discharge diagnosis of acute MI, initial episode of care, using International Classification of Diseases, 9th Edition (ICD-9) codes 410.×1. We defined the NSTEMI subgroup as patients with a primary diagnosis of subendocardial myocardial infarction (ICD-9 code 410.71), the STEMI subgroup as patients with a primary diagnosis of a ST-elevation myocardial infarction (STEMI, ICD-9 codes 410.×1, excluding 410.71), and the shock/arrest subgroup as patients with additional diagnostic codes for cardiogenic shock or cardiac arrest (ICD-9 codes 785.51 and 427.5, respectively). We also divided the sample into two age groups (65 to 74 and ≥ 75 years of age), based on current guidelines for use of PCI in the setting of STEMI.12
As is convention with prior work in this area, transferred patients were assigned to the receiving hospital. We only included acute MI patients who were admitted or transferred to a hospital that was PCI capable in each of the study years.
Massachusetts, Pennsylvania, and New York constituted our public reporting states for our primary analysis. Because of wide regional differences in practice patterns, we selected a regional control group consisting of the remaining New England and Mid-Atlantic states (Maine, Vermont, New Hampshire, Connecticut, Rhode Island, Maryland, and Delaware). We excluded New Jersey from our analyses because it has been collecting data, though not reporting, since 2007. We also performed sensitivity analyses using national controls; these results are available in the Appendix (tables 9–12).
New York began to publicly report performance on PCI over 2 decades ago (1991) and Pennsylvania began in 2001. Over the past decade, only Massachusetts started to publicly report performance for PCI (with the first report coming out in late 2005). Therefore, for our longitudinal analysis, we only examined Massachusetts as our public reporting state, used the same control states as described above, and excluded New York and Pennsylvania from our analyses. We defined the pre-reporting period for this analysis as 2002 through 2004, and the post-reporting period as 2006 through 2010, excluding 2005 because reporting began during that year.
Our primary outcome was the receipt of a PCI (procedure codes 0066, 3601, 3602, 3605, 3606, 3607, 3609) during hospitalization for an acute MI. We calculated unadjusted PCI rates and ranked all states by the proportion of acute MI patients in 2010 who received a PCI during their hospitalization. Then, to examine whether patients with an acute MI were less likely to receive a PCI in public reporting states compared with control states, we created patient-level hierarchical logistic regression models, first calculating the unadjusted odds of receiving a PCI in reporting versus non-reporting states. We then accounted for clustering of patients within hospitals, included a random effect for hospital, and calculated rates adjusted for age, sex, race, and 29 comorbid medical conditions using the Medicare risk adjustment model developed by the Agency for Healthcare Quality and Research (AHRQ, Appendix Table 1). We repeated these analyses for cardiac catheterization (procedure codes 3722, 3723, 8855, 8856, 8857) and coronary artery bypass grafting (CABG, procedure codes 3610–3619), as well as for a combined endpoint of PCI or CABG.
In our longitudinal analysis, we again used patient-level hierarchical logistic regression models, first calculating unadjusted rates and odds and then creating fully-adjusted models, and examined changes between the pre-reporting period and the post-reporting period in Massachusetts versus control states. We tested for interaction and repeated both the cross-sectional and longitudinal models in each of our three cohorts based on risk (NSTEMI, STEMI, and shock/arrest) and the two cohorts based on age (65 – 74 years and ≥ 75 years), as well as for each of the procedural outcomes (PCI, catheterization, CABG, and the combined endpoint of PCI or CABG).
We were also interested in whether public reporting led to better outcomes for the overall population of patients with acute MI as well as for the acute MI patients who underwent PCI. Therefore, we calculated 30-day mortality rates using hierarchical logistic regression models, again both with and without adjustment for age, sex, race, and medical comorbidities using the AHRQ method. Unadjusted analyses account for concerns that public reporting states code comorbidities more aggressively, thus making similar patients appear sicker in the public reporting states than in the control states. We compared outcomes in public reporting versus control states in 2010. Next, we repeated longitudinal analyses, comparing outcomes in Massachusetts versus control states. In both cases, we repeated these models for all three risk cohorts and both age groups. Finally, we repeated the mortality analyses stratified by receipt of PCI.
Based on the baseline mortality rates in our dataset, we had 90% power to detect a difference of 1.7% between the reporting states in cross-sectional analyses and 90% power to detect a difference in differences of 0.9% in longitudinal analyses. A p value of less than 0.05 was considered to be statistically significant. All analyses were conducted using SAS 9.3 software. The study was approved by the Harvard School of Public Health Office of Human Research Administration.
Our patient population consisted of 97,802 discharges with a primary diagnosis of acute MI. In our cross-sectional cohort, patients in reporting states were slightly older; medical comorbidities were well-matched (Table 1). There were similar proportions of NSTEMI, STEMI, and shock/arrest patients in public reporting and non-reporting states. Patient characteristics for the longitudinal cohort, as well as patient characteristics stratified by receipt of PCI, are available in Appendix Tables 2–4.
When we ranked the 50 U.S. states and the District of Columbia by their proportion of acute MI patients who received a PCI in 2010, the three public reporting states ranked 42nd (Pennsylvania), 48th (Massachusetts), and 50th (New York, Appendix Figure 1); patterns were similar for shock/arrest patients (Appendix Figure 2).
Among all patients admitted for an acute MI in 2010, those in public reporting states were significantly less likely to receive a PCI compared to the regional control group of patients in non-reporting states (unadjusted rate 37.7% versus 42.7%, adjusted odds ratio [OR] 0.82, 95% confidence interval [CI] 0.71 to 0.93, p=0.003, Table 2). This was most pronounced in the STEMI and shock/arrest groups (p-value for interaction <0.001). The odds of receiving a PCI for patients in the NSTEMI group in public reporting states versus non-reporting states were similar (30.3% versus 33.7%, OR 0.87, 95% CI 0.73 to 1.04, p=0.118), while for the STEMI (61.8% versus 68.0%, OR 0.73, 95% CI 0.59 to 0.89, p=0.002) and shock/arrest groups (41.5% versus 46.7%, OR 0.79,95% CI 0.64 to 0.98, p=0.030), the odds were significantly lower. These differences were similar in the older (≥ 75) age group.
Patterns were very similar for the odds of receiving cardiac catheterization (unadjusted rates 58.0% versus 62.8%, adjusted OR 0.82, 95% CI 0.69 to 0.99, p=0.041, Appendix Table 5). There were no differences between reporting and non-reporting states in the odds of receiving CABG (unadjusted rates 8.3% versus 9.3%, adjusted OR 1.01, 95% CI 0.80 to 1.26, p=0.951, Table 3).
We next examined changes in PCI rates in Massachusetts, the only state which initiated public reporting for PCI in the recent era. We found that through 2004, patients in Massachusetts and control states appeared to have comparable PCI rates. However, in 2005, the year in which Massachusetts began public reporting, the patterns began to diverge. By 2010, patients in Massachusetts appeared to have a lower likelihood of receiving a PCI than did patients in control states (Figure 1).
Using a difference in differences model, we found that the risk-adjusted odds of receiving a PCI changed in Massachusetts (compared to control states) after the advent of public reporting. In the pre-reporting period, patients in Massachusetts were similarly likely to undergo PCI as patients in control states (unadjusted rates 40.6% versus 41.8%, adjusted OR 1.00, 95% CI 0.71 to 1.41, Table 4). However, by the post-reporting period, the odds of receiving PCI had dropped in Massachusetts relative to controls (41.1% versus 45.6%, OR 0.81, 95% CI 0.47 to 1.38, p value for difference in differences=0.030). These associations were most pronounced for shock/arrest patients (p value for interaction <0.001): for these patients, the relative odds of receiving a PCI in Massachusetts versus controls dropped significantly from the pre-reporting period (44.2% versus 36.6%, OR 1.40, 95% CI 0.85 to 2.32), to the post-reporting period (43.9% versus 44.8%, OR 0.92, 95% CI 0.38 to 2.22, p-value for difference in differences=0.028). We found comparable relationships in our group of older patients.
Patterns were similar for the odds of receiving cardiac catheterization (pre-reporting unadjusted rates 59.7% versus 63.2%, adjusted OR 0.94, 95% CI 0.63 to 1.42; post-reporting 59.0% versus 67.0%, OR 0.71, 95% CI 0.38 to 1.31, p for difference in differences=0.006, Appendix Table 6), suggesting that some decisions not to proceed took place prior to seeing patients’ coronary anatomy. Over this same time period, there was an increase in the odds of receiving CABG in Massachusetts compared to controls (pre-reporting unadjusted rates 10.8% versus 12.9%, adjusted OR 0.90, 95% CI 0.64 to 1.21; post-reporting 10.8% versus 11.1%, OR 1.16, 95% CI 0.67 to 1.90, p for difference in differences=0.013, Table 5 and Figure 2). This trend was driven primarily by a relative increase in the use of CABG in the NSTEMI group (Table 5). When we summed the rates of PCI and CABG, the differences in trends between Massachusetts and controls were somewhat attenuated (Figure 3).
We found no overall difference in 30-day mortality among patients with acute MI in public reporting versus non-reporting states (unadjusted rates 12.8% versus 12.1%, adjusted OR 1.08, 95% CI 0.96 to 1.20, p=0.196, Table 6). However, we did note higher mortality in the STEMI subgroup (13.5% versus 11.0%, OR 1.35, 95% CI 1.10 to 1.66, p=0.004).
Mortality was comparable in Massachusetts and non-reporting states prior to public reporting (unadjusted rates 13.1% versus 13.8%, adjusted OR 0.89, 95% CI 0.77 to 1.02, Table 7). After the initiation of public reporting, we found no significant difference between Massachusetts and controls (11.7% versus 12.1%, OR 0.99, 95% CI 0.76 to 1.30; p for difference in differences=0.096). These results were consistent across risk groups, in our older cohort, and in both patients undergoing and not undergoing PCI ( Appendix Tables 7–8).
We examined mandatory public reporting of patient outcomes for PCI and found that, compared to states without public reporting, states with reporting programs in place had substantially lower rates of PCI among their acute MI patients. The lower PCI rates were particularly pronounced for patients with STEMI, shock, or cardiac arrest. In Massachusetts, which adopted public reporting relatively recently, the initiation of public reporting was associated with a significant decrease in the odds of receiving PCI. We found no evidence that public reporting was associated with lower overall 30-day mortality rates for patients with acute MI.
There are at least two possible explanations for why public reporting was associated with lower rates of PCI for patients admitted for an acute MI. It is possible that many of the foregone procedures were futile or unnecessary, and public reporting focused clinicians on ensuring that only the most appropriate procedures were performed. Alternatively, public reporting may have led clinicians to avoid PCI in eligible patients due to concern over the risk of poor outcomes. While policymakers have made efforts to ensure that risk adjustment models account for patient complexity, prior qualitative work suggests that clinicians remain concerned about receiving adequate “credit” for the comorbidities of their own patient population.1 Our data cannot definitively differentiate between these two potential mechanisms.
One way to ascertain whether the foregone procedures were appropriate or not is to examine mortality rates for AMI patients. In our analyses, we found that patients admitted for an acute MI had a somewhat higher mortality rate in public reporting states, although the differences were small, inconsistent, and usually not statistically significant. The weakness of this association suggests that the foregone procedures might have been a mix of appropriate and inappropriate PCIs. Another potential explanation for our mortality findings is that physicians in reporting states may have changed their coding practices in ways that made patients appear sicker than they actually were. These changes would likely bias our analyses away from finding an association between public reporting and worse outcomes. Therefore, although the lack of such an association is reassuring, we need new approaches to definitively understand whether outcomes were different in public reporting states.
Strategies to help clinicians differentiate between patients likely to benefit from PCI and patients for whom it would be futile are critically important. Promising work in this area is already underway.13,14 Providing real-time models of both risk and benefit may help physicians, patients, and families make more informed decisions about when to pursue PCI. Similarly, strategies to provide adequate credit for taking care of the sickest patients would also be useful. Massachusetts recently introduced a “compassionate use” category in their PCI reporting program that more accurately classifies extremely high-risk patients,15 and in 2008, New York began excluding patients in cardiogenic shock from its publicly reported outcomes.15 Whether these changes, instituted only recently, have been effective is not yet known. Follow-on data will be critical for determining if and how these policy changes influence patient selection for PCI.
Our findings are consistent with prior reports from single-state public reporting experiences. A study examining public reporting in the 1990s found that patients in New York were significantly less likely to undergo PCI after an acute MI than comparable patients in Michigan, a state without reporting.16 A more recent analysis of a small registry of patients with cardiogenic shock found that New York patients in shock were less likely than non–New York patients in shock to undergo PCI; in this study, the lower access to PCI was associated with higher mortality, although in a significantly younger population than our study.3 Studies examining public reporting and access to CABG surgery have yielded more mixed results, with some studies suggesting decreased access6,8 and others not demonstrating this relationship.7
Our study has several limitations. First, while we focused our analyses on patients admitted for an acute MI, public reporting targeted all patients undergoing PCI, and we could not determine whether public reporting was associated with a mortality benefit for non-MI patients. Of course, patients undergoing PCI for indications other than acute MI have very low mortality rates (0.65% in data from the National Cardiovascular Data Registry).14 Second, there was substantial heterogeneity within the public reporting states. For example, two of the three states only reported outcomes for hospitals while the third also reported outcomes for individual clinicians. Each state’s system for adjudication and monitoring of data is different. Finally, one of the three states (Massachusetts) expanded access to health insurance during the study period, which could have increased demand for PCI. Therefore, whether our results would be generalizable to all other states contemplating public reporting is unclear.
Because we used administrative data, we were unable to determine whether PCI was the most appropriate treatment in any specific clinical situation. However, indications for PCI should not be substantively different in public reporting states than in control states, and should not have changed over time in Massachusetts. Further, our use of administrative data limited our ability to fully account for potential up-coding by hospitals in public reporting states. We attempted to address this limitation by examining unadjusted outcomes. Additionally, any upcoding in the public reporting states should bias us towards finding better outcomes after the advent of public reporting. Thus, our findings may actually underestimate the relationship between public reporting and PCI utilization or outcomes. Finally, our analysis was limited to Medicare patients over the age of 65. Whether our findings would extend to a younger patient population is unclear.
We found that, among Medicare beneficiaries with acute MI, the use of PCI was lower for patients treated in three states with public reporting of PCI outcomes compared with patients treated in seven regional control states without public reporting. These differences were particularly large in the highest-risk patients. However, we found no evidence that public reporting was associated with better overall mortality for patients with an acute MI.
Dr. Joynt had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Funding for this study was provided by grant 1K23HL109177-01 from the National Heart, Lung and Blood Institute. The funder of this study, the National Heart, Lung, and Blood Institute of the National Institutes of Health, had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; and preparation, review, or approval of the manuscript.
Dr. Joynt, Dr. Blumenthal, Dr. Orav, Dr. Resnic, and Dr. Jha have no potential conflicts of interest to disclose.