|Home | About | Journals | Submit | Contact Us | Français|
Drug-eluting stents (DES) reduce rates of restenosis compared with bare metal stents (BMS). A number of observational studies have also found lower rates of mortality and non-fatal myocardial infarction with DES compared with BMS, findings not observed in randomized clinical trials. In order to explore reasons for this discrepancy, we compared outcomes after percutaneous coronary intervention (PCI) with DES or BMS by multiple statistical methods.
We compared short-term rates of all-cause mortality and myocardial infarction for patients undergoing PCI with DES or BMS using propensity-score adjustment, propensity-score matching, and a stent-era comparison in a large, integrated health system between 1998 and 2007. For the propensity-score adjustment and stent era comparisons, we used multivariable logistic regression to assess the association of stent type with outcomes. We used McNemar's Chi-square test to compare outcomes for propensity-score matching.
Between 1998 and 2007, 35,438 PCIs with stenting were performed among health plan members (53.9% DES and 46.1% BMS). After propensity-score adjustment, DES was associated with significantly lower rates of death at 30 days (OR 0.49, 95% CI 0.39 - 0.63, P < 0.001) and one year (OR 0.58, 95% CI 0.49 - 0.68, P < 0.001), and a lower rate of myocardial infarction at one year (OR 0.72, 95% CI 0.59 - 0.87, P < 0.001). Thirty day and one year mortality were also lower with DES after propensity-score matching. However, a stent era comparison, which eliminates potential confounding by indication, showed no difference in death or myocardial infarction for DES and BMS, similar to results from randomized trials.
Although propensity-score methods suggested a mortality benefit with DES, consistent with prior observational studies, a stent era comparison failed to support this conclusion. Unobserved factors influencing stent selection in observational studies likely account for the observed mortality benefit of DES not seen in randomized clinical trials.
The comparison of alternative treatments has long been a primary mission of both randomized trials and observational studies. With the commitment of $1.1 billion in support of comparative effectiveness research in the American Recovery and Reinvestment Act of 2009, the number of studies comparing different drugs, devices, techniques and systems will undoubtedly increase dramatically . While randomized clinical trials are likely to remain the gold standard for comparing alternative treatments, observational studies should continue to have significant, if not leading, roles in comparative effectiveness research moving forward, particularly in light of recommendations to prioritize assessments of community-based interventions within populations traditionally underrepresented in clinical trials .
However, observational studies are subject to a number of limitations, foremost among them the potential for unmeasured variables that confound results. While a number of methods directed at assessing causal effects and eliminating confounding have been developed, few clinical studies describe reasons for the specific choice of method used, and fewer present multiple methods to help corroborate findings[3,4].
Observational studies comparing drug-eluting stents (DES) and bare metal stents (BMS) for percutaneous coronary intervention (PCI) have consistently shown lower mortality and myocardial infarction associated with DES [5-20], findings not seen in randomized clinical trials [21-27]. We applied three common methods to compare DES to BMS within a large observational study population to: 1) determine whether mortality benefit for DES was observed in our study population and 2) to identify potential challenges to the application of these methods to compare treatments in the presence of strong treatment selection.
We conducted a retrospective dynamic cohort study within Kaiser Permanente of Northern California (KPNC), a large integrated healthcare delivery system caring for > 3.2 million individuals that are broadly representative of the local surrounding and statewide population. All health plan members aged 30 years and older between January 1998 and the end of December 2007 were considered eligible. The study was reviewed by the institutional review board of the Kaiser Division of Research, and requirement for informed consent was waived due to the nature of the study. From this cohort, we identified all PCI procedures using either DES or BMS based on relevant International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) and Current Procedural Terminology (CPT) codes that occurred within health plan and non-health plan hospitals . Consecutive procedures occurring within 7 days of one another were considered part of the same clinical episode. Procedures in which both DES and BMS were used were excluded from the analysis.
Patient demographic information was obtained from health plan electronic databases. Relevant ICD-9 or CPT codes found in hospital discharge databases during the eight years before the procedure date were used to identify prior cardiovascular disease including prior myocardial infarction, prior angina, prior PCI or coronary artery bypass graft (CABG) surgery, history of ischemic stroke and peripheral arterial disease. Prior chronic heart failure was determined based on a validated algorithm using information from hospitalization, outpatient and emergency department diagnostic codes.
Relevant data sources were searched for cardiovascular risk factors. Diabetes mellitus was identified from a validated longitudinal Diabetes Registry relying on inpatient and outpatient diabetes diagnoses, receipt of anti-diabetic therapies, and abnormal glycosylated hemoglobin or blood glucose levels . Hypertension was based on serial ambulatory diagnoses or the combination of diagnoses and receipt of anti-hypertensive medications. Chronic lung disease, systemic malignancy, and history of gastrointestinal bleeding were determined based on validated methods [29-35]. The presence and severity of chronic kidney disease was ascertained using the abbreviated four-variable Modification of Diet in Renal Disease (MDRD) equation for estimated glomerular filtration rate based on most recent outpatient determination of serum creatinine within the previous 24 months before the year examined.
Prior observational studies comparing outcomes for DES and BMS have not included the chronic outpatient use of cardioprotective medications, an important potential confounder [6,19,36]. Outpatient medication use within 30 days before PCI was ascertained from health plan pharmacy records for therapies known to lower cardiovascular risk, including beta-blockers , angiotensin-converting enzyme (ACE) inhibitors or angiotensin receptor blockers (ARBs), statins , and thienopyridines . Outpatient warfarin use was also assessed. More than 90% of patients had a drug benefit that provided strong financial incentives to obtain medications from health plan pharmacies.
Death from any cause occurring within thirty days and one year after the procedure was identified from health plan administrative databases, proxy information, Social Security Administration vital status files, and California state death certificate information[41,42]. Subsequent hospitalized myocardial infarction based on ICD-9-CM codes (primary discharge diagnosis coded as 410.x1) within one year was also identified.
Three separate methods were used to compare outcomes after DES and BMS: 1) statistical adjustment based on propensity score decile, 2) propensity-score matching, and 3) a stent era comparison, in which outcomes before and after the introduction of DES in 2003 were compared.
Because patients were not eligible to receive DES until after April of 2003, only PCI procedures after this date were included in the propensity score methods. A non-parsimonious logistic regression model was used to generate propensity scores for the likelihood of receiving DES based on 23 demographic and clinical variables. Propensity score deciles were created, and logistic regression was performed modeling the association of DES with outcomes, adjusted for propensity score decile (categorical variable) as well as the complete set of patient covariates. To accommodate data from individuals who may have undergone multiple procedures during the study period, robust standard errors accounting for clustering by patient were used.
In the second method, one-to-one nearest neighbor matching was performed to compare outcomes after DES and BMS in patients undergoing PCI after April 2003. Caliper width was set at 0.0001 using the STATA program psmatch2 (Leuven and Sianesi, 2003) . Adequacy of the match was assessed by estimating the standardized differences between DES and BMS patients for all variables. Outcomes were compared between groups using the McNemar's Chi-square test for matched comparisons.
Because DES was not available before 2003 and from 2004 onward DES was used in the vast majority of cases, a comparison of outcomes of PCI prior to and after DES introduction serves as a useful surrogate for assessing the impact of DES (Figure (Figure1)1) [4,36,44]. Procedures between January 1998 and March 2003 were assigned to the 'BMS era', while those between April 2003 and December 2007 were assigned to the 'DES era'. This variable would be expected to be strongly associated with treatment assignment, and less prone to unmeasured confounding by stent indication. Logistic regression was then performed to determine the association of stent era with outcomes, adjusted for all assessed covariates.
Analyses were performed using STATA (Stata Corp. 2008. Stata Statistical Software: Release 10, College Station, TX: StataCorp LP). A two-sided P value of < 0.05 was considered significant for all analyses.
A total of 35,438 PCIs with stent placement between January 1998 and December 2007 were included. Of these, 19,103 (53.9%) utilized DES and 16,335 (46.1%) utilized BMS. Patients receiving DES were significantly different than those receiving BMS in nearly all baseline characteristics, including having significantly higher rates of hypertensions, diabetes, and congestive heart failure (Table (Table1).1). DES patients also had lower rates of presentation with ST-elevation myocardial infarction within 30 days prior to PCI, and were more likely to be receiving outpatient cardioprotective medications including statins, beta-blockers and thienopyridines. Crude-mortality was significantly higher at thirty days (2.2% versus 1.6%, P < .001) and one year (5.1% versus 4.6%, P = 0.04) after PCI with BMS compared to DES. The rate of myocardial infarction at one year was 4.7% after BMS and 4.3% after DES (P = 0.06).
Patients receiving BMS after April 2003 were markedly different than those receiving BMS before April 2003, having higher rates of hypertension (65.0% versus 46.8%, P < 0.001), congestive heart failure (4.3% versus 3.2%, P = 0.008), prior PCI (9.3% versus 5.6%, P < 0.001), systemic malignancy (9.3% versus 5.2%, P < 0.001) and prior gastrointestinal bleeding requiring hospitalization (3.0% versus 1.7%, P < 0.001). Crude-mortality at thirty days and one year and myocardial infarction at one year after PCI were significantly higher in patients receiving BMS after April 2003 compared those receiving BMS before 2003 (Figure (Figure22).
After propensity-score adjustment, DES was associated with significantly lower mortality at thirty days (OR 0.49, 95% CI 0.39 - 0.63, P < 0.001) and one year (OR 0.58, 95% CI 0.49 - 0.68, P < 0.001), as well as a lower rate of myocardial infarction at one year (OR 0.72, 95% CI 0.59 - 0.87, P < 0.001). After 1:1 propensity-score matching (n = 4,126), DES was again associated with significantly lower mortality at thirty days (OR 0.55, 95% CI 0.35 - 0.85, P = 0.005), and at one year (OR 0.70, 95% CI 0.52 - 0.92, P = 0.01). Myocardial infarction at one year was similar (OR 0.94, 95% CI 0.71 - 1.25, P = 0.67). All standardized differences between groups were < 10%, suggesting an adequate match (Table (Table2).2). In the stent era comparison, no differences in mortality were seen when comparing the DES era versus BMS era at thirty days (OR 0.92, 95% CI 0.78 - 1.10, P = 0.39) or one year (0.92, 95% CI 0.82 - 1.02, P = 0.12), or in the rate of myocardial infarction at one year (OR 0.92, 0.82 - 1.03, P = 0.16) (Figure (Figure33).
We have shown that after the introduction of drug-eluting stents, bare metal stents were reserved for patients with significant comorbidities and high mortality rates. Commonly used propensity-score methods continued to suggest a mortality benefit to DES, while a stent era comparison showed no significant differences in all outcomes for DES and BMS, similar to results from randomized clinical trials.
A number of prior observational studies comparing DES to BMS have suggested reductions in mortality and myocardial infarction associated with DES, the majority of which have utilized propensity-score methods to mitigate potential confounding [5-20]. However, data from randomized trials have not corroborated these findings [21-27]. In a meta-analysis of 22 randomized trials and 34 observational studies, Kirtane et al found no differences in mortality or myocardial infarction in randomized trials, but consistent reductions for both outcomes in observational studies, and attributed these differences, at least in part, to residual confounding by unmeasured differences between patients receiving BMS and DES . Our results are consistent with this conclusion. In our cohort, patients who received BMS after April 2003 had significantly greater comorbidities and unadjusted death and myocardial infarction rates compared to patients who received BMS before this date, suggesting that physicians selectively reserved BMS for a sicker population of patients after DES was introduced. While propensity-score adjustment and matching both attempt to account for such differences, they only account for variables that are assessed. In the presence of such a high degree of patient selection, any residual confounding in this case is likely to lead to the appearance of improved outcomes with DES, as was seen in this study. While propensity scores have been put forward as a method which overcomes some of the limitations of traditional logistic regression, they do not address a primary threat to the validity of such studies - unmeasured confounding.
A comparison of the BMS era (100% BMS) to the DES era (88.3% DES) as a surrogate for a BMS-DES comparison eliminates the unwanted influence of patient selection . Consistent with this, the stent era comparison showed no differences in mortality or myocardial infarction between BMS and DES, most closely approximating results seen in randomized trials. However, because such an analysis compares patients undergoing PCI at two different times, important secular trends may have influenced the results. For example, comparisons of stent eras in the Medicare population and in the New York PCI Registry both suggested lower rates of myocardial infarction in the DES era compared to BMS era [36,44]. However, neither study adjusted for differences in the use of outpatient medications such as statins and beta-blockers, which have increasingly been used in patients with coronary disease over time . In this study, we found significantly higher rates of use of beta blockers, statins, angiotensin-converting enzyme inhibitors, and thienopyridines in the DES era. After accounting for these and other secular trends, no differences in myocardial infarction were observed between DES and BMS patients.
A number of circumstances specific to stent use allowed for the critical examination of these analytical methods. First, the rapid uptake of DES over BMS created a natural experiment allowing use of a stent era comparison. Second, because a large number of patients from randomized clinical trials comparing DES with BMS have been performed, these studies help provide an estimate of the 'truth.' However, in other examples in which the questions of interest may be less well studied, there could be much greater difficulty interpreting disparate results.
In summary, we have compared outcomes after PCI with either DES or BMS, using three commonly used methods of adjustment in observational studies. Large baseline differences in BMS and DES groups suggested strong treatment selection, which were incompletely adjusted for by propensity-score methods. In the presence of such treatment selection, alternative methods of analysis of observational data should be considered.
ACE: angiotensin-converting enzyme; ARBs: angiotensin receptor blockers; BMS: bare metal stents; CABG: coronary artery bypass graft; CPT: Current Procedural Terminology; DES: drug-eluting stents; ICD-9-CM: International Classification of Diseases: Ninth Edition: Clinical Modification; KPNC: Kaiser Permanente of Northern California; MDRD: modification of diet in renal disease; PCI: percutaneous coronary intervention.
The authors declare that they have no competing interests.
RWY conceived of the original idea for the study, conducted statistical analyses, and drafted the manuscript. MC assisted with analysis of the data and critically reviewed the manuscript. CEC advised on biostatistical methodology of the study and provided critical revisions of the manuscript. ASG supervised in the conception and design of the study and provided critical revisions of the manuscript. All authors read and approved the final manuscript.
No additional acknowledgements.
This work was funded by institutional funds from the Massachusetts General Hospital and Kaiser Permanente of Northern California.
The pre-publication history for this paper can be accessed here: