|Home | About | Journals | Submit | Contact Us | Français|
The purpose of this study was to develop a long-term model to predict mortality after percutaneous coronary intervention (PCI) both in patients with ST elevation myocardial infarction (STEMI) and those with more stable coronary disease.
The American College of Cardiology Foundation CathPCI Registry® data was linked to the Centers for Medicare and Medicaid Services (CMS) 100% denominator file by probabilistic matching. Pre-procedure demographic and clinical variables from the CathPCI registry were used to predict the probability of death over three years as recorded in the CMS database. Between 2004 and 2007, 343,466 patients (66%) of 518,195 patients age 65 or older undergoing first PCI in the CathPCI Registry were successfully linked to CMS data. This study population was randomly divided into 60% derivation and 40% validation cohorts. Median follow-up was 15 months, with mortality of 3.0% at 30 days, and 8.6%, 13.4% and 18.3% at 1, 2 and 3 years, respectively. Twenty-four characteristics related to demographics, clinical co-morbidity, prior history of disease, as well as indices of disease severity and acuity, were identified as being associated with mortality. The c indices in the validation cohorts for patients with and without STEMI were 0.79 and 0.78. The model calibrated well across a wide range of predicted probabilities.
Based on the large and nationally representative CathPCI Registry, we have developed a model that has excellent discrimination, calibration and validation to predict survival up to three years after PCI.
Prediction models are useful in understanding the long-term clinical outcome after an index event. Prediction models can also be useful in guiding decision making. Multiple such models have been created in patients with cardiovascular diseases. Percutaneous coronary intervention (PCI) is the most common form of myocardial revascularization, and thus a validated long-term prediction model after PCI would be of considerable interest. To date, however, PCI prediction models have generally concentrated on early results either in-hospital or 30 day outcomes.1, 2
The American College of Cardiology Foundation (ACCF) and The Society of Thoracic Surgeons (STS) Collaboration on the Comparative Effectiveness of Revascularization Strategies (ASCERT) project represents a unique collaboration among two professional societies and an Academic Research Organization (the Duke Clinical Research Institute). The project has been funded by the National Heart, Lung and Blood Institute (NHLBI) to study the comparative effectiveness of PCI versus coronary artery bypass graft (CABG) surgery.3 The clinical databases in ASCERT are composed of the ACCF National Cardiovascular Data Registry (NCDR) CathPCI Registry® and the STS Adult Cardiac Surgery Database.4 Long-term follow-up data are provided by linking these clinical databases to the Centers for Medicaid and Medicare Services (CMS) 100% denominator file.5 The current study uses the linked CathPCI Registry-CMS database to examine the longitudinal outcomes of patients age 65 years or older for up to three years following PCI.
CathPCI Registry® is an initiative of the American College of Cardiology Foundation and The Society for Cardiovascular Angiography and Interventions. The CathPCI Registry includes 2,001,529 PCI records from patients discharged between 2004 and 2009 from 1,032 participating hospitals (Figure 1).5 Patients aged 65 years or older who underwent PCI between 2004 and 2007, with data in the ACC-NCDR v.3 dataset, were linked to CMS claims files using a probabilistic matching algorithm that overcomes the need for a universal patient identifier.5 The establishment of this CMS-registry link then permitted the identification of patients in common with the CathPCI Registry®.
The NCDR has several mechanisms in place to ensure data quality, including electronic data capture by hospitals using a certified data collection tool, automated checks of submitted data for completeness, and an annual on-site audit program that involves reviewing charts at a select number of sites to ensure data accuracy. A professional audit, independent of the NCDR, reported a 93% median agreement between hospital records and data submitted to the CathPCI Registry in 2010.
Records in these two databases were considered to refer to the same patient if they matched on a set of indirect identifiers including patient date of birth and sex, and hospital name, admission date and discharge date. The matching algorithm required an exact match on some of these variables and a partial matching on several variables. Once the individual patient records were linked, longitudinal records were created containing follow-up information, such as subsequent death and all subsequent hospitalizations, as a unified record. Follow-up for mortality was available for all patients until December 31, 2007.
This project was approved by the ACCF’s independent Institutional Review Board (IRB), Chesapeake Research Review, Inc. (CRRI), as well as the IRBs of Duke University and Christiana Care Health System.
Candidate variables, 25 total, were selected from the CathPCI Registry. All pre-procedural variables from the diagnostic catheterization and PCI database v3.0 were considered.6 Glomerular filtration rate (GFR) was estimated using the Modified Diet and Renal Disease equation.7. The number of diseased vessels was derived from variables in the CathPCI Registry, using ≥70% diameter stenosis for the left anterior descending, right coronary, circumflex and ramus coronary arteries and saphenous vein or arterial bypass grafts to define significant obstruction, except for the left main where >50% diameter stenosis was considered significant. Subgroups of interest were defined in advance, and included gender, age ≤75 and >75, diabetes, chronic kidney disease, number of coronary arteries diseased, left ventricular function defined by ejection fraction, chronic lung disease and peripheral vascular disease. Data are displayed as proportions or mean ± standard deviation. The models were based on variables available at the start of the PCI procedure; thus, procedural details and complications were not included.
The study sample was stratified by the presence or absence of ST elevation myocardial infarction (STEMI) at admission and each group was randomly divided into 60% derivation and 40% validation cohorts. The development sample was used to determine the form of the model and estimate regression coefficients. Data from the validation sample were used to assess model discrimination and calibration.
Predictors of mortality were determined by Cox model analysis.8 Variables were included in the final model for clinical relevance and/or if they contributed to the model significantly (95% CI of the hazard ratio did not include 1) to at least in one of the periods in at least one group, STEMI or Without STEMI. We used cubic spline plots to explore the functional form of continuous variables. For each continuous variable, we found that its effect on the log-hazard of mortality was approximately linear below a threshold and approximately constant above the same threshold. Based on these exploratory analyses, ejection fraction (EF) was modeled as linear below 60% and constant above 60%. Body mass index (BMI) was modeled as linear below 30 kg/m2 and constant above 30 kg/m2. We found that patients with GFR ≤ 30 had similar risk as dialysis patients. Therefore, dialysis and GFR≤30 were combined into a single “renal failure” category. For patients without renal failure, the effect of GFR was modeled as linear between 30 and 70 ml/min/1.73 m2 and constant above 70 ml/min/1.73 m2.
The goal of the study was to produce a predictive model, potentially including covariates for which the hazard ratios varied over time. A single model incorporating ‘time-dependent covariates’, however, would have resulted in a model that depended on continuous time parameters and would not have led to a straightforward presentation of results. Therefore, to investigate the proportional hazards assumption and to simplify the modeling and presentation of results, a discrete mechanism was constructed to reflect these calculations in reasonable intervals. Hazard ratios for all variables were assessed separately for the following time periods: from the day of the procedure to 1 month, 1 to 12 months and more than 12 months. These time periods were selected to account for peri-procedural, mid-term and longer term mortality. To account for within-hospital clustering, 95% confidence intervals were computed using sandwich standard error estimates.9 As expected, several variables demonstrated evident differences in hazard ratios over the three time periods. Final variables were kept in the models based on whether they either demonstrated significant interactions over the time intervals or contributed significantly within an interval. The final model was constructed by first including all of these variables in Cox proportional hazards models for each of the three intervals. Because each subsequent interval is conditional on survival in the previous one, we combined the models together to compute unconditional probabilities of survival from procedure to 3 years after PCI. Mathematically, this was equivalent to fitting a single Cox model with piecewise-constant hazard ratios for all model variables..
Separate models were created for patients undergoing primary PCI for an ST elevation myocardial infarction (STEMI) and all other patients (Without STEMI). The decision to create separate models was based on the different pathophysiologic and clinical circumstances of primary PCI as well as the considerably higher mortality in these patients. In each model, interactions were examined by identifying 5 predictors with the highest global chi-square statistics and creating all possible pairwise interactions among them. While some interaction terms were statistically significant, measures of model calibration and discrimination were not materially affected by their inclusion, and models without interactions were considered to be substantially more interpretable and usable. Therefore, models with only main effects were presented.
Predictor data were highly complete with most covariates having ≤1% missing data. An exception was ejection fraction (missing 28%), which was modeled separately with an indicator variable for missing values. This approach was adopted because failure to measure ejection fraction may be a proxy for unmeasured patient baseline characteristics which are potentially informative for the purpose of predicting the patient’s subsequent risk of death. For all other variables, missing categorical variables were imputed to the most common value and missing continuous variables were imputed to relevant group-specific medians to improve prediction of missing values.
Finally, model calibration and discrimination were evaluated in the 40% validation sample. Predicted survival curves were generated by applying regression estimates from the development sample to covariate data from the validation sample. To assess calibration graphically, the average model-predicted survival probabilities among patients in the validation sample were plotted as a function of time and compared to non-parametric (Kaplan-Meier) survival estimates, overall and for each subgroup of interest. Model-based and non-parametric estimates of 1-year mortality risk were compared across deciles of model-based estimated 1-year risk. Finally, model discrimination was assessed using Harrell’s C statistic, which represents the probability that among two randomly selected patients, the patient who survived longer had a lower predicted risk of mortality.10 The C statistic was estimated separately for survival up to 30 days and at 1, 2 and 3 years. However, results were very similar and we only report two-year C-indices, overall and for each subgroup of interest. SAS (version 9.2, SAS Institute, Cary NC) and R (version 2.13.0) statistical software were used for all statistical testing.
Among 2,001,529 PCI admissions in the NCDR database between January 1, 2004 and June 20, 2009 (Figure 1), there were 1,040,889 patients discharged after PCI between 2004 and 2007, of whom 518,195 were age 65 or older. Of these 518,195 patients, successful linkage to the CMS database was accomplished in 343,496 (66%). The comparison of linked to unlinked data is available in an electronic appendix (Table e1). Due to uncertainty about the date of death, 30 additional patients were excluded from the final dataset for analysis, resulting in a final study population of 343,466.
The 343,466 patients in the study population were randomly divided into a derivation cohort with 206,081 patients (60%) and a validation cohort with 137,385 patients (40%). Patient characteristics overall, in the STEMI and in the Without STEMI study populations of the derivation cohort are displayed in Table 1. Overall, these groups are representative of patients undergoing PCI with the exception that the study population was limited to those over age 65. The mean age of our study population was 75 years and the majority of patients were white. The mean BMI was within the overweight range, and just under half never smoked. Diabetes was present in a substantial minority, and the majority had hypertension. Prior revascularization was common. A history of heart failure was noted in a minority, and comorbidities were common. Mean ejection fraction was below 0.50 for STEMI patients, above 0.50 for Without STEMI. While multivessel disease was common, left main disease remained a small minority. Within the group Without STEMI, most cases were elective or urgent, while most cases of STEMI were emergent or salvage.
Median follow-up was 15 months from index procedure. Mortality is summarized in Table 2. Of the 206,081 patients in the derivation cohort, 22,012 died, of whom 4,526 were in the STEMI group and 17,486 in the Without STEMI group. Mortality at 3 years was 18.7% overall, 25.4% in the STEMI group and 17.7% in the group Without STEMI.
Multivariable predictors of mortality in the derivation study populations for STEMI and Without STEMI are shown in Table 3. The 30-day, 1-year and 2-year c indices in the validation dataset were 0.78, 0.79 and 0.79 for STEMI and 0.76, 0.78 and 0.78 for the Without STEMI models, respectively. The hazard ratios (HRs) and 95% confidence intervals (CIs) are shown for each group at less than 1 month, 1 to 12 months and greater than 12 months. In general, the 95% confidence intervals were relatively small due to the size of the study population. Many variables were predictive, although few had large hazard ratios. Predictive variables include demographics, co-morbidity, prior procedures, severity of illness, and urgency of presentation. Variables related to anatomical severity of disease, such as ejection fraction and left main disease, or to severity of disease at presentation, such as cardiogenic shock, were relatively powerful predictors of mortality. Some variables, such as age, had relatively constant hazard ratios across the time periods. Others, such as male sex, were initially neutral or associated with decreased risk, but showed increased risk over time. Some, such as prior valve surgery, were unstable due to low numbers of patients. Finally, variables associated with the acuteness of the presentation, such as cardiogenic shock, portended increased risk during the first month, but not long term.
Calibration of the model in the validation population is presented in Figure 2, with observed and predicted risk displayed across the spectrum of risk from under 5% to over 70% risk in STEMI (Panel A) and from approximately 2% to 40% in patients Without STEMI (Panel 2b). The observed and predicted risks were almost identical up to 20% predicted mortality; above this range mortality risks was underestimated to some extent.. Survival curves for both STEMI and Without STEMI groups in the validation population are displayed in Figures 3, with slight under prediction of mortality in the STEMI patients (Panel 3A), and with observed and predicted curves being nearly identical for patients Without STEMI (Panel B). Note that there is significantly higher initial mortality with STEMI than Without STEMI, accounting for most of the difference in survival by 1000 days. Observed and predicted survival in subgroups of STEMI and Without STEMI in the validation population are displayed in Figures 4 and and5.5. There is a difference in survival for all subgroups except by sex. The observed and predicted survival rates are quite similar in subgroups with larger sample sizes, but varied somewhat from the predicted in STEMI subgroups with smaller sample sizes.
By linking the CathPCI Registry to the CMS 100% denominator file we have developed prediction models for survival for up to three years following PCI in a dataset of 343,466 patients ages 65 and older. The large size of the study population has resulted in narrow 95% CIs, both for estimates of survival and hazard ratios. The variables observed to be predictive of mortality included demographics, co-morbidity, severity of disease, and acuteness of presentation. Acuteness of presentation is most predictive early in follow-up, while older age, co-morbidity and severity of disease are more predictive over the longer term. The models had excellent c indices for long-term models as well as nearly perfect calibration. Restriction of the study population to patients over age 65 removes much of the influence of age on outcome and thus leads to a lower c index than would be observed in the larger dataset. Furthermore, the observed and predicted survivals curves and estimated risk by decile in the validation dataset almost completely overlapped out to three years of follow-up, both in the overall cohort and within multiple subgroups.
The models that we have developed are consistent with those previously developed from the NCDR for in-hospital mortality.1, 2, 11 Variables that predict mortality generally include demographics, co-morbidities, severity of disease, and acuteness of presentation. Variables reflecting acuteness at presentation generally have the largest hazard ratios. In-hospital models generally calibrate well and have high c indices as a measure of discrimination. By linking the NCDR CathPCI Registry to the CMS 100% denominator file, Curtis et al12 created models to predict 30-day mortality in patients with STEMI and in all other patients using data from patients undergoing PCI in 2006. Acuteness of presentation was the strongest predictor in both models. The models had both good discrimination (c index 0.83 for the STEMI/shock model and 0.82 for the non-STEMI/non-shock model) and calibration. Validation of these models was performed by re-analysis in the group undergoing PCI in 2005.
There are also a number of models that have been developed utilizing other databases from the Society for Cardiac Angiography and Intervention,13 Northern New England,14 Michigan,15 Washington State,16 New York State,17 as well as single site databases18–20 to predict in-hospital mortality and complications after PCI. Most models included demographic variables, co-morbidity, disease severity and acuteness.
There have also been several long-term models. Addala et al21 developed a risk score for long-term mortality after primary intervention in acute myocardial infarctions using pooled data from the Primary Angiography in MI (PAMI) trials. Mackenzie et al22 developed models to predict death long-term after PCI using data from the Northern New England database, with data on PCI’s from 1992 through 2001. There were 19,806 patients with mortality data obtained from the National Death Index. Data were divided into follow-up periods of 0 to 91 days, 4 to 18 months and more than 18 months, with c indices of 0.83, 0.78 and 0.76 respectively. Calibration was excellent. Correlates of mortality included demographics, co-morbidity, severity of disease, and acuteness of clinical presentation. The observed predictors were similar to those in the present study. For instance, the hazard ratio was below 1.0 for male sex short-term, but above 1.0 longer term. A history of diabetes, heart failure, or kidney disease and decreased ejection fraction increased risk during all time periods. Emergent procedures increased risk primarily short-term.
The model we present in this paper is by far the largest to date with long term mortality outcome of PCI. By combining the clinical and angiographic predictors of NCDR with the outcomes collected by CMS, this is a unique collaboration, combining two databases of unquestioned authority. This model offers the best available prediction tool for the Medicare population. This model can therefore be utilized by clinicians and payors to evaluate the likely outcomes of PCI in this population, supplementing clinical judgment. Moreover, the results are based on contemporary data from the period when drug eluting stents were used. The size of the dataset permits remarkably high precision in the estimates of hazard ratios and expected survival. The models have been thoroughly evaluated for discrimination, calibration and validation. The dataset also comes from 791 sites all over the United States, making it the most representative and generalizable of its kind. Previous models using the NCDR were limited to in-hospital results and more recently 30-day outcomes.2, 12 Short-term models are clearly limited when compared to long-term models in predicting outcome.
The present study is based on linkage of the CathPCI Registry to the CMS 100% denominator file, which is an administrative database. Covariate information related to co-morbidity, severity of disease and acuteness of presentation is limited in such an administrative database.23–25 Linkage of clinical and administrative databases offers the more accurate assessment of clinical covariates along with the availability of long-term outcome data. Probabilistic matching offers an approach to link the databases without patient identifiers, which can resolve both Health Insurance Portability and Accountability Act (HIPAA) and informed consent limitations. Careful IRB approval is clearly needed for such projects.
The present model will be available to CathPCI Registry participants and through the ASCERT website to aid in individual risk assessment. The model, however, is only directly applicable in patients who have had the procedure. There is added uncertainty when such models are used to compare outcomes of several alternative therapeutic choices, as this would assume that the patients being considered for alternative approaches would be fully described by the variables considered in the model. Alternative treatments are best considered in the context of randomized trials or carefully conducted observational studies. Such methods, however, cannot consider specific patient characteristics, which may best be accounted for using risk models such as the one presented here. Thus, the ability to assess risk for each patient can aid in decision making.
This study has a number of limitations. The model is only as good as the quality of the data. Furthermore, a number of the data elements, of which several are related to acuteness of the procedure, are relatively subjective and have relatively high hazard ratios. Hospitals are aware that coding of such variables can affect benchmarking of their outcomes, which might potentially lead to preferential coding bias. All variables, however, are carefully defined and an audit process is in place to verify the data. The study may not represent all hospitals, as there are hospitals that do not participate in the CathPCI Registry. It is, however, the most representative national database of outcomes following PCI. The CathPCI Registry has only limited angiographic data and limited data related to severity of ischemia, both of which may limit the ability of these models to predict outcome. More detailed anatomic data, such as that derived from the SYNTAX score, may enhance the model.26, 27 Finally, the results of this analysis can only be applied to patients ages 65 or older.
In conclusion, we have developed a long-term mortality model after PCI based on the largest data set to date, using contemporary data from the CathPCI Registry and long-term outcome data from the CMS 100% denominator file. This model may be used to predict mortality of patients undergoing PCI, and may aid in medical decision making. Future studies will consider models that include non-fatal outcomes and cost. It may also be possible to investigate the impact of such models on medical decision making and outcomes. These models may also help inform the design of future clinical trials concerning revascularization strategies. This study did not require any new data collection, and thus the model can be updated or extended relatively easily. This model, as well as future models based on these methods, will permit physicians and patients to estimate prognosis both initially and during follow-up after a procedure or event with considerable precision.
Support: The ASCERT Study is supported by Award Number RC2HL101489 from the National Heart, Lung, and Blood Institute. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Heart, Lung, And Blood Institute or the National Institutes of Health. This award has been issued under the American Recovery and Reinvestment Act of 2009 for a two-year period. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Heart, Lung, And Blood Institute or the National Institutes of Health
Grants/grants pending: Dangas, DeLong, Edwards, Garratt, Grau-Sepulveda, Grover, Klein, Kolm, Mayer, Moussa, O’Brien, Peterson, Popma, Ritzenthaler, Shahian, Shaw, Weintraub, Weiss
Expert Witness: Dangas, Weintraub
Consultancy: Dangas, Popma, Weintraub