|Home | About | Journals | Submit | Contact Us | Français|
The impact of risk factors upon perioperative mortality might differ for patients undergoing open versus endovascular repair (EVAR) of abdominal aortic aneurysms (AAA). In order to investigate this, we developed a differential predictive model of perioperative mortality after AAA repair.
A total of 45,660 propensity score matched Medicare beneficiaries undergoing elective open or endovascular AAA repair from 2001–2004 were studied. Using half the dataset we developed a multiple logistic regression model for a matched cohort of open and EVAR patients and used this to derive an easily evaluable risk prediction score. The remainder of the dataset formed a validation cohort used to confirm results.
The derivation cohort included 11,415 open and 11,415 endovascular repairs. Perioperative mortality was 5.3% and 1.8% respectively. Independent predictors of mortality (RR, 95% CI) were open repair (3.2, 2.7–3.8), age (71–75 years 1.2, 0.9–1.6; 76–80 years 1.9, 1.4–2.5; >80 years 3.1, 2.4–4.2), female sex (1.5, 1.3–1.8), dialysis (2.6, 1.5–4.6), chronic renal insufficiency (2.0, 1.6–2.6), congestive heart failure (1.7, 1.5–2.1), and vascular disease (1.3, 1.2–1.6). There were no differential predictors of mortality across the two procedures. A simple scoring system was developed from a logistic regression model fit to both endovascular and open patients (area under the ROC curve of 72.6) from which low, medium, and high risk groups were developed. The absolute predicted mortality ranged from 0.7% for an EVAR patient ≤ 70 years of age with no comorbidities to 38% for an open patient > 80 with all the comorbidities considered. Although relative risk was similar among age groups, the absolute difference was greater for older patients (with higher baseline risk).
Mortality after AAA repair is predicted by comorbidities, sex, and age and these predictors have similar effects for both methods of AAA repair. This simple scoring system can predict repair mortality for both treatment options and thus may help guide clinical decisions.
Open abdominal aortic aneurysm (AAA) repair has been shown to have higher early mortality compared to endovascular (EVAR) repair.1–3 For this reason, when anatomic factors allow, there is a mounting preference toward using EVAR, particularly for older and sicker patients who would be less likely to survive open surgery. Long-term outcomes, however eventually converge, with similar survival after several years of follow-up.1,4,5 Also favoring endovascular repair are shorter duration hospital stays, quicker functional recovery, fewer post-operative complications, fewer laparotomy-related reinterventions, and lower initial costs. Follow-up monitoring regimens, however, are more intensive after endovascular repair, and aneurysm-related re-interventions are more frequent.1 Thus, the decision to pursue open versus endovascular repair hinges on many factors and is not always straightforward.
Using comprehensive data on elderly enrollees in the Medicare program, we previously found that in-hospital mortality for endovascular AAA repair and open repair were 1.2% and 4.8% respectively, but that early mortality differences increased with age.1 What remains unknown, however, is if there are specific clinical factors that might differentially impact early survival after open and endovascular AAA repair and thus influence the decision to perform one procedure over the other.
Predictive risk models such as the Glasgow Aneurysm Score (GAS), Leiden Score, and Hardman Index have been derived for open AAA repair and, while they have variable utility for open repair, they overpredict mortality in the EVAR population.6–13 To our knowledge, no independent predictive models have been devised from EVAR data nor have these studies attempted to assess outcomes in terms of whether factors may differentially affect mortality after EVAR versus open repair. We therefore sought to identify clinical and demographic factors related to perioperative mortality for both types of repairs and to identify specifically predictors with a differential effect to the extent that they exist. The results of this analysis will provide a clinical tool for the selection of patients best suited for endovascular or open repair and will assist clinicians in estimating mortality probabilities for Medicare patients undergoing AAA repair.
We used data from the Medicare population to identify all traditional Medicare beneficiaries who underwent either open or endovascular repair of an AAA during the time period 2001–2004. Data from Part A and B claims as well as the Medicare Denominator file were used to allow analysis of inpatient, outpatient, and survival information. Using a split sample approach, we developed clinical models predicting perioperative mortality and identified clinical predictors that differentially impacted each of the procedures. The models were validated on the other portion of the split sample. The study was approved by the Institutional Review Board at Harvard Medical School and all analyses were performed by a collaborating author (P.C.) at the Centers for Medicare and Medicaid Services (CMS).
Using both ICD-9-CM diagnosis and procedure codes (International Classification of Diseases, 9th Revision, Clinical Modification) and CPT codes (Current Procedural Terminology, American Medical Association), we identified all traditional Medicare beneficiaries age 67 or older with at least two years of prior Medicare claims who underwent either endovascular or open repair of an elective AAA from 2001–2004. Patients with a diagnosis of ruptured AAA, thoracic aneurysm, thoracoabdominal aneurysms, or aortic dissection were excluded along with patients having procedure codes for repair of the thoracic aorta or visceral bypass. Also excluded were beneficiaries enrolled in health maintenance organizations at any time during the study period and those beneficiaries not enrolled in Medicare Part B. The patient selection methods are available in detail elsewhere.1
We measured clinical co-morbidities during the two years prior to the index admission using a version of the Elixhauser algorithm that was adapted to also include diagnoses that occurred only in the outpatient setting.14,15 Candidate comorbid conditions included were those that were identified in prior studies as impacting mortality (Table I). Baseline beneficiary demographic characteristics were obtained from the Medicare denominator file. Age was categorized as 67–70, 71–75, 76–80, or > 80 years and race was classified as white, black, or other. Prior vascular disease was defined as either a prior history of cerebrovascular disease or peripheral vascular disease.
Perioperative mortality was defined as in-hospital death or death within 30 days of the procedure. Date of death was identified from the Medicare denominator file. We included in-hospital deaths that occurred more than 30 days after the procedure because a substantial number of deaths occur in this period and these deaths were thought to be directly related to the procedure or complications from the procedure.16
In order to assure that the populations undergoing each procedure were comparable, we created matched cohorts of patients using a propensity score approach. As explanatory variables we used available baseline beneficiary demographic and clinical characteristics obtained from claims during the 2-year period prior to and not including the index admission. We measured clinical co-morbidities using a version of the Elixhauser algorithm as noted above.14,15 We matched each beneficiary who underwent endovascular repair to a beneficiary who underwent open repair based on the estimated propensity score. To ensure close matches we required the estimated log-odds of endovascular repair between a patient who underwent endovascular repair and one who underwent open repair to be within 0.60 standard deviations. This value removes approximately 90% of the bias in estimates of effects due to differences in covariate distributions between “treatment” and comparison groups.17,18
After matching, we randomly divided the study population into derivation and validation cohorts using a computer generated randomization program. The bivariate relationships between perioperative mortality and categorical variables were compared using χ2 tests. All statistical analyses were performed using SAS 9.1 statistical software (SAS Institute Inc., Cary, NC). Using the derivation cohort, we identified significant variables by analyzing mortality risk associated with each demographic and comorbid factor in univariate and multivariate logistic regression models. This was done separately for both endovascular and open repair. Variables meeting statistical significance at the P <.10 level were then included as predictor variables in a combined model with endovascular repair. Interaction terms were tested for each demographic and comorbid variable to test for differential effects based upon repair type. The final regression model included all age categorical variables and all other variables and interaction terms that remained significant at the P <.05 level. This final model was then translated into a probability formula, which we used to derive a simple scoring system. We used the weighted sum of the predictors from the logistic regression model multiplied by their regression coefficients, and then transformed these onto the [0,1] interval to yield a predicted probability of mortality for each possible combination of characteristics. To make the predictive model as easy as possible to present, we compute “scores” for patients based on the weighted sum as opposed to presenting the probabilities themselves. We then partitioned the predicted mortality rates for each repair type into low, medium, and high risk scores range based on the 25th and 75th percentiles of their probability distributions as these corresponded to clinically appropriate mortality probability cut-offs. This assigned 25% of the patients as low risk and 25% as high risk.
We then tested the scoring system using the validation cohort of our sample by examining the area under the receiver operator characteristic (ROC) curve (the c-statistic). A similar area under the curve (AUC) for the derivation and validation cohorts suggests that the model discriminates the outcome of interest (mortality) equally well in the two datasets and that there is no evidence of “overfitting” (i.e., including predictors that correlate with random variation in the derivation cohort).
We identified a total of 61,598 patients undergoing repair of an intact AAA during the years 2001–2004. Endovascular repair was performed in 29,542 and open repair in 32,056. After propensity matching and randomization into derivation and validation cohorts, this left a total of 22,830 patients (11, 415 EVAR, 11,415 Open Repair) in both the derivation and validation cohorts. The population characteristics for each repair group within both cohorts are listed in Table I. About 30% were between the ages of 71–75 years old and 20% were female. Approximately 8% had suffered a myocardial infarction in the last two years and just over 15% had diabetes. There were few significant differences between the patients undergoing EVAR or open repair in either the derivation or validation cohorts. For instance, there were slightly more patients who had undergone CABG surgery within the last two years in the open as compared to the EVAR cohorts (4.4% v. 6.9% for the derivation cohort, p<.001).
Overall perioperative mortality over the study period was 1.8% (95% CI 1.5–2.0) after EVAR and 5.3% (95% CI 4.9–5.7) after open repair. There were no significant differences in mortality from 2001 to 2004 for either method of repair. The relative risk (RR) of mortality after open repair versus EVAR was similar across all age groups, ranging from 2.6 to 3.2. The absolute mortality difference between open and EVAR was lowest for the youngest age group at 1.6% and rose incrementally with age to a maximum of 6.1% for patients over 80 years. Mortality was higher for females than males for both repair types (EVAR RR=2.0, P<.0001; Open RR=1.5, P<.0001).
Significant univariate predictors of mortality after EVAR were age, female sex, renal disease, recent myocardial infarction, congestive heart failure, vascular disease, valvular heart disease, and chronic obstructive pulmonary disease (Table II). Coronary artery disease with recent percutaneous coronary angioplasty was associated with lower mortality.
Comparatively, univariate predictors of mortality after open repair included age, female gender, renal disease, coronary artery disease without a recent procedure, congestive heart failure, vascular disease, valvular heart disease, hypertension, diabetes mellitus, and chronic obstructive pulmonary disease (Table II). Coronary artery disease with a recent coronary artery bypass surgery was associated with lower mortality after open repair. Most predictors had similar effects for both EVAR and open repair while some varied by repair method, suggesting it would be worthwhile to test the associated interaction effects.
Open repair was a significant predictor of mortality with a 3-fold increase compared to EVAR (Table III). The most significant predictors of mortality (OR, 95%CI) were open repair (3.2, 2.7–3.8), age > 80 years (3.1, 2.4–4.2), and end-stage renal disease (2.6, 1.5–4.6). Age from 76–80 (1.9, 1.4–2.5), female gender (1.5, 1.3–1.8), chronic renal insufficiency (2.0, 1.6–2.6), congestive heart failure (1.7, 1.5–2.1), and vascular disease (1.3, 1.2–1.6) were other significant predictors. After testing for interactions between the type of repair and each of the significant predictors, no variables were identified that had a differential impact between the two repair types.
Based on the predictive probabilities of mortality, the differences in absolute mortality and relative risk between EVAR and open repair were 16% and 2.4 respectively for the oldest age group with all other predictive factors whereas they were 4.3% and 3.1 for the same age group with no other predictive factors (Table IV). Conversely, for the youngest age group the absolute mortality difference and relative risk were 11.3% and 2.9 with all other predictive factors present and 1.3% and 3.2 without any other predictive factors.
The full logistic regression equation for calculation of mortality probability* was simplified into a risk score as represented by the following sequence of equations (I(event)= 1 if event is true and 0 otherwise).
Another way of stating this is to simply sum the coefficients for each characteristic as shown in Table V. A coefficient of +12 would be attributed to open repair, making it the highest weighted component. As each characteristic was equally weighted regardless of repair type, however, it was decided to apply the effect of open repair on translation of the score into mortality estimation. The division of low, medium, and high risk corresponds to score ranges of <3 (low), 3 to 11 (medium), and >11 (high) with respective mortality probabilities of <1%, 1% to 2%, and >2% for EVAR and <3%, 3% to 6%, and >6% for open repair (Table V).
The area under the ROC curve (AUC) for the predictive probabilities was 72.6 within the derivation cohort and 71.8 within the validation cohort. Applying the cutoff value of the high risk group, there was a 56.7 and 68.0 sensitivity and specificity for open patients and a 61.2 and 66.8 sensitivity and specificity for EVAR patients for prediction of mortality. The corresponding values on the validation cohort were 53.8 and 67.9 for open patients and 58.6 and 66.5 for EVAR patients. Like the AUC, the sensitivity and specificity values are almost as high in the validation cohort as in the derivation cohort essentially ruling out over-fitting and providing validation for the model.
In this study, we used a comprehensive dataset of Medicare patients undergoing intact AAA repair to develop a risk prediction model based on preoperative characteristics that would be applicable to patients undergoing either open or endovascular repair. We found the absolute mortality of open repair to be greater than that of endovascular repair by a factor of 3-fold for all patients with mortality further predicted by age, female sex, and selected comorbidities. There were no relative differences in predictors of perioperative mortality between the two repair approaches. Our scoring system classifies patients into three risk classes (depending on the procedure) that can be easily calculated using available clinical or administrative data.
Previous studies have used administrative data to develop predictors of mortality after AAA repair and others have created risk models for mortality associated with AAA repair using clinical cohorts. These studies have been limited by examining open or endovascular repair alone, examining both procedures without differentiation, or by using data sets that lack prior data that can be used to differentiate pre-existing conditions from complications occurring within the hospitalization.6–8,19–24
Multivariate statistical models from administrative database studies generally find age and female sex to be important predictors, but have shown variable results for other clinical comorbidities dependent upon the database used and inclusion of open repair and/or EVAR.19–24 Hua et al. used the National Surgical Quality Improvement Program-Private Sector database, which uses nurse chart review for data entry and thus has more reliable comorbidity data, and found that open repair, age, angina, poor functional status, recent weight loss, and dialysis were all associated with increased mortality when including both methods of repair.19
As we have done here, other studies have translated risk prediction models into clinical scoring systems to provide estimates of an individual’s predicted mortality after repair. These prediction models, however, were all derived from cohorts restricted to open AAA repair. A few of the more well-known are the Glasgow Aneurysm Score (GAS), the Leiden Score, and the Hardman Index.8–10 It is important to recognize that there may be considerable limitations to these when looking at modern elective AAA repair, because of the significant mortality differences when including EVAR.9–13 For instance, the GAS was developed from a cohort of 268 open AAA repairs of which 41% were ruptured. The overall mortality of the group was 20% (8.6% for intact AAA repairs). Age (included as a continuous variable), is the most significant factor in this scoring model.6,11,12,25 Nesi et al. showed that age alone was at least as good a predictor of mortality as the GAS after elective open AAA repair.25
The Leiden Score was developed in 1995 from a cohort of 238 patients undergoing open AAA repair as well as a meta-analysis of published literature. Overall mortality in the Leiden data were 7.3% and 6.8% depending on the data source, which more closely approximates current results of elective open AAA repair.8 This mortality rate, however, is still substantially higher than we observed in our more recent Medicare cohort and from recent NIS population data.1,22,26 Faizer et al. performed a comparison of both of these scoring methods to evaluate their applicability to mortality prediction for EVAR. The GAS was found to have a c-statistic of 0.47 for EVAR showing poor predictive ability. Although the M-LS had a c-statistic of 0.7, all of the deaths occurred in the highest quartile of the score, so this was not an effective way of distinguishing risk factors for most of the population undergoing repair.11
Similarly, some scoring systems have been applied retrospectively to participants in randomized clinical trials including the UK EVAR trials and the Dutch Randomized Endovascular Aneurysm Management (DREAM) trial. Baas et al. used the GAS scoring model to predict mortality in patients from the DREAM trial. They found that a higher GAS was associated with increased 30-day mortality in open repair but not EVAR.9 Brown et al. used the Customized Probability Index score to assess whether patient fitness determined the relative benefits of open or endovascular repair within the EVAR I trial. The study found that only patients receiving scores corresponding to the fittest group had a 30-day mortality benefit from EVAR versus open repair (OR 0.24, P = .03) and that no group had a 4-year mortality benefit.27 While these show that scoring models may be useful in preoperative risk stratification before AAA repair, the trials and subsequent tests of score models were relatively small compared to our current study (35 deaths in 1205 patients in EVAR-1, 10 deaths in 345 patients in DREAM vs. 810 deaths in 22,830 patients in the current study). We have also specifically included repair type as a preoperative variable and tested for interaction effects to ensure that the other preoperative factors have the same predictive effect across both repair types.
An area under the ROC curve of 0.73 implies that the model has an approximate 73% chance of correctly classifying into the high risk category a randomly selected patient who dies compared to one who survives. Given the relatively low mortality now associated with AAA repair, a model with a high sensitivity is nearly impossible and, given the inverse relationship of sensitivity and specificity, would minimize its ability to identify a low risk population. As the goal of this model is to predict risk categories, we believe this provides a useful tool that could aid in patient discussions and clinical decision making.
We used administrative data to measure comorbidities that are dependent upon accurate coding of diagnoses and procedures. The dataset also does not have pre-operative laboratory results or information on patient symptoms. Additionally, there are no anatomic descriptive data apart from the diagnosis coding for aneurysm location as “abdominal,” thus we cannot control for anatomic features such as aneurysm diameter, neck length, or level of clamp placement. A significant benefit of Medicare data versus other administrative data is that with the addition of Part B (physician claims) and denominator files, we have universal pre-existing patient data that was input by physicians and complete mortality data during the follow-up period. CPT coding is more detailed than ICD-9 coding which allows for greater accuracy and specificity for identifying both procedures and diagnoses. We excluded patients with CPT codes for concomitant visceral-renal bypass or thoracoabdominal aneurysm to exclude suprarenal aneurysm repair. Incorporating prior claims data allowed us to measure comorbidities reliably and made it possible to distinguish comorbidities from complications. It is also important to recognize that this study examined perioperative mortality (30-day and in-hospital). These results, therefore, cannot be used to predict longer term survival differences.
A further limitation of applying scores developed for open repair to endovascular repair and vice versa is that such applications may be inappropriate due to the differing candidate populations.28 To address this issue, we created carefully matched populations of patients undergoing each of the procedures. Based on age and other predictors, some patients are only considered candidates for EVAR, while previously younger and healthier patients have preferentially been offered open repair. This allows our model to be relevant for clinical decision making in patients who are reasonable candidates for either type of repair in order to determine differences in predicted mortality. Given our requirement for two years of preoperative data, the youngest patients in this study are 67 years of age. Increasing age is a major predictor of perioperative mortality compared to patients younger than 70 years. If younger patients were included it is likely that age would be an even stronger predictor but it has to be recognized that this model was developed from the Medicare population and may not be applicable to younger patients.
Endovascular AAA repair has significantly lower perioperative mortality than open AAA repair (by 3-fold) for all ages and comorbid conditions. Older age, female gender, renal disease, CHF, and vascular disease increases the relative mortality risk equally for either repair method. Given a higher baseline mortality of open repair, absolute mortality increases as more risk factors are added, making the absolute benefit of EVAR greater. We detail a preoperative scoring system for mortality risk prediction that may be used in the clinical setting to estimate the risk of mortality for any patient eligible for open and EVAR.
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Presented at the New England Society for Vascular Surgery 35th Annual Meeting, Newport, RI October 4th, 2008
*logit(p)= −5.02+0.42*Female+0.15*Age(70 to 75)+0.63*Age(75 to 80)+1.14*Age(>80)+0.71*CRI+0.95*ESRD+0.55*CHF +0.30*Vascular Disease+1.17*Open Repair
Kristina A. Giles, Beth Israel Deaconess Medical Center.
Marc L. Schermerhorn, Beth Israel Deaconess Medical Center.
A. James O’Malley, Department of Health Care Policy, Harvard Medical School.
Philip Cotterill, Centers for Medicare and Medicaid Services.
Ami Jhaveri, Beth Israel Deaconess Medical Center.
Frank Pomposelli, Beth Israel Deaconess Medical Center.
Bruce E. Landon, Department of Health Care Policy, Harvard Medical School.