|Home | About | Journals | Submit | Contact Us | Français|
National guidelines on carotid endarterectomy(CEA) for asymptomatic patients state that the procedure should be performed with a ≤3% risk of perioperative death or stroke. We developed and validated a multivariate model of risk of death or stroke within 30 days of CEA for asymptomatic disease and a related clinical prediction rule.
We analyzed asymptomatic cases in a population-based cohort of CEA performed in Medicare beneficiaries in NY. Medical records were abstracted for: sociodemographics, neurological history, disease severity, data, comorbidities, and deaths and strokes within 30 days of surgery. We use multivariate logistic regression to identify independent predictors of perioperative death/stroke. The CEA-8 Clinical Risk Score was derived from the final model.
Among the 6553 patients, mean age was 74 years, 55% were male, 62% had coronary artery disease(CAD), and 22% history of distant stroke or TIA. The perioperative rate of death/stroke rate was 3.0%. Multivariable predictors of perioperative events were: female(OR, 1.5; 95% CI, 1.1–1.9), non-white(OR, 1.8; 1.1–2.9), severe disability(OR, 3.7; 1.8–7.7), CHF (OR, 1.6; 1.1–2.4), CAD(OR, 1.6; 1.2–2.2), valvular heart disease(OR, 1.5; 1.1–2.3), distant history of stroke/TIA(OR, 1.5; 1.1–2.0), and non-operated stenosis ≥50%(OR, 1.8; 1.3–2.3). The CEA-8 Risk Score stratified patients with predicted probability of death/stroke rate from 0.6% to 9.6%.
Several sociodemographic, neurological severity, and comorbidity factors predicted risk of perioperative death/stroke in asymptomatic patients. The CEA-8 risk score can help clinicians calculate a predicted probability of complications for an individual patient to help inform the decision about revascularization.
Three-quarters of the 117,000 carotid endarterectomies (CEA) done annually in the US are for asymptomatic disease. The overall magnitude of benefit of CEA in asymptomatic patients is modest. In randomized controlled trials (RCT), CEA reduced the absolute risk of stroke or perioperative death among asymptomatic patients approximately 6% over 5 years compared to medical therapy alone.[2, 3] Because there was a 3% risk of death or stroke due to the procedure in the RCTs, national guidelines emphasize that the overall benefit of surgery for asymptomatic disease is critically dependent on the perioperative risk.[4, 5] Patients with ≤3% complication rate would benefit more, those with >3% risk would benefit less, and for those at high operative risk, the risks might outweigh the benefits. Having an evidence-based way to calculate the personalized risk of CEA for an asymptomatic patient would help physicians counsel patients about the potential benefits and harms of carotid revascularization.
Unfortunately, no validated model for predicting perioperative complications of CEA in asymptomatic disease exists. Most previous studies of perioperative risk after CEA have several limitations. Most studies lack multivariate analyses,[6–9] or focused on all-comers (symptomatic and asymptomatic patients),[6, 10–13] single institutions,[6, 10–13] in-hospital complications,[7, 8, 14–16] or highly selected surgeons and patients in the RCTs.[17–21]
The few prior studies that proposed a clinical risk prediction rule for CEA were either developed in all-comers(symptomatic and asymptomatic).[22, 23] or symptomatic patients alone.[19, 21] Extrapolation about risk factors for asymptomatic patients from studies of symptomatic patients or all-comers is problematic for several reasons. Symptomatic patients have greater severity of carotid disease, more comorbidities, and double the rates of perioperative death and stroke (6% vs. 3%) compared with asymptomatic ones.[5, 9] The proportion of asymptomatic patients in all-comer studies have ranged from about one-quarter to three-quarters.
This study sought to: 1) Develop and validate a multivariable model predicting the risk of death and stroke within 30 days of CEA for asymptomatic patients; and 2) Create a practical, clinical prediction rule that could be used by physicians and patients to weigh the risks and benefits of surgery.
The New York Carotid Artery Surgery (NYCAS) Study included all Medicare beneficiaries who had a CEA (ICD-9 code 38.12) in any hospital in New York State between January 1, 1998 and June 30, 1999. Details about NYCAS have been published previously.[1, 24] Cases were identified using Medicare and NY state hospital discharge data. The study was approved by the University of Texas Southwestern Medical Center Institutional Review Board. Of 11,406 potentially eligible cases, medical records for 10,817 cases (94.8%) were reviewed. Among these, we excluded: 110 cases with no CEA performed, 308 with same side operations for restenosis, 490 with CEA combined with other procedures, and 601 lacking clinical risk factor data. Out of 9,308 complete cases, 6553 were performed on clinically defined asymptomatic patients.
Hospital charts were abstracted by trained nurse abstractors for: sociodemographics, neurological examination, functional status, laboratory values, medications, diagnostic imaging results, neurological, medical, and surgical history. We calculated Revised Cardiac Risk Index and Charlson comorbidity scores. Severe disability was defined as a modified Rankin score of 4–5. Asymptomatic patients were defined as those with no history of carotid stroke or transient ischemic attack(TIA) ≤12 months before surgery. History of distant cerebrovascular disease was defined as stroke or TIA >12 months before surgery.
Carotid imaging data included the percent stenosis of the operated and non-operated artery and presence of plaque ulcers. Angiography was considered to be the most accurate imaging test, followed by Doppler ultrasonography, and then MR angiography. If no imaging results were available, stenosis information was abstracted from preoperative notes. Abstractors passed quality assurance tests and inter-rater reliability was high (Kappas 0.60–1.0).
We identified potential perioperative deaths, strokes and TIAs by abstracting the medical record of the index admission, as well as the charts for readmissions to any hospital in the state within 30 days of surgery using Medicare and NY state hospital discharge databases. Research nurses reviewed the admission and progress notes, discharge summaries, and brain imaging reports. Cases identified by the abstractors as having a death, stroke, or TIA were independently reviewed and confirmed by two physicians (including a neurologist). Initial agreement was 95%, and disagreements resolved by consensus.
The primary outcome of interest was death or non-fatal stroke within 30 days of surgery—the composite outcome used in the RCTs and national guidelines. We also developed a risk model for the secondary outcome of all strokes (fatal and non-fatal) within 30 days of surgery. We first assessed univariate associations between outcomes and risk factors. Categorical variables were examined with χ2 tests and Cochrane-Mantel-Haenszel tests for trend and continuous data with t tests and Wilcoxon rank sum tests, as indicated. Age was examined as a continuous variable, intervals and thresholds (75 and 80 years old). Non-white refers to Blacks and Hispanics. The other/unknown race group was very small and had outcomes similar to whites and were combined with them. Univariate clinical predictors significant at p <0.2 level were entered into a hierarchical multivariable logistic model. We used generalized estimating equations to account for the effects of clustering of patients by surgeon and surgeon by hospital.
We used cross-validation to examine the predictive power of the multivariable model by performing 100 iterations of a random 75%:25% split of the data into derivation and validation samples. We further validated the risk model by plotting the observed rates of adverse events by quintile of predicted probability of events for the derivation and validation groups. Model discrimination was assessed with the c-statistic and calibration via the Hosmer Lemeshow test.
Based on the final multivariable model of death or stroke, we developed a clinical risk score assigning in accordance with published principles. Seven risk factors with similar standardized beta-coefficients were assigned 1 point (female, non-white, congestive heart failure, coronary artery disease, valvular heart disease, cerebrovascular disease, and non-operated stenosis ≥50%), and severe disability 2 points (because it had twice as a large a beta-coefficient). The total risk score ranged from 0 to 9 points. Given the small percentage of patients with ≥5 points (N=159, 0.2% of the total), these cases were grouped as the highest risk group. Analyses were performed using SAS 9.2 (Gary, NC) and R 2.9.0 (Vienna, Austria).
NYCAS included 6553 CEAs performed on asymptomatic patients. Table 1 shows the patient characteristics. The mean age was 74.5±6.6 years (range 42–98) and 54.9% were male. Nearly three-quarters of patients had hypertension, 62.4% had coronary artery disease, 29.2% diabetes, 13.4% valvular heart disease, and 9.2% congestive heart failure. With regards to neurological history, 21.8% had a history of distant stroke or TIA, and 1.4% had severe neurological disability. Nearly all patients (96.4%) had high grade stenosis (70–99%). There was a wide range of stenosis on the non-operated artery, with 37.1% having ≥50% contralateral stenosis, and 5.6% having total occlusion. The CEAs were performed by 435 surgeons in 157 hospitals.
Within 30 days of surgery, there were 55 deaths (0.84%) and 165 (2.52%) strokes. The overall rate of combined perioperative death or non-fatal stroke was 3.05% (N=200). Table 1 shows the univariate predictors of adverse outcomes. Patients that were ≥80 years had higher rates of death or stroke (3.81% vs. 2.76%; p<.03). Using a cut-point of ≥75 years did not differentiate groups with significantly different outcomes. Women and Non-Whites had significantly higher rates of death or stroke. Cardiovascular comorbidities like coronary artery disease, valvular heart disease, atrial fibrillation, and congestive heart failure also increased the risk of complications, as did diabetes. The higher the overall comorbid illness burden, the higher the chance of death or stroke. Those with distant stroke or TIA also had worse outcomes (4.06% vs. 2.71%). Severe disability greatly increased the risk of adverse events (10.75% vs. 2.90%). The degree of stenosis of the operated artery was not significantly associated with outcomes. However, those with ≥50% stenosis of the non-operated artery or ulcerated plaques had higher risks of death or stroke. Univariate predictors of stroke alone were similar to those for the combined outcome of death or stroke with the exception of older age.
Eight variables were independent predictors of death or stroke (Table 2). Two were sociodemographics factors (female and Non-white race), three were neurological (history of distant stroke or TIA, non-operated stenosis ≥50%, and severe disability), and three were cardiac comorbidities (coronary artery disease, congestive heart failure, and valvular heart disease). Neither age ≥75 nor ≥80 years were significant multivariate predictors; there was no interaction between older age and gender. Among comorbidities, diabetes was eliminated by the other cardiac conditions. Figure 1-A shows that the final multivariable model of death and stroke performed similarly in both the derivation and validation analyses (C statistic=0.64 and 0.62).
The independent risk factors for any stroke were nearly identical to those for combined death and stroke with the exception of congestive heart failure, which did not meet statistical significance for the any stroke model. Comparison of the final stroke model in the derivation and validation sample is shown in Figure 1-B (C statistic = 0.65 and 0.64). Both multivariable models were well-calibrated according to the Hosmer-Lemeshow test.
We produced two clinical prediction rule versions of the final multivariable model. One version uses all eight variables in the final model with all worth one point except for disability worth two points (Table 3). Figure 2-A shows that the CEA-8 Clinical Risk Score nicely stratified patients from an observed death/stroke rate of 0.6% to 9.6%. Based on national guideline benchmarks (≤3% death/stroke rate), patients with a score of 0–2 would be classified low risk, those with 3 points moderate risk (4.7%), and ≥4 points high risk (7.5%). Based on this CEA-8 Clinical Risk Score and observed outcomes, 70.7% of patients in NYCAS were low risk, 19.7% moderate risk, and 9.6% high risk cases. Based on predicted probability of complications, one-quarter (26.5%) of patients had a higher predicted probability of death and stroke rate than the national guideline recommend (3% threshold).
Figure 2-B shows a “Patient Friendly” CEA-7 version of the risk score that a lay person could use themselves to estimate their risk using the simple sociodemographic (gender, race) and clinical information they should know. The CEA-7 version omits “non-operative stenosis ≥50%” based on the assumption that many patients might not know this detail. The seven factor “Patient Friendly” Risk Score also nicely stratified the risk of adverse outcomes (1.4% to 8.3%).
In this population-based study of 6553 asymptomatic patients undergoing CEA from 435 surgeons in 157 hospitals, we examined the potential impact of over 26 patient and clinical characteristics on risk of stroke or death within 30 days of CEA. The final multivariable model identified several sociodemographic, neurological severity and comorbidity factors independently associated with higher risk of perioperative complications.
Among sociodemographic factors, our finding that women had higher risk-adjusted odds of complications is consistent with prior studies.[30, 31], The higher risk in women may help explain why subgroup analyses of the RCTs and the meta-analyses found that asymptomatic women did not appear to benefit from CEA. If women have higher upfront rates of death or stroke due to the procedure, it will be harder for them to accrue the long term benefits. The higher perioperative risk among non-white patients has been reported in most studies of all-comers having CEA, though this has been attributed to higher rates of symptomatic disease among minorities[14, 33–35] Finding such a disparity among asymptomatic persons implies the differences are likely a function of a combination of patient, surgeon and hospital factors. Among the neurological severity variables, two markers of underlying prior cerebrovascular disease (severe disability and distant stroke or TIA) greatly increased the odds of adverse events as did ≥50% contralateral stenosis. These neurological severity variables have been found to increase risk in all-comer CEA populations.[9, 15, 24] In the asymptomatic ACAS trial, both distant stroke and contralateral stenosis ≥60% were univariate predictors of 30 day death or stroke, though these analyses were not risk-adjusted.20 That the severity of ipsilateral stenosis was not related to short term complications is consistent with the RCTs and meta-analyses which also found that long term benefits from CEA in asymptomatic patients were similar in moderate and severe stenosis. The presence of major cardiovascular problems (congestive heart failure, coronary artery disease, and valvular heart disease) as risk factors is consistent with the literature.[8, 23],,[36, 37] We have previously identified predictors of perioperative complications in all-comers in NYCAS (asymptomatic and symptomatic patients), this study focused on stratifying risk among the 6653 asymptomatic patients in the cohort. Both the all-comer and asymptomatic risk models included: Non-White, distant stroke or TIA, non-operated stensosis ≥50%, disability, and coronary artery disease as significant multivariables factors.24
The clinical prection rule version of the final risk models represents a new and practical contribution to the literature as it is the first risk score focused on asymptomatic patients. Prior risk scores in CEA were either for all-comers (symptomatic and asymptomatic), or the select patients and surgeons in the symptomatic ECST trial. While these other prediction rules were developed for different patient populations, our CEA-8 risk score shares in common with some of these other indices: female gender, prior cerebroavascular disease, contralateral stenosis, coronary artery disease, and congesetive heart failure (See Supplemental Table 5 for details).
The CEA-8 risk score can help referring physicians, surgeons, neurologists, and anesthesiologists calculate the short term risk of CEA in an individual patient. Because the absolute benefits of carotid revascularization among patients with with asymptomatic disease is very modest (reduction of about 1% lower risk of stroke per year), understanding the probability of upfront harm due to the procedure is critical to weighing its long term net benefit for a given patient.
The RCTs and national guidelines stress the importance of having CEA performed by a surgeon and hospital with ≤3% risk of death or stroke within 30 days. The good news from NYCAS was that the average 30 day death and stroke rate among asymptomatic patients having surgery in community practice was at the 3.0% benchmark. The cautionary news was that over one in four Medicare patients had a predicted probability of perioperative events >3%. We envision clinicians using the CEA-8 risk score in the following fashion. Patients with 0–2 points would be classified as low risk, safely below the 3% guideline threshold. Those with 3 points, with a predicted 4.7% death and stroke rate, would be moderate risk (slightly above 3% threshold). Those with 4 points (with a ≥7.5% risk of serious complications), are high risk and CEA would be unlikely to be beneficial for most patients with this profile. The seven factor “Patient-Friendly” CEA-7 risk score also clearly identifies low, moderate and high risk groups, and could be used by patients to help them make a shared, informed decision about surgery.
The study strengths should be viewed in the context of a few limitations. While NYCAS collected detailed data on potential risk factors and outcomes from medical records, it was based on documentation in real world practice. There was no prospective data collection or outcome examination as done in the RCTs. However, clinical events were reveiwed by two study investigators, and the richness of our data is much greater than most prior studies of CEA risk factors. NYCAS reflects practice in one state, albeit a large one, during 1998 and 1999. Table 4 shows the perioperative outcomes and patient characteristics in NYCAS compared to the landmark a RCTs and two very recent large, observational cohort studies of CEA for asymptomatic disease. This shows that the outcome rates in NYCAS were comparable to those in RCTs[2, 3] and similar to those reported among Medicare beneficiaries during 2004–2005. It is worth noting, that a recent study by Woo et al of asymptomatic patients of all ages having CEA in the National Surgical Quality Improvement Program (NQSIP) registry reported lower absolute rates of perioperative death and stroke. These differences are likely due to to the fact that NYCAS Medicare patients were older and sicker than those in NSQIP and voluntary registries tend over-represent high performing institutions. The Woo et al study also excluded patients with a distant history of stroke and TIA, which represented 21.8% of asymptomatic patients in NYCAS—a risk factor that increased the risk-adjusted odds of death or stroke by 48%. However, the applicability of our results to non-Medicare populations merits further investigation, and future work should include validation of the prediction rule in an independent, more contemporary sample.
It is also worth acknowledging that a patient’s risk of surgical complications is a function of both their individual characteristics, as well as the skill and experience of surgeon and hospital team caring for them, so both elements of risk should be factored in an evidence-based decision about revascularization. Ultimately, the decision about revascularization needs to weigh both the short term risks of surgery against its long term benefits. More research is needed with regards to developing evidence-based estimates of the long term benefit of CEA for an individual asymptomatic patient.
In summary, though the rate of perioperative death and stroke after CEA in this population-based cohort were similar to those deemed optimal in the RCTs and national guidelines, the range of risk faced by asymptomatic patients undergoing surgery in community practice range from <1% to almost 10%. The new CEA-8 risk score we developed can help physicians counsel patients about the potential risks and benefits of CEA and help rationalize selection of appropriate candidates. Further prediction rules that could help asymptomatic patients individualize the long term benefits of CEA would be additionally valuable in improving overall decision making.
The authors acknowledge the assistance of the Island Peer Review Organization (IPRO) and the Centers for Medicare and Medicaid Services (CMS) in providing the data which made this research possible. The conclusions presented are solely those of the authors and do not represent those of IPRO or CMS. All authors made substantial contributions to the manuscript.
This study was supported by the Agency for Healthcare Research and Quality (RO1 HS09754-01), Center for Medicare Services, Robert Wood Johnson Foundation (#020803) and National Institute of Neurological Diseases and Stroke (R01 NSO56028-02). Dr. Calvillo-King was supported by Diversity Supplement from the National Institute of Neurological Diseases and Stroke.
Conflicts of Interest Disclosure
There are no conflicts of interest or disclosures to report.
Linda Calvillo-King, Division of General Internal Medicine, University of Texas Southwestern Medical Center, 5323 Harry Hines Blvd, Dallas, TX 75390-8889.
Lei Xuan, Department of Clinical Sciences, University of Texas Southwestern Medical Center, 5323 Harry Hines Blvd, Dallas, TX 75390-8889.
Song Zhang, Department of Clinical Sciences, University of Texas Southwestern Medical Center, 5323 Harry Hines Blvd, Dallas, TX 75390-8889.
Stanley Tuhrim, Department of Neurology, Mount Sinai School of Medicine, NY, One Gustave L. Levy Place, New York, NY 10029.
Ethan A. Halm, Division of General Internal Medicine, and Department of Clinical Sciences, University of Texas Southwestern Medical Center, 5323 Harry Hines Blvd, Dallas, TX 75390-8889, Tel: (214) 648-2841, Fax: (214) 648-8727.