|Home | About | Journals | Submit | Contact Us | Français|
To develop standardized metrics for expected outcomes in lower extremity revascularization for critical limb ischemia (CLI), the SVS has developed objective performance goals (OPGs), based on aggregate data from randomized trials of lower extremity bypass. It remains unknown, however, if these targets can be achieved in everyday vascular surgery practice.
We applied SVS OPG criteria to 1,039 patients undergoing 1,039 lower extremity bypass operations (LEB) for CLI with autogenous vein (excluding dialysis patients) within the Vascular Study Group of New England (VSGNE). Each of the individual OPGs was calculated within the VSGNE dataset, along with its surrounding 95% confidence intervals, and compared to published SVS OPGs using chi squared comparisons and survival analysis.
Across most risk strata, patients in the VSGNE and SVS OPG cohorts were similar (clinical high-risk [age>80 years and tissue loss]: 15.3% VSGNE, 16.2% SVS OPG, p=0.58; anatomic high risk [infrapopliteal target artery]: 57.8% VSGNE, 60.2% SVS OPG, p=0.32). However, the proportion of VSGNE patients designated as conduit high-risk (lack of single segment greater saphenous vein) was lower (10.2% VSGNE, 26.9% SVS OPG, p<0.001). The primary safety endpoint, major adverse limb event at 30 days (MALE), was lower in the VSGNE cohort (3.2%%, 95% CI 2.3–4.6%) than the SVS OPG cohort (6.2%, 95% CI 4.2–8.1) (p=0.05). The primary efficacy OPG endpoint, freedom from any major adverse limb event or post-operative death within the first year (MALE+POD), was similar between VSGNE and SVS OPG cohorts (77% (95% CI 74–80%) SVS OPG, 74% (95% CI 71–77%) VSGNE, p=0.58). In the remaining safety and efficacy OPGs, the VSGNE cohort met or exceeded the benchmarks established by the SVS OPG cohort.
Community and academic centers in everyday vascular surgery practice can meet OPGs derived from centers of excellence in LEB. Quality improvement initiatives, as well as clinical trials, should incorporate OPGs in their outcome measures to facilitate communication and comparison of risk-adjusted outcomes in the treatment of critical limb ischemia.
Over 12 million patients in the United States suffer from peripheral arterial disease (PAD)1–4, and recent studies estimate that nearly 1 million have the most severe form of PAD, critical limb ischemia (CLI)5. Surgical revascularization, while an effective treatment for CLI6, is associated with significant morbidity and mortality7–10, thus making less invasive treatment strategies desirable. Accordingly, endovascular interventions have emerged as an alternative to surgical arterial reconstruction1, 11. However, while many have sought to compare these new and competing endovascular interventions to surgical bypass12–14, methodologic challenges exist in designing and executing these studies, and differences in patient characteristics and prohibitive costs have made randomized trials and other high-quality studies difficult to design and perform15.
Therefore, in order to establish objective benchmarks for comparison of safety and efficacy in lower extremity revascularization, the SVS recently developed objective performance goals (OPGs) for the treatment of CLI16, 17 (Table 1). These standards were derived from the surgical results of randomized controlled trials of several different treatments for CLI, and offer a benchmark for non-randomized evaluation of new treatments for lower extremity PAD, using the results from lower extremity bypass as a comparator.
It remains unknown, however, if SVS OPGs accurately reflect everyday vascular surgery practice. The SVS OPGs were derived from carefully controlled studies, using highly selected patients undergoing surgery in centers of excellence, where outcomes were recorded within closely monitored, independently adjudicated clinical trials. Whether the trials and results that the SVS OPGs were derived upon are broadly representative of real-world community and academic vascular surgery practice is unclear.
To determine if SVS OPGs accurately represent outcomes found in the everyday practice of vascular surgery, we studied patients undergoing lower extremity bypass in the Vascular Study Group of New England (VSGNE), a regional quality improvement group. We applied the SVS OPGs to patients in the VSGNE, to discern if these targets could be reliably obtained by surgeons practicing in a variety of settings.
The SVS OPGs were derived in recent work by Conte et al16, 17, and the details of this derivation have been published previously16, 17. In brief, the SVS OPGs were calculated using independently adjudicated, line-item data from the surgical bypass arms of five multicenter randomized trials for CLI12, 18–21. The SVS designated these OPGs as safety and efficacy measures as outlined in Table 1. CLI was defined as ischemic rest pain or tissue loss, as well as compromised hemodynamics (ankle pressure <50, toe pressure <30, TcPO2 <30). To standardize the effect of bypass conduit, only those patients with autogenous vein available for bypass conduit were included. Further, congruent with the definitions listed in the OPG cohort, patients with end-stage renal disease (ESRD) on dialysis were eliminated from the analysis, given the disproportionately poor outcomes achieved in these patients.
Patients undergoing lower extremity bypass in hospitals participating in the Vascular Study Group of New England between 2003 and 2009 (2,899 bypass procedures) (Appendix A) were used to form the cohort for this analysis. Further details on this registry, derived from a broad specturm of 15 community and academic centers across New England, have been published previously22, and others are available at vsgne.org.
To construct a cohort of patients with characteristics similar to those patients used to derive SVS OPGs, we included only patients who underwent open infrainguinal bypass procedures for CLI, and eliminated all patients with claudication (n=609, 21% of the total). Additionally, we eliminated patients who underwent bypass with prosthetic conduit (n=564, 20% of the total), and those patients with end-stage renal disease (n=181, 6% of the total). We limited this analysis to patients with occlusive disease, and eliminated all patients undergoing lower extremity bypass for aneurysmal disease (n=203, 7% of the total). Inflow arteries for bypass procedures could be iliac, femoral, or popliteal. Bypass target arteries could be above-knee or below-knee, popliteal, tibial, or pedal. Patients with concomitant endovascular procedures (such as an iliac stent done at the time of bypass surgery) were included in the analysis as well. Patients in whom long-term follow-up was unavailable were also excluded (n=93, 3% of the total). Even though the laterality of procedures is recorded within the VSGNE, both before surgery and during follow-up, only the first limb operated on per patient was entered in the analysis, to avoid confounding secondary to within-patient dependence. This occurred in 74 patients (3% of the total). Lastly, we eliminated any patient who was enrolled in the PREVENT III trial or any other cohort represented in the SVS OPG dataset. This occurred in 24 patients across 2 centers (<1% of the total), and these patients were therefore eliminated from the analyses. Therefore, our cohort consisted of 1,039 patients who underwent 1,039 LEB procedures. Follow up times varied from 1 month to 35 months, with a median of 12.8 months (standard deviation of 5.2 months).
Our unit of analysis was the patient, given that only one procedure per patient was analyzed. Patients were evaluated for pre-existing medical comorbidities, and these data were prospectively entered into our registry by specifically trained surgeons, nurses or clinical data abstractors. Over 70 clinical and demographic variables were collected on each patient22, 23. Validation analyses to ensure complete submission of data were performed using hospital discharge abstracts. Further, we used the Social Security Death Index to ensure that all deaths had been recorded in our dataset. Demographic data and the incidence of patient-level comorbidities are outlined in Table 2.
Our main outcome measures were the OPGs themselves, recorded at 30 days for the safety OPGs, and 1 year for the efficacy OPGs (Table 1). In univariate analyses, we compared the OPGs from the SVS data to those results derived from procedures performed within the VSGNE, using chi-squared for categorical variables (the safety measures studied at 30 days). We examined the efficacy outcomes at one year, using log-rank comparisons between high and low risk strata within VSGNE analytic groups over a one year period, and comparing survival data and its surrounding 95% confidence intervals between OPG data and VSGNE datasets.
The SVS OPG investigators used regression models to identify key variables that were associated with poor outcomes at one year. They designated patients with several important characteristics to be “high-risk.” These high-risk groups were designated as clinical high-risk [age over 80 and tissue loss], anatomic high risk [infrapopliteal distal target], and conduit high-risk [absence of single-segment greater saphenous vein greater than 3mm in diameter]. For these efficacy outcome measures, comparisons between high and low-risk strata were made using log-rank tests at one-year follow up.
Further, we were then able to use these high risk criteria and their associated beta coefficients from univariate regression models to calculate population-specific OPGs for the VSNGE cohort, based on each individual patient’s high-risk criteria. An example of how the VSGNE population specific OPG for MALE+POD OPG outcome is calculated is illustrated below.
The coefficients used in the four remaining efficacy outcomes are shown in Appendix B. Finally, to account for all of the patient-level covariates in the calculation of each individual OPG, all the covariates were entered into multivariate models derived to predict each efficacy OPG based on each patient’s individual characteristics. The multivariate equations for each OPG are shown in Appendix B. These multivariable models take into consideration the patient-specific variables of anatomic, clinical, and conduit risk in determining each patient’s individual risk of meeting each of the OPGs. By aggregating patients across the cohort, we were then able to generate population-specific OPGs for the patients that comprised the VSGNE cohort.. All analyses were performed using Microsoft Excel (Redmond, WA) and STATA (College Station, TX). The Institutional Review Board at Dartmouth Medical School reviewed and approved our study protocol.
The 1,039 patients who underwent 1,039 lower extremity bypass procedures in the VSGNE cohort were most commonly male (66%), elderly (mean age of 73 years), and Caucasian (99%) (Table 2 and Appendix C). Nearly all patients had a history of either prior or current smoking (79%). Over half (58%) of patients had a history of diabetes, 40% had coronary disease, and nearly a third (30%) had a history of COPD. The majority of patients with CLI presented with tissue loss (65%), as opposed to rest pain (35%). Further, while all patients had available autogenous conduit, 13% used alternative types of autogenous conduit such as composite vein (7%), arm vein, (4%), or lesser saphenous vein (2%). Further details about the characteristics of the cohort used are available in Table 2 and Appendix C.
Overall, while patients in the VSGNE cohort were similar to patients in the SVS OPG cohort, small but significant differences exist between the two groups (Table 2). For example, patients in the VSGNE cohort were less likely to be African American (0.7% VSGNE, 13.8% SVS OPG, p<0.001) and more likely to have hypertension (86.3% VSGNE, 76.4% SVS OPG, p=0.02). VSGNE patients were also more likely to be on statins (54.3% VSGNE, 49.1% SVS OPG, p=0.016) and to have undergone previous vascular interventions (29.0% VSGNE, 16.2% SVS OPG, p<0.001), but fewer had tissue loss (65.5% VSGNE, 73.7% SVS OPG, p=0.02) Overall, the proportion who were designated as clinically high-risk was slightly lower (15.3 VSGNE, 16.2% SVS OPG, p=0.58), as was the proportion designated as anatomically high risk (57.8% VSGNE, 60.2% SVS OPG, p=0.48. However, the proportion designated as conduit high risk differed between datasets (10.2.0% VSGNE, 26.9% SVS OPG, p<0.001). Some of this difference may be attributable to differences in definition of an acceptable saphenous vein. It is important to note that while the SVS cohort required the vein to be greater than 3mm in diameter, the VSGNE cohort did not have size parameters on their definition; rather, the vein was simply categorized as “acceptable.” The more stringent definition used in the SVS OPG cohort, may have therefore contributed to the higher proportion of patients designated as “conduit high-risk” in the SVS OPG cohort.
The three safety OPGs studied at 30 days following surgery were major adverse limb events (MALE) (amputation or major reintervention), major adverse cardiovascular events (MACE) (myocardial infarction, stroke, or death), and major above-ankle amputation. The main safety OPG, major adverse limb event (MALE) at 30 days, occurred in 6.1% (95% CI 4.6–7.9%) of patients in the SVS derivation dataset. In our cohort, we found that the MALE rate was slightly lower (3.2%%, 95% CI 2.3–4.6%, p=0.05). The second safety OPG, major amputation rate before thirty days, was similar in both the SVS and VSGNE datasets (SVS = 1.9%, (95% CI 1.1–3.1), VSGNE = 1.9 (95% CI 1.2–2.9, p=0.71). Finally, the third safety OPG, major adverse cardiac event rate, was also similar between the two groups, as shown in Table 3 (SVS = 6.2%, (95% CI 4.7–8.1), VSGNE = 4.2 (95% CI 3.3–5.4, p=0.35).
We compared the efficacy outcomes established by the SVS OPGs with those calculated in the VSGNE. We found that the VSGNE dataset met all the efficacy OPGs suggested by the SVS (Table 3). For example, for the main efficacy OPG, freedom from MALE or post operative death (MALE+POD), results were very similar between the SVS OPG dataset and the VSGNE (77% (95% CI 74–80%) SVS OPG, 74% (95% CI 71–77%) VSGNE, p=0.58). This trend was similar for the remaining OPGs, including limb salvage, survival, and amputation-free survival rates (Table 3). Freedom from reintervention or amputation within the first year continued to reflect the slightly lower rate of re-interventions in the VSGNE dataset within the first year.
Next, we studied outcomes within the three high-risk subgroups (anatomic, clinical, and conduit) defined by the SVS OPG working group (Table 4). Overall, we found that in the VSGNE, 15% of patients were defined as clinical high-risk, 58% of patients were anatomic high risk, and 10% used bypass conduit designated as high-risk.
First, we studied if the variables that were important in delineating high and low-risk populations in the SVS OPG cohort accomplished the same task in the VSGNE cohort. We found that these risk subgroups discriminated performance well (Figure 1). For example, across all three subgroups, using the MALE+POD efficacy OPG, patients in the lower risk category had significantly better outcomes at one year (log-rank p = 0.01). Findings were similar across all of the remaining efficacy OPGs (Appendix D).
Second, we questioned if outcomes were similar across high-risk subpopulations. For example, the population-specific OPG for MALE+POD in the anatomic high-risk group is 74.0%. While the MALE+POD result in the VSGNE cohort was slightly lower at 69.3%, the 95% confidence intervals around the VSGNE dataset (65.1–74.0) included the adjusted OPG. Therefore, the VSGNE dataset met the OPG for this measure. Across all of the high-risk subgroups, the VSGNE cohort met or exceeded the OPG, indicating that even in high-risk patient subgroups, OPGs represent an attainable benchmark in real-world practice.
But what if a patient is high-risk across more than one category? This same calculation can be undertaken, adjusting for all of the risk categories simultaneously, using the multivariate models described in Appendix B. These multivariate models calculate population-specific OPGs, based on each of the four inputs used in the SVS OPG multivariate model (infrapopliteal anatomy, age over 80, tissue loss, and conduit). We found these population-specific “adjusted” OPGs were, on average, similar to the SVS OPGs– reflecting the relative similarities in the VSGNE and SVS patient populations.
As shown in Figure 2, the outcomes achieved in the VSGNE cohort generally met or exceeded the adjusted OPG for all of the efficacy endpoints. Any instances wherein differences occurred between the VSG outcomes and OPGs benchmarks (such as survival) were small and not clinically significant, with two notable exceptions: RAS and RAO. These differences in the RAS and RAO endpoints, where the VSGNE cohort had significantly fewer adverse events of reintervention, amputation, or graft stenosis, likely represent an underestimate of the occurrence of these endpoints in the VSGNE dataset, because not all patients in the VSGNE dataset underwent routine surveillance duplex at 1 year follow up.
Patients and their physicians are faced with difficult choices when making treatment decisions for CLI. While traditional surgical bypass is established and effective, alternative endovascular therapies have emerged as a popular alternative, most likely because they avoid invasive therapies in this frail patient population11. However, trials comparing these two modalities have proven difficult to design and expensive to perform. Therefore, the SVS has designed OPGs as a mechanism to provide benchmarks, by which the performance of endovascular interventions and other treatment modalities can be measured. As our study demonstrates, even though these benchmarks were derived using data from highly selected clinical trials, they appear to accurately reflect the outcomes achieved in real-world lower extremity bypass.
The generalizability of SVS OPGs, across highly selected clinical trials as well as within everyday vascular surgery practice, suggests that these benchmarks are an accurate performance measurement tool in lower extremity revascularization for patients with CLI. Further, the strong correlation between the outcomes of clinical trials and everyday practice suggests that, in the “real world”, patients have widely available access to high-quality surgical lower extremity revascularization for CLI. Unlike early clinical trials of asymptomatic carotid endarterectomy24, wherein clinical trial results didn’t always translate into real-world outcomes, it appears that patients can be assured of similar outcomes regardless of the setting in which they undergo surgery.
This report adds to the growing evidence from clinical trials12, 18–21, academic centers25, 26, and regional collaboratives8 that demonstrate that lower extremity bypass is effective, reproducible, and broadly available. Therefore, policymakers and payers should ensure that endovascular interventions meet a similar benchmark. Many will argue that patients may accept shorter patency or higher reintervention rates for endovascular interventions, based on their less-invasive approach. This process occurred in the treatment of coronary artery disease27–30. While it is well-known that patients will often “discount” a desired outcome based on a less morbid approach and shorter recovery31, 32, the extent of this trade-off has yet to be defined. As less invasive procedures challenge traditional open surgery, it is likely that patients and providers will “vote with their feet” and choose the surgical procedure or endovascular intervention that they believe will provide the most effective treatment. However, the first step in determining treatment efficacy and effectiveness is to define the outcome measures, and our analysis demonstrates that the SVS OPGs represent plausible, achievable targets for patients with CLI.
Many will question why the SVS OPGs, which were primarily derived to determine endpoints for clinical trials, should matter to surgeons interested in quality improvement, such as those surgeons in the VSGNE. However, the essential components of any quality improvement effort, as outlined by Deming, usually involve a Plan/Do/Study/Act (or PDSA) cycle35. In a PDSA cycle, a process is improved by planning how change will be implemented, implementing the change, and studying the effect of change and acting upon the results of these studies. We believe the SVS OPGs offer the advantage of standardization of outcomes during the “study” phase of these efforts. Many have documented wide variation in the assessment of efficacy in the study of lower extremity revascularization 1, 2, 8, 10. This problem limits effectiveness and generalizability, not only in clinical trials, but quality improvement efforts as well. By standardizing outcome assessment, OPGs can help to further efforts in the study of lower extremity revascularization, across both clinical trials of new devices as well as in structured quality improvement efforts.
Our study has several limitations. First, there are small differences between the definitions in the SVS OPG dataset and our VSGNE dataset. For example, reinterventions are categorized as catheter-based, surgical, or both in the VSGNE. However, a major/minor categorization scheme is used in the SVS OPG definition. Despite these disparities, the absolute differences in most of the efficacy endpoints between the two datasets were small (less than 5% at one year). Second, data within the VSGNE is self-reported by its participating centers, and is not independently adjudicated, nor was duplex evalution at one year mandated in our dataset. However, for two reasons, we believe our results accurately reflect outcomes with lower extremity bypass in our region. First, participation in our regional quality improvement database is voluntary, and surgeons who freely allocate their time and effort are unlikely to purposefully contribute inaccurate results. Second, in the majority of VSGNE centers, follow-up visits are coded by nursing or research personnel, limiting (to some extent) reporting bias, as compared to self-reporting by the operating surgeon. And third, our dataset is validated for inclusion and outcomes by comparing VSGNE outcomes with hospital billing data and the Social Security Death Index, as outlined in several prior reports8, 23, 33, 34. Our third limitation centers around our measurement and definition of CLI in our cohort. In the SVS OPG cohort, hemodynamic criteria were used to support the diagnosis of critical limb ischemia in the randomized trials. Extensive information was collected as line-item data to determine each patient’s ankle pressure, toe pressure, or TCPO2, with specific criteria required for inclusion (ankle pressure <50, toe pressure <30, TcPO2 <30). In the VSGNE, hemodynamic critieria was available to confirm the diagnosis of critical limb ischemia both before and after surgery in 47% of our patients with CLI. Within this group, the mean toe pressure was 0.2, and the mean ankle pressure 0.38, lending credence to the diagnosis of CLI, at least in these patients. However, a sizeable portion of patients in our cohort do not have these variables routinely recorded, as our registry does not mandate practice patterns, and some surgeons in our dataset do not routinely measure ABIs at follow-up, especially in clinical settings where the significance is uncertain (such as a palpable graft pulse). However, realizing the value of this data in comparative research, more complete evaluation of hemodynamic data in patients with CLI remains an ongoing focus of our data collection improvement process. And fourth, while VSGNE outcome satisfied the SVS OPGs in our analyses, in some instances, this occurred only by using the 95% confidence interval around the point estimate of the VSGNE outcome. For example, the SVS OPG Freedom from MALE/POD outcome at 1 year was 77%, while this value in the VSGNE was only 74%, with a 95% confidence interval that included 77%. Whether or not the SVS OPGs should be considered as a “hard target”, and be considered unsatisfied if the actual point estimate is not reached, remains a question for future discussion. And fifth, given the observational nature VSGNE dataset, unmeasured confounders may have impacted our outcomes. However, the similarities between the results from the VSGNE dataset and the randomized trials from which the SVS OPGs were derived argue against any significant effect of any unmeasured confounders.
In summary, the development and broad implementation of the SVS OPGs in clinical trials, quality improvement efforts, and overall clinical assessment of lower extremity revascularization represents an achievable opportunity to advance the science of how vascular surgeons measure success or failure. However, looking forward, the current SVS OPGs represent the beginning of this effort, and expansion towards even more in-depth OPGs lies ahead. For example, the current SVS OPGs do not take into account quality of life, nor do they reflect the use of medical adjuncts such as antiplatelet agent or statin use. Given the focus of payers, providers, and policymakers on functional outcomes as well as evidence-based medicine, our future efforts will hope to shape new OPGs that will address the ability of vascular surgeons to design and achieve benchmarks in these critical areas.
The OPGs suggested by the SVS accurately represent broadly achievable targets in lower extremity bypass surgery, irrespective of whether the procedure is performed within a clinical trial, an academic center, or in a community setting. Further, in our region, lower extremity bypass surgery performed in the context of a regional quality improvement registry demonstrates outcomes that are consistent with those found in multicenter randomized trials. Vascular surgeons interested in the measuring the quality of treatment of patients with lower extremity peripheral arterial disease should use the SVS OPGs to study, report, and improve their outcomes.
Appendix A. Hospitals participating in the Vascular Study Group of New England.
Appendix B: Beta coefficients utilized to calculate population-specific OPGs in the SVS CLI dataset.
Appendix C: Additional patient characteristics from the VSGNE dataset.
Appendix D: Efficacy OPGs at one year, in the high and low-risk subgroups
Presented at the Society for Vascular Surgery’s Vascular Annual Meeting,, June 10th, 2010, Boston, MA
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.