Patients and their physicians are faced with difficult choices when making treatment decisions for CLI. While traditional surgical bypass is established and effective, alternative endovascular therapies have emerged as a popular alternative, most likely because they avoid invasive therapies in this frail patient population11
. However, trials comparing these two modalities have proven difficult to design and expensive to perform. Therefore, the SVS has designed OPGs as a mechanism to provide benchmarks, by which the performance of endovascular interventions and other treatment modalities can be measured. As our study demonstrates, even though these benchmarks were derived using data from highly selected clinical trials, they appear to accurately reflect the outcomes achieved in real-world lower extremity bypass.
The generalizability of SVS OPGs, across highly selected clinical trials as well as within everyday vascular surgery practice, suggests that these benchmarks are an accurate performance measurement tool in lower extremity revascularization for patients with CLI. Further, the strong correlation between the outcomes of clinical trials and everyday practice suggests that, in the “real world”, patients have widely available access to high-quality surgical lower extremity revascularization for CLI. Unlike early clinical trials of asymptomatic carotid endarterectomy24
, wherein clinical trial results didn’t always translate into real-world outcomes, it appears that patients can be assured of similar outcomes regardless of the setting in which they undergo surgery.
This report adds to the growing evidence from clinical trials12, 18–21
, academic centers25, 26
, and regional collaboratives8
that demonstrate that lower extremity bypass is effective, reproducible, and broadly available. Therefore, policymakers and payers should ensure that endovascular interventions meet a similar benchmark. Many will argue that patients may accept shorter patency or higher reintervention rates for endovascular interventions, based on their less-invasive approach. This process occurred in the treatment of coronary artery disease27–30
. While it is well-known that patients will often “discount” a desired outcome based on a less morbid approach and shorter recovery31, 32
, the extent of this trade-off has yet to be defined. As less invasive procedures challenge traditional open surgery, it is likely that patients and providers will “vote with their feet” and choose the surgical procedure or endovascular intervention that they believe will provide the most effective treatment. However, the first step in determining treatment efficacy and effectiveness is to define the outcome measures, and our analysis demonstrates that the SVS OPGs represent plausible, achievable targets for patients with CLI.
Many will question why the SVS OPGs, which were primarily derived to determine endpoints for clinical trials, should matter to surgeons interested in quality improvement, such as those surgeons in the VSGNE. However, the essential components of any quality improvement effort, as outlined by Deming, usually involve a Plan/Do/Study/Act (or PDSA) cycle35
. In a PDSA cycle, a process is improved by planning how change will be implemented, implementing the change, and studying the effect of change and acting upon the results of these studies. We believe the SVS OPGs offer the advantage of standardization of outcomes during the “study” phase of these efforts. Many have documented wide variation in the assessment of efficacy in the study of lower extremity revascularization 1, 2, 8, 10
. This problem limits effectiveness and generalizability, not only in clinical trials, but quality improvement efforts as well. By standardizing outcome assessment, OPGs can help to further efforts in the study of lower extremity revascularization, across both clinical trials of new devices as well as in structured quality improvement efforts.
Our study has several limitations. First, there are small differences between the definitions in the SVS OPG dataset and our VSGNE dataset. For example, reinterventions are categorized as catheter-based, surgical, or both in the VSGNE. However, a major/minor categorization scheme is used in the SVS OPG definition. Despite these disparities, the absolute differences in most of the efficacy endpoints between the two datasets were small (less than 5% at one year). Second, data within the VSGNE is self-reported by its participating centers, and is not independently adjudicated, nor was duplex evalution at one year mandated in our dataset. However, for two reasons, we believe our results accurately reflect outcomes with lower extremity bypass in our region. First, participation in our regional quality improvement database is voluntary, and surgeons who freely allocate their time and effort are unlikely to purposefully contribute inaccurate results. Second, in the majority of VSGNE centers, follow-up visits are coded by nursing or research personnel, limiting (to some extent) reporting bias, as compared to self-reporting by the operating surgeon. And third, our dataset is validated for inclusion and outcomes by comparing VSGNE outcomes with hospital billing data and the Social Security Death Index, as outlined in several prior reports8, 23, 33, 34
. Our third limitation centers around our measurement and definition of CLI in our cohort. In the SVS OPG cohort, hemodynamic criteria were used to support the diagnosis of critical limb ischemia in the randomized trials. Extensive information was collected as line-item data to determine each patient’s ankle pressure, toe pressure, or TCPO2, with specific criteria required for inclusion (ankle pressure <50, toe pressure <30, TcPO2 <30). In the VSGNE, hemodynamic critieria was available to confirm the diagnosis of critical limb ischemia both before and after surgery in 47% of our patients with CLI. Within this group, the mean toe pressure was 0.2, and the mean ankle pressure 0.38, lending credence to the diagnosis of CLI, at least in these patients. However, a sizeable portion of patients in our cohort do not have these variables routinely recorded, as our registry does not mandate practice patterns, and some surgeons in our dataset do not routinely measure ABIs at follow-up, especially in clinical settings where the significance is uncertain (such as a palpable graft pulse). However, realizing the value of this data in comparative research, more complete evaluation of hemodynamic data in patients with CLI remains an ongoing focus of our data collection improvement process. And fourth, while VSGNE outcome satisfied the SVS OPGs in our analyses, in some instances, this occurred only by using the 95% confidence interval around the point estimate of the VSGNE outcome. For example, the SVS OPG Freedom from MALE/POD outcome at 1 year was 77%, while this value in the VSGNE was only 74%, with a 95% confidence interval that included 77%. Whether or not the SVS OPGs should be considered as a “hard target”, and be considered unsatisfied if the actual point estimate is not reached, remains a question for future discussion. And fifth, given the observational nature VSGNE dataset, unmeasured confounders may have impacted our outcomes. However, the similarities between the results from the VSGNE dataset and the randomized trials from which the SVS OPGs were derived argue against any significant effect of any unmeasured confounders.
In summary, the development and broad implementation of the SVS OPGs in clinical trials, quality improvement efforts, and overall clinical assessment of lower extremity revascularization represents an achievable opportunity to advance the science of how vascular surgeons measure success or failure. However, looking forward, the current SVS OPGs represent the beginning of this effort, and expansion towards even more in-depth OPGs lies ahead. For example, the current SVS OPGs do not take into account quality of life, nor do they reflect the use of medical adjuncts such as antiplatelet agent or statin use. Given the focus of payers, providers, and policymakers on functional outcomes as well as evidence-based medicine, our future efforts will hope to shape new OPGs that will address the ability of vascular surgeons to design and achieve benchmarks in these critical areas.