|Home | About | Journals | Submit | Contact Us | Français|
Use of the World Health Organization’s Surgical Safety Checklist has been associated with a significant reduction in major post-operative complications after inpatient surgery. We hypothesized that implementation in the United States would be cost saving at the hospital level. We performed a decision analysis comparing implementation of the checklist to existing practice in American hospitals. In a hospital with a baseline major complication rate after surgery of at least 3%, the checklist is cost saving once it prevents at least 5 major complications. Implementation and use of the checklist is a cost saving quality improvement strategy.
The World Health Organization (WHO) launched the Safe Surgery Saves Lives campaign to improve consistency of surgical care and adherence to beneficial safety practices. To help achieve this goal, the Surgical Safety Checklist was created through an international consultative process. The checklist is a two-minute tool, much like a pilot’s checklist, and is designed to help operating room staff improve teamwork and ensure the consistent use of safety processes.1 It consists of a series of checks that occur before the induction of anesthesia, before skin incision, and before the patient leaves the operating room (Exhibit 1). These checks include such items as confirming appropriate antibiotics have been given to prevent infection, necessary equipment is available, and that no members of the team have any unaddressed questions or concerns before proceeding with the operation.
In a pilot study of systematic implementation of the checklist, complications were markedly decreased for patients undergoing non-cardiac surgery in eight diverse international hospitals.2 Four out of the eight study sites were in high-income countries with well-developed health care infrastructures. Among these sites, there was a 30% relative reduction in major complications after introduction of the checklist.
With evidence that systematic use of checklists can result in decreased rates of surgical complications2 as well as catheter-related bloodstream infections,3 the use of this type of intervention is gaining acceptance.4 However, one line of criticism of checklists is that they may be cost-ineffective because of time, effort and the variable levels of risk involved.5 In this paper, we therefore examined the costs of implementation and use of the WHO Surgical Safety Checklist in the United States to determine whether it is cost saving at the hospital level.
We performed a decision analysis of implementation and use of the checklist in an American hospital over a one-year time horizon. The analysis was performed from a hospital’s perspective with respect to costs. Costs were adjusted for inflation to 2008 dollars based on the Consumer Price Index and the medical care price index.6 A discount rate was not applied given the one-year time horizon. Costs associated with the checklist were divided into one-time start-up costs from implementation and recurrent costs from use of the checklist.
Estimates of implementation costs were based on pilot site experience in the Safe Surgery Saves Lives Study,2 experience in our own institution, and communication with hospitals adopting the checklist in the United States.7
Implementation of the checklist generally requires collaboration between departments of surgery, anesthesia and operating room nursing. Representatives from each department work together to introduce the checklist to their staff, modify it to meet the conditions in their hospital, and make appropriate systems changes to ensure successful use of the checklist.8 Clinician champions from each department as well as an implementation coordinator generally oversee this implementation process. The implementation coordinator is usually a bachelor’s or master’s level quality improvement officer who serves as a facilitator for a hospital’s adoption of the checklist.
At both our institution and the United States’ pilot study site, senior clinicians in leadership roles within their own departments were involved in checklist implementation. These clinicians championed use of the checklist within their departments and they worked with other departments to provide multidisciplinary leadership during implementation. The time commitment of individual champions varied between institutions. For our analysis we selected the highest estimate and applied it to all three champions. The time commitment of the implementation coordinator was similar at each institution.
We estimated the cost of implementation as the opportunity cost of the work that would have otherwise been performed by the three department champions and the implementation coordinator. We calculated this cost by summing the time spent on implementation multiplied by the mean hourly wage for each champion and the implementation coordinator.9 Based on the experience at our institution and at the pilot study site in the United States, we estimated the time spent on checklist implementation at 40 hours for each champion and 120 hours for the implementation coordinator. Using this estimate, we arrived at an implementation cost of $12,635 for our base-case analysis.
Thus far, the checklist has been commonly introduced to individual clinicians during a portion of a grand rounds or a regularly scheduled staff meeting. Because clinicians do not usually see patients or operate during this time, they do not have to trade-off between learning about the checklist and generating revenue. In this case, the opportunity cost of time spent discussing the checklist is foregone educational or meeting time which we considered negligible and excluded from our analysis.
Although there has been concern about the time required to perform the checklist, early adopting institutions have not experienced decreased throughput or disruptions in workflow, including our own institution.10 Therefore, the cost of time spent performing the checklist in the operating room was assumed to be zero in our base-case analysis. However, as reflected in the per-use cost, we varied the cost of time spent running through the checklist in our sensitivity analysis.
Most of the checklist items have little to no direct cost, as they tend to consist of verbal confirmations that a routine safety measure was performed. As such, consistent performance of these checklist items would be expected to result in minimal added cost. An exception is antibiotic prophylaxis, which requires the use of a consumable good as opposed to the performance of a verbal check.
The per-use cost of the checklist was calculated by estimating the increase in the appropriate use of antibiotic prophylaxis observed with completion of the checklist. In the pilot study of eight hospitals, antibiotic prophylaxis increased by 26.5% after implementation of the checklist.2 We applied this rate of increased antibiotic use to the cost of using cefazolin for prophylaxis or vancomycin in the event of a penicillin or cephalosporin allergy.11 Based on these assumptions, the per-use cost of the checklist was estimated at $11 for our base-case analysis. We excluded costs associated with site-marking and pulse oximetry. Although the checklist is intended to ensure site-marking, it is not clear whether the practice increases with use of the checklist as adherence to this safety measure was not studied in the Safe Surgery Saves Lives Study. The cost of pulse oximetry was excluded from the analysis because its use is nearly universal in the United States. The range for per-use checklist cost in the sensitivity analysis was $5.50-$22.12
The cost of a major surgical complication was estimated from the literature.13 In our base-case analysis, this cost was $13,372 after adjusting for medical price inflation. Because our analysis is from a hospital’s perspective, we did not include outpatient costs or costs to the patient.
Since there is no national estimate of complication rates across all types of surgery and the operating rooms chosen for study in the pilot sites had high baseline complication rates, we used an estimate of complication rates from the literature. This estimate was based on a retrospective review of discharges for all types of surgical procedures from hospitals in Utah and Colorado.14 Based on this estimate, the complication rate for our base-case analysis was 3%.
The relative rate of reduction of major complications with the checklist was estimated from the reduction in complications observed in the high-income sites in the Safe Surgery Saves Lives Study.2 Although there was a 30% relative reduction of major complications at those sites,2 we assumed a 10% relative reduction in major complications with the checklist to account for the possibility that hospitals may experience less dramatic results. We did not include a reduction in post-operative mortality from the checklist in the analysis because the reduction observed at high-income pilot sites was not statistically significant.
We estimated the number of inpatient operations performed on an annual basis at the hospital level based on the literature in conjunction with data from the American Hospital Association regarding the proportion of operations that are inpatient and the number of hospitals performing surgery.15 For our base-case we estimated an annual number of 4,000 inpatient non-cardiac operations.
To determine whether the checklist is cost saving, we compared its use to current practice. The cost associated with current practice was calculated by multiplying the number of non-cardiac inpatient operations performed per year by the complication rate and the cost per major complication. To calculate the cost of the checklist we summed the per-use cost, implementation cost and cost from major complications. Net cost was then calculated by subtracting the cost of checklist use and implementation from the cost of current practice.
In addition to our base-case analysis, we completed one-way sensitivity analyses as well as threshold-level analyses. A summary of the inputs for our base-case and sensitivity analyses is included in Exhibit 2.
When compared to current practice in the base-case analysis, implementation and use of the checklist saves $103,829 for a hospital performing 4000 non-cardiac operations per year. This equates to a savings of $26.96 per operation performed. For every complication averted, there is a net savings of $8,652. To achieve cost savings, at least 5 major complications would need to be prevented with use of the checklist. Cost savings are possible when the additional cost per major complication is as low as $1,574 (Exhibit 3).
For a given baseline complication rate, cost savings achieved with the checklist increase as the relative reduction in complications increases (Exhibit 4). At a 3% baseline complication rate, if there is a relative reduction in complications of only 1% (i.e. the complication rate drops to 2.97%) the checklist costs the hospital $40,589 per year. When the relative reduction in complications increases to at least 3.53% (i.e. the complication rates drops to 2.89%), the checklist is cost saving as demonstrated in the base-case. If complications are reduced by 30%, as observed in high-income sites in the Safe Surgery Saves Lives Study,2 savings would increase to $424,757 per year. If, in addition, the baseline complication rate were as high as 17%,16 savings would be $2,671,253 per year.
As the relative reduction in complications with the checklist increases, the initial complication rate required for a hospital to achieve cost savings with the checklist decreases (Exhibit 5). A baseline complication rate of at least 2.12% would lead to cost savings with the checklist if a 5% relative reduction in complications is achieved. If there is a 15% relative reduction in complications, the checklist is cost saving for a collection of procedures starting with a complication rate as low as 0.71%.
One-way sensitivity analysis demonstrates that the checklist is cost saving except for in instances of either a baseline complication rate of ≤1.06% or a relative reduction in complications of ≤3.53%. Varying the number of operations per year, the additional cost per major complication, and the cost of implementation and use of the checklist does not affect whether the checklist is cost saving (Exhibit 6). However, varying each of these inputs does affect cost savings by one order of magnitude. When the number of inpatient non-cardiac operations is as low as 1000 per year with a 3% complication rate and 10% relative reduction in major complications, the cost savings is $16,481. If the number of operations increases to 8,000 per year, the cost savings increase to $220,293. The variation in cost savings is greatest with variation in the additional cost per major complication, ranging from $23,597–$264,293. Cost savings are relatively insensitive to variation in implementation cost, ranging from $91,194–$110,146.
This study demonstrates that implementation and use of the WHO Surgical Safety Checklist is a cost saving quality improvement strategy. If at least 5 major complications are prevented within the first year of using the checklist, a hospital will realize a return on its investment within that time frame. Since implementation costs make up the majority of the costs associated with the checklist and do not recur, cost savings may occur beyond the first year of use. Furthermore, hospitals that do not prevent 5 major complications in the first year may still achieve savings as the number of complications averted accumulates over a longer period of time.
A previous study of use of a checklist to prevent catheter-related bloodstream infections suggested potential cost savings from checklist use.3 Other studies have suggested that use of a daily goals checklist in an intensive care unit is a cost-effective means for reducing nosocomial infections.17 This study is consistent with these reports and extends upon them by finding that systematic implementation of a simple, relatively inexpensive, low-technology intervention such as a checklist is cost saving.
Hospitals may realize savings through gains in efficiency that we did not include in our analysis because the causal mechanism for these improvements is not yet clear. However, use of a “preflight checklist” in Kaiser Permanente Southern California’s operating rooms resulted in improved nurse retention as turnover decreased from 23% to 7%.18 Also, after implementation of Kaiser Permanente’s checklist, there was a decrease in the number of operative cases that were canceled or delayed.18 Additionally, there is further evidence to suggest that operative briefings may actually decrease delays19 and surgical flow disruptions.20
Because we only accounted for cost-savings through the first year of checklist use, we may have actually underestimated its potential benefit. While there may be costs associated with training new hires and with maintaining checklist use with current employees, we suspect these costs would not be as considerable as the implementation costs. We also suspect that the benefits from reducing complications would persist leading to continued savings. Since use of the checklist is in its early stages, further investigation of the costs associated with continued use as well as its long-term effectiveness is necessary. For example, hospitals may find with time that more dedicated checklist training, as in aviation, produces improved results.
The findings of this study should be interpreted in the context that implementation and use of the checklist is in its early phase. Data on the time required to use the checklist remain limited. However, sensitivity analysis shows that even with added time in the operating room, cost-savings persist. Further, we estimated implementation and peruse costs based on pilot site experience and the experience of early adopters. Although further study of the checklist implementation process and its costs is necessary, we found the occurrence of cost savings to be relatively insensitive to the cost associated with implementation and use. Additionally, we likely overestimated per-use cost of the checklist in our base-case analysis because antibiotic prophylaxis increased by 5.3% after implementation in high-income pilot sites and we based our cost estimate on the increased antibiotic use (26.5%) seen across all study sites.2
Another limitation of this study is that the benefit obtained from use of the checklist is based on the results of a pilot study that had only four high-income sites. While there was a statistically significant 30% relative reduction in major complications observed in the high-income pilot sites, it is not yet clear to what extent this result will prove generalizable nationwide. This concern is mitigated by our conservative assumption of a 10% relative reduction in complications with the checklist in our base-case. Further, we selected the lowest baseline complication rate available in the literature.15,17,21 At a 3% baseline complication rate, a 10% relative reduction in complications equals an absolute reduction to 2.7%, a conservative reduction given studies showing higher baseline complication rates.
While our analysis was at the hospital level, hospitals may not be the sole beneficiaries of cost-savings from the checklist. Since payers are thought to bear a greater burden of the monetary costs associated with surgical complications,22 they may realize a greater proportion of the cost-savings. As such, payers could consider providing hospitals with financial incentives for implementation and use of the checklist.23
Preventing medical errors and adverse events are benefits to society even when not cost saving. There are important quality improvement programs that may not be cost saving but are necessary to improve care. However, cost can be either a barrier or an incentive to implementation. In the current economic climate, evidence of cost-savings may increase the rate of a program’s spread. We recognize that hospital leaders may be sensitive to financial considerations when they consider implementing a quality improvement program.24 We believe there is sufficient evidence of both effectiveness and savings with the WHO Surgical Safety Checklist for hospital leaders in the United States to support a trial of adoption at their institution.