|Home | About | Journals | Submit | Contact Us | Français|
Central venous catheter placement is a common procedure with a high incidence of error. Other fields requiring high reliability have used Failure Mode and Effects Analysis (FMEA) to prioritize quality and safety improvement efforts.
To use FMEA in the development of a formal, standardized curriculum for central venous catheter training.
We surveyed interns regarding their prior experience with central venous catheter placement. A multidisciplinary team used FMEA to identify high-priority failure modes and to develop online and hands-on training modules to decrease the frequency, diminish the severity, and improve the early detection of these failure modes. We required new interns to complete the modules and tracked their progress using multiple assessments.
Survey results showed new interns had little prior experience with central venous catheter placement. Using FMEA, we created a curriculum that focused on planning and execution skills and identified 3 priority topics: (1) retained guidewires, which led to training on handling catheters and guidewires; (2) improved needle access, which prompted the development of an ultrasound training module; and (3) catheter-associated bloodstream infections, which were addressed through training on maximum sterile barriers. Each module included assessments that measured progress toward recognition and avoidance of common failure modes. Since introducing this curriculum, the number of retained guidewires has fallen more than 4-fold. Rates of catheter-associated infections have not yet declined, and it will take time before ultrasound training will have a measurable effect.
The FMEA provided a process for curriculum development. Precise definitions of failure modes for retained guidewires facilitated development of a curriculum that contributed to a dramatic decrease in the frequency of this complication. Although infections and access complications have not yet declined, failure mode identification, curriculum development, and monitored implementation show substantial promise for improving patient safety during placement of central venous catheters.
Central venous catheter (CVC) placement is a common procedure with a high incidence of error, resulting in failure to place a functional catheter, arterial puncture, pneumothorax, catheter-associated bloodstream infection, or retained guidewires.1–,4 Improving the performance of this procedure requires a systematic strategy.5 Because residents perform most CVC placements in teaching settings, we developed a simulation-based training program for this intervention. Several studies have reported the benefits of simulation-based training for CVC placement. A prospective, randomized, controlled trial of standard training followed by practice on patients versus simulator-based training found that the simulator training group had a significantly higher level of comfort and ability in placing CVCs and significantly less complications than the group who trained using real patients.6 Another study showed that the knowledge and confidence gained from simulator-based CVC training was retained after 18 months.7
Systematic efforts to detect and address sources of error can result in further improvements. High-reliability industries, such as nuclear power and aerospace use Failure Mode and Effects Analysis (FMEA)8 to identify high-priority failure modes. The FMEA process recognizes that it is impossible to eliminate all failures and focuses on reducing the frequency, decreasing the severity, and improving the detection of failures before they lead to harm. These systems also continually monitor performance and assess whether interventions have had the desired effect on frequency, severity, and detection of failure modes.5,8
Cardiopulmonary resuscitation (CPR) training already uses elements of FMEA, providing a standardized education format that includes opportunities for practice and for multiple assessments.9–,14 In contrast, CVC placement is typically learned through “on-the-job” training sessions,15 which follow the tradition of “see one, do one, teach one.” Training in CVC placement rarely includes strategies to avoid or detect known failure modes and usually lacks an assessment component. As a result, it is difficult to ascertain whether a trainee has mastered the desired learning objectives before performing the procedure on a patient without direct supervision. The CVC curricula lag far behind CPR curricula, even though the average new intern can expect to encounter many more instances of CVC insertion.
This prompted us to develop a formal, standardized curriculum for CVC training based on data from our concurrent failure mode analysis. The curriculum builds upon simulation-based training that focuses instruction on important failure modes, an approach that is well established in other fields16–,19 and has increasingly been adopted in health care programs.20,21 The process creates feedback loops similar to those found in the process-improvement literature.16,22–,25
In 2006, our 1200-bed, tertiary hospital launched an initiative to improve patient safety during CVC placement. A multidisciplinary committee was formed with representation from internal medicine, surgery, radiology, emergency medicine, anesthesiology, neurology, and obstetrics and gynecology to develop and implement recommendations for improving performance during CVC placement in adult patients. The committee also sought the expertise of vascular access nurses, critical care nurses, process improvement consultants, and simulation experts. The effort was complemented by a formal value stream analysis conducted in March 2007. After consultation with our Institutional Review Board, it was determined that informed consent was not needed to participate in this curriculum because this training was a required part of new intern orientation.
In 2007, 112 physician trainees from 6 departments participated in the training curriculum. Over the course of 3 years, the number and diversity of trainees has expanded, and in 2009, the course included 124 physicians and 6 advanced-practice nurses from 9 departments.
The resulting modular curriculum included segments that were delivered online or through hands-on training sessions. Each hands-on training session was hosted in the medical center's simulation facilities and used 16 to 19 instructors. The online content was delivered through 2 different learning management systems.
A series of internal and external data sources were used to identify high-priority failure modes.5 Because reliable and objective data on specific failure modes were lacking when the project was initiated, our primary data sources were morbidity and mortality conferences, the Joint Commission's sentinel events, national patient safety goals, the Centers for Medicare and Medicaid Services “never events” list, and the personal observations of the multidisciplinary committee members. This analysis is described in detail in supplement 1 of the online version of this article. The committee then used simulation to investigate the mechanisms for these failures. Once failure modes were defined, learning objectives were developed to address them.
To develop the online modules, the committee created a storyboard that included content and short assessments, using a commercial software package (Adobe Captivate 3.0, San Jose, CA) for early testing because it generated content in a file format (HTML) that could be viewed with Internet browsing software. Prototype training modules were then loaded onto a commercial hosting service that included a learning management system (Adobe Connect Pro, San Jose, CA) to facilitate more widespread testing and to track trainee responses to the online assessments. After revision, the online training modules were hosted on 1 of 2 commercial systems (Pathlore, BJC Learning Center, Town and Country, MO, or Etrinsic, a division of Simbionix, Denver, CO).
To develop the hands-on training sessions, the committee defined learning objectives for each simulation station. These instructor-led sessions were designed to last 20 to 30 minutes and covered 6 to 8 learning objectives. The instructors worked with the simulators and other materials to develop an approach that suited their individual style.
The online courses were available by the first of June during each year of the study, and the hands-on sessions were held during the second and third weeks of June. These sessions were a required part of new intern orientation. For the online courses, trainees were notified through an e-mail that contained the link to the course's secure website. All online modules included an assessment, and some modules, such as catheter-associated bloodstream infection, required trainees to obtain a passing score before starting their internship.
The hands-on courses were held in the medical school's simulation center over 3 to 4 days. Initially, training followed a serial pattern, but given the limited time for these sessions and the knowledge that repetition is a crucial aspect of learning any psychomotor skill,26 we moved toward a parallel training structure where trainees are able to repeatedly practice the skill during the 20 to 40 minute training session. In 2009, each day, groups of 25 to 45 trainees rotating through a series of 10 different stations, and each station was hosted by 1 or more instructors who led the trainees through a series of exercises.
We assessed the effectiveness of training through several different methods. For the online modules, in addition to trainee acceptance of the material, we used item analysis to identify problematic content or other factors that affected achievement of learning objectives. The learning management systems were capable of recording the number of incorrect responses to each question. Although most questions had a single best answer, several matching questions were used, and other questions required trainees to create an ordered list. Starting in 2009, a standardized assessment was added. This assessment required each trainee to return to the simulation center and complete an entire simulated procedure. Trainees were graded using standardized criteria, and a passing grade was required before completing postgraduate year 1 (PGY-1).
Survey data indicated that new interns (PGY-1 trainees) had little practical experience with CVC placement (Figure 1) or with crucial diagnostic tools, such as ultrasound. Of the 72 interns completing the survey as part of the training course, only 25% (18 of 72) indicated they had participated in a formal training program for central venous access during medical school. The data also indicated that most had little or no experience with placing CVCs, and even fewer had experience with ultrasound guidance.
We estimated that these trainees would participate in more than 5000 central line placement procedures each year. Even in cases where they might not place the catheter, the trainees would still be expected to identify appropriate candidates for CVC placement, to care for patients having many different types of CVCs, and to make decisions about removing those catheters.27
The resulting training program evolved considerably during the past 3 years (online supplement 2). The current curriculum contains 4 parts: (1) a series of online training modules review the indications, contraindications, complications, and basic technique for CVC; (2) trainees participate in hands-on, simulation-based, instructor-led training sessions; (3) trainees are required to pass a standardized assessment during the latter months of their PGY-1; and (4) training in ultrasound image interpretation and scanning was added in 2009.
The FMEA process was used to identify high-priority topics for training.5,8 Priority numbers were determined by multiplying frequency, severity, and ability to escape detection (online supplement 1) and were initially based on rough estimates. In the coming years, our ability to track specific failure modes and refine those estimates will improve with the implementation of electronic health records.
Retained guidewires became a high-priority topic when the Joint Commission designated it a sentinel event in 2006. Although the number of retained guidewires observed at our institution fell in 2007 (Figure 2), the 2007 rate was still considered unacceptable. The failure modes that caused retained guidewires were examined in detail (online supplement 1), and the analysis resulted in changing the emphasis of training from keeping a firm grip on the guidewire to inserting the guidewire to its 20-cm mark and maintaining that position while the tract was dilated and the catheter inserted (Table 1). Since introducing the catheter and guidewire skills module, the number of retained guidewires has decreased further (Figure 2).).
Failure to access the vein and unintended arterial puncture were considered a linked pair of high-priority failure modes that could be addressed by ultrasound guidance.28,29 To help address this need, the hospital purchased a number of ultrasound units in 2008. That purchase prompted the multidisciplinary group to begin developing and testing training materials for ultrasound guidance. Module development required a detailed evaluation of the tasks performed during ultrasound-guided access and the possible failure modes. That analysis suggested dividing ultrasound guidance into 2 parts30,31: (1) interpreting the ultrasound image and using it to determine a suitable needle path from the skin to the internal jugular vein (Figure 3C), and (2) using real-time ultrasound to confirm that the needle was being advanced along the chosen path. The failure modes for image interpretation were catalogued and used to develop training objectives. The resulting online training modules were introduced in 2009. At the same time, the team began creating content needed for a hands-on training module that included objective assessment tools.
Breaks in sterile technique have always been a high-priority failure mode because catheter-related bloodstream infections cause substantial, excess morbidity and mortality.32–,34 Studies have illustrated how the rate of such infections can be decreased through maximum sterile barriers and hand hygiene.35 Before the 2008 course, the multidisciplinary committee revised the contents of the standard central line tray so that it included larger drapes. Since 2008, the course has emphasized the rationale for these changes and this includes a demonstration of how guidewires could easily extend beyond the edge of the smaller drapes. Assessment of competence in sterile techniques was also added in 2009. The analysis of operational data (Figure 2) has yet to show significantly fewer catheter-associated bloodstream infections.
The learning objectives provided a means of measuring the curriculum's effectiveness (Table 2). The online training modules combined explanations and examples of the desired mental models with assessments designed to elicit evidence that the trainee had mastered these objectives (Figure 3). This paradigm was based on motor control theory, which contends that all planned actions consist of planning and execution stages. The online assessments focused on planning skills because cognitive psychologists have emphasized the importance of errors in planning, which unlike errors in execution, are often missed.19,36,37 They also stress how robust planning skills allow the operator to anticipate execution errors and to develop appropriate contingency plans, which further improve system reliability.38 As shown in Figure 3, the online modules were well suited to the assessment of planning skills. In contrast, execution skills were best evaluated by direct observation during the hands-on sessions.
The assessment process also revealed which segments of the current curriculum were effective and which required improvements (Figure 4). Although the best students demonstrated few errors, a pattern emerged when analyzing the performance of good and fair students. A few questions accounted for most of the errors. Applying FMEA to this data set led to 2 potential remedies: (1) errors that resulted from poorly worded questions could be addressed by revising the test, and (2) errors that resulted from inadequate time, context, or exercises for a topic could be addressed by revising elements of the curriculum. Monitoring the results after changing the content or the question provides feedback on whether the changes have the intended results. This overall process of developing our curriculum is summarized in online supplement 3, a systematic approach to venous access.
Our attempt to develop a standardized curriculum for CVC placement began with a global needs assessment and used task and failure mode analysis to identify specific training needs. Failure modes were then separated into planning and execution failures. This division prompted us to develop a combination of online training modules, which focused on planning skills, and hands-on exercises, which targeted execution skills. Planning skills were assessed by incorporating questions into the online modules. Execution skills were assessed by having instructors observe performance during simulated procedures. Assessment results were used to complete a data-driven, curriculum-development cycle.
Aspects of developing a standardized curriculum for CVC placement proved challenging. First, trainees from at least 8 different departments were organized into more than 15 different teams that place CVCs. The resulting variation in processes confounded efforts to systematically collect data on failure modes and to develop standardized training. Second, this effort required substantial resources. Experts from multiple disciplines spent considerable time in meetings to determine objectives and coordinate training approaches. Creating the online modules required multiple development and testing cycles before they were ready for use. The hands-on training and assessment sessions required coordinating the schedules of multiple instructors with the availability of the simulation center.
The effectiveness of any curricula should be judged according to its ability to change the learner's behavior. Too often training is judged according to the ease with which it is produced, packaged, and delivered. Instead, curricula should be structured as environments that facilitate the creation and refinement of each trainee's mental models.39–,41 As such, we intend to continually focus, not on the amount of content delivered or performance observed during simulated procedures, but on evidence of how trainees used the desired mental models when caring for patients. Achieving this goal requires verifying that trainees not only learn the overall process needed to reliably choose, place, maintain, and remove CVCs (online supplement 4) but also reliably apply the resulting knowledge, skills, and abilities to their daily work.
One of the major advantages of the FMEA approach is that it creates a feedback cycle where data from clinical operations can be integrated into curriculum development. By requiring objective definitions of individual failure modes, FMEA improved our understanding of the underlying system. This improved our ability to measure performance during both simulated and actual patient procedures. Said another way, one does not fully understand how any complex system works until one has to correctly diagnose its failures, prescribe appropriate remedies, and assess the effectiveness of the actions taken. Indeed, medical care itself benefits from repeated cycles of diagnosis, treatment, and monitoring.
We caution other programs and institutions against adopting our training strategies without first conducting their own FMEA. Although failure modes are universal, the frequency, severity, and detection of each individual failure mode will depend on the capabilities of each system's control strategies. Thus, FMEA results and the priority for individual failure modes will vary between programs.5,8 As a result, our curriculum will not be universally effective. However, given that agencies such as the Joint Commission and Centers for Medicare and Medicaid Services have set national priorities that can be mapped back to certain failure modes, a few failure modes will be high-priority targets for many systems. This suggests that effective control strategies will likely prove useful in a wide variety of care settings.
This manuscript focuses on training because, for the foreseeable future, system performance will remain heavily dependent on operator skill. The benefits from ultrasound, maximum sterile barriers, and improved catheter and guidewire skills will only be derived if the human operators effectively employ ultrasound guidance, rigorously maintain sterility, and adroitly manipulate the catheters and guidewires. We contend that the traditional, informal training method of “see one, do one, teach one” is highly variable and leads to variation in skill that degrades system reliability far more than the decrements attributable to variation in tools, environment, or patients.
Still, we recognize that overzealous efforts to standardize training can have undesirable consequences,31 such as a loss of system flexibility. Rigid application of inflexible mental models will lead to complications when unforeseen or “latent” failure modes are encountered.36,37 Heterogeneous training populations are another shortcoming of standardized training programs because standardized training follows a “one size fits all” approach. Individuals with preexisting skills and who are fast learners will become bored when reviewing introductory material, whereas true novices will suffer from cognitive overload if the training pace is hastened.40 We expect that adaptive training programs might overcome these limitations because they use assessments to diagnose preexisting skills and prescribe coursework accordingly. However, such adaptive programs require substantially more resources to develop, and they introduce their own set of potential failure modes.
Curricula must be continually revised to maximize the efficacy and efficiency of training.39,42–,48 The marked decrease in retained guidewires suggests the training program has effectively addressed the underlying failure modes. The lack of marked improvement in catheter-associated bloodstream infections suggests that either the training program was ineffective at changing physician behavior or that factors other than breaks in sterile technique during catheter insertion are the primary cause for these infections. Our data on curriculum effectiveness suggest the current curriculum is marginally effective, and the observed improvements in patient outcomes could easily be attributed to other factors. However, the current effort has convinced members of the multidisciplinary committee that there is a clear need for the wide variety of teams involved in central venous access to continue working together. As described by Senge and Argote, organizational learning is driven by feedback loops.16,17 These loops require data, and they, in turn, require the creation of operational definitions of success and failure at numerous steps in the process. Learning organizations rely on personal mastery and explicit communication of the resulting mental models. Although there will always be reasons why a particular mental model might fail for a specific patient, without an explicit description and communication of the mental model and the nature of the failure, the organization will miss an opportunity to learn from that event. As a result, curriculum development never ends. Performance can always be improved.
James R. Duncan, MD, PhD, is an Associate Professor of Radiology at Washington University School of Medicine and St. Louis Children's Hospital; Katherine Henderson, MD, is an Assistant Professor of Clinical Medicine at Washington University School of Medicine and Barnes-Jewish Hospital; Mandie Street, RT, is a Quality and Safety Coordinator at Washington University School of Medicine; Amy Richmond, RN, an Infection Control Nurse at Barnes-Jewish Hospital; Mary Klingensmith, MD, is a Professor of Surgery at Washington University School of Medicine; Elio Beta, BS, is a Medical Student at the University of Illinois School of Medicine; Andrea Vannucci, MD, is an Assistant Professor of Anesthesiology at Washington University School of Medicine and Barnes-Jewish Hospital; and David Murray, MD, is a Professor of Anesthesiology at Washington University School of Medicine and St. Louis Children's Hospital.
The authors report no conflicts of interest.
This work was supported by grants from the Barnes-Jewish Hospital Foundation and the Anesthesia Patient Safety Foundation.
The authors wish to acknowledge Dr Jonathan Gottlieb for his enthusiastic support of this project. Drs Gottlieb and Vannucci served on the multidisciplinary Central Venous Access Committee along with Drs Geoffrey Cislo, Perren Cobb, Daniel Theodoro, Brent Ruoff, Michael Diringer, Katherine Henderson, David Murray, and James Duncan. Terra Mouser, MBA; Theresa Sieber; Amy Richmond, RN, MHS, CIC; and Cathy Robinson, BA, RN, CRNI, also served on the committee. Julie Woodhouse BSN, MEd, and Rebecca Snider, BSN, RN, MHS, were instrumental in setting up and conducting the hands-on simulation training sessions and assessments.
Editor's Note: The online version of this article contains supplemental material such as determining and investigating high-priority failure modes, the evolution of the curriculum, process map and feedback loops for curriculum development, and a systematic approach to venous access.