Implementation and evaluation of a simulation curriculum for paediatric residency programs including just-in-time in situ mock codes

Paediatr Child Health. 2012 February; 17(2): e16–e20.

PMCID: PMC3299361

Implementation and evaluation of a simulation curriculum for paediatric residency programs including just-in-time in situ mock codes

Jonathan Sam, MD FRCPC, Michael Pierse, MD FRCPC, Abdullah Al-Qahtani, MD, and Adam Cheng, MD FRCPC

Department of General Pediatrics, BC Children’s Hospital, Vancouver, British Columbia

Correspondence: Dr Jonathan Sam, Department of General Pediatrics, BC Children’s Hospital, 4480 Oak Street, Vancouver, British Columbia V6H 3V4. Telephone 604-314-8338, fax 604-875-2890, e-mail Jonathan.Sam/at/utoronto.ca

Accepted March 22, 2011.

Abstract

OBJECTIVE:

To develop, implement and evaluate a simulation-based acute care curriculum in a paediatric residency program using an integrated and longitudinal approach.

DESIGN:

Curriculum framework consisting of three modular, year-specific courses and longitudinal just-in-time, in situ mock codes.

SETTING:

Paediatric residency program at BC Children’s Hospital, Vancouver, British Columbia.

INTERVENTIONS:

The three year-specific courses focused on the critical first 5 min, complex medical management and crisis resource management, respectively. The just-in-time in situ mock codes simulated the acute deterioration of an existing ward patient, prepared the actual multidisciplinary code team, and primed the surrounding crisis support systems. Each curriculum component was evaluated with surveys using a five-point Likert scale.

RESULTS:

A total of 40 resident surveys were completed after each of the modular courses, and an additional 28 surveys were completed for the overall simulation curriculum. The highest Likert scores were for hands-on skill stations, immersive simulation environment and crisis resource management teaching. Survey results also suggested that just-in-time mock codes were realistic, reinforced learning, and prepared ward teams for patient deterioration.

CONCLUSIONS:

A simulation-based acute care curriculum was successfully integrated into a paediatric residency program. It provides a model for integrating simulation-based learning into other training programs, as well as a model for any hospital that wishes to improve paediatric resuscitation outcomes using just-in-time in situ mock codes.

Keywords: Curriculum, Education, Paediatric, Residency, Resuscitation, Simulation

Résumé

OBJECTIF :

Élaborer, mettre en œuvre et évaluer un cursus de soins aigus fondé sur la simulation dans un programme de résidence en pédiatrie au moyen d’une démarche intégrée et longitudinale.

MÉTHODOLOGIE :

Le cadre du cursus contient trois cours modulaires enseignés lors d’une année donnée et des scénarios longitudinaux juste-à-temps et in situ.

LIEU :

Programme de résidence en pédiatrie du BC Children’s Hospital de Vancouver, au Canada.

INTERVENTIONS :

Les trois cours enseignés lors d’une année donnée traitent des cinq premières minutes, essentielles, de la prise en charge médicale complexe et de la gestion des ressources en cas d’urgence, respectivement. Dans le cadre des scénarios juste-à-temps et in situ, on simulait la détérioration aiguë d’un patient déjà hospitalisé dans un service, on préparait une véritable équipe multidisciplinaire d’urgence et on organisait des systèmes de soutien connexes en cas d’urgence. Les chercheurs ont évalué chaque élément du cursus au moyen de sondages construits selon l’échelle de cinq points de Likert.

RÉSULTATS :

Au total, 40 sondages auprès des résidents ont été remplis après les cours modulaires, et 28 autres à l’égard de l’ensemble du cursus sur la simulation. Les indices de Likert les plus élevés portaient sur les ateliers pratiques, les milieux de simulation immersifs et l’enseignement de la gestion des ressources en cas d’urgence. Les résultats du sondage indiquaient également que les scénarios juste-à-temps étaient réalistes, qu’ils renforçaient l’apprentissage et préparaient les équipes des services à la détérioration de l’état des patients.

CONCLUSIONS :

L’hôpital a réussi à intégrer un cursus de soins aigus fondé sur la simulation à son programme de résidence en pédiatrie. Il fournit un modèle d’intégration de l’apprentissage par la simulation à d’autres programmes de formation et un modèle pour tout hôpital qui désire améliorer les issues de réanimation en pédiatrie au moyen des scénarios d’urgence juste-à-temps et in situ.

Critically ill children requiring acute resuscitation are rare (1,2). When they do occur, outcomes are poor with reports of survival-to-discharge rates at 14% to 36% (3–5). In many paediatric hospitals, residents are the first physicians to respond, but studies show that their experiences in leading or participating in real resuscitations are limited and that current training methods are inadequate (6). While Paediatric Advanced Life Support courses increase resuscitation knowledge in the short term (7), retention of skills and knowledge without the opportunity for hands-on practice has been poor (8,9).

The effectiveness of simulation in teaching acute care is well established. By creating a realistic and safe learning environment that focuses on both experiential and reflective learning, simulation has been proven to teach the knowledge, procedural skills and crisis resource management required during successful resuscitations (10–13). While multiple reports introducing mock code programs into paediatric centres have been published (14–17), few describe integrating a simulation-based curriculum into paediatric training (18,19). The goal of our project was to develop, implement and evaluate a simulation-based acute care curriculum including modular, year-specific courses and longitudinal just-in-time mock codes for our paediatric residency program at British Columbia’s Children’s Hospital (BCCH) in Vancouver, British Columbia.

METHODS

Needs assessment

A committee of content experts from BCCH reviewed the Royal College of Physicians and Surgeons of Canada (RCPSC) and Accreditation Council for Graduate Medical Education (ACGME) objectives for paediatric residency training (20,21). From these, the committee identified acute care objectives that were best suited for simulation-based learning. Resident focus groups were then conducted to determine acute care topics and skills that they felt least comfortable with. With the information obtained from the focus groups and review of RCPSC and ACGME objectives, the committee allocated learning objectives for each of the simulation-based modular courses. A summary of the objectives is outlined in Table 1.

TABLE 1

Paediatric simulation curriculum objectives

Modular year-specific courses

Development:

Three full-day courses were designed to meet the distinct learning objectives of paediatric residents in years 1 through 3 respectively. A blended approach was chosen to address various learning styles and core knowledge was emphasized with didactic teaching, technical competence with skill stations and clinical application with simulation scenarios. All scenarios were reviewed and pilot tested as part of the existing paediatric emergency medicine simulation-based curriculum at BCCH. Although many of the cases were similar across different years, their specific objectives and subsequent learning outcomes were noticeably different.

Implementation:

Year 1 and 2 courses were scheduled early in the academic year, while the year 3 course occurred later to prepare incoming senior residents for the code team leader role. Year-specific prereading materials including selected hospital clinical guidelines, and evidence-based consensus statements were circulated one to two weeks before the start of each course. All courses were held at the Centre for Excellence in Simulation Education and Innovation at Vancouver General Hospital (British Columbia). Courses were scheduled during protected academic time to ensure residents were able to attend. Instructors consisted of four attending staff (from emergency, intensive care and general paediatrics) familiar with simulation-based learning methodology, and each was paired with one chief resident or emergency fellow at each learning station. Instructors were provided with teaching assignments and appropriate materials one to two weeks before the start of the course.

Course agendas were developed using a similar format (Table 2). Residents were divided into groups of three to four for skill and simulation stations. Simulation stations were 15 min for the scenario followed by 15 min for staff-facilitated debriefing. Residents rotated through different roles for each simulation, providing multiple opportunities to be the team leader during each course.

TABLE 2

Typical modular year-specific course schedule

To meet the distinct learning objectives of each year, course structure and design were varied for each year. The year 1 course was designed to emphasize the assessment and management of a critically ill child in the first 5 min. This was achieved by conducting shorter, 5 min simulation scenarios with 5 min debriefings in the morning. To reinforce the importance of quality cardiopulmonary resuscitation (CPR), a CPR feedback device was used to provide real-time feedback on depth, rate and recoil of chest compressions. A fluid administration skills station was designed to give residents the opportunity for hands-on practice and to compare the efficiency of various methods. The year 2 course focused on complex medical management with longer and more challenging scenarios. The final year 3 course emphasized crisis resource management (CRM) principles. These were highlighted with carefully scripted scenarios and included one scenario with a blindfolded leader to illustrate the importance of closed-loop communication and situational awareness.

Evaluation:

Each year-specific course was evaluated with resident surveys using a five-point Likert scale, with 1 indicating strong disagreement and 5 indicating strong agreement. Categories for each course differed and were based on their unique learning objectives.

Longitudinal just-in-time mock codes

Development:

To complement and reinforce the learning from year-specific modular courses, a just-in-time, in-situ mock code program was instituted and run in a longitudinal manner by the paediatric chief residents. These simulation-based mock codes were run one to two times per month on BCCH wards. To build the just-in-time component into the mock codes, paediatric chief residents selected one of the sickest patients on the clinical teaching unit (CTU) that day and created a simulation scenario where that patient acutely deteriorated. Learning objectives for each case were determined in discussion with the CTU director and ward nurse educators (See Table 3 for sample learning objectives).

TABLE 3

Sample learning objectives for just-in-time mock codes

Implementation:

A portable, high-fidelity infant simulator was set up in a standard patient ward room. Relevant patient documents were copied and modified to coincide with deterioration while protecting patient confidentiality. Nursing records, flow sheet vital signs, medication administration records and recent physician orders were made available when appropriate, but identifying information was hidden. The actual bedside nurse was involved whenever possible and often initiated the mock code with a concerned call to the junior resident without prior notification of a mock code event.

At BCCH, the multidisciplinary code team consists of CTU residents, bedside and ward charge nurses, respiratory therapists and intensive care unit code team (nurses and residents). Each mock code was carried out in real time with progressive involvement of all members of the code team as needed and generally lasted 20 min in total. The chief resident, CTU director and nurse educator directly observed and facilitated a 10 min debriefing session immediately afterward. In addition to an annual year-specific simulation course, each paediatric resident should participate in four to eight (average of six) just-in-time mock codes on CTU and intensive care unit rotations during a three-year training period.

Evaluation:

Residents completed a survey six months after the implementation of just-in-time mock codes and after all modular courses had been run at least once. It contained 10 items and used a five-point Likert scale.

RESULTS

Modular year-specific courses

The year 1 course was evaluated with 14 resident surveys using a five-point Likert scale. Because this course focused on the critical first 5 min, it was important that residents felt shorter scenarios in the morning emphasized initial assessment and management (mean score 4.7) and felt more comfortable with initial assessment and management of sick patients at the end of the course (mean score 4.8). The year 2 course was evaluated with eight resident surveys. See Figure 1 for a summary of year 1 and 2 course survey data. The year 3 course was evaluated with 18 resident surveys. See Figure 2 for a summary of the quantitative results from the year 3 course, and Table 4 for a summary of qualitative feedback from all three courses.

Figure 1)

Summary of year 1 and 2 course evaluation survey

Figure 2)

Summary of year 3 course evaluation survey

TABLE 4

Summary of qualitative feedback from modular courses

Longitudinal just-in-time mock codes

Just-in-time mock codes were evaluated with 28 resident surveys (Figure 3). Residents agreed that they reinforced learning from year-specific courses (mean score 4.3), creating scenarios using real ward patients made mock codes more realistic (mean score 4.4), including multidisciplinary team improved learning (mean score 4.3), and they prepared ward teams for potential deterioration of sicker patients (mean score 4.5).

Figure 3)

Just-in-time mock codes evaluation survey

Overall simulation curriculum

The early impact of the entire simulation curriculum was evaluated by 28 resident surveys (Figure 4). Residents found that simulation provided a safe learning environment (mean score 4.6) and was better than didactic lectures at teaching acute care, communication and teamwork issues (mean score 4.7). They also believed that the simulation curriculum would help to decrease their level of anxiety in future resuscitations (mean score 4.5) and should be implemented in all paediatric residency programs (mean score 4.6).

Figure 4)

Overall simulation curriculum evaluation survey

DISCUSSION

Previous generations of health care providers have learned acute care medicine based on the “see one, do one, teach one” mantra, while the current generation of trainees have the opportunity to practice to perfection in simulation, grounded in the principles of improving patient safety and outcomes (22). Our curriculum safely increased resident exposure to critically ill children in a realistic, yet controlled, environment, while concurrently addressing RCPSC and ACGME learning objectives required for certification of training. Furthermore, an emerging body of evidence suggests that the greatest benefit of simulation may reside in its ability to train multidisciplinary teams, help identify human errors, and modify team behaviour leading to a reduction in medical errors (23–25). In one study, Hunt et al (26) examined 34 unannounced, hospital-based paediatric mock codes and found that 100% had communication errors and represented an important opportunity to improve real patient outcomes. We addressed this need throughout our curriculum by using multidisciplinary teams whenever possible, discussing communication and teamwork issues during all debriefing sessions, and focusing our final modular course on CRM skills. Our survey data reflect this potential benefit with the strongest scores occurring in the areas of communication and teamwork.

The evidence also suggests that teaching resuscitation through Paediatric Advanced Life Support courses and didactic lectures lead to significant decay in knowledge and skills (8,9). Without the reinforcement of regular hands-on practice, subsequent resuscitations on real patients are likely to be suboptimal. A recent review of existing best practices for resuscitation education coincidently included reducing course duration and instead distributing practice sessions over time (27). It also suggested maximizing the time spent in deliberate practice on manikins, constructing scenarios that match the learners’ usual clinical practice, and integrating simulation into resuscitation curricula. In addition, a recent study has shown that as few as two ‘rolling refreshers’ per month in the critical care setting using just-in-time CPR training can improve performance in subsequent real resuscitations (28). In designing our own curriculum and considering the evidence, the concept of just-in-time mock codes was incorporated to address multiple learning needs simultaneously. First, they were distributed at monthly intervals to allow regular hands-on practice and help prevent knowledge and skill decay. Second, they were multidisciplinary and involved the actual paediatric code team to help identify and reduce team-work and communication errors. Third and most importantly, just-in-time mock codes were as realistic as possible and meant to be predictive of patient deterioration. By using real patient information and documents in combination with a portable paediatric simulator, we were able to recruit the actual bedside nurse and have the responsible CTU residents as first responders. This not only enhanced the situational and emotional realism, but often identified issues with actual equipment, access to resources, and other obstacles to a timely response that are otherwise difficult to recognize. In essence, just-in-time mock codes trained the human element and primed the surrounding crisis support systems to have the greatest potential for improving our sick children’s care.

Our curriculum was by no means a static proposition but an evolving educational process. In comparison to the simulation-based curriculum for paediatric emergency fellows at our centre (18), our residency trainees’ learning objectives varied widely. From the first-year resident with minimal clinical experience to the fourth year resident who will soon graduate to assume responsibilities as a community paediatrician, we approximated the steep learning curve with our integrated curriculum framework while allowing for flexibility in the longitudinal growth of any given cohort as they pass through the curriculum. In accordance, we noted an evolution in perceived value of various teaching methods at different stages of training. First-year resident survey scores and feedback suggested greater relative benefit from lectures and may reflect increased knowledge gaps. In comparison, second-year residents appreciated the hands-on procedural skills and complex medical simulation scenarios, while senior resident feedback emphasized the benefit of focused teaching on communication and leadership skills. Each year-specific course built upon the foundation from the previous, every just-in-time mock code reinforced the knowledge and skills that could have potentially been lost, and annual needs assessments ensured that the curriculum continued to adapt. While many paediatric centres have used simulation in some capacity, our experience suggests that an understanding of your centre’s unique resources and opportunities, an evolving assessment of your trainees’ needs, combined with a commitment to an integrated and longitudinal approach for teaching resuscitation through simulation, may have the most lasting impact on future generations of trainees and their patients.

Our study’s primary limitation was that we were not able to evaluate resident performance because the study design did not include a pre- and postintervention comparison or control group. However, studies have shown positive correlations between resident self-assessment surveys and performance in resuscitation team leadership skills and critical resuscitation procedures (29,30). The overall curriculum surveys may have been susceptible to recall bias because they were performed at least six months after the trainees’ modular course, but this increased the likelihood of both participating in just-in-time mock codes and having the opportunity to apply their learning in real situations. Future research is needed to determine the impact of educational interventions such as ours on resuscitation performance and patient outcomes.

CONCLUSION

We used an integrated and longitudinal approach to develop, implement and evaluate a successful simulation-based, acute care curriculum for our paediatric residency program. Resident satisfaction surveys highlighted the effectiveness of hands-on skill stations, immersive simulation environment and CRM teaching. The curriculum provides a model for integrating simulation-based learning into other training programs as well as a model for any hospital that wishes to improve paediatric resuscitation outcomes using just-in-time in-situ mock codes. Further research is required to assess the impact of educational interventions such as these on performance and patient outcomes.

Acknowledgments

The authors thank Dr Jennifer Druker for her support in her roles as paediatric residency program director and CTU director.

REFERENCES

1. de Mos N, van Litsenburg RR, McCrindle B, et al. Pediatric in-intensive-care-unit cardiac arrest: incidence, survival, and predictive factors. Crit Care Med. 2006;34:1209–15. [PubMed]

2. Slonim AD, Patel KM, Ruttimann UE, et al. Cardiopulmonary resuscitation in pediatric intensive care units. Crit Care Med. 1997;25:1951–5. [PubMed]

3. Reis AG, Nadkarni V, Perondi MB, et al. A prospective investigation into the epidemiology of in-hospital pediatric cardiopulmonary resuscitation using the international Utstein reporting style. Pediatrics. 2002;109:200–9. [PubMed]

4. Young KD, Seidel JS. Pediatric cardiopulmonary resuscitation: A collective review. Ann Emerg Med. 1999;33:195–205. [PubMed]

5. Tibballs J, Kinney S. A prospective study of outcome of inpatient paediatric cardiopulmonary arrest. Resuscitation. 2006;71:310–8. [PubMed]

6. Donoghue AJ, Durbin DR, Nadel FM, et al. Effect of high-fidelity simulation on Pediatric Advanced Life Support training in pediatric house staff: A randomized trial. Pediatr Emerg Care. 2009;25:139–44. [PubMed]

7. Brown TB, Dias JA, Saini D, et al. Relationship between knowledge of cardiopulmonary resuscitation guidelines and performance. Resuscitation. 2006;69:253–61. [PubMed]

8. Jewkes F, Phillips B. Resuscitation training of paediatricians. Arch Dis Child. 2003;88:118–21. [PMC free article] [PubMed]

9. Nadel FM, Lavelle JM, Fein JA, et al. Teaching resuscitation to pediatric residents: The effects of an intervention. Arch Pediatr Adolesc Med. 2000;154:1049–54. [PubMed]

10. Cheng A, Duff J, Grant E, et al. Simulation in paediatrics: An educational revolution. Paediatr Child Health. 2007;12:465–8. [PMC free article] [PubMed]

11. Eppich WJ, Adler MD, McGaghie WC. The use of medical simulation in the training of acute pediatric emergencies. Curr Opin Pediatr. 2006;18:266–71. [PubMed]

12. Marshall RL, Smith JS, Gorman PJ, et al. Use of a human patient simulator in the development of resident trauma management skills. J Trauma. 2001;51:17–21. [PubMed]

13. Holcomb JB, Dumire RD, Crommett JW, et al. Evaluation of trauma team performance using an advanced human patient simulator for reusscitation training. J Trauma. 2001;52:1078–86. [PubMed]

14. Andreatta P, Saxton E, Thompson M, Annich G. Simulation-based mock codes significantly correlate with improved pediatric patient cardiopulmonary arrest survival rates. Pediatr Crit Care Med. 2011;12:33–8. [PubMed]

15. Friedman D, Zaveri P, O’Connell K. Pediatric mock code curriculum: Improving resident resuscitations. Pediatr Emerg Care. 2010;26:490–4. [PubMed]

16. Tofil NM, Lee White M, Manzella B, et al. Initiation of a pediatric mock code program at a children’s hospital. Med Teach. 2009;31:e241–7. [PubMed]

17. Mikrogianakis A, Osmond MH, Nuth JE, et al. Evaluation of a multidisciplinary pediatric mock trauma code educational initiative: A pilot study. J Trauma. 2008;64:761–7. [PubMed]

18. Cheng A, Goldman RD, Aish MA, et al. A simulation-based acute care curriculum for pediatric emergency medicine fellowship training programs. Pediatr Emerg Care. 2010;26:475–80. [PubMed]

19. Adler MD, Vozenilek JA, Trainor JL, et al. Development and evaluation of a simulation-based pediatric emergency medicine curriculum. Acad Med. 2009;84:935–41. [PubMed]

20. Royal College Physicians and Surgeons of Canada, Pediatrics, Objectives of Training < http://rcpsc.medical.org/information> (Accessed January 2011).

21. Accreditation Council for Graduate Medical Education, Pediatrics Program Requirements < http://www.acgme.org/acWebsite/RRC_320/320_prIndex.asp> (Accessed January 2011).

22. Nishisaki A, Keren R, Nadkarni V. Does simulation improve patient safety? Self-efficacy, competence, operational performance, and patient safety. Anesthesiol Clin. 2007;25:225–36. [PubMed]

23. Shapiro MJ, Morey JC, Small SD, et al. Simulation based teamwork training for emergency department staff: Does it improve clinical team performance when added to an existing didactic teamwork curriculum? Qual Saf Health Care. 2004;13:417–21. [PMC free article] [PubMed]

24. Morey JC, Simon R, Jay GD, et al. Error reduction and performance improvement in the emergency department through formal teamwork training: Evaluation results of the MedTeams project. Health Serv Res. 2002;37:1553–81. [PMC free article] [PubMed]

25. Thomas EJ, Taggart B, Crandell S, et al. Teaching teamwork during the Neonatal Resuscitation Program: A randomized trial. J Perinatol. 2007;27:409–414. [PubMed]

26. Hunt EA, Walker AR, Shaffner DH, et al. Simulation of in-hospital pediatric medical emergencies and cardiopulmonary arrests: Highlighting the importance of the first 5 minutes. Pediatrics. 2008;121:e34–e43. [PubMed]

27. Hunt EA, Fiedor-Hamilton M, Eppich WJ. Resuscitation education: Narrowing the gap between evidence-based resuscitation guidelines and performance using best educational practices. Pediatr Clin North Am. 2008;55:1025–50. [PubMed]

28. Niles D, Sutton RM, Donoghue A, et al. “Rolling Refreshers”: A novel approach to maintain CPR psychomotor skill competence. Resuscitation. 2009;80:909–12. [PubMed]

29. Gilfoyle E, Gottesman R, Razack S. Development of a leadership skills workshop in paediatric advanced resuscitation. Medical Teacher. 2007;29:e276–83. [PubMed]

30. Langhan TS, Rigby IJ, Walker IW, et al. Simulation-based training in critical resuscitation procedures improves residents’ competence. CJEM. 2009;11:535–9. [PubMed]

Articles from Paediatrics & Child Health are provided here courtesy of
Pulsus Group