Development and Implementation of an Oral Sign-out Skills Curriculum

doi:10.1007/s11606-007-0331-0

J Gen Intern Med. 2007 October; 22(10): 1470–1474.

Published online 2007 August 3. doi: 10.1007/s11606-007-0331-0

PMCID: PMC2305855

Development and Implementation of an Oral Sign-out Skills Curriculum

Leora I. Horwitz, MD, MHS,^1,² Tannaz Moin, MD, MBA,² and Michael L. Green, MD, MSc²

¹Center for Outcomes Research and Evaluation, Yale-New Haven Hospital, New Haven, CT USA

²Department of Internal Medicine, Yale University School of Medicine, I.E.-61 SHM, P.O. Box 208088 New Haven, CT USA

Leora I. Horwitz, Phone: +1-203-7855469, Email: leora.horwitz/at/yale.edu.

Corresponding author.

Received February 2, 2007; Revised May 24, 2007; Accepted July 24, 2007.

This article has been cited by other articles in PMC.

Abstract

Introduction

Imperfect sign-out of patient information between providers has been shown to contribute to medical error, but there are no standardized curricula to teach sign-out skills. At our institution, we identified several deficiencies in skills and a lack of any existing training.

Aim

To develop a sign-out curriculum for medical house staff. Setting: Internal medicine residency program.

Program description

We developed a 1-h curriculum and implemented it in August of 2006 at three hospital sites. Teaching strategies included facilitated discussion, modeling, and observed individual practice with feedback. We emphasized interactive communication, a structured sign-out format summarized by an easy-to-remember mnemonic (“SIGNOUT”), consistent inclusion of key content items such as anticipatory guidance, and use of concrete language.

Program evaluation

We received 34 evaluations. The mean score for the course was 4.44

0.61 on a 1–5 scale. Perceived usefulness of the structured oral communication format was 4.46

0.78. Participants rated their comfort with providing oral sign-out significantly higher after the session than before (3.27

1.0 before vs. 3.94

0.90 after; p

.001).

Discussion

We developed an oral sign-out curriculum that was brief, structured, and well received by participants. Further study is necessary to determine the long-term impact of the curriculum.

KEY WORDS: medical student and residency education, communication skills, curriculum development/evaluation

INTRODUCTION

Communication failures figure in 25–67% of adverse events.¹^–⁵ Transfers of care in hospitalized patients are particularly vulnerable to communication failures during the sign-out of clinical information between providers.⁶^–¹¹ Consequently, the Joint Commission on Accreditation of Healthcare Organizations now mandates a standardized approach to hand-offs of patient care.¹² In addition, the Accreditation Council on Graduate Medical Education requires that residents be proficient in communication skills that “result in effective information exchange and teaming with patients, their patients’ families, and professional associates.”¹³

At present, however, residency sign-out systems are not standardized, and skills training is rare.¹⁴^,¹⁵ Furthermore, there are no publicly available or published curricula for improving physicians’ sign-out skills, and there are no universally accepted standards for hand-offs of patient care among physicians, though some groups have recently issued recommendations for best practices.¹⁶^–¹⁸

To address these gaps, we developed a brief, structured sign-out curriculum for Internal Medicine house staff. In this paper we describe the curriculum development, instructional strategies, and initial implementation experience and present our plans for further expansion and evaluation of the curriculum.

PROGRAM DEVELOPMENT AND DESCRIPTION

Needs Assessment

We followed a six-step model in developing our curriculum.¹⁹ First, we conducted an in-depth local needs assessment. We informally surveyed house staff and discovered that none had had any training in sign-out skills during medical school. We also examined our internal practices and found that there were no institutional standards as to how and when hand-offs were to take place, no requirements as to who participated in the hand-off, and no consistent method of conveying information orally or in written form. In addition, although ward attendings were urged to evaluate residents’ sign-out skills, they were given no formal guidance or evaluative criteria.

To better understand existing practices, we audiotaped sign-outs and interviewed post-call and night float house staff about the previous days’ sign-outs. From these direct observations, we concluded that the quality of written sign-out was dependent largely on the technological support available to residents, whereas the quality of verbal sign-outs was more variable and appeared driven primarily by the skill set of individual participants. We therefore decided to focus our curriculum on oral sign-out skills and to direct efforts to improve written sign-out at systems issues, such as developing standardized templates.

Literature Review

First, we attempted to identify extant sign-out curricula by searching MEDLINE and numerous curricular web sites.²⁰^–²⁷ While we identified an extensive literature on physician–patient communication, we were unable to locate any curriculum specifically for teaching physician–physician communication skills. We therefore determined it would be necessary to develop our own.

Next we conducted an additional literature search, seeking to identify explicit processes that improve hand-offs. We reviewed effectiveness studies and recommendations from recognized authorities.¹⁶^–¹⁸^,²⁸^–³⁶ We also included articles discussing hand-offs in other high-reliability fields.³⁷^–⁴¹ Several studies suggested that a quiet setting, structured format, and opportunity for interaction represent key elements of successful sign-outs.¹⁷^,¹⁸^,²⁸^,⁴²^,⁴³ From our direct observations we also noted the importance of concrete language, description of the clinical condition of the patients, anticipatory guidance, and clear plans and rationales for all assigned tasks. These elements became highlights of our curriculum.

Curriculum Description

Although we considered incorporating the curriculum into the intern orientation before the start of residency, we rejected this idea because orientation was already very full and we felt the curriculum would be most useful to interns who had already had some experience with sign-out. We decided on a noon conference format, with identical sessions to take place in the summer at all three of the hospitals through which our interns rotate. This allowed us to reach interns early enough for the workshop to be useful but well enough into internship for them to have had some practical experience with sign-out and cross coverage. We also designed a pocket card, placed a structured oral sign-out format on the residency web site, and added a section on sign-out to the “Intern Survival Guide” each intern receives on entering the program.

The goals of the curriculum were to sensitize interns to the concept of sign-out as an important contributor to patient safety and to improve their skills in providing sign-out. To achieve these goals we developed several specific curricular objectives, covering cognitive, affective, and psychomotor domains (Table 1). Based on these objectives, we determined that a combination of instructional strategies would be necessary and, therefore, structured the workshop in two parts.

Table 1

Curriculum Objectives

First, we designed a 30-min interactive large-group discussion facilitated by a study investigator (L.I.H.). We covered five topics: the importance of sign-out and consequences of poor sign-out; the ideal content of written and oral sign-outs; the differences between oral and written sign-out aims and techniques; the SBAR (situation, background, assessment, recommendation) format for communication about emergent situations;⁴⁴ and a new structured form of oral sign-out.

Our insistence on a formal approach to oral sign-out represents the most novel part of the curriculum. There is no uniform standard for sign-out format akin to that of the history and physical, making it difficult for interns to consistently include all desirable content. We developed a standardized format modeled on SBAR that includes all the essential elements of an oral sign-out in an easy-to-remember order: SIGNOUT (Table 2). The facilitator demonstrated the use of this format by modeling a sign-out for the large group.

Table 2

SIGNOUT Format for Oral Communication

In the second half of the workshop senior residents were dismissed to provide a low-key atmosphere for the remaining interns and students. Remaining participants were divided into small groups, with each facilitated by a study investigator (L.I.H.) or a chief resident. Each participant took it in turn to sign out the first patient on his own patient list to the person on his right while the remainder of the group listened. Immediately following, the small group offered feedback for both the provider and receiver of the sign-out, including discussion of structure, clarity, information that should have been included or omitted, and additional questions that could have been asked. The next intern then signed out, until each intern had had the opportunity both to present and receive an oral sign-out. Learners thus each had the opportunity for real-time practice, evaluation, and feedback as an integral part of the seminar. The small group part of the session lasted about 20 min.

Evaluation

At the conclusion of the workshop, participants were asked to complete an evaluation form, which assessed satisfaction with the course, attitude towards sign-out importance, and retrospective pre–post comfort with providing sign-out. The simultaneous self-assessment of pre and post skills at the conclusion of the workshop avoids response-shift bias, in which participants’ internal standard changes as the result of the educational intervention itself.⁴⁵ This method has been shown to be more consistent with independent skills assessments than prospective pre to post self-evaluation.⁴⁶ Pre–post comfort levels were compared with a paired t test using SAS 9.1.2 (SAS Institute, Cary, NC, USA).

The Human Investigation Committee at Yale-New Haven Hospital and the Institutional Review Board at the West Haven VA Hospital reviewed and approved the project and granted an exemption of signed informed consent to maintain anonymity of participants.

PROGRAM IMPLEMENTATION AND ASSESSMENT

We conducted an hour-long sign-out session in August at each of the three hospitals through which interns rotate. Thirty-four evaluations were completed, 14 by interns, 14 by students, and 6 by others. We did not take attendance, but we received evaluations from 92% of interns who could have been at the conference at two sites. At the third site, attendance was very poor and we received only two intern evaluations (15% of eligible). Because evaluation forms were distributed at the conclusion of the workshop, we received very few evaluations from residents, who left partway through the session.

The overall mean score given to the course on a 1-to-5 scale, where 5 is best, was 4.44

0.61. Mean score for the large group session was 4.35

0.73; for the small group session it was 4.55

0.71. The course was rated similarly by interns and students (4.50

0.52 vs. 4.33

0.72; p

0.49). All participants rated sign-out as important or very important to patient care (mean score 4.88

0.33) and the SIGNOUT mnemonic as likely to be useful or very useful (mean score 4.46

0.78). The SBAR format rated a lower mean score of 4.18

0.83. Perceived comfort at providing sign-out increased significantly (3.27

1.0 before vs. 3.94

0.90 after; p

0.001). Interns as a group reported higher pre and post scores than students, but their improvement in subjective comfort with sign-out was still significant (3.57

0.65 vs. 4.21

0.58; p

0.002).

During the small group session, interns tended to ramble rather than keep to a concise, structured format, but they recognized the deficiency. While they made efforts to provide anticipatory guidance, they did not always recognize when it was needed. In addition, despite repeated feedback after each subsequent sign-out, it was challenging to get interns to read back instructions, as has been recommended by JCAHO. This is a completely novel form of communication for most house staff, and their discomfort was evident.

DISCUSSION

We developed a brief, formal curriculum to teach sign-out skills to interns that incorporated a variety of instructional strategies, including discussion, modeling, individual practice, evaluation, and feedback. Overall satisfaction with the course was high and participants reported improved comfort with sign-out skills at the conclusion of the workshop.

The most common suggestion for improving the curriculum was to offer it even earlier in the academic year. One alternative would be to present the curriculum at the June intern orientation. However, this would make real-time practice with actual patients impossible. In addition, interns who have not yet done any night-time cross-coverage might have less appreciation for the types of problems commonly encountered overnight that can be avoided with good sign-out. In fact, we found that interns had difficulty with the anticipatory guidance part of sign-out, although they had already had some experience on the wards.

We have several plans for further expansion and evaluation of the curriculum. We are now developing a sign-out evaluation tool so that residents can provide evaluation and feedback for their interns throughout the year. This tool will include areas of deficiency noted during the small group sessions, such as structured format, anticipatory guidance, and read-back. In addition, we plan to repeat our direct observation of house staff sign-outs to determine the curriculum’s effect on actual sign-out practices. Because of the inherent steep learning curve of the intern year, we cannot compare practices before and after the curriculum in the intervention group itself. Rather, we will observe practices in the intervention group in the spring and compare them to those of a previous cohort of interns who did not receive any sign-out education or evaluation. Finally, based on feedback from medical student participants, we are exploring the possibility of developing a similar curriculum for the medical school.

In the next iteration, we will also address two challenges that we encountered during the first implementation. First, several participants were on outpatient or elective rotations and were disappointed not to have the opportunity to practice giving sign-out. We have developed written sign-outs for use by participants who do not have their own patients so that all attendees can participate in the small group session. Second, despite presenting the curriculum at each of the hospitals through which our house staff rotate, we only reached approximately a quarter of all interns because those who were post-call or on vacation or elective and intensive care rotations were not available, and the new ward interns had difficulty making time to attend educational conferences. Next time, therefore, we will offer the workshop at each participating hospital during two different rotations.

In summary, we developed a formal sign-out skills curriculum incorporating a variety of teaching techniques; a novel structured sign-out format; and real-time practice, evaluation, and feedback for all participants. We intend this to be one step in improving residents’ communication competence and reducing medical errors from poor sign-out.

Acknowledgements

Dr. Horwitz was supported by the Department of Veterans Affairs at the time this study was conducted. We thank Cyrus Kapadia, M.D., and Stephen J. Huot, M.D., Ph.D., directors of the categorical and primary care Internal Medicine residency programs, respectively, for their cooperation in implementing this curriculum. We also thank chief residents Francis W. Chan, M.D.; Caleb Moore, M.D.; Juliette Spelman, M.D.; and Matthew Stopper, M.D., for their assistance during the small-group sessions.

Conflict of Interest None disclosed.

References

1. Joint Commission on Accreditation of Healthcare Organizations. Root Causes of Sentinel Events. http://www.jointcommission.org/SentinelEvents/Statistics/. Accessed January 20, 2007.

2. Jagsi R, Kitch BT, Weinstein DF, Campbell EG, Hutter M, Weissman JS. Residents report on adverse events and their causes. Arch Intern Med. 2005;165:2607–13. [PubMed]

3. Lofgren RP, Gottlieb D, Williams RA, Rich EC. Post-call transfer of resident responsibility: its effect on patient care. J Gen Intern Med. 1990;5:501–5. [PubMed]

4. Moore C, Wisnivesky J, Williams S, McGinn T. Medical errors related to discontinuity of care from an inpatient to an outpatient setting. J Gen Intern Med. 2003;18:646–51. [PMC free article] [PubMed]

5. Petersen LA, Brennan TA, O’Neil AC, Cook EF, Lee TH. Does housestaff discontinuity of care increase the risk for preventable adverse events? Ann Intern Med. 1994;121:866–72. [PubMed]

6. Beach C. Lost in Transition. AHRQ WebM&M (serial online), February, 2006. http://webmm.ahrq.gov/case.aspx?caseID=116. Accessed January 20, 2007.

7. Beach C, Croskerry P, Shapiro M. Profiles in patient safety: emergency care transitions. Acad Emerg Med. 2003;10:364–7. [PubMed]

8. Chassin MR, Becher EC. The wrong patient. Ann Intern Med. 2002;136:826–33. [PubMed]

9. Gandhi TK. Fumbled handoffs: one dropped ball after another. Ann Intern Med. 2005;142:352–8. [PubMed]

10. Mukherjee S. A precarious exchange. N Engl J Med. 2004;351:1822–4. [PubMed]

11. Vidyarthi A. Fumbled Handoff. March, 2004. http://webmm.ahrq.gov/case.aspx?caseID=55. Accessed January 20, 2006.

12. Joint Commission on Accreditation of Healthcare Organizations. 2006 Critical Access Hospital and Hospital National Patient Safety Goals. http://www.jointcommission.org/PatientSafety/NationalPatientSafetyGoals/06_npsg_cah.htm). Accessed October 8, 2005.

13. Accreditation Council for Graduate Medical Education. Advancing Education in Interpersonal and Communication Skills: An educational resource from the ACGME Outcome Project. http://www.acgme.org/outcome/implement/interperComSkills.pdf. Accessed January 20, 2007.

14. Horwitz LI, Krumholz HM, Green ML, Huot SJ. Transfers of patient care between house staff on internal medicine wards: a national survey. Arch Intern Med. 2006;166:1173–7. [PubMed]

15. Sinha M, Shriki J, Salness R, Blackburn PA. Need for standardized sign-out in the emergency department: a survey of emergency medicine residency and pediatric emergency medicine fellowship program directors. Acad Emerg Med. 2006.

16. Joint Commission on Accreditation of Healthcare Organizations. Patient Safety Practices related to Patient Safety Goal 2E. http://www.jcipatientsafety.org/22791/). Accessed January 20, 2007.

17. British Medical Association, National Patient Safety Agency, NHS Modernisation Agency. Safe Handover: Safe Patients. London: British Medical Association; 2004.

18. University HealthSystem Consortium. UHC Best Practice Recommendation: Patient Hand Off Communication: White Paper. Oak Brook, IL: UHC; 2006.

19. Kern DE, Thomas PA, Howard DM, Bass EB. Curriculum Development for Medical Education: A Six-Step Approach. Baltimore, MD: The Johns Hopkins University Press; 1998.

20. Institute of Education Sciences. Education Resources Information Center. http://www.eric.ed.gov/ERICWebPortal/Home.portal. Accessed January 20, 2007.

21. American Association of Medical Colleges. AAMC Curriculum Management and Information Tool (CurrMIT©). http://services.aamc.org/currdir/start.cfm). Accessed January 20, 2007.

22. American Association of Medical Colleges. MedEdPORTAL: Providing Online Resources to Advanced Learning in Medical Education. http://services.aamc.org/jsp/mededportal/goLinkPage.do?link=home). Accessed January 20, 2007.

23. Alliance for Academic Internal Medicine. Educational Tools: Curricula. http://www.im.org/AAIM/Tools/Curricula.htm). Accessed January 20, 2007.

24. Council on Medical Student Education in Pediatrics. APA/COMSEP General Pediatric Clerkship Curriculum. http://www.comsep.org/Curriculum/). Accessed January 20, 2007.

25. Josiah Macy Jr. Foundation. http://www.josiahmacyfoundation.org/). Accessed January 20, 2007.

26. Thomas PA, Kern DE. Internet resources for curriculum development in medical education: an annotated bibliography. J Gen Intern Med. 2004;19:599–605. [PMC free article] [PubMed]

27. Green ML. Identifying, appraising, and implementing medical education curricula: a guide for medical educators. Ann Intern Med. 2001;135:889–96. [PubMed]

28. Arora V, Johnson J, Lovinger D, Humphrey HJ, Meltzer DO. Communication failures in patient sign-out and suggestions for improvement: a critical incident analysis. Qual Saf Health Care. 2005;14:401–7. [PMC free article] [PubMed]

29. Hanna D, Griswold P, Leape LL, Bates DW. Communicating critical test results: safe practice recommendations. Jt Comm J Qual Patient Saf. 2005;31:68–80. [PubMed]

30. Lee LH, Levine JA, Schultz HJ. Utility of a standardized sign-out card for new medical interns. J Gen Intern Med. 1996;11:753–5. [PubMed]

31. Marill KA, Gauharou ES, Nelson BK, Peterson MA, Curtis RL, Gonzalez MR. Prospective, randomized trial of template-assisted versus undirected written recording of physician records in the emergency department. Ann Emerg Med. 1999;33:500–9. [PubMed]

32. Parker J, Coiera E. Improving clinical communication: a view from psychology. J Am Med Inform Assoc. 2000;7:453–61. [PMC free article] [PubMed]

33. Petersen LA, Orav EJ, Teich JM, O’Neil AC, Brennan TA. Using a computerized sign-out program to improve continuity of inpatient care and prevent adverse events. Jt Comm J Qual Improv. 1998;24:77–87. [PubMed]

34. Ram R, Block B. Signing out patients for off-hours coverage: comparison of manual and computer-aided methods. Proc Annu Symp Comput Appl Med Care. 1992:114–8.

35. Van Eaton EG, Horvath KD, Lober WB, Rossini AJ, Pellegrini CA. A randomized, controlled trial evaluating the impact of a computerized rounding and sign-out system on continuity of care and resident work hours. J Am Coll Surg. 2005;200:538–45. [PubMed]

36. Vidyarthi AR, Arora V, Schnipper JL, Wall SD, Wachter RM. Managing discontinuity in academic medical centers: Strategies for a safe and effective resident sign-out. J Hosp Med. 2006;1:257–266. [PubMed]

37. Helmreich RL. On error management: lessons from aviation. BMJ. 2000;320:781–5. [PMC free article] [PubMed]

38. Burke CS, Salas E, Wilson-Donnelly K, Priest H. How to turn a team of experts into an expert medical team: guidance from the aviation and military communities. Qual Saf Health Care. 2004;13(Suppl 1):i96–i104. [PMC free article] [PubMed]

39. Carvalho PV, Dos Santos IL, Vidal MC. Safety implications of cultural and cognitive issues in nuclear power plant operation. Appl Ergon. 2006;37:211–23. [PubMed]

40. Patterson ES, Roth EM, Woods DD, Chow R, Gomes JO. Handoff strategies in settings with high consequences for failure: lessons for health care operations. Int J Qual Health Care. 2004;16:125–32. [PubMed]

41. Patterson ES, Woods DD. Shift changes, updates, and the on-call architecture in space shuttle mission control. Comput Support Coop Work. 2001;10:317–46. [PubMed]

42. National Coordinating Council for Medication Error Reporting and Prevention. Recommendations to Reduce Medication Errors Associated with Verbal Medication Orders and Prescriptions: Adopted February 20, 2001. http://www.nccmerp.org/council/council2001-02-20.html). Accessed January 20, 2007.

43. Volpp KG, Grande D. Residents’ suggestions for reducing errors in teaching hospitals. N Engl J Med. 2003;348:851–5. [PubMed]

44. Haig KM, Sutton S, Whittington J. SBAR: a shared mental model for improving communication between clinicians. Jt Comm J Qual Patient Saf. 2006;32:167–75. [PubMed]

45. Howard GS, Dailey PR. Response-shift bias: A source of contamination of self-report measures. J Appl Psychol. 1979;64:144–50.

46. Skeff KM, Stratos GA, Bergen MR. Evaluation of a medical faculty development program: a comparison of traditional pre/post and retrospective pre/post self-assessment ratings. Eval Health Prof. 1992;15:350–66.

Articles from Journal of General Internal Medicine are provided here courtesy of
Society of General Internal Medicine