PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of jgmeLink to Publisher's site
 
J Grad Med Educ. Jun 2012; 4(2): 170–175.
PMCID: PMC3399608
A Randomized Controlled Trial of the Impact of a Teaching Procedure Service on the Training of Internal Medicine Residents
Michelle Mourad, MD, Sumant Ranji, MD, and Diane Sliwka, MD
Corresponding author: Michelle Mourad, MD, 505 Parnassus Avenue, M1287, Box 0131, San Francisco, CA 94143, 415.476.2264, Michelle.Mourad/at/ucsf.edu
Received June 18, 2011; Revised October 16, 2011; Revised November 25, 2011; Accepted November 29, 2011.
Introduction
Academic medical centers must provide safe inpatient procedures while balancing resident autonomy and education. We performed a randomized, controlled trial to evaluate the effect of a 2-week hospitalist procedure service (HPS) rotation on interns' self-perceived procedure ability, knowledge, and autonomy versus the standard curriculum.
Methods
We randomly selected 16 of 57 internal medicine interns (28%) to participate in the intervention group rotation, with 29 interns in the control group. All interns were surveyed before the start of residency and at the end of the postgraduate year-1 (PGY-1) and PGY-2 years to evaluate self-reported knowledge and ability to (1) safely perform procedures, (2) supervise procedures, and (3) use bedside ultrasound.
Results
Ninety-four percent of HPS interns (15/16) and 71% of control interns (29/41) completed all surveys. Baseline knowledge and experience did not differ significantly between the groups. The intervention group performed significantly more paracentesis (9 versus 4; P < .001), thoracentesis (6 versus 2; P < .001), and lumbar puncture (4 versus 3; P < .001) procedures than did the control group. After their first year, residents who completed the HPS rotation rated their ability to safely perform and supervise all of the assessed procedures as higher (P < .05 for all procedures) and were more likely to rate self-perceived knowledge as very good or excellent in all surveyed aspects of procedure performance (P < .05).
Discussion
A 2-week hospitalist-supervised procedure service rotation substantially improved residents' experience, confidence, and knowledge in performing bedside procedures early in their training, with this effect sustained through the PGY-2 year. Standardized procedure service rotations are a viable solution for programs seeking to improve their procedure-based education.
What was known
Residents early in their training are expected to perform and supervise bedside procedures and have indicated a lack of confidence and desire for additional formal training.
What is new
A randomized trial of a 2-week procedural training rotation for interns with intensive didactics and added experience found higher self-reported confidence in the intervention group in performing and supervising procedures and use of ultrasound.
Limitations
Single-site, single-specialty study, with self-reported, mostly subjective data (some data on complications were collected).
Bottom line
A hospitalist-supervised procedure service rotation substantially improved interns' experience, confidence, and knowledge in performing and supervising bedside procedures, with this benefit sustained well into the PGY-2 year.
A 1989 study1 reported that internal medicine residents failed to master routine procedures and made recommendations for increased training, supervision, and competency assessment. Since that time, residents have continued to report discomfort with procedural skills, as well as a desire for increased training.2,5 Certification requirements currently exclude performance and volume-based metrics, requiring only that residents, “recognize indications, contraindications and manage complications” of selected bedside procedures.6 The literature suggests that residents routinely perform procedures,5,7,8 and frequently do this with uncomfortable or underqualified peer supervisors.5,9
With increased attention on patient safety and resident work hours, faculty-staffed procedure services have been developed to handle common bedside procedures, such as paracentesis, thoracentesis, lumbar puncture, arthrocentesis, and central line insertion.7,10 Such a service can simultaneously provide residents with procedural training and offer a possible solution to balance quality procedural performance with resident education. Most programs to date have been for postgraduate year-2 (PGY-2) or PGY-3 residents,2,3,5 despite procedural expectations for residents earlier in training.5,9
To better address procedural training early in residency, we created a comprehensive hospitalist procedure service (HPS) with a 2-week rotation for internal medicine interns. We performed a randomized, controlled trial to evaluate the effect of participation in the HPS rotation on interns' longitudinal, self-perceived procedure ability and knowledge as compared with the standard curriculum.
Site and Participants
Participants included interns at a university-based, academic medical center in the 2008–2009 academic year. The intervention group comprised 16 of 57 randomly selected internal medicine interns (28%) who participated in a 2-week procedure service rotation at the 560-bed university hospital setting (1 of 3 hospital sites in the program). Interns in both the intervention group and control group who did not complete all of the surveys were excluded from the study (n  = 13; 23%). The study was reviewed and approved by our Institutional Review Board, The Committee on Human Research.
Procedure Service
The HPS began offering procedure services for paracentesis, thoracentesis, and lumbar puncture in November 2008. Patients were voluntarily referred to this service. Interns rotating on the HPS performed referred procedures during a 2-week rotation; all procedures were supervised by an attending hospitalist with additional ultrasound and procedural training. Procedures not referred to the HPS were independently performed by residents on the primary team or referred to interventional radiology. Additionally, residents and interns continued to perform procedures independently at the 2 other hospital sites. A detailed description of the structure of the service and faculty training has been described previously.11
Resident Education
We employed a best-practices technique to teach interns, which included repetitive practice to develop proper technique, real-time ultrasonographic guidance, direct supervision by expert faculty, and a team-based approach. All interns received the standard procedure curricular instruction: a half-day simulation course dedicated to providing hands-on instruction with ultrasound technique, central venous catheter insertion, thoracentesis, and lumbar puncture performance. In addition, the intervention group participated in 10 didactic sessions that included procedure safety, informed consent, infection control and documentation practices, ultrasound basics, and in-depth, procedure-based lectures reviewing indications and contraindications, procedure preparation, anatomy, technique, troubleshooting, management of complications, and follow-up studies and their interpretation. These sessions were supplemented with independent readings and 3 evidence-based, interactive computer modules reviewing best practices for each procedure. Each intern was required to demonstrate procedure technique using step-by-step hands-on simulation with the procedure kit before performance in a clinical setting.
Survey Development and Distribution
The 3 survey instruments for this study were developed by the HPS faculty in conjunction with divisional research leadership and the program director and were reviewed by a committee of local survey-design experts. The survey tools were pilot-tested by a group of attending physicians, residents, and the investigative team but were not formally validated.
The initial PGY-1 survey captured incoming interns' experience, knowledge, and confidence in performing common, invasive, bedside procedures listed in the American Board of Internal Medicine's Policies and Procedures for Certification.6 The survey also asked about baseline experience and confidence in observing, performing, and supervising procedures as well as interns' confidence in using ultrasound for procedure guidance. Interns specifically rated their confidence to (1) safely perform specific bedside procedures, and (2) safely supervise others in conducting procedures. The surveys at the end of their PGY-1 and PGY-2 year captured changes in experience and knowledge over time and self-perceived procedure knowledge on a range of topics, including indications and contraindications, landmarks, use of procedure-specific ultrasound, evidence-based procedure technique, troubleshooting, and management of complications. Questions regarding central-line performance served as a check item because that procedure was not taught or performed by the HPS.
Surveys were distributed via e-mail in July 2008 and 2009 and in June 2010. Up to 3 reminder e-mails were sent to nonrespondents.
Analysis
We analyzed baseline and yearly differences between the intervention and nonintervention group using χ2 tests for categoric and nominal variables. Differences in Likert scale ratings for “confidence” to perform and supervise procedures were compared with Student t tests. Residents were considered to have self-perceived knowledge of a subject area if they rated their knowledge as 4 (very good) or 5 (excellent). Knowledge was compared between intervention and control groups using Fisher exact tests. Multivariate logistic regression was used to analyze the contribution of age, sex, future subspecialty, and procedure experience.
Reported complications and those found in patients' medical charts were classified into major and minor based on reported definitions in the literature.12,15 Major complications included bleeding requiring transfusion, pneumothorax requiring a chest tube, respiratory failure, bowel perforation, cerebral herniation or shock, cerebrospinal fluid leak requiring intervention, and transfer to a higher level of care. For patients receiving thoracentesis, the patient's medical record was reviewed to check for a follow-up chest x-ray, the presence of a pneumothorax, or clinical evidence for re-expansion pulmonary edema.
Demographics of Respondents
Ninety-four percent (15/16) of HPS interns and 71% of controls (29/41) completed all 3 surveys. Baseline self-reported experience and confidence was low and not significantly different between the intervention and the control groups (table 1).
TABLE 1
TABLE 1
Baseline Demographics of Study Participants
Procedure Volume
In its first 8 months, the 16 interns on the procedure service performed 265 supervised procedures (122 paracenteses, 77 thoracenteses, and 66 lumbar punctures). Interns performed an average of 16 procedures per intern during the 2-week period. During the 8 months, the intervention group performed significantly more paracenteses (median, 9 versus 4; P < .001), thoracenteses (median, 6 versus 2; P < .001), and lumbar punctures (median, 4 versus 3; P < .001) than did the control group. The number of central lines performed did not differ between the 2 groups (median, 2 versus 3; P  =  .83), and the volume of procedures performed by the control group was similar to previously published data.9
Procedure Performance
Interns who completed the rotation rated their ability to safely perform each of the assessed procedures higher than did the controls (P < .036 for paracentesis, thoracentesis, lumbar puncture; table 2). For central lines (not taught or performed by the HPS), there was no difference between the intervention and control groups. The HPS interns were also more confident in their use of ultrasound (P < .001) and were more likely to use bedside ultrasound for paracentesis and thoracentesis, and many of those gains persisted through the end of the PGY-2 year (P < .041; table 3). The largest and most sustained gains in confidence between the intervention and control groups were seen in thoracentesis confidence. In a multivariate analysis of procedure confidence adjusting for age, sex, future invasive subspecialty, and procedure volume at the end of the PGY-1, the number of procedures performed was a significant predictor for confidence performing thoracentesis, lumbar puncture, and central line insertion.
TABLE 2
TABLE 2
Resident Confidence in Performing Bedside, Invasive Procedures and Use of Ultrasound
TABLE 3
TABLE 3
Resident Report of Likelihood of Bedside Ultrasound Use
Procedure Supervision
Interns rotating on the HPS reported significantly higher levels of confidence in safely supervising bedside invasive procedures offered by the service (P < .023; table 4), and these gains persisted until the end of the PGY-2 year. In a multivariate analysis of procedure confidence adjusting for age, sex, future invasive subspecialty, and procedure volume at the end of the PGY-1 year, the number of procedures performed was a significant predictor for confidence supervising thoracentesis, lumbar puncture, and central line insertion.
TABLE 4
TABLE 4
Resident Confidence in Supervising Bedside, Invasive Procedures
Self-Rated Knowledge
The HPS interns were more likely to rate self-perceived knowledge as very good or excellent in all surveyed aspects of procedure performance, including indications and contraindications, landmarks, procedure-specific ultrasound use, evidence-based procedure technique, troubleshooting, and management of complications (P < .022 for all). Significant relative gains in all knowledge-based areas persisted until at least the end of PGY-2 year (P < .043 for all).
Complications
Of the 265 procedures performed by the interns during the initial year of the service, 4 patients (1.5%) had a major complication. All were patients who received a paracentesis and required transfusion by their primary team on the same day as the procedure; none required further intervention. Of the 77 patients that had a thoracentesis performed, all (100%) had a follow-up chest x-ray, and none (0%) suffered an iatrogenic pneumothorax or re-expansion pulmonary edema.
Feasibility
A 0.72 full-time equivalent faculty member was devoted to the training, education, and supervision of interns on the procedure service. In addition, the medical center purchased an ultrasound for the use of the service and provided procedure kits and training supplies, which were a significant expense. Training for faculty was provided by colleagues in interventional radiology and the emergency department at no cost. At the end of our first year, the service averaged almost 3 procedures per day, which represents a third of procedures needed for a self-supporting service. We estimate that we would need to perform at least 4 therapeutic thoracentesis or, at most, 9 lumbar punctures per day to be self-supporting. If a mix of procedures were performed, the service would need to perform more than 6 procedures per day.
A 2-week rotation on the HPS provided interns with significantly more procedure experience and confidence to safely perform and supervise bedside, invasive procedures versus the standard curriculum. Benefits included a low rate of complications but sustained resident confidence, which persisted until the end of the academic year and, for many, until the completion of their PGY-2 year. The gains in confidence from the structure of the procedure service could not be separated from those resulting from the increased experience interns gained by participating in the HPS. These gains were sustained despite the relatively brief exposure to the HPS. The experience allows for repetitive performance in a well-supervised setting with dedicated feedback after each experience, which has been recommended as the “gold standard” for procedural education.17,18 Simulation has seen similar gains in procedure-related knowledge and comfort, although most have not demonstrated such a sustained response19,20 nor has increased confidence to supervise been assessed after simulation. Our emphasis on the modeling of supervision might also explain the sustained confidence in supervising, despite similar numbers of procedures performed and supervised during the interns' PGY-2 year. Although the intervention group may have overestimated their abilities because of having completed an intensive procedure-training program, the lack of confidence to perform or supervise procedures demonstrated by the control group remains troubling and likely represents residents' accurate assessment of their procedural deficits.
Residents routinely perform bedside procedures, often without adequate supervision, particularly during nights and weekends. Patient safety demands that residents be properly trained to perform and supervise bedside procedures. Residency programs and teaching hospitals should consider implementing programs like the HPS, which can provide supervised, dedicated procedural training that positively affects residents' comfort with performing and supervising procedures. The most notable gap between the 2 groups was in experience and confidence to perform and supervise thoracenteses.
Our study has several limitations. First, it was a single-site intervention, which limits generalizability. Second, the cost of our model could be an obstacle to wide adoption. Finally, data were self-reported, and the measures for procedure confidence and self-rated knowledge were subjective and collected using a nonvalidated survey instrument.
Conclusions
A 2-week hospitalist-supervised procedure service rotation substantially improved residents' experience, confidence, and knowledge in performing invasive bedside procedures. In addition, many residents who had completed the rotation also felt qualified to supervise these procedures throughout their training.
Footnotes
All authors are with the Department of Medicine, Division of Hospital Medicine, University of California, San Francisco. Michelle Mourad, MD, is Assistant Clinical Professor; Sumant Ranji, MD, is Associate Professor; and Diane Sliwka, MD, is Associate Professor.
Funding: The authors report no external funding source for this study.
A preliminary version of this article was previously presented as a poster at the Society of Hospital Medicine Conference in Washington, DC, in April 2010.
We would like to thank Andy Auerbach, MD, Patricia O'Sullivan, EdD, and the members of the Educational Scholars Committee and Incubator group for their contribution to study methodology, survey design, and data analysis.
1. Wigton RS, Blank LL, Nicolas JA, Tape TG. Procedural skills training in internal medicine residencies: a survey of program directors. Ann Intern Med. 1989;111(11):932–938. [PubMed]
2. Hicks CM, Gonzalez R, Morton MT, Gibbons RV, Wigton RS, Anderson RJ. Procedural experience and comfort level in internal medicine trainees. J Gen Intern Med. 2000;15(10):716–722. [PMC free article] [PubMed]
3. Smith CC, Gordon CE, Feller-Kopman D, Huang GC, Weingart SN, Davis RB, et al. Creation of an innovative inpatient medical procedure service and a method to evaluate house staff competency. J Gen Intern Med. 2004;19(5, pt 2):510–513. [PMC free article] [PubMed]
4. Huang GC, Smith CC, Gordon CE, Feller-Kopman DJ, Davis RB, Phillips RS, et al. Beyond the comfort zone: residents assess their comfort performing inpatient medical procedures. Am J Med. 2006;119(1):71.e17–71.e24. doi:10.1016/j.amjmed.2005.08.007. [PubMed]
5. Lenhard A, Moallem M, Marrie RA, Becker J, Garland A. An intervention to improve procedure education for internal medicine residents. J Gen Intern Med. 2008;23(3):288–293. [PMC free article] [PubMed]
6. American Board of Internal Medicine. Policies and procedures for certification, February 2011. http://www.abim.org/certification/policies/imss/im.aspx#procedures. Accessed May 23, 2011.
7. Lucas BP, Asbury JK, Wang Y, Lee K, Kumapley R, Mba B, et al. Impact of a bedside procedure service on general medicine inpatients: a firm-based trial. J Hosp Med. 2007;2(3):143–149. [PubMed]
8. Huang GC, Newman LR, Schwartzstein RM, Clardy PF, Feller-Kopman D, Irish JT, et al. Procedural competence in internal medicine residents: validity of a central venous catheter insertion assessment instrument. Acad Med. 2009;84(8):1127–1134. [PubMed]
9. Mourad M, Kohlwes J, Maselli J. MERN Group. Auerbach AD. Supervising the supervisors—procedural training and supervision in internal medicine residency. J Gen Intern Med. 2010;25(4):351–356. [PMC free article] [PubMed]
10. Ault MJ, Rosen BT. Proceduralists—leading patient-safety initiatives. N Engl J Med. 2007;356(17):1789–1790. [PubMed]
11. Mourad M, Auerbach A, Maselli J, Sliwka D. Patient satisfaction with a hospitalist procedure service: is bedside procedure teaching reassuring to patients. J Hosp Med. 2011;6(4):219–224. [PubMed]
12. Grogan DR, Irwin RS, Channick R, Raptopoulos V, Curley F. J., Bartter T, et al. Complications associated with thoracentesis: a prospective, randomized study comparing three different methods. Arch Intern Med. 1990;150(4):873–877. [PubMed]
13. De Gottardi A, Thévenot T, Spahr L, Morard I, Bresson-Hadni S, Torres F, et al. Risk of complications after abdominal paracentesis in cirrhotic patients: a prospective study. Clin Gastroenterol Hepatol. 2009;7(8):906–909. [PubMed]
14. Grabau CM, Crago SF, Hoff LK, Simon JA, Melton CA, Ott BJ, et al. Performance standards for therapeutic abdominal paracentesis. Hepatology. 2004;40(2):484–488. [PubMed]
15. Sempere AP, Berenguer-Ruiz L, Lezcano-Rodas M, Mira-Berenguer F, Waez M. Lumbar puncture: its indications, contraindications, complications and technique [in Spanish] Rev Neurol. 2007;45(7):433–436. [PubMed]
16. Grantcharov TP, Reznick RK. Teaching procedural skills. BMJ. 2008;336(7653):1129–1131. [PMC free article] [PubMed]
17. Rodriguez-Paz JM, Kennedy M, Salas E, Wu AW, Sexton JB, Hunt E. A, et al. Beyond “see one, do one, teach one”: toward a different training paradigm. Qual Saf Health Care. 2009;18(1):63–68. [PubMed]
18. Wayne DB, Barsuk JH, O'Leary KJ, Fudala MJ, McGaghie WC. Mastery learning of thoracentesis skills by internal medicine residents using simulation technology and deliberate practice. J Hosp Med. 2008;3(1):48–54. [PubMed]
19. Millington SJ, Wong RY, Kassen BO, Roberts JM, Ma IW. Improving internal medicine residents' performance, knowledge, and confidence in central venous catheterization using simulators. J Hosp Med. 2009;4(7):410–416. [PubMed]
Articles from Journal of Graduate Medical Education are provided here courtesy of
Accreditation Council for Graduate Medical Education