Background

Residents-as-teachers (RATs) programs have been shown to improve trainees' teaching skills, yet these decline over time.

Intervention

We adapted a commercial Web-based system to maintain resident teaching skills through reflection and deliberate practice and assessed the system's ability to (1) prevent deterioration of resident teaching skills and (2) provide information to improve residents' teaching skills and teaching program quality.

Methods

Ten first-year obstetrics-gynecology (Ob-Gyn) residents participated in a RATs program. Following the program, they used a commercial evaluation system to complete self-assessments of their teaching encounters with medical students. Students also evaluated the residents. To assess the system's effectiveness, we compared these residents to historical controls with an Objective Structured Teaching Examination (OSTE) and analyzed the ratings and the free text comments of residents and students to explore teaching challenges and improve the RATs program.

Results

The intervention group outscored the control group on the OSTE (mean score ± SD  =  81 ± 8 versus 74 ± 7; P  =  .05, using a 2-tailed Student t-test). Rating scale analysis showed resident self-assessments were consistently lower than student evaluations, with the difference reaching statistical significance in 3 of 6 skills (P < .05). Comments revealed that residents most valued using innovative teaching techniques, while students most valued a positive educational climate and interpersonal connections with residents. Recommended targets for RATs program improvement included teaching feedback, time-limited teaching, and modeling professionalism behaviors.

Conclusions

Our novel electronic Web-based reinforcement system shows promise in preventing deterioration of resident teaching skills learned during an Ob-Gyn RATs program. The system also was effective in gaining resident and student insights to improve RATs programs. Because our intervention was built upon a commercially available program, our approach could prove useful to the large population of current subscribers.

Background

The Accreditation Council for Graduate Medical Education (ACGME) requires pediatric residency programs to teach professionalism but does not provide concrete guidance for fulfilling these requirements. Individual programs, therefore, adopt their own methods for teaching and evaluating professionalism, and published research demonstrating how to satisfy the ACGME professionalism requirement is lacking.

Methods

We surveyed pediatric residency program directors in 2008 to explore the establishment of expectations for professional conduct, the educational experiences used to foster learning in professionalism, and the evaluation of professionalism.

Results

Surveys were completed by 96 of 189 program directors (51%). A majority reported that new interns attend a session during which expectations for professionalism are conveyed, either verbally (93%) or in writing (65%). However, most program directors reported that “None or Few” of their residents engaged in multiple educational experiences that could foster learning in professionalism. Despite the identification of professionalism as a core competency, a minority (28%) of programs had a written curriculum in ethics or professionalism. When evaluating professionalism, the most frequently used assessment strategies were rated as “very useful” by only a modest proportion (26%–54%) of respondents.

Conclusions

Few programs have written curricula in professionalism, and opportunities for experiential learning in professionalism may be limited. In addition, program directors express only moderate satisfaction with current strategies for evaluating professionalism that were available through 2008.

Introduction

Quality improvement (QI) activities are an important part of residency training. National studies are needed to inform best practices in QI training and experience for residents. The impact of the Institutional Review Board (IRB) process on such studies is not well described.

Methods

This observational study looked at time, length, comfort level, and overall quality of experience for 42 residency training programs in obtaining approval or exemption for a nationally based educational QI study.

Results

For the 42 programs in the study, the time period to IRB approval/exemption was highly variable, ranging from less than 1 week to 56.5 weeks; mean and median time was approximately 18 weeks (SD, 10.8). Greater reported comfort with the IRB process was associated with less time to obtain approval (r  =  −.50; P < .01; 95% CI, −0.70 to −0.23). A more positive overall quality of experience with the IRB process was also associated with less time to obtain IRB approval (r  =  −.60; P < .01; 95% CI, −0.74 to −0.36).

Discussion

The IRB process for residency programs initiating QI studies shows considerable variance that is not explained by attributes of the projects. New strategies are needed to assist and expedite IRB processes for QI research in educational settings and reduce interinstitutional variability and increase comfort level among educators with the IRB process.

Background

Efforts are underway to enhance learner input into the accreditation of educational programs, including residencies and fellowships.

Objectives

To aggregate the perspectives of residents and fellows from a cross-section of specialties to highlight common dimensions in learners' perceptions of strengths and opportunities for improvement (OFIs) in their program and to assess whether the ACGME Resident Survey captures areas important to residents' perceptions of their learning environment.

Results

The data set included 206 core and 193 subspecialty programs representing a wide range of specialties and subspecialties. Comments on strengths and OFIs addressed most of the items in the Resident Survey, with items not addressed in the survey also not represented in the ACGME requirements. The findings suggest that some program attributes are mentioned only in their absence (hygiene factors), whereas other dimensions (satisfiers), particularly quality and quantity of residents' interactions with faculty, procedural volume, and variety and didactic offerings, are critical to learners' perceptions of the quality of their learning environment. For some strengths, residents indicated their programs exceeded the ACGME standards, and for areas identified as OFIs, comments suggested programs were in compliance, but the residents desired more. Mentioned in this context were opportunities to perform research, access to board preparation courses and career counseling, and availability of new technology, including new patient care modalities.

Conclusions

The findings allow insight into program attributes important to residents' perceptions of their learning environment. Programs may find the results helpful in suggesting areas for improvement in their learning environment.

Background

New approaches to enhance access in primary care necessitate change in the model for residency education.

Purpose

To describe instrument design, development and testing, and data collection strategies for residency programs, continuity clinics, residents, and program graduates participating in the Preparing the Personal Physician for Practice (P4) project.

Methods

We developed and pilot-tested surveys to assess demographic characteristics of residents, clinical and operational features of the continuity clinics and educational programs, and attitudes about and implementation status of Patient Centered Medical Home (PCMH) characteristics. Surveys were administered annually to P4 residency programs since the project started in 2007. Descriptive statistics were used to profile data from the P4 baseline year.

Results

Most P4 residents were non-Hispanic white women (60.7%), married or partnered, attended medical school in the United States and were the first physicians in their families to attend medical school. Nearly 85% of residency continuity clinics were family health centers, and about 8% were federally qualified health centers. The most likely PCMH features in continuity clinics were having an electronic health record and having fully secure remote access available; both of which were found in more than 50% of continuity clinics. Approximately one-half of continuity clinics used the electronic health record for safety projects, and nearly 60% used it for quality-improvement projects.

Conclusions

We created a collaborative evaluation model in all 14 P4 residencies. Successful implementation of new surveys revealed important baseline features of residencies and residents that are pertinent to studying the effects of new training models for the PCMH.

Background

Paracentesis is a commonly performed bedside procedure that has the potential for serious complications. Therefore, simulation-based education for paracentesis is valuable for clinicians.

Objective

To assess internal medicine residents' procedural skills before and after simulation-based mastery learning on a paracentesis simulator.

Methods

A team with expertise in simulation and procedural skills developed and created a high fidelity, ultrasound-compatible paracentesis simulator. Fifty-eight first-year internal medicine residents completed a mastery learning-based intervention using the paracentesis simulator. Residents underwent baseline skill assessment (pretest) using a 25-item checklist. Residents completed a posttest after a 3-hour education session featuring a demonstration of the procedure, deliberate practice, ultrasound training, and feedback. All residents were expected to meet or exceed a minimum passing score (MPS) at posttest, the key feature of mastery learning. We compared pretest and posttest checklist scores to evaluate the effect of the educational intervention. Residents rated the training sessions.

Results

Residents' paracentesis skills improved from an average pretest score of 33.0% (SD  =  15.2%) to 92.7% (SD  =  5.4%) at posttest (P < .001). After the training intervention, all residents met or exceeded the MPS. The training sessions and realism of the simulation were rated highly by learners.

Conclusion

This study demonstrates the ability of a paracentesis simulator to significantly improve procedural competence.

Background

Transitional Year (TY) programs meet an important need by preparing residents for specialties that accept individuals after an initial preparatory year. To our knowledge, no surveys to date have been conducted to identify attributes of TY programs and concerns of TY program directors.

Purpose

The purpose of this study was to review TY program characteristics and identify critical issues and concerns of TY program directors (TYPDs).

Methods

A web-based, 22-question survey was sent to all 114 TYPDs of programs accredited by the Accreditation Council for Graduate Medical Education between January and April 2011. The survey included open-formatted and closed-formatted questions addressing program and institution demographics, program director time, administrative support, satisfaction, and future plans.

Results

The survey response rate was 86%. The median age of TY programs was 28 years, with few new programs. More than 80% of TY programs were conducted at community hospitals and university-affiliated community hospitals. Of the responding TYPDs, 17% had served less than 2 years, and 32% had served 10 years or more. Common sponsoring TY programs included internal medicine (88%), general surgery (42%), family medicine (25%), emergency medicine (24%), and pediatrics (18%). Overall, TYPDs were satisfied with their positions. They expressed concerns about inadequate time to complete duties, salary support, and administrative duties assigned to program coordinators. Forty-nine percent of TYPDs reported they planned to leave the position within the next 5 years.

Conclusions

Our survey provides useful information to assist institutions and the graduate medical education community in meeting the needs of TYPDs and strengthening TY programs.

Background

Residents and interns are recognized as important clinical teachers and mentors. Resident-as-teacher training programs are known to improve resident attitudes and perceptions toward teaching, as well as their theoretical knowledge, skills, and teaching behavior. The effect of resident-as-teacher programs on learning outcomes of medical students, however, remains unknown. An intervention cohort study was conducted to prospectively investigate the effects of a teacher-training workshop on teaching behavior of participating interns and on the clerkship learning outcomes of instructed fourth-year medical students.

Methods

The House Officer-as-Teacher Training Workshop was implemented in November 2009 over 1.5 days and attended by all 34 interns from one teaching hospital. Subsequently, between February and August 2010, 124 fourth-year medical students rated the observable teaching behavior of interns during 6-week general surgery clerkships at this intervention hospital as well as at 2 comparable hospitals serving as control sites. Ratings were collected using an anonymous 15-item Intern Clinical Teaching Effectiveness Instrument. Student achievement of clerkship learning outcomes during this period was evaluated using a validated and centralized objective structured clinical examination.

Results

Medical students completed 101 intern clinical teaching effectiveness instruments. Intern teaching behavior at the intervention hospital was found to be significantly more positive, compared with observed behavior at the control hospitals. Objective structured clinical examination results, however, did not demonstrate any significant intersite differences in student achievement of general surgery clerkship learning outcomes.

Conclusions

The House Officer-as-Teacher Training Workshop noticeably improved teaching behavior of surgical interns during general surgery clerkships. This improvement did not, however, translate into improved achievement of clerkship learning outcomes by medical students during the study period.

Background

High-quality, shift-to-shift handovers by residents are critical to ensuring to patient safety. The 2011 Accreditation Council for Graduate Medical Education duty hour requirements have increased the number of handovers occurring daily, necessitating new approaches to this challenge. Research suggests standardized approaches, electronic systems, and education programs can improve the handover process.

Methods

We conducted a 2-phase, observational study comparing an electronic handover system (experimental) in one clinical setting to a standard card-based system (control) at a second site. Outcome data included an objective assessment of the completeness and accuracy of handovers, and resident assessment of the handover systems. In phase 1, data were recorded at both sites and not shared with residents. In phase 2, data from the experimental system were used to provide standardized feedback to residents on the quality of their handovers.

Results

A total of 3184 individual patient sign-outs were evaluated during the 11-month period. Following introduction of a feedback intervention in the experimental arm, errors were present in only 5.2% of handovers, compared with 16.1% of controls (P < .001), and 67% of the 38 residents responding reported they perceived the experimental system as facilitating better patient care.

Conclusion

Regular, real-time feedback through an electronic handover system can improve the accuracy and completeness of handovers in patient care.

Introduction

Postrotation evaluations are frequently used by residency program directors for early detection of residents with academic difficulties; however, the accuracy of these evaluations in assessing resident performance has been questioned.

Methods

This retrospective case-control study examines the ability of postrotation evaluation characteristics to predict the need for remediation. We compared the evaluations of 17 residents who were placed on academic warning or probation, from 2000 to 2007, with those for a group of peers matched on sex, postgraduate year (PGY), and entering class.

Results

The presence of an outlier evaluation, the number of words written in the comments section, and the percentage of evaluations with negative or ambiguous comments were all associated with the need for remediation (P  =  .01, P  =  .001, P  =  .002, P  =  < .001, respectively). In contrast, United States Medical Licensing Examination step 1 and step 2 scores, total number of evaluations received, and percentage of positive comments on the evaluations were not associated with the need for remediation (P  =  .06, P  =  .87, P  =  .55, respectively).

Discussion

Despite ambiguous evaluation comments, the length and percentage of ambiguous or negative comments did indicate future need for remediation.

Conclusions

Our study demonstrates that postrotation evaluation characteristics can be used to identify residents as risk. However, larger prospective studies, encompassing multiple institutions, are needed to validate various evaluation methods in measuring resident performance and to accurately predict the need for remediation.

Background

Failures of communication during the transfer of patient care errors.

Methods

We created a new format for written sign-out material, based on aviation industry practice and cognitive psychology theory, designed to improve interns' and senior medical students' communication during transfers of patient care responsibility. We carried out a randomized, blinded, crossover trial, comparing a new, narrative, written sign-out report to a usual written sign-out. Thirty-two interns and fourth-year medical students rated their confidence across various clinical tasks and answered clinical questions regarding hypothetical patients presented to them in written, new, narrative sign-out compared with the customary format.

Results

There was no statistical difference in confidence when interns and senior medical students received usual versus narrative sign-outs.

Conclusions

Although a limited measure suggested some improvement in competence, the narrative format did not improve participants' self-rated confidence during patient-care transfer.

Background

The Screening, Brief Intervention and Referral to Treatment (SBIRT) service for unhealthy alcohol use has been shown to be one of the most cost-effective medical preventive services and has been associated with long-term reductions in alcohol use and health care utilization. Recent studies also indicate that SBIRT reduces illicit drug use. In 2008 and 2009, the Substance Abuse Mental Health Service Administration funded 17 grantees to develop and implement medical residency training programs that teach residents how to provide SBIRT services for individuals with alcohol and drug misuse conditions. This paper presents the curricular activities associated with this initiative.

Methods

We used an online survey delivery application (Qualtrics) to e-mail a survey instrument developed by the project directors of 4 SBIRT residency programs to each residency grantee's director. The survey included both quantitative and qualitative data.

Results

All 17 (100%) grantees responded. Respondents encompassed residency programs in emergency medicine, family medicine, pediatrics, obstetrics-gynecology, psychiatry, surgery, and preventive medicine. Thirteen of 17 (76%) grantee programs used both online and in-person approaches to deliver the curriculum. All 17 grantees incorporated motivational interviewing and validated screening instruments in the curriculum. As of June 2011, 2867 residents had been trained, and project directors reported all residents were incorporating SBIRT into their practices. Consistently mentioned challenges in implementing an SBIRT curriculum included finding time in residents' schedules for the modules and the need for trained faculty to verify resident competence.

Conclusions

The SBIRT initiative has resulted in rapid development of educational programs and a cohort of residents who utilize SBIRT in practice. Skills verification, program dissemination, and sustainability after grant funding ends remain ongoing challenges.