|Home | About | Journals | Submit | Contact Us | Français|
Research on the outcomes of pre-clinical curricula for clinical skills development is needed to assess their influence on medical student performance in clerkships.
To better understand the impact of a clinical-skills curriculum in the pre-clinical setting on student performance.
We conducted a non-randomized, retrospective, pre-post review of student performance evaluations from 3rd-year clerkships, before and after implementation of a clinical-skills curriculum, the Colleges (2001–2007).
Comparisons of clerkship performance data revealed statistically significant differences favoring the post-Colleges group in the Internal Medicine clerkship for 9 of 12 clinical-skills domains, including Technical Communication Skills (p<0.023, effect size 0.16), Procedural Skills (p<0.031, effect size 0.17), Communication Skills (p<0.003, effect size 0.21), Patient Relationships (p<0.003, effect size 0.21), Professional Relationships (p<0.021, effect size 0.17), Educational Attitudes (p<0.001, effect size 0.24), Initiative and Interest (p<0.032, effect size 0.15), Attendance and Participation (p<0.007, effect size 0.19), and Dependability (p<0.008, effect size 0.19). Statistically significant differences were identified favoring the post-Colleges group in technical communication skills for three of six basic clerkships (Internal Medicine, Surgery, and Pediatrics).
Implementation of a pre-clinical fundamental skills curriculum appears to be associated with improved clerkship performance in the 3rd year of medical school, particularly in the Internal Medicine clerkship. Similar curricula, focused on teaching clinical skills in small groups at the bedside with personalized mentoring from faculty members, may improve student performance. Continued efforts are needed to understand how to best prepare students for clinical clerkships and how to evaluate outcomes of similar pre-clinical skills programs.
Teaching fundamental clinical skills should be a priority in medical school curricula, in keeping with the importance of mastering clinical skills for becoming a physician.1–8 However, there is a paucity of curricula devoted to formal clinical skills instruction in US medical schools.9–19 As academic faculty increasingly focus on their own clinical and research efforts, declining resources support bedside teaching and clinical skills development for medical students.20–25
In response, the University of Washington School of Medicine (UWSOM) in 2001 implemented a competency-based curriculum (the Colleges program) designed to teach fundamental clinical skills at the bedside to 2nd-year medical students; this program has been described previously.26 Published outcomes of similar teaching models are scarce. A previous study of this UWSOM curricular model suggests that early introduction of clinical skills increases student comfort on the wards at the start of the 3rd year;27 several other studies of similar models have shown improved interpersonal and communication skills.28–29 Despite the contemporary focus in medical education research on outcomes, few meaningful outcome measures assess the influence of specific teaching innovations on students’ clinical performance.
As a part of ongoing efforts to investigate the effectiveness and impact of the Colleges program, this study explored whether the Colleges curriculum improved 3rd-year clerkship performance as assessed by performance evaluations. We conducted a non-randomized, retrospective, pre-post review of 3rd-year clerkship student performance evaluations from 2001 to 2007. We hypothesized that the Colleges curriculum would result in improved student performance in basic clinical skills on which the Colleges focus most strongly (communication, history-taking, physical exam, oral case presentations, and preparing write-ups), compared with ratings prior to inception of the Colleges.
The UW Colleges curriculum, implemented in 2001, aims to teach fundamental clinical skills and to provide longitudinal mentoring for medical students. Thirty-six College faculty mentors meet their small groups of six students weekly throughout the students’ second year. The first several sessions are conducted in a conventional curricular setting using observation, practice with peers, and standardized patients to teach history and physical exam skills. The mentors then take students to the hospital wards for weekly bedside teaching sessions interspersed with monthly organ-system-specific advanced-exam teaching sessions.
At each bedside teaching session, two students interview and examine hospitalized patients under their mentor’s supervision and then present their patients at the bedside. The other four students observe the bedside presentation and provide feedback. Through the year, each student independently interviews, examines, and performs bedside presentations on six hospitalized patients and observes their peers’ presentations on 30 additional patients. Faculty mentors provide additional teaching and skills modeling for the group at the bedside. The emphasis on focused teaching of clinical skills at the bedside with direct observation and feedback by faculty physicians enables role modeling, careful monitoring of skills development, and personalized feedback. Specific skills targeted by the curriculum include patient communication, history-taking, physical exam, oral case presentations, write-ups, professional values, and introduction to clinical reasoning.
The UWSOM Department of Medical Education and Biomedical Informatics, which maintains longitudinal student performance evaluation data from basic 3rd-year clerkships, prepared de-identified end-of-clerkship evaluations for the purposes of aggregate analyses. This research was approved by Human Subjects at the University of Washington prior to data analysis.
The Colleges program was implemented for medical students matriculating in fall 2001 and was fully phased in for students in fall 2002. We identified 3 years of performance evaluations prior to the Colleges (E-98, E-99, E-00) and 3 years after the Colleges implementation (E-02, E-03, E-04). We excluded the E-01 class, as the Colleges curriculum underwent substantial formative development and revision during that period; we also excluded students who did not proceed directly to 3rd-year clerkships within 3 years of matriculation (i.e., MD/PhD students, students in an expanded curriculum).
For each basic 3rd-year clerkship at the University of Washington School of Medicine (Family Medicine, Internal Medicine, Obstetrics-Gynecology, Pediatrics, Psychiatry, and Surgery), students receive end-of-clerkship evaluations with the purpose of providing feedback on performance to students and to the Dean’s Office. Similar global ratings are frequently used in medical education to assess performance.30–32 Faculty ratings of students’ clinical competence have been predictive of performance on medical licensing examinations and clinical competence ratings in residency.33
Evaluations at the UWSOM include a final numeric rating on a Likert scale (1–7) for each of 12 clinical skills domains (Table 1). Clinical clerkships at the UWSOM are conducted at clinical sites throughout five states—Washington, Wyoming, Alaska, Montana, and Idaho. Evaluations by residents and faculty at individual sites are merged by site coordinators and clerkship directors to create final numeric scores. Common descriptive anchors on each clerkship evaluation provide stability in assessment. We analyzed six separate performance evaluation data sets from each of the basic clerkships; variables include matriculation year, clerkship dates, and numeric values for each clinical-skill domain.
On students’ final evaluation, a final grade was also reported. To understand how a final grade is calculated for each clerkship, we interviewed the six clerkship directors. Each director reported a distinct system, communicated to students in writing at the beginning of the clerkship, for determining the final grade. All clerkships reported at least one structural change in the prior 7 years that may have impacted how grades were determined or that could have influenced the average final grade. The system for assigning final clerkship grades does not provide an anchor system to give stability from year to year. Because of the variability between clerkships and the lack of an anchor system, we elected not to include final grades in analyses.
STATA 10, a statistical software package, was used for statistical analyses of student performance evaluations.34 Analyses consisted of t-tests comparing the pre and post groups and assumed unequal variance. Standard alpha (0.05) was used to identify statistical significance. Effect sizes were calculated using Cohen’s d for comparisons in which a statistically significant difference was found pre- and post-Colleges. Pre-Colleges and post-Colleges cohorts were compared within each clerkship for statistically significant differences, but not across clerkships (i.e., clerkships were not compared to one another).
A total of 5,240 performance evaluations were analyzed (2,634 pre-Colleges and 2,606 post-Colleges implementation). Total numbers of evaluations were similar for each clerkship: Internal Medicine (871), Family Medicine (908), Obstetrics—Gynecology (819), Pediatrics (875), Psychiatry (934), and Surgery (833). Each clerkship had similar total numbers of evaluations in the pre- and post-College groups.
Comparisons of Internal Medicine clerkship performance data revealed statistically significant increases in the post-Colleges group in 9 of 12 clinical-skills domains (Table 2). Calculation of Cohen’s d effect sizes for comparisons found to be statistically significant in the Internal Medicine clerkship showed moderate effects; the largest effect sizes were for educational attitudes (0.24), patient relationships (0.21), and communication skills (0.21).
Analyses in other clerkships revealed few individual statistically significant differences pre- and post-Colleges, and no clear trends (Table 3). Across clerkships, there was a trend towards improvement post-Colleges in technical communication skills, with statistically significant improvements in Internal Medicine, Pediatrics, and Surgery.
Our study examined the influence of a structured, bedside clinical-skills curriculum for 2nd-year medical students, the Colleges program, on students’ performances in basic 3rd-year clerkships. Analyses identified improvement in student performance ratings on a substantial set of clinical skills domains within the Internal Medicine clerkship. The improvements were primarily in relational, communications, or attitudinal domains. These findings confirm the value of a preclinical skills curriculum focused on bedside teaching under the guidance of faculty mentors for advancing students’ internal medicine skills beyond the 2nd year.
The positive findings specifically for the Internal Medicine clerkship may be due to several factors. The specialty of internal medicine has historically been described as “the mother of all clinical divisions,” the foundation on which the instruction of the art of medicine is built.35,36 This may account for why internal medicine faculty are heavily involved in teaching the Introduction to Clinical Medicine courses nationwide.9 Faculty in the Colleges represent a diverse range of specialties; approximately one-third are internists. Further, the skills taught in the Colleges program are the foundational internal medicine skills of history-taking, physical examination, communication, oral case presentation, and write-ups. The patients with whom students work in the Colleges program are predominantly adult inpatients, often on general medicine wards. This increases the preclinical skills curriculum’s internal medicine orientation and enhances students’ future comfort and confidence with adult inpatients. Students’ skills in other areas are developed later and differentiated into other clerkships and skill sets, but the foundational approach to the history and physical lies within the specialty of internal medicine. In addition, students spend more time in the internal medicine clerkship than in other basic 3rd-year clerkships—12 weeks compared with 4–8 weeks. Thus, performance in internal medicine should provide a more in-depth portrait of a student’s performance, enhancing the likelihood that positive (or negative) results may be demonstrated.
It is also likely that the statistically significant improvements seen in the domain of technical communication in Internal Medicine, Surgery, and Pediatrics are related to the Colleges curriculum. Technical communication refers to skills in oral case presentation and write-ups, described on performance evaluation forms as “completeness, logic, and accuracy of communications (with the medical team) regarding patient’s problems.” This curricular element has been particularly emphasized by the Colleges curriculum and was minimally covered for pre-clinical students prior to the Colleges.
While we expected to demonstrate improvement in performance evaluations in each clerkship, we did not find consistent improvements. Isolated declines in performance for individual areas within a few clerkships were identified, but it is difficult to assign meaning. Students may not have been as confident working with a more diverse patient panel in these clerkships (including pediatric, psychiatric, surgical, or obstetrics patients). Skills specific to each of these clerkships are more specialized than the general skills taught by the Colleges.
Interpretation of retrospective pre-post data analyses is subject to flaws. Though the initiation of the Colleges represents a significant change in the curriculum, we cannot determine a causal relationship; however, this was the only substantive change to the curriculum affecting preclinical students at UWSOM in the prior several years. Structural changes in individual clerkships may have contributed to changes in average grades and could have contributed to trends in clinical skills domains over time, despite constant anchors on evaluation forms. Data obtained for clerkship performance evaluations do not represent evaluations for every class member; the Department of Medical Education and Biomedical Informatics does not routinely receive all evaluations for its school-wide analyses, but for most years and clerkships, the data represent well over 90% of the class.
Clerkship performance evaluations do not necessarily measure the entire range of performance. However, a prior study evaluating the predictive validity of student evaluations in basic clerkships on performance for USMLE Step 2 and 3 scores and supervisors’ ratings in residency found the strongest association in evaluations from the internal medicine clerkship.33 This affirms the use of clerkship performance evaluations across a range of skills and to assess change over time. Finally, our analyses are limited by an inability to test for other potential predictor variables, such as clerkship site (location of clerkship) or setting (inpatient vs. outpatient), due to restrictions on data use by Human Subjects considerations.
Implementation of the Colleges pre-clinical curriculum appears to be associated with increased confidence27 and improved student performance in 3rd-year clerkships, especially in the Internal Medicine clerkship. Continued efforts to elucidate how to best prepare students for clerkships and how to evaluate outcomes of clinical skills training programs should be encouraged. The Colleges clinical skills curriculum, featuring guided bedside teaching on the wards with a dedicated faculty mentor during the 2nd year of medical school, appears to be an effective method for improving some aspects of student performance in basic clerkships.
Funding support for Dr. Jackson for her work in medical education research and program evaluation comes from the Office of the CEO and Dean of the School of Medicine at the University of Washington. Funding support for Dr. Keen for his work as a UW College faculty member is provided by UW School of Medicine.
Conflicts of Interest None disclosed.