PMCCPMCCPMCC

Search tips
Search criteria 

Advanced


 
Formats
Article sections
Authors
Related links
Logo of jgimedspringer.comThis journalToc AlertsSubmit OnlineOpen Choice
 
J Gen Intern Med. 2009 September; 24(9): 1018–1022.
Published online 2009 July 5. doi:  10.1007/s11606-009-1045-2
PMCID: PMC2726882
Copyright © Society of General Internal Medicine 2009
Association Between Hand-off Patients and Subject Exam Performance in Medicine Clerkship Students
Valerie J. Lang, MD,corresponding author Christopher J. Mooney, MA, Alec B. O’Connor, MD, MPH, Donald R. Bordley, MD, and Stephen J. Lurie, MD, PhD
University of Rochester School of Medicine and Dentistry, 601 Elmwood Avenue, Box MED-HMD, Rochester, NY 14642 USA
Valerie J. Lang, Phone: +1-585-2734881, Fax: +1-585-2762212, Valerie_Lang/at/urmc.rochester.edu.
corresponding authorCorresponding author.
Received October 10, 2008; Revised April 9, 2009; Accepted June 3, 2009.
ABSTRACT
BACKGROUND
Teaching hospitals increasingly rely on transfers of patient care to another physician (hand-offs) to comply with duty hour restrictions. Little is known about the impact of hand-offs on medical students.
OBJECTIVE
To evaluate the impact of hand-offs on the types of patients students see and the association with their subsequent Medicine Subject Exam performance.
DESIGN
Observational study over 1 year.
PARTICIPANTS
Third-year medical students in an Inpatient Medicine Clerkship at five hospitals with night float systems.
MEASUREMENTS
Primary outcome: Medicine Subject Exam at the end of the clerkship; explanatory variables: number of fresh (without prior evaluation) and hand-off patients, diagnoses, subspecialty patients, and full evaluations performed during the clerkship, and United Stated Medical Licensing Examination (USMLE) Step I scores.
MAIN RESULTS
Of the 2,288 patients followed by 89 students, 990 (43.3%) were hand-offs. In a linear regression model, the only variables significantly associated with students’ Subject Exam percentile rankings were USMLE Step I scores (B = 0.26, P < 0.001) and the number of full evaluations completed on fresh patients (B =0.20, P = 0.048; model r2 = 0.58). In other words, for each additional fresh patient evaluated, Subject Exam percentile rankings increased 0.2 points. For students in the highest quartile of Subject Exam percentile rankings, only Step I scores showed a significant association (B = 0.22, P = 0.002; r2 = 0.5). For students in the lowest quartile, only fresh patient evaluations demonstrated a significant association (B = 0.27, P = 0.03; r2 = 0.34).
CONCLUSIONS
Hand-offs constitute a substantial portion of students’ patients and may have less educational value than “fresh” patients, especially for lower performing students.
KEY WORDS: medical student and residency education, medical education- instructional design, medical education-cognition/problem solving
Restrictions on resident duty hours were implemented nationally in July 2003 by the Accreditation Council for Graduate Medical Education.1 In response to duty hours restrictions, many residency programs implemented night float systems and other schedule changes that increase the number of patient hand-offs.2–6 At the University of California, San Francisco, the number of hand-offs among interns increased 40%.3 In a national survey of medicine chief residents, hand-offs were estimated to have increased by 11%.7 Programs that instituted float systems were more likely to report an increase in patients admitted by one physician and then transferred to a different team (55% vs. 29%; P = 0.007).7 In an overview of the implications of resident duty hour restrictions, Fletcher and colleagues noted that the impact of this loss of continuity on medical student education has not been adequately evaluated.8
Studies have revealed different impressions among clerkship directors and medical students regarding the impact of resident duty hour restrictions on students’ educational experiences. In a national survey of medicine clerkship directors, the majority disagreed or strongly disagreed that residents and attendings had more time to teach students.9 Medicine clerkship students at the University of Minnesota reported no significant change in the average amount of time they were taught by residents or attendings.10 At the University of Pennsylvania, students reported no significant difference in the proportion of time spent in educational activities.11 At the University of Michigan, students reported no significant difference in faculty and resident accessibility and quality of teaching, but they did report a significant decrease in the quality of feedback and an increase in the amount of scutwork they performed.12 Students at Harvard Medical School reported no significant change in the interest or skill of the residents and attendings they worked with and an increase in the availability of residents.13 However, evaluations from six clerkships were combined, and information on the medicine clerkship was not reported separately.
The preceding studies focused on process measures, such as time spent in educational activities, and perceived quality and accessibility of teachers. We found no studies evaluating the impact of duty hour restrictions and/or increased hand-offs on educational outcomes. A concern raised by hand-offs is the loss of opportunities for students to develop their knowledge and clinical reasoning skills by evaluating “fresh” patients (i.e., those who have not yet been given a diagnosis to explain their clinical presentation). Students in the medicine clerkship must practice the skills of evaluating a new patient, framing the central problem, and developing a differential diagnosis, evaluation plan, and management plan in order to develop and organize their knowledge base and expertise with clinical reasoning.14,15 In a system of hand-offs, where other physicians have already completed these tasks and sometimes resolved the presenting problem before patients are handed off to students, students’ knowledge base and clinical reasoning skills may not develop as effectively.
At the University of Rochester, students in the Inpatient Medicine Clerkship rotate at five hospitals. The residency programs were subject to New York State duty hours restrictions for 15 years prior to the implementation of national duty hour restrictions, and all five hospitals have well-established night float systems for the general medicine teams. Patients evaluated and treated by the night float residents are handed off to the day teams. Students rotating on day teams may be assigned to follow these night float hand-offs or other types of “hand-offs,” including patients who were already on the resident team or patients transferred from the ICU; they may also be assigned “fresh” patients, who are generally admitted directly from the emergency department.
Objective
The purpose of this study was to evaluate the extent to which students received patients as hand-offs during the medicine clerkship and whether the frequency with which students evaluated fresh patients was associated with their performance on a standardized examination of knowledge and clinical reasoning at the end of the clerkship.
Setting and Participants At the University of Rochester, third-year students participate in an 8-week Inpatient Internal Medicine Clerkship. From April 2005 through March 2006, students were assigned to resident teams at five hospitals. All were assigned to a general medicine inpatient team for the first half of the clerkship. During the second half of the clerkship, the majority of students switched to a general medicine team at a different hospital, while a smaller number of students spent 2 weeks each on subspecialty acute services (hematology-oncology unit, cardiac care unit, or urgent care). Two of the hospitals were staffed by the same residency program, but all five had night float systems for their general medicine teams. Night float residents completed evaluations of new patients, documented their admission notes in the chart, and initiated treatment. Patients admitted by the night float were assigned to daytime resident teams the following morning. There was no overnight call for the daytime teams. The subset of students who spent half of their clerkship on hematology-oncology and cardiac care unit teams did not stay overnight, though the residents on these units did; they did not have a separate night float resident. Students ranked their preferred sites before the clerkship began, but were not guaranteed assignments according to their preferences.
Three of the five sites had a modified every 4th day call structure for the General Medicine teams, where teams stayed late and admitted patients every 4th day and admitted a smaller number of patients on 1 short call day (Table 1). At the two other sites, General Medicine teams admitted daily or almost daily, with 1 day per week when they were exempt from admitting patients (Table 1). The subspecialty teams had a hybrid schedule; residents admitted daily, but students admitted every 3–4 days.
Table 1
Table 1
Number of Fresh vs. Hand-Off Patients Seen by Students on General Medicine Teams at Five Hospitals
Data Collection
All students recorded their patient encounters in a database document that they kept in their e-mail folders. Students recorded the dates of the encounters, diagnoses, whether they completed a full evaluation (as opposed to a focused, follow-up evaluation), and whether the patient was a “hand-off” or “fresh.” “Hand-offs” were designated as night float hand-offs, ICU transfers, or patients who were already on the resident team before the students assumed their care part way through the hospitalization (“assumed care”). “Fresh” patients were considered new admissions to the student’s team, not previously evaluated by a night float or ICU physician. Students were required to submit the completed log before the end of the clerkship in order to sit for the final exam.
Students selected the primary and secondary diagnoses for each patient from a list of 73 options. For purposes of analysis, the 73 options were condensed into the 33 training problems outlined by the Clerkship Directors in Internal Medicine-Society of General Internal Medicine (CDIM-SGIM) Core Curriculum.16 During the year of the study, students were not instructed to see any specific conditions, but they were expected to directly follow a minimum of two to four new patients per week, and complete histories, physicals, notes, and presentations daily on those patients.
Assessment
On the last day of the clerkship, students took the National Board of Medical Examiners (NBME) subject examination in Internal Medicine. The test is a standardized, 100-item multiple-choice examination testing knowledge and reasoning skills using clinical vignettes. Nationally and at our institution, mean scores on the NBME examination differed by the quarter of the year in which students rotated. Thus, the percentile rankings provided by the NBME for each quarter were utilized.17
The NBME subject examination percentile rankings were compared with the number of “fresh” and handed-off patients on whom students did full evaluations, the number of patients whom they evaluated during general and subspecialty rotations, the total number of patients seen (with and without full evaluations), the number of core training problems encountered, and students’ prior scores on Step I of the United States Medical Licensing Exam (USMLE).
The study was approved and monitored by the University of Rochester Medical Center Research Subjects Review Board.
Statistical Analysis
For the general medicine teams, data on the proportion of hand-off patients at the three sites with every 4th day call structures were grouped together, and data from the two sites with daily admission structures were grouped together. These two groups were compared to each other and analyzed by chi-square analsis.
Bivariate analysis of the variables USMLE Step I score, total number of fresh patients (general medicine and subspecialty), number of fresh general medicine patients, total number of hand-off patients, total number of hand-off patients from the night float, and number of core diagnoses encountered compared with the outcome of NBME subject examination percentile ranking was conducted using Pearson’s r. The number of patients that did not have a full initial evaluation was very small; therefore, this variable was not included in the analysis.
Variables with moderate to strong correlation were included in a linear regression analysis to develop a model to predict students’ scores on the NBME subject examination for internal medicine. We tested for collinearity by computing variance inflation factors for our regression matrix. All of the computed variance inflation factors were well below ten, the threshold that is generally associated as a sign of severe multi-collinearity. In order to assess whether clinical experiences may variably affect the outcome of students with high vs. low levels of knowledge, we recalculated the model using the upper and lower quartile of students based on NBME subject examination percentile scores.
A two-tailed P with a cutoff of 0.05 was used to determine statistical significance. All analyses were performed with SAS version 9.1 (Cary, N.C.).
Ninety students completed the clerkship, and all (100%) returned their patient logs. Eighty-nine students included information regarding fresh vs. hand-off patients; of these 88 (97.8%) students provided adequate information for the regression analysis.
Of the 2,288 patients followed by 89 students (including both general medicine and subspecialty patients), 1,298 (56.7%) were fresh patients, and 990 (43.3%) were hand-offs. Of the hand-off patients, 480 (48.5%) were previously admitted by the night float team, 447 (45.2%) were already on the day team when students were assigned to them, and 63 (6.4%) were transferred from the ICU.
Because the subspecialty teams had no night float and a hybrid call schedule (residents stayed overnight and admitted daily, whereas students admitted every 3–4 days), only the general medicine teams were included in further analyses. A total of 1,873 (81.8%) patients were followed by students on general medicine teams, with a similar distribution of fresh (56.8%) and hand-off (43.2%) patients (Table 1).
For the general medicine teams, the proportion of hand-off patients at each site varied significantly (Table 1). The sites where teams admitted almost daily had significantly more hand-offs to students than sites with modified every 4th day admitting schedules (mean 51.3% vs. 31.2%, p < 0.0001).
Students followed up a median of 26 (IQR 23, 28) patients with a median of 16 (IQR 14, 17) of the core training problems. They performed full evaluations on a median of 19 (IQR 16, 23) patients, including 80.7% of the night-floated patients, 8.9% of the patients who were already on the team, and 72.9% of the ICU transfers, vs. 94.7% of the fresh admissions. They performed a median of 10 (IQR 7, 15) full evaluations on fresh patients. The mean USMLE Step I score was 222.76 (range 168–269; SD 20.43).
Bivariate analysis of subject exam percentile rankings revealed strong correlation with USMLE Step I scores (r = 0.75) and small but positive correlation (r = 0.13–0.19) with the total number of fresh patients evaluated and the number of core diagnoses encountered (Table 2). These three variables were selected for inclusion in a linear regression model. For the full sample, only USMLE Step I scores (B = 0.26, P < 0.0001) and the number of full evaluations completed on fresh patients (P = 0.048) demonstrated significant association with subject exam percentile rankings (r2 = 0.58; Table 3). In other words, for each additional fresh patient evaluated, subject exam rankings increased 0.2 percentiles (or five fresh patients for a 1-point percentile change). When the analysis was confined to students in the highest quartile of subject exam performance (n = 23), the only significant predictor was Step I score (P = 0.002; r2 = 0.5). For students in the lowest quartile (n = 23), the number of full evaluations completed on fresh patients was the only significant variable (P = 0.03; r2 = 0.34) (Table 3).
Table 2
Table 2
Correlation of Predictor Variables with NBME Subject Exam Percentile Rankings
Table 3
Table 3
Association Between Number of Fresh Patients Seen and NBME Subject Exam Percentile Rankings
We found a small but statistically significant association between the number of fresh patients seen by medical students on a third-year clerkship and NBME subject exam performance. Assuming this association is causal, it suggests that exposure to hand-off patients may have less educational value compared to learning from patients with undiagnosed presentations. However, our results suggest that the types of patients students encounter during the clerkship may play a less important role for the students who score higher on the subject exam and a more important role for students who score lower.
Bowen describes several categories of skills involved in diagnostic reasoning, including data acquisition and reporting, problem representation, and generation of hypotheses, which involves searching for and selecting an illness script.15 The development of each of these skills may be negatively impacted by hand-offs. For example, a night float resident might admit a patient complaining of headache, weakness, poor appetite, generalized aches, and a cough and appropriately hypothesize that the patient has influenza. A student would likely have more difficulty sifting through these complaints, prioritizing them, and linking the other symptoms to the cough. A student who saw the patient on initial presentation is more likely to learn from this process than a student who received the patient as a hand-off the next morning, after these clinical and cognitive tasks were completed. Although night float residents are often expected to complete rudimentary evaluations of new admissions, they usually perform the most important elements, including gathering key data, developing an assessment and differential diagnosis, and initiating an evaluation and treatment plan. It is possible that students who scored lower on the exam were more dependent on practicing these skills on fresh patients.
Nearly half of patients followed by medical students in our clerkship were hand-offs, thereby limiting students’ opportunities to evaluate “fresh” patients with undiagnosed presentations. The impact of night float systems on the number of hand-off patients students see may be influenced by a number of factors, including the daytime call structure. Although all five of our hospitals have night float systems for the general medical teams, students saw approximately 50% more hand-off patients if the teams admitted patients daily than if the teams admitted on an every 4th day call cycle. This is likely due to the need for the teams who admit daily to fill up their census early with night float patients and complete their work early, in contrast to teams who are expected to stay late into the evening twice a week. It is possible that residents in either call structure preferentially assign night float patients to students to ensure that the students will have enough patients to follow, and the students will have more time to finish their assessments before the end of the day. Also, residents may preferentially assign stabilized hand-off patients to students whom they perceive as less proficient.
The results of our study might reflect baseline characteristics of the students that were not assessed; for example, highly motivated students might request fresh patients and also be more likely to study for the exam. They may also be able to learn more from any patients they see, compensating for patients with lower learning value by reading or other activities, or have better skills in taking standardized exams. Prior studies have reported an association between USMLE Step I performance and Medicine Subject Exam performance.18,19 Students who scored in the lower quartile may be less equipped to compensate for seeing patients with lower learning value and gain more from the practice of repeatedly evaluating fresh patients. While it is possible that the loss of association between USMLE Step I score and Medicine Subject Exam performance among our students in the lowest quartile was an artifact of reduction in the range of exam scores with the smaller sample, this association persisted in the subsample of students with the highest scores, where the range of exam scores was similarly restricted.
The association between the number of fresh vs. hand-off admissions and subsequent exam performance is small. However, little else is known about what factors affect student learning in the clerkship. Griffith and colleagues found a small but significant association between working with a highly rated teacher and subsequent clerkship exam performance.18,19 We did not assess the students’ teachers. However, our students rotated with many different teaching faculty and residents (approximately 12 per student); the impact of a single teacher was likely to be attenuated.
A strength of this study was its inclusion of hospitals where night float systems were well established, minimizing the impact of the turmoil that programs inevitably face after adjusting to duty hour restrictions. It also had the advantage of including five different hospitals with different systems.
This study has several limitations. Because our data were observational, we cannot ascribe causality or rule out the possibility of unmeasured confounding variables. We did not assess the correlation between clerkship site and subject exam performance. Students were assigned to two different sites among the five hospitals in 12 different combinations; the number of students in any combination of sites was too small for analysis. The generalizability to other institutions also merits further study. We relied partially on students’ logbooks, and a small portion had incomplete data. Finally, the outcome assessed was cognitive performance as measured by a standardized multiple choice question exam. Further study is needed to assess the impact of hand-offs on clinical skills.
CONCLUSIONS
Fresh patients may have greater educational value for 3rd-year medical students than hand-off patients. In maximizing students’ opportunities to evaluate “fresh” patients, potentially helpful interventions might include rotating students onto night float teams, having students do “blind” evaluations of patients before reviewing the charts, practicing with simulations, and implementing every 4th day instead of daily admission schedules.
Acknowledgements
The authors acknowledge Kevin R. McCormick, MD (Highland Hospital), Walter Polashenski, MD (Rochester General Hospital), Philip Bonanni, MD (Unity Health), and Alan Kozak, MD (Mary Imogene Bassett Hospital) for their contributions to the data collection. There was no funding for this study.
Funding None.
Conflict of Interest None disclosed.
Footnotes
An abstract from this paper was presented at the Clerkship Directors in Internal Medicine National Conference in October 2007 and is published in the proceedings of that conference.
1. Accreditation Council for Graduate Medical Education. Common Program Requirements. Available at: http://www.acgme.org/acwebsite/dutyhours/dh_lang703.pdf; Accessed May 29, 2009.
2. Vidyarthi AR, Katz PP, Wall SD, Wachter RM, Auerbach AD. Impact of reduced duty hours on residents’ educational satisfaction at the University of California, San Francisco. Acad Med. 2006;81(1):76–81. [PubMed]
3. Vidyarthi AR, Arora V, Schnipper JL, Wall SD, Wachter RM. Managing discontinuity in academic medical centers: Strategies for a safe and effective resident sign-out. J Hosp Med. 2006;1(4):257–66. [PubMed]
4. Goldstein MJ, Kim E, Widmann WD, Hardy MA. A 360˚ evaluation of a night-float system for general surgery: A response to mandated work-hours reduction. Curr Surg. 2004;61(5):445–51. [PubMed]
5. Wallach SL, Alam K, Diaz N, Shine D. How do internal medicine residency programs evaluate their resident float experiences? South Med J. 2006;99(9):919–23. [PubMed]
6. Peterson LE, Johnson H, Pugno PA, Bazemore A, Phillips RL. Training on the clock: Family medicine residency directors’ responses to resident duty hours reform. Acad Med. 2006;81(12):1032–7. [PubMed]
7. Horwitz LI, Krumholz HM, Green ML, Huot SJ. Transfers of patient care between staff on internal medicine wards. Arch Intern Med. 2006;166:1173–7. [PubMed]
8. Fletcher K, Saint S, Mangrulkar RS. Balancing continuity of care with residents’ limited work hours: Defining the implications. Acad Med. 2005;80(1):39–43. [PubMed]
9. Kogan JR, Pinto-Powell R, Brown LA, Hemmer P, Bellini LM, Peltier D. The impact of resident duty hours reform on the internal medicine core clerkship: Results from the Clerkship Directors in Internal Medicine survey. Acad Med. 2006;81(12):1038–44. [PubMed]
10. Nixon LJ, Benson BJ, Rogers T, Sick BT, Miller WJ. Effects of Accreditation Council for Graduate Medical Education work restrictions on medical student experience. J Gen Intern Med. 2007;22:937–41. [PMC free article] [PubMed]
11. Kogan JR, Bellini LM, Shea JA. The impact of resident duty hour reform in a medicine core clerkship. Acad Med. 2004;79(10S):S58–61. [PubMed]
12. White CB, Haftel HM, Purkiss JA, Schigelone AS, Hammoud MM. Multidimensional effects of the 80-hour work week at the University of Michigan Medical School. Acad Med. 2006;81(1):57–62. [PubMed]
13. Jagsi R, Shapiro J, Weinstein DF. Perceived impact of resident work hour limitations on medical student clerkships: A survey study. Acad Med. 2005;80(8):752–7. [PubMed]
14. Norman G. Building on experience: The development of clinical reasoning. N Engl J Med. 2006;335(21):2251–2. [PubMed]
15. Bowen J. Educational strategies to promote clinical diagnostic reasoning. N Engl J Med. 2006;335(21):2217–25. [PubMed]
16. Defer TM, Fazio SB. Clerkship Directors in Internal Medicine-Society of General Internal Medicine. Core medicine clerkship curriculum guide: A resource for teachers and learners, version 3.0. Available at: http://www.im.org/Resources/Education/Students/Learning/Pages/CDIM-SGIMCoreMedicineClerkshipCurriculum.aspx. Accessed May 29, 2009.
17. National Board of Medical Examiners. Subject Examination Program Information Guide. 2007; p. 9. Available at: http://www.nbme.org/PDF/2007subexaminfoguide.pdf. Accessed May 29, 2009.
18. Griffth CH III, Wilson JF, Haist SA, Ramsbottom-Lucier M. Do students who work with better housestaff in their medicine clerkships learn more? Acad Med. 1998;73(10):S57–9. [PubMed]
19. Griffith CH III, Wilson JF, Haist SA, Ramsbottom-Lucier M. Relationships of how well attending physicians teach to their students’ performances and residency choices. Acad Med. 1997;72(10):S118–20. [PubMed]
Articles from Journal of General Internal Medicine are provided here courtesy of
Society of General Internal Medicine