|Home | About | Journals | Submit | Contact Us | Français|
To determine the prevalence and influence of specific attending teaching practices on student evaluations of the quality of attendings' teaching in the inpatient component of Internal Medicine clerkships.
Nationwide survey using a simple random sample.
One hundred and twenty-one allopathic 4-year medical schools in the United States.
A total of 2,250 fourth-year medical students.
In the spring of 2002, student satisfaction with the overall quality of teaching by attendings in the inpatient component of Internal Medicine clerkships was measured on a 5-point scale from very satisfied to very dissatisfied (survey response rate, 68.3%). Logistic regression was used to determine the association of specific teaching practices with student evaluations of the quality of their attendings' teaching. Attending physicians' teaching practices such as engaging students in substantive discussions (odds ratio (OR)=3.0), giving spontaneous talks and prepared presentations (OR=1.6 and 1.8), and seeing new patients with the team (OR=1.2) were strongly associated with higher student satisfaction, whereas seeming rushed and eager to finish rounds was associated with lower satisfaction (OR=0.6).
Findings suggest that student satisfaction with attendings' teaching is high overall but there is room for improvement. Specific teaching behaviors used by attendings affect student satisfaction. These specific behaviors could be taught and modified for use by attendings and clerkship directors to enhance student experiences during clerkships.
At a time when clinical faculty in academic departments of Internal Medicine have been urged to increase their patient care activities1 and subjected to increased reporting requirements,2 it is important to remain cognizant of the role of attending physicians in the training of medical students. In the team-oriented practice of Internal Medicine in clinical teaching settings, attending physicians have a great deal of influence in setting the tone for the learning environment, which in turn can influence student learning and their opinions regarding Internal Medicine as a specialty.
Several studies have assessed student satisfaction with clerkships. Myers3 and Mazor et al.4 found that students tend to rate outpatient medicine clerkships and attending physicians higher than inpatient clerkships and physicians. Only a few studies, however, have linked specific attending physician behaviors to students' assessments of their clerkships. Ramsey et al.5 and Irby6 found that greater instructor involvement with students was associated with greater satisfaction with Internal Medicine clerkships and ambulatory care rotations. Mazor et al.4 found that students rated Internal Medicine attending physicians highly on interest in teaching, showing respect for students' opinions, and being available to students. Xu et al.7 found an association between student satisfaction in surgery clerkships and student evaluations of attending physicians along dimensions such as showing respect for students, offering frequent contact, and serving as positive role models. These investigators also found that higher ratings of teaching rounds and conferences were associated with greater satisfaction.
Whereas many of these studies focus primarily on the attitudes or levels of involvement with students that attendings display, the goal of our study is to identify specific teaching practices used by attendings in Internal Medicine clerkships that affect student satisfaction with the quality of the teaching they receive. There is evidence that student assessments are linked to the actual quality of teaching. In educational settings outside the medical context, student evaluations of teaching have been shown to be correlated with a variety of indicators of effective teaching.8 In this study, we assess several teaching practices that may be affected by the pressures that have increasingly been placed on clinical faculty in medicine. Most of these practices are modifiable, however, and could be addressed by increasing the attention to pedagogy and enhancing faculty development for attending physicians.
We conducted a nationwide survey of a simple random sample of 2,250 fourth-year medical students drawn from 121 allopathic 4-year medical schools in the United States. We excluded medical schools in Puerto Rico and 2-year medical schools in the United States. Student contact information, demographics, and Medical College Admission Test (MCAT) scores were furnished by the Association of American Medical Colleges (AAMC). The AAMC also provided information pertaining to 2,500 subjects randomly selected from its roster of all medical students at the 121 schools who were in their fourth year of medical school in the 2001 to 2002 academic year. Of the 2,500 students, we randomly selected 250 for cognitive and pilot testing of the survey and reserved the rest for the final survey sample. The cognitive testing process, in which a sample of students who had taken a preliminary version of the survey were interviewed, led to slight refinements in the survey instrument.
The survey instrument was designed by physicians and other health services researchers and was based on a review of the literature, focus groups with students, and interviews with medical school faculty involved in the clinical education of medical students. The RAND Institutional Review Board approved the research protocol. The RAND Survey Research Group pretested the survey using established cognitive testing procedures, piloted the survey, and then administered it by mail in the spring of 2002. Surveys were designed to take 30 minutes to complete and were confidential. Respondents were asked to fill in blanks or choose among specified response categories. Response enhancement techniques included multiple mailings, telephone follow-up, and a monetary incentive of $40 for completing the survey. No significant problems with questions were observed during pretesting. However, a few questions were eliminated or combined to decrease administration time, and some other minor wording changes were made. The incentive level of $40 was determined as the most cost-effective for enhancing response.
Our survey questions were grouped conceptually into 4 different categories: attending teaching practices and behaviors in the Internal Medicine clerkship, organizational features of the clerkship, student characteristics, and student satisfaction with the quality of teaching by attendings.
Several survey questions were devoted to the teaching practices and behaviors of attending physicians—how often attendings engaged students in substantive discussions of differential diagnosis, pathophysiology, or work-up and management, how often they gave prepared presentations on a topic that came up during rounds on a previous day, how often they gave spontaneous or unplanned talks on a topic related to a patient on the ward service, how often they typically saw new patients with the team, how often they seemed rushed and eager to finish rounds, how often they got temporarily called away from rounds, how often they performed the initial examination and history on new patients with the students. Because students often have more than 1 attending physician during a clerkship, we asked them to respond to questions about all the attending physicians in their Internal Medicine clerkships, taken as a group. Through the cognitive testing process, we found that it was feasible for students to average their responses across attendings. An example of a survey question was the following: “During your Internal Medicine clerkship, how often did attendings typically see new patients with the team?” The 5 response options were “never or almost never,”“rarely,”“sometimes,”“often,” and “almost always or always.” We also asked students to tell us the amount of time in a typical week they spent with attendings outside of rounds or formal teaching sessions.
Structural or organizational features of the clerkships included the type of hospital through which students rotated, length of the clerkship, hours worked per week, amount of time spent on call, allocation of time across different activities, and other aspects of the workload assigned to students.
Our survey asked students whether they were married and whether they had children. In addition, our survey data were merged with demographic characteristics obtained from the Association of American Medical Colleges consisting of age, sex, race/ethnicity, and test scores on the verbal reasoning, physical science, and biological science sections of the MCAT.
Finally, we asked students to rate their level of satisfaction with the overall quality of teaching by attending physicians in their Internal Medicine clerkship on a 5-point scale: very dissatisfied, dissatisfied, neutral, satisfied, and very satisfied.
In addition to the survey data, we obtained a measure of the research intensity of the medical schools attended by the students. The measure consisted of NIH research dollars9 awarded to the school divided by the number of faculty in the school.10
We conducted both descriptive and multivariate analyses using students as the unit of analysis. The multivariate analysis consisted of a logistic regression model to determine the association of specific teaching practices and attending physician behaviors with student satisfaction with the overall quality of attending physician teaching. For ease of interpretation, the dependent variable in the model was a dichotomous measure of satisfaction that we created by collapsing the 5-point scale to 2 categories: “very satisfied or satisfied” versus “neutral, dissatisfied or very dissatisfied.” The results obtained using the dichotomous measure did not differ qualitatively or in statistical significance from those obtained using an ordered logistic model or an ordinary least squares regression model that treated the outcome as continuous, with minor exceptions discussed in the next section.
The independent variables of primary interest in the regression model were those describing attending physicians' teaching practices and behaviors. We entered the variables reporting the frequency of instructional practices (e.g., how often did attendings typically see new patients with the team?), which were measured on a 5-point scale, as “continuous” variables, owing to their ordinal nature. Other independent variables of interest in the model were organizational features of the clerkships. Last, the model included student characteristics and the measure of the research intensity of the medical school as controls.
Standard errors were adjusted for clustering of students within medical schools. A P value less than .05 was chosen as the criterion for statistical significance.
We collected 1,530 completed surveys, resulting in a response rate of 68.3% after adjusting for 11 ineligible responses. Comparison of respondent characteristics (age, sex, race/ethnicity, and MCAT scores) to those of students in the original sample showed no significant differences large enough to warrant the application of weights. African-American students were slightly less likely to respond but the difference was small, and sensitivity analyses weighting for non-response revealed no substantive differences.
Students were, on average, 28 years old (Table 1)The majority of the students were white, slightly fewer than half were women, nearly a third were married, and 11% had children.
Descriptive information on attending physicians' teaching practices and behaviors and on organizational features of the Internal Medicine clerkship is presented in Table 2. For those variables with 5 response categories, we collapsed the categories to 3 for ease of presentation.
Most students were satisfied with the quality of teaching of attending physicians, but over 20% were either neutral or dissatisfied. The reported teaching practices of attending physicians varied considerably. Most students (62%) reported that attending physicians often engaged them in substantive discussions related to differential diagnosis, pathophysiology, work-up, and management, but nearly a third reported that they were only sometimes engaged and more than 7% were rarely or never engaged in this type of discussion with attending physicians. More than one third of students reported that attending physicians rarely or never gave prepared presentations on topics that had come up during rounds. Attending physicians were more likely to give spontaneous or unplanned talks on topics related to patient care, although only half of the students reported that attending physicians did this often. About half the students reported that attending physicians typically saw new patients together with the team, but more than a third of students never or only rarely had this experience. In only very few instances did students report that attending physicians performed the initial exam and history with students; students reported doing this alone about 40% of the time and with a resident about a third of the time. Students reported that others had performed the exam and history before they saw the patient 23% of the time. A small fraction of the average student's week was spent with attending physicians outside of rounds or formal teaching sessions.
The settings in which students completed their Internal Medicine rotations were university hospitals, Veterans Affairs hospitals, and county or city hospitals (the majority of students—approximately 70%—rotated through more than 1 type of setting in their Internal Medicine clerkship). Students reported spending slightly more time per day doing “scut work” than talking to or examining patients. On a weekly basis, they reported spending the greatest amount of time reading for their clerkship, but also substantial amounts of time in rounds and lectures.
Table 3 reports odds ratios and confidence intervals for the independent variables in the logistic regression model.
We found that nearly all of the teaching practices and attending physician behaviors that we measured in our survey were significantly and substantially related to student satisfaction with the overall quality of attending teaching. The more frequently attending physicians saw new patients with the team, gave spontaneous or unplanned talks on topics relating to patients on the ward service, gave prepared presentations on topics that came up on prior rounds, and engaged students in substantive discussions relating to diagnoses, pathophysiology, or work-up, the more satisfied students were with attendings' teaching. The more that attending physicians seemed rushed and eager to finish rounds, however, the less satisfied students felt. These associations were highly significant and robust to different model specifications.
One organizational feature was also related to student satisfaction with the quality of teaching by attending physicians. Specifically, satisfaction was positively associated with the amount of time students spent examining patients. This result was not robust to model specification, however, and disappeared in the ordered logit and ordinary least squares regressions. In addition, these 2 specifications indicated that staying overnight when on call increased student satisfaction levels. None of the variables representing student characteristics was significantly related to student satisfaction with attendings' teaching in any model, nor was the research intensity of the institution (data not shown).
Although we found that a majority of students were satisfied with the quality of the teaching they received from their attendings, we also found that satisfaction levels varied considerably in relation to the teaching practices used by attendings. Attending physicians who engaged students in substantive discussions relating to patient care provided what students perceived to be particularly high-quality teaching.Figure 1 illustrates the potential impact of various attending behaviors on student satisfaction with the quality of teaching. Students who reported that attendings engaged them in substantive discussions “almost always or always” had a 97% likelihood of reporting that they were satisfied, whereas students who reported that attendings did this “almost never or never” had about a 28% likelihood of reporting that they were satisfied. Second in impact was giving spontaneous or prepared talks on topics that came up during rounds. Attending physicians who frequently saw new patients with the ward team raised satisfaction levels, whereas attending physicians who often seemed rushed or eager to finish rounds greatly depressed satisfaction. The only organizational feature of clerkships that was associated with satisfaction was the time students spent examining patients: students who spent more time on this activity were more satisfied with attendings' teaching.
When we place these findings in context by referring to the descriptive information on teaching practices and attending physician behaviors (Table 2), we find substantial room for improvement. For example, although attending physicians who engaged students in substantive discussions, gave spontaneous talks, or gave prepared presentations relating to patient care increased student satisfaction substantially, large percentages of students reported that their attending physicians used these 3 teaching techniques only sometimes or rarely.
The teaching practices that enhance student satisfaction with the quality of teaching—engaging students in substantive discussions, giving talks and presentations, and seeing new patients with the team—all require conscious attempts to integrate students into the patient care process and, possibly, spending additional time in teaching, as well. Spending additional time is problematic in the context of the increasing time pressures being experienced by clinical faculty and raises a concern that effective teaching practices may become less frequent. From this perspective, our finding that attending physicians least often gave prepared presentations is not surprising, since preparing up-to-date presentations based on the most recent literature can be time-consuming. The high prevalence of attending physicians who seemed rushed or eager to finish rounds is additional evidence that a lack of time may have a negative impact on teaching.
Our study has some limitations. First, all information on teaching practices, other attending physician behaviors, and organizational features was derived from student reports and not from actual monitoring and observation of attending physicians. Little information is known about the attendings being assessed. Thus, we do not know if other unobserved characteristics of attendings, students, or unobserved elements of student-attending interactions influenced both student reports of teaching practices and their satisfaction with the quality of teaching. For instance, students who were favorably predisposed towards Internal Medicine clerkships to begin with might have been both more attuned to attending physicians' positive behaviors and more likely to be satisfied with the quality of their teaching.
Second, although 99% of the respondents reported having completed or currently undergoing an Internal Medicine rotation at the time of the survey in the spring of 2002, we did not ask them to specify the dates of their clerkships. Students' survey responses may have differed depending on the timing of their individual clerkships.
Third, our study did not examine the association between student satisfaction with attendings' teaching and their subsequent clinical performance. It is not possible to know from these data what impact the identified instructional practices, satisfaction levels, and other aspects of clerkships have on the future clinical abilities of students. Nonetheless, as mentioned earlier, the quality of teaching has been shown to be correlated with student evaluations.
We also underscore that our focus was on student satisfaction with their attendings' teaching, not with their clerkships. Earlier studies have linked specific attending physician behaviors to students' assessments of their clerkships.5–7 These studies did not examine the role of specific instructional behaviors and practices, however. Based on these studies, it seems likely that instructional practices that promote higher satisfaction with attending teaching enhance overall satisfaction with the clerkship as well, but our study offers no direct evidence on this question.
Despite these limitations, our study offers insight into the importance of specific instructional practices in generating student satisfaction with the teaching of attending physicians—a key component of their Internal Medicine clerkships. Clinical faculty in Internal Medicine often face important tradeoffs when exercising their dual roles as educator and practitioner. The findings of this study point to specific teaching practices that may enhance student satisfaction in a learning climate in which time constraints are often pressing. In general, continued attention to teaching on the part of those with important roles in training students is clearly warranted in an era of increasing pressures that may pose a threat to the quality of clinical training. Simply engaging students in substantive discussions related to patient care will increase satisfaction levels and, most likely, improve instructional quality. In a period in which faculty time is scarce, this study identifies simple, easily replicable techniques by which attending physicians can enhance their perceived effectiveness with students. Encouraging the use of these techniques is likely to raise satisfaction and the quality of medical education.
We would like to thank Melissa Bradley, RAND Survey Research Group, for overseeing survey administration. The Atlantic Philanthropies provided grant support to this RAND Project HE387.