PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of jgmeLink to Publisher's site
 
J Grad Med Educ. 2016 May; 8(2): 185–190.
PMCID: PMC4857508

Quantitative Study of the Characteristics of Effective Internal Medicine Noon Conference Presentations

Abstract

Background Increasing demands on residents' time have made it critically important to maximize the effectiveness of didactic activities and motivate independent study.

Objective Our aim was to correlate characteristics of noon conferences with internal medicine (IM) residents' ratings of perceived effectiveness and intent to pursue independent reading.

Methods We assessed characteristics of each noon conference by direct observation using predetermined metrics. We surveyed IM residents to assess their perception of the conference's effectiveness and their intention to pursue additional reading. A variety of modeling techniques were used to discern meaningful correlations of effectiveness and motivation.

Results A total of 649 evaluations of 29 conferences were submitted by 153 of 185 (83%) residents in the program. Median effectiveness score was 6 (on a scale of 1 to 7). Clinicopathological conferences had 0.55-point higher effectiveness scores than traditional conferences (P = .011). In multivariable analyses focusing on traditional conferences, summary statement inclusion was significantly associated with 0.43-point higher effectiveness scores (P = .016), and having resident speakers was associated with 0.50-point higher effectiveness scores than unfamiliar faculty (P = .045). Conferences with higher effectiveness scores had significantly higher proportions of respondents indicating intention to read.

Conclusions This is the first study to quantitatively assess correlations of high effectiveness ratings of noon conferences in a residency program. Intention to read improved with increasing effectiveness scores of conferences, suggesting residents are more inclined to pursue self-directed learning when topics are well presented. Considering these attributes in designs of didactic sessions may enhance their educational value.

What was known and gap

Didactic sessions are an important element of resident education, yet little is known about what constitutes effective didactics.

What is new

A study quantified attributes of didactic sessions that enhanced their effectiveness and promoted resident self-learning.

Limitations

Single site study reduces generalizability; lack of assessment of objective learning outcomes.

Bottom line

Residents are more inclined to pursue self-directed learning when topics are well presented, focus on clinical reasoning, and use a summary statement.

Editor's Note: The online version of this article contains the noon conference data collection sheet

Introduction

The Accreditation Council for Graduate Medical Education requires that internal medicine (IM) residency programs provide a didactic curriculum based on the core knowledge content.1 The ability of this curriculum to motivate independent study among residents is not known.29

A study in continuing medical education found that lecture effectiveness was correlated with the identification of important points, lecture clarity, and the ability to engage participants.7 An education task force made up of experts developed an instrument for peer assessment of faculty medical lectures and endorsed the following dimensions of effective didactics: clarity, enthusiasm, statement of importance and goals, summarization, subject command, audiovisual aids, audience interaction, and query of understanding.8 In a focus group study, IM residents reported that effective lectures were clinically relevant and practical, presented a balance of evidence, and included cases and questions.9

Characteristics of resident noon conferences have not been assessed with respect to lecture evaluations, and little is known about lecture characteristics that motivate independent study. Our aim was to evaluate the characteristics of noon conferences at a large academic teaching hospital, as well as to find correlations with IM residents' ratings of perceived effectiveness and intent to pursue independent reading.

Methods

Setting and Participants

This study was conducted at Massachusetts General Hospital, where the IM residency program offers 60-minute noon conference sessions covering a core set of clinical topics each weekday. While attendance is strongly encouraged, it is not required, and an average of 45 (out of a total of 185) residents attend each conference. Historically, 80% of the program's residents have entered a subspecialty fellowship.

Between October and December 2013, we assessed the characteristics of each noon conference by direct observation, and we surveyed residents about the educational value of each conference. Conferences were excluded if study staff were not available to observe, if the conference was not intended to be educational, or if resident survey data were not available.

Measures

Conference Characteristics

:

Guided by a review of the literature,79 we hypothesized that the educational value of noon conferences could be affected by who presents the conference (speaker characteristics), what is presented (content characteristics), and how the material is delivered (presentation characteristics). Specific characteristics for study were selected within this framework and based on feasibility of measurement.

Study investigators or chief residents trained by study investigators attended each noon conference to assess the speaker, content, and presentation characteristics according to a checklist (provided as online supplemental material). When more than 1 observer assessed a conference, the authors discussed discordant results to obtain a consensus.

Speaker characteristics included sex, specialty area (general IM, IM subspecialty, or non-IM specialty), and academic profile (resident, familiar faculty, or unfamiliar faculty). A faculty speaker was considered familiar to residents if he or she spent at least 2 months as an attending physician on a resident service each year or had an administrative role in the residency program.

Content characteristics included the estimated frequency with which a topic is encountered on resident services (≥ weekly, < weekly but ≥ monthly, or < monthly), the use of a patient case for at least 5 minutes (yes or no), and the use of at least 3 multiple-choice questions (yes or no). Additionally, we distinguished between traditional conferences and clinicopathological conferences (CPCs). CPCs include a brief case presentation by a resident, an in-depth discussion of diagnostic clinical reasoning by a faculty member, and a summary of the patient's diagnosis by a clinical pathologist.10

Presentation characteristics include the following binary variables: having an early statement of objectives, interacting with the audience, stepping away from the podium for at least 5 minutes, using humor (audience laughter at least 3 times), making a specific suggestion for further reading, including a summary statement (brief emphasis of take-home points at the end), and ending late (at least 5 minutes overtime).

Resident Evaluations

:

To assess residents' perceptions of the educational quality of each noon conference, we conducted a voluntary, deidentified electronic survey of all residents after each noon conference. We requested that residents complete the survey only if they had attended at least 50% of the conference. Survey participants rated the conference effectiveness on a scale of 1 to 7 (with 7 being the most effective). Additionally, participants indicated their intention to read about the conference topic in the following week, their postgraduate training year, and their sex. Four IM residents reviewed the survey for clarity prior to its deployment.

This study was declared exempt by the Partners Human Research Committee.

Statistical Analysis

Our primary analytic objective was to determine which noon conference characteristics were most strongly associated with 2 outcome measures: (1) the resident-rated effectiveness score (range 1 to 7), and (2) the intention to pursue additional reading (yes versus no).

We used generalized linear mixed models to account for correlations in the data arising from each conference having multiple evaluations and each resident providing multiple conference evaluations over time. We specified a normal response distribution and used the identity link function to model the continuous outcome of effectiveness score. We also specified a binary response distribution and the logit link function to model the dichotomous outcome of intention to read. For both model types, we entered conference characteristics as fixed effects and estimated their associations with each outcome. We treated evaluations from multiple residents about the same conference as repeated measures with compound symmetric covariance structure. We modeled a resident identifier variable (based on usernames provided with each survey) as a random effect for each outcome to account for multiple conference evaluations arising from a single resident over time. Surveys without an identifier were treated as independent observations.

We first compared effectiveness scores and intention to read outcomes for traditional conferences versus CPCs. Because CPCs are predetermined and differ considerably from traditional conferences, we focused subsequent modeling on traditional conference formats only. In the absence of a specific predictor variable of interest, we took an exploratory approach to model building to determine which characteristics of speakers, content, and presentation were most strongly associated with resident effectiveness scores and intention to read. Prior to modeling, we assessed the intention to read outcome's association with effectiveness score.

We began the modeling process by examining the bivariate associations between the independent variables (conference characteristics) and dependent variables (effectiveness scores and intention to read). Independent variables associated with dependent variables at a significance level of P < .20 were then entered in multivariable models. We considered variables associated with each outcome in multivariable models to be statistically significant at P < .05. We conducted all analyses using SAS version 9.4 (SAS Institute Inc, Cary, NC).

Results

A total of 35 noon conferences occurred during the study period. From the analysis, we omitted 2 conferences because the survey was not sent to the residents, 1 conference because no designated observers were available, and 3 conferences because these were resident meetings without educational intent. The final analytic data set included 29 noon conferences, all of which were either traditional lectures or CPCs. Characteristics for 15 conferences (52%) were recorded by more than 1 person. In most cases, the majority of raters agreed and the majority opinion was used in the analysis.

Conference Characteristics

The characteristics of the 29 noon conferences are shown in table 1. A total of 86% (25 of 29) of noon conferences opened with a statement of objectives, and 62% (18 of 29) concluded with a summary statement (table 1). Most presenters were general or subspecialty internists, and more than two-thirds were men. On average, 22.4 (SD = 5.5) evaluations were submitted for each conference.

table 1
Characteristics of Noon Conferences (N = 29)

Resident Respondents

Residents submitted a total of 649 evaluations. Of these, 577 evaluations were associated with 153 distinct resident usernames (median 2 evaluations per resident, range 1 to 19 per resident), comprising approximately 83% of the 185 residents in the program during the study period. Of those who provided identifying information, 37% (57 of 153) were postgraduate year (PGY) 1, 29% (45 of 153) were PGY-2, 33% (51 of 153) were PGY-3, and 56% (86 of 153) were men.

Effectiveness Scores

Across all conference evaluations, the mean effectiveness score was 6 out of 7 (Q1 = 5, Q3 = 7) , and 91% (591 of 649) of scores were ≥ 5. CPCs had 0.55-point higher effectiveness scores than traditional conferences (95% CI 0.13–0.98, P = .011).

In unadjusted analyses of traditional conferences, having a summary statement (P = .005) and the academic profile of the speaker (P = .032 for overall effect) were significantly associated with higher effectiveness scores (table 2). In adjusted multivariable analyses, a summary statement was significantly associated with 0.43-point higher effectiveness scores (95% CI 0.08–0.77, P = .016), and resident speakers had 0.50-point higher effectiveness scores than unfamiliar faculty (95% CI 0.01–0.99, P = .045).

table 2
Regression Analysis of Lecture Characteristics With Effectiveness Scores of Residents

Intention to Read

Across all conference evaluations, 58% (376 of 649) indicated an intention to read about the topic the following week. Conferences with higher effectiveness scores had significantly higher proportions of respondents indicating an intention to read, such that for each 1-point increase in the effectiveness score, the odds of reading were 1.9-fold higher (95% CI 1.6–2.3). Unlike effectiveness scores, intention to read was not significantly different between CPC and traditional conference formats (OR = 0.81, 95% CI 0.40–1.62).

In unadjusted analyses of traditional conferences, having a summary statement (OR = 1.90, 95% CI 1.02–3.54) and speaker specialty (P = .007) were significantly associated with intention to read (table 2). In analyses mutually adjusted for these variables, only speaker specialty remained significant (P = .034), with conferences given by subspecialty internist speakers associated with 2-fold higher odds of intention to read relative to conferences given by general internists (OR = 2.04, 95% CI 1.11–3.75).

Discussion

To our knowledge, this is the first study to quantitatively assess the correlates of effective noon conferences in a residency training program. Conferences with a clinicopathological conference format were rated as more effective than traditional conferences, which is concordant with the findings of a prior qualitative study documenting residents' preferences for lectures that focus on clinical cases and questions.9 Despite this, the mere use of a patient case for at least 5 minutes was not significantly correlated with higher effectiveness scores among traditional conferences. While a possible advantage of CPCs may be the central role of the patient case throughout the conference, the emphasis on clinical reasoning rather than the features of a particular illness may be more important to the CPC format's perceived effectiveness.

Among traditional conferences, the use of a summary was a significant predictor of effectiveness scores. This finding supports the frequent suggestion by educational experts to summarize key points at the end of presentations.11,12 Additionally, resident speakers received higher effectiveness scores than either familiar or unfamiliar faculty members, pointing toward a potential role for peer-based educational strategies. Peer teaching is increasingly used in medical schools,13,14 and reports on its use in both surgical and radiology residency conferences have been positive.1518 Peer teaching may also foster mastery of the topic in residents who give the talks.

Intention to read was bolstered with increasing effectiveness scores, suggesting that residents are more inclined to pursue self-directed learning when topics are well presented. Although there were few significant correlations of intention to read, talks given by subspecialty IM faculty were associated with greater reading intentions than talks by general IM faculty. While the basis for this finding is uncertain, talks with a more specialized orientation may prompt learners to extend their own knowledge in less familiar topic areas.

Limitations of this study include that it was conducted at a single, large residency program, with a high proportion of graduates entering an IM subspecialty education. This may limit generalizability. The small number of lectures and the ceiling effect of ratings limit the ability to detect potentially important predictors of effective conferences. The outcome variables were subjective, and it is not known to what degree they correlate with subject mastery, actual independent reading, or standardized examination performance. Future research should include multi-institution studies that assess more distal educational outcomes.

Conclusion

Our findings suggest that residents are more inclined to pursue self-directed learning when topics are well presented. A noon conference curriculum that focuses on clinical reasoning skills, utilizes a summary statement, and incorporates resident speakers may have enhanced educational value.

Supplementary Material

References

1. Accreditation Council for Graduate Medical Education. ACGME program requirements for graduate medical education in internal medicine. http://www.acgme.org/Portals/0/PFAssets/ProgramRequirements/140_internal_medicine_2016.pdf. Accessed April 8, 2016.
2. Cacamese SM, Eubank KJ, Hebert RS, Wright SM. Conference attendance and performance on the in-training examination in internal medicine. Med Teach. 2004; 26 7: 640– 644. [PubMed]
3. FitzGerald JD, Wenger NS. Didactic teaching conferences for IM residents: who attends, and is attendance related to medical certifying examination scores? Acad Med. 2003; 78 1: 84– 89. [PubMed]
4. McDonald FS, Zeger SL, Kolars JC. Associations of conference attendance with internal medicine in-training exam scores. Mayo Clin Proc. 2008; 83 4: 449– 453. [PubMed]
5. McDonald FS, Zeger SL, Kolars JC. Factors associated with medical knowledge acquisition during internal medicine residency. J Gen Intern Med. 2007; 22 7: 962– 968. [PMC free article] [PubMed]
6. Batalden MK, Warm EJ, Logio LS. Beyond a curricular design of convenience: replacing the noon conference with an academic half day in three internal medicine residency programs. Acad Med. 2013; 88 5: 644– 651. [PubMed]
7. Copeland HL, Longworth DL, Hewson MG, Stoller JK. Successful lecturing: a prospective study to validate attributes of the effective medical lecture. J Gen Int Med. 2000; 15 6: 366– 371. [PMC free article] [PubMed]
8. Newman LR, Brodsky DD, Roberts DH, Pelletier SR, Johansson A, Vollmer CM, et al. Developing expert-derived rating standards for the peer assessment of lectures. Acad Med. 2012; 87 3: 356– 363. [PubMed]
9. Sawatsky AP, Zickmund SL, Berlacher K, Lesky D, Granieri R. Understanding resident learning preferences within an internal medicine noon conference lecture series: a qualitative study. J Grad Med Educ. 2014; 6 1: 32– 38. [PMC free article] [PubMed]
10. Hunt DP, Scheske MD, Dudzinski DM, Arvikar SL. Case records of a Massachusetts general hospital. Case 22-2015. A 20-year-old man with sore throat, fever, myalgias, and a pericardial effusion. N Engl J Med. 2015; 373 3: 263– 271. [PubMed]
11. Skeff KM, Stratos GA. Methods for Teaching Medicine (ACP Teaching Medicine Series). 1st ed. Philadelphia, PA: American College of Physicians Press; 2010.
12. Schwenk TL, Whitman N. The Physician as Teacher. 1st ed. Baltimore, MD: Williams & Wilkins; 1987.
13. Benè KL, Bergus G. When learners become teachers: a review of peer teaching in medical student education. Fam Med. 2014; 46 10: 783– 787. [PubMed]
14. Hoogenes J, Mironova P, Safir O, McQueen SA, Abdelbary H, Drexler M, et al. Student-led learning: a new teaching paradigm for surgical skills. Am J Surg. 2015; 209 1: 107– 114. [PubMed]
15. Collins J, Miller SS, Albanese MA. Resident learning and knowledge retention from resident-prepared chest radiology conferences. Acad Radiol. 1997; 4 11: 732– 735. [PubMed]
16. Mainiero MB, Collins J, Primack SL. Effectiveness of resident-prepared conferences in teaching imaging utilization guidelines to radiology residents. Acad Radiol. 1999; 6 12: 749– 751. [PubMed]
17. Bull DA, Stringham JC, Karwande SV, Neumayer LA. Effect of a resident self-study and presentation program on performance on the thoracic surgery in-training examination. Am J Surg. 2001; 181 2: 142– 144. [PubMed]
18. Schuller MC, DaRosa DA, Clandall ML. Using just-in-time teaching and peer instruction in a residency program's core curriculum: enhancing satisfaction, engagement, and retention. Acad Med. 2015; 90 3: 384– 391. [PubMed]

Articles from Journal of Graduate Medical Education are provided here courtesy of Accreditation Council for Graduate Medical Education