Search tips
Search criteria 


Logo of jgimedspringer.comThis journalToc AlertsSubmit OnlineOpen Choice
J Gen Intern Med. 2006 May; 21(5): 538–541.
PMCID: PMC1484798

Beyond Journal Clubs

Moving Toward an Integrated Evidence-Based Medicine Curriculum
Rose Hatala, MD, MSc,1 Sheri A Keitz, MD, PhD,2 Mark C Wilson, MD, MPH,3 and Gordon Guyatt, MD, MSc4


Incorporating evidence-based medicine (EBM) into clinical practice is an important competency that residency training must address. Residency program directors, and the clinical educators who work with them, should develop curricula to enhance residents' capacity for independent evidence-based practice. In this article, the authors argue that residency programs must move beyond journal club formats to promote the practice of EBM by trainees. The authors highlight the limitations of journal club, and suggest additional curricular approaches for an integrated EBM curriculum. Helping residents become effective evidence users will require a sustained effort on the part of residents, faculty, and their educational institutions.

Keywords: evidence-based medicine, medical education, curriculum, internship and residency

The practice of evidence-based medicine (EBM) has gained widespread attention as an important competency for clinicians and both undergraduate and postgraduate learners. North American organizations responsible for the standards for residency education, such as the Accreditation Council for Graduate Medical Education1 and the Royal College of Physicians and Surgeons of Canada,2 consider evidence-based practice a core competency that accredited training programs must address.

Many residency programs, as they struggle to meet new duty hour limitations3 and implement competency-based education,4 are undergoing intense reassessment and ongoing redesign. Now is the optimal time to ensure that core educational objectives to meet 1 of the 6 U.S. competency domains, practice-based learning and improvement (for Canadian programs, the CanMEDS domains of Medical Expert and Scholar), are clearly articulated and new opportunities to teach and assess this competency are well-integrated throughout our residency programs.

Helping our learners acquire key EBM skills during their residency training requires thoughtful curriculum development.5 At many institutions, the predominant curricular focus for EBM during residency training has been journal club, a well-ingrained tradition defined as a resident-centered, small-group discussion of research papers.6 However, there are significant limitations to journal club that may be restricting the development of EBM competency in our learners and ultimately the incorporation of evidence into practice. We will address these limitations and suggest additional curricular approaches to complement journal clubs. As with many educational endeavors, empiric research into specific EBM curricular innovations is limited.7 Here, we highlight some interventions that offer potential solutions for training programs.


A recent overview identified 23 articles on postgraduate EBM curricula, with 78% of the programs based on a seminar or journal club format.8 This tradition may have become ingrained among our training programs owing to ease of implementation, adaptability, faculty comfort, minimal preparation time for group participants, regular provision of food, and a resident-centered approach to teaching.9

Nevertheless, journal club has important limitations. Traditional journal clubs tend to focus excessively on critical appraisal skills.6 Heterogeneous studies examining the impact of journal club in postgraduate medical training have revealed inconsistent, small, and/or short-term improvements in knowledge of clinical epidemiology and biostatistics, critical appraisal skills, self-reported reading habits, and use of the medical literature.6,8,10 Self-assessed attitudes toward EBM and incorporation of EBM-related behaviors into clinical practice have shown either no improvement or inconsistent results using a standalone curricular approach.8

Attempting to improve upon these limitations, many programs have adopted the use of learning packages that contain a clinical case, a relevant article, a critical appraisal worksheet, and subsequent group discussion as to how to use the new information in the context of the specific scripted patient.9 Small group formats structured on mentored, case-based clinical problem-solving have provided more encouraging improvements in EBM skills and behaviors.11 However, even this approach to journal club has met with inconsistent results. A controlled trial of emergency medicine residents receiving a year-long, mentored, case-based journal club showed no difference between intervention and control residents in their ability to critically appraise a fabricated paper filled with methodological flaws.12 Conversely, The Hospital for Tropical Diseases in London has described success with a journal club that fostered the implementation of changes in management approaches for specific patient problems.13

Despite modifications to journal club formats, a recent large cross-sectional survey of incoming U.S. chief medical residents over the past 5 years reveals limited self-perceived skills in practicing or teaching in an evidence-based fashion.14 Perhaps the greatest limitation of a traditional journal club is that this format encourages residents to see EBM as a “separate” exercise that is not integrated into daily patient care.


Journal clubs' limitations are highlighted when we consider the breadth of competencies that describe an EBM curriculum. These competencies include abilities to: (1) recognize a patient problem and construct a structured clinical question; (2) efficiently and effectively search information resources to retrieve the best available evidence to answer the clinical question; (3) critically appraise this evidence; (4) gain a full understanding of the study results; and (5) integrate the evidence with all aspects of individual patient decision making to determine the best clinical care options for the patient.15


A recent overview of postgraduate EBM education emphasized that clinically integrated teaching of EBM is more likely to bring about desired changes in attitudes, skills, and behaviors.8 Offering multiple EBM interventions in multiple learning venues throughout residency training will facilitate the reinforcement and retention of key skills and behaviors. When residents have the time and opportunity to integrate EBM competencies during real-time patient care, they shift from a paternalistic to a participatory decision-making style.16

Echoing the results of the recent overview,8 we believe it is time to limit the emphasis on journal club and to move our postgraduate training programs toward integrated EBM curricula. The standalone formats of journal club or seminars may be useful tools to teach critical appraisal skills and expose learners to new literature in small group settings but they are not a substitute for learners pursuing an EBM exercise to inform the care of their own patients.11,17 To this end, EBM education should be broadened across the training program and not limited to journal clubs alone.

A 1998 survey of North American internal medicine residency programs revealed that a minority of programs attempted to integrate EBM teaching into real-time clinical venues such as attending rounds, morning report, and the ambulatory care clinic.18 One advantage of teaching EBM in “real-time” clinical practice is that educational opportunities abound to discuss clinical decision making when ideal evidence is lacking. In the remainder of this article, we suggest practical approaches to move beyond reliance on journal clubs alone and to implement an integrated EBM curricular change emphasizing 4 strategies: developing faculty role models as EBM practitioners, integrating EBM into clinical settings (inpatient and ambulatory), integrating EBM into morning report, and assessment of residents' EBM skills and behaviors.

In parallel to any of these initiatives, institutions must bring immediate, 24-hour access to electronic resources to the point of patient care. A minority of institutions provide resources beyond MEDLINE for their faculty and trainees,18 in part because many resources are expensive. However, limited access to electronic resources is a key barrier to the practice of EBM.19


What trainees observe in their clinical experience, particularly in the practices of respected clinicians, affects their future attitudes and behaviors and influences their career choices.20 As a result, compelling role models offer critical tools for teaching EBM in any clinical setting.21 If residents train in an environment in which educators address EBM as an isolated topic, and visible role models demonstrating the utility of EBM in clinical practice are lacking, they will assume that EBM is tangential to real clinical medicine, and thus, deserves only passing attention.19

Consequently, EBM curricular innovations must address faculty development. Institutions may, however, face substantial challenges in ensuring trainee exposure to high-quality role models. Few faculty may have the requisite EBM knowledge and training and those who do not, may not perceive acquiring these skills as a high priority.22 Faculty may lack confidence in searching the literature, critically appraising relevant articles, and applying the relevant literature to their patient care.22 Faculty may experience EBM as a challenge to their expert knowledge or leadership.19 Without adequate training, clinician-teachers may find it very difficult to devote the teaching time to demonstrate integration of evidence-based approaches.23 In 1 surgical training program, many of these factors led surgical residents to identify faculty as a significant barrier to practicing EBM.19

Committed institutions can look to successful models of faculty development programs.24,25 One regional faculty development program developed a cadre of EBM educators among New York Internal Medicine residency training programs.24 The 3½ day workshop included 5 educational strategies: (1) administrative buy-in, as each residency program or library director had to nominate faculty as course participants; (2) priority to training programs that nominated a critical mass of 2 or more physician and librarian faculty; (3) use of a collaborative, interactive, small-group learning model; (4) completion of a commitment to change contract by course participants outlining their action plan for curricular change in their home institution; (5) postcourse support including a website, newsletter, and continuing education sessions. Participants' post-course ratings demonstrated an increase in self-reported EBM knowledge and a desire to implement EBM teaching within their home institutions.

At an individual institutional level, educators at 1 U.S. medical school developed a faculty development program based on a faculty needs assessment.25 The 3-month course included a pre and postcourse EBM knowledge test, 4 didactic sessions, and 5 small-group meetings with physician and librarian co-tutors. Participants' EBM knowledge improved significantly and self-reported EBM skills remained high 9 months postcourse.25


Inpatient Rotations

Educators have described a number of curricular approaches to EBM teaching in the inpatient setting. McGinn et al.26 established an “EBM attending month.” Every weekday during a month-long inpatient general medicine rotation, 1 ward team developed an answerable clinical question relevant to the care of a new admission. The team searched the literature, critically appraised their findings, and discussed their results during attending rounds the following day. Surveying the participants at the end of the month, 50% believed that the EBM process had significantly influenced the care of their patients and 75% believed it would change the way they managed future patients.26

Two studies described the development of a 2-week block EBM elective.27,28 In 1 elective, family practice residents attended inpatient rounds, developed an answerable clinical question, searched the literature with a librarian coach, appraised the article, and presented a critically appraised topic (CAT) to the inpatient team.27 The rotation also included a web-based EBM curriculum and a journal club presentation. Resident satisfaction with the rotation was high, and participants' self-reported EBM skills and behaviors improved.27 A similar elective for internal medicine residents attached the resident to an inpatient team, without direct patient care responsibilities.28

Another study aided residents' ability to create an answerable clinical question during an inpatient general internal medicine rotation.29 Team residents received a 1-hour didactic session on building a clinical question using the population, intervention, comparison, outcome (PICO)30 format, and were given PICO file cards to record clinical questions during patient admissions. During an on-call shift, the chief medical resident reviewed a resident's question cards and encouraged residents to search the literature. Intervention residents demonstrated a 2-fold increase in their use of MEDLINE during the rotation.29

Ambulatory Clinic

In ambulatory settings, effective teaching techniques such as the 1-minute preceptor (OMP) are well suited to EBM teaching.31 The OMP focuses on 5 teaching microskills used during residents' case presentations: (1) getting a commitment from the learner as to the patient problem; (2) probing the learner for their underlying reasoning; (3) teaching a general principle; (4) providing positive feedback; and (5) correcting mistakes. Each of these steps is readily adaptable to EBM teaching, and we can take advantage of nonurgent decisions by turning the precepting moment into a multiple step exercise from question formulation to accessing and incorporating evidence as the clinical care of the patient unfolds over several weeks.

For example, Ross and Verdieck32 describe an EBM curriculum integrated into family practice residents' continuity clinic. Although their approach used a seminar format, the continuous nature of the teaching and its integration into the resident's outpatient experience resulted in an increase in EBM behaviors as recorded during resident-preceptor interactions. The EBM behaviors of the preceptors also increased during this intervention, potentially because of the residents' influence.32

An alternative approach used the outpatient setting to teach question formulation and information retrieval. Residents spent 30 minutes of protected time during 1 outpatient clinic/wk answering a clinical question generated during 1 of their patient encounters.33 They received a session on how to construct an answerable clinical question and search the literature, and had access to computer resources and general internal medicine faculty to mentor their searches. The residents completed 68% of their searches, and felt that the information they retrieved helped them in their daily patient care.33


Case-based discussion in morning report leads logically into explicitly acknowledging what is known and unknown and can result in pursuit of the best evidence to inform management decisions.34 One pediatrics program described an EBM morning report: once a week, an inpatient team identified a clinical question related to their patient, searched for an answer with the aid of a clinical librarian, critically appraised the evidence, and presented their summary during a morning report devoted to their team.35

Another internal medicine residency program described a slightly different approach.34 Their evidence-based morning report generated clinical questions based on 2 or 3 new admissions discussed during morning report. The residents subsequently searched and appraised the relevant literature and presented a 1-page summary to the group at the next morning report.34 Another variation on this approach had clinical librarians attend morning report to undertake the literature search and information retrieval “real-time” with the group.36 Any of these approaches should be applicable to other case-based formats such as traditional morbidity and mortality rounds or patient safety conferences.


To establish whether an education intervention has been successful, and to powerfully motivate learning, the intervention must include learner assessment and program evaluation. There are few established EBM assessment tools and as a result, assessment of our trainees is a challenging aspect of EBM curricula. The available assessment tools predominantly focus on specific skills such as critical appraisal.6

Developing a competent evidence user involves individual behavior change in addition to the acquisition of a new set of knowledge and skills. Thus, our assessment tools must measure these behaviors in simulated and real clinical settings. Standardized patient encounters during a clinical performance assessment can assess EBM skills.37 A few studies have used audiotapes in outpatient clinics to determine the frequency with which residents and faculty incorporate EBM principles in their interactions.32,38 Others have assessed residents search strategies by directly tracking on-line searches on the wards.29 Learning portfolios might provide another assessment tool to track changes in residents' EBM skills.39

Developing valid assessment tools goes hand in hand with improving the quality of research into effective EBM curricula. Using complementary quantitative and qualitative methodologies will allow us to more fully assess the impact of EBM curricular innovations.7


Many clinicians understand that practicing EBM is aspiring to the ideal attributes of the very best physicians. We must move away from the isolation of standalone EBM curricular efforts (often embodied in journal club) and bring EBM teaching prominently into our direct patient-care realms. Integrated curricular approaches, some of which we have highlighted in this article, offer alternatives to enhance our learners competence as EBM practitioners. Helping our residents become effective evidence users will require a sustained effort on the part of residents, faculty, and their educational institutions.


1. ACGME [Web document] Chicago, IL: Accreditation Council on Graduate Medical Education; 2001. [May 28, 2005]. Outcome Project: General Competencies. Available at:
2. The Royal College of Physicians and Surgeons of Canada [Web document] [May 28, 2005]. Documents and Publications.
3. Report of the ACGME Work Group on Resident Duty Hours. Chicago, IL: Accreditation Council for Graduate Medical Education; 2002.
4. Goroll AH, Sirio C, Duffy FD, et al. A new model for accreditation of residency programs in internal medicine. Ann Intern Med. 2004;140:902–9. [PubMed]
5. Kern DE, Thomas PA, Howard DM, Bass EB. Curriculum Development for Medical Education: A Six-Step Approach. Baltimore, MD: John Hopkins University Press; 1998.
6. Green ML. Graduate medical education training in clinical epidemiology, critical appraisal, and evidence-based medicine: a critical review of curricula. Acad Med. 1999;74:686–94. [PubMed]
7. Hatala R, Guyatt GH. Evaluating the teaching of evidence-based medicine. JAMA. 2002;288:1110–11. [PubMed]
8. Coomarasamy A, Khan KS. What is the evidence that postgraduate teaching in evidence based medicine changes anything? A systematic review. BMJ. 2004;329:1017–21. [PMC free article] [PubMed]
9. Alguire P. A review of journal clubs in postgraduate medical education. J Gen Intern Med. 1998;13:347–53. [PMC free article] [PubMed]
10. Ebbert JO, Montori VM, Schultz HJ. The journal club in postgraduate medical education: a systematic review. Med Teach. 2001;23:455–61. [PubMed]
11. Green ML, Ellis PJ. Impact of an evidence-based medicine curriculum based on adult learning theory. J Gen Intern Med. 1997;12:742–50. [PMC free article] [PubMed]
12. Bazarian JJ, Davis CO, Spillane LL, Blumstein H, Schneider SM. Teaching emergency medicine residents evidence-based critical appraisal skills: a controlled trial. Ann Emerg Med. 1999;34:148–54. [PubMed]
13. Lockwood D, Armstrong M, Grant A. Integrating evidence-based medicine into routine clinical practice: seven years' experience at the hospital for tropical diseases, London. BMJ. 2004;329:1020–3. [PMC free article] [PubMed]
14. Watkins R, Wilson MC, Richardson S. Chief residents' skills with evidence-based medicine: still hazy after all these years. J Gen Intern Med. 2004;19(suppl):130.
15. Guyatt G, Rennie D, editors. Chicago, IL: AMA Press; 2002. Users' Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice.
16. Montori VM, Tabini CC, Ebbert JO. A qualitative assessment of 1st-year internal medicine residents' perceptions of evidence-based clinical decision-making. Teach Learn Med. 2001;14:114–8. [PubMed]
17. Ghali WA, Saitz R, Eskew AH, Gupta M, Quan H, Hershman WY. Successful teaching in evidence-based medicine. Med Educ. 2000;34:18–22. [PubMed]
18. Green M. Evidence-based medicine training in internal medicine residency programs. A National survey. J Gen Intern Med. 2000;15:129–33. [PMC free article] [PubMed]
19. Bhandari M, Montori V, Deveraux PJ, Dosanjh S, Sprague S, Guyatt GH. Challenges to the practice of evidence-based medicine during residents' surgical training: a qualitative study using grounded theory. Acad Med. 2003;78:1183–90. [PubMed]
20. Wright S, Carrese JA. Excellence in role modeling: insight and perspectives from the pros. CMAJ. 2002;167:638–43. [PMC free article] [PubMed]
21. Sackett DL, Straus SE, Richardson WS, Rosenberg W, Haynes RB. 2. Edinburgh: Churchill Livingstone; 2000. Evidence-Based Medicine. How to Practice and Teach EBM.
22. McAlister FA, Graham I, Karr GW, Laupacis A. Evidence-based medicine and the practicing clinician. J Gen Intern Med. 1999;14:236–42. [PMC free article] [PubMed]
23. McColl A, Smith H, White P, Field J. General practitioners' perceptions of the route to evidence based medicine: a questionnaire survey. BMJ. 1998;316:361–65. [PMC free article] [PubMed]
24. Leipzig RM, Wallace EZ, Smith LG, Sullivant J, Dunn K, McGinn T. Teaching evidence-based medicine: a regional dissemination model. Teach Learn Med. 2003;15:204–9. [PubMed]
25. Crites GE, Chrisagis X, Patel V, Little D, Drehmer T. A locally created EBM course for faculty development. Med Teach. 2004;26:74–8. [PubMed]
26. McGinn T, Seltz M, Korenstein D. A method for real-time, evidence-based general medical attending rounds. Acad Med. 2002;77:1150–52. [PubMed]
27. Thom DH, Haugen J, Sommers PS, Lovett P. Description and evaluation of an EBM curriculum using a block rotation. BMC Med Educ. 2004;4:19. [PMC free article] [PubMed]
28. Akl EA, Izuchukwu IS, El-Dika S. Integrating an evidence-based medicine rotation into an internal medicine residency curriculum. Acad Med. 2004;79:897–904. [PubMed]
29. Cabell CH, Schardt C, Sanders L, Corey GR, Keitz SA. Resident utilization of information technology. J Gen Intern Med. 2001;16:838–44. [PMC free article] [PubMed]
30. Richardson WS, Wilson MC, Nishikawa J, Hayward RS. The well-built clinical question: a key to evidence-based decisions. ACP J Club. 1995;123:A12–A13. [PubMed]
31. Neher JO, Gordon KC, Meyer B, Stevens N. A five-step “microskills” model of clinical teaching. J Am Board Fam Pract. 1992;5:419–24. [PubMed]
32. Ross R, Verdieck A. Introducing an evidence-based medicine curriculum into a family practice residency–is it effective? Acad Med. 2003;78:412–17. [PubMed]
33. Schilling LM, Steiner JF, Lundahl K, Anderson RJ. Residents' patient-specific clinical questions: opportunities for evidence-based learning. Acad Med. 2005;80:51–6. [PubMed]
34. Reilly B, Lemon M. Evidence-based morning report: a popular new format in a large teaching hospital. Am J Med. 1997;103:419–26. [PubMed]
35. Schwartz A, Hupert J, Elstein AS, Noronha P. Evidence-based morning report for inpatient pediatric rotations. Acad Med. 2000;75:1229. [PubMed]
36. Atlas MC, Smigielski EM, Wulff JL, Coleman MT. Case studies from morning report: librarians' role in helping residents find evidence-based clinical information. Med Ref Serv Q. 2003;22:1–14. [PubMed]
37. Davidson RA, Duerson M, Romrell L, Pauly R, Watson RT. Evaluating evidence-based medicine skills during a performance-based examination. Acad Med. 2004;79:272–5. [PubMed]
38. Flynn C, Helwig A. Evaluating an evidence-based medicine curriculum. Acad Med. 1997;72:454–5. [PubMed]
39. Fung MF, Walker M, Fung KF, et al. An internet-based learning portfolio in resident education: the KOALA multicentre programme. Med Educ. 2000;34:474–9. [PubMed]

Articles from Journal of General Internal Medicine are provided here courtesy of Society of General Internal Medicine