Over the last years key stake holders in the healthcare sector have increasingly recognised evidence based medicine (EBM) as a means to improving the quality of healthcare. However, there is considerable uncertainty about the best way to disseminate basic knowledge of EBM. As a result, huge variation in EBM educational provision, setting, duration, intensity, content, and teaching methodology exists across Europe and worldwide. Most courses for health care professionals are delivered outside the work context ('stand alone') and lack adaptation to the specific needs for EBM at the learners' workplace. Courses with modern 'adaptive' EBM teaching that employ principles of effective continuing education might fill that gap. We aimed to develop a course for post-graduate education which is clinically integrated and allows maximum flexibility for teachers and learners.
A group of experienced EBM teachers, clinical epidemiologists, clinicians and educationalists from institutions from eight European countries participated. We used an established methodology of curriculum development to design a clinically integrated EBM course with substantial components of e-learning. An independent European steering committee provided input into the process.
We defined explicit learning objectives about knowledge, skills, attitudes and behaviour for the five steps of EBM. A handbook guides facilitator and learner through five modules with clinical and e-learning components. Focussed activities and targeted assignments round off the learning process, after which each module is formally assessed.
The course is learner-centred, problem-based, integrated with activities in the workplace and flexible. When successfully implemented, the course is designed to provide just-in-time learning through on-the-job-training, with the potential for teaching and learning to directly impact on practice.
Evidence based medicine (EBM) is considered an integral part of medical training, but integration of teaching various EBM steps in everyday clinical practice is uncommon. Currently EBM is predominantly taught through theoretical courses, workshops and e-learning. However, clinical teachers lack confidence in teaching EBM in workplace and are often unsure of the existing opportunities for teaching EBM in the clinical setting. There is a need for continuing professional development (CPD) courses that train clinical trainers to teach EBM through on-the-job training by demonstration of applied EBM real time in clinical practice. We developed such a course to encourage clinically relevant teaching of EBM in post-graduate education in various clinical environments.
We devised an e-learning course targeting trainers with EBM knowledge to impart educational methods needed to teach application of EBM teaching in commonly used clinical settings. The curriculum development group comprised experienced EBM teachers, clinical epidemiologists, clinicians and educationalists from institutions in seven European countries. The e-learning sessions were designed to allow participants (teachers) to undertake the course in the workplace during short breaks within clinical activities. An independent European steering committee provided input into the process.
The curriculum defined specific learning objectives for teaching EBM by exploiting educational opportunities in six different clinical settings. The e-modules incorporated video clips that demonstrate practical and effective methods of EBM teaching in everyday clinical practice. The course encouraged focussed teaching activities embedded within a trainer's personal learning plan and documentation in a CPD portfolio for reflection.
This curriculum will help senior clinicians to identify and make the best use of available opportunities in everyday practice in clinical situations to teach various steps of EBM and demonstrate their applicability to clinical practice. Once fully implemented, the ultimate outcome of this pilot project will be a European qualification in teaching EBM, which will be used by doctors, hospitals, professional bodies responsible for postgraduate qualifications and continuing medical education.
Evidence-based medicine (EBM) involves approaching a clinical problem using a four-step method: (1) formulate a clear clinical question from a patient’s problem, (2) search the literature for relevant clinical articles, (3) evaluate (critically appraise) the evidence for its validity and usefulness, (4) implement useful findings into clinical practice. EBM has now been incorporated as an integral part of the medical curriculum in many faculties of medicine around the world. The Faculty of Medicine, King Abdulaziz University, started its process of curriculum reform and introduction of the new curriculum 4 years ago. One of the most characteristic aspects of this curriculum is the introduction of special study modules and electives as a student-selected component in the fourth year of study; the Introduction to Evidence-Based Medicine course was included as one of these special study modules. The purpose of this article is to evaluate the EBM skills of medical students after completing the course and their perceptions of the faculty member delivering the course and organization of the course.
Materials and methods
The EBM course was held for the first time as a special study module for fourth-year medical students in the first semester of the academic year 2009–2010. Fifteen students were enrolled in this course. At the end of the course, students anonymously evaluated aspects of the course regarding their EBM skills and course organization using a five- point Likert scale in response to an online course evaluation questionnaire. In addition, students’ achievement was evaluated with regard to the skills and competencies taught in the course.
Medical students generally gave high scores to all aspects of the EBM course, including course organization, course delivery, methods of assessment, and overall. Scores were also high for students’ self-evaluation of skill level and EBM experience. The results of a faculty member’s evaluation of the students’ achievement showed an average total percentage (92.2%) for all EBM steps.
The EBM course at the Faculty of Medicine, King Abdulaziz University, is useful for familiarizing medical students with the basic principles of EBM and to help them in answering routine questions of clinical interest in a systematic way. In light of the results obtained from implementing this course with a small number of students, and as a student-selected component, the author believes integrating EBM longitudinally throughout the curriculum would be beneficial for King Abdulaziz University medical students. It would provide a foundation of knowledge, offer easy access to resources, promote point-of-care and team learning, help students to develop applicable skills for lifelong learning, and help the faculty to achieve its goals of becoming more student-centered and encouraging students to employ more self-directed learning strategies.
student-selected component; evidence-based medicine; learning; curriculum
Health care professionals worldwide attend courses and workshops to learn evidence-based medicine (EBM), but evidence regarding the impact of these educational interventions is conflicting and of low methodologic quality and lacks generalizability. Furthermore, little is known about determinants of success. We sought to measure the effect of EBM short courses and workshops on knowledge and to identify course and learner characteristics associated with knowledge acquisition.
Health care professionals with varying expertise in EBM participated in an international, multicentre before–after study. The intervention consisted of short courses and workshops on EBM offered in diverse settings, formats and intensities. The primary outcome measure was the score on the Berlin Questionnaire, a validated instrument measuring EBM knowledge that the participants completed before and after the course.
A total of 15 centres participated in the study and 420 learners from North America and Europe completed the study. The baseline score across courses was 7.49 points (range 3.97–10.42 points) out of a possible 15 points. The average increase in score was 1.40 points (95% confidence interval 0.48–2.31 points), which corresponded with an effect size of 0.44 standard deviation units. Greater improvement in scores was associated (in order of greatest to least magnitude) with active participation required of the learners, a separate statistics session, fewer topics, less teaching time, fewer learners per tutor, larger overall course size and smaller group size. Clinicians and learners involved in medical publishing improved their score more than other types of learners; administrators and public health professionals improved their score less. Learners who perceived themselves to have an advanced knowledge of EBM and had prior experience as an EBM tutor also showed greater improvement than those who did not.
EBM course organizers who wish to optimize knowledge gain should require learners to actively participate in the course and should consider focusing on a small number of topics, giving particular attention to statistical concepts.
Background and objectives
Evidence-based health care requires clinicians to engage with use of evidence in decision-making at the workplace. A learner-centred, problem-based course that integrates e-learning in the clinical setting has been developed for application in obstetrics and gynaecology units. The course content uses the WHO reproductive health library (RHL) as the resource for systematic reviews. This project aims to evaluate a clinically integrated teaching programme for incorporation of evidence provided through the WHO RHL. The hypothesis is that the RHL-EBM (clinically integrated e-learning) course will improve participants' knowledge, skills and attitudes, as well as institutional practice and educational environment, as compared to the use of standard postgraduate educational resources for EBM teaching that are not clinically integrated.
The study will be a multicentre, cluster randomized controlled trial, carried out in seven countries (Argentina, Brazil, Democratic Republic of Congo, India, Philippines, South Africa, Thailand), involving 50-60 obstetrics and gynaecology teaching units. The trial will be carried out on postgraduate trainees in the first two years of their training. In the intervention group, trainees will receive the RHL-EBM course. The course consists of five modules, each comprising self-directed e-learning components and clinically related activities, assignments and assessments, coordinated between the facilitator and the postgraduate trainee. The course will take about 12 weeks, with assessments taking place pre-course and 4 weeks post-course. In the control group, trainees will receive electronic, self-directed EBM-teaching materials. All data collection will be online.
The primary outcome measures are gain in EBM knowledge, change in attitudes towards EBM and competencies in EBM measured by multiple choice questions (MCQs) and a skills-assessing questionniare administered eletronically. These questions have been developed by using questions from validated questionnaires and adapting them to the current course. Secondary outcome measure will be educational environment towards EBM which will be assessed by a specifically developed questionnaire.
The trial will determine whether the RHL EBM (clinically integrated e-leraning) course will increase knowledge, skills and attitudes towards EBM and improve the educational environment as compared to standard teaching that is not clinically integrated. If effective, the RHL-EBM course can be implemented in teaching institutions worldwide in both, low-and middle income countries as well as industrialized settings. The results will have a broader impact than just EBM training because if the approach is successful then the same educational strategy can be used to target other priority clinical and methodological areas.
To evaluate the educational effectiveness of a clinically integrated e-learning course for teaching basic evidence-based medicine (EBM) among postgraduate medical trainees compared to a traditional lecture-based course of equivalent content.
We conducted a cluster randomized controlled trial to compare a clinically integrated e-learning EBM course (intervention) to a lecture-based course (control) among postgraduate trainees at foundation or internship level in seven teaching hospitals in the UK West Midlands region. Knowledge gain among participants was measured with a validated instrument using multiple choice questions. Change in knowledge was compared between groups taking into account the cluster design and adjusted for covariates at baseline using generalized estimating equations (GEE) model.
There were seven clusters involving teaching of 237 trainees (122 in the intervention and 115 in the control group). The total number of postgraduate trainees who completed the course was 88 in the intervention group and 72 in the control group. After adjusting for baseline knowledge, there was no difference in the amount of improvement in knowledge of EBM between the two groups. The adjusted post course difference between the intervention group and the control group was only 0.1 scoring points (95% CI −1.2–1.4).
An e-learning course in EBM was as effective in improving knowledge as a standard lecture-based course. The benefits of an e-learning approach need to be considered when planning EBM curricula as it allows standardization of teaching materials and is a potential cost-effective alternative to standard lecture-based teaching.
Evidence-based medicine (EBM) has become standard approach in medicine. Patients and health authorities increasingly claim active patient roles in decision making. Education to cope with these roles might be useful. We investigated the feasibility, acceptability and possible impact of EBM training courses for patient and consumer representatives.
We designed a generic one-week EBM course based on previous experience with EBM courses for non-medical health professionals. A course specific competence test has been developed and validated to measure EBM skills. Formative and summative evaluation of the course comprised: 1) EBM skills; 2) individual learning goals; 3) self-reported implementation after six months using semi-structured interviews; 4) group-based feedback by content analysis. EBM skills' achievement was compared to results gathered by a group of undergraduate University students of Health Sciences and Education who had attended a comparable EBM seminar.
Fourteen EBM courses were conducted including 161 participants without previous EBM training (n = 54 self-help group representatives, n = 64 professional counsellors, n = 36 patient advocates, n = 7 others); 71% had a higher education degree; all but five finished the course. Most participants stated personal learning goals explicitly related to practicing EBM such as acquisition of critical appraisal skills (n = 130) or research competencies (n = 67). They rated the respective relevance of the course on average with 80% (SD 4) on a visual analogue scale ranging from 0 to 100%.
Participants passed the competence test with a mean score of 14.7 (SD 3.0, n = 123) out of 19.5 points. The comparison group of students achieved a mean score of 14.4 (SD 3.3, n = 43). Group-based feedback revealed increases of self confidence, empowerment through EBM methodology and statistical literacy, and acquisition of new concepts of patient information and counselling. Implementation of EBM skills was reported by 84 of the 129 (65%) participants available for follow-up interviews. Barriers included lack of further support, limited possibilities to exchange experiences, and feeling discouraged by negative reactions of health professionals.
Training in basic EBM competencies for selected patient and consumer representatives is feasible and accepted and may affect counselling and advocacy activities. Implementation of EBM skills needs support beyond the training course.
Introduction: Evidence-based medicine (EBM) has become increasingly important in the practice of gastroenterology and endoscopy, and the training of future gastroenterology physicians. The objectives were to assess the attitudes/opinions of gastroenterology specialists towards EBM, and evaluate possible gaps in education for certain EBM-related concepts.
Methods: An internet-based survey was emailed to 4073 gastroenterology specialists. The main outcome measurements were physicians’ endorsement of EBM, impact of EBM on clinical practice, utilization of EBM-specific resources, self-assessed understanding of EBM concepts (EBM familiarity score), and actual knowledge of EBM concepts (EBM competency score).
Results: A total of 337 gastroenterology specialists participated. On a sale of 1–10, there was widespread agreement that EBM improves patient care (median score = 9, interquartile range (IQR) = 7–10), and physicians should be familiar with techniques for critical appraisal of studies (median = 9, IQR = 8–10). Most (64.0%) utilized the EBM-related resource UpToDate™ regularly, as opposed to PubMed™ (47.1%) or Clinical Evidence™ (5.4%). The mean EBM familiarity score was 3.4 ± 0.6 on a scale of 1–4. Out of a maximum 49 points, the mean EBM competency score was 35 ± 4.9. There was poor concordance among EBM familiarity and competency scores (r = 0.161; p = 0.005). Academic practice (p < 0.001), research/teaching (p < 0.001), advanced degree (p = 0.012), and recent EBM training (p = 0.001) were all associated with improved EBM competency.
Conclusion: The attitudes and opinions of EBM are extremely favorable among gastroenterology physicians. Although gastroenterology physicians report familiarity with most EBM-related concepts, there is poor correlation with their actual knowledge of EBM. Further educational initiatives should be undertaken to address methods in which EBM skills are reinforced among all gastroenterology practitioners.
clinical practice; evidence-based medicine; survey
Teaching evidence-based medicine (EBM) should be evaluated and guided by evidence of its own effectiveness. However, no data are available on adoption of EBM by Syrian undergraduate, postgraduate, or practicing physicians. In fact, the teaching of EBM in Syria is not yet a part of undergraduate medical curricula. The authors evaluated education of evidence-based medicine through a two-day intensive training course.
The authors evaluated education of evidence-based medicine through a two-day intensive training course that took place in 2011. The course included didactic lectures as well as interactive hands-on workshops on all topics of EBM. A comprehensive questionnaire, that included the Berlin questionnaire, was used to inspect medical students’ awareness of, attitudes toward, and competencies’ in EBM.
According to students, problems facing proper EBM practice in Syria were the absence of the following: an EBM teaching module in medical school curriculum (94%), role models among professors and instructors (92%), a librarian (70%), institutional subscription to medical journals (94%), and sufficient IT hardware (58%). After the course, there was a statistically significant increase in medical students' perceived ability to go through steps of EBM, namely: formulating PICO questions (56.9%), searching for evidence (39.8%), appraising the evidence (27.3%), understanding statistics (48%), and applying evidence at point of care (34.1%). However, mean increase in Berlin scores after the course was 2.68, a non-statistically significant increase of 17.86%.
The road to a better EBM reality in Syria starts with teaching EBM in medical school and developing the proper environment to facilitate transforming current medical education and practice to an evidence-based standard in Syria.
Although evidence-based medicine (EBM) is a useful method for integrating evidence into the decision-making process of occupational physicians, occupational physicians lack EBM knowledge and skills, and do not have the time to learn the EBM method. In order to enable them to educate themselves at the time and place they prefer, we designed an electronic EBM course. We studied the feasibility and utility of the course as well as its effectiveness in increasing EBM knowledge, skills, and behaviour.
Occupational physicians from various countries were included in a within-subjects study. Measurements were conducted on participants' EBM knowledge, skills, behaviour, and determinants of behaviour at baseline, directly after finishing the course and 2 months later (n = 36). The feasibility and utility of the course were evaluated directly after the course (n = 42).
The course is applicable as an introductory course on EBM for occupational physicians in various countries. The course is effective in enhancing EBM knowledge and self-efficacy in practising EBM. No significant effect was found on EBM skills, behaviour, and determinants of behaviour. After the course, more occupational physicians use the international journals to solve a case.
An electronic introductory EBM course is suitable for occupational physicians. Although it is an effective method for increasing EBM knowledge, it does not seem effective in improving skills and behaviour. We recommend integrating e-learning courses with blended learning, where it can be used side by side with other educational methods that are effective in changing behaviour.
Evidence-based medicine; Distance education; Occupational health; Medical education
Evidence-based medicine (EBM) is an important element of medical education. However, limited information is available on effective curricula.
To evaluate a longitudinal medical school EBM curriculum using validated instruments.
DESIGN, PARTICIPANTS, MEASUREMENTS
We evaluated EBM attitudes and knowledge of medical students as they progressed through an EBM curriculum. The first component of the curriculum was an EBM “short course” with didactic and small-group sessions occurring at the end of the second year. The second component integrated EBM assignments with third-year clinical rotations. The 15-point Berlin Questionnaire was administered before the course in 2006 and 2007, after the short course, and at the end of the third year. The 212-point Fresno Test was administered before the course in 2007 and 2008, after the short course, and at the end of the third year. Self-reported knowledge and attitudes were also assessed in all three classes of medical students.
EBM knowledge scores on the 15-point Berlin Questionnaire increased from baseline by 3.0 points (20.0%) at the end of the second year portion of the course (p < 001) and by 3.4 points (22.7%) at the end of the third year (p < 001). EBM knowledge scores on the 212-point Fresno Test increased from baseline by 39.7 points (18.7%) at the end of the second year portion of the course (p < 001) and by 54.6 points (25.8%) at the end of the third year (p < 001). On a 5-point scale, self-rated EBM knowledge increased from baseline by 1.0 and 1.4 points, respectively (both p < 001). EBM was felt to be of high importance for medical education and clinical practice at all time points, with increases noted after both components of the curriculum.
A longitudinal medical school EBM was associated with markedly increased EBM knowledge on two validated instruments. Both components of the curriculum were associated with gains in knowledge. The curriculum was also associated with increased perceptions of the importance of EBM for medical education and clinical practice.
Electronic supplementary material
The online version of this article (doi:10.1007/s11606-011-1642-8) contains supplementary material, which is available to authorized users.
medical education; evidence-based medicine; medical school
Teaching the steps of evidence-based medicine (EBM) to undergraduate as well as postgraduate health care professionals is crucial for implementation of effective, beneficial health care practices and abandonment of ineffective, harmful ones. Stellenbosch University in Cape Town, South Africa, offers a 12-week, completely online module on EBM within the Family Medicine division, to medical specialists in their first year of training. The aim of this study was to formatively evaluate this module; assessing both the mode of delivery; as well as the perceived effectiveness and usefulness thereof.
We used mixed methods to evaluate this module: A document review to assess whether the content of the module reflects important EBM competencies; a survey of the students to determine their experiences of the module; and semi-structured interviews with the tutors to explore their perspectives of the module. Ethics approval was obtained.
The document review indicated that EBM competencies were covered adequately, although critical appraisal only focused on randomised controlled trials and guidelines. Students had a positive attitude towards the module, but felt that they needed more support from the tutors. Tutors felt that students engaged actively in discussions, but experienced difficulties with understanding certain concepts of EBM. Furthermore, they felt that it was challenging explaining these via the online learning platform and saw the need to incorporate more advanced technology to better connect with the students. In their view the key to successful learning of EBM was to keep it relevant and applicable to everyday practice. Tutors also felt that an online module on EBM was advantageous, since doctors from all over the world were able to participate.
Our study has shown that the online module on EBM was effective in increasing EBM knowledge and skills of postgraduate students and was well received by both students and tutors. Students and tutors experienced generic challenges that accompany any educational intervention of EBM (e.g. understanding difficult concepts), but in addition had to deal with challenges unique to the online learning environment. Teachers of EBM should acknowledge these so as to enhance and successfully implement EBM teaching and learning for all students.
Evidence-based medicine; Postgraduate; Online learning; Evaluation
To evaluate the educational effects of a clinically integrated e-learning course for teaching basic evidence-based medicine (EBM) among postgraduates compared to a traditional lecture-based course of equivalent content.
We conducted a cluster randomised controlled trial in the Netherlands and the UK involving postgraduate trainees in six obstetrics and gynaecology departments. Outcomes (knowledge gain and change in attitude towards EBM) were compared between the clinically integrated e-learning course (intervention) and the traditional lecture based course (control). We measured change from pre- to post-intervention scores using a validated questionnaire assessing knowledge (primary outcome) and attitudes (secondary outcome).
There were six clusters involving teaching of 61 postgraduate trainees (28 in the intervention and 33 in the control group). The intervention group achieved slightly higher scores for knowledge gain compared to the control, but these results were not statistically significant (difference in knowledge gain: 3.5 points, 95% CI -2.7 to 9.8, p = 0.27). The attitudinal changes were similar for both groups.
A clinically integrated e-learning course was at least as effective as a traditional lecture based course and was well accepted. Being less costly than traditional teaching and allowing for more independent learning through materials that can be easily updated, there is a place for incorporating e-learning into postgraduate EBM curricula that offer on-the-job training for just-in-time learning.
Trial registration number: ACTRN12609000022268.
Evidence-Based Medicine (EBM) is an important competency for the healthcare professional. Experimental evidence of EBM educational interventions from rigorous research studies is limited. The main objective of this study was to assess EBM learning (knowledge, attitudes and self-reported skills) in undergraduate medical students with a randomized controlled trial.
The educational intervention was a one-semester EBM course in the 5th year of a public medical school in Mexico. The study design was an experimental parallel group randomized controlled trial for the main outcome measures in the 5th year class (M5 EBM vs. M5 non-EBM groups), and quasi-experimental with static-groups comparisons for the 4th year (M4, not yet exposed) and 6th year (M6, exposed 6 months to a year earlier) groups. EBM attitudes, knowledge and self-reported skills were measured using Taylor’s questionnaire and a summative exam which comprised of a 100-item multiple-choice question (MCQ) test.
289 Medical students were assessed: M5 EBM=48, M5 non-EBM=47, M4=87, and M6=107. There was a higher reported use of the Cochrane Library and secondary journals in the intervention group (M5 vs. M5 non-EBM). Critical appraisal skills and attitude scores were higher in the intervention group (M5) and in the group of students exposed to EBM instruction during the previous year (M6). The knowledge level was higher after the intervention in the M5 EBM group compared to the M5 non-EBM group (p<0.001, Cohen's d=0.88 with Taylor's instrument and 3.54 with the 100-item MCQ test). M6 Students that received the intervention in the previous year had a knowledge score higher than the M4 and M5 non-EBM groups, but lower than the M5 EBM group.
Formal medical student training in EBM produced higher scores in attitudes, knowledge and self-reported critical appraisal skills compared with a randomized control group. Data from the concurrent groups add validity evidence to the study, but rigorous follow-up needs to be done to document retention of EBM abilities.
Evidence-based medicine; Undergraduate medical education; Curriculum development; Educational assessment; Critical appraisal skills
Evidence Based Medicine (EBM) is a core unit delivered across many medical schools. Few studies have investigated the most effective method of teaching a course in EBM to medical students. The objective of this study was to identify whether a blended-learning approach to teaching EBM is more effective a didactic-based approach at increasing medical student competency in EBM.
A mixed-methods study was conducted consisting of a controlled trial and focus groups with second year graduate medical students. Students received the EBM course delivered using either a didactic approach (DID) to learning EBM or a blended-learning approach (BL). Student competency in EBM was assessed using the Berlin tool and a criterion-based assessment task, with student perceptions on the interventions assessed qualitatively.
A total of 61 students (85.9%) participated in the study. Competency in EBM did not differ between the groups when assessed using the Berlin tool (p = 0.29). Students using the BL approach performed significantly better in one of the criterion-based assessment tasks (p = 0.01) and reported significantly higher self-perceived competence in critical appraisal skills. Qualitative analysis identified that students had a preference for the EBM course to be delivered using the BL approach.
Implementing a blended-learning approach to EBM teaching promotes greater student appreciation of EBM principles within the clinical setting. Integrating a variety of teaching modalities and approaches can increase student self-confidence and assist in bridging the gap between the theory and practice of EBM.
Evidence based medicine; Blended learning; Graduate medical education; Pedagogy
Rationale, aims and objectives
Efforts to implement evidence-based medicine (EBM) training in developing countries are limited. We describe the results of an international effort to improve research capacity in a developing country; we conducted a course aimed at improving basic EBM attitudes and identified challenges.
Between 2005 and 2009, we conducted an annual 3-day course in Perú consisting of interactive lectures and case-based workshops. We assessed self-reported competence and importance in EBM using a Likert scale (1 = low, 5 = high).
Totally 220 clinicians participated. For phase I (2005–2007), self-reported EBM competence increased from a median of 2 to 3 (P < 0.001) and the perceived importance of EBM did not change (median = 5). For phase II (2008–2009), before the course, 8–72% graded their competence very low (score of 1–2). After the course, 67–92% of subjects graded their increase in knowledge very high (score of 4–5). The challenges included limited availability of studies relevant to the local reality written in Spanish, participants’ limited time and lack of long-term follow-up on practice change. Informal discussion and written evaluation from participants were universally in agreement that more training in EBM is needed.
In an EBM course in a resource-poor country, the baseline self-reported competence and experience on EBM were low, and the course had measurable improvements of self-reported competence, perceived utility and readiness to incorporate EBM into their practices. Similar to developed countries, translational research and building the research capacity in developing countries is critical for translating best available evidence into practice.
developing countries; education; evidence-based medicine; evidence-based practice; international cooperation
Evidence-based medicine (EBM) is increasingly taught in medical schools, but few curricula have been evaluated using validated instruments.
To evaluate a longitudinal medical school EBM curriculum using a validated instrument.
Design, Participants, Measurements
We evaluated EBM attitudes and knowledge of 32 medical students as they progressed through an EBM curriculum. The first part was an EBM “short course” with didactic and small-group sessions occurring at the end of the second year. The second part integrated EBM assignments with third-year clinical rotations. The validated 15-item Berlin Questionnaire was administered before the course, after the short course, and at the end of the third year.
EBM knowledge scores increased from baseline by 2.8 points at the end of the second year portion of the course (p = .0001), and by 3.7 points at the end of the third year (p < .0001). Self-rated EBM knowledge increased from baseline by 0.8 and 1.1 points, respectively (p = .0006 and p < .0001, respectively). EBM was felt to be of high importance for medical education and clinical practice at all time points, peaking after the short course.
A longitudinal medical school EBM curriculum was associated with increased EBM knowledge. This knowledge increase was sustained throughout the curriculum.
Electronic supplementary material
The online version of this article (doi:10.1007/s11606-008-0625-x) contains supplementary material, which is available to authorized users.
medical education; evidence-based medicine; medical school
Evidence-Based Medicine (EBM) is an important method used by occupational physicians (OPs) to deliver high quality health care. The presence and quality of a knowledge infrastructure is thought to influence the practice of EBM in occupational health care. This study explores the facilities in the knowledge infrastructure being used by OPs in different countries, and their perceived importance for EBM practice.
Thirty-six OPs from ten countries, planning to attend an EBM course and to a large extent recruited via the European Association of Schools of Occupational Medicine (EASOM), participated in a cross-sectional study.
Research and development institutes, and knowledge products and tools are used by respectively more than 72% and more than 80% of the OPs and they are rated as being important for EBM practice (more than 65 points (range 0–100)). Conventional knowledge access facilities, like traditional libraries, are used often (69%) but are rated as less important (46.8 points (range 0–100)) compared to the use of more novel facilities, like question-and-answer facilities (25%) that are rated as more important (48.9 points (range 0–100)). To solve cases, OPs mostly use non evidence-based sources. However, they regard the evidence-based sources that are not often used, e.g. the Cochrane library, as important enablers for practising EBM. The main barriers are lack of time, payment for full-text articles, language barrier (most texts are in English), and lack of skills and support.
This first exploratory study shows that OPs use many knowledge infrastructure facilities and rate them as being important for their EBM practice. However, they are not used to use evidence-based sources in their practice and face many barriers that are comparable to the barriers physicians face in primary health care.
To develop and implement an evidence-based medicine (EBM) curriculum and determine its effectiveness in improving residents' EBM behaviors and skills.
Description of the curriculum and a multifaceted evaluation, including a pretest-posttest controlled trial.
University-based primary care internal medicine residency program.
Second- and third-year internal medicine residents (N =34).
A 7-week EBM curriculum in which residents work through the steps of evidence-based decisions for their own patients. Based on adult learning theory, the educational strategy included a resident-directed tutorial format, use of real clinical encounters, and specific EBM facilitating techniques for faculty.
MEASUREMENTS AND MAIN RESULTS
Behaviors and self-assessed competencies in EBM were measured with questionnaires. Evidence-based medicine skills were assessed with a 17-point test, which required free text responses to questions based on a clinical vignette and a test article. After the intervention, residents participating in the curriculum (case subjects) increased their use of original studies to answer clinical questions, their examination of methods and results sections of articles, and their self-assessed EBM competence in three of five domains of EBM, while the control subjects did not. The case subjects significantly improved their scores on the EBM skills test (8.5 to 11.0, p =.001), while the control subjects did not (8.5 to 7.1, p =.09). The difference in the posttest scores of the two groups was 3.9 points (p =.001, 95% confidence interval 1.9, 5.9).
An EBM curriculum based on adult learning theory improves residents' EBM skills and certain EBM behaviors. The description and multifaceted evaluation can guide medical educators involved in EBM training.
evidence-based medicine (EBM); curriculum; residents; medical education; adult learning theory
While a variety of instruments have been developed to assess knowledge and skills in evidence based medicine (EBM), few assess all aspects of EBM - including knowledge, skills attitudes and behaviour - or have been psychometrically evaluated. The aim of this study was to develop and validate an instrument that evaluates medical trainees’ competency in EBM across knowledge, skills and attitude.
The ‘Assessing Competency in EBM’ (ACE) tool was developed by the authors, with content and face validity assessed by expert opinion. A cross-sectional sample of 342 medical trainees representing ‘novice’, ‘intermediate’ and ‘advanced’ EBM trainees were recruited to complete the ACE tool. Construct validity, item difficulty, internal reliability and item discrimination were analysed.
We recruited 98 EBM-novice, 108 EBM-intermediate and 136 EBM-advanced participants. A statistically significant difference in the total ACE score was observed and corresponded to the level of training: on a 0-15-point test, the mean ACE scores were 8.6 for EBM-novice; 9.5 for EBM-intermediate; and 10.4 for EBM-advanced (p < 0.0001). Individual item discrimination was excellent (Item Discrimination Index ranging from 0.37 to 0.84), with internal reliability consistent across all but three items (Item Total Correlations were all positive ranging from 0.14 to 0.20).
The 15-item ACE tool is a reliable and valid instrument to assess medical trainees’ competency in EBM. The ACE tool provides a novel assessment that measures user performance across the four main steps of EBM. To provide a complete suite of instruments to assess EBM competency across various patient scenarios, future refinement of the ACE instrument should include further scenarios across harm, diagnosis and prognosis.
Evidence based medicine; Assessment; Medical students
To measure the effectiveness of an educational intervention designed to teach residents four essential evidence-based medicine (EBM) skills: question formulation, literature searching, understanding quantitative outcomes, and critical appraisal.
Firm-based, controlled trial.
Urban public hospital.
Fifty-five first-year internal medicine residents: 18 in the experimental group and 37 in the control group.
An EBM course, taught 2 hours per week for 7 consecutive weeks by senior faculty and chief residents focusing on the four essential EBM skills.
MEASUREMENTS AND MAIN RESULTS
The main outcome measure was performance on an EBM skills test that was administered four times over 11 months: at baseline and at three time points postcourse. Postcourse test 1 assessed the effectiveness of the intervention in the experimental group (primary outcome); postcourse test 2 assessed the control group after it crossed over to receive the intervention; and postcourse test 3 assessed durability. Baseline EBM skills were similar in the two groups. After receiving the EBM course, the experimental group achieved significantly higher postcourse test scores (adjusted mean difference, 21%; 95% confidence interval, 13% to 28%; P < .001). Postcourse improvements were noted in three of the four EBM skill domains (formulating questions, searching, and quantitative understanding [P < .005 for all], but not in critical appraisal skills [P = .4]). After crossing over to receive the educational intervention, the control group achieved similar improvements. Both groups sustained these improvements over 6 to 9 months of follow-up.
A brief structured educational intervention produced substantial and durable improvements in residents' cognitive and technical EBM skills.
evidence-based medicine; clinical trial; graduate medical education; internship and residency
While previous authors have emphasized the importance of integrating and reinforcing evidence-based medicine (EBM) skills in residency, there are few published examples of such curricula. We designed an EBM curriculum to train family practice interns in essential EBM skills for information mastery using clinical questions generated by the family practice inpatient service. We sought to evaluate the impact of this curriculum on interns, residents, and faculty.
Interns (n = 13) were asked to self-assess their level of confidence in basic EBM skills before and after their 2-week EBM rotation. Residents (n = 21) and faculty (n = 12) were asked to assess how often the answers provided by the EBM intern to the inpatient service changed medical care. In addition, residents were asked to report how often they used their EBM skills and how often EBM concepts and tools were used in teaching by senior residents and faculty. Faculty were asked if the EBM curriculum had increased their use of EBM in practice and in teaching.
Interns significantly increased their confidence over the course of the rotation. Residents and faculty felt that the answers provided by the EBM intern provided useful information and led to changes in patient care. Faculty reported incorporating EBM into their teaching (92%) and practice (75%). Residents reported applying the EBM skills they learned to patient care (86%) and that these skills were reinforced in the teaching they received outside of the rotation (81%). All residents and 11 of 12 faculty felt that the EBM curriculum had improved patient care.
To our knowledge, this is the first published EBM curriculum using an individual block rotation format. As such, it may provide an alternative model for teaching and incorporating EBM into a residency program.
Recent surveys suggest few emergency medicine (EM) training programs have formal evidence-based medicine (EBM) or journal club curricula. Our primary objective was to describe the methods of EBM training in Royal College of Physicians and Surgeons of Canada (RCPSC) EM residencies. Secondary objectives were to explore attitudes regarding current educational practices including e-learning, investigate barriers to journal club and EBM education, and assess the desire for national collaboration.
A 16-question survey containing binary, open-ended, and 5-pt Likert scale questions was distributed to the 14 RCPSC-EM program directors. Proportions of respondents (%), median, and IQR are reported.
The response rate was 93% (13/14). Most programs (85%) had established EBM curricula. Curricula content was delivered most frequently via journal club, with 62% of programs having 10 or more sessions annually. Less than half of journal clubs (46%) were led consistently by EBM experts. Four programs did not use a critical appraisal tool in their sessions (31%). Additional teaching formats included didactic and small group sessions, self-directed e-learning, EBM workshops, and library tutorials. 54% of programs operated educational websites with EBM resources. Program directors attributed highest importance to two core goals in EBM training curricula: critical appraisal of medical literature, and application of literature to patient care (85% rating 5 - “most importance”, respectively). Podcasts, blogs, and online journal clubs were valued for EBM teaching roles including creating exposure to literature (4, IQR 1.5) and linking literature to clinical practice experience (4, IQR 1.5) (1-no merit, 5-strong merit). Five of thirteen respondents rated lack of expert leadership and trained faculty educators as potential limitations to EBM education. The majority of respondents supported the creation of a national unified EBM educational resource (4, IQR 1) (1-no support, 5- strongly support).
RCPSC-EM programs have established EBM teaching curricula and deliver content most frequently via journal club. A lack of EBM expert educators may limit content delivery at certain sites. Program directors supported the nationalization of EBM educational resources. A growing usage of electronic resources may represent an avenue to link national EBM educational expertise, facilitating future collaborative educational efforts.
Evidence-based medicine; Medical education; Emergency medicine; E-learning; Journal club
As the overall evidence for the effectiveness of teaching of evidence based medicine (EBM) is not strong, and the impact of cultural and societal influences on teaching method is poorly understood, we undertook a randomised-controlled trial to test the effectiveness and learning satisfaction with two different EBM teaching methods (usual teaching vs. problem based learning (PBL)) for undergraduate medical students.
A mixed methods study that included a randomised-controlled crossover trial with two intervention arms (usual teaching and PBL) and a nested qualitative study with focus groups to explore student perceptions of learning and to assess the effectiveness and utility of the two teaching methods.
All 129 second-year medical students at the University of Hong Kong in 2007.
The main outcomes measures were attitudes towards EBM; personal application and current use of EBM; EBM knowledge; future use of EBM.
PBL was less effective at imparting knowledge than usual teaching consisting of a lecture followed by a group tutorial. After usual teaching students showed improvement in scores for 'attitudes towards EBM', 'personal application and current use of EBM' and 'EBM knowledge, which were not evident after PBL. In contrast to the usual teaching, students found PBL difficult as they lacked the statistical knowledge necessary to support discussion, failed to understand core concepts, and lost direction.
The evidence presented here would suggest that the teaching of EBM within an Asian environment should adopt a format that facilitates both the acquisition of knowledge and encourages enquiry.
The Education in Palliative and End-of-life Care for Emergency Medicine Project (EPEC™-EM) is a comprehensive curriculum in palliative and end-of-life care for emergency providers. We assessed the adaptation of this course to an EM residency program using synchronous and asynchronous learning.
Curriculum adaptation followed Kern’s standardized six-step curriculum design process. Post-graduate year (PGY) 1–4 residents were taught all EPEC™-EM cognitive domains, divided as seven synchronous and seven asynchronous modules. All synchronous modules featured large group didactic lectures and review of EPEC™-EM course materials. Asynchronous modules use only EPEC™-EM electronic course media for resident self-study. Targeted evaluation for EPEC™-EM knowledge objectives was conducted by a prospective case-control crossover study, with synchronous learning serving as the quasi-control, using validated exam tools. We compared de-identified test scores for effectiveness of learning method, using aggregate group performance means for each learning strategy.
Of 45 eligible residents 55% participated in a pre-test for local needs analysis, and 78% completed a post-test to measure teaching method effect. Post-test scores improved across all EPEC™-EM domains, with a mean improvement for synchronous modules of +28% (SD=9) and a mean improvement for asynchronous modules of +30% (SD=18). The aggregate mean difference between learning methods was 1.9% (95% CI −15.3, +19.0). Mean test scores of the residents who completed the post-test were: synchronous modules 77% (SD=12); asynchronous modules 83% (SD=13); all modules 80% (SD=12).
EPEC™-EM adapted materials can improve resident knowledge of palliative medicine domains, as assessed through validated testing of course objectives. Synchronous and asynchronous learning methods appear to result in similar knowledge transfer, feasibly allowing some course content to be effectively delivered outside of large group lectures.