To evaluate the educational effectiveness of a clinically integrated e-learning course for teaching basic evidence-based medicine (EBM) among postgraduate medical trainees compared to a traditional lecture-based course of equivalent content.
We conducted a cluster randomized controlled trial to compare a clinically integrated e-learning EBM course (intervention) to a lecture-based course (control) among postgraduate trainees at foundation or internship level in seven teaching hospitals in the UK West Midlands region. Knowledge gain among participants was measured with a validated instrument using multiple choice questions. Change in knowledge was compared between groups taking into account the cluster design and adjusted for covariates at baseline using generalized estimating equations (GEE) model.
There were seven clusters involving teaching of 237 trainees (122 in the intervention and 115 in the control group). The total number of postgraduate trainees who completed the course was 88 in the intervention group and 72 in the control group. After adjusting for baseline knowledge, there was no difference in the amount of improvement in knowledge of EBM between the two groups. The adjusted post course difference between the intervention group and the control group was only 0.1 scoring points (95% CI −1.2–1.4).
An e-learning course in EBM was as effective in improving knowledge as a standard lecture-based course. The benefits of an e-learning approach need to be considered when planning EBM curricula as it allows standardization of teaching materials and is a potential cost-effective alternative to standard lecture-based teaching.
Objective To evaluate the effects of standalone versus clinically integrated teaching in evidence based medicine on various outcomes in postgraduates.
Design Systematic review of randomised and non-randomised controlled trials and before and after comparison studies.
Data sources Medline, Embase, ERIC, Cochrane Library, DARE, HTA database, Best Evidence, BEME, and SCI.
Study selection 23 studies: four randomised trials, seven non-randomised controlled studies, and 12 before and after comparison studies. 18 studies (including two randomised trials) evaluated a standalone teaching method, and five studies (including two randomised trials) evaluated a clinically integrated teaching method.
Main outcome measures Knowledge, critical appraisal skills, attitudes, and behaviour.
Results Standalone teaching improved knowledge but not skills, attitudes, or behaviour. Clinically integrated teaching improved knowledge, skills, attitudes, and behaviour.
Conclusion Teaching of evidence based medicine should be moved from classrooms to clinical practice to achieve improvements in substantial outcomes.
At postgraduate level evidence based medicine (EBM) is currently taught through tutor based lectures. Computer based sessions fit around doctors' workloads, and standardise the quality of educational provision. There have been no randomized controlled trials comparing computer based sessions with traditional lectures at postgraduate level within medicine.
This was a randomised controlled trial involving six postgraduate education centres in the West Midlands, U.K. Fifty five newly qualified foundation year one doctors (U.S internship equivalent) were randomised to either computer based sessions or an equivalent lecture in EBM and systematic reviews. The change from pre to post-intervention score was measured using a validated questionnaire assessing knowledge (primary outcome) and attitudes (secondary outcome).
Both groups were similar at baseline. Participants' improvement in knowledge in the computer based group was equivalent to the lecture based group (gain in score: 2.1 [S.D = 2.0] versus 1.9 [S.D = 2.4]; ANCOVA p = 0.078). Attitudinal gains were similar in both groups.
On the basis of our findings we feel computer based teaching and learning is as effective as typical lecture based teaching sessions for educating postgraduates in EBM and systematic reviews.
Background and objectives
Evidence-based health care requires clinicians to engage with use of evidence in decision-making at the workplace. A learner-centred, problem-based course that integrates e-learning in the clinical setting has been developed for application in obstetrics and gynaecology units. The course content uses the WHO reproductive health library (RHL) as the resource for systematic reviews. This project aims to evaluate a clinically integrated teaching programme for incorporation of evidence provided through the WHO RHL. The hypothesis is that the RHL-EBM (clinically integrated e-learning) course will improve participants' knowledge, skills and attitudes, as well as institutional practice and educational environment, as compared to the use of standard postgraduate educational resources for EBM teaching that are not clinically integrated.
The study will be a multicentre, cluster randomized controlled trial, carried out in seven countries (Argentina, Brazil, Democratic Republic of Congo, India, Philippines, South Africa, Thailand), involving 50-60 obstetrics and gynaecology teaching units. The trial will be carried out on postgraduate trainees in the first two years of their training. In the intervention group, trainees will receive the RHL-EBM course. The course consists of five modules, each comprising self-directed e-learning components and clinically related activities, assignments and assessments, coordinated between the facilitator and the postgraduate trainee. The course will take about 12 weeks, with assessments taking place pre-course and 4 weeks post-course. In the control group, trainees will receive electronic, self-directed EBM-teaching materials. All data collection will be online.
The primary outcome measures are gain in EBM knowledge, change in attitudes towards EBM and competencies in EBM measured by multiple choice questions (MCQs) and a skills-assessing questionniare administered eletronically. These questions have been developed by using questions from validated questionnaires and adapting them to the current course. Secondary outcome measure will be educational environment towards EBM which will be assessed by a specifically developed questionnaire.
The trial will determine whether the RHL EBM (clinically integrated e-leraning) course will increase knowledge, skills and attitudes towards EBM and improve the educational environment as compared to standard teaching that is not clinically integrated. If effective, the RHL-EBM course can be implemented in teaching institutions worldwide in both, low-and middle income countries as well as industrialized settings. The results will have a broader impact than just EBM training because if the approach is successful then the same educational strategy can be used to target other priority clinical and methodological areas.
Although several studies have shown that teaching EBM is effective in improving knowledge, at present, there is no convincing evidence that teaching EBM also changes professional behaviour in practice. Therefore, the primary aim of this study was to evaluate the effectiveness of a clinically integrated post-graduate training programme in EBM on evidence-based disability evaluation.
Methods and Findings
In a cluster randomised controlled trial, fifty-four case-based learning groups consisting of 132 physicians and 1680 patients were randomly assigned to the intervention or control groups. A clinically integrated, post-graduate, 5-day training programme in evidence-based medicine, consisting of (home) assignments, peer teaching, interactive training in searching databases, lectures and brainstorming sessions was provided to the intervention group. The control group received no training. The primary outcome was evidence-based disability evaluation, as indicated by the frequency in use of evidence of sufficient quality in disability evaluation reports. There are no general EBM behaviour outcome measures available. Therefore, we followed general guidelines for constructing performance indicators and defined an a priori cut-off for determination of sufficient quality as recommended for evaluating EB training. Physicians trained in EBM performed more evidence-based disability evaluations compared to physicians in the control group (difference in absolute proportion 9.7%, 95% CI 3.5 to 15.9). The primary outcome differences between groups remained significant after both cluster-adjusted analysis and additional sensitivity analyses accounting for subjects lost to follow-up.
A EBM programme successfully improved the use of evidence in a non-hospital based medical specialty. Our findings support the general recommendations to use multiple educational methods to change physician behaviour. In addition, it appeared important that the professional context of the intervention was very supportive in the sense that searches in databases, using and applying guidelines and other forms of evidence are considered standard practice and are encouraged by colleagues and management.
As the overall evidence for the effectiveness of teaching of evidence based medicine (EBM) is not strong, and the impact of cultural and societal influences on teaching method is poorly understood, we undertook a randomised-controlled trial to test the effectiveness and learning satisfaction with two different EBM teaching methods (usual teaching vs. problem based learning (PBL)) for undergraduate medical students.
A mixed methods study that included a randomised-controlled crossover trial with two intervention arms (usual teaching and PBL) and a nested qualitative study with focus groups to explore student perceptions of learning and to assess the effectiveness and utility of the two teaching methods.
All 129 second-year medical students at the University of Hong Kong in 2007.
The main outcomes measures were attitudes towards EBM; personal application and current use of EBM; EBM knowledge; future use of EBM.
PBL was less effective at imparting knowledge than usual teaching consisting of a lecture followed by a group tutorial. After usual teaching students showed improvement in scores for 'attitudes towards EBM', 'personal application and current use of EBM' and 'EBM knowledge, which were not evident after PBL. In contrast to the usual teaching, students found PBL difficult as they lacked the statistical knowledge necessary to support discussion, failed to understand core concepts, and lost direction.
The evidence presented here would suggest that the teaching of EBM within an Asian environment should adopt a format that facilitates both the acquisition of knowledge and encourages enquiry.
Evidence-Based Medicine (EBM) is an important competency for the healthcare professional. Experimental evidence of EBM educational interventions from rigorous research studies is limited. The main objective of this study was to assess EBM learning (knowledge, attitudes and self-reported skills) in undergraduate medical students with a randomized controlled trial.
The educational intervention was a one-semester EBM course in the 5th year of a public medical school in Mexico. The study design was an experimental parallel group randomized controlled trial for the main outcome measures in the 5th year class (M5 EBM vs. M5 non-EBM groups), and quasi-experimental with static-groups comparisons for the 4th year (M4, not yet exposed) and 6th year (M6, exposed 6 months to a year earlier) groups. EBM attitudes, knowledge and self-reported skills were measured using Taylor’s questionnaire and a summative exam which comprised of a 100-item multiple-choice question (MCQ) test.
289 Medical students were assessed: M5 EBM=48, M5 non-EBM=47, M4=87, and M6=107. There was a higher reported use of the Cochrane Library and secondary journals in the intervention group (M5 vs. M5 non-EBM). Critical appraisal skills and attitude scores were higher in the intervention group (M5) and in the group of students exposed to EBM instruction during the previous year (M6). The knowledge level was higher after the intervention in the M5 EBM group compared to the M5 non-EBM group (p<0.001, Cohen's d=0.88 with Taylor's instrument and 3.54 with the 100-item MCQ test). M6 Students that received the intervention in the previous year had a knowledge score higher than the M4 and M5 non-EBM groups, but lower than the M5 EBM group.
Formal medical student training in EBM produced higher scores in attitudes, knowledge and self-reported critical appraisal skills compared with a randomized control group. Data from the concurrent groups add validity evidence to the study, but rigorous follow-up needs to be done to document retention of EBM abilities.
Evidence-based medicine; Undergraduate medical education; Curriculum development; Educational assessment; Critical appraisal skills
Critical appraisal of scientific literature is an integral part of Evidence Based Medicine (EBM). Many medical practitioners have either limited or no formal education in research and are inadequately prepared to critically analyze the quality of research they are reading. This study presents the instructional strategy, students’ evaluation and the feedback of the undergraduate and postgraduate students on teaching critical appraisal of published medical literature to undergraduate and postgraduate students in the Ziauddin Medical University, Karachi, Pakistan.
Two batches of undergraduate medical students of Year-3 (n = 85) and a group of (n = 18) postgraduate students in basic sciences, community health sciences and family medicine.
After 170 hours of teaching of biostatistics, epidemiology and survey methodology in Year-1 & 2, in Year-3 of undergraduate curriculum, six 2-hour structured sessions for critical appraisal of research articles published in peer reviewed journals were held.
All (N=103) students who took the course appeared in the objective structured practical examination (OSPE), where out of 100 they scored 74.3 ± 9.1. The studentds’ feedback on a 5-point Likert’s scale questionnaire showed the mean of overall satisfaction of the students is 3.93, and appreciation of relevance of quantitative subjects to understand medical literature is 4.89. All respondents agreed and strongly agreed the course helped them appreciate the relevance of quantitative subjects to understanding of medical literature
This course should be considered as the first step in the journey of becoming a competent self learner and should be followed by courses on EBM.
Evidence based medicine; critical appraisal skills
Evidence based medicine (EBM) is considered an integral part of medical training, but integration of teaching various EBM steps in everyday clinical practice is uncommon. Currently EBM is predominantly taught through theoretical courses, workshops and e-learning. However, clinical teachers lack confidence in teaching EBM in workplace and are often unsure of the existing opportunities for teaching EBM in the clinical setting. There is a need for continuing professional development (CPD) courses that train clinical trainers to teach EBM through on-the-job training by demonstration of applied EBM real time in clinical practice. We developed such a course to encourage clinically relevant teaching of EBM in post-graduate education in various clinical environments.
We devised an e-learning course targeting trainers with EBM knowledge to impart educational methods needed to teach application of EBM teaching in commonly used clinical settings. The curriculum development group comprised experienced EBM teachers, clinical epidemiologists, clinicians and educationalists from institutions in seven European countries. The e-learning sessions were designed to allow participants (teachers) to undertake the course in the workplace during short breaks within clinical activities. An independent European steering committee provided input into the process.
The curriculum defined specific learning objectives for teaching EBM by exploiting educational opportunities in six different clinical settings. The e-modules incorporated video clips that demonstrate practical and effective methods of EBM teaching in everyday clinical practice. The course encouraged focussed teaching activities embedded within a trainer's personal learning plan and documentation in a CPD portfolio for reflection.
This curriculum will help senior clinicians to identify and make the best use of available opportunities in everyday practice in clinical situations to teach various steps of EBM and demonstrate their applicability to clinical practice. Once fully implemented, the ultimate outcome of this pilot project will be a European qualification in teaching EBM, which will be used by doctors, hospitals, professional bodies responsible for postgraduate qualifications and continuing medical education.
Evidence Based Medicine (EBM) is a core unit delivered across many medical schools. Few studies have investigated the most effective method of teaching a course in EBM to medical students. The objective of this study was to identify whether a blended-learning approach to teaching EBM is more effective a didactic-based approach at increasing medical student competency in EBM.
A mixed-methods study was conducted consisting of a controlled trial and focus groups with second year graduate medical students. Students received the EBM course delivered using either a didactic approach (DID) to learning EBM or a blended-learning approach (BL). Student competency in EBM was assessed using the Berlin tool and a criterion-based assessment task, with student perceptions on the interventions assessed qualitatively.
A total of 61 students (85.9%) participated in the study. Competency in EBM did not differ between the groups when assessed using the Berlin tool (p = 0.29). Students using the BL approach performed significantly better in one of the criterion-based assessment tasks (p = 0.01) and reported significantly higher self-perceived competence in critical appraisal skills. Qualitative analysis identified that students had a preference for the EBM course to be delivered using the BL approach.
Implementing a blended-learning approach to EBM teaching promotes greater student appreciation of EBM principles within the clinical setting. Integrating a variety of teaching modalities and approaches can increase student self-confidence and assist in bridging the gap between the theory and practice of EBM.
Evidence based medicine; Blended learning; Graduate medical education; Pedagogy
Health care professionals worldwide attend courses and workshops to learn evidence-based medicine (EBM), but evidence regarding the impact of these educational interventions is conflicting and of low methodologic quality and lacks generalizability. Furthermore, little is known about determinants of success. We sought to measure the effect of EBM short courses and workshops on knowledge and to identify course and learner characteristics associated with knowledge acquisition.
Health care professionals with varying expertise in EBM participated in an international, multicentre before–after study. The intervention consisted of short courses and workshops on EBM offered in diverse settings, formats and intensities. The primary outcome measure was the score on the Berlin Questionnaire, a validated instrument measuring EBM knowledge that the participants completed before and after the course.
A total of 15 centres participated in the study and 420 learners from North America and Europe completed the study. The baseline score across courses was 7.49 points (range 3.97–10.42 points) out of a possible 15 points. The average increase in score was 1.40 points (95% confidence interval 0.48–2.31 points), which corresponded with an effect size of 0.44 standard deviation units. Greater improvement in scores was associated (in order of greatest to least magnitude) with active participation required of the learners, a separate statistics session, fewer topics, less teaching time, fewer learners per tutor, larger overall course size and smaller group size. Clinicians and learners involved in medical publishing improved their score more than other types of learners; administrators and public health professionals improved their score less. Learners who perceived themselves to have an advanced knowledge of EBM and had prior experience as an EBM tutor also showed greater improvement than those who did not.
EBM course organizers who wish to optimize knowledge gain should require learners to actively participate in the course and should consider focusing on a small number of topics, giving particular attention to statistical concepts.
Postgraduate medical education and training in many specialties, including Clinical Radiology, is undergoing major changes. In part this is to ensure that shorter training periods maximise the learning opportunities but it is also to bring medical education in line with broader educational theory. Learning outcomes need to be defined so that there is no doubt what knowledge, skills, attitudes and behaviours are expected of those in training. Curricula should be developed into competency or outcome based models and should state the aims, objectives, content, outcomes and processes of a training programme. They should include a description of the methods of learning, teaching, feedback and supervision. Assessment systems must be matched to the curriculum and must be fair, reliable and valid. Workplace based assessments including the use of multisource feedback need to be developed and validated for use during radiology training. These should be used in a formative and developmental way, although the overall results from a series of such assessments can be used in a more summative way to determine progress to the next phase of training. Formal standard setting processes need to be established for ‘high stakes’ summative assessments such as examinations. In addition the unique skills required of a radiologist in terms of image interpretation, pattern recognition, deduction and diagnosis need to be evaluated in robust, reliable and valid ways. Through a combination of these methods we can be assured that decisions about trainees’ progression through training is fair and standardised and that we are protecting patients by establishing national standards for training, curricula and assessment methods.
Postgraduate; radiology; training; education
Evidence-based medicine (EBM) is increasingly taught in medical schools, but few curricula have been evaluated using validated instruments.
To evaluate a longitudinal medical school EBM curriculum using a validated instrument.
Design, Participants, Measurements
We evaluated EBM attitudes and knowledge of 32 medical students as they progressed through an EBM curriculum. The first part was an EBM “short course” with didactic and small-group sessions occurring at the end of the second year. The second part integrated EBM assignments with third-year clinical rotations. The validated 15-item Berlin Questionnaire was administered before the course, after the short course, and at the end of the third year.
EBM knowledge scores increased from baseline by 2.8 points at the end of the second year portion of the course (p = .0001), and by 3.7 points at the end of the third year (p < .0001). Self-rated EBM knowledge increased from baseline by 0.8 and 1.1 points, respectively (p = .0006 and p < .0001, respectively). EBM was felt to be of high importance for medical education and clinical practice at all time points, peaking after the short course.
A longitudinal medical school EBM curriculum was associated with increased EBM knowledge. This knowledge increase was sustained throughout the curriculum.
Electronic supplementary material
The online version of this article (doi:10.1007/s11606-008-0625-x) contains supplementary material, which is available to authorized users.
medical education; evidence-based medicine; medical school
Learning styles determine how people manage new information. Evidence-based medicine (EBM) involves the management of information in clinical practice. As a consequence, the way in which a person uses EBM can be related to his or her learning style. In order to tailor EBM education to the individual learner, this study aims to determine whether there is a relationship between an individual's learning style and EBM competence (knowledge/skills, attitude, behaviour).
In 2008, we conducted a survey among 140 novice GP trainees in order to assess their EBM competence and learning styles (Accommodator, Diverger, Assimilator, Converger, or mixed learning style).
The trainees' EBM knowledge/skills (scale 0-15; mean 6.8; 95%CI 6.4-7.2) were adequate and their attitudes towards EBM (scale 0-100; mean 63; 95%CI 61.3-64.3) were positive. We found no relationship between their knowledge/skills or attitudes and their learning styles (p = 0.21; p = 0.19). Of the trainees, 40% used guidelines to answer clinical questions and 55% agreed that the use of guidelines is the most appropriate way of applying EBM in general practice. Trainees preferred using evidence from summaries to using evidence from single studies. There were no differences in medical decision-making or in EBM use (p = 0.59) for the various learning styles. However, we did find a link between having an Accommodating or Converging learning style and making greater use of intuition. Moreover, trainees with different learning styles expressed different ideas about the optimal use of EBM in primary care.
We found that EBM knowledge/skills and EBM attitudes did not differ with respect to the learning styles of GP trainees. However, we did find differences relating to the use of intuition and the trainees' ideas regarding the use of evidence in decision-making.
The challenge of imparting a large amount of knowledge within a limited time period in a way it is retained, remembered and effectively interpreted by a student is considerable. This has resulted in crucial changes in the field of medical education, with a shift from didactic teacher centered and subject based teaching to the use of interactive, problem based, student centered learning. This study tested the hypothesis that learning styles (visual, auditory, read/write and kinesthetic) and approaches to learning (deep, strategic and superficial) differ among first and final year undergraduate medical students, and postgraduates medical trainees.
We used self administered VARK and ASSIST questionnaires to assess the differences in learning styles and approaches to learning among medical undergraduates of the University of Colombo and postgraduate trainees of the Postgraduate Institute of Medicine, Colombo.
A total of 147 participated: 73 (49.7%) first year students, 40 (27.2%) final year students and 34(23.1%) postgraduate students. The majority (69.9%) of first year students had multimodal learning styles. Among final year students, the majority (67.5%) had multimodal learning styles, and among postgraduates, the majority were unimodal (52.9%) learners.
Among all three groups, the predominant approach to learning was strategic. Postgraduates had significant higher mean scores for deep and strategic approaches than first years or final years (p < 0.05). Mean scores for the superficial approach did not differ significantly between groups.
The learning approaches suggest a positive shift towards deep and strategic learning in postgraduate students. However a similar difference was not observed in undergraduate students from first year to final year, suggesting that their curriculum may not have influenced learning methodology over a five year period.
Learning methods; Medicine; Post graduate; Undergraduate curriculum evaluation
Evidence-based medicine (EBM) is an indispensable tool in clinical practice. Teaching and training of EBM to trainee clinicians is patchy and fragmented at its best. Clinically integrated teaching of EBM is more likely to bring about changes in skills, attitudes and behaviour. Provision of evidence-based health care is the most ethical way to practice, as it integrates up-to-date, patient-oriented research into the clinical decision making process, thus improving patients' outcomes. In this article, we aim to dispel the myth that EBM is an academic and statistical exercise removed from practice by providing practical tips for teaching the minimum skills required to ask questions and critically identify and appraise the evidence and presenting an approach to teaching EBM within the existing clinical and educational training infrastructure.
Evidence-based medicine (EBM) has been widely integrated into residency curricula, although results of randomized controlled trials and long term outcomes of EBM educational interventions are lacking. We sought to determine if an EBM workshop improved internal medicine residents' EBM knowledge and skills and use of secondary evidence resources.
This randomized controlled trial included 48 internal medicine residents at an academic medical center. Twenty-three residents were randomized to attend a 4-hour interactive workshop in their PGY-2 year. All residents completed a 25-item EBM knowledge and skills test and a self-reported survey of literature searching and resource usage in their PGY-1, PGY-2, and PGY-3 years.
There was no difference in mean EBM test scores between the workshop and control groups at PGY-2 or PGY-3. However, mean EBM test scores significantly increased over time for both groups in PGY-2 and PGY-3. Literature searches, and resource usage also increased significantly in both groups after the PGY-1 year.
We were unable to detect a difference in EBM knowledge between residents who did and did not participate in our workshop. Significant improvement over time in EBM scores, however, suggests EBM skills were learned during residency. Future rigorous studies should determine the best methods for improving residents' EBM skills as well as their ability to apply evidence during clinical practice.
Evidence-based medicine (EBM) is an important element of medical education. However, limited information is available on effective curricula.
To evaluate a longitudinal medical school EBM curriculum using validated instruments.
DESIGN, PARTICIPANTS, MEASUREMENTS
We evaluated EBM attitudes and knowledge of medical students as they progressed through an EBM curriculum. The first component of the curriculum was an EBM “short course” with didactic and small-group sessions occurring at the end of the second year. The second component integrated EBM assignments with third-year clinical rotations. The 15-point Berlin Questionnaire was administered before the course in 2006 and 2007, after the short course, and at the end of the third year. The 212-point Fresno Test was administered before the course in 2007 and 2008, after the short course, and at the end of the third year. Self-reported knowledge and attitudes were also assessed in all three classes of medical students.
EBM knowledge scores on the 15-point Berlin Questionnaire increased from baseline by 3.0 points (20.0%) at the end of the second year portion of the course (p < 001) and by 3.4 points (22.7%) at the end of the third year (p < 001). EBM knowledge scores on the 212-point Fresno Test increased from baseline by 39.7 points (18.7%) at the end of the second year portion of the course (p < 001) and by 54.6 points (25.8%) at the end of the third year (p < 001). On a 5-point scale, self-rated EBM knowledge increased from baseline by 1.0 and 1.4 points, respectively (both p < 001). EBM was felt to be of high importance for medical education and clinical practice at all time points, with increases noted after both components of the curriculum.
A longitudinal medical school EBM was associated with markedly increased EBM knowledge on two validated instruments. Both components of the curriculum were associated with gains in knowledge. The curriculum was also associated with increased perceptions of the importance of EBM for medical education and clinical practice.
Electronic supplementary material
The online version of this article (doi:10.1007/s11606-011-1642-8) contains supplementary material, which is available to authorized users.
medical education; evidence-based medicine; medical school
Several studies have evaluated whether evidence-based medicine (EBM) training courses can improve skills such as literature searching and critical appraisal but to date, few data exist on whether teaching EBM skills and providing evidence-based resources result in change in behavior or clinical outcomes. This study was conducted to evaluate whether a multifaceted EBM intervention consisting of teaching EBM skills and provision of electronic evidence resources changed clinical practice.
The medical inpatient units at a district general hospital.
Thirty-five attending physicians and 12 medicine residents.
A multicomponent EBM intervention was provided including an EBM training course of seven 1-hour sessions, an EBM syllabus and textbook, and provision of evidence-based resources on the hospital network.
MEASUREMENTS AND MAIN RESULTS
The primary outcome of the study was the quality of evidence in support of therapies initiated for the primary diagnoses in 483 consecutive patients admitted during the month before and the month after the intervention. Patients admitted after implementation of the EBM intervention were significantly more likely to receive therapies proven to be beneficial in randomized controlled trials (62% vs 49%; P = .016). Of these trial-proven therapies, those offered after the EBM intervention were significantly more likely to be based on high-quality randomized controlled trials (95% vs 87%; P = .023).
A multifaceted intervention designed to teach and support EBM significantly improved evidence-based practice patterns in a district general hospital.
evidence-based medicine; medical education; practice of medicine
Teaching the steps of evidence-based medicine (EBM) to undergraduate as well as postgraduate health care professionals is crucial for implementation of effective, beneficial health care practices and abandonment of ineffective, harmful ones. Stellenbosch University in Cape Town, South Africa, offers a 12-week, completely online module on EBM within the Family Medicine division, to medical specialists in their first year of training. The aim of this study was to formatively evaluate this module; assessing both the mode of delivery; as well as the perceived effectiveness and usefulness thereof.
We used mixed methods to evaluate this module: A document review to assess whether the content of the module reflects important EBM competencies; a survey of the students to determine their experiences of the module; and semi-structured interviews with the tutors to explore their perspectives of the module. Ethics approval was obtained.
The document review indicated that EBM competencies were covered adequately, although critical appraisal only focused on randomised controlled trials and guidelines. Students had a positive attitude towards the module, but felt that they needed more support from the tutors. Tutors felt that students engaged actively in discussions, but experienced difficulties with understanding certain concepts of EBM. Furthermore, they felt that it was challenging explaining these via the online learning platform and saw the need to incorporate more advanced technology to better connect with the students. In their view the key to successful learning of EBM was to keep it relevant and applicable to everyday practice. Tutors also felt that an online module on EBM was advantageous, since doctors from all over the world were able to participate.
Our study has shown that the online module on EBM was effective in increasing EBM knowledge and skills of postgraduate students and was well received by both students and tutors. Students and tutors experienced generic challenges that accompany any educational intervention of EBM (e.g. understanding difficult concepts), but in addition had to deal with challenges unique to the online learning environment. Teachers of EBM should acknowledge these so as to enhance and successfully implement EBM teaching and learning for all students.
Evidence-based medicine; Postgraduate; Online learning; Evaluation
Aims: In order to develop the e-learning teaching material for medical professionals who are not physicians, we compared solution-based interactive and reading-based reproductive learning with regard to the increase of knowledge. Furthermore we tried to identify additional factors influencing learning.
Methods: We used a quasi-experimental double-stage study design with pre-test (point of time t1), intervention and post-test (point of time t2). The classification into three comparable groups was carried out according to the pre-test results. The interactive and reproductive group participated in the intervention but not the control group. All three groups consisted of graduates of medical schools (N=150) and more experienced physiotherapists during continuing training (N=66). The increase of knowledge was assessed by the post-test. The analysis of variance was the most important statistical tool.
Results: Interactive learning generated a higher increase of knowledge than reproductive learning but was more time consuming. The two groups which participated in the intervention obtained better results than the control group. The level of education and the prior knowledge also influenced the post-test results.
Conclusion: We recommend combining interactive and reproductive approaches for designing the e-learning platform.
interactive learning; reproductive learning; analysis of variance; continuing professional development (CPD); anatomy
Introduction: Evidence-based medicine (EBM) has become increasingly important in the practice of gastroenterology and endoscopy, and the training of future gastroenterology physicians. The objectives were to assess the attitudes/opinions of gastroenterology specialists towards EBM, and evaluate possible gaps in education for certain EBM-related concepts.
Methods: An internet-based survey was emailed to 4073 gastroenterology specialists. The main outcome measurements were physicians’ endorsement of EBM, impact of EBM on clinical practice, utilization of EBM-specific resources, self-assessed understanding of EBM concepts (EBM familiarity score), and actual knowledge of EBM concepts (EBM competency score).
Results: A total of 337 gastroenterology specialists participated. On a sale of 1–10, there was widespread agreement that EBM improves patient care (median score = 9, interquartile range (IQR) = 7–10), and physicians should be familiar with techniques for critical appraisal of studies (median = 9, IQR = 8–10). Most (64.0%) utilized the EBM-related resource UpToDate™ regularly, as opposed to PubMed™ (47.1%) or Clinical Evidence™ (5.4%). The mean EBM familiarity score was 3.4 ± 0.6 on a scale of 1–4. Out of a maximum 49 points, the mean EBM competency score was 35 ± 4.9. There was poor concordance among EBM familiarity and competency scores (r = 0.161; p = 0.005). Academic practice (p < 0.001), research/teaching (p < 0.001), advanced degree (p = 0.012), and recent EBM training (p = 0.001) were all associated with improved EBM competency.
Conclusion: The attitudes and opinions of EBM are extremely favorable among gastroenterology physicians. Although gastroenterology physicians report familiarity with most EBM-related concepts, there is poor correlation with their actual knowledge of EBM. Further educational initiatives should be undertaken to address methods in which EBM skills are reinforced among all gastroenterology practitioners.
clinical practice; evidence-based medicine; survey
Over the last years key stake holders in the healthcare sector have increasingly recognised evidence based medicine (EBM) as a means to improving the quality of healthcare. However, there is considerable uncertainty about the best way to disseminate basic knowledge of EBM. As a result, huge variation in EBM educational provision, setting, duration, intensity, content, and teaching methodology exists across Europe and worldwide. Most courses for health care professionals are delivered outside the work context ('stand alone') and lack adaptation to the specific needs for EBM at the learners' workplace. Courses with modern 'adaptive' EBM teaching that employ principles of effective continuing education might fill that gap. We aimed to develop a course for post-graduate education which is clinically integrated and allows maximum flexibility for teachers and learners.
A group of experienced EBM teachers, clinical epidemiologists, clinicians and educationalists from institutions from eight European countries participated. We used an established methodology of curriculum development to design a clinically integrated EBM course with substantial components of e-learning. An independent European steering committee provided input into the process.
We defined explicit learning objectives about knowledge, skills, attitudes and behaviour for the five steps of EBM. A handbook guides facilitator and learner through five modules with clinical and e-learning components. Focussed activities and targeted assignments round off the learning process, after which each module is formally assessed.
The course is learner-centred, problem-based, integrated with activities in the workplace and flexible. When successfully implemented, the course is designed to provide just-in-time learning through on-the-job-training, with the potential for teaching and learning to directly impact on practice.
Training in Evidence-Based Practice (EBP) has been widely implemented throughout medical school and residency curricula. The aim of this study is to systematically review studies that assessed the effectiveness of EBP teaching to improve knowledge, skills, attitudes and behavior of postgraduate healthcare workers, and to describe instruments available to evaluate EBP teaching.
The design is a systematic review of randomized, non-randomized, and before-after studies. The data sources were MEDLINE, Cochrane Library, EMBASE, CINAHL and ERIC between 1966 and 2006. Main outcomes were knowledge, skills, attitudes and behavior towards EBP. Standardized effect sizes (E-S) were calculated. The E-S was categorized as small (E-S < 0.2), small to moderate (E-S between 0.2 and 0.5), moderate to large (E-S between 0.51 and 0.79), large (E-S > 0.79). Reliability and validity of instruments for evaluating education were assessed. Studies excluded were those that were not original, performed in medical students, focused on prescribing practices, specific health problems, theoretical reviews of different components of EBP, continuing medical education, and testing the effectiveness of implementing guidelines.
Twenty-four studies met our inclusion criteria. There were 15 outcomes within the 10 studies for which E-S could be calculated. The E-S ranged from 0.27 (95%CI: -0.05 to 0.59) to 1.32 (95%CI: 1.11 to 1.53). Studies assessing skills, behavior and/or attitudes had a "small to moderate" E-S. Only 1 of the 2 studies assessing knowledge had E-S of 0.57 (95 CI: 0.32 to 0.82) and 2 of the 4 studies that assessed total test score outcomes had "large" E-S. There were 22 instruments used, but only 10 had 2 or more types of validity or reliability evidence.
Small improvements in knowledge, skills, attitudes or behavior are noted when measured alone. A large improvement in skills and knowledge in EBP is noted when measured together in a total test score. Very few studies used validated measures tests.
To measure the effectiveness of an educational intervention designed to teach residents four essential evidence-based medicine (EBM) skills: question formulation, literature searching, understanding quantitative outcomes, and critical appraisal.
Firm-based, controlled trial.
Urban public hospital.
Fifty-five first-year internal medicine residents: 18 in the experimental group and 37 in the control group.
An EBM course, taught 2 hours per week for 7 consecutive weeks by senior faculty and chief residents focusing on the four essential EBM skills.
MEASUREMENTS AND MAIN RESULTS
The main outcome measure was performance on an EBM skills test that was administered four times over 11 months: at baseline and at three time points postcourse. Postcourse test 1 assessed the effectiveness of the intervention in the experimental group (primary outcome); postcourse test 2 assessed the control group after it crossed over to receive the intervention; and postcourse test 3 assessed durability. Baseline EBM skills were similar in the two groups. After receiving the EBM course, the experimental group achieved significantly higher postcourse test scores (adjusted mean difference, 21%; 95% confidence interval, 13% to 28%; P < .001). Postcourse improvements were noted in three of the four EBM skill domains (formulating questions, searching, and quantitative understanding [P < .005 for all], but not in critical appraisal skills [P = .4]). After crossing over to receive the educational intervention, the control group achieved similar improvements. Both groups sustained these improvements over 6 to 9 months of follow-up.
A brief structured educational intervention produced substantial and durable improvements in residents' cognitive and technical EBM skills.
evidence-based medicine; clinical trial; graduate medical education; internship and residency