While the above findings support the acceptance and perceived utility of our EBM program, they do not provide a formal measure of its effectiveness. Changes in resident knowledge, skills and confident were measured over a short period of time. The frequency of reinforcement of EBM and the impact of EBM on clinical care was by resident and faculty report and may have been biased. No attempt was made to observe changes in physician behaviors or patient outcomes. There were no measures of EBM use prior to the introduction of the EBM rotation and no comparison group was available. It is therefore not possible to objectively determine to what degree current levels of awareness and utilization of EBM are the result of the rotation. It is also not possible to separate out the effects of the 2-week EBM rotation from the adjunct changes of establishing a medical information website or promoting the use of PDAs, except to the extent that the questions asked specifically about the 2-week EBM rotation.
Our block EBM rotation differs substantially from the approaches reported in previous studies [9
] in that it utilizes a concentrated, individual experience that combines formal learning (via tutorials and a web-based course) with immediate application of EBM to answer an important clinical questions for individual patients – the target goal for EBM training. An advantage of our approach is the ability to tailor the curriculum to the background and needs of each intern and to provide interns with dedicated time during which they can rapidly acquire EBM knowledge and skills and apply them to "real time" clinical questions under the supervision of a faculty member and a librarian information specialist. We believe that our curriculum could be implemented by any faculty member with a working knowledge of EBM. The website tutorial [15
] provides the core didactic portion for the rotation. Moreover, directing the rotation naturally increases the experience and expertise of the faculty member involved.
Interns reported greater confidence in their search skills after the two week rotation. Two randomized controlled trials and a controlled before-after study have demonstrated benefits of training in electronic search skills [17
]. Interns also reported significantly greater confidence in their ability to apply EBM knowledge and principals in critical appraisal. This is consistent with previous studies that have found training improves critical appraisal skills for residents and practicing physicians [20
One study that evaluated the impact of a 1-month pilot program to use EBM methods on an inpatient service, reported results roughly similar to ours [21
]. Our curriculum also includes the archiving of EBM answers in a standardized format for future reference by residents and faculty. While our study did not include an evaluation of the usefulness of the archived EBM answers generated by the rotation, archiving EBM answers for web-based access has been shown to provide a useful resource for resident physicians in an internal medicine residency program [22
Previous studies have found only small increases in residents' knowledge and skills from journal clubs alone, leading to the suggestion that journal clubs should be used as a component of EBM training rather then being 'stand alone' activities [6
]. While we did not attempt to evaluate the effects of the journal club per se, including it as a component of our EBM curriculum is concordant with this suggestion.
Other investigators have reported educational interventions aimed at improving faculty knowledge and skills in medical informatics and EBM. One study reported increases in faculty self-rating of EBM skills following an intervention consisting of 2 half-day workshops and substantial amount of individual mentoring [23
]. We found that faculty reported our EBM rotation has increased their use of EBM in their clinical practice, as well as their teaching of EBM; this was echoed by residents. While it is difficult to compare the two studies, our findings suggest that integrating EBM into the residency via resident training may improve faculty application of EBM to clinical care and in their teaching, and may be a cost-effective way to reinforce faculty EBM skills.
Our EBM curriculum, based on an individual 2-week EBM rotation in the first year, appears to be successful in increasing resident's EBM skills and confidence. In addition, resident and faculty both perceive EBM as being incorporated and reinforced beyond the rotation, and that the presence of EBM in the residency improves the quality of patient care. We hope that our experience provides a useful model for teaching and integrating EBM into a busy, resource-limited, family practice residency.