This paper is the first to document knowledge of and attitudes toward EBM in a large group of community-based volunteer faculty from the departments of Internal Medicine, Family Medicine, and Pediatrics. The study compares their characteristics to those of the full-time faculty members, utilizing a novel approach. We used scales to assess how important EBM was to their everyday practice and to self-evaluate their own understanding of EBM terminology. We also used a short test to assess their knowledge about EBM. Our findings are instructive.
Fifty-two percent of primary care volunteer community-based faculty had essentially no background in research, more than a 5-fold difference compared to their full-time and volunteer community-based subspecialty counterparts. Fewer PC faculty (21%) admitted reading about EBM concepts, fewer still (16%) read EBM reviews (as in the ACP Journal Club
POEMS), and they read fewer original clinical studies per month than FT or SS faculty. Respondents' reliance on textbooks and colleagues to answer patient care questions echoes a prior study of Canadian internists.9
PC faculty say they do not incorporate EBM into their teaching on a regular basis, have a lower confidence in their own understanding of statistics, and assigned a lower importance to core EBM skills compared to FT or SS faculty. Despite these attitudinal differences, there was no significant difference in the knowledge scores of PC and SS respondents, and both were significantly lower than those of the FT faculty. After we accounted for other characteristics in the multivariate analyses, specializing in Family Practice—a large proportion of the PC faculty—contributed significantly to the overall EBM score.
Having a research background was the strongest independent predictor of the EBM score, other than being a full-time faculty member. A higher self-evaluation of EBM understanding was also highly predictive of success on the test. On the whole, our regression model predicted 39% of the variability in the EBM score. Variables not accounted for by our model would include a lack of incentive to perform well on the knowledge test, differences in how the respondents received training in EBM, and differences in the test-taking abilities of the respondents.
Three quarters of medical school graduates in 1999 believed their instruction during medical school in EBM was “appropriate” in quantity.10
Green surveyed Internal Medicine program directors in 1998 to characterize EBM curricula in residency programs. About one third of the programs had a free-standing EBM curriculum, and most tried to integrate EBM instruction into a variety of learning situations, including attending rounds (84%) and resident report (82%).11
Studies at individual institutions have shown that these EBM curricular efforts have increased learners' confidence in their critical appraisal skills,12,13
and improved their cognitive and technical skills in EBM.14
Controlled trials using an EBM curriculum as an intervention have shown an impact in learners' ability to appraise articles,15
but have not resolved whether behavior at the point of care can be influenced.16
As EBM moves to the forefront of medical education, it is extolled as a new way to teach medicine.17
However, residents' behavior in providing care likely will mirror the patterns modeled by their mentors. While our full-time faculty may be getting the EBM message, the enlistment of community-based preceptors has raised new questions. Can they teach effectively? What should they be teaching? How and when should their faculty development occur? Family Medicine residencies have utilized voluntary community physicians since the inception of their programs thirty years ago; General Internal Medicine and Pediatrics have done so only more recently.
The Preceptor Education Project of the Society of Teachers of Family Medicine is a project started in the early 1990s that is devoted to increasing the teaching skills of office-based Family Practice preceptors.18
The preceptor manual and workshop materials, widely distributed through most of the 120 Family Practice residencies in this country, discuss how “evidence-based clinical practice” is a particularly productive application of collaborative learning. Concomitantly, there has been a dramatic increase in the discussion of EBM principles in the most prominent peer-reviewed Family Practice journals. Based on a medline
search of the 6 leading Family Practice journals (Family Medicine
, Journal of Family Practice
, Journal of the American Board of Family Practice
, American Family Physician
, Archives of Family Medicine
, and Family Practice Management
) using only the keywords “evidence-based medicine,” there was almost no mention of EBM in 1995, compared to 4.1% of all abstracts mentioning it in 2000 (a 6-year average of 2.2%). This compares very favorably with the frequency of such discussions in prestigious general medical journals (JAMANEJM
, 0.5%) using the same keywords in the same time period. It seems that EBM is beginning to soak into the fabric of family medicine. This major change in emphasis may explain why Family Practice as a specialty was an independent predictor of success on the EBM test in our study.
Not surprisingly, physicians who were farther in years from their training did poorer on the EBM test and were less likely to incorporate EBM into their teaching. Previous studies have shown that the knowledge base of physicians decreases with time, and can be responsive to interventions.19
Particularly important is the fact that the preponderance of EBM knowledge dissemination has occurred since 1990, and the data help identify a high-impact group for faculty development in EBM. In Green's study, only about 45% of the Internal Medicine residency programs provided any faculty development in this area.12
This study has some limitations. First, it took place in only 1 city. It may be that “Family Practice” as a characteristic would not be predictive of EBM knowledge in studies at other institutions. Secondly, this was a questionnaire study and is subject to response bias, as well as to the respondents' ability for self-evaluation. Fortunately, we were able to diminish response bias by achieving a very reasonable response rate (64%). Our knowledge test was able to verify any inflation of the respondents' self-reported understanding of EBM content areas. We suspect, based on the disparity, that this may have been the case among the volunteer subspecialty faculty, who reported a score on EBM understanding as statistically high as the full-time faculty, but who scored lower than the full-time faculty on the knowledge test. Also, there are EBM skills that we did not assess in our study, such as articulating answerable questions, searching for evidence, and applying the evidence in decision making. Finally, not all respondents completed the entire EBM test (91% completed 4 or more of the 7 questions) and far fewer attempted to complete the final 2 short-answer questions. However, given that the respondents are mainly busy volunteer faculty, we were impressed by the number who attempted it at all.
In conclusion, we found that community-based volunteer faculty are not as equipped or motivated to incorporate EBM into their clinical teaching as are full-time faculty. We identified which EBM concepts were not disseminated well into the collective knowledge base of community faculty. We found a few characteristics that were independently associated with having a higher knowledge of EBM concepts, including at least some research background, specializing in Family Practice, and the number of years since residency (a negative predictor). As a whole, this needs assessment should be helpful for faculty development planners who create curricula for their community volunteer faculty in using and teaching basic EBM concepts.