|Home | About | Journals | Submit | Contact Us | Français|
In response to the American Osteopathic Association’s Commission on Osteopathic College Accreditation (COCA) standards set forth in 2008, osteopathic medical schools are restructuring curricula to demonstrate they are teaching the seven core competencies and integrating evidence-based medicine (EBM) throughout all 4 years of training.
To describe and evaluate the efforts of a college of osteopathic medicine to integrate EBM concepts into its curriculum while maintaining existing course content and faculty contact hours.
One-group pre- and posttest study.
Kirksville College of Osteopathic Medicine-A.T. Still University (KCOM) in Missouri.
KCOM course directors in workshop series I (n=20) and KCOM faculty workshop series II (n=14).
A faculty development workshop series based on the diffusion of innovations model was instituted to facilitate cultural change, gain faculty support, and accelerate the implementation of EBM throughout KCOM’s curriculum.
Faculty attitudes, confidence levels, and the number of courses that included instruction of EBM concepts were measured in August 2007 and May 2008.
Faculty attitudes about integrating EBM into the curriculum and confidence levels measured pre- and postworkshop series found that 21 of 26 participants believed they improved their ability to locate primary EBM resources using the Internet; 21 of 28 improved their ability to teach EBM concepts to students. Fifteen of 16 faculty course directors agreed to find ways to incorporate EBM into their classes. Review of KCOM’s course syllabi in April 2009 demonstrated a statistically significant difference (P<.001) in the number of faculty teaching EBM concepts after the faculty development workshop series concluded in March 2008 compared to before the series commenced in March 2006. An unexpected outcome was the implementation of a faculty-conceived, standalone EBM course in fall 2007.
A workshop series based on the diffusion of innovations model is effective in garnering faculty support for the implementation of a change in curriculum that emphasizes EBM content without increasing faculty contact hours or eliminating existing curricular content. A faculty development model emphasizing a “bottom-to-top” approach is effective in achieving workplace culture changes and seamless curricular transitions. Results have shown that a consensus building model is conducive to engaging faculty and garnering its support to effect curricular change, which, ultimately, ensures success.
Osteopathic medical schools are being asked by accrediting boards and related professional societies to demonstrate where within their curriculum they are teaching the seven core competencies1 of osteopathic medicine as well as how they suggest students be evaluated toward mastery of the competencies. Evidence-based medicine (EBM) is a component of competencies 3 (Patient Care) and 6 (Practice-Based Learning and Improvement). To adequately address these competencies, colleges of osteopathic medicine must be able to integrate EBM into all 4 years of their curriculum. Finding room in the curriculum and enlisting faculty support are potential challenges that must be overcome to successfully teach these two competencies. We investigated the use of a faculty-driven model to add EBM content into the existing curriculum without sacrificing existing course content at the Kirksville College of Osteopathic Medicine-A.T. Still University (KCOM) in Missouri.
Attempts by medical schools to integrate EBM into their curriculum have centered predominantly on infusing EBM concepts into fourth year students’ Journal Club, initiating library-centric EBM training, or teaching medical informatics during senior year clerkships vs infusing EBM concepts across all 4 years of medical school.2–7 Because of the limited success of these prior efforts, KCOM elected to use a theoretical model—Rogers’ diffusion of innovation (DOI)8—to guide its approach to curricular revision. “Diffusion is the process through which innovation is communicated through certain channels over time among members of a social system.”8 According to DOI, five distinct phases influence the decision-making process when groups are contemplating adopting an innovation. “Knowledge,” the first stage, takes place when a potential adopter is exposed to an innovation, solicits additional information, and develops an adequate understanding of it. In the subsequent “persuasion” stage, it is essential that those seeking to effect change be client focused rather than innovation centered to compel potential adopters to form a favorable attitude of the innovation.8 “Decision” occurs once an innovation is adopted or rejected. “Implementation,” the fourth stage, signifies that the innovation has been put to use. “Confirmation” denotes the final stage, when the adopter evaluates the innovation’s efficacy and makes a decision to move forward as it is, make modifications, or discontinue its use.8,9
Although effectuating change can sometimes be a lengthy process, DOI concepts have been used with positive results to launch new products and implement scientific innovations in various industries ranging from agriculture to information technology to healthcare.8,10 As part of the grant application process, we hypothesized that 40 students would complete EBM training by the end of Year 2. Therefore, we incorporated the five stages of the DOI model into its strategic plan to achieve expeditious diffusion of EBM throughout the KCOM curriculum.
We devised a series of four workshops based on the five stages of DOI to facilitate adoption of EBM into the existing curriculum. Gaining faculty consensus to adopt EBM concepts throughout the curriculum was not a single act, but a process that occurred over time. People are more apt to adopt innovations when the advantages outweigh the risks, are compatible with adopters’ needs, and are easily implemented.8,11,12 Thus, a faculty needs assessment (Appendix 1) was administered online via Zoomerang software to capture baseline measurements of faculty attitudes about EBM and their confidence to teach EBM concepts before workshop development.
Based on faculty’s feedback from the needs assessment, we created an introductory computer skills laboratory and a series of three workshops based on the three stages of DOI to facilitate adoption of EBM into the existing curriculum: “Knowledge,” “Persuasion,” and “Implementation” (Figure 1). The DOI stages “Decision” and “Confirmation” were not integrated into the workshop because the training series was designed to encourage adoption of the curricular change, which could not take place until faculty arrived at consensus to revise the curriculum. A cogent, multiprong strategy was launched using what research literature labels “consistently effective”13 best practices to disseminate knowledge and facilitate behavioral change among faculty.13–15 Consensus-building opportunities, educational outreach visits by other faculty, and recruitment of influential colleagues who endorsed and taught EBM concepts to champion curricular revision were incorporated into the workshop series design.
The introductory computer skills laboratory covered basic computer skills used in conducting research, document creation, and group communication as well as instruction on directing students in using the computer for classroom and clinical work. Each workshop examined and taught the four components of EBM: asking a clinically relevant question, acquiring appropriate evidence, critically appraising the evidence, and applying the best evidence for clinical practice. The first EBM workshop’s content centered on clarifying the grant’s project goals and objectives, initiating teams of basic scientists and clinicians to establish the common ground of critical thinking between the researchable question of basic science and the clinical question of EBM, and formulating appropriate researchable questions.
Once faculty needs were determined, KCOM and its partner allopathic school, Penn State University College of Medicine (Penn State) in Hershey, Pennsylvania, provided instructors to lead the KCOM workshop series. Instructors educated participating KCOM faculty about the purpose of the curriculum change, bolstered the faculty’s comfort level with the new material, and communicated the importance associated with integrating EBM across KCOM’s 4-year training continuum.
Workshop evaluations were used as an ongoing evaluation of the series and led to minor revisions of the series in subsequent years. Planners administered a 12-item paper-based EBM posttest (Appendix 2) at the end of each training session to evaluate the workshop’s effectiveness and ascertain faculty’s self-reported attitudes toward EBM. Faculty members were surveyed again between 1 and 3 months after completing the entire workshop series. The authors considered any improvement in scores from pre- to posttest as a measure of success and looked for changes in the “Disagree” and “Agree” categories in the Likert scales as an indication of faculty acceptance of EBM. Survey instruments were formulated by an evaluation team that comprised KCOM faculty content experts and consultants from Penn State. This team was independent of the planning team to ensure objectivity. Survey results were subjected to item analysis to determine the effectiveness of each question and to ensure the questions were reliable and valid measures of the intended element within the evaluation instrument.
We perceived that KCOM faculty was more willing to candidly discuss their reactions to the workshop series with neutral third-party consultants rather than with the associate deans or the KCOM administration. Consequently, we arranged for Penn State faculty to lead impartial discussions with KCOM faculty about enhancing the curriculum and sharing ideas how to integrate EBM concepts into course content. In addition, we actuated five quality circles, which are defined in Schriver16 as follows:
Quality Circles are groups or team processes designed to improve work procedures and reduce problems. Periodic staff meetings involve examining and suggesting work improvement to their superiors. Line workers rather than supervisors or mangers usually chair these meetings. Management is committed to taking these meetings seriously.
The team is a cohesive work group with the ability to operate with a substantial degree of autonomy in the areas for which it is responsible.16 The quality circles used in the present study comprised 6 randomly selected clinical and basic science faculty to guide and improve the adoption rate of the EBM process and methodologies, to obtain critical input on faculty development efforts, to identify opportunities for improvement, and to promote consensus building on the need to integrate EBM.
Of the 57 KCOM faculty members, 37 (16 basic science, 15 clinical science, and 6 other) voluntarily completed the pre-workshop needs assessment, for a response rate of 45%. The 2007 and 2008 workshops resulted in faculty establishing consensus that (1) it was important to integrate EBM into the curriculum, and (2) it was crucial to incorporate EBM concepts into courses without replacing current content or adding faculty contact hours. Of 57 on-campus core teaching faculty members in 2007, 20 (35%) completed 13 contact hours of EBM faculty development workshops, as did 14 (29%) of 49 faculty in 2008. Two themes emerged from the data collected after the workshop series: (1) faculty felt more confident in their ability to teach EBM after the training and (2) faculty agreed that infusing EBM concepts without increasing faculty contact hours or diminishing existing course content was necessary to prepare future physicians for research purposes.
Faculty attitudes were measured before and after the workshop series using a 19-item survey instrument. Before the first workshop, 12 (32%) of 37 participating faculty reported they were teaching research design content and EBM-related topics, and 26 (70%) of 37 agreed that additional faculty development regarding EBM concepts and ideas of how to teach EBM were needed. After completion of the first workshop series, 15 (94%) of 16 participating faculty course directors agreed to find ways to incorporate EBM into their courses, thus signaling the adoption of the EBM curricular change initiative.
The DOI stage of sharing knowledge and the relative benefits of infusing EBM in the EMB training series in 2007 and 2008 contributed to a jump in faculty’s confidence levels: 25 (83%) of 30 responding participants indicated they had a clear understanding of EBM; 21 (88%) of 26 believed they had improved their electronic resource capabilities, which would allow them to better instruct students on proper search capabilities regarding EBM; and 21 (75%) of 28 faculty reported they had improved their ability to teach students how to critically appraise research results and apply them to patient care—two important aspects of EBM instruction. Faculty reported that before launching the faculty development workshop series, only 4 (8%) of 48 courses contained EBM content. However, a review of course syllabi revealed that during the 3-year period from 2007 to 2009, faculty consciously integrated EBM exercises and modules into 34 (72%) of 47 courses (Figure 2).
We initiated a curricular mapping tool, which allows one to graphically portray where in the curriculum each element of the EBM process has been integrated into ongoing course material,17 as a quality control measurement to track the infusion of EBM content into courses, identify gaps and instructional redundancies, and document the extent that the four elements of EBM are being taught in each course. The number of faculty teaching EBM concepts in 2008 compared with the number in 2006 substantially jumped from only 5 (9%) of 57 in 2006 to 33 (67%) of 49 in 2008—a statistically significant (P<.001) change (Figure 3). Moreover, without prompting by KCOM administration, faculty petitioned the curriculum committee to reallocate existing contact hours to form a new course that provides students a solid foundation of EBM knowledge and adequately prepares them to apply EBM principles in their subsequent courses and clinical experiences.
We developed a curriculum map to determine the extent of EBM content in each course and steadily assessed student performance to evaluate the effectiveness of EBM instruction after it was institutionalized. On average, 115 of 158 students from the class of 2010 stated they constructed EBM clinical questions weekly for their coursework during their first and second years of medical school. This same group of students was surveyed prior to attending school—67 of 108 respondents stated they had not received instruction in research design content and EBM-related topics.
The current study demonstrates that a series of faculty workshops based on the DOI model can successfully effect a change in curriculum to include EBM training without adding faculty contact hours. Because we studied Rogers’ DOI8 and consulted Penn State instructors who advised us on the use of DOI principles in the planning of the workshop series, we recognized that mandating faculty to immediately infuse EBM concepts into course curriculum would circumvent the fostering of authentic faculty support and create resistance. We opted to cultivate favorable attitudes among faculty by personally engaging them in the curriculum enhancement process and transferring cutting-edge knowledge about the relative benefits8 of incorporating EBM concepts. On completion of the faculty development workshops, faculty responses toward implementing EBM concepts were exceedingly positive.
Application of Rogers’ five stages of DOI was integral to building faculty consensus and accelerated assimilation of EBM content into the curriculum. Consensus was evidenced when 14 newly trained faculty integrated EBM concepts into their courses. These additions impacted an estimated 349 osteopathic medical students, compared to planners’ original Year 2 projection of reaching 40 students.
An unexpected innovation resulting from the workshop series was the formation of a foundational EBM course that was implemented 1 year after the completion of the first workshop. The Fundamentals of EBM course initiated EBM training for 523 students, far beyond the original target of the NIH grant and in less time than originally anticipated. This faculty-initiated, stand-alone course equips students with the latest EBM concepts and prepares them for in-depth EBM exercises that they will encounter in basic science classes and clinical rotations. Additional opportunities to integrate EBM were proposed by faculty and included incorporating EBM into Journal Club activities during rotating students’ third and fourth years of medical school, research electives, and oral case study presentations. Based on advice from Penn State consultants, KCOM also launched a Web-based tool to provide professional development to its community-based faculty to enhance their EBM evaluation skills. The online program teaches clinical preceptors how to apply evaluation criteria using a competency-based rubric to objectively evaluate students’ mastery of EBM skills in their clinical presentations.
Based on assessments of student performance to evaluate the effectiveness of EMB after it was institutionalized, course content has enhanced medical students’ knowledge of EBM concepts, and students possess the knowledge and skills to competently apply EBM concepts to various clinical scenarios. These impressive results demonstrate how faculty’s enthusiasm and competency increased as a result of the faculty development workshop model, which contributed to the improvement of student performance.
Because KCOM sought to be in compliance with the American Osteopathic Association’s Commission on Osteopathic College Accreditation standards, enhancing KCOM’s curriculum to include EBM competencies without increasing faculty contact hours was considered a high priority. We successfully realized curricular change within a relatively brief period by employing DOI principles and developing a strategic framework that fostered faculty support and interdisciplinary collaboration.
The findings from our project coalesce with research literature’s findings and demonstrate that the quality of collegial relationships, methods of dispensing knowledge transfer, and how a call for action is communicated are salient factors that influence an innovation’s rate of adoption or rejection.8,14,18,19 In other words, when DOIs are implemented appropriately, it results in improved collegial relationships and knowledge transfer; when a call to action is communicated appropriately, it influences an innovation’s rate of adoption or rejection.
The consensus building that occurred in workshops led to the creation of a faculty-inspired foundational EBM course for first-year students. This action signaled faculty cohesion and was the impetus that accelerated the curricular revision to include EBM content. The faculty development workshop model fulfilled its purpose; it spurred faculty to expeditiously institutionalize EBM concepts across all 4 years of training without increasing faculty contact hours, and it paved the way for KCOM to demonstrate to the American Osteopathic Association that it provides continuous EBM course content to its medical students.
We thank KCOM faculty for their contributions and commitment to the present study. We also thank Stephanie Willett, MA, the curriculum specialist, for providing data used in the preparation of this study.
This project was made possible by grant number 1 R25 AT003579-01 from the National Center for Complementary and Alternative Medicine.
Disclaimer: The contents of this article are solely the responsibility of the authors and do not necessarily represent the official views of the National Center for Complementary and Alternative Medicine or the National Institutes of Health.
Financial Disclosures: None reported.