|Home | About | Journals | Submit | Contact Us | Français|
Despite increased demand for new curricula in medical education, most academic medical centers have few faculty with training in curriculum development.
To describe and evaluate a longitudinal mentored faculty development program in curriculum development.
A 10-month curriculum development program operating one half-day per week of each academic year from 1987 through 2003. The program was designed to provide participants with the knowledge, attitudes, skills, and experience to design, implement, evaluate, and disseminate curricula in medical education using a 6-step model.
One-hundred thirty-eight faculty and fellows from Johns Hopkins and other institutions and 63 matched nonparticipants.
Pre- and post-surveys from participants and nonparticipants assessed skills in curriculum development, implementation, and evaluation, as well as enjoyment in curriculum development and evaluation. Participants rated program quality, educational methods, and facilitation in a post-program survey.
Sixty-four curricula were produced addressing gaps in undergraduate, graduate, or postgraduate medical education. At least 54 curricula (84%) were implemented. Participant self-reported skills in curricular development, implementation, and evaluation improved from baseline (p<.0001), whereas no improvement occurred in the comparison group. In multivariable analyses, participants rated their skills and enjoyment at the end of the program significantly higher than nonparticipants (all p<.05). Eighty percent of participants felt that they would use the 6-step model again, and 80% would recommend the program highly to others.
This model for training in curriculum development has long-term sustainability and is associated with participant satisfaction, improvement in self-rated skills, and implementation of curricula on important topics.
Medical education requires ongoing curriculum development (CD) to incorporate new knowledge and competencies.1 The Liaison Committee on Medical Education, the Accreditation Council for Graduate Medical Education (ACGME), and the American Academy of Continuing Medical Education have called for curricular changes to enhance the ability of physicians to fulfill their societal contract of providing quality medical care.2–11 Currently, medical schools and residency programs are required to define learning objectives and methods for their trainees.8,9 Soon, they will be expected to demonstrate the attainment of program objectives and trainee competence.11
Despite demands for curricular change, most faculty in academic medical centers have no instruction in education or CD.12,13 Curricula produced are often suboptimal and do not follow CD principles.14 Faculty development programs that have arisen to address this need vary in intensity, duration, and focus.12,13,15–22 Most of them aim to improve teaching skills of their faculty.13 Few publications describe faculty training in CD. Those that do focus only partially on CD16,19,20 or include few participants.20–22 This paper describes and evaluates a 10-month mentored program in existence since 1987, which trains faculty and fellows in CD skills.
The Longitudinal Program in Curriculum Development (CD) was established in 1987 as part of the Johns Hopkins Faculty Development Program at the Johns Hopkins Bayview Medical Center. The Faculty Development Program also includes teaching skills,17,18 consultation, and facilitator training programs.
The CD program serves faculty and fellows from Johns Hopkins and other medical institutions within the geographic region. Applicants self-select to participate. In general, no specific criteria are used for accepting applicants other than involvement in medical education and developing or joining a curricular project deemed acceptable to program facilitators. Participation is required for fellows in the clinician-educator track of the general internal medicine fellowship. Due to grant stipulations, the program trained mainly general internal medicine faculty and fellows during the years of our study. The number of participants from other specialties has increased in recent years, a trend facilitated by the initiation of program tuition and the program becoming recognized as a resource for the entire School of Medicine.
The goals of the program are for participants to: (1) develop the knowledge, attitudes, and skills to design, implement, evaluate, and disseminate a curriculum in medical education; and (2) design, pilot, implement, evaluate, write up, and present a curriculum.
The program operates one half-day per week from early September to mid June of each academic year. Components include:
The Bureau of Health Professions, Health Resources, and Services Administration, U.S. Public Health Service provided grant support in all but 1 year from 1987 to 2006. In 1993, tuition was initated to partly cover expenses. In all years, the Johns Hopkins Bayview Medical Center has provided meeting space and approximately 30% full-time equivalent administrative support. Beginning in 2006, the program became financially independent. We adjusted tuition to cover faculty and food expenses and a 15% continuing education office charge. Two or three faculty experienced in CD, teaching skills, and mentoring facilitate each year depending on the number of curricular projects. Salary support is based on activities provided by faculty members: 5% FTE each for facilitating group sessions, 1.5% FTE for each curriculum mentored, and 4% FTE for one faculty to administer the program.
Five of the 152 participants who enrolled in the program from 1988 through 2003 (cohorts 1–16) dropped out, and two participated twice, leaving 145 individuals who completed the course. Each of the 64 program completers in cohorts 2–9 named a colleague they knew during the first session who was similar to themselves in age, gender, professional training, and professional status. These nonparticipants served as a comparison group for self-assessed CD skills and enjoyment. We recruited nonparticipants until 64 were obtained, providing 80% power to detect a difference of 0.5 on Likert-scaled questions (α=0.05).
A preprogram survey asked the participants (cohorts 1–16) and nonparticipants (cohorts 2–9) to indicate their age, gender, professional training, work setting, academic activities, and prior experience in CD. Pre- and post-surveys ascertained participant and nonparticipant self-assessments of their skills in CD, implementation, and evaluation, as well as their enjoyment in CD and evaluation. Participants completed questionnaires during the first and last workshop sessions. Nonparticipants received mailed questionnaires within 3 months of their matched participant’s first and last curricular sessions. We rated skills on a 6-point scale (0=none, 5=excellent) and enjoyment on a 4-point scale (0=none, 3=a lot).
In a post-program survey, participants in all 16 cohorts rated the program’s quality, methods,and facilitation on 5-point scales. Open-ended questions asked the participants to assess the program’s strengths and weaknesses and to comment on constraints that might inhibit their curricular work.
We determined whether curricular projects for all 16 cohorts had been implemented through contact with participants or direct knowledge of the facilitators. We obtained characteristics of curricula (topics, targeted learners) from project titles, facilitators, and written products of the project teams. We confirmed all publications in medical journals by PubMed searches.
The Johns Hopkins IRB determined that this project qualified for exemption from review under guidelines regarding education program evaluation.
We calculated baseline characteristics by percentage. To assess whether participants and nonparticipants in cohorts 2–9 differed in baseline characteristics, or pre- or post-program self-reported skills and enjoyment, we performed Student’s t tests, Fisher’s exact tests, or chi-squared analyses. We performed multiple linear regression analyses to determine whether participants differed from nonparticipants in baseline and end-of-program self-assessment of skills and enjoyment in CD activities while controlling for baseline characteristics that were statistically different between groups. Baseline covariates included type of fellowship training; academic appointment; time spent in hospital-based practice, community-based practice, teaching, and administration; research salary support; and previous experience in evaluating curricula. We used ANCOVA to compare change scores from baseline between participants and nonparticipants while controlling for baseline characteristics that differed between groups.
Two study investigators (AG, DK) independently analyzed responses to open-ended questions, marking comments representing discrete thoughts. The resulting comments were separated into categories based on the actual words used by participants. Consensus was reached by discussion.
Of the 145 participants in cohorts 1–16, 138 (95.2%) completed the pre- and post-questionnaires. For cohorts 2–9, 63 (98.4%) participants and 63 (98.4%) nonparticipants completed pre- and post-questionnaires. Baseline characteristics for cohort 1–16 and cohort 2–9 participants were similar except that cohort 1–16 participants were more likely to be women and to have associate professor/professor academic appointments. Baseline characteristics for cohort 2–9 participants versus nonparticipants are displayed in Table 1. Participants were more likely than nonparticipants to have fellowship training in general internal medicine, practice medicine in a community-based setting, and have salary supported by grant funding. Participants were less likely to have an academic appointment, teaching and administrative responsibilities and report past evaluation of a curriculum.
At baseline, as many as 54% (range 40–54%) of participants and 68% (range 27–68%) of nonparticipants in cohorts 2–9 indicated that they did not perform CD, implementation, or evaluation activities at baseline and did not rate their skills and enjoyment in these categories. On the post-program survey, seven participants and seven nonparticipants indicated that they had received training in curriculum/program development beyond our CD Program during the prior 10 months. Participants (cohorts 2–9) rated their pre-program skills in CD and implementation less favorably than nonparticipants and their post-program skills in CD, implementation, and evaluation more favorably (Table 2). Self-reported skills improved significantly for participants from baseline in curricular development, implementation, and evaluation. Ratings decreased for nonparticipants in CD and did not change in curricular implementation or evaluation (Table 3). Participants enjoyed CD more than nonparticipants (Table 2), and enjoyment did not change after the program (Table 3). These findings remained after controlling for baseline characteristics. Pre–post changes for participants were similar in each area described when data from all 16 cohorts were analyzed.
Nearly 90% of the participants indicated that the program offered a good balance between faculty input and self-directed learning. Most felt confident that their curriculum would be piloted and implemented at their respective workplaces and that they could develop curricula in the future (Table 4). Eighty percent of the participants indicated that they would use the 6-step model in the next 2 years, and 80% would recommend this program to others as a good or outstanding experience.
Over 90% of the participants responded to ≥1 open-ended questions. Two-hundred two comments related to program strengths: 31% focused on organization, structure, and content of the program; 29% on the expertise, support, and facilitation of program faculty; 14% on experiential learning and development of a curriculum; 12% on working in and interactions among groups; and 5% on quality of course materials. There were 217 responses on curriculum implementation and evaluation barriers: 24% mentioned time constraints; 22.6% finances/funding; 12.4% institutional constraints; 9.7% learner interest; 7.4% lack of faculty continuity; 8.3% logistics/feasibility; and 6.9% personal constraints such as time management or energy. Of the 70 comments on program weaknesses, the most common related to heavy workload and time commitment.
Cohort 1–16 participants produced 64 curricula. At least 54 (84%) were implemented in total or part. Thirteen curricular projects (20.3%) led to publication of 18 articles in peer-reviewed journals, and 2 books related to project work. Curricula targeted residents (n=48), practicing physicians (n=13), students (n=7), fellows (n=3), and nonphysicians (n=2) (some addressed >1 level of learner). Curricula were typically directed toward trainees or practitioners in general internal medicine (n=49), but several targeted physicians in other specialties (n=4), physicians in mixed specialties (n=3), or nonphysicians (n=2).
Curricula mostly addressed gaps in existing education. Topics included: psychosocial or behavioral subjects [19 projects (30%), e.g., communication skills, substance abuse, ethics]; general clinical issues [16 (25%), e.g., evidence-based practice, introduction to clinical medicine and clinical reasoning, physical examination skills]; internal medicine topics directed toward generalists [13 (20%), e.g., ambulatory medicine, HIV care, hospitalist care]; other topics for internists [9 (14%), e.g., musculoskeletal disorders, women’s health, dermatology]; preventive medicine topics [6 (9%), e.g., colorectal cancer screening, smoking cessation counseling, obesity management]; and educational and research skills training [3 (5%)].
Most participants (86%) worked in pairs or groups and rated this highly in terms of productivity, developing collegial relationships, and cooperativeness (all mean ≥4.0, where 1=poor, 5=excellent). Of those who worked in groups, 86% felt it moderately to very important to work collaboratively. Of those who worked alone, 75% would have preferred to work with others.
This paper describes a 10-month mentored faculty development program in CD in existence since 1987. Our results show improvement in self-assessed CD, implementation, and evaluation skills of participants but not in those of nonparticipants. Participants’ end-of-program self-assessed skills were higher than those of nonparticipants, whereas some of their pre-program self-assessed skills were lower even after controlling for differences in baseline characteristics. The program’s success was also seen in the number of curricular products implemented.
Institutions have recognized the need for faculty development of clinician-educators. The focus of this training has often centered on teaching skills.12,17,21,24–29 Although skilled teachers remain critical for role modeling30,31 and the dissemination of knowledge and skills to future physicians, new curricula are required to meet changing educational demands. Some institutions have recognized this need and instituted programs to train faculty in CD.16,19–22 Most programs have integrated CD training into a more comprehensive program and only report in part on this aspect of the training.16,19,20 Evaluations of training have usually been post-only,16,19–22 involved small numbers,20–22 or did not include a comparison group.16,19–22 We add to the literature by reporting a pre–post-evaluation (with a comparison group) of a long-standing mentored faculty development program focused solely on CD.
The methods used in our CD program mirror methods used in research training and other CD programs15,19–22,32: skills training sessions, independent projects, and regular meetings with mentors and with feedback from them (the mentors). We also include periodic deadlines, work-in-progress presentations, and the oral and written presentation of a final project. Participants valued each of these components highly, reported they had improved their skills, and felt confident that they could develop other curricula in the future. While published guides are available to provide independent direction for curriculum developers,23,33 it is unlikely that these alone or even short 1- to 2-day workshops could have accomplished these objectives. Other faculty development programs that have led to successful project implementation have also used a longitudinal approach with at least some of the methods described above.15,16,19–22
At least 84% of curricula developed were fully or partly implemented. Possible explanations for the high implementation rate include: (a) the structure and mentorship provided by the program; (b) most curricula were group projects, so that if a participant moved after completion of the program, another participant was likely to continue the work; and (c) as years passed, facilitators focused increasingly on the implementation step of CD. In 2003, facilitators began screening the proposed projects of potential applicants to the program before accepting them to ensure support within the participants’ institutions.
Success in implementing curricula can help clinician-educators meet the criteria for academic advancement. Previous surveys have demonstrated that promotion committees and department chairs assign high importance to a clinician-educator’s performance in the design, implementation, and evaluation of educational programs.34,35 Successes in CD can be included in the educational portfolios that are being increasingly used to support applications for promotion.36–38
Despite successful implementation, only 20% of the projects led to publication. Because publication continues to be an important promotion criterion,34,39 and because the steps of well-conducted CD projects offer multiple opportunities for publication,23 program faculty recently began to emphasize this prospect. Starting with cohort 16, a session was added on publishing and disseminating curricular work. For cohort 18, the program addressed searching educational databases and obtaining IRB approval. Most participants now identify dissemination and/or publication as a goal.
There are limitations of our work. First, we used self-reports rather than objective measures of CD skills. Second, although participants produced and implemented a large number of curricula, we do not have measures of the quality of the products produced. We do know from final reports that the curricula included a needs assessment, objectives, educational strategies based on the objectives, and an evaluation plan. Third, our participants were self-selected. Fourth, our comparison group was self-selected by participants. Because we did not have a randomized controlled group, baseline characteristics were not completely identical for participants and nonparticipants. Nevertheless, we were able to adjust for differences in our multivariable analyses. Fifth, many participants and nonparticipants did not evaluate their skills and enjoyment of curricular development activities, especially at baseline. This limited our ability to make comparisons between and among groups. Sixth, we only present data until the end of the program. A separate long-term follow-up study is under way. Seventh, we report on a program from one institution that served predominantly general internal medicine faculty.
This report presents a model for training in CD that has long-term sustainability, has resulted in implementation of many new curricula on important topics, and is associated with participant satisfaction and improvement in self-rated skills. Overall, the outcomes are positive enough to demonstrate that longitudinal training in CD can be successful. Given the needs for improved medical curricula and for academic medical institutions to have a talented cadre of medical educators who can accomplish this work and be promoted, other institutions may want to institute similar programs.
We thank Ken Kolodner, Sc.D., for critically reviewing the statistical methodology used in the paper.
Dr. Windish was a General Internal Medicine Fellow at Johns Hopkins University School of Medicine, Baltimore, MD, at the time of project initiation, whereas Dr. Howard is currently retired and was previously affiliated with the Division of General Internal Medicine, Johns Hopkins University School of Medicine.
Potential Financial Conflicts of Interest None disclosed.