Search tips
Search criteria 


Logo of jgmeLink to Publisher's site
J Grad Med Educ. 2012 March; 4(1): 76–82.
PMCID: PMC3312539

Integrative Medicine in Residency Education: Developing Competency Through Online Curriculum Training



The Integrative Medicine in Residency (IMR) program, a 200-hour Internet-based, collaborative educational initiative was implemented in 8 family medicine residency programs and has shown a potential to serve as a national model for incorporating training in integrative/complementary/alternative medicine in graduate medical education.


The curriculum content was designed based on a needs assessment and a set of competencies for graduate medical education developed following the Accreditation Council for Graduate Medical Education outcome project guidelines. The content was delivered through distributed online learning and included onsite activities. A modular format allowed for a flexible implementation in different residency settings.


To assess the feasibility of implementing the curriculum, a multimodal evaluation was utilized, including: (1) residents' evaluation of the curriculum; (2) residents' competencies evaluation through medical knowledge testing, self-assessment, direct observations, and reflections; and (3) residents' wellness and well-being through behavioral assessments.


The class of 2011 (n  =  61) had a high rate of curriculum completion in the first and second year (98.7% and 84.2%) and course evaluations on meeting objectives, clinical utility, and functioning of the technology were highly rated. There was a statistically significant improvement in medical knowledge test scores for questions aligned with content for both the PGY-1 and PGY-2 courses.


The IMR program is an advance in the national effort to make training in integrative medicine available to physicians on a broad scale and is a success in terms of online education. Evaluation suggests that this program is feasible for implementation and acceptable to residents despite the many pressures of residency.

Editor's Note: The online version of this article contains additional details of the evaluation of competencies described in this study.


Graduate medical education is challenged to train enough primary care physicians to meet the nation's growing chronic illness burden.1 Primary care physicians need training in approaches that engage patients in self-care, health promotion, and prevention, including nutrition, diet, physical activity/exercise, stress management, and use of nutritional supplements. Because 75% of patients use complementary and alternative medicine (CAM) together with conventional medicine, the Institute of Medicine recommends that health professional schools incorporate information about CAM into standard curricula at undergraduate, graduate, and postgraduate levels.2,3

Integrative medicine (IM) is healing-oriented medicine that takes account of the whole person—body, mind, and spirit—including lifestyle. It emphasizes a therapeutic relationship and use of conventional and alternative approaches to care.4 Curriculum guidelines for IM have been published for medical students and residents.4,6 Residency might be the most appropriate level to teach IM as demonstrated by a 4-year model that included a full family medicine residency plus an IM fellowship.7 This model successfully increased graduates' IM competencies and enhanced student recruitment.8 A limitation is that the program educated just 6 to 8 residents annually.8 To promote IM training on a larger scale, we developed the Integrative Medicine in Residency (IMR) program, a 200-hour, competency-based online curriculum incorporated throughout all 3 years of a primary care residency. We describe the development and content of the IMR program and its evaluation design, and present preliminary findings from 3 measures (course completion rates, course evaluations, and medical knowledge test scores).


Curriculum Development

In 2007, the Arizona Center for Integrative Medicine solicited participation from family medicine faculty trained in IM to implement a 5-year pilot project to determine the feasibility of incorporating 200 hours of IM content into residency training. Eight programs (table 1) committed participation for 3 graduating classes (2011, 2012, 2013). The IMR curriculum followed guidelines from the Accreditation Council for Graduate Medical Education (ACGME). Content development was guided by data from a faculty and residents needs assessment.9 Previously published IM competencies were referenced to develop IMR core competencies and link them with specific evaluation methods (table 2).5,10

Integrative Medicine in Residency (IMR) Program Sites and Control Sites
Integrative Medicine in Residency Competencies and Evaluation Strategies
Integrative Medicine in Residency Competencies and Evaluation Strategies

The online modular curriculum allows maximum implementation flexibility. Self-contained units can be adapted to use longitudinally, within individual rotations, as electives or tracks. Case-based teaching reflecting diverse populations is infused throughout, and maximum interactivity holds learners' interest. Most content (166 hours) is delivered online (table 3), leaving 34 hours for topics taught in face-to-face sessions based on faculty expertise and resources. Standard onsite sessions include curriculum orientation, annual resident wellness and self-care assessments, and experiential workshops in mind-body medicine and motivational interviewing. Other onsite activities include a yoga class, a visit to a health food store and potlucks with healthy foods. The first unit, Prevention and Wellness, emphasizes conventional recommendations for preventive services along the life cycle and includes courses in nutrition, dietary supplements, physical activity, mind-body tools for stress management, and sleep. Interactive cases provide clinical integration. Tools in IM include courses on botanical medicine, manual medicine, mind-body medicine, spirituality, and an introduction to whole systems (naturopathy, homeopathy, traditional Chinese medicine, Ayurveda). All other units address problems of specific patient populations (table 3). The majority of the content was written by integrative primary care physicians and reviewed by faculty at participating residency programs.

IMR Curriculum Content

The IMR website contains a learning community section and online portfolio.11 In the community section, residents' pictures and profiles are posted. Online dialogues (a “watercooler” and “curbside consult”) are facilitated. The portfolio is a repository of completion reports, evaluations, and reflections completed in different modules.


IMR evaluation is organized in 4 components: (1) evaluation of resident competencies; (2) course completion; (3) resident evaluation of the curriculum; and (4) assessment of resident wellness behaviors and well-being. Both quantitative and qualitative methods are used. With the exception of faculty ratings on direct observation checklists (DOCs), data are collected through the website.

The IMR group is composed of 3 cohorts—graduating classes of 2011, 2012, and 2013. The pilot sites represent a mix of community (n  =  3) and university affiliated/university-based residencies (n  =  5). To serve as controls, residents were recruited from the 2012 and 2013 classes from 4 residencies that do not have the IMR curriculum but attract similar residents as the pilot sites based on geographical proximity or overall residency characteristics (table 1). Controls receive an annual incentive to complete the comparison measures of medical knowledge and resident wellness behaviors and well-being. Comparison data are still being collected and are not presented.

Approval for the study was granted by the University of Arizona Institutional Review Board (IRB) and the IRBs of the pilot sites and control sites.

Evaluation of Resident Competencies

The evaluation of resident competence in IM encompasses medical knowledge, practice-based learning and improvement, patient care, interpersonal and communication skills, professionalism, and systems-based practice. Details of the evaluation are provided as online supplemental material.

Course Completion

Upon completion of each course, residents click a “finish” button that records course completion. A minimum of 80% completed courses is required.

Resident Evaluation of the Curriculum

At the end of each course, residents complete a course evaluation. Items use a 5-point Likert-type scale to rate how well residents felt the course covered its learning objectives, whether the depth of material was challenging, the usefulness of the online resources, the usefulness of the course for patient care, and the functioning of the online technology. Residents also estimate the length of time it took to complete the course and can provide free-text comments. Appropriate curricular revisions and enhancements are made as a result of the residents' feedback.

Assessment of Resident Wellness and Well-Being

Instruments with known evidence of validity are used to assess residents' well-being: perceived stress, depression, burnout, emotional intelligence, mindfulness, gratitude, satisfaction with life, and positive and negative affect (table 4).12,20 A new instrument to assess wellness behaviors (eg, restful sleep, physical activity, diet) was developed by faculty of the Arizona Center for Integrative Medicine. Residents are assessed at the beginning of each year and at the end of residency.

Wellness and Well-Being Measures


Sample and Demographics

Our results are based on 61 residents in the 2011 IMR class, who completed the first 2 years of the curriculum (of 69 entering residents, 6 left residency and 2 were reassigned to the 2012 class). The majority of the cohort were women (66.6%, n  =  42) and US medical school graduates (55.5%, n  =  35). Ten residents (15.9%) were osteopathic graduates, and 18 (28.6%) were international medical graduates.

Course Completion

The PGY-1 curriculum consisted of 8 courses, and the mean percentage of course completions was very high, 98.7% (range 98.4%–100%). The PGY-2 curriculum consisted of 16 courses, and mean percentage of completion was high at 84.2% (range 61.1%–96.2%).

Course Evaluation

Resident ratings of meeting stated learning objectives (adequately and definitely) for the PGY-1 and PGY-2 courses combined averaged 93.8% (range 84.6%–98%). Ratings on clinical utility (useful and very useful) of courses averaged 89.7% (SD  =  5.9%, range 77.8%–98.7%), and ratings of the functioning of the online technology (smooth and very smooth) averaged 84.6% (range 67.4%–97.4%).

Box 1 Qualitative Resident Feedback on the Integrative Medicine Residency Program

“I most valued information about natural therapies, how to prescribe them, and the evidence for them.”

“Great set of modules. These are situations we all see in clinic at least a few times a day and these modules provided great resources to give patients.”

“I liked the video where I could see the interaction between the provider and the patient.”

“Combining and comparing the integrative medical remedies and the medications. It helps me put it all together in my head as to what tools I have to treat my patients with.”

Medical Knowledge Test

For scores on first- and second-year tests, there was statistically significant improvement from pre- to posttest. The mean percentage correct increased from 50.98% to 64.71%, {t[50]  =  −6.129, P  =  .001} for PGY-1 and from 43.82% to 51.37%, {t[45]  =  −3.397, P  =  .001} for PGY-2.


The IMR program represents an important advance in efforts to make training in IM broadly available in graduate medical education. Preliminary results presented here, including high levels of course completion and resident satisfaction, suggest the program is feasible during residency and is useful to residents. The high rates of course completions address a concern from the faculty and program directors' needs assessment9 that a 200-hour curriculum would constitute a barrier to implementation. The modest yet significant gain in medical knowledge between pre- and postintervention scores suggests the curriculum might be effective in increasing baseline IM knowledge. Additional data from control sites will provide more comprehensive results.

Certain measures can assure successful implementation of the IMR. Although distributed learning allows residents to access content developed by leaders in IM, program success requires that some onsite faculty have IM expertise. The IMR web-based format can facilitate faculty development, allowing faculty to increase their own knowledge and skills in IM. Further, support from the program director is required, as is a faculty leader with at least 10% full-time equivalent (FTE) to coordinate onsite activities. It is also critical that residents apply IM knowledge within the outpatient setting. A strength of the web-based curriculum is that it can be accessed from anywhere at any time, avoiding potential duty hour violations. The potential for further dissemination of the IMR program is evidenced by the fact that 12 additional family medicine and 2 internal medicine residencies adopted the curriculum as of July 2011.

Our intervention has some limitations. We report preliminary findings for the first 2 years of the first cohort, and there was no control group. Data from subsequent classes will enable a more robust analysis with control group comparisons.


Early evaluation data suggest the IMR curriculum may be suitable for widespread dissemination in graduate medical education. The IMR program demonstrates feasibility of incorporating a standardized, web-based curriculum into diverse residency settings. Future planned analyses of the curriculum will provide more definitive evidence of its effectiveness and potential for dissemination.


Patricia Lebensohn, MD, is Professor of Clinical Family and Community Medicine at the University of Arizona College of Medicine, and is Director of the Integrative Medicine in Residency program at the Arizona Center for Integrative Medicine; Benjamin Kligler, MD, MPH, is Vice Chair and Research Director of the Beth Israel Department of Integrative Medicine, Continuum for Health and Healing; Sally Dodds, PhD, is Associate Professor/Research Scholar of Medicine at the University of Arizona College of Medicine, and is the Evaluator of the Integrative Medicine in Residency program at the Arizona Center for Integrative Medicine; Craig Schneider, MD, is Director of Integrative Medicine in the Department of Family Practice at Maine Medical Center, Portland; Selma Sroka, MD, is Assistant Professor Department of Family Medicine and Community Health, University of Minnesota, Department of Family and Community Medicine, Hennepin County Medical Center; Rita Benn, PhD, is a Research Investigator, Department of Family Medicine and Director of Education, University of Michigan Integrative Medicine, as well as an Assistant Research Scientist, Institute of Research onWomen and Gender; Paula Cook, BA, is a Senior Research Specialist for the Arizona Center for Integrative Medicine, University of Arizona; Mary Guerrera, MD, is a Professor and Director of Integrative Medicine in the Department of Family Medicine at the University of Connecticut School of Medicine; Tieraona Low Dog, MD, is Clinical Associate Professor of Medicine, University of Arizona, College ofMedicine, and is the Director of the Fellowship in Integrative Medicine of the Arizona Center for Integrative Medicine; Victor Sierpina, MD, is the WD and Laura Nell Nicholson Family Professor of Integrative Medicine and Professor of Family Medicine, University of Texas Medical Branch; Raymond Teets, MD, is the Director of Integrative Medicine at the Beth Israel Residency in Urban Family Practice, and Clinical Instructor in the Department of Family Medicine at the Albert Einstein School ofMedicine; Dael Waxman, MD, is Medical Director of Behavioral and Integrative Medicine, Department of Family Medicine, Carolinas Medical Center, Charlotte; John Woytowicz, MD, is Director of Integrative Medicine at Maine-Dartmouth Family Medicine Residency in Augusta; Andrew Weil, MD, is Professor of Medicine and Public Health, and Jones-Lovell Endowed Chair in Integrative Rheumatology, University of Arizona College of Medicine, and is the Founder and Director of the Arizona Center for Integrative Medicine; and Victoria Maizes, MD, is Professor of Clinical Medicine, Family Medicine, and Public Health at the University of Arizona College of Medicine, and is the Executive Director of the Arizona Center for Integrative Medicine.

Funding/Support: Funding for the Integrative Medicine in Residency program and its evaluation has been provided by a Congressionally-directed grant administered through the US Department of Education. Contents do not necessarily represent the policy of the Department of Education and do not assume endorsement by the Federal Government. Additional support comes from the Weil Foundation, the David C. and Lura M. Lovell Foundation, the Joan B. Diamond Charitable Lead Trust, and multiple generous individuals. Additional funding for Dr Kligler is from NIH/NCCAM K-07 Award (AT004570-01A2).

The authors wish to thank Rhonda Hallquist, Web Developer, and Gates Matthew Stoner, Web Platform Manager, for their invaluable assistance in implementing the Integrative Medicine in Residency curriculum in an online format. Audrey Brooks, PhD, provided statistical analysis for the results. In addition, Emily Sherbrooke, Program Coordinator, is thanked for her contributions.


1. Robert Wood Johnson Foundation. 2010 report, chronic care: making the case for ongoing care. Accessed June 6, 2011.
2. Eisenberg DM, Kessler RC, Foster C, Norlock FE, Calkins DR, Delbanco TL. Unconventional medicine in the United States. N Engl J Med. 1993;328:246–252. [PubMed]
3. Institute of Medicine. January 12, 2005 report, complementary and alternative medicine in the United States. Accessed June 2, 2011.
4. Maizes V, Schneider C, Bell I, Weil A. Integrative medical education: development and implementation of a comprehensive curriculum at the University of Arizona. Acad Med. 2002;77:851–860. [PubMed]
5. Kligler B, Gordon A, Stuart M, Sierpina V. Suggested curriculum guidelines on complementary and alternative medicine: recommendations of the Society of Teachers of Family Medicine Group on Alternative Medicine. Fam Med. 2000;2:30–33. [PubMed]
6. Kligler B, Maizes V, Schachter S, Park CM, Gaudet T, Benn R, Lee R, Remen RN. Education Working Group, Consortium of Academic Health Centers for Integrative Medicine. Core competencies in integrative medicine for medical school curricula: a proposal. Acad Med. 2004;79:521–531. [PubMed]
7. Maizes V, Silverman H, Lebensohn P, Koithan M, Kligler B, Rakel D, Schneider C, Kohatsu W, Hayes M. The Integrative Family Medicine Program: an innovation in residency education. Acad Med. 2006;81:583–589. [PubMed]
8. Kligler B, Lebensohn P, Koithan M, Schneider C, Rakel D, Cook P, Kohatsu W, Maizes V. Measuring the “whole system” outcomes of an educational innovation: experience from the Integrative Family Medicine Program. Fam Med. 2009;41:342–349. [PubMed]
9. Benn R, Maizes V, Guerrera M, Sierpina V, Cook P, Lebensohn P. Integrative Medicine in Residency: assessing curricular needs in eight programs. Fam Med. 2009;41:708–714. [PubMed]
10. Kligler B, Koithan M, Maizes V, Hayes M, Schneider C, Lebensohn P, Hadley S. Competency-based evaluation tools for integrative medicine training in family medicine residency: a pilot study. BMC Med Educ. 2007;7:7. [PMC free article] [PubMed]
11. Tochel C, Haig A, Hesketh A, Cadzow A, Beggs K, Colthart I, Peacock H. The effectiveness of portfolios for post-graduate assessment and education: BEME Guide No 12. Med Teach. 2009;31:299–318. [PubMed]
12. Cohen S, Kamarck T, Mermelstein R. A global measure of perceived stress. J Health Soc Behav. 1983;24:385–396. [PubMed]
13. Radloff LS. The CES-D Scale: a self-report depression scale for research in the general population. Appl Psychol Meas. 1977;1:385–401.
14. Maslach C, Jackson SE, Leiter MP. Maslach Burnout Inventory. 3rd ed. Palo Alto, CA: Consulting Psychologists Press; 1996.
15. Watson D, Clark LA, Tellegen A. Development and validation of brief measures of positive and negative affect: the PANAS scales. J Pers Soc Psychol. 1988;54:1063–1070. [PubMed]
16. Diener E, Emmons RA, Larsen RJ, Griffin SG. The Satisfaction with Life Scale. J Pers Assess. 1985;49:71–75. [PubMed]
17. McCullough ME, Emmons RA, Tsang J-A. The grateful disposition: a conceptual and empirical topography. J Pers Soc Psychol. 2002;82:112–127. [PubMed]
18. Walach H, Bucheld N, Buttenmuller V, Kleinknecht N, Schmidt S. Measuring mindfulness—the Freiburg Mindfulness Inventory (FMI) Pers Individ Dif. 2006;40:1543–1555.
19. Salovey P, Mayer JD, Goldman SL, Turvey C, Palfai TP. Emotional attention, clarity, and repair: exploring emotional intelligence using the Trait Meta-Mood Scale. In: Pennebaker J, editor. Emotion, Disclosure, and Health. Washington, DC: American Psychological Association; 1995. pp. 125–154.
20. Davis MH. A multidimensional approach to individual differences in empathy. JSAS Catalog of Selected Documents in Psychology. 1980;10:85.

Articles from Journal of Graduate Medical Education are provided here courtesy of Accreditation Council for Graduate Medical Education