PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of jgmeLink to Publisher's site
 
J Grad Med Educ. 2009 September; 1(1): 5–20.
PMCID: PMC2931179

Charting the Road to Competence: Developmental Milestones for Internal Medicine Residency Training

Abstract

Background

The Accreditation Council for Graduate Medical Education (ACGME) Outcome Project requires that residency program directors objectively document that their residents achieve competence in 6 general dimensions of practice.

Intervention

In November 2007, the American Board of Internal Medicine (ABIM) and the ACGME initiated the development of milestones for internal medicine residency training. ABIM and ACGME convened a 33-member milestones task force made up of program directors, experts in evaluation and quality, and representatives of internal medicine stakeholder organizations. This article reports on the development process and the resulting list of proposed milestones for each ACGME competency.

Outcomes

The task force adopted the Dreyfus model of skill acquisition as a framework the internal medicine milestones, and calibrated the milestones with the expectation that residents achieve, at a minimum, the “competency” level in the 5-step progression by the completion of residency. The task force also developed general recommendations for strategies to evaluate the milestones.

Discussion

The milestones resulting from this effort will promote competency-based resident education in internal medicine, and will allow program directors to track the progress of residents and inform decisions regarding promotion and readiness for independent practice. In addition, the milestones may guide curriculum development, suggest specific assessment strategies, provide benchmarks for resident self-directed assessment-seeking, and assist remediation by facilitating identification of specific deficits. Finally, by making explicit the profession's expectations for graduates and providing a degree of national standardization in evaluation, the milestones may improve public accountability for residency training.

Introduction

In July 2002, the Accreditation Council for Graduate Medical Education (ACGME) Outcome Project changed the currency of accreditation from process and structure (capturing a program's potential to educate) to outcomes (capturing a program's actual accomplishments).1 Residency program directors were asked to provide more than a schedule of rotations, a written curriculum, and agreements with clinical training venues. They also must objectively document that their residents achieve competence in 6 general dimensions of practice. In phase 1 of the Outcome Project, programs defined objectives to demonstrate learning in the competencies. In phase 2 they integrated the competencies into their curricula and expanded their evaluation systems to assess performance in them. Programs are currently in phase 3, which requires them to use aggregate performance data for curriculum reform. Phase 4 intends to focus on identification of benchmark programs. This paradigm shift in training, hailed as the Flexnerian revolution of the 21st century,2 is aimed at enhancing our profession's ability to verify that graduates of residency programs are competent, at a minimum, to deliver safe and effective patient care.

Although the Outcome Project has advanced residency training in important ways, it has not resulted in widespread “operationalizing” of outcomes in the evaluation of residents or in the accreditation of programs. This may be partly because of the complex nature of the competencies, which reflect high-level syntheses of more operationally measurable learning objectives. To move the process forward, the ACGME has engaged the medical education community in “articulating milestones of competency development in each discipline.”3 The milestones would explicate the 6 ACGME general competencies by describing a developmental progression of observable behaviors. Programs would use the milestones to provide more specific feedback and evaluation to residents and ensure that they acquire the necessary knowledge, skills, and attitudes for advancing in the program and entering the next phase of their careers. The ACGME would use program performance on the milestones as currency for accreditation actions.

In November 2007, the American Board of Internal Medicine (ABIM) and the ACGME sponsored an initiative to develop milestones for internal medicine residency training. Herein we report the development process and the resulting list of proposed milestones for each ACGME competency.

Methods

The ABIM and ACGME convened a 33-member milestones task force (appendix 1) composed of program directors, experts in evaluation and quality, and representatives of internal medicine stakeholder organizations, including the Alliance for Academic Internal Medicine, American College of Physicians, American Medical Association, Association of Program Directors in Internal Medicine, and the Society of General Internal Medicine. These individuals participated with the understanding that the resulting milestones document would not signify official policy of their respective organizations. Even though ABIM and ACGME provided funding, meeting space, and administrative support for this project, they agreed to maintain the editorial independence of the task force. An initial 2-day meeting included an overview of the Alliance for Academic Internal Medicine Education Redesign Task Force Consensus Report,4 a brainstorming session on the potential utility of developmental milestones, presentations of milestones initiatives at 3 residency programs (Michigan State,5 Lehigh Valley, and Baystate), a facilitated discussion of several conceptual frameworks of competence,6,9 and division of the task force into subcommittees representing the 6 ACGME general competencies.

The subcommittees worked independently, via conference calls and a PBwiki collaboration Internet site (PBworks, San Mateo, CA), to develop an initial set of milestones and suggested evaluation strategies for each competency. In their work, they reviewed the revised ACGME common program requirements and Residency Review Committee for Internal Medicine program requirements (effective July 2009),10 relevant medical education literature, and several internal medicine program curricula. The larger task force assembled for a second 2-day meeting in May 2008. The agenda included (1) achieving a consensus for a minimum standard for “competence” for internal medicine residents, (2) aligning evaluation strategies with particular milestones, and (3) considering practical issues, such as the resource and expertise requirements for programs. In addition, a writing committee was constituted from representatives from each subcommittee. Its charge was to refine and standardize the milestone language, reconcile redundancies and conflicts, and compose a document that articulated the need for developmental milestones, recorded the task force's process, and placed the initiative in the context of ongoing graduate medical education reform.

The members of the larger task force reviewed the document individually and provided additional commentary at a third meeting in December 2008. In particular, the task force recommended condensing the original detailed 64-page document to a briefer overview for the purpose of sharing the milestones with the broader medical education community. The writing committee revised the document accordingly and prepared it for external review. In total, members of the writing committee participated in 6 conference calls, 2 in-person meetings, and numerous e-mail exchanges.

Results

The task force adopted the Dreyfus model8,9 of skill acquisition as a framework for developing milestones for internal medicine residency training. Specifically, we calibrated the milestones with the expectation that residents achieve, at a minimum, the “competency” level in the 5-step progression before completion of residency training. This threshold is consistent with other applications of the Dreyfus model to medical education.11,15 By the time a learner reaches “competence,” he or she has already progressed from simply applying rules to facts and features without context (novice) to considering the specific features of concrete “situations” (advanced beginner). The competent learner considers both context-free and situational elements but also hierarchically organizes and reduces them to a smaller set on which to base a decision. In addition, he or she becomes more intimately involved in the process and feels more responsibility for the outcome. In the next stage, proficiency, learners solve problems with an intuition that usually derives from some time in independent practice. Thus, although it is expected that some residents will achieve proficiency in some competencies, the task force decided not to set “proficiency” as a minimum threshold.

appendix 2 lists the developmental milestones for internal medicine residency training, organized in terms of the ACGME general competencies and the extended specialty-specific requirements added by the Residency Review Committee for Internal Medicine. The ACGME-proposed bullets subdividing the competencies were used as the framework to organize the milestones. These subdivisions are either included verbatim or collapsed into a smaller number of categories. Recognizing that competence can be observed only in performance, we phrased the milestones in behavioral terms. We also suggested approximate time frames for residents to reach each milestone, recognizing a certain amount of arbitrariness in the process and anticipating that, for some milestones, achievement times may vary widely among programs with different curricula. For example, residents in a program that does not offer a quality improvement curriculum until the third year may not meet many of the practice-based learning and improvement milestones until then.

Finally, we confined our recommendations for evaluating the milestones (appendix 2) to general strategies. The writing committee decided that a detailed discussion about the availability, formats, feasibility, and psychometric characteristics of specific assessment instruments was beyond the scope of this initiative, which focused on articulating the milestones. Among the general strategies, we did not include global ratings because faculty scoring fails to distinguish between performance in the 6 competencies.16 We did not link learning portfolios to particular milestones, as these collections may include evaluation items from all 6 competencies. The recommended evaluation strategies are intended not as prescriptions but rather as a range of options for program directors, who may choose among them or develop their own, based on their expertise, resources, programmatic objectives, and institutional values.

Discussion

We propose this list of milestones to promote competency-based training in internal medicine. Residency program directors may use them to track the progress of trainees in the 6 general competencies and inform decisions regarding promotion and readiness for independent practice. In addition, the milestones may guide curriculum development, suggest specific assessment strategies, provide benchmarks for resident self-directed assessment-seeking,17 assist remediation by facilitating identification of specific deficits, and provide a degree of national standardization in evaluation. Finally, by explicitly enumerating the profession's expectations for graduates, they may improve public accountability for residency training.

It is worth noting that many of the milestones—particularly in practice-based learning and improvement, systems-based practice, communication and interpersonal skills, and professionalism competencies—are not unique to internal medicine. Physicians in any specialty should demonstrate competence in these “horizontal” dimensions of clinical practice. Thus, educators in other specialties may adopt some of our work as they develop milestones for their residency programs.

Some may find that the “generous” time frames set a low bar, believing that residents should reach some of the early milestones sooner. Indeed, in keeping with our decision to set a floor rather than a ceiling, we set the time frames with the expectation that a resident's failure to reach them would trigger further assessment and possibly remediation. At a programmatic level, a significant deviation from the expected progression along the milestones may trigger an accreditation action. Thus, we expect that many normally progressing residents will reach many of the milestones in advance of the “deadline.” A few exceptional graduating medical students may even begin their internship part of the way “down the road.” Finally, these time frames represent a starting point of an ongoing dialogue. We expect them to be refined based on the implementation pilot projects planned for the next phase.

We also anticipate that some program directors, weary from complying with the “musts” and “shoulds” handed down from the ACGME, may receive the milestones as yet another bureaucratic burden. On the contrary, we foresee the milestones making their jobs easier. The specific observable behaviors embodied in them, for instance, should assist program directors, who have hitherto struggled to translate the more general language of the 6 competencies into concrete assessments.18 Nor should this initiative stifle creativity and innovation. In the spirit of the Outcome Project, program directors remain free to develop innovative structures, curricula, and evaluation systems, provided they demonstrate learning “outcomes” in the 6 competencies, which are now elaborated in the milestones. Finally, we expect that residents, who often receive feedback lacking a specific action plan,19 will welcome the more actionable feedback afforded by the milestones framework.

Of course, “This is not the end,” as Churchill said in 1942. “It is not even the beginning of the end. But it is, perhaps, the end of the beginning.” More work is needed before these milestones can be successfully integrated into competency-based evaluation systems. Specifically, we will solicit commentary from the broader medical education community to help us refine the proposed milestones and correct any omissions or redundancies. We must also articulate concrete behavioral anchors for each developmental stage, identify psychometrically robust and feasible evaluation instruments to assess residents' progress, and train faculty to use these instruments effectively.20 Finally, we will learn practical lessons from the initial implementation experience, as diverse residency programs, beginning with pilot projects, integrate developmental milestones into their evaluation systems.

This will be a challenging task but, we believe, one that is well within our reach. We do not share the skepticism of others who lament the perceived inadequacy of currently available evaluation instruments.21 On the contrary, the “tool box” contains many robust instruments.22,29 The problem lies in the variable use of the instruments by faculty who do not share a common understanding of expected behaviors.30 The milestones provide a set of consistent expectations that should reduce this variability.

As representatives of the internal medicine education community, we articulated the milestones to embody our vision of the development of a competent internist. We ask the ACGME only to hold us to this standard.

Appendix 1 Milestones Task Force

Eva M. Aagaard, MD

Jane H. Barnsteiner, PhD, FAAN

Thomas A. Blackwell, MD

Karen Hsu Blatman, MD

Donald R. Bordley, MD

Kelly Caverzagie, MD

Davoren A. Chick, MD, FACP

Charles P. Clayton

Thomas G. Cooney, MD

Rosemarie L. Fisher, MD

Michael L. Green, MD, MSc

Luke Hansen, MD

Linda A. Headrick, MD

Kevin T. Hinchey, MD

Eric S. Holmboe, MD, FACP

Holly J. Humphrey, MD, FACP

William F. Iobst, MD

Gregory C. Kane, MD

David Karlson, PhD

Charles M. Kilo, MD, MPH

Lynne M. Kirk, MD

Catherine R. Lucey, MD, FACP

Thomas J. Nasca, MD, MACP

Eileen E. Reynolds, MD

Eugene C. Rich, MD, FACP

Paul H. Rockey, MD, MPH

William E. Rodak, PhD

Michele Sanders, MD

Henry J. Schultz, MD

Lawrence Smith, MD

Abraham Verghese, MD, MACP, DSc

Steven E. Weinberger, MD, FACP

Brent C. Williams, MD, MPH

Appendix 2.1 Developmental Milestones for Internal Medicine Training—Patient Care

An external file that holds a picture, illustration, etc.
Object name is i1949-8357-1-1-5-t01.jpg

Appendix 2.2 Developmental Milestones for Internal Medicine Training—Medical Knowledge

An external file that holds a picture, illustration, etc.
Object name is i1949-8357-1-1-5-t02.jpg

Appendix 2.3 Developmental Milestones for Internal Medicine Training—Practice-Based Learning and Improvement

An external file that holds a picture, illustration, etc.
Object name is i1949-8357-1-1-5-t03.jpg

Appendix 2.4 Developmental Milestones for Internal Medicine Training—Interpersonal and Communication Skills

An external file that holds a picture, illustration, etc.
Object name is i1949-8357-1-1-5-t04.jpg

Appendix 2.5 Developmental Milestones for Internal Medicine Training—Professionalism

An external file that holds a picture, illustration, etc.
Object name is i1949-8357-1-1-5-t05.jpg

Appendix 2.6 Developmental Milestones for Internal Medicine Training—Systems-Based Practice

An external file that holds a picture, illustration, etc.
Object name is i1949-8357-1-1-5-t06.jpg

Footnotes

Michael L. Green, MD, MSc, is an Associate Professor and Associate Residency Program Director in the Department of Internal Medicine, Yale University School of Medicine; Eva M. Aagaard, MD, is Vice Chair for Education in the Department of Internal Medicine, University of Colorado School of Medicine; Kelly J. Caverzagie, MD, is Associate Residency Program Director in the Department of Internal Medicine, Henry Ford Hospital; Davoren A. Chick, MD, is Associate Residency Program Director in the Department of Internal Medicine, University of Michigan Medical School; Eric Holmboe, MD, is Senior Vice President for Quality Research and Academic Affairs with the American Board of Internal Medicine; Gregory Kane, MD, is Professor, Vice Chair for Education, and Residency Program Director in the Department of Internal Medicine, Jefferson Medical College; Cynthia D. Smith, MD, is Residency Program Director in the Department of Internal Medicine, Lankenau Hospital; and William Iobst, MD, is Director of Academic Affairs with the American Board of Internal Medicine.

This study was funded by the American Board of Internal Medicine and the Accreditation Council for Graduate Medical Education.

We would like to thank Sarah Hood at the American Board of Internal Medicine for her tireless work organizing the milestones initiative and assisting with this manuscript.

References

1. Accreditation Council for Graduate Medical Education. Outcome project. http://www.acgme.org/outcome/. Accessed November 15, 2008.
2. Carraccio C., Wolfsthal S. D., Englander R., Ferentz K., Martin C. Shifting paradigms: from Flexner to competencies. Acad Med. 2002;77(5):361–367. [PubMed]
3. Nasca T. J. The next step in the outcomes-based accreditation project. ACGME Bulletin. May, 2008. pp. 2–4.
4. Meyers F. J., Weinberger S. E., Fitzgibbons J. P., Glassroth J., Duffy F. D., Clayton C. P. Redesigning residency training in internal medicine: the consensus report of the Alliance for Academic Internal Medicine Education Redesign Task Force. Acad Med. 2007;82(12):1211–1219. [PubMed]
5. Chick D., Laird-Fick H., Sousa A. Michigan State University Internal Medicine Residency Competency Curricula and Learning Objectives. 3rd ed. 2007. East Lansing, MI: Michigan State University.
6. Miller G. E. The assessment of clinical skills/competence/performance. Acad Med. 1990;65(suppl 9):S63–S67. [PubMed]
7. Epstein R. M., Hundert E. M. Defining and assessing professional competence. JAMA. 2002;287(2):226–235. [PubMed]
8. Dreyfus S. E., Dreyfus H. L. A five stage model of the mental activities involved in directed skill acquisition. Air Force Office of Scientific Research under contract F49620-79-C-0063. Berkeley: University of California; 1980. http://oai.dtic.mil/oai/oai?verb=getRecord&metadataPrefix=html&identifier=ADA084551. Accessed November 15, 2008.
9. Dreyfus H. L. On the Internet: Thinking in Action. New York: Routledge; 2003.
10. Accreditation Council for Graduate Medical Education. Program requirements for residency education in internal medicine. http://www.acgme.org/acWebsite/downloads/RRC_progReq/140_im_07012007.pdf. Accessed November 15, 2008. [PubMed]
11. Benner P. Using the Dreyfus model of skill acquisition to describe and interpret skill acquisition and clinical judgment in nursing practice and education. Bull Sci Technol Soc. 2004;24(3):188–199.
12. Oetting T. A., Lee A. G., Beaver H. A. Teaching and assessing surgical competency in ophthalmology training programs. Ophthalmic Surg Lasers Imaging. 2006;37(5):384–393. [PubMed]
13. Leach D. C. Competence is a habit. JAMA. 2002;287(2):243–244. [PubMed]
14. Ogrinc G., Headrick L. A., Mutha S., Coleman M. T., O'Donnell J., Miles P. V. A Framework for teaching medical students and residents about practice-based learning and improvement, synthesized from a literature review. Acad Med. 2003;78(7):748–756. [PubMed]
15. Carraccio C. L., Benson B. J., Nixon L. J., Derstine P. L. From the educational bench to the clinical bedside: translating the Dreyfus developmental model to the learning of clinical skills. Acad Med. 2008;83(8):761–767. [PubMed]
16. Silber C. G., Nasca T. J., Paskin D. L., Eiger G., Robeson M., Veloski J. J. Do global rating forms enable program directors to assess the ACGME competencies? Acad Med. 2004;79(6):549–556. [PubMed]
17. Eva K. W., Regehr G. “I'll never play professional football” and other fallacies of self-assessment. J Contin Educ Health Prof. 2008;28(1):14–19. [PubMed]
18. Heard J. K., Allen R. M., Clardy J. Assessing the needs of residency program directors to meet the ACGME general competencies. Acad Med. 2002;77(7):750. [PubMed]
19. Holmboe E. S., Yepes M., Williams F., Huot S. J. Feedback and the mini clinical evaluation exercise. J Gen Intern Med. 2004;19(5, pt 2):558–561. [PMC free article] [PubMed]
20. Holmboe E. S., Hawkins R. E., Huot S. J. Effects of training in direct observation of medical residents' clinical competence: a randomized trial. Ann Intern Med. 2004;140(11):874–881. [PubMed]
21. Lurie S. J., Mooney C. J., Lyness J. M. Measurement of the general competencies of the Accreditation Council for Graduate Medical Education: a systematic review. Acad Med. 2009;84(3):301–309. [PubMed]
22. Accreditation Council for Graduate Medical Education. Outcome project: assessment. http://www.acgme.org/outcome/assess/assHome.asp. Accessed November 15, 2008. [PubMed]
23. Holmboe E. S., Hawkins R. E., editors. A Practical Guide To the Evaluation of Clinical Competence. Philadelphia: Mosby-Elsevier; 2008.
24. Duffy F. D., Gordon G. H., Whelan G. Assessing competence in communication and interpersonal skills: the Kalamazoo II report. Acad Med. 2004;79(6):495–507. [PubMed]
25. Rider E. A., Keefer C. H. Communication skills competencies: definitions and a teaching toolbox. Med Educ. 2006;40(7):624–629. [PubMed]
26. Lynch D. C., Surdyk P. M., Eiser A. R. Assessing professionalism: a review of the literature. Med Teach. 2004;26(4):366–373. [PubMed]
27. Shaneyfelt T., Baum K. D., Bell D. Instruments for evaluating education in evidence-based practice: a systematic review. JAMA. 2006;296(9):1116–1127. [PubMed]
28. Lynch D. C., Swing S. R., Horowitz S. D., Holt K., Messer J. V. Assessing practice-based learning and improvement. Teach Learn Med. 2004;16(1):85–92. [PubMed]
29. Ogrinc G., Headrick L. A., Morrison L. J., Foster T. Teaching and assessing resident competence in practice-based learning and improvement. J Gen Intern Med. 2004;19(5, pt 2):496–500. [PMC free article] [PubMed]
30. Holmboe E. S. Faculty and the observation of trainees' clinical skills: problems and opportunities. Acad Med. 2004;79(1):16–22. [PubMed]

Articles from Journal of Graduate Medical Education are provided here courtesy of Accreditation Council for Graduate Medical Education