Related Articles

Background

The Accreditation Council for Graduate Medical Education (ACGME) requires an annual evaluation of all ACGME-accredited residency and fellowship programs to assess program quality. The results of this evaluation must be used to improve the program. This manuscript describes a metric to be used in conducting ACGME-mandated annual program review of ACGME-accredited anesthesiology residencies and fellowships.

Methods

A variety of metrics to assess anesthesiology residency and fellowship programs are identified by the authors through literature review and considered for use in constructing a program "report card."

Results

Metrics used to assess program quality include success in achieving American Board of Anesthesiology (ABA) certification, performance on the annual ABA/American Society of Anesthesiology In-Training Examination, performance on mock oral ABA certification examinations, trainee scholarly activities (publications and presentations), accreditation site visit and internal review results, ACGME and alumni survey results, National Resident Matching Program (NRMP) results, exit interview feedback, diversity data and extensive program/rotation/faculty/curriculum evaluations by trainees and faculty. The results are used to construct a "report card" that provides a high-level review of program performance and can be used in a continuous quality improvement process.

Conclusions

An annual program review is required to assess all ACGME-accredited residency and fellowship programs to monitor and improve program quality. We describe an annual review process based on metrics that can be used to focus attention on areas for improvement and track program performance year-to-year. A "report card" format is described as a high-level tool to track educational outcomes.

Background

In 1999, the Accreditation Council for Graduate Medical Education (ACGME) Outcome Project began to focus on resident performance in the 6 competencies of patient care, medical knowledge, professionalism, practice-based learning and improvement, interpersonal communication skills, and professionalism. Beginning in 2007, the ACGME began collecting information on how programs assess these competencies. This report provides information on the nature and extent of those assessments.

Methods

Using data collected by the ACGME for site visits, we use descriptive statistics and percentages to describe the number and type of methods and assessors accredited programs (n  =  4417) report using to assess the competencies. Observed differences among specialties, methodologies, and assessors are tested with analysis of variance procedures.

Results

Almost all (>97%) of programs report assessing all of the competencies and using multiple methods and multiple assessors. Similar assessment methods and evaluator types were consistently used across the 6 competencies. However, there were some differences in the use of patient and family as assessors: Primary care and ambulatory specialties used these to a greater extent than other specialties.

Conclusion

Residency programs are emphasizing the competencies in their evaluation of residents. Understanding the scope of evaluation methodologies that programs use in resident assessment is important for both the profession and the public, so that together we may monitor continuing improvement in US graduate medical education.

Background

The American Board of Internal Medicine Certification Examination (ABIM-CE) is one of several methods used to assess medical knowledge, an Accreditation Council for Graduate Medical Education (ACGME) core competency for graduating internal medicine residents. With recent changes in graduate medical education program directors and internal medicine residents are seeking evidence to guide decisions regarding residency elective choices. Prior studies have shown that formalized elective curricula improve subspecialty ABIM-CE scores. The primary aim of this study was to evaluate whether the number of subspecialty elective exposures or the specific subspecialties which residents complete electives in impact ABIM-CE scores.

Methods

ABIM-CE scores, elective exposures and demographic characteristics were collected for MedStar Georgetown University Hospital internal medicine residents who were first-time takers of the ABIM-CE in 2006–2010 (n=152). Elective exposures were defined as a two-week period assigned to the respective subspecialty. ABIM-CE score was analyzed using the difference between the ABIM-CE score and the standardized passing score (delta-SPS). Subspecialty scores were analyzed using percentage of correct responses. Data was analyzed using GraphPad Prism version 5.00 for Windows.

Results

Paired elective exposure and ABIM-CE scores were available in 131 residents. There was no linear correlation between ABIM-CE mean delta-SPS and the total number of electives or the number of unique elective exposures. Residents with ≤14 elective exposures had higher ABIM-CE mean delta-SPS than those with ≥15 elective exposures (143.4 compared to 129.7, p=0.051). Repeated electives in individual subspecialties were not associated with significant difference in mean ABIM-CE delta-SPS.

Conclusions

This study did not demonstrate significant positive associations between individual subspecialty elective exposures and ABIM-CE mean delta-SPS score. Residents with ≤14 elective exposures had higher ABIM-CE mean delta-SPS than those with ≥15 elective exposures suggesting there may be an “ideal” number of elective exposures that supports improved ABIM-CE performance. Repeated elective exposures in an individual specialty did not correlate with overall or subspecialty ABIM-CE performance.

Background

The Accreditation Council for Graduate Medical Education (ACGME) Outcome Project requires that residency program directors objectively document that their residents achieve competence in 6 general dimensions of practice.

Intervention

In November 2007, the American Board of Internal Medicine (ABIM) and the ACGME initiated the development of milestones for internal medicine residency training. ABIM and ACGME convened a 33-member milestones task force made up of program directors, experts in evaluation and quality, and representatives of internal medicine stakeholder organizations. This article reports on the development process and the resulting list of proposed milestones for each ACGME competency.

Outcomes

The task force adopted the Dreyfus model of skill acquisition as a framework the internal medicine milestones, and calibrated the milestones with the expectation that residents achieve, at a minimum, the “competency” level in the 5-step progression by the completion of residency. The task force also developed general recommendations for strategies to evaluate the milestones.

Discussion

The milestones resulting from this effort will promote competency-based resident education in internal medicine, and will allow program directors to track the progress of residents and inform decisions regarding promotion and readiness for independent practice. In addition, the milestones may guide curriculum development, suggest specific assessment strategies, provide benchmarks for resident self-directed assessment-seeking, and assist remediation by facilitating identification of specific deficits. Finally, by making explicit the profession's expectations for graduates and providing a degree of national standardization in evaluation, the milestones may improve public accountability for residency training.

Background

In July 2003, the Accreditation Council for Graduate Medical Education (ACGME) introduced a set of regulations that mandated a reduction in the number of hours that medical residents can work. These requirements have generated controversy among medical educators, with some expressing concern that reducing resident hours may limit clinical exposure and competency, particularly in surgical specialties.

Objective

This study examines the impact of duty hour restrictions on resident operative experience in residents in 2 surgical subspecialties since the implementation of the ACGME duty hour limits.

Method

We examined operative log data for vascular surgery and pediatric surgery, using the academic year immediately preceding the duty hour restrictions, 2002 to 2003, as a baseline for comparison to subsequent academic years through 2006 to 2007 for vascular surgery and 2007 to 2008 for pediatric surgery.

Results

Graduating fellows in pediatric surgery showed no change in their total operative volume following duty hour restrictions. The pediatric-defined category of neonate procedures showed an increase following duty hour restrictions. Graduating fellows in vascular surgery showed an increase in total major procedures as surgeon. The vascular-defined categories of endovascular-diagnostic, endovascular-therapeutic, and endovascular-graft procedures also increased.

Conclusions

The reduction of duty hours has not resulted in a decrease in operative volume as some have predicted. Operative volume in pediatric surgery remained mainly unchanged, whereas operative volume in vascular surgery increased. We explore possible explanations for the observed findings.

Background

An important expectation of pediatric education is assessing, resuscitating, and stabilizing ill or injured children.

Objective

To determine whether the Accreditation Council for Graduate Medical Education (ACGME) minimum time requirement for emergency and acute illness experience is adequate to achieve the educational objectives set forth for categorical pediatric residents. We hypothesized that despite residents working five 1-month block rotations in a high-volume (95 000 pediatric visits per year) pediatric emergency department (ED), the comprehensive experience outlined by the ACGME would not be satisfied through clinical exposure.

Study Design

This was a retrospective, descriptive study comparing actual resident experience to the standard defined by the ACGME. The emergency medicine experience of 35 categorical pediatric residents was tracked including number of patients evaluated during training and patient discharge diagnoses. The achievability of the ACGME requirement was determined by reporting the percentage of pediatric residents that cared for at least 1 patient from each of the ACGME-required disorder categories.

Results

A total of 11.4% of residents met the ACGME requirement for emergency and acute illness experience in the ED. The median number of patients evaluated by residents during training in the ED was 941. Disorder categories evaluated least frequently included shock, sepsis, diabetic ketoacidosis, coma/altered mental status, cardiopulmonary arrest, burns, and bowel obstruction.

Conclusion

Pediatric residents working in one of the busiest pediatric EDs in the country and working 1 month more than the ACGME-recommended minimum did not achieve the ACGME requirement for emergency and acute illness experience through direct patient care.

Background

In the United States, the Accreditation Council of graduate medical education (ACGME) requires all accredited Internal medicine residency training programs to facilitate resident scholarly activities. However, clinical experience and medical education still remain the main focus of graduate medical education in many Internal Medicine (IM) residency-training programs. Left to design the structure, process and outcome evaluation of the ACGME research requirement, residency-training programs are faced with numerous barriers. Many residency programs report having been cited by the ACGME residency review committee in IM for lack of scholarly activity by residents.

Methods

We would like to share our experience at Lincoln Hospital, an affiliate of Weill Medical College Cornell University New York, in designing and implementing a successful structured research curriculum based on ACGME competencies taught during a dedicated "research rotation".

Results

Since the inception of the research rotation in 2004, participation of our residents among scholarly activities has substantially increased. Our residents increasingly believe and appreciate that research is an integral component of residency training and essential for practice of medicine.

Conclusion

Internal medicine residents' outlook in research can be significantly improved using a research curriculum offered through a structured and dedicated research rotation. This is exemplified by the improvement noted in resident satisfaction, their participation in scholarly activities and resident research outcomes since the inception of the research rotation in our internal medicine training program.

OBJECTIVE: To describe the views of residency program directors regarding the effect of the 2010 duty hour recommendations on the 6 core competencies of graduate medical education.

METHODS: US residency program directors in internal medicine, pediatrics, and general surgery were e-mailed a survey from July 8 through July 20, 2010, after the 2010 Accreditation Council for Graduate Medical Education (ACGME) duty hour recommendations were published. Directors were asked to rate the implications of the new recommendations for the 6 ACGME core competencies as well as for continuity of inpatient care and resident fatigue.

RESULTS: Of 719 eligible program directors, 464 (65%) responded. Most program directors believe that the new ACGME recommendations will decrease residents' continuity with hospitalized patients (404/464 [87%]) and will not change (303/464 [65%]) or will increase (26/464 [6%]) resident fatigue. Additionally, most program directors (249-363/464 [53%-78%]) believe that the new duty hour restrictions will decrease residents' ability to develop competency in 5 of the 6 core areas. Surgery directors were more likely than internal medicine directors to believe that the ACGME recommendations will decrease residents' competency in patient care (odds ratio [OR], 3.9; 95% confidence interval [CI], 2.5-6.3), medical knowledge (OR, 1.9; 95% CI, 1.2-3.2), practice-based learning and improvement (OR, 2.7; 95% CI, 1.7-4.4), interpersonal and communication skills (OR, 1.9; 95% CI, 1.2-3.0), and professionalism (OR, 2.5; 95% CI, 1.5-4.0).

CONCLUSION: Residency program directors' reactions to ACGME duty hour recommendations demonstrate a marked degree of concern about educating a competent generation of future physicians in the face of increasing duty hour standards and regulation.

The reactions of residency program directors to the ACGME duty hour recommendations demonstrate a marked degree of concern about educating a competent generation of future physicians in the face of increasing duty hour standards and regulation.

Background

The Accreditation Council on Graduate Medical Education (ACGME) supports chart audit as a method to track competency in Practice-Based Learning and Improvement. We examined whether peer chart audits performed by internal medicine residents were associated with improved documentation of foot care in patients with diabetes mellitus.

Methods

A retrospective electronic chart review was performed on 347 patients with diabetes mellitus cared for by internal medicine residents in a university-based continuity clinic from May 2003 to September 2004. Residents abstracted information pertaining to documentation of foot examinations (neurological, vascular, and skin) from the charts of patients followed by their physician peers. No formal feedback or education was provided.

Results

Significant improvement in the documentation of foot exams was observed over the course of the study. The percentage of patients receiving neurological, vascular, and skin exams increased by 20% (from 13% to 33%) (p = 0.001), 26% (from 45% to 71%) (p < 0.001), and 18% (51%–72%) (p = 0.005), respectively. Similarly, the proportion of patients receiving a well-documented exam which includes all three components – neurological, vascular and skin foot exam – increased over time (6% to 24%, p < 0.001).

Conclusion

Peer chart audits performed by residents in the absence of formal feedback were associated with improved documentation of the foot exam in patients with diabetes mellitus. Although this study suggests that peer chart audits may be an effective tool to improve practice-based learning and documentation of foot care in diabetic patients, evaluating the actual performance of clinical care was beyond the scope of this study and would be better addressed by a randomized controlled trial.

Objective: To review the current status and recent trends in the American Board of Psychiatry and Neurology (ABPN) specialties and neurologic subspecialties and discuss the implications of those trends for subspecialty viability.

Methods: Data on numbers of residency and fellowship programs and graduates and ABPN certification candidates and diplomates were drawn from several sources, including ABPN records, Web sites of the Accreditation Council for Graduate Medical Education and the American Medical Association, and the annual medical education issues of the Journal of the American Medical Association.

Results: About four-fifths of neurology graduates pursue fellowship training. While most recent neurology and child neurology graduates attempt to become certified by the ABPN, many clinical neurophysiologists elect not to do so. There appears to have been little interest in establishing fellowships in neurodevelopmental disabilities. The pass rate for fellowship graduates is equivalent to that for the “grandfathers” in clinical neurophysiology. Lower percentages of clinical neurophysiologists than specialists participate in maintenance of certification, and maintenance of certification pass rates are high.

Conclusion: The initial enthusiastic interest in training and certification in some of the ABPN neurologic subspecialties appears to have slowed, and the long-term viability of those subspecialties will depend upon the answers to a number of complicated social, economic, and political questions in the new health care era.

Background

The Accreditation Council for Graduate Medical Education (ACGME) uses a 29-question Resident Survey for yearly residency program assessments. This article describes methodology for aggregating Resident Survey data into 5 discrete areas of program performance for use in the accreditation process. This article also describes methodology for setting thresholds that may assist Residency Review Committees in identifying programs with potential compliance problems.

Methods

A team of ACGME staff and Residency Review Committee chairpersons reviewed the survey for content and proposed thresholds (through a modified Angoff procedure) that would indicate problematic program functioning.

Results

Interrater agreement was high for the 5 content areas and for the threshold values (percentage of noncompliant residents), indicating that programs above these thresholds may warrant follow-up by the accrediting organization. Comparison of the Angoff procedure and the actual distribution of the data revealed that the Angoff thresholds were extremely similar to 1 standard deviation above the content area mean.

Conclusion

Data from the ACGME Resident Survey may be aggregated into internally consistent and consensually valid areas that may help Residency Review Committees make more targeted and specific judgments about program compliance.

OBJECTIVES

1) To describe how internal medicine residency programs fulfill the Accreditation Council for Graduate Medical Education (ACGME) scholarly activity training requirement including the current context of resident scholarly work, and 2) to compare findings between university and nonuniversity programs.

DESIGN

Cross-sectional mailed survey.

SETTING

ACGME-accredited internal medicine residency programs.

PARTICIPANTS

Internal medicine residency program directors.

MEASUREMENTS

Data were collected on 1) interpretation of the scholarly activity requirement, 2) support for resident scholarship, 3) scholarly activities of residents, 4) attitudes toward resident research, and 5) program characteristics. University and nonuniversity programs were compared.

MAIN RESULTS

The response rate was 78%. Most residents completed a topic review with presentation (median, 100%) to fulfill the requirement. Residents at nonuniversity programs were more likely to complete case reports (median, 40% vs 25%; P =.04) and present at local or regional meetings (median, 25% vs 20%; P =.01), and were just as likely to conduct hypothesis-driven research (median, 20% vs 20%; P =.75) and present nationally (median, 10% vs 5%; P =.10) as residents at university programs. Nonuniversity programs were more likely to report lack of faculty mentors (61% vs 31%; P <.001) and resident interest (55% vs 40%; P =.01) as major barriers to resident scholarship. Programs support resident scholarship through research curricula (47%), funding (46%), and protected time (32%).

CONCLUSIONS

Internal medicine residents complete a variety of projects to fulfill the scholarly activity requirement. Nonuniversity programs are doing as much as university programs in meeting the requirement and supporting resident scholarship despite reporting significant barriers.

Background

The Accreditation Council for Graduate Medical Education (ACGME) core competencies are used to assess resident performance, and recently similar competencies have become an accepted framework for evaluating medical student achievements as well. However, the utility of incorporating the competencies into the resident application has not yet been assessed.

Purpose

The objective of this study was to examine letters of recommendation (LORs) to identify ACGME competency–based themes that might help distinguish the least successful from the most successful residents.

Methods

Residents entering a university-based residency program from 1994 to 2004 were retrospectively evaluated by faculty and ranked in 4 groups according to perceived level of success. Applications from residents in the highest and lowest groups were abstracted. LORs were qualitatively reviewed and analyzed for 9 themes (6 ACGME core competencies and 3 additional performance measures). The mean number of times each theme was mentioned was calculated for each student. Groups were compared using the χ2 test and the Student t test.

Results

Seventy-five residents were eligible for analysis, and 29 residents were ranked in the highest and lowest groups. Baseline demographics and number of LORs did not differ between the two groups. Successful residents had statistically significantly more comments about excellence in the competency areas of patient care, medical knowledge, and interpersonal and communication skills.

Conclusion

LORs can provide useful clues to differentiate between students who are likely to become the least versus the most successful residency program graduates. Greater usage of the ACGME core competencies within LORs may be beneficial.

The Accreditation Council of Graduate Medical Education’s (ACGME) Data Accreditation System indicates 124 of 152 orthopaedic surgery residency program directors have 5 or fewer years of tenure. The qualifications and responsibilities of the position based on the requirements of orthopaedic surgery residency programs, the institutions that support them, and the ACGME Outcome Project have evolved the role of the program coordinator from clerical to managerial. To fill the void of information on the coordinators’ expanding roles and responsibilities, the 2006 Association of Residency Coordinators in Orthopaedic Surgery (ARCOS) Career survey was designed and distributed to 152 program coordinators in the United States. We had a 39.5% response rate for the survey, which indicated a high level of day-to-day managerial oversight of all aspects of the residency program; additional responsibilities for other department or division functions for fellows, rotating medical students, continuing medical education of the faculty; and miscellaneous business functions. Although there has been expansion of the role of the program coordinator, challenges exist in job congruence and position reclassification. We believe use of professional groups such as ARCOS and certification of program coordinators should be supported and encouraged.

Background

The Outcome Project requires high-quality assessment approaches to provide reliable and valid judgments of the attainment of competencies deemed important for physician practice.

Intervention

The Accreditation Council for Graduate Medical Education (ACGME) convened the Advisory Committee on Educational Outcome Assessment in 2007–2008 to identify high-quality assessment methods. The assessments selected by this body would form a core set that could be used by all programs in a specialty to assess resident performance and enable initial steps toward establishing national specialty databases of program performance. The committee identified a small set of methods for provisional use and further evaluation. It also developed frameworks and processes to support the ongoing evaluation of methods and the longer-term enhancement of assessment in graduate medical education.

Outcome

The committee constructed a set of standards, a methodology for applying the standards, and grading rules for their review of assessment method quality. It developed a simple report card for displaying grades on each standard and an overall grade for each method reviewed. It also described an assessment system of factors that influence assessment quality. The committee proposed a coordinated, national-level infrastructure to support enhancements to assessment, including method development and assessor training. It recommended the establishment of a new assessment review group to continue its work of evaluating assessment methods. The committee delivered a report summarizing its activities and 5 related recommendations for implementation to the ACGME Board in September 2008.

Background

In 2006, the University of Virginia became one of the first academic medical institutions to be placed on probation, after the Accreditation Council for Graduate Medical Education (ACGME) Institutional Review Committee implemented a new classification system for institutional reviews.

Intervention

After University of Virginia reviewed its practices and implemented needed changes, the institution was able to have probation removed and full accreditation restored. Whereas graduate medical education committees and designated institutional officials are required to conduct internal reviews of each ACGME–accredited program midway through its accreditation cycle, no similar requirement exists for institutions.

Learning

As we designed corrective measures at the University of Virginia, we realized that regularly scheduled audits of the entire institution would have prevented the accumulation of deficiencies. We suggest that institutional internal reviews be implemented to ensure that the ACGME institutional requirements for graduate medical education are met. This process represents practice-based learning and improvement at the institutional level and may prevent other institutions from receiving unfavorable accreditation decisions.

BACKGROUND

In July 2011, the Accreditation Council for Graduate Medical Education (ACGME) will implemented stricter duty-hour limits and related changes to the training environment. This may affect preventable adverse event (PAE) rates.

OBJECTIVES

To estimate direct costs under various implementation approaches, and examine net costs to teaching hospitals and cost-effectiveness to society across a range of hypothetical changes in PAEs.

DESIGN

A decision-analytical model represented direct costs and PAE rates, mortality, and costs.

DATA SOURCES

Published literature and publicly available data.

TARGET POPULATION

Patients admitted to hospitals with ACGME-accredited programs.

TIME HORIZON

One year.

PERSPECTIVES

All teaching hospitals, major teaching hospitals, society.

INTERVENTION

ACGME’s 2011 Common Program Requirements.

OUTCOME MEASURES

Direct annual costs (all accredited hospitals), net cost (major teaching hospitals), cost per death averted (society).

RESULTS OF BASE-ANALYSIS

Nationwide, duty-hour changes would cost $177 million annually if interns maintain current productivity, vs. up to $982 million if they transfer work to a mixture of substitutes; training-environment changes will cost $204 million. If PAEs decline by 7.2–25.8%, net costs to major teaching hospitals will be zero. If PAEs fall by 3%, the cost to society per death averted would be –$523,000 (95%-confidence interval: –$1.82 million to $685,000) to $2.44 million ($271,000 to $6.91 million). If PAEs rise, the policy will be cost-increasing for teaching hospitals and society.

RESULTS OF SENSITIVITY ANALYSIS

The total direct annual cost nationwide would be up to $1.34 billion using nurse practitioners/physician assistants, $1.64 billion using attending physicians, $820 million hiring additional residents, vs. 1.42 billion using mixed substitutes.

LIMITATIONS

The effect on PAEs is unknown. Data were limited for some model parameters.

CONCLUSION

Implementation decisions greatly affect the cost. Unless PAEs decline substantially, teaching hospitals will lose money. If PAEs decline modestly, the requirements might be cost-saving or cost-effective to society.

Electronic supplementary material

The online version of this article (doi:10.1007/s11606-011-1775-9) contains supplementary material, which is available to authorized users.

Background

The Accreditation Council for Graduate Medical Education (ACGME) requirements stipulate that psychiatry residents need to be educated in the area of emergency psychiatry. Existing research investigating the current state of this training is limited, and no research to date has assessed whether the ACGME Residency Review Committee requirements for psychiatry residency training are followed by psychiatry residency training programs.

Methods

We administered, to chief resident attendees of a national leadership conference, a 24‐item paper survey on the types and amount of emergency psychiatry training provided by their psychiatric residency training programs. Descriptive statistics were used in the analysis.

Results

Of 154 surveys distributed, 111 were returned (72% response rate). Nearly one‐third of chief resident respondents indicated that more than 50% of their program's emergency psychiatry training was provided during on‐call periods. A minority indicated that they were aware of the ACGME program requirements for emergency psychiatry training. While training in emergency psychiatry occurred in many programs through rotations—different from the on‐call period—direct supervision was available during on‐call training only about one‐third of the time.

Conclusions

The findings suggest that about one‐third of psychiatry residency training programs do not adhere to the ACGME standards for emergency psychiatry training. Enhanced knowledge of the ACGME requirements may enhance psychiatry residents' understanding on how their programs are fulfilling the need for more emergency psychiatry training. Alternative settings to the on‐call period for emergency psychiatry training are more likely to provide for direct supervision.

Objective

To assess the value of a faculty and resident medical education development program.

Study Design

Modules on Accreditation Council for Graduate Medical Education (ACGME) competencies and evaluation, teaching methods, and Residency Review Committee guidelines were created, beta tested, and installed on a website. Pretests and posttests were developed. Faculty and residents were required to complete the course. At initiation and 6 months after training, residents completed a feedback perception survey. Statistical analysis was performed using Student t test. P < .05 was considered significant.

Results

Forty-nine voluntary faculty members and residents completed the course. The posttest scores on all the ACGME competencies were significantly higher than the pretest scores (P < .05). The results of the residents' survey indicated that the educational development program significantly improved their perceptions of corrective and immediate feedback by faculty.

Conclusion

A formal Internet-based program significantly increases short-term cognitive knowledge about the ACGME competencies among participants and improves trainees' perceptions of the quality of faculty feedback up to 6 months after training.

Background

Because of the Accreditation Council for Graduate Medical Education (ACGME) and the Residency Review Committee (RRC) approval timelines, new residency programs cannot use Electronic Residency Application Service (ERAS) during their first year of applicants.

Aim

We sought to identify differences between program directors’ subjective ratings of applicants from an emergency medicine (EM) residency program’s first year (in which ERAS was not used) to their ratings of applicants the following year in which ERAS was used.

Method

The University of Utah Emergency Medicine Residency Program received approval from the ACGME in 2004. Applicants for the entering class of 2005 (year 1) did not use ERAS, submitting a separate application, while those applying for the following year (year 2) used ERAS. Residency program directors rated applicants using subjective components of their applications, assigning scores on scales from 0–10 or 0–5 (10 or 5 = highest score) for select components of the application. We retrospectively reviewed and compared these ratings between the 2 years of applicants.

Results

A total of 130 and 458 prospective residents applied during year 1 and year 2, respectively. Applicants were similar in average scores for research (1.65 vs. 1.81, scale 0–5, p = 0.329) and volunteer work (5.31 vs. 5.56, scale 0–10, p = 0.357). Year 1 applicants received higher scores for their personal statement (3.21 vs. 2.22, scale 0–5, p < 0.001), letters of recommendation (7.0 vs. 5.94, scale 0–10, p < 0.001), dean’s letter (3.5 vs. 2.7, scale 1–5, p < 0.001), and in their potential contribution to class characteristics (4.64 vs. 3.34, scale 0–10, p < 0.001).

Conclusion

While the number of applicants increased, the use of ERAS in a new residency program did not improve the overall subjective ratings of residency applicants. Year 1 applicants received higher scores for the written components of their applications and in their potential contributions to class characteristics.

Background

The Accreditation Council for Graduate Medical Education (ACGME) and the American Board of Pediatrics (ABP) have partnered to initiate the Pediatrics Milestone Project to further refine the 6 ACGME competencies and to set performance standards as part of the continued commitment to document outcomes of training and program effectiveness.

Intervention

Members of the Pediatrics Milestone Project Working Group searched the medical literature and beyond to create a synopsis of models and evidence for a developmental ontogeny of the elements for 52 subcompetencies. For each subcompetency, we created a series of Milestones, grounded in the literature. The milestones were vetted with the entire working group, engaging in an iterative process of revisions until reaching consensus that their narrative descriptions (1) included all critical elements, (2) were behaviorally based, (3) were properly sequenced, and (4) represented the educational continuum of training and practice.

Outcomes

We have completed the first iteration of milestones for all subcompetencies. For each milestone, a synopsis of relevant literature provides background, references, and a conceptual framework. These milestones provide narrative descriptions of behaviors that represent the ontogeny of knowledge, skill, and attitude development across the educational continuum of training and practice.

Discussion

The pediatrics milestones take us a step closer to meaningful outcome assessment. Next steps include undertaking rigorous study, making appropriate modifications, and setting performance standards. Our aim is to assist program directors in making more reliable and valid judgments as to whether a resident is a “good doctor” and to provide outcome evidence regarding the program's success in developing doctors.

Background

The Accreditation Council for Graduate Medical Education (ACGME) program requirements mandate “adequate supervision,” of residents, but there is little guidance for sports medicine fellowship directors regarding the transition from direct to indirect supervision of fellows covering football games.

Objective

We sought to gather evidence of current supervision practices in the context of injury outcomes.

Methods

Fellows and program directors of ACGME-accredited sports medicine fellowship programs were invited to complete an online survey regarding their experience and current supervision practice at football games. Criteria for transition to autonomy and desired changes in supervision practice were elicited. Player safety was quantified by noting the number of field-side emergencies, whether an attending was present, and whether better outcomes might have resulted from the presence of an attending.

Results

A total of 80 fellows and 50 program directors completed the online survey. Direct supervision was lacking in about 50% of high school games and 20% of college games. A resulting cost in terms of player safety was estimated to apply to 5% of serious injuries by fellows' report but less than 0.5% by directors' report. Written criteria for transitioning from direct supervision to autonomy were the exception rather than the rule. The majority of fellows and directors expressed satisfaction with the current level of supervision, but 20% of fellows would prefer more supervision through postgame review.

Conclusions

Football games covered by fellows are often not directly supervised. Absence of an attending affected the outcomes of 5% or less of serious injuries. Transition to autonomy does not usually require meeting written criteria. Fellows might benefit from additional off-site supervision.

Background

Board certification is an important professional qualification and a prerequisite for credentialing, and the Accreditation Council for Graduate Medical Education (ACGME) assesses board certification rates as a component of residency program effectiveness. To date, research has shown that preresidency measures, including National Board of Medical Examiners scores, Alpha Omega Alpha Honor Medical Society membership, or medical school grades poorly predict postresidency board examination scores. However, learning styles and temperament have been identified as factors that 5 affect test-taking performance. The purpose of this study is to characterize the learning styles and temperaments of pediatric residents and to evaluate their relationships to yearly in-service and postresidency board examination scores.

Methods

This cross-sectional study analyzed the learning styles and temperaments of current and past pediatric residents by administration of 3 validated tools: the Kolb Learning Style Inventory, the Keirsey Temperament Sorter, and the Felder-Silverman Learning Style test. These results were compared with known, normative, general and medical population data and evaluated for correlation to in-service examination and postresidency board examination scores.

Results

The predominant learning style for pediatric residents was converging 44% (33 of 75 residents) and the predominant temperament was guardian 61% (34 of 56 residents). The learning style and temperament distribution of the residents was significantly different from published population data (P  =  .002 and .04, respectively). Learning styles, with one exception, were found to be unrelated to standardized test scores.

Conclusions

The predominant learning style and temperament of pediatric residents is significantly different than that of the populations of general and medical trainees. However, learning styles and temperament do not predict outcomes on standardized in-service and board examinations in pediatric residents.

Purpose

The Accreditation Council for Graduate Medical Education (ACGME) expects programs to engage in ongoing, meaningful improvement, facilitated in part through an annual process of program assessment and improvement. The Duke University Hospital Office of Graduate Medical Education (OGME) used an institutional practice-based learning and improvement strategy to improve the annual evaluation and improvement of its programs.

Methods

The OGME implemented several strategies including the development and dissemination of a template for the report, program director and coordinator development, a reminder and tracking system, incorporation of the document into internal reviews, and use of incentives to promote program adherence.

Results

In the first year of implementation (summer 2005), 27 programs (37%) submitted documentation of their annual program evaluation and improvement to the OGME; this increased to 100% of programs by 2009. A growing number of programs elected to use the template in lieu of written minutes. The number of citations related to required program review and improvement decreased from 12 in a single academic year to 3 over the last 5 years.

Conclusion

Duke University Hospital's institutional initiative to incorporate practice-based learning and improvement resulted in increased documentation, greater use of a standardized template, fewer ACGME-related citations, and enhanced consistency in preparing for ACGME site visits.

Purpose

A design conference with participants from accredited programs and institutions was used to explore how the principles of patient- and family-centered care (PFCC) can be implemented in settings where residents learn and participate in care, as well as identify barriers to PFCC and simple strategies for overcoming them.

Approach

In September 2009, the Accreditation Council for Graduate Medical Education (ACGME) held a conference with 74 participants representing a diverse range of educational settings and a group of expert presenters and facilitators. Small group sessions explored the status of PFCC in teaching settings, barriers that need to be overcome in some settings, simple approaches, and the value of a national program and ACGME support.

Findings

Participants shared information on the state of their PFCC initiatives, as well as barriers to implementing PFCC in the learning environment. These emerged in 6 areas: culture, the physical environment, people, time and other constraints, skills and capabilities, and teaching and assessment, as well as simple strategies to help overcome these barriers. Two Ishikawa (Fishbone) diagrams (one for barriers and one for simple strategies) make it possible to select strategies for overcoming particular barriers.

Conclusions

A group of participants with a diversity of approaches to incorporating PFCC into the learning environment agreed that respectful communication with patients/families needs to be learned, supported, and continuously demanded of residents. In addition, for PFCC to be sustainable, it has to be a fundamental expectation for resident learning and attainment of competence. Participants concurred that improving the environment for patients concurrently improves the environment for learners.