PMCC PMCC

Search tips
Search criteria

Advanced
Results 1-25 (964148)

Clipboard (0)
None

Related Articles

1.  Impact of subspecialty elective exposures on outcomes on the American board of internal medicine certification examination 
BMC Medical Education  2012;12:94.
Background
The American Board of Internal Medicine Certification Examination (ABIM-CE) is one of several methods used to assess medical knowledge, an Accreditation Council for Graduate Medical Education (ACGME) core competency for graduating internal medicine residents. With recent changes in graduate medical education program directors and internal medicine residents are seeking evidence to guide decisions regarding residency elective choices. Prior studies have shown that formalized elective curricula improve subspecialty ABIM-CE scores. The primary aim of this study was to evaluate whether the number of subspecialty elective exposures or the specific subspecialties which residents complete electives in impact ABIM-CE scores.
Methods
ABIM-CE scores, elective exposures and demographic characteristics were collected for MedStar Georgetown University Hospital internal medicine residents who were first-time takers of the ABIM-CE in 2006–2010 (n=152). Elective exposures were defined as a two-week period assigned to the respective subspecialty. ABIM-CE score was analyzed using the difference between the ABIM-CE score and the standardized passing score (delta-SPS). Subspecialty scores were analyzed using percentage of correct responses. Data was analyzed using GraphPad Prism version 5.00 for Windows.
Results
Paired elective exposure and ABIM-CE scores were available in 131 residents. There was no linear correlation between ABIM-CE mean delta-SPS and the total number of electives or the number of unique elective exposures. Residents with ≤14 elective exposures had higher ABIM-CE mean delta-SPS than those with ≥15 elective exposures (143.4 compared to 129.7, p=0.051). Repeated electives in individual subspecialties were not associated with significant difference in mean ABIM-CE delta-SPS.
Conclusions
This study did not demonstrate significant positive associations between individual subspecialty elective exposures and ABIM-CE mean delta-SPS score. Residents with ≤14 elective exposures had higher ABIM-CE mean delta-SPS than those with ≥15 elective exposures suggesting there may be an “ideal” number of elective exposures that supports improved ABIM-CE performance. Repeated elective exposures in an individual specialty did not correlate with overall or subspecialty ABIM-CE performance.
doi:10.1186/1472-6920-12-94
PMCID: PMC3480921  PMID: 23057635
Resident education; Gender; Elective; Subspecialty; Graduate medical education
2.  Optimizing the Implementation of Practice Improvement Modules in Training: Lessons from Educators 
Background
The American Board of Internal Medicine approved the use of Practice Improvement Modules (PIMs) to help training programs teach and assess practice-based learning and improvement (PBLI) and systems-based practice (SBP).
Methods
We surveyed individuals who ordered a PIM in a residency or fellowship training program between June 2006 and August 2009. The 43 programs that volunteered to participate completed a 30-minute anonymous online survey.
Results
Program directors or associate program directors led the PIM process in 30 programs (70%). Trainees' degrees of involvement in PIMs were highly variable between programs, and several respondents felt that trainees were either not sufficiently engaged or not engaged with enough consistency. The most common activity for trainee involvement was data collection through patient surveys or chart review, although only 17 programs (40%) provided protected time for this activity. Few trainees participated in higher level activities such as data analysis or identification for areas of improvement or were given leadership roles; yet most respondents reported that completing the PIM helped trainees learn basic principles of QI and develop competence in PBLI and SBP and that PIM completion improved the program's ability to develop QI initiatives and resulted in program or institutional improvements, including sustainable improvement in patient care. Most respondents reported that the outcome warranted the effort to complete PIMs.
Conclusions
PIMs may be a valuable but underused educational experience for trainees as well as training programs. Focusing on particular themes and facets of PIMs may facilitate implementation.
doi:10.4300/JGME-D-11-00281.1
PMCID: PMC3613323  PMID: 24404231
3.  Teaching Internal Medicine Residents to Sustain Their Improvement Through the Quality Assessment and Improvement Curriculum 
ABSTRACT
INTRODUCTION
Although sustainability is a key component in the evaluation of continuous quality improvement (CQI) projects, medicine resident CQI projects are often evaluated by immediate improvements in targeted areas without addressing sustainability.
AIM/SETTING
To assess the sustainability of resident CQI projects in an ambulatory university-based clinic.
PROGRAM DESCRIPTION
During their ambulatory rotation, all second year internal medicine residents use the American Board of Internal Medicine’s Clinical Preventive Services (CPS) Practice Improvement Modules (PIM) to complete chart reviews, patient surveys, and a system survey. The residents then develop a group CQI project and collect early post data. Third year residents return to evaluate their original CQI project during an ambulatory rotation two to six months later and complete four plan-do-study-act (PDSA) cycles on each CQI project.
PROGRAM EVALUATION
From July 2006 to June 2009, 64 (100%) medicine residents completed the CQI curriculum. Residents completed six group projects and examined their success using early (2 to 6 weeks) and late (2 to 6 months) post-intervention data. Three of the projects demonstrated sustainable improvement in the resident continuity clinic.
DISCUSSION
When residents are taught principles of sustainability and spread and asked to complete multiple PDSA cycles, they are able to identify common themes that may contribute to success of QI projects over time.
Electronic supplementary material
The online version of this article (doi:10.1007/s11606-010-1547-y) contains supplementary material, which is available to authorized users.
doi:10.1007/s11606-010-1547-y
PMCID: PMC3019318  PMID: 21053089
resident education; quality improvement; sustainability; practice-based learning and improvement; system-based practice
4.  Instituting systems-based practice and practice-based learning and improvement: a curriculum of inquiry 
Medical Education Online  2013;18:10.3402/meo.v18i0.21612.
Background
The Accreditation Council for Graduate Medical Education (ACGME) requires that training programs integrate system-based practice (SBP) and practice-based learning and improvement (PBLI) into internal medicine residency curricula.
Context and setting
We instituted a seminar series and year-long-mentored curriculum designed to engage internal medicine residents in these competencies.
Methods
Residents participate in a seminar series that includes assigned reading and structured discussion with faculty who assist in the development of quality improvement or research projects. Residents pursue projects over the remainder of the year. Monthly works in progress meetings, protected time for inquiry, and continued faculty mentorship guide the residents in their project development. Trainees present their work at hospital-wide grand rounds at the end of the academic year. We performed a survey of residents to assess their self-reported knowledge, attitudes and skills in SBP and PBLI. In addition, blinded faculty scored projects for appropriateness, impact, and feasibility.
Outcomes
We measured resident self-reported knowledge, attitudes, and skills at the end of the academic year. We found evidence that participants improved their understanding of the context in which they were practicing, and that their ability to engage in quality improvement projects increased. Blinded faculty reviewers favorably ranked the projects’ feasibility, impact, and appropriateness. The ‘Curriculum of Inquiry’ generated 11 quality improvement and research projects during the study period. Barriers to the ongoing work include a limited supply of mentors and delays due to Institutional Review Board approval. Hospital leadership recognizes the importance of the curriculum, and our accreditation manager now cites our ongoing work.
Conclusions
A structured residency-based curriculum facilitates resident demonstration of SBP and practice-based learning and improvement. Residents gain knowledge and skills though this enterprise and hospitals gain access to trainees who help to solve ongoing problems and meet accreditation requirements.
doi:10.3402/meo.v18i0.21612
PMCID: PMC3776321  PMID: 24044686
graduate medical education; competencies; longitudinal curriculum
5.  A nomogram to predict the probability of passing the American Board of Internal Medicine examination 
Medical Education Online  2012;17:10.3402/meo.v17i0.18810.
Background
Although the American Board of Internal Medicine (ABIM) certification is valued as a reflection of physicians’ experience, education, and expertise, limited methods exist to predict performance in the examination.
Purpose
The objective of this study was to develop and validate a predictive tool based on variables common to all residency programs, regarding the probability of an internal medicine graduate passing the ABIM certification examination.
Methods
The development cohort was obtained from the files of the Cleveland Clinic internal medicine residents who began training between 2004 and 2008. A multivariable logistic regression model was built to predict the ABIM passing rate. The model was represented as a nomogram, which was internally validated with bootstrap resamples. The external validation was done retrospectively on a cohort of residents who graduated from two other independent internal medicine residency programs between 2007 and 2011.
Results
Of the 194 Cleveland Clinic graduates used for the nomogram development, 175 (90.2%) successfully passed the ABIM certification examination. The final nomogram included four predictors: In-Training Examination (ITE) scores in postgraduate year (PGY) 1, 2, and 3, and the number of months of overnight calls in the last 6 months of residency. The nomogram achieved a concordance index (CI) of 0.98 after correcting for over-fitting bias and allowed for the determination of an estimated probability of passing the ABIM exam. Of the 126 graduates from two other residency programs used for external validation, 116 (92.1%) passed the ABIM examination. The nomogram CI in the external validation cohort was 0.94, suggesting outstanding discrimination.
Conclusions
A simple user-friendly predictive tool, based on readily available data, was developed to predict the probability of passing the ABIM exam for internal medicine residents. This may guide program directors’ decision-making related to program curriculum and advice given to individual residents regarding board preparation.
doi:10.3402/meo.v17i0.18810
PMCID: PMC3475012  PMID: 23078794
board examination; in-training examination; internal medicine; residents; program directors
6.  Implementing the 2009 Institute of Medicine recommendations on resident physician work hours, supervision, and safety 
Long working hours and sleep deprivation have been a facet of physician training in the US since the advent of the modern residency system. However, the scientific evidence linking fatigue with deficits in human performance, accidents and errors in industries from aeronautics to medicine, nuclear power, and transportation has mounted over the last 40 years. This evidence has also spawned regulations to help ensure public safety across safety-sensitive industries, with the notable exception of medicine.
In late 2007, at the behest of the US Congress, the Institute of Medicine embarked on a year-long examination of the scientific evidence linking resident physician sleep deprivation with clinical performance deficits and medical errors. The Institute of Medicine’s report, entitled “Resident duty hours: Enhancing sleep, supervision and safety”, published in January 2009, recommended new limits on resident physician work hours and workload, increased supervision, a heightened focus on resident physician safety, training in structured handovers and quality improvement, more rigorous external oversight of work hours and other aspects of residency training, and the identification of expanded funding sources necessary to implement the recommended reforms successfully and protect the public and resident physicians themselves from preventable harm.
Given that resident physicians comprise almost a quarter of all physicians who work in hospitals, and that taxpayers, through Medicare and Medicaid, fund graduate medical education, the public has a deep investment in physician training. Patients expect to receive safe, high-quality care in the nation’s teaching hospitals. Because it is their safety that is at issue, their voices should be central in policy decisions affecting patient safety. It is likewise important to integrate the perspectives of resident physicians, policy makers, and other constituencies in designing new policies. However, since its release, discussion of the Institute of Medicine report has been largely confined to the medical education community, led by the Accreditation Council for Graduate Medical Education (ACGME).
To begin gathering these perspectives and developing a plan to implement safer work hours for resident physicians, a conference entitled “Enhancing sleep, supervision and safety: What will it take to implement the Institute of Medicine recommendations?” was held at Harvard Medical School on June 17–18, 2010. This White Paper is a product of a diverse group of 26 representative stakeholders bringing relevant new information and innovative practices to bear on a critical patient safety problem. Given that our conference included experts from across disciplines with diverse perspectives and interests, not every recommendation was endorsed by each invited conference participant. However, every recommendation made here was endorsed by the majority of the group, and many were endorsed unanimously. Conference members participated in the process, reviewed the final product, and provided input before publication. Participants provided their individual perspectives, which do not necessarily represent the formal views of any organization.
In September 2010 the ACGME issued new rules to go into effect on July 1, 2011. Unfortunately, they stop considerably short of the Institute of Medicine’s recommendations and those endorsed by this conference. In particular, the ACGME only applied the limitation of 16 hours to first-year resident physicans. Thus, it is clear that policymakers, hospital administrators, and residency program directors who wish to implement safer health care systems must go far beyond what the ACGME will require. We hope this White Paper will serve as a guide and provide encouragement for that effort.
Resident physician workload and supervision
By the end of training, a resident physician should be able to practice independently. Yet much of resident physicians’ time is dominated by tasks with little educational value. The caseload can be so great that inadequate reflective time is left for learning based on clinical experiences. In addition, supervision is often vaguely defined and discontinuous. Medical malpractice data indicate that resident physicians are frequently named in lawsuits, most often for lack of supervision. The recommendations are: The ACGME should adjust resident physicians workload requirements to optimize educational value. Resident physicians as well as faculty should be involved in work redesign that eliminates nonessential and noneducational activity from resident physician dutiesMechanisms should be developed for identifying in real time when a resident physician’s workload is excessive, and processes developed to activate additional providersTeamwork should be actively encouraged in delivery of patient care. Historically, much of medical training has focused on individual knowledge, skills, and responsibility. As health care delivery has become more complex, it will be essential to train resident and attending physicians in effective teamwork that emphasizes collective responsibility for patient care and recognizes the signs, both individual and systemic, of a schedule and working conditions that are too demanding to be safeHospitals should embrace the opportunities that resident physician training redesign offers. Hospitals should recognize and act on the potential benefits of work redesign, eg, increased efficiency, reduced costs, improved quality of care, and resident physician and attending job satisfactionAttending physicians should supervise all hospital admissions. Resident physicians should directly discuss all admissions with attending physicians. Attending physicians should be both cognizant of and have input into the care patients are to receive upon admission to the hospitalInhouse supervision should be required for all critical care services, including emergency rooms, intensive care units, and trauma services. Resident physicians should not be left unsupervised to care for critically ill patients. In settings in which the acuity is high, physicians who have completed residency should provide direct supervision for resident physicians. Supervising physicians should always be physically in the hospital for supervision of resident physicians who care for critically ill patientsThe ACGME should explicitly define “good” supervision by specialty and by year of training. Explicit requirements for intensity and level of training for supervision of specific clinical scenarios should be providedCenters for Medicare and Medicaid Services (CMS) should use graduate medical education funding to provide incentives to programs with proven, effective levels of supervision. Although this action would require federal legislation, reimbursement rules would help to ensure that hospitals pay attention to the importance of good supervision and require it from their training programs
Resident physician work hours
Although the IOM “Sleep, supervision and safety” report provides a comprehensive review and discussion of all aspects of graduate medical education training, the report’s focal point is its recommendations regarding the hours that resident physicians are currently required to work. A considerable body of scientific evidence, much of it cited by the Institute of Medicine report, describes deteriorating performance in fatigued humans, as well as specific studies on resident physician fatigue and preventable medical errors.
The question before this conference was what work redesign and cultural changes are needed to reform work hours as recommended by the Institute of Medicine’s evidence-based report? Extensive scientific data demonstrate that shifts exceeding 12–16 hours without sleep are unsafe. Several principles should be followed in efforts to reduce consecutive hours below this level and achieve safer work schedules. The recommendations are: Limit resident physician work hours to 12–16 hour maximum shiftsA minimum of 10 hours off duty should be scheduled between shiftsResident physician input into work redesign should be actively solicitedSchedules should be designed that adhere to principles of sleep and circadian science; this includes careful consideration of the effects of multiple consecutive night shifts, and provision of adequate time off after night work, as specified in the IOM reportResident physicians should not be scheduled up to the maximum permissible limits; emergencies frequently occur that require resident physicians to stay longer than their scheduled shifts, and this should be anticipated in scheduling resident physicians’ work shiftsHospitals should anticipate the need for iterative improvement as new schedules are initiated; be prepared to learn from the initial phase-in, and change the plan as neededAs resident physician work hours are redesigned, attending physicians should also be considered; a potential consequence of resident physician work hour reduction and increased supervisory requirements may be an increase in work for attending physicians; this should be carefully monitored, and adjustments to attending physician work schedules made as needed to prevent unsafe work hours or working conditions for this group“Home call” should be brought under the overall limits of working hours; work load and hours should be monitored in each residency program to ensure that resident physicians and fellows on home call are getting sufficient sleepMedicare funding for graduate medical education in each hospital should be linked with adherence to the Institute of Medicine limits on resident physician work hours
Moonlighting by resident physicians
The Institute of Medicine report recommended including external as well as internal moonlighting in working hour limits. The recommendation is: All moonlighting work hours should be included in the ACGME working hour limits and actively monitored. Hospitals should formalize a moonlighting policy and establish systems for actively monitoring resident physician moonlighting
Safety of resident physicians
The “Sleep, supervision and safety” report also addresses fatigue-related harm done to resident physicians themselves. The report focuses on two main sources of physical injury to resident physicians impaired by fatigue, ie, needle-stick exposure to blood-borne pathogens and motor vehicle crashes. Providing safe transportation home for resident physicians is a logistical and financial challenge for hospitals. Educating physicians at all levels on the dangers of fatigue is clearly required to change driving behavior so that safe hospital-funded transport home is used effectively. Fatigue-related injury prevention (including not driving while drowsy) should be taught in medical school and during residency, and reinforced with attending physicians; hospitals and residency programs must be informed that resident physicians’ ability to judge their own level of impairment is impaired when they are sleep deprived; hence, leaving decisions about the capacity to drive to impaired resident physicians is not recommendedHospitals should provide transportation to all resident physicians who report feeling too tired to drive safely; in addition, although consecutive work should not exceed 16 hours, hospitals should provide transportation for all resident physicians who, because of unforeseen reasons or emergencies, work for longer than consecutive 24 hours; transportation under these circumstances should be automatically provided to house staff, and should not rely on self-identification or request
Training in effective handovers and quality improvement
Handover practice for resident physicians, attendings, and other health care providers has long been identified as a weak link in patient safety throughout health care settings. Policies to improve handovers of care must be tailored to fit the appropriate clinical scenario, recognizing that information overload can also be a problem. At the heart of improving handovers is the organizational effort to improve quality, an effort in which resident physicians have typically been insufficiently engaged. The recommendations are: Hospitals should train attending and resident physicians in effective handovers of careHospitals should create uniform processes for handovers that are tailored to meet each clinical setting; all handovers should be done verbally and face-to-face, but should also utilize written toolsWhen possible, hospitals should integrate hand-over tools into their electronic medical records (EMR) systems; these systems should be standardized to the extent possible across residency programs in a hospital, but may be tailored to the needs of specific programs and services; federal government should help subsidize adoption of electronic medical records by hospitals to improve signoutWhen feasible, handovers should be a team effort including nurses, patients, and familiesHospitals should include residents in their quality improvement and patient safety efforts; the ACGME should specify in their core competency requirements that resident physicians work on quality improvement projects; likewise, the Joint Commission should require that resident physicians be included in quality improvement and patient safety programs at teaching hospitals; hospital administrators and residency program directors should create opportunities for resident physicians to become involved in ongoing quality improvement projects and root cause analysis teams; feedback on successful quality improvement interventions should be shared with resident physicians and broadly disseminatedQuality improvement/patient safety concepts should be integral to the medical school curriculum; medical school deans should elevate the topics of patient safety, quality improvement, and teamwork; these concepts should be integrated throughout the medical school curriculum and reinforced throughout residency; mastery of these concepts by medical students should be tested on the United States Medical Licensing Examination (USMLE) stepsFederal government should support involvement of resident physicians in quality improvement efforts; initiatives to improve quality by including resident physicians in quality improvement projects should be financially supported by the Department of Health and Human Services
Monitoring and oversight of the ACGME
While the ACGME is a key stakeholder in residency training, external voices are essential to ensure that public interests are heard in the development and monitoring of standards. Consequently, the Institute of Medicine report recommended external oversight and monitoring through the Joint Commission and Centers for Medicare and Medicaid Services (CMS). The recommendations are: Make comprehensive fatigue management a Joint Commission National Patient Safety Goal; fatigue is a safety concern not only for resident physicians, but also for nurses, attending physicians, and other health care workers; the Joint Commission should seek to ensure that all health care workers, not just resident physicians, are working as safely as possibleFederal government, including the Centers for Medicare and Medicaid Services and the Agency for Healthcare Research and Quality, should encourage development of comprehensive fatigue management programs which all health systems would eventually be required to implementMake ACGME compliance with working hours a “ condition of participation” for reimbursement of direct and indirect graduate medical education costs; financial incentives will greatly increase the adoption of and compliance with ACGME standards
Future financial support for implementation
The Institute of Medicine’s report estimates that $1.7 billion (in 2008 dollars) would be needed to implement its recommendations. Twenty-five percent of that amount ($376 million) will be required just to bring hospitals into compliance with the existing 2003 ACGME rules. Downstream savings to the health care system could potentially result from safer care, but these benefits typically do not accrue to hospitals and residency programs, who have been asked historically to bear the burden of residency reform costs. The recommendations are: The Institute of Medicine should convene a panel of stakeholders, including private and public funders of health care and graduate medical education, to lay down the concrete steps necessary to identify and allocate the resources needed to implement the recommendations contained in the IOM “Resident duty hours: Enhancing sleep, supervision and safety” report. Conference participants suggested several approaches to engage public and private support for this initiativeEfforts to find additional funding to implement the Institute of Medicine recommendations should focus more broadly on patient safety and health care delivery reform; policy efforts focused narrowly upon resident physician work hours are less likely to succeed than broad patient safety initiatives that include residency redesign as a key componentHospitals should view the Institute of Medicine recommendations as an opportunity to begin resident physician work redesign projects as the core of a business model that embraces safety and ultimately saves resourcesBoth the Secretary of Health and Human Services and the Director of the Centers for Medicare and Medicaid Services should take the Institute of Medicine recommendations into consideration when promulgating rules for innovation grantsThe National Health Care Workforce Commission should consider the Institute of Medicine recommendations when analyzing the nation’s physician workforce needs
Recommendations for future research
Conference participants concurred that convening the stakeholders and agreeing on a research agenda was key. Some observed that some sectors within the medical education community have been reluctant to act on the data. Several logical funders for future research were identified. But above all agencies, Centers for Medicare and Medicaid Services is the only stakeholder that funds graduate medical education upstream and will reap savings downstream if preventable medical errors are reduced as a result of reform of resident physician work hours.
doi:10.2147/NSS.S19649
PMCID: PMC3630963  PMID: 23616719
resident; hospital; working hours; safety
7.  Charting the Road to Competence: Developmental Milestones for Internal Medicine Residency Training 
Background
The Accreditation Council for Graduate Medical Education (ACGME) Outcome Project requires that residency program directors objectively document that their residents achieve competence in 6 general dimensions of practice.
Intervention
In November 2007, the American Board of Internal Medicine (ABIM) and the ACGME initiated the development of milestones for internal medicine residency training. ABIM and ACGME convened a 33-member milestones task force made up of program directors, experts in evaluation and quality, and representatives of internal medicine stakeholder organizations. This article reports on the development process and the resulting list of proposed milestones for each ACGME competency.
Outcomes
The task force adopted the Dreyfus model of skill acquisition as a framework the internal medicine milestones, and calibrated the milestones with the expectation that residents achieve, at a minimum, the “competency” level in the 5-step progression by the completion of residency. The task force also developed general recommendations for strategies to evaluate the milestones.
Discussion
The milestones resulting from this effort will promote competency-based resident education in internal medicine, and will allow program directors to track the progress of residents and inform decisions regarding promotion and readiness for independent practice. In addition, the milestones may guide curriculum development, suggest specific assessment strategies, provide benchmarks for resident self-directed assessment-seeking, and assist remediation by facilitating identification of specific deficits. Finally, by making explicit the profession's expectations for graduates and providing a degree of national standardization in evaluation, the milestones may improve public accountability for residency training.
doi:10.4300/01.01.0003
PMCID: PMC2931179  PMID: 21975701
8.  A Self-instructional Model to Teach Systems-based Practice and Practice-based Learning and Improvement 
Background
When mandated as resident competencies in 1999, systems-based practice (SBP) and practice-based learning and improvement (PBLI) were new concepts to many.
Objective
To describe and evaluate a 4-week clinical elective (Achieving Competence Today—ACT) to teach residents SBP and PBLI.
Design
ACT consisted of a four-week active learning course and follow-up teaching experience, guided and supported by web-based materials. The curriculum included readings, scheduled activities, work products including an improvement project, and weekly meetings with a non-expert preceptor. The evaluation used a before–after cross-comparison of ACT residents and their peers.
Participants
Seventy-eight residents and 42 faculty in 18 US Internal Medicine residency programs participated between 2003 and 2005.
Results and Main Measurements
All residents and faculty preceptors responded to a knowledge test, survey of attitudes, and self-assessment of competency to do 15 tasks related to SBP/PBLI. All measures were normalized to a 100-point scale. Each program’s principal investigator (PI) identified aspects of ACT that were most and least effective in enhancing resident learning. ACT residents’ gains in knowledge (4.4 on a 100-point scale) and self-assessed competency (11.3) were greater than controls’ (−1.9, −8.0), but changes in attitudes were not significantly different. Faculty preceptors’ knowledge scores did not change, but their attitudes became more positive (15.8). PIs found a ready-to-use curriculum effective (rated 8.5 on a 10-point scale).
Conclusions
ACT increased residents’ knowledge and self-assessment of their own competency and raised faculty’s assessment of the importance of residents’ learning SBP/PBLI. Faculty content expertise is not required for residents to learn SBP/PBLI.
doi:10.1007/s11606-008-0517-0
PMCID: PMC2517944  PMID: 18612719
residency training; systems-based practice; practice-based learning and improvement
9.  Pilot Study Evaluating a Practice-Based Learning and Improvement Curriculum Focusing on the Development of System-Level Quality Improvement Skills 
Background
We developed a practice-based learning and improvement (PBLI) curriculum to address important gaps in components of content and experiential learning activities through didactics and participation in systems-level quality improvement projects that focus on making changes in health care processes.
Methods
We evaluated the impact of our curriculum on resident PBLI knowledge, self-efficacy, and application skills. A quasi-experimental design assessed the impact of a curriculum (PBLI quality improvement systems compared with non-PBLI) on internal medicine residents' learning during a 4-week ambulatory block. We measured application skills, self-efficacy, and knowledge by using the Systems Quality Improvement Training and Assessment Tool. Exit evaluations assessed time invested and experiences related to the team projects and suggestions for improving the curriculum.
Results
The 2 groups showed differences in change scores. Relative to the comparison group, residents in the PBLI curriculum demonstrated a significant increase in the belief about their ability to implement a continuous quality improvement project (P  =  .020), comfort level in developing data collection plans (P  =  .010), and total knowledge scores (P < .001), after adjusting for prior PBLI experience. Participants in the PBLI curriculum also demonstrated significant improvement in providing a more complete aim statement for a proposed project after adjusting for prior PBLI experience (P  =  .001). Exit evaluations were completed by 96% of PBLI curriculum participants who reported high satisfaction with team performance.
Conclusion
Residents in our curriculum showed gains in areas fundamental for PBLI competency. The observed improvements were related to fundamental quality improvement knowledge, with limited gain in application skills. This suggests that while heading in the right direction, we need to conceptualize and structure PBLI training in a way that integrates it throughout the residency program and fosters the application of this knowledge and these skills.
doi:10.4300/JGME-D-10-00104.1
PMCID: PMC3186260  PMID: 22379523
10.  Are Commonly Used Resident Measurements Associated with Procedural Skills in Internal Medicine Residency Training? 
Background
Acquisition of competence in performing a variety of procedures is essential during Internal Medicine (IM) residency training.
Purposes
Determine the rate of procedural complications by IM residents; determine whether there was a correlation between having 1 or more complications and institutional procedural certification status or attending ratings of resident procedural skill competence on the American Board of Internal Medicine (ABIM) monthly evaluation form (ABIM-MEF). Assess if an association exists between procedural complications and in-training examination and ABIM board certification scores.
Methods
We retrospectively reviewed all procedure log sheets, procedural certification status, ABIM-MEF procedural skills ratings, in-training exam and certifying examination (ABIM-CE) scores from the period 1990–1999 for IM residency program graduates from a training program.
Results
Among 69 graduates, 2,212 monthly procedure log sheets and 2,475 ABIM-MEFs were reviewed. The overall complication rate was 2.3/1,000 procedures (95% CI: 1.4–3.1/1,000 procedure). With the exception of procedural certification status as judged by institutional faculty, there was no association between our resident measurements and procedural complications.
Conclusions
Our findings support the need for a resident procedural competence certification system based on direct observation. Our data support the ABIM’s action to remove resident procedural competence from the monthly ABIM-MEF ratings.
doi:10.1007/s11606-006-0068-1
PMCID: PMC1824756  PMID: 17356968
procedural skills; Internal Medicine residency training program; ABIM evaluation
11.  Are Commonly Used Resident Measurements Associated with Procedural Skills in Internal Medicine Residency Training? 
Background
Acquisition of competence in performing a variety of procedures is essential during Internal Medicine (IM) residency training.
Purposes
Determine the rate of procedural complications by IM residents; determine whether there was a correlation between having 1 or more complications and institutional procedural certification status or attending ratings of resident procedural skill competence on the American Board of Internal Medicine (ABIM) monthly evaluation form (ABIM-MEF). Assess if an association exists between procedural complications and in-training examination and ABIM board certification scores.
Methods
We retrospectively reviewed all procedure log sheets, procedural certification status, ABIM-MEF procedural skills ratings, in-training exam and certifying examination (ABIM-CE) scores from the period 1990–1999 for IM residency program graduates from a training program.
Results
Among 69 graduates, 2,212 monthly procedure log sheets and 2,475 ABIM-MEFs were reviewed. The overall complication rate was 2.3/1,000 procedures (95% CI: 1.4–3.1/1,000 procedure). With the exception of procedural certification status as judged by institutional faculty, there was no association between our resident measurements and procedural complications.
Conclusions
Our findings support the need for a resident procedural competence certification system based on direct observation. Our data support the ABIM’s action to remove resident procedural competence from the monthly ABIM-MEF ratings.
doi:10.1007/s11606-006-0068-1
PMCID: PMC1824756  PMID: 17356968
procedural skills; Internal Medicine residency training program; ABIM evaluation
12.  Using the American Board of Internal Medicine Practice Improvement Modules to Teach Internal Medicine Residents Practice Improvement 
Background
Although residency programs must prepare physicians who can analyze and improve their practice, practice improvement (PI) is new for many faculty preceptors. We describe the pilot of a PI curriculum incorporating a practice improvement module (PIM) from the American Board of Internal Medicine for residents and their faculty preceptors.
Methods
Residents attended PI didactics and completed a PIM during continuity clinic and outpatient months working in groups under committed faculty.
Results
All residents participated in PI group projects. Residents agreed or strongly agreed that the projects and the curriculum benefited their learning and patient care. A self-assessment revealed significant improvement in PI competencies, but residents were just reaching a “somewhat confident” level.
Conclusion
A PI curriculum incorporating PIMs is an effective way to teach PI to both residents and faculty preceptors. We recommend the team approach and use of the PIM tutorial approach especially for faculty.
doi:10.4300/JGME-D-09-00032.1
PMCID: PMC2931217  PMID: 21975892
13.  Associations between quality indicators of internal medicine residency training programs 
BMC Medical Education  2011;11:30.
Background
Several residency program characteristics have been suggested as measures of program quality, but associations between these measures are unknown. We set out to determine associations between these potential measures of program quality.
Methods
Survey of internal medicine residency programs that shared an online ambulatory curriculum on hospital type, faculty size, number of trainees, proportion of international medical graduate (IMG) trainees, Internal Medicine In-Training Examination (IM-ITE) scores, three-year American Board of Internal Medicine Certifying Examination (ABIM-CE) first-try pass rates, Residency Review Committee-Internal Medicine (RRC-IM) certification length, program director clinical duties, and use of pharmaceutical funding to support education. Associations assessed using Chi-square, Spearman rank correlation, univariate and multivariable linear regression.
Results
Fifty one of 67 programs responded (response rate 76.1%), including 29 (56.9%) community teaching and 17 (33.3%) university hospitals, with a mean of 68 trainees and 101 faculty. Forty four percent of trainees were IMGs. The average post-graduate year (PGY)-2 IM-ITE raw score was 63.1, which was 66.8 for PGY3s. Average 3-year ABIM-CE pass rate was 95.8%; average RRC-IM certification was 4.3 years. ABIM-CE results, IM-ITE results, and length of RRC-IM certification were strongly associated with each other (p < 0.05). PGY3 IM-ITE scores were higher in programs with more IMGs and in programs that accepted pharmaceutical support (p < 0.05). RRC-IM certification was shorter in programs with higher numbers of IMGs. In multivariable analysis, a higher proportion of IMGs was associated with 1.17 years shorter RRC accreditation.
Conclusions
Associations between quality indicators are complex, but suggest that the presence of IMGs is associated with better performance on standardized tests but decreased duration of RRC-IM certification.
doi:10.1186/1472-6920-11-30
PMCID: PMC3126786  PMID: 21651768
program quality; Residency Review Committee; American Board of Internal Medicine Certifying Examination
14.  Description of a Developmental Criterion-Referenced Assessment for Promoting Competence in Internal Medicine Residents 
Rationale
End-of- rotation global evaluations can be subjective, produce inflated grades, lack interrater reliability, and offer information that lacks value. This article outlines the generation of a unique developmental criterion-referenced assessment that applies adult learning theory and the learner, manager, teacher model, and represents an innovative application to the American Board of Internal Medicine (ABIM) 9-point scale.
Intervention
We describe the process used by Southern Illinois University School of Medicine to develop rotation-specific, criterion-based evaluation anchors that evolved into an effective faculty development exercise.
Results
The intervention gave faculty a clearer understanding of the 6 Accreditation Council for Graduate Medical Education competencies, each rotation's educational goals, and how rotation design affects meaningful work-based assessment. We also describe easily attainable successes in evaluation design and pitfalls that other institutions may be able to avoid. Shifting the evaluation emphasis on the residents' development of competence has made the expectations of rotation faculty more transparent, has facilitated conversations between program director and residents, and has improved the specificity of the tool for feedback. Our findings showed the new approach reduced grade inflation compared with the ABIM end-of-rotation global evaluation form.
Discussion
We offer the new developmental criterion-referenced assessment as a unique application of the competences to the ABIM 9-point scale as a transferable model for improving the validity and reliability of resident evaluations across graduate medical education programs.
doi:10.4300/01.01.0012
PMCID: PMC2931180  PMID: 21975710
15.  Protocol-directed care in the ICU: making a future generation of intensivists less knowledgeable? 
Critical Care  2012;16(2):307.
Expanded abstract
Citation
Prasad M, Holmboe ES, Lipner RS, Hess BJ, Christie JD, Bellamy SL, Rubenfeld GD, Kahn JM. Clinical Protocols and Trainee Knowledge About Mechanical Ventilation. JAMA. 2011; 306(9):935-941. PubMed PMID: 21900133 This is available on http://www.pubmed.gov
Background
Clinical protocols are associated with improved patient outcomes; however, they may negatively affect medical education by removing trainees from clinical decision making.
Methods
Objective: To study the relationship between critical care training with mechanical ventilation protocols and subsequent knowledge about ventilator management.
Design: A retrospective cohort equivalence study linking a national survey of mechanical ventilation protocol availability with knowledge about mechanical ventilation. Exposure to protocols was defined as high intensity if an intensive care unit had 2 or more protocols for at least 3 years and as low intensity if 0 or 1 protocol.
Setting: Accredited US pulmonary and critical care fellowship programs.
Subjects: First-time examinees of the American Board of Internal Medicine (ABIM) Critical Care Medicine Certification Examination in 2008 and 2009.
Intervention: N/A
Outcomes: Knowledge, measured by performance on examination questions specific to mechanical ventilation management, calculated as a mechanical ventilation score using item response theory. The score is standardized to a mean (SD) of 500 (100), and a clinically important difference is defined as 25. Variables included in adjusted analyses were birth country, residency training country, and overall first-attempt score on the ABIM Internal Medicine Certification Examination.
Results
The 90 of 129 programs (70%) responded to the survey. Seventy seven programs (86%) had protocols for ventilation liberation, 66 (73%) for sedation management, and 54 (60%) for lung-protective ventilation at the time of the survey. Eighty eight (98%) of these programs had trainees who completed the ABIM Critical Care Medicine Certification Examination, totaling 553 examinees. Of these 88 programs, 27 (31%) had 0 protocols, 19 (22%) had 1 protocol, 24 (27%) had 2 protocols, and 18 (20%) had 3 protocols for at least 3 years. 42 programs (48%) were classified as high intensity and 46 (52%) as low intensity, with 304 trainees (55%) and 249 trainees (45%), respectively. In bi-variable analysis, no difference in mean scores was observed in high-intensity (497; 95% CI, 486-507) vs low-intensity programs (497; 95% CI, 485-509). Mean difference was 0 (95% CI, -16 to 16), with a positive value indicating a higher score in the high-intensity group. In multivariable analyses, no association of training was observed in a high-intensity program with mechanical ventilation score (adjusted mean difference, -5.36; 95% CI, -20.7 to 10.0).
Conclusions
Among first-time ABIM Critical Care Medicine Certification Examination examinees, training in a high-intensity ventilator protocol environment compared with a low-intensity environment was not associated with worse performance on examination questions about mechanical ventilation management.
doi:10.1186/cc11257
PMCID: PMC3681378  PMID: 22494787
16.  An Alternative Practice Model: Residents Transform Continuity Clinic and Become Systems Thinkers 
Background
A changing health care environment has created a need for physicians trained in health system improvement. Residency programs have struggled to teach and assess practice-based learning and improvement and systems-based practice competencies, particularly within ambulatory settings.
Intervention
We describe a resident-created and resident-led quality and practice-improvement council in an internal medicine continuity clinic. We conducted focus groups and report on residents' perspectives on council membership, practice management experiences, quality improvement projects, and resident satisfaction.
Method
Focus groups were held from May 2009 to March 2010 with internal medicine residents (N  =  5/focus group) who participated in the Continuity Clinic Ownership in Resident Education (CCORE) council. Data were analyzed with a grounded theory approach.
Results
During the focus groups, residents responded to the question: “Do you have any new insights into delivering quality patient care in an outpatient clinic as a result of this experience (CCORE membership)?” The qualitative analysis resulted in 6 themes: systems thinking and systems-based care skills; improving quality of patient care; improved clinic efficiency; ownership of patients; need for improved communication of practice changes; and a springboard for research.
Conclusions
CCORE residents participated in system changes and acquired leadership skills while working on practice-based and system problems in a clinic microsystem. We believe this model can be implemented by other residency programs to promote the development of systems thinking in residents, increase their ownership of continuity clinic, and empower them to implement system changes.
doi:10.4300/JGME-D-11-00133.1
PMCID: PMC3399618  PMID: 23730447
17.  Maintenance of certification in Internal Medicine: participation rates and patient outcomes 
The clinical practice of internal medicine continues to evolve with the addition of new information and new technology. Most internists in practice will have erosion of their knowledge after they complete training unless life-long learning occurs. The American Board of Internal Medicine (ABIM) began to issue time-limited certification in 1990 and asserts that the Maintenance of Certification (MOC) program promotes the professional development of internists. However, the available medical literature does not provide strong support for the assumption that internists with certification or recertification have better patient outcomes. This relationship between recertification and patient outcomes needs more study. In addition, the participation in the Maintenance of Certification program by internists with lifetime certifications has been low, and recertification by leaders in internal medicine has also been relatively low. Some physicians in practice have concerns about the relevance of the program and the cost. Our review suggests that the ABIM needs to review its current Maintenance of Certification program and make changes to enhance its clinical relevance and educational value. We suggest that professional development should be based on focused reviews of the current literature, which is immediately relevant to clinical practice, and that recertification could be based on completion of modules and more frequent, less onerous testing.
doi:10.3402/jchimp.v2i4.19753
PMCID: PMC3715151  PMID: 23882382
certification; recertification; internal medicine; patient outcomes; mortality
18.  Developing a Practice-Based Learning and Improvement Curriculum for an Academic General Surgery Residency 
Background
Program Directors in Surgery are now facing the challenge of incorporating the ACGME's practice-based learning and improvement (PBLI) competency into residency curriculum. We introduced a comprehensive PBLI experience for PG2 residents designed to integrate specific competency goals (quality improvement, clinical thinking, and self-directed learning) within the context of residents’ clinical practice.
Study Design
Fourteen PG2 residents participated in a three-week PBLI curriculum consisting of three components: Complex Clinical Decision Making (CCDM), Individual Learning Plan, and Quality Improvement (QI). To assess how effectively the curriculum addressed these three competencies, residents rated their understanding of PBLI by answering a 12-question written survey given pre- and post-rotation. Resident satisfaction was assessed through standard post-rotation evaluations.
Results
Analysis of the pre and post rotation surveys from the fourteen participants showed an increase in all measured elements, including knowledge of PBLI (p<0.001), ability to assess learning needs (p<0.001) and set learning goals (p<0.001), understanding of QI concepts (p=0.001), and experience with QI projects (p<0.001). Fourteen QI projects were developed. Although many residents found the creation of measurable learning goals to be challenging, the process of identifying strengths and weaknesses enhanced the resident's self-understanding, and contributed to overall satisfaction with the rotation.
Conclusions
The initial implementation of our PBLI curriculum demonstrated that residents report personal progress in their clinical decision making, self-directed learning, and familiarity with quality improvement. This comprehensive PBLI curriculum was accepted by surgical residents as a valuable part of their training. We are encouraged to continue a clinically-grounded PBLI experience for PG2 residents.
doi:10.1016/j.jamcollsurg.2010.01.017
PMCID: PMC2887484  PMID: 20347732
19.  Mastery Learning of Advanced Cardiac Life Support Skills by Internal Medicine Residents Using Simulation Technology and Deliberate Practice 
BACKGROUND
Internal medicine residents must be competent in advanced cardiac life support (ACLS) for board certification.
OBJECTIVE
To use a medical simulator to assess postgraduate year 2 (PGY-2) residents' baseline proficiency in ACLS scenarios and evaluate the impact of an educational intervention grounded in deliberate practice on skill development to mastery standards.
DESIGN
Pretest-posttest design without control group. After baseline evaluation, residents received 4, 2-hour ACLS education sessions using a medical simulator. Residents were then retested. Residents who did not achieve a research-derived minimum passing score (MPS) on each ACLS problem had more deliberate practice and were retested until the MPS was reached.
PARTICIPANTS
Forty-one PGY-2 internal medicine residents in a university-affiliated program.
MEASUREMENTS
Observational checklists based on American Heart Association (AHA) guidelines with interrater and internal consistency reliability estimates; deliberate practice time needed for residents to achieve minimum competency standards; demographics; United States Medical Licensing Examination Step 1 and Step 2 scores; and resident ratings of program quality and utility.
RESULTS
Performance improved significantly after simulator training. All residents met or exceeded the mastery competency standard. The amount of practice time needed to reach the MPS was a powerful (negative) predictor of posttest performance. The education program was rated highly.
CONCLUSIONS
A curriculum featuring deliberate practice dramatically increased the skills of residents in ACLS scenarios. Residents needed different amounts of training time to achieve minimum competency standards. Residents enjoy training, evaluation, and feedback in a simulated clinical environment. This mastery learning program and other competency-based efforts illustrate outcome-based medical education that is now prominent in accreditation reform of residency education.
doi:10.1111/j.1525-1497.2006.00341.x
PMCID: PMC1828088  PMID: 16637824
mastery learning; medical simulation; residency education
20.  A Multi-Institutional Quality Improvement Initiative to Transform Education for Chronic Illness Care in Resident Continuity Practices 
Journal of General Internal Medicine  2010;25(Suppl 4):574-580.
BACKGROUND
There is a gap between the need for patient-centered, evidence-based primary care for the large burden of chronic illness in the US, and the training of resident physicians to provide that care.
OBJECTIVE
To improve training for residents who provide chronic illness care in teaching practice settings.
DESIGN
US teaching hospitals were invited to participate in one of two 18-month Breakthrough Series Collaboratives—either a national Collaborative, or a subsequent California Collaborative—to implement the Chronic Care Model (CCM) and related curriculum changes in resident practices. Most practices focused on patients with diabetes mellitus. Educational redesign strategies with related performance measures were developed for curricular innovations anchored in the CCM. In addition, three clinical measures—HbA1c <7%, LDL <100 mg/dL, and blood pressure ≤130/80—and three process measures—retinal and foot examinations, and patient self-management goals—were tracked.
PARTICIPANTS
Fifty-seven teams from 37 self-selected teaching hospitals committed to implement the CCM in resident continuity practices; 41 teams focusing on diabetes improvement participated over the entire duration of one of the Collaboratives.
INTERVENTIONS
Teaching-practice teams—faculty, residents and staff—participated in Collaboratives by attending monthly calls and regular 2-day face-to-face meetings with the other teams. The national Collaborative faculty led calls and meetings. Each team used rapid cycle quality improvement (PDSA cycles) to implement the CCM and curricular changes. Teams reported education and clinical performance measures monthly.
RESULTS
Practices underwent extensive redesign to establish CCM elements. Education measures tracked substantial development of CCM-related learning. The clinical and process measures improved, however inconsistently, during the Collaboratives.
CONCLUSIONS
These initiatives suggest that systematic practice redesign for implementing the CCM along with linked educational approaches are achievable in resident continuity practices. Improvement of clinical outcomes in such practices is daunting but achievable.
doi:10.1007/s11606-010-1392-z
PMCID: PMC2940442  PMID: 20737232
residency training; chronic illness; teaching hospitals; Chronic Care Model
21.  An initiative to improve adherence to evidence-based guidelines in the treatment of URIs, sinusitis, and pharyngitis 
Background
Upper respiratory infections, acute sinus infections, and sore throats are common symptoms that cause patients to seek medical care. Despite well-established treatment guidelines, studies indicate that antibiotics are prescribed far more frequently than appropriate, raising a multitude of clinical issues.
Methods
The primary goal of this study was to increase guideline adherence rates for acute sinusitis, pharyngitis, and upper respiratory tract infections (URIs). This study was the first Plan-Do-Study-Act (PDSA) cycle in a quality improvement program at an internal medicine resident faculty practice at a university-affiliated community hospital internal medicine residency program. To improve guideline adherence for respiratory infections, a package of small-scale interventions was implemented aimed at improving patient and provider education regarding viral and bacterial infections and the necessity for antibiotics. The data from this study was compared with a previously published study in this practice, which evaluated the adherence rates for the treatment guidelines before the changes, to determine effectiveness of the modifications. After the first PDSA cycle, providers were surveyed to determine barriers to adherence to antibiotic prescribing guidelines.
Results
After the interventions, antibiotic guideline adherence for URI improved from a rate of 79.28 to 88.58% with a p-value of 0.004. The increase of adherence rates for sinusitis and pharyngitis were 41.7–57.58% (p=0.086) and 24.0–25.0% (p=0.918), respectively. The overall change in guideline adherence for the three conditions increased from 57.2 to 78.6% with the implementations (p<0.001). In planning for future PDSA cycles, a fishbone diagram was constructed in order to identify all perceived facets of the problem of non-adherence to the treatment guidelines for URIs, sinusitis, and pharyngitis. From the fishbone diagram and the provider survey, several potential directions for future work are discussed.
Conclusions
Passive interventions can result in small changes in antibiotic guideline adherence, but further PDSA cycles using more active methodologies are needed.
doi:10.3402/jchimp.v4.22958
PMCID: PMC3937558  PMID: 24596644
antibiotic; guideline; adherence
22.  Lessons Learned From a 5-Year Experience With a 4-Week Experiential Quality Improvement Curriculum in a Preventive Medicine Fellowship 
Background
Competency in practice-based learning and improvement (PBLI) and systems-based practice (SBP) empowers learners with the skills to plan, lead, and execute health care systems improvement efforts. Experiences from several graduate medical education programs describe the implementation of PBLI and SBP curricula as challenging because of lack of adequate curricular time and faculty resources, as well as a perception that PBLI and SBP are not relevant to future careers. A dedicated experiential rotation that requires fellow participation in a specialty-specific quality improvement project (QIP) may address some of these challenges.
Method
We describe a retrospective analysis of our 5-year experience with a dedicated 3-week PBLI-SBP experiential curriculum in a preventive medicine fellowship program at Mayo Clinic, Rochester, Minnesota.
Results
Between 2004 and 2008, 19 learners including 7 preventive medicine fellows participated in the rotation. Using just-in-time learning, fellows work together on a relatively complex QIP of community or institutional significance. Since 2004, all 19 learners (100%) participating in this rotation have consistently demonstrated statistically significant increase in their quality improvement knowledge application tool (QIKAT) scores at the end of the rotation. At the end of the rotation, all 19 learners stated that they were either confident or very confident of making a change to improve health care in a local setting. Most of the QIPs resulted in sustainable practice improvements, and resultant solutions have been disseminated beyond the location of the original QIP.
Conclusion
A dedicated experiential rotation that requires learner participation in a QIP is one of the effective methods to address the needs of the SBP and PBLI competencies.
doi:10.4300/01.01.0015
PMCID: PMC2931202  PMID: 21975713
23.  Clinical instructors' perception of a faculty development programme promoting postgraduate year-1 (PGY1) residents' ACGME six core competencies: a 2-year study 
BMJ Open  2011;1(2):e000200.
Objective
The six core competencies designated by Accreditation Council for Graduate Medical Education (ACGME) are essential for establishing a patient centre holistic medical system. The authors developed a faculty programme to promote the postgraduate year 1 (PGY1) resident, ACGME six core competencies. The study aims to assess the clinical instructors' perception, attitudes and subjective impression towards the various sessions of the ‘faculty development programme for teaching ACGME competencies.’
Methods
During 2009 and 2010, 134 clinical instructors participated in the programme to establish their ability to teach and assess PGY1 residents about ACGME competencies.
Results
The participants in the faculty development programme reported that the skills most often used while teaching were learnt during circuit and itinerant bedside, physical examination teaching, mini-clinical evaluation exercise (mini-CEX) evaluation demonstration, training workshop and videotapes of ‘how to teach ACGME competencies.’ Participants reported that circuit bedside teaching and mini-CEX evaluation demonstrations helped them in the interpersonal and communication skills domain, and that the itinerant teaching demonstrations helped them in the professionalism domain, while physical examination teaching and mini-CEX evaluation demonstrations helped them in the patients' care domain. Both the training workshop and videotape session increase familiarity with teaching and assessing skills. Participants who applied the skills learnt from the faculty development programme the most in their teaching and assessment came from internal medicine departments, were young attending physician and had experience as PGY1 clinical instructors.
Conclusions
According to the clinical instructors' response, our faculty development programme effectively increased their familiarity with various teaching and assessment skills needed to teach PGY1 residents and ACGME competencies, and these clinical instructors also then subsequently apply these skills.
Article summary
Article focus
In order to train PGY1 residents, we need to help clinical instructors to become familiar with the teaching and assessment skills that form the Accreditation Council for Graduate Medical Education six core-competencies.
Our study used a self-reported questionnaires based analysis to evaluate the clinical instructors' perception to our faculty development programme.
Key messages
Participants reported that their most commonly used skills were learnt from itinerant and circuit bedside teaching, and mini-clinical evaluation exercise evaluation demonstration in our programme.
Participants also reported that the 40 h basic training course improved their abilities to train and assess PGY1 residents in patient care, interpersonal and communication skills, and medical knowledge domains whereas postcourse training workshop and videotape session enhanced their ability in system-based practice, practice-based learning and improvement, and professionalism domains.
A serial follow-up questionnaire suggested that the degree of participant application of skills learnt from our programme increased progressively after finishing the 40 h basic training course, the postcourse training workshop and videotape session.
Strengths and limitations of this study
According to the clinical instructors' responses, our programme effectively increased their familiarity with teaching and assessment skills needed when teaching PGY1 residents' Accreditation Council for Graduate Medical Education competencies and that these skills were subsequently applies.
This study was limited by the fact that questionnaire used to track and assess the effectiveness of the training programme may have had information and recall bias. In addition, this study had a relatively small sample size and did not contain a control group. However, no controlled educational trials on this subject have been published as yet.
doi:10.1136/bmjopen-2011-000200
PMCID: PMC3225591  PMID: 22116089
24.  Description and evaluation of an EBM curriculum using a block rotation 
Background
While previous authors have emphasized the importance of integrating and reinforcing evidence-based medicine (EBM) skills in residency, there are few published examples of such curricula. We designed an EBM curriculum to train family practice interns in essential EBM skills for information mastery using clinical questions generated by the family practice inpatient service. We sought to evaluate the impact of this curriculum on interns, residents, and faculty.
Methods
Interns (n = 13) were asked to self-assess their level of confidence in basic EBM skills before and after their 2-week EBM rotation. Residents (n = 21) and faculty (n = 12) were asked to assess how often the answers provided by the EBM intern to the inpatient service changed medical care. In addition, residents were asked to report how often they used their EBM skills and how often EBM concepts and tools were used in teaching by senior residents and faculty. Faculty were asked if the EBM curriculum had increased their use of EBM in practice and in teaching.
Results
Interns significantly increased their confidence over the course of the rotation. Residents and faculty felt that the answers provided by the EBM intern provided useful information and led to changes in patient care. Faculty reported incorporating EBM into their teaching (92%) and practice (75%). Residents reported applying the EBM skills they learned to patient care (86%) and that these skills were reinforced in the teaching they received outside of the rotation (81%). All residents and 11 of 12 faculty felt that the EBM curriculum had improved patient care.
Conclusions
To our knowledge, this is the first published EBM curriculum using an individual block rotation format. As such, it may provide an alternative model for teaching and incorporating EBM into a residency program.
doi:10.1186/1472-6920-4-19
PMCID: PMC524496  PMID: 15476556
25.  Setting a Fair Performance Standard for Physicians’ Quality of Patient Care 
Background
Assessing physicians’ clinical performance using statistically sound, evidence-based measures is challenging. Little research has focused on methodological approaches to setting performance standards to which physicians are being held accountable.
Objective
Determine if a rigorous approach for setting an objective, credible standard of minimally-acceptable performance could be used for practicing physicians caring for diabetic patients.
Design
Retrospective cohort study.
Participants
Nine hundred and fifty-seven physicians from the United States with time-limited certification in internal medicine or a subspecialty.
Main Measures
The ABIM Diabetes Practice Improvement Module was used to collect data on ten clinical and two patient experience measures. A panel of eight internists/subspecialists representing essential perspectives of clinical practice applied an adaptation of the Angoff method to judge how physicians who provide minimally-acceptable care would perform on individual measures to establish performance thresholds. Panelists then rated each measure’s relative importance and the Dunn–Rankin method was applied to establish scoring weights for the composite measure. Physician characteristics were used to support the standard-setting outcome.
Key Results
Physicians abstracted 20,131 patient charts and 18,974 patient surveys were completed. The panel established reasonable performance thresholds and importance weights, yielding a standard of 48.51 (out of 100 possible points) on the composite measure with high classification accuracy (0.98). The 38 (4%) outlier physicians who did not meet the standard had lower ratings of overall clinical competence and professional behavior/attitude from former residency program directors (p = 0.01 and p = 0.006, respectively), lower Internal Medicine certification and maintenance of certification examination scores (p = 0.005 and p < 0.001, respectively), and primarily worked as solo practitioners (p = 0.02).
Conclusions
The standard-setting method yielded a credible, defensible performance standard for diabetes care based on informed judgment that resulted in a reasonable, reproducible outcome. Our method represents one approach to identifying outlier physicians for intervention to protect patients.
Electronic supplementary material
The online version of this article (doi:10.1007/s11606-010-1572-x) contains supplementary material, which is available to authorized users.
doi:10.1007/s11606-010-1572-x
PMCID: PMC3077491  PMID: 21104453
clinical performance assessment; standard setting; composite measures; diabetes care

Results 1-25 (964148)