PMCC PMCC

Search tips
Search criteria

Advanced
Results 1-25 (921136)

Clipboard (0)
None

Related Articles

1.  Impact of subspecialty elective exposures on outcomes on the American board of internal medicine certification examination 
BMC Medical Education  2012;12:94.
Background
The American Board of Internal Medicine Certification Examination (ABIM-CE) is one of several methods used to assess medical knowledge, an Accreditation Council for Graduate Medical Education (ACGME) core competency for graduating internal medicine residents. With recent changes in graduate medical education program directors and internal medicine residents are seeking evidence to guide decisions regarding residency elective choices. Prior studies have shown that formalized elective curricula improve subspecialty ABIM-CE scores. The primary aim of this study was to evaluate whether the number of subspecialty elective exposures or the specific subspecialties which residents complete electives in impact ABIM-CE scores.
Methods
ABIM-CE scores, elective exposures and demographic characteristics were collected for MedStar Georgetown University Hospital internal medicine residents who were first-time takers of the ABIM-CE in 2006–2010 (n=152). Elective exposures were defined as a two-week period assigned to the respective subspecialty. ABIM-CE score was analyzed using the difference between the ABIM-CE score and the standardized passing score (delta-SPS). Subspecialty scores were analyzed using percentage of correct responses. Data was analyzed using GraphPad Prism version 5.00 for Windows.
Results
Paired elective exposure and ABIM-CE scores were available in 131 residents. There was no linear correlation between ABIM-CE mean delta-SPS and the total number of electives or the number of unique elective exposures. Residents with ≤14 elective exposures had higher ABIM-CE mean delta-SPS than those with ≥15 elective exposures (143.4 compared to 129.7, p=0.051). Repeated electives in individual subspecialties were not associated with significant difference in mean ABIM-CE delta-SPS.
Conclusions
This study did not demonstrate significant positive associations between individual subspecialty elective exposures and ABIM-CE mean delta-SPS score. Residents with ≤14 elective exposures had higher ABIM-CE mean delta-SPS than those with ≥15 elective exposures suggesting there may be an “ideal” number of elective exposures that supports improved ABIM-CE performance. Repeated elective exposures in an individual specialty did not correlate with overall or subspecialty ABIM-CE performance.
doi:10.1186/1472-6920-12-94
PMCID: PMC3480921  PMID: 23057635
Resident education; Gender; Elective; Subspecialty; Graduate medical education
2.  Predicting Pass Rates on the American Board of Internal Medicine Certifying Examination 
Our objective was to determine the ability of the internal medicine In-Training Examination (ITE) to predict pass or fail outcomes on the American Board of Internal Medicine (ABIM) certifying examination and to develop an externally validated predictive model and a simple equation that can be used by residency directors to provide probability feedback for their residency programs. We collected a study sample of 155 internal medicine residents from the three Virginia internal medicine programs and a validation sample of 64 internal medicine residents from a residency program outside Virginia. Scores from both samples were collected across three class cohorts. The Kolmogorov-Smirnov z test indicated no statistically significant difference between the distribution of scores for the two samples (z = 1.284, p = .074). Results of the logistic model yielded a statistically significant prediction of ABIM pass or fail performance from ITE scores (Wald = 35.49, SE = 0.036, df = 1, p < .005) and overall correct classifications for the study sample and validation sample at 79% and 75%, respectively. The ITE is a useful tool in assessing the likelihood of a resident's passing or failing the ABIM certifying examination but is less predictive for residents who received ITE scores between 49 and 66.
doi:10.1046/j.1525-1497.1998.00122.x
PMCID: PMC1496976  PMID: 9669571
certifying examination; in-training examination; education; predictions; residents
3.  Are Commonly Used Resident Measurements Associated with Procedural Skills in Internal Medicine Residency Training? 
Background
Acquisition of competence in performing a variety of procedures is essential during Internal Medicine (IM) residency training.
Purposes
Determine the rate of procedural complications by IM residents; determine whether there was a correlation between having 1 or more complications and institutional procedural certification status or attending ratings of resident procedural skill competence on the American Board of Internal Medicine (ABIM) monthly evaluation form (ABIM-MEF). Assess if an association exists between procedural complications and in-training examination and ABIM board certification scores.
Methods
We retrospectively reviewed all procedure log sheets, procedural certification status, ABIM-MEF procedural skills ratings, in-training exam and certifying examination (ABIM-CE) scores from the period 1990–1999 for IM residency program graduates from a training program.
Results
Among 69 graduates, 2,212 monthly procedure log sheets and 2,475 ABIM-MEFs were reviewed. The overall complication rate was 2.3/1,000 procedures (95% CI: 1.4–3.1/1,000 procedure). With the exception of procedural certification status as judged by institutional faculty, there was no association between our resident measurements and procedural complications.
Conclusions
Our findings support the need for a resident procedural competence certification system based on direct observation. Our data support the ABIM’s action to remove resident procedural competence from the monthly ABIM-MEF ratings.
doi:10.1007/s11606-006-0068-1
PMCID: PMC1824756  PMID: 17356968
procedural skills; Internal Medicine residency training program; ABIM evaluation
4.  Are Commonly Used Resident Measurements Associated with Procedural Skills in Internal Medicine Residency Training? 
Background
Acquisition of competence in performing a variety of procedures is essential during Internal Medicine (IM) residency training.
Purposes
Determine the rate of procedural complications by IM residents; determine whether there was a correlation between having 1 or more complications and institutional procedural certification status or attending ratings of resident procedural skill competence on the American Board of Internal Medicine (ABIM) monthly evaluation form (ABIM-MEF). Assess if an association exists between procedural complications and in-training examination and ABIM board certification scores.
Methods
We retrospectively reviewed all procedure log sheets, procedural certification status, ABIM-MEF procedural skills ratings, in-training exam and certifying examination (ABIM-CE) scores from the period 1990–1999 for IM residency program graduates from a training program.
Results
Among 69 graduates, 2,212 monthly procedure log sheets and 2,475 ABIM-MEFs were reviewed. The overall complication rate was 2.3/1,000 procedures (95% CI: 1.4–3.1/1,000 procedure). With the exception of procedural certification status as judged by institutional faculty, there was no association between our resident measurements and procedural complications.
Conclusions
Our findings support the need for a resident procedural competence certification system based on direct observation. Our data support the ABIM’s action to remove resident procedural competence from the monthly ABIM-MEF ratings.
doi:10.1007/s11606-006-0068-1
PMCID: PMC1824756  PMID: 17356968
procedural skills; Internal Medicine residency training program; ABIM evaluation
5.  Associations between quality indicators of internal medicine residency training programs 
BMC Medical Education  2011;11:30.
Background
Several residency program characteristics have been suggested as measures of program quality, but associations between these measures are unknown. We set out to determine associations between these potential measures of program quality.
Methods
Survey of internal medicine residency programs that shared an online ambulatory curriculum on hospital type, faculty size, number of trainees, proportion of international medical graduate (IMG) trainees, Internal Medicine In-Training Examination (IM-ITE) scores, three-year American Board of Internal Medicine Certifying Examination (ABIM-CE) first-try pass rates, Residency Review Committee-Internal Medicine (RRC-IM) certification length, program director clinical duties, and use of pharmaceutical funding to support education. Associations assessed using Chi-square, Spearman rank correlation, univariate and multivariable linear regression.
Results
Fifty one of 67 programs responded (response rate 76.1%), including 29 (56.9%) community teaching and 17 (33.3%) university hospitals, with a mean of 68 trainees and 101 faculty. Forty four percent of trainees were IMGs. The average post-graduate year (PGY)-2 IM-ITE raw score was 63.1, which was 66.8 for PGY3s. Average 3-year ABIM-CE pass rate was 95.8%; average RRC-IM certification was 4.3 years. ABIM-CE results, IM-ITE results, and length of RRC-IM certification were strongly associated with each other (p < 0.05). PGY3 IM-ITE scores were higher in programs with more IMGs and in programs that accepted pharmaceutical support (p < 0.05). RRC-IM certification was shorter in programs with higher numbers of IMGs. In multivariable analysis, a higher proportion of IMGs was associated with 1.17 years shorter RRC accreditation.
Conclusions
Associations between quality indicators are complex, but suggest that the presence of IMGs is associated with better performance on standardized tests but decreased duration of RRC-IM certification.
doi:10.1186/1472-6920-11-30
PMCID: PMC3126786  PMID: 21651768
program quality; Residency Review Committee; American Board of Internal Medicine Certifying Examination
6.  Protocol-directed care in the ICU: making a future generation of intensivists less knowledgeable? 
Critical Care  2012;16(2):307.
Expanded abstract
Citation
Prasad M, Holmboe ES, Lipner RS, Hess BJ, Christie JD, Bellamy SL, Rubenfeld GD, Kahn JM. Clinical Protocols and Trainee Knowledge About Mechanical Ventilation. JAMA. 2011; 306(9):935-941. PubMed PMID: 21900133 This is available on http://www.pubmed.gov
Background
Clinical protocols are associated with improved patient outcomes; however, they may negatively affect medical education by removing trainees from clinical decision making.
Methods
Objective: To study the relationship between critical care training with mechanical ventilation protocols and subsequent knowledge about ventilator management.
Design: A retrospective cohort equivalence study linking a national survey of mechanical ventilation protocol availability with knowledge about mechanical ventilation. Exposure to protocols was defined as high intensity if an intensive care unit had 2 or more protocols for at least 3 years and as low intensity if 0 or 1 protocol.
Setting: Accredited US pulmonary and critical care fellowship programs.
Subjects: First-time examinees of the American Board of Internal Medicine (ABIM) Critical Care Medicine Certification Examination in 2008 and 2009.
Intervention: N/A
Outcomes: Knowledge, measured by performance on examination questions specific to mechanical ventilation management, calculated as a mechanical ventilation score using item response theory. The score is standardized to a mean (SD) of 500 (100), and a clinically important difference is defined as 25. Variables included in adjusted analyses were birth country, residency training country, and overall first-attempt score on the ABIM Internal Medicine Certification Examination.
Results
The 90 of 129 programs (70%) responded to the survey. Seventy seven programs (86%) had protocols for ventilation liberation, 66 (73%) for sedation management, and 54 (60%) for lung-protective ventilation at the time of the survey. Eighty eight (98%) of these programs had trainees who completed the ABIM Critical Care Medicine Certification Examination, totaling 553 examinees. Of these 88 programs, 27 (31%) had 0 protocols, 19 (22%) had 1 protocol, 24 (27%) had 2 protocols, and 18 (20%) had 3 protocols for at least 3 years. 42 programs (48%) were classified as high intensity and 46 (52%) as low intensity, with 304 trainees (55%) and 249 trainees (45%), respectively. In bi-variable analysis, no difference in mean scores was observed in high-intensity (497; 95% CI, 486-507) vs low-intensity programs (497; 95% CI, 485-509). Mean difference was 0 (95% CI, -16 to 16), with a positive value indicating a higher score in the high-intensity group. In multivariable analyses, no association of training was observed in a high-intensity program with mechanical ventilation score (adjusted mean difference, -5.36; 95% CI, -20.7 to 10.0).
Conclusions
Among first-time ABIM Critical Care Medicine Certification Examination examinees, training in a high-intensity ventilator protocol environment compared with a low-intensity environment was not associated with worse performance on examination questions about mechanical ventilation management.
doi:10.1186/cc11257
PMCID: PMC3681378  PMID: 22494787
7.  Description of a Developmental Criterion-Referenced Assessment for Promoting Competence in Internal Medicine Residents 
Rationale
End-of- rotation global evaluations can be subjective, produce inflated grades, lack interrater reliability, and offer information that lacks value. This article outlines the generation of a unique developmental criterion-referenced assessment that applies adult learning theory and the learner, manager, teacher model, and represents an innovative application to the American Board of Internal Medicine (ABIM) 9-point scale.
Intervention
We describe the process used by Southern Illinois University School of Medicine to develop rotation-specific, criterion-based evaluation anchors that evolved into an effective faculty development exercise.
Results
The intervention gave faculty a clearer understanding of the 6 Accreditation Council for Graduate Medical Education competencies, each rotation's educational goals, and how rotation design affects meaningful work-based assessment. We also describe easily attainable successes in evaluation design and pitfalls that other institutions may be able to avoid. Shifting the evaluation emphasis on the residents' development of competence has made the expectations of rotation faculty more transparent, has facilitated conversations between program director and residents, and has improved the specificity of the tool for feedback. Our findings showed the new approach reduced grade inflation compared with the ABIM end-of-rotation global evaluation form.
Discussion
We offer the new developmental criterion-referenced assessment as a unique application of the competences to the ABIM 9-point scale as a transferable model for improving the validity and reliability of resident evaluations across graduate medical education programs.
doi:10.4300/01.01.0012
PMCID: PMC2931180  PMID: 21975710
8.  Development of an Ambulatory Geriatrics Knowledge Examination for Internal Medicine Residents 
Background
The number of older adults needing primary care exceeds the capacity of trained geriatricians to accommodate them. All physicians should have basic knowledge of optimal outpatient care of older adults to enhance the capacity of the system to serve this patient group. To date, there is no knowledge-assessment tool that focuses specifically on geriatric ambulatory care.
Objective
We developed an examination to assess internal medicine residents' knowledge of ambulatory geriatrics.
Methods
A consensus panel developed a 30-question examination based on topics in the American Board of Internal Medicine (ABIM) Certification Examination Blueprint, the ABIM in-training examinations, and the American Geriatrics Society Goals and Objectives. Questions were reviewed, edited, and then administered to medical students, internal medicine residents, primary care providers, and geriatricians.
Results
Ninety-eight individuals (20 fourth-year medical students, 57 internal medicine residents, 11 primary care faculty members, and 10 geriatrics fellowship-trained physicians) took the examination. Based on psychometric analysis of the results, 5 questions were deleted because of poor discriminatory power. The Cronbach α coefficient of the remaining 25 questions was 0.48; however, assessment of interitem consistency may not be an appropriate measure, given the variety of clinical topics on which questions were based. Scores increased with higher levels of training in geriatrics (P < .001).
Conclusion
Our preliminary study suggests that the examination we developed is a reasonably valid method to assess knowledge of ambulatory geriatric care and may be useful in assessing residents.
doi:10.4300/JGME-D-13-00123.1
PMCID: PMC3886473  PMID: 24455023
9.  Burnout and Distress Among Internal Medicine Program Directors: Results of A National Survey 
Journal of General Internal Medicine  2013;28(8):1056-1063.
BACKGROUND
Physician burnout and distress has been described in national studies of practicing physicians, internal medicine (IM) residents, IM clerkship directors, and medical school deans. However, no comparable national data exist for IM residency program directors.
OBJECTIVE
To assess burnout and distress among IM residency program directors, and to evaluate relationships of distress with personal and program characteristics and perceptions regarding implementation and consequences of Accreditation Council for Graduate Medical Education (ACGME) regulations.
DESIGN AND PARTICIPANTS
The 2010 Association of Program Directors in Internal Medicine (APDIM) Annual Survey, developed by the APDIM Survey Committee, was sent in August 2010 to the 377 program directors with APDIM membership, representing 99.0 % of the 381 United States categorical IM residency programs.
MAIN MEASURES
The 2010 APDIM Annual Survey included validated items on well-being and distress, including questions addressing quality of life, satisfaction with work-life balance, and burnout. Questions addressing personal and program characteristics and perceptions regarding implementation and consequences of ACGME regulations were also included.
KEY RESULTS
Of 377 eligible program directors, 282 (74.8 %) completed surveys. Among respondents, 12.4 % and 28.8 % rated their quality of life and satisfaction with work-life balance negatively, respectively. Also, 27.0 % reported emotional exhaustion, 10.4 % reported depersonalization, and 28.7 % reported overall burnout. These rates were lower than those reported previously in national studies of medical students, IM residents, practicing physicians, IM clerkship directors, and medical school deans. Aspects of distress were more common among younger program directors, women, and those reporting greater weekly work hours. Work–home conflicts were common and associated with all domains of distress, especially if not resolved in a manner effectively balancing work and home responsibilities. Associations with program characteristics such as program size and American Board of Internal Medicine (ABIM) pass rates were not found apart from higher rates of depersonalization among directors of community-based programs (23.5 % vs. 8.6 %, p = 0.01). We did not observe any consistent associations between distress and perceptions of implementation and consequences of program regulations.
CONCLUSIONS
The well-being of IM program directors across domains, including quality of life, satisfaction with work-life balance, and burnout, appears generally superior to that of medical trainees, practicing physicians, and other medical educators nationally. Additionally, it is reassuring that program directors' perceptions of their ability to respond to current regulatory requirements are not adversely associated with distress. However, the increased distress levels among younger program directors, women, and those at community-based training programs reported in this study are important concerns worthy of further study.
doi:10.1007/s11606-013-2349-9
PMCID: PMC3710382  PMID: 23595924
graduate medical education; residency; burnout; well-being
10.  Charting the Road to Competence: Developmental Milestones for Internal Medicine Residency Training 
Background
The Accreditation Council for Graduate Medical Education (ACGME) Outcome Project requires that residency program directors objectively document that their residents achieve competence in 6 general dimensions of practice.
Intervention
In November 2007, the American Board of Internal Medicine (ABIM) and the ACGME initiated the development of milestones for internal medicine residency training. ABIM and ACGME convened a 33-member milestones task force made up of program directors, experts in evaluation and quality, and representatives of internal medicine stakeholder organizations. This article reports on the development process and the resulting list of proposed milestones for each ACGME competency.
Outcomes
The task force adopted the Dreyfus model of skill acquisition as a framework the internal medicine milestones, and calibrated the milestones with the expectation that residents achieve, at a minimum, the “competency” level in the 5-step progression by the completion of residency. The task force also developed general recommendations for strategies to evaluate the milestones.
Discussion
The milestones resulting from this effort will promote competency-based resident education in internal medicine, and will allow program directors to track the progress of residents and inform decisions regarding promotion and readiness for independent practice. In addition, the milestones may guide curriculum development, suggest specific assessment strategies, provide benchmarks for resident self-directed assessment-seeking, and assist remediation by facilitating identification of specific deficits. Finally, by making explicit the profession's expectations for graduates and providing a degree of national standardization in evaluation, the milestones may improve public accountability for residency training.
doi:10.4300/01.01.0003
PMCID: PMC2931179  PMID: 21975701
11.  A tool for self-assessment of communication skills and professionalism in residents 
Background
Effective communication skills and professionalism are critical for physicians in order to provide optimum care and achieve better health outcomes. The aims of this study were to evaluate residents' self-assessment of their communication skills and professionalism in dealing with patients, and to evaluate the psychometric properties of a self-assessment questionnaire.
Methods
A modified version of the American Board of Internal Medicine's (ABIM) Patient Assessment survey was completed by 130 residents in 23 surgical and non-surgical training programs affiliated with a single medical school. Descriptive, regression and factor analyses were performed. Internal consistency, inter-item gamma scores, and discriminative validity of the questionnaire were determined.
Results
Factor analysis suggested two groups of items: one group relating to developing interpersonal relationships with patients and one group relating to conveying medical information to patients. Cronbach's alpha (0.86) indicated internal consistency. Males rated themselves higher than females in items related to explaining things to patients. When compared to graduates of U.S. medical schools, graduates of medical schools outside the U.S. rated themselves higher in items related to listening to the patient, yet lower in using understandable language. Surgical residents rated themselves higher than non-surgical residents in explaining options to patients.
Conclusion
This appears to be an internally consistent and reliable tool for residents' self-assessment of communication skills and professionalism. Some demographic differences in self-perceived communication skills were noted.
doi:10.1186/1472-6920-9-1
PMCID: PMC2631014  PMID: 19133146
12.  Procedural Experience and Comfort Level in Internal Medicine Trainees 
BACKGROUND
The American Board of Internal Medicine (ABIM) has recommended a specific number of procedures be done as a minimum standard for ensuring competence in various medical procedures. These minimum standards were determined by consensus of an expert panel and may not reflect actual procedural comfort or competence.
OBJECTIVE
To estimate the minimum number of selected procedures at which a majority of internal medicine trainees become comfortable performing that procedure.
DESIGN
Cross-sectional, self-administered survey.
SETTING
A military-based, a community-based, and 2 university-based programs.
PARTICIPANTS
Two hundred thirty-two internal medicine residents.
MEASUREMENTS
Survey questions included number of specific procedures performed, comfort level with performing specific procedures, and whether respondents desired further training in specific procedures. The comfort threshold for a given procedure was defined as the number of procedures at which two thirds or more of the respondents reported being comfortable or very comfortable performing that procedure.
RESULTS
For three of seven procedures selected, residents were comfortable performing the procedure at or below the number recommended by the ABIM as a minimum requirement. However, residents needed more procedures than recommended by the ABIM to feel comfortable with central venous line placement, knee joint aspiration, lumbar puncture, and thoracentesis. Using multivariate logistic regression analysis, variables independently associated with greater comfort performing selected procedures included increased number performed, more years of training, male gender, career goals, and for skin biopsy, training in the community-based program. Except for skin biopsy, comfort level was independent of training site. A significant number of advanced-year house officers in some programs had little experience in performing selected common ambulatory procedures.
CONCLUSION
Minimum standards for certifying internal medicine residents may need to be reexamined in light of house officer comfort level performing selected procedures.
doi:10.1046/j.1525-1497.2000.91104.x
PMCID: PMC1495602  PMID: 11089715
ABIM; procedure comfort level; residents
13.  Maintenance of certification in Internal Medicine: participation rates and patient outcomes 
The clinical practice of internal medicine continues to evolve with the addition of new information and new technology. Most internists in practice will have erosion of their knowledge after they complete training unless life-long learning occurs. The American Board of Internal Medicine (ABIM) began to issue time-limited certification in 1990 and asserts that the Maintenance of Certification (MOC) program promotes the professional development of internists. However, the available medical literature does not provide strong support for the assumption that internists with certification or recertification have better patient outcomes. This relationship between recertification and patient outcomes needs more study. In addition, the participation in the Maintenance of Certification program by internists with lifetime certifications has been low, and recertification by leaders in internal medicine has also been relatively low. Some physicians in practice have concerns about the relevance of the program and the cost. Our review suggests that the ABIM needs to review its current Maintenance of Certification program and make changes to enhance its clinical relevance and educational value. We suggest that professional development should be based on focused reviews of the current literature, which is immediately relevant to clinical practice, and that recertification could be based on completion of modules and more frequent, less onerous testing.
doi:10.3402/jchimp.v2i4.19753
PMCID: PMC3715151  PMID: 23882382
certification; recertification; internal medicine; patient outcomes; mortality
14.  Teaching Internal Medicine Residents Quality Improvement Techniques using the ABIM’s Practice Improvement Modules 
Summary
Introduction/aim
Standard curricula to teach Internal Medicine residents about quality assessment and improvement, important components of the Accreditation Council for Graduate Medical Education core competencies practiced-based learning and improvement (PBLI) and systems-based practice (SBP), have not been easily accessible.
Program description
Using the American Board of Internal Medicine’s (ABIM) Clinical Preventative Services Practice Improvement Module (CPS PIM), we have incorporated a longitudinal quality assessment and improvement curriculum (QAIC) into the 2 required 1-month ambulatory rotations during the postgraduate year 2. During the first block, residents complete the PIM chart reviews, patient, and system surveys. The second block includes resident reflection using PIM data and the group performing a small test of change using the Plan–Do–Study–Act (PDSA) cycle in the resident continuity clinic.
Program Evaluation
To date, 3 resident quality improvement (QI) projects have been undertaken as a result of QAIC, each making significant improvements in the residents’ continuity clinic. Resident confidence levels in QI skills (e.g., writing an aim statement [71% to 96%, P < .01] and using a PDSA cycle [9% to 89%, P < .001]) improved significantly.
Discussion
The ABIM CPS PIM can be used by Internal Medicine residency programs to introduce QI concepts into their residents’ outpatient practice through encouraging practice-based learning and improvement and systems-based practice.
doi:10.1007/s11606-008-0549-5
PMCID: PMC2517947  PMID: 18449612
Internal Medicine residents; quality improvement; practiced-based learning and improvement; systems-based practice; practice improvement module
15.  Relationship between internal medicine program board examination pass rates, accreditation standards, and program size 
Objectives: To determine Internal Medicine residency program compliance with the Accreditation Council for Graduate Medical Education 80% pass-rate standard and the correlation between residency program size and performance on the American Board of Internal Medicine Certifying Examination.
Methods
Using a cross-sectional study design from 2010-2012 American Board of Internal Medicine Certifying Examination data of all Internal Medicine residency programs, comparisons were made between program pass rates to the Accreditation Council for Graduate Medical Education pass-rate standard. To assess the correlation between program size and performance, a Spearman’s rho was calculated. To evaluate program size and its relationship to the pass-rate standard, receiver operative characteristic curves were calculated.
Results
Of 372 Internal Medicine residency programs, 276 programs (74%) achieved a pass rate of ≥80%, surpassing the Accreditation Council for Graduate Medical Education minimum standard. A weak correlation was found between residency program size and pass rate for the three-year period (ρ=0.19, p<0.001). The area underneath the receiver operative characteristic curve was 0.69 (95% Confidence Interval [0.63-0.75]), suggesting programs with less than 12 examinees/year are less likely to meet the minimum Accreditation Council for Graduate Medical Education pass-rate standard (sensitivity 63.8%, specificity 60.4%, positive predictive value 82.2%, p<0.001).
Conclusions
Although a majority of Internal Medicine residency programs complied with Accreditation Council for Graduate Medical Education pass-rate standards, a quarter of the programs failed to meet this requirement. Program size is positively but weakly associated with American Board of Internal Medicine Certifying Examination performance, suggesting other unidentified variables significantly contribute to program performance.
doi:10.5116/ijme.52c5.6602
PMCID: PMC4207188  PMID: 25341205
Certification; educational measurement; ROC curve; sensitivity and specificity; specialty boards
16.  Correlation of the Emergency Medicine Resident In-Service Examination with the American Osteopathic Board of Emergency Medicine Part I 
Introduction: Eligible residents during their fourth postgraduate year (PGY-4) of emergency medicine (EM) residency training who seek specialty board certification in emergency medicine may take the American Osteopathic Board of Emergency Medicine (AOBEM) Part 1 Board Certifying Examination (AOBEM Part 1). All residents enrolled in an osteopathic EM residency training program are required to take the EM Resident In-service Examination (RISE) annually. Our aim was to correlate resident performance on the RISE with performance on the AOBEM Part 1. The study group consisted of osteopathic EM residents in their PGY-4 year of training who took both examinations during that same year.
Methods: We examined data from 2009 to 2012 from the National Board of Osteopathic Medical Examiners (NBOME). The NBOME grades and performs statistical analyses on both the RISE and the AOBEM Part 1. We used the RISE exam scores, as reported by percentile rank, and compared them to both the score on the AOBEM Part 1 and the dichotomous outcome of passing or failing. A receiver operating characteristic (ROC) curve was generated to depict the relationship.
Results: We studied a total of 409 residents over the 4-year period. The RISE percentile score correlated strongly with the AOBEM Part 1 score for residents who took both exams in the same year (r=0.61, 95% confidence interval [CI] 0.54 to 0.66). Pass percentage on the AOBEM Part 1 increased by resident percent decile on the RISE from 0% in the bottom decile to 100% in the top decile. ROC analysis also showed that the best cutoff for determining pass or fail on the AOBEM Part 1 was a 65th percentile score on the RISE.
Conclusion: We have shown there is a strong correlation between a resident's percentile score on the RISE during their PGY-4 year of residency training and first-time success on the AOBEM Part 1 taken during the same year. This information may be useful for osteopathic EM residents as an indicator as to how well prepared they are for the AOBEM Part 1 Board Certifying Examination.
doi:10.5811/westjem.2013.7.17904
PMCID: PMC3952889  PMID: 24696749
17.  A Multiple Choice Testing Program Coupled with a Year-long Elective Experience is Associated with Improved Performance on the Internal Medicine In-Training Examination 
Journal of General Internal Medicine  2011;26(11):1253-1257.
Background
The Internal Medicine In-Training Exam (IM-ITE) assesses the content knowledge of internal medicine trainees. Many programs use the IM-ITE to counsel residents, to create individual remediation plans, and to make fundamental programmatic and curricular modifications.
Objective
To assess the association between a multiple-choice testing program administered during 12 consecutive months of ambulatory and inpatient elective experience and IM-ITE percentile scores in third post-graduate year (PGY-3) categorical residents.
Design
Retrospective cohort study.
Participants
One hundred and four categorical internal medicine residents. Forty-five residents in the 2008 and 2009 classes participated in the study group, and the 59 residents in the three classes that preceded the use of the testing program, 2005–2007, served as controls.
Intervention
A comprehensive, elective rotation specific, multiple-choice testing program and a separate board review program, both administered during a continuous long-block elective experience during the twelve months between the second post-graduate year (PGY-2) and PGY-3 in-training examinations.
Measures
We analyzed the change in median individual percent correct and percentile scores between the PGY-1 and PGY-2 IM-ITE and between the PGY-2 and PGY-3 IM-ITE in both control and study cohorts. For our main outcome measure, we compared the change in median individual percentile rank between the control and study cohorts between the PGY-2 and the PGY-3 IM-ITE testing opportunities.
Results
After experiencing the educational intervention, the study group demonstrated a significant increase in median individual IM-ITE percentile score between PGY-2 and PGY-3 examinations of 8.5 percentile points (p < 0.01). This is significantly better than the increase of 1.0 percentile point seen in the control group between its PGY-2 and PGY-3 examination (p < 0.01).
Conclusion
A comprehensive multiple-choice testing program aimed at PGY-2 residents during a 12-month continuous long-block elective experience is associated with improved PGY-3 IM-ITE performance.
doi:10.1007/s11606-011-1696-7
PMCID: PMC3208479  PMID: 21499831
Internal Medicine In-Training Exam; multiple-choice testing; medical knowledge
18.  Ambulatory Morning Report 
We assessed the ability of a novel ambulatory morning report format to expose internal medicine residents to the breadth of topics covered by the American Board of Internal Medicine (ABIM) exam. Cases were selected by the Ambulatory Assistant Chief Residents and recorded in a logbook to limit duplication. We conducted a retrospective review of 406 cases discussed from July 1998 to July 2000 and cataloged each according to the primary content area. The percentage of cases in each area accurately reflected that covered by the ABIM exam, with little redundancy or over-selection of esoteric diseases. Our data suggest that a general medicine clinic is capable of exposing house staff to the wide breadth of internal medicine topics previously thought to be unique to subspecialty clinics.
doi:10.1046/j.1525-1497.2002.10202.x
PMCID: PMC1495020  PMID: 11929507
postgraduate education; ambulatory care; internal medicine residency; morning report
19.  Mastery Learning of Advanced Cardiac Life Support Skills by Internal Medicine Residents Using Simulation Technology and Deliberate Practice 
BACKGROUND
Internal medicine residents must be competent in advanced cardiac life support (ACLS) for board certification.
OBJECTIVE
To use a medical simulator to assess postgraduate year 2 (PGY-2) residents' baseline proficiency in ACLS scenarios and evaluate the impact of an educational intervention grounded in deliberate practice on skill development to mastery standards.
DESIGN
Pretest-posttest design without control group. After baseline evaluation, residents received 4, 2-hour ACLS education sessions using a medical simulator. Residents were then retested. Residents who did not achieve a research-derived minimum passing score (MPS) on each ACLS problem had more deliberate practice and were retested until the MPS was reached.
PARTICIPANTS
Forty-one PGY-2 internal medicine residents in a university-affiliated program.
MEASUREMENTS
Observational checklists based on American Heart Association (AHA) guidelines with interrater and internal consistency reliability estimates; deliberate practice time needed for residents to achieve minimum competency standards; demographics; United States Medical Licensing Examination Step 1 and Step 2 scores; and resident ratings of program quality and utility.
RESULTS
Performance improved significantly after simulator training. All residents met or exceeded the mastery competency standard. The amount of practice time needed to reach the MPS was a powerful (negative) predictor of posttest performance. The education program was rated highly.
CONCLUSIONS
A curriculum featuring deliberate practice dramatically increased the skills of residents in ACLS scenarios. Residents needed different amounts of training time to achieve minimum competency standards. Residents enjoy training, evaluation, and feedback in a simulated clinical environment. This mastery learning program and other competency-based efforts illustrate outcome-based medical education that is now prominent in accreditation reform of residency education.
doi:10.1111/j.1525-1497.2006.00341.x
PMCID: PMC1828088  PMID: 16637824
mastery learning; medical simulation; residency education
20.  Cancer-specific survival after radical nephroureterectomy for upper urinary tract urothelial carcinoma: proposal and multi-institutional validation of a post-operative nomogram 
British Journal of Cancer  2012;106(6):1083-1088.
Background:
Owing to the scarcity of upper urinary tract urothelial carcinoma (UUT-UC) it is often necessary for investigators to pool data. A patient-specific survival nomogram based on such data is needed to predict cancer-specific survival (CSS) post nephroureterectomy (NU). Herein, we propose and validate a nomogram to predict CSS post NU.
Patients and methods:
Twenty-one French institutions contributed data on 1120 patients treated with NU for UUT-UC. A total of 667 had full data for nomogram development. Study population was divided into the nomogram development cohort (397) and external validation cohort (270). Cox proportional hazards regression models were used for univariate and multivariate analyses and to build a nomogram. A reduced model selection was performed using a backward step-down selection process, and Harrell's concordance index (c-index) was used for quantifying the nomogram accuracy. Internal validation was performed by bootstrapping and the reduced nomogram model was calibrated.
Results:
Of the 397 patients in the nomogram development cohort, 91 (22.9%) died during follow-up, of which 66 (72.5%) died as a consequence of UUT-UC. The actuarial CSS probability at 5 years was 0.76 (95% CI, 71.62-80.94). On multivariate analysis, T stage (P<0.0001), N status (P=0.014), grade (P=0.026), age (P=0.005) and location (P=0.022) were associated with CSS. The reduced nomogram model had an accuracy of 0.78. We propose a nomogram to predict 3 and 5-year CSS post NU for UUT-UC.
Conclusion:
We have devised and validated an accurate nomogram (78%), superior to any single clinical variable or current model, for predicting 5-year CSS post NU for UUT-UC.
doi:10.1038/bjc.2012.64
PMCID: PMC3304431  PMID: 22374463
nomogram; urothelial carcinoma; renal pelvis: ureter; survival; prognosis
21.  Patients’ assessment of professionalism and communication skills of medical graduates 
BMC Medical Education  2014;14:28.
Background
Professionalism and communication skills constitute important components of the integral formation of physicians which has repercussion on the quality of health care and medical education. The objective of this study was to assess medical graduates’ professionalism and communication skills from the patients’ perspective and to examine its association with patients’ socio-demographic variables.
Methods
This is a hospital based cross-sectional study. It involved 315 patients and 105 medical graduates selected by convenient sampling method. A modified and validated version of the American Board of Internal Medicine’s (ABIM) Patient Assessment survey questionnaire was used for data collection through a face to face interview. Data processing and analysis were performed using the Statistical Package for Social Science (SPSS) 16.0. Mean, frequency distribution, and percentage of the variables were calculated. A non-parametric Kruskal Wallis test was applied to verify whether the patients’ assessment was influenced by variables such as age, gender, education, at a level of significance, p ≤ 0.05.
Results
Female patients constituted 46% of the sample, whereas males constituted 54%. The mean age was 36 ± 16. Patients’ scoring of the graduate’s skills ranged from 3.29 to 3.83 with a mean of 3.64 on a five-point Likert scale. Items assessing the “patient involvement in decision-making” were assigned the minimum mean values, while items dealing with “establishing adequate communication with patient” assigned the maximum mean values. Patients, who were older than 45 years, gave higher scores than younger ones (p < 0.001). Patients with higher education reported much lower scores than those with lower education (p = 0.003). Patients’ gender did not show any statistically significant influence on the rating level.
Conclusion
Generally patients rated the medical graduates’ professionalism and communication skills at a good level. Patients’ age and educational level were significantly associated with the rating level.
doi:10.1186/1472-6920-14-28
PMCID: PMC3923249  PMID: 24517316
22.  Nomogram to Predict Insignificant Prostate Cancer at Radical Prostatectomy in Korean Men: A Multi-Center Study 
Yonsei Medical Journal  2010;52(1):74-80.
Purpose
Due to the availability of serum prostate specific antigen (PSA) testing, the detection rate of insignificant prostate cancer (IPC) is increasing. To ensure better treatment decisions, we developed a nomogram to predict the probability of IPC.
Materials and Methods
The study population consisted of 1,471 patients who were treated at multiple institutions by radical prostatectomy without neoadjuvant therapy from 1995 to 2008. We obtained nonrandom samples of n = 1,031 for nomogram development, leaving n = 440 for nomogram validation. IPC was defined as pathologic organ-confined disease and a tumor volume of 0.5 cc or less without Gleason grade 4 or 5. Multivariate logistic regression model (MLRM) coefficients were used to construct a nomogram to predict IPC from five variables, including serum prostate specific antigen, clinical stage, biopsy Gleason score, positive cores ratio and maximum % of tumor in any core. The performance characteristics were internally validated from 200 bootstrap resamples to reduce overfit bias. External validation was also performed in another cohort.
Results
Overall, 67 (6.5%) patients had a so-called "insignificant" tumor in nomogram development cohort. PSA, clinical stage, biopsy Gleason score, positive core ratio and maximum % of biopsy tumor represented significant predictors of the presence of IPC. The resulting nomogram had excellent discrimination accuracy, with a bootstrapped concordance index of 0.827.
Conclusion
Our current nomogram provides sufficiently accurate information in clinical practice that may be useful to patients and clinicians when various treatment options for screen-detected prostate cancer are considered.
doi:10.3349/ymj.2011.52.1.74
PMCID: PMC3017711  PMID: 21155038
Prostatic neoplasms; nomograms; insignificant
23.  Examination outcomes for international medical graduates pursuing or completing family medicine residency training in Quebec 
Canadian Family Physician  2010;56(9):912-918.
ABSTRACT
OBJECTIVE
To review the success of international medical graduates (IMGs) who are pursuing or have completed a Quebec residency training program and examinations.
DESIGN
We retrospectively reviewed IMGs’ success rates on the pre-residency Collège des médecins du Québec medical clinical sciences written examination and objective structured clinical examination, as well as on the post-residency Certification Examination in Family Medicine.
SETTING
Quebec.
PARTICIPANTS
All IMGs taking their examinations between 2001 and 2008, inclusive, and Canadian and American graduates taking their examinations during this same period.
MAIN OUTCOME MEASURES
Success rates for IMGs on the pre-residency and post-residency examinations, compared with success rates for Canadian and American graduates.
RESULTS
Success rates on the pre-residency clinical examinations remained below 50% from 2001 to 2008 for IMGs. Similarly, during the same period, the average success rate on the Certification examination was 56.0% for IMGs, compared with 93.5% for Canadian and American medical graduates.
CONCLUSION
Despite pre-residency competency screening and in-program orientation and supports, a substantial number of IMGs in Quebec are not passing their Certification examinations. Another study is under way to analyze reasons for some IMGs’ lack of success and to find ways to help IMGs complete residency training successfully and pass the Certification examination.
PMCID: PMC2939121  PMID: 20841596
24.  How do IMGs compare with Canadian medical school graduates in a family practice residency program? 
Canadian Family Physician  2010;56(9):e318-e322.
ABSTRACT
OBJECTIVE
To compare international medical graduates (IMGs) with Canadian medical school graduates in a family practice residency program.
DESIGN
Analysis of the results of the in-training evaluation reports (ITERs) and the Certification in Family Medicine (CCFP) examination results for 2 cohorts of IMGs and Canadian-trained graduates between the years 2006 and 2008.
SETTING
St Paul’s Hospital (SPH) in Vancouver, BC, a training site of the University of British Columbia (UBC) Family Practice Residency Program.
PARTICIPANTS
In-training evaluation reports were examined for 12 first-year and 9 second-year Canadian-trained residents at the SPH site, and 12 first-year and 12 second-year IMG residents at the IMG site at SPH; CCFP examination results were reviewed for all UBC family practice residents who took the May 2008 examination and disclosed their results.
MAIN OUTCOME MEASURES
Pass or fail rates on the CCFP examination; proportions of evaluations in each group of residents given each of the following designations: exceeds expectations, meets expectations, or needs improvement. The May 2008 CCFP examination results were reviewed.
RESULTS
Compared with the second-year IMGs, the second-year SPH Canadian-trained residents had a greater proportion of exceeds expectations designations than the IMGs. For the first-year residents, both the SPH Canadian graduates and IMGs had similar results in all 3 categories. Combining the results of the 2 cohorts, the Canadian-trained residents had 310 (99%) ITERs that were designated as either exceeds expectations or meets expectations, and only 3 (1%) ITERs were in the needs improvement category. The IMG results were 362 (97.6%) ITERs in the exceeds expectations or meets expectations categories; 9 (2%) were in the needs improvement category. Statistically these are not significant differences. Seven of the 12 (58%) IMG candidates passed the CCFP examination compared with 59 of 62 (95%) of the UBC family practice residents.
CONCLUSION
The IMG residents compared favourably with their Canadian-trained colleagues when comparing ITERs but not in passing the CCFP examination. Further research is needed to elucidate these results.
PMCID: PMC2939132  PMID: 20841570
25.  Setting a Fair Performance Standard for Physicians’ Quality of Patient Care 
Background
Assessing physicians’ clinical performance using statistically sound, evidence-based measures is challenging. Little research has focused on methodological approaches to setting performance standards to which physicians are being held accountable.
Objective
Determine if a rigorous approach for setting an objective, credible standard of minimally-acceptable performance could be used for practicing physicians caring for diabetic patients.
Design
Retrospective cohort study.
Participants
Nine hundred and fifty-seven physicians from the United States with time-limited certification in internal medicine or a subspecialty.
Main Measures
The ABIM Diabetes Practice Improvement Module was used to collect data on ten clinical and two patient experience measures. A panel of eight internists/subspecialists representing essential perspectives of clinical practice applied an adaptation of the Angoff method to judge how physicians who provide minimally-acceptable care would perform on individual measures to establish performance thresholds. Panelists then rated each measure’s relative importance and the Dunn–Rankin method was applied to establish scoring weights for the composite measure. Physician characteristics were used to support the standard-setting outcome.
Key Results
Physicians abstracted 20,131 patient charts and 18,974 patient surveys were completed. The panel established reasonable performance thresholds and importance weights, yielding a standard of 48.51 (out of 100 possible points) on the composite measure with high classification accuracy (0.98). The 38 (4%) outlier physicians who did not meet the standard had lower ratings of overall clinical competence and professional behavior/attitude from former residency program directors (p = 0.01 and p = 0.006, respectively), lower Internal Medicine certification and maintenance of certification examination scores (p = 0.005 and p < 0.001, respectively), and primarily worked as solo practitioners (p = 0.02).
Conclusions
The standard-setting method yielded a credible, defensible performance standard for diabetes care based on informed judgment that resulted in a reasonable, reproducible outcome. Our method represents one approach to identifying outlier physicians for intervention to protect patients.
Electronic supplementary material
The online version of this article (doi:10.1007/s11606-010-1572-x) contains supplementary material, which is available to authorized users.
doi:10.1007/s11606-010-1572-x
PMCID: PMC3077491  PMID: 21104453
clinical performance assessment; standard setting; composite measures; diabetes care

Results 1-25 (921136)