We have created a nomogram that may be used to help residency programs to predict the probability of an individual resident passing the ABIM examination on their first attempt. To our knowledge, this is the first tool to accurately predict the chance of success on the ABIM examination. It has the advantage of using common predictors that are available to all the internal medicine residents and program directors.
United States Medical Licensing Examination is a three-step examination with four tests, sponsored by the Federation of State Medical Boards and the National Board of Medical Examiners (7
). Passing the first two steps is required before graduates start their postgraduate training, while passing the third step is required before graduation from the residency, to obtain a license to practice medicine in the United States (8
). ITE was developed in 1988 by the American College of Physicians, the Association of Professors of Medicine, and the Association of Program Directors in Internal Medicine to help program directors assess the residents and identify potential knowledge deficiencies (9
The initial univariate analysis () in the development cohort showed a strong correlation between the results of the ABIM examination and the USMLE 1 and 2 CK scores, the assessment score received by participants at the time they were interviewed for obtaining a position in the residency program, and the ITE scores in the PGY 1, 2, and 3. There was also a weak association with the time passed between the medical school graduation and beginning of the residency program, suggesting maybe that candidates having more time to study or getting some experience before starting the residency have a higher chance of succeeding in the ABIM exam. The USMLE scores and the interview scores did not portend the ABIM results in the final multivariable logistic regression model, as ITE and numbers of calls in the last 6 months of residency training had a more predictive weight. This could actually be explained by the fact that residents included in the study have already been selected based on the USMLE scores. Regarding the interview score, some of the determinants of this score take into account precisely the USMLE scores.
Previously, few studies looked at the association between the board results and ITE scores during the second (10
) or third (10
) year of internal medicine training. Similarly, our study found a very good CI for the ITE scores in the PGY 2 and 3 years () and is the first one to show that actually the ITE results during every training year (PGY 1, 2 and 3) are important in predicting the board results and provide an estimated probability in passing the ABIM exam. The ITE score during PGY 3 had the highest weight in predicting the probability score. This has very important implications for the residents considering that the ITE takes place every year during fall. Knowing the score in advance could make them change their study habits and improve the chance of passing the ABIM exam. In addition, it would give program directors the ability to identify the residents who are likely to benefit the most from additional board review curriculum.
Other studies looked at the correlation between the ability to pass the board examination and other criteria, including background resident features, namely age and gender, time passed between medical school graduation and starting the residency, USMLE scores, interest in pursuing fellowships, and attendance to teaching conferences (14
). Although in the above mentioned studies, USMLE scores, analyzed separately, were associated with the passing of the board examination, there is no mathematical model in the literature on how to predict the ability to pass the ABIM exam when the USMLE, ITE scores, and other resident individual characteristics are analyzed together.
The ACGME requires residents to attend the regularly scheduled didactic sessions (3
). Our study confirmed the FitzGerald findings (14
) that there is no association between residents’ attendance to conferences and the board score results. This cannot be generalized to other programs but has implications for the type and content of medical topics presented during these conferences. It also suggests that conferences cannot replace the individual effort each resident invests into studying.
We found no difference between residents from osteopathic and allopathic schools in the success rate of passing the board examination. The background resident characteristics, namely age and sex, did not influence the success either, but the resident age distribution was narrow, therefore findings cannot be generalized to other programs with a different age distribution.
Residents fellowship aspiration did not correlate with the board pass rate, suggesting that although residents interested in subspecialties might have extra elective rotations in their favorite specialty, they receive an equally effective training when compared to the ones planning to pursue a carrier in general internal medicine.
An interesting finding is the influence that the number of calls during the past 6 months of residency has upon the ABIM probability of success. Although small in comparison to the ITE score relevance, it suggests that easier rotation months at the end of the residency give residents an additional advantage and increases their chance of passing the board. This could be effectively used by the program directors, when scheduling rotations for the residents.
The strength of the article resides in the fact that the nomogram uses simple variables common to all the residency programs and is validated with an external cohort of residents from two other independent internal medicine residency programs.
A limitation of our study is that we had the ABIM examination as a binary outcome variable (pass/fail) as it is known that there is a wide range of variance both below and above the cut-off passing score. Having the actual passing score and knowing which residents will be immediately above the cut-off point might help target specific interventions during residency training that will help promote the resident from a competent to an outstanding one. Similarly, the ability to show which resident will be immediately below the threshold might help motivate them and intensify the efforts toward personal improvement. However, from a career standpoint, passing the ABIM examination and not the score is the most important outcome.
This article has also several limitations common to retrospective studies. We did not have enough data on USMLE step 3 results to conclude on the importance of this variable. We used noon conference, but not morning report or grand rounds attendance, because of insufficient recorded data. In addition, the nomogram itself has some limitations, having good prediction for the residents with high likelihood of passing the ABIM examination but slightly overestimating the passing rate for those with low likelihood. Due to these limitations, we feel that a larger sample of residents, the possibility of additional novel predictors that we did not include, and data collection from different residency programs are likely to improve the nomogram's accuracy and portability.
In conclusion, we created and validated a simple tool that predicts the probability of internal medicine residents passing the ABIM examination. The nomogram has the potential to help programs design curricular activities to support at-risk residents preparing for the ABIM examination.