Our findings regarding educational outcomes for students with first-attempt Step 1 failing scores should be considered in the context of previous studies of examinees’ Step 1 and Step 2CK performance, evolving medical school policies for advancement/graduation, and the implications of these failing scores for students who ultimately graduate from medical school and seek graduate medical education (GME) positions.
The finding that older students (>24 years) who initially failed Step 1 were less likely to initially pass Step 2CK in our national sample is aligned with a previous single-institutional report, published in 2009, of a significant, inverse relationship between age and Step 2 performance among all examinees, regardless of Step l score.11
The finding that students with higher MCAT scores were more likely to pass Step 2CK on the first attempt extends previous observations of significant positive correlations between MCAT scores and 3-digit Step 2CK scores.8,9
In addition, our finding that students with failing Step l scores in the lowest and middle tertiles were less likely to pass Step 2CK on the first attempt both extends the results of other studies that have reported positive correlations between 3-digit Step l and Step 2CK scores7,9
and suggests that the lower that students’ first-attempt failing Step 1 scores are, rather than merely having a failing score, the more academic support that those students may need to advance and pass Step 2CK on the first attempt. The issue of support for students who initially fail Step l is likely to be of growing interest for faculty and administrators at U.S. medical schools because of increases in the first-attempt Step 1 failure rate in the past 4 years, from 5% in 20073
to 8% in 2010,12
as well as increases in the total number of Step l examinees in the past 4 years.3,12
The number of Step 1 examinees may be expected to grow further in coming years, reflecting rising enrollment at U.S. LCME-accredited medical schools.13
Comprehensive medical school remediation and other academic support programs for the increasing number of students who initially fail Step l, especially those students with very low failing scores who are at a particularly high risk of failing Step 2CK, might minimize the likelihood of repeated licensure examination failures.14
Although most U.S. medical school matriculants in the academic years from 1993 through 2000 were men and of white race/ethnicity,15
most students in our study sample of students who initially failed Step 1 were women and of non-white race/ethnicity. On an annual basis, of all U.S. medical school matriculants, women constituted from 42.0% (in 1993-1994) to 45.8% (in 2000-2001), and non-white students from 31.8% (in 1993-1994) to 33.9% (in 2000-2001);15
whereas, women constituted 55.5% of students in our sample of matriculants who initially failed Step l, and non-white students constituted 62.9%, of students in our sample of matriculants who initially failed Step l. The relative over-representation of women and non-white students in our sample of students is consistent with previously-reported differences in first-attempt Step l passing rates on the basis of sex and race/ethnicity.7
Among a national cohort of nearly 15,000 students enrolled in U.S. LCME-accredited medical schools, 89.1% passed Step l on the first attempt, including 91.1% of men and 85.9% of women. First-attempt Step l passing rates also differed by race/ethnicity; 93.4% of white, 86.8% of Asian,77.5% of Hispanic, and 58.2% of African American students in this national cohort passed Step l on the first attempt.7
Nevertheless, among students in our sample who initially failed Step l, women were more likely than men to take and to pass Step 2CK. In the national cohort study of U.S. medical students by Case and colleagues, 92.9% of women and 92.3% of men passed Step 2CK on the first attempt.7
Other investigators have reported that women performed better than men on clinical skills’ assessments16-18
and also obtained higher numeric scores on Step 2CK.19
Thus, among students who initially fail Step l and advance to the clinical years, women’s performance in medical school might generally be expected to improve during the clinical years.
The finding that each of URM and Asian/Pacific Islander race/ethnicity was independently associated with a lower likelihood of passing Step 2CK similarly extends a previous observation by Case and colleagues that the percentages of Asian (87.6%), Hispanic (86.9%), and African American (71.5%) students who passed Step 2CK on the first attempt were lower than the percentage of white students (96.3%).7
Our observation that URM and Asian/Pacific Islander students were less likely to pass Step 2CK in a model that controlled for Step l scores, premedical debt, and MCAT scores should be of concern to U.S. medical schools in light of ongoing efforts to promote greater racial and ethnic diversity in the emerging physician workforce.20
Further research is needed to identify other variables, amenable to intervention, that might disproportionately and negatively impact the subsequent Step 2CK success of non-white students who experience difficulties with Step l.14
The finding that a more recent matriculation year and Step l scores in the lowest and middle tertiles were associated with a lower likelihood of taking Step 2CK might be explained by changes in school policies regarding advancement/graduation and increases in Step l passing score cut-offs. Our results suggest that, as the number of schools mandating a passing score on Step l for advancement/graduation increases and as the minimum Step l passing score increases (from 176 in 199421
to 188 in 201022
), more students who initially fail Step l may not advance sufficiently to take Step 2CK but rather will be dismissed or will withdraw from medical school. Our observation that there was a small percentage of students in our study sample who never took Step 2CK but graduated is consistent both with the wide range in medical school policies regarding passage of Step l and Step 2CK to advance and with the continual evolution of these policies since the introduction of the USMLE sequence. In the academic year 2009-2010, 117 of 130 U.S. LCME-accredited medical schools required a passing Step l score, 11 schools required students only to take (but not pass) Step l, and two schools had no Step l requirements for advancement/graduation; similarly, 98 of 130 schools required a passing Step 2CK score, 30 schools required students only to take (but not pass) Step 2CK, and two schools had no Step 2CK requirements for advancement/graduation.6
It was beyond the scope of our study to examine whether there was a relationship between school policy regarding passing Step l for advancement/promotion and the likelihood of a student attempting, or passing, Step 2CK. However, such studies in the future could further inform the academic medicine community’s understanding of the variables associated with educational outcomes among students at risk for experiencing difficulties with the USMLE sequence.
Our observation that a more recent matriculation year was associated with a greater likelihood of passing Step 2CK might seem counterintuitive, particularly since the minimum Step 2CK passing score cut-off, in 1994, of 16721
has also been raised several times, most recently in 2010, to the current cut-off of 189.22
However, our findings are consistent with the hypothesis that the poorest-performing students of those who failed Step 1 become less likely to advance sufficiently to attempt Step 2CK, so the subset of Step l examinees who failed yet still took Step 2CK had obtained increasingly higher first-attempt Step l scores.
Students at private medical schools who initially failed Step 1 were neither more nor less likely to attempt Step 2CK than their counterparts at public medical schools, but they were less likely to pass Step 2CK on their first attempt. We speculate that this observation might reflect school specific differences in the extent and nature of interventions to support students who experienced difficulties with Step 1.5
Medical schools have not uniformly provided remediation for students who failed Step l. According to the LCME Annual Medical School Questionnaire responses from the 124-125 medical schools in the U.S. that completed the questionnaire in academic years 1993-1994 through 2001-2002, only 60-68 schools annually reported that they provided remediation for students who failed Step l.23
Robust support programs for at-risk students might be particularly well developed at public schools with a strong commitment to recruiting and graduating state residents.8
Future research to identify model Step l remediation programs that best promote student success on subsequent Step l, and initial Step 2CK, attempts could inform the efforts of all medical schools to provide effective support for their students who experience particular difficulties with the USMLE sequence.
Finally, as most students in our study sample ultimately graduated, including those who failed both Step 1 and Step 2CK on the first attempt, the implications of first-attempt Step 1 failing scores for graduates should be considered. First-attempt failing Step l scores may adversely affect the efforts of graduating students to secure GME positions in programs accredited by the Accreditation Council for Graduate Medical Education (ACGME). Many program directors use Step l results to select applicants to interview and to place on their rank lists for GME positions; most of these program directors seldom or never consider applicants who failed Step l or Step 2CK on the first attempt.24
With increasing competition for positions in ACGME-accredited residency training programs, un-matched students who have experienced difficulties with the USMLE sequence during medical school also may face growing challenges in securing unfilled GME positions.25
Thus, students who fail Step l, even if they subsequently pass Step 2CK on the first attempt and graduate from medical school, remain at risk for experiencing difficulties in gaining entry into GME programs, particularly in their preferred specialties and programs. The risk for experiencing such difficulties is likely even greater for those students who also failed Step 2CK on the first attempt.
A strength of our study was that we included a national cohort of students who never took Step 2CK and/or did not graduate. Including these students provided a more complete picture of the educational outcomes for all students who failed Step 1 than have previous studies, which included only those students who also took Step 2CK7
or graduated from medical school.26
A limitation of our study was that we did not include information about school-specific requirements regarding passing scores on Step l for advancement/graduation. When we conducted this study, the NBME had not limited the number of times that an examinee could attempt Step l. However, some schools that require a passing score on Step l for advancement/graduation may have had policies in place regarding the number of attempts allowed to pass Step l. For example, failure to pass Step l in three attempts is grounds for dismissal at some schools.27,28
Effective January 1, 2012, however, the USMLE Program placed a six-attempt limit to pass each Step or Step component of the USMLE sequence for all examinees.29
We also lacked information about school-specific differences in academic support programs that might have contributed to observed differences in passing Step 2CK on the first attempt.5,14,23
Finally, as the mean Step l score of matriculants excluded from our final study sample was significantly lower than the mean Step l score of matriculants included, outcomes among all U.S. medical school matriculants who fail Step l may be somewhat less favorable than the outcomes that we observed among matriculants in our study sample. Despite these limitations, our results may inform medical schools’ efforts both to better assess the risk of experiencing subsequent difficulties for the growing numbers of students who initially fail Step l and to develop interventions tailored to meet the educational needs of these students to maximize the likelihood that they will take Step 2CK and pass it on the first attempt.