|Home | About | Journals | Submit | Contact Us | Français|
The relationship between increasing numbers and diversity of medical-school enrollees and the US physician workforce size and composition has not been described.
Identify demographic and pre-matriculation factors associated with medical-school matriculants’ outcomes.
De-identified data for the 1994–1999 national cohort of 97445 matriculants, followed through March 2, 2009 to graduation or withdrawal/dismissal, were analyzed using multivariable logistic regression to identify factors associated with suboptimal outcomes.
Academic withdrawal/dismissal, non-academic withdrawal/dismissal, and graduation without first-attempt passing scores on United States Medical Licensing Examination (USMLE) Step 1 and/or Step 2CK, each compared with graduation with first-attempt passing scores on both examinations.
Of 84018 (86.2%) matriculants, 74494 (88.7%) graduated with first-attempt passing scores on Step 1 and Step 2CK, 6743 (8.0%) graduated without first-attempt passing scores on Step 1 and/or Step 2CK, 1049 (1.2%) withdrew/were dismissed for academic reasons, and 1732 (2.1%) withdrew/were dismissed for non-academic reasons. Variables associated with greater likelihood of graduation without first-attempt passing scores on Step 1 and/or Step 2CK and of academic withdrawal/dismissal included Medical College Admission Test (MCAT) scores (MCAT 18–20 [2.9% of sample]: adjusted odds ratio [OR]=13.06, 95% confidence interval [95% CI]=11.56–14.76 and OR=11.08, 95% CI=8.50–14.45, respectively; MCAT 21–23 [5.6%]: OR=7.52, 95% CI=6.79–8.33 and OR=5.97, 95% CI=4.68–7.62; MCAT 24–26 [13.9%]: OR=4.27, 95% CI=3.92–4.65 and OR=3.56, CI=2.88–4.40) each compared with MCAT>29; Asian/Pacific Islander ([18.2%]: OR=2.15, 95% CI=2.00–2.32 and OR=1.69, 95% CI = 1.37–2.09) or underrepresented minority ([14.9%]: OR=2.30, 95% CI=2.13–2.48 and OR=2.96, 95% CI=2.48–3.54) compared with white race/ethnicity, and premedical debt > $50,000 ([1.0%]:OR=1.68, 95% CI=1.35–2.08 and OR=2.33, 95% CI=1.57–3.46) compared with no debt.
Lower MCAT scores, non-white race/ethnicity, and premedical debt > $50,000 were independently associated with greater likelihood of academic withdrawal/dismissal and graduating without first-attempt passing scores on USMLE Step 1 and/or Step 2CK.
The US physician workforce is not of sufficient size or diversity to meet the population’s health care needs.1–5 Ongoing efforts to address these workforce needs include increasing both numbers and demographic diversity of medical-school matriculants.1, 3, 6 The effect of these efforts on the physician workforce size and diversity will depend in large part on the extent to which matriculants complete medical school and graduate in a timely manner, prepared for advancement through graduate medical education (GME). Since the introduction of the United States Medical Licensing Examination (USMLE) sequence, increasing numbers of medical schools have included USMLE Step 1 and/or Step 2 Clinical Knowledge (Step 2CK) passing-scores as criteria for advancement and graduation.7–9 We identified demographic and prematriculation variables associated with outcomes for matriculants who entered medical school since introduction of the USMLE sequence. Matriculants who graduated with first-attempt passing scores on USMLE Step 1 and Step 2CK comprised the optimal outcome group because they are most favorably positioned for advancement through GME. Suboptimal outcomes included “graduation without first-attempt passing scores on USMLE Step 1 and Step 2CK,” “academic withdrawal/dismissal” and “non-academic withdrawal/dismissal.”
A database constructed for this study included individualized, de-identified records for all 1994–1999 matriculants enrolled in Liaison Committee on Medical Education (LCME)-accredited US medical schools. 1994 was selected as the initial matriculation year because the USMLE sequence was not fully introduced until 1994,10 and 1999 as the latest year to allow a sufficient follow-up period for all matriculants in our study; 96% of all matriculants reportedly graduate within 10 years of matriculation.11
The Association of American Medical Colleges (AAMC) provided individualized, de-identified Student Record System (SRS) records updated through March 2, 2009 for all 1994–1999 matriculants, including matriculation year, sex, race/ethnicity, Carnegie Classification for undergraduate degree-granting institution, last-status description for matriculants no longer in medical school (academic withdrawal/dismissal, non-academic withdrawal/dismissal, or graduated), and last-status year. The AAMC also provided responses to selected items on the AAMC Matriculating Student Questionnaire (MSQ), administered annually to incoming students and completed voluntarily on an identifiable but confidential basis.12 Overall, the 1994–1999 MSQ response rates ranged between 93.6% in 1994 and 96.6 % in 1997 (David Matthew, PhD, AAMC Senior Research Analyst, personal written communication, March 4, 2010).
Matriculants’ most-recent-attempt verbal reasoning (VR), physical science (PS), and biological science (BS) subscores on the revised Medical College Admission Test (MCAT) were also provided by the AAMC, as were matriculants’ USMLE Step 1 and Step 2CK first-attempt three-digit scores and pass/fail results, released with permission from the National Board of Medical Examiners. Records for each student were linked using a unique AAMC-generated identification number. The Institutional Review Board at Washington University School of Medicine approved this study as non-human-subjects research with a waiver of consent.
Student Record System demographic variables included matriculation year, last-status date, sex, and self-identified race/ethnicity as reported to the AAMC by matriculants from a list of options on the American Medical College Application Service questionnaire. Race/ethnicity was categorized as Asian/Pacific Islander, underrepresented minority in medicine (including Black, Hispanic, and American Indian/Alaska Native), other/unknown (including matriculants who self-identified as “other,” as multiple races, or did not respond to this question), or white (reference group). Medical school duration was calculated as years elapsed from matriculation to final-status year.
Additional MSQ variables included age at matriculation and premedical debt, which were categorized as “≥$50000,” “$25000-$49999,” “$10000-$24999,” and “$100-$9999” vs “no debt.” Based on responses to the MSQ item, “Type of degree program in which you are enrolled,” matriculants enrolled in MD-only degree programs were included; those enrolled in dual-degree programs were excluded. For the MSQ item, “Indicate any programs you participated in to prepare for a career in medicine or science,” yes/no responses were analyzed for “Laboratory research-apprenticeship for college students” and “Summer academic-enrichment program for college students.”
Matriculants obtained undergraduate degrees from institutions that included 29 different Carnegie classification categories of educational institutions. 13 A 6-category variable was created for undergraduate degree-granting institution Carnegie classification, including: 1) Baccalaureate Colleges – Arts & Sciences; 2) Research Universities – High Research Activity and Doctoral/Research Universities; 3) Master’s Colleges/Universities; 4) all other Carnegie classifications of non-research-oriented undergraduate institutions (“Other Institutions”); 5) Carnegie classification “Not specified”; and 6) Research Universities – Very High Research Activity (reference).
A composite MCAT score was computed as the sum of VR, BS, and PS subscores, and a 7-category variable was created for analysis: score not available (“N/A”, to include students without MCAT scores), <18, 18–20, 21–23, 24–26, 27–29 and >29 (reference). Composite MCAT scores <18 were combined into a single category to ensure sufficient numbers in that low-score category, and all scores >29 were combined because scores in this group were similarly associated with lower likelihood of academic difficulty during medical school.14 (Fig. 3f, p. 916)
A 3-category variable was created for each of Step 1 and Step 2CK (first-attempt pass, first-attempt fail, and “did not take exam”). A 4-category outcome variable was created for all matriculants in the sample using the SRS variable, last-status description, and the 3-category Step 1 and Step 2CK variables: academic withdrawal/dismissal, non-academic withdrawal/dismissal, graduated without first-attempt-passing scores on Step 1 and/or Step 2CK, and graduated with first-attempt-passing scores Step 1 and Step 2CK as the optimal outcome and reference category.
Descriptive statistics are reported for each independent variable and the dependent variable. We report adjusted odds ratios (OR) and 95% confidence intervals (CI) from separate multivariable logistic regression models, which identified variables independently associated with each suboptimal-outcome group compared with the optimal-outcome group. Predictor variables were entered into each model in 3 blocks: 1) MCAT scores, 2) sociodemographic variables (sex, race/ethnicity, and age), and 3) premedical variables (participation in laboratory research apprenticeship, participation in summer academic enrichment program, college Carnegie classification, and premedical debt). Separate logistic regression models were run to examine the associations between suboptimal outcomes and the main effects of each variable of interest, as well as models that added the interaction between the categorical race/ethnicity and MCAT variables in a fourth block. All tests were performed using SPSS version 17.0.3 (SPSS, Inc., Chicago, IL, 2009). Two-sided P-values <.05 were considered significant.
Of the 97445 matriculants in the 1994–1999 cohort, 178 who were still in school as of March 2, 2009, 81 who were deceased and one whose degree was revoked were excluded. Of the remaining 97185 matriculants no longer in school, 91929 (94.3% of all 97445) completed the MSQ at least in part. After excluding 5815 MSQ respondents not enrolled in MD-only degree programs, 86114 eligible MSQ respondents in MD-degree programs remained. The final sample included 84018 MD-program matriculants with data for all variables of interest (86.2% of all 97445 matriculants and 97.6% of 86114 MD-program matriculants no longer in medical school). Of the 84018 matriculants, 81237 (96.7%) had graduated, 1049 (1.2%) were no longer in medical school for academic reasons (653 dismissed, 396 withdrew), and 1732 (2.1%) were no longer in medical school for non-academic reasons (121 dismissed, 12 withdrew for financial, 105 withdrew for health, and 1494 withdrew for other reasons). Of the 178 matriculants still in school, 40 of 141 (28.4%) who had taken Step 1 and 18 of 42 (42.9%) who had taken Step 2CK had first-attempt failures. Final-status outcomes by definition are suboptimal for these matriculants.
MSQ respondents included 95.1% of women and 94.2% of men, and 92.9% of underrepresented minorities, 94.4% of Asian/Pacific Islanders, 95.3% of whites, and 81.2% of unknown/other race/ethnicity. Mean (SD) MCAT score was higher among MSQ respondents than nonrespondents (29.3 [4.5] vs 28.9 [4.9]; P < .001).
Table 1 shows sample characteristics grouped by outcome categories. Mean MCAT scores differed by race/ethnicity (P<.001) and were lower among matriculants who were underrepresented minorities (24.3 [4.8]) than among matriculants who were white (29.9 [3.7]), Asian/Pacific Islander (30.8 [3.8]), and other/unknown race/ethnicity (30.7 [4.2]).
The proportions of matriculants in each race/ethnicity group varied between 1994 and 1999 (P <.001), from 15.8% to 19.9% of matriculants for Asians/Pacific Islanders, from 15.3% to 14.0% for underrepresented minorities, and from 65.9% to 65.3% for white matriculants. Mean MCAT scores increased from 28.5 (4.6) for matriculants in 1994 to 29.6 (4.3) in 1999 (P <.001). Mean Step 1 scores increased from 211.1 (21.0) in 1994 to 216.2 (23.4) in 1999 (P <.001), and mean Step 2CK scores increased from 209.7 (22.9) in 1994 to 216.9 (22.7) in 1999 (P <.001).
Of 82090 Step 1 examinees, 4920 (6.0%) failed on their first attempt. Of 81275 Step 2CK examinees, 3580 (4.4%) failed on their first attempt. Of the 3580 examinees who failed Step 2CK, 1313 (36.7%) also had failed Step 1. There were 1918 matriculants in the sample without USMLE records for Step 1 or Step 2CK (2.3% of 84018), including 12 matriculants who graduated without first-attempt-passing scores on Step 1 and/or Step 2CK, 626 in the academic withdrawal/dismissal group, and 1280 in the non-academic withdrawal/dismissal group.
Of matriculants in the sample with last-status date, 87% (72145/82971) arrived at final-status within 4 years of matriculation, including 89.6% (65909/73526) who graduated with first-attempt-passing scores on Step 1 and Step 2CK, 85.3% (1475/1729) of the non-academic withdrawal/dismissal group, 77.2% (808/1046) of the academic withdrawal/dismissal group, and 59.3% (3953/6670) who graduated without first-attempt-passing scores on Step 1 and/or Step 2CK.
Table 2 shows results of the 3 final multivariable logistic regression models of variables associated with each suboptimal outcome. The goodness-of-fit of the partial models and the final models were all acceptable (Hosmer-Lemeshow statistic >.05). Matriculants were more likely to have suboptimal outcomes if they were Asian/Pacific Islander or underrepresented minority race/ethnicity; were older; obtained undergraduate degrees from institutions in Carnegie Classification categories other than “Research Universities – Very High Research Activity”; had MCAT scores ≤ 29; had premedical debt ≥ $10000; or reported summer-academic-enrichment program participation. Matriculants were less likely to have suboptimal outcomes if they were women or reported research-apprenticeship-program participation.
In separate models (data not shown) the effect of the interaction between MCAT scores and race/ethnicity was examined for each suboptimal outcome, but there was a significant interaction effect only for the suboptimal outcome of academic withdrawal/dismissal. In this model with the interaction effect, there was no change in the significance of any of the main effects of the predictor variables of interest after the interaction effect was entered. The only categorical comparison that was significant was for the group of underrepresented minority matriculants without MCAT scores (OR=4.48, 95% confidence interval, 1.43–14.01; P=.01).
Suboptimal outcomes were observed for 11.3% of matriculants in our sample. Graduates with and without first-attempt-passing scores on Step 1 and Step 2CK were distinguished because graduates with first-attempt-passing scores are more favorably positioned for entry into and progression through GME compared with graduates without first-attempt-passing scores.
GME program directors place importance on Step 1 and Step 2CK scores in resident selection.15 In the 2008 National Resident Matching Program (NRMP) Program Director Survey results, USMLE Step 1 score was the most frequently cited factor in selecting interviewees.16 Furthermore, among program directors who required applicants to submit Step 1 and/or Step 2CK scores, 83.5% reported that they would seldom or never consider interviewing an applicant with a first-attempt Step 1 failure, and 87.7% reported that they would seldom or never consider interviewing an applicant with a first-attempt Step 2CK failure.16 Applicants with first-attempt-failing (or even lower passing) scores remain over-represented among unmatched applicants in the NRMP.17, 18
Graduates with first-attempt Step 1 and/or Step 2CK failures also face challenges during GME. Many GME programs require USMLE sequence completion for contract renewal beyond the initial GME year(s).19–21 To do so, graduates must pass USMLE Step 3.22 Step 1 and Step 2CK passing scores are prerequisites for Step 3 eligibility, and Step 3 scores correlate with MCAT, Step 1, and Step 2CK scores.22–24 Thus, graduates without first-attempt-passing scores on Step 1 and/or Step 2CK are at risk for difficulty in timely USMLE sequence completion and are vulnerable to program dismissal. Many state-licensing boards limit the number of attempts to pass each licensing examination and/or the time for USMLE sequence completion.25 For these reasons, the optimal medical-school outcome is graduation with first-attempt-passing scores on Step 1 and Step 2CK.
That lower MCAT scores were associated with an increased likelihood of suboptimal outcomes is consistent with other multi-institutional studies and a meta-analysis that documented positive associations between MCAT scores and each of Step 1 scores, third-year-clerkships’ grade-point average, and Step 2CK scores.14, 24, 26, 27 The observations that each of underrepresented minority and Asian/Pacific Islander race/ethnicity was associated with a greater likelihood of academic withdrawal/dismissal and of graduation without first-attempt- passing scores on Step 1 and Step 2CK in a model that controlled for MCAT score is consistent with a report that non-white students performed more poorly in medical school compared with white students with the same MCAT scores.26 Because these observations are from a model that also controlled for other variables including premedical debt, further research seems warranted to identify additional variables amenable to intervention that may contribute to the disparate outcomes observed on the basis of race/ethnicity.
Lower MCAT scores did not preclude an optimal outcome for many matriculants. Because medical schools accept applicants with a wide range of MCAT scores, these findings may be of value in identifying matriculants who may benefit from additional support to maximize their likelihood of an optimal outcome.14, 28 The outcomes observed among matriculants without MCAT scores, and among the race-by-MCAT-interaction group of underrepresented minority matriculants without MCAT scores, may be of interest to medical schools with special admissions programs that waive MCAT-score requirements.
That women were at lower risk of academic withdrawal/dismissal differs from findings of an earlier study, which reported that women were at greater risk for academic difficulty.29 As matriculation of women in medical school has reached parity with that of men, the physician workforce gender gap may continue to narrow.
Consistent with previous reports, older age at matriculation was associated with a greater likelihood of suboptimal outcomes.27, 29 These matriculants might have had additional responsibilities (e.g., family) during medical school or might have had difficulty gaining medical-school acceptance and devoted additional years to study or research.
Almost 50% of matriculants in the sample received undergraduate degrees from “Research Universities – Very High Research Activity” institutions; graduates from other undergraduate-institution categories were more likely to have a suboptimal outcome. These findings suggest that student experiences in very high research-activity university settings may be associated with success in the medical school environment.
College research-apprenticeship-program participation was associated with a lower likelihood of suboptimal outcomes, but summer-academic-enrichment-program participation during college was associated with a higher likelihood of suboptimal outcomes. Many summer-academic-enrichment programs are specifically intended for students interested in health professions careers who seek to strengthen their performance in premedical courses and on the MCAT, and so may be at greater risk for performance difficulties in medical school.
Because higher premedical debt was associated with greater likelihood of suboptimal outcomes, the low levels of socioeconomic diversity that exist among medical school matriculants may be even more pronounced among graduates.31 The findings regarding premedical debt and participation in college programs to prepare for a career in medicine, both of which are amenable to intervention, may be of particular interest to medical schools as they seek to meet the revised LCME accreditation standards on diversity.32
More than 40% of matriculants who graduated without first-attempt-passing scores on Step 1 and/or Step 2CK were enrolled in medical school for more than 4 years. This likely reflects, at least in part, delays in advancement or graduation among matriculants enrolled at schools with Step 1 and/or Step 2CK passing score requirements for advancement/graduation. In 1994–1995, 87 schools had such Step 1 requirements and 53 schools had such Step 2CK requirements for advancement/graduation;7 in 2000–2001, 103 schools had Step 1 requirements and 72 schools had Step 2CK requirements.8 In 2008–2009, 112 schools had Step 1 requirements and 93 had Step 2CK requirements.9 Therefore, most contemporary matriculants who initially fail Step 1 and/or Step 2CK are subject to delayed advancement or graduation if they eventually pass the examination(s), or to withdrawal/dismissal if they do not.
Despite trends towards increasing MCAT, Step 1, and Step 2CK scores, the proportion of matriculants in the optimal outcome group did not increase over time, which was likely due at least in part to changes in minimum passing scores on Step 1 and Step 2CK. The initial Step 1 passing score of 176 in 199410 was revised to 179 in 1998,33 182 in 2001,34 185 in 200735 and 188 in 2010.36 Similarly, the initial Step 2CK passing score of 167 in 199410 was revised to 170 in 1996,37 174 in 2000,34 182 in 200338 and 184 in 2007.39
Many matriculants in this study who withdrew or were dismissed from medical school had no USMLE records. School-specific curricula that facilitate identification and counseling of matriculants with difficulties before they attempt the USMLE sequence might be among the contributory factors.40
This study of a nationally representative sample of medical school matriculants should be interpreted within the context of its limitations. Since we excluded matriculants who entered other types of medical degree programs, the findings can be generalized only to MD-degree program enrollees. MSQ variables in the study were by self-report, which may be prone to self-protection bias. Inclusion of only MSQ respondents may have introduced some selection bias, since MSQ respondents had higher MCAT scores than nonrespondents. Matriculants’ experiences during medical school and medical-school-specific variables, such as cultural climate for demographically diverse student populations, curriculum, USMLE sequence policies, and quality of student support services would be expected to contribute substantially to attrition and graduation outcomes.40–43 Furthermore, because most matriculants with nonacademic withdrawal/dismissal were not dismissed, but had withdrawn from medical school for unspecified reasons, other unmeasured variables likely contributed particularly to this outcome. Because this is an observational study, causation cannot be inferred.
US LCME-accredited medical schools are currently in a period of concerted efforts to increase enrollment and diversity of enrollees. These results regarding prematriculation variables associated with suboptimal medical school outcomes may help inform these endeavors.6
Funding/Support: Funding for the study was provided by the National Institutes of Health National Institute of General Medical Sciences (R01 GM085350-01).
Role of the Sponsor: The National Institute of General Medical Sciences was not involved in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review or approval of the manuscript.
Author Contributions: Dr Jeffe had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.Study concept and design: Andriole, Jeffe
Acquisition of data: Andriole, Jeffe
Analysis and interpretation of data: Andriole, Jeffe
Drafting of the manuscript: Andriole, Jeffe
Critical revision of the manuscript for important intellectual content: Andriole, Jeffe.
Financial Disclosures: None reported.
Disclaimer: The conclusions of the authors are not necessarily those of the Association of American Medical Colleges, National Board of Medical Examiners, National Institutes of Health or their respective staff members.
Additional contributions: Data management and statistical services were provided by James Struthers, BA, and Yan Yan, MD, PhD (Washington University School of Medicine) who were supported in part by this grant. We thank our colleagues Paul Jolly, PhD, Gwen Garrison, PhD, and David Matthew, PhD, (Association of American Medical Colleges) for their support of our research efforts through provision of data and assistance with coding; Robert M. Galbraith, MD, MBA and Jillian Ketterer, (National Board of Medical Examiners) for assistance with USMLE Step 1 and Step 2 Clinical Knowledge data; and Yan Yan, MD, PhD and Mario Schootman, PhD, (Washington University School of Medicine) for their critiques of an earlier draft.