In 1998, a new selection process which utilised an aptitude test and an interview in addition to previous academic achievement was introduced into an Australian undergraduate medical course.
To test the outcomes of the selection criteria over an 11-year period.
1174 students who entered the course from secondary school and who enrolled in the MBBS from 1999 through 2009 were studied in relation to specific course outcomes. Regression analyses using entry scores, sex and age as independent variables were tested for their relative value in predicting subsequent academic performance in the 6-year course. The main outcome measures were assessed by weighted average mark for each academic year level; together with results in specific units, defined as either ‘knowledge'-based or ‘clinically’ based.
Previous academic performance and female sex were the major independent positive predictors of performance in the course. The interview score showed positive predictive power during the latter years of the course and in a range of ‘clinically' based units. This relationship was mediated predominantly by the score for communication skills.
Results support combining prior academic achievement with the assessment of communication skills in a structured interview as selection criteria into this undergraduate medical course.
Selection of medical students in the UK is still largely based on prior academic achievement, although doubts have been expressed as to whether performance in earlier life is predictive of outcomes later in medical school or post-graduate education. This study analyses data from five longitudinal studies of UK medical students and doctors from the early 1970s until the early 2000s. Two of the studies used the AH5, a group test of general intelligence (that is, intellectual aptitude). Sex and ethnic differences were also analyzed in light of the changing demographics of medical students over the past decades.
Data from five cohort studies were available: the Westminster Study (began clinical studies from 1975 to 1982), the 1980, 1985, and 1990 cohort studies (entered medical school in 1981, 1986, and 1991), and the University College London Medical School (UCLMS) Cohort Study (entered clinical studies in 2005 and 2006). Different studies had different outcome measures, but most had performance on basic medical sciences and clinical examinations at medical school, performance in Membership of the Royal Colleges of Physicians (MRCP(UK)) examinations, and being on the General Medical Council Specialist Register.
Correlation matrices and path analyses are presented. There were robust correlations across different years at medical school, and medical school performance also predicted MRCP(UK) performance and being on the GMC Specialist Register. A-levels correlated somewhat less with undergraduate and post-graduate performance, but there was restriction of range in entrants. General Certificate of Secondary Education (GCSE)/O-level results also predicted undergraduate and post-graduate outcomes, but less so than did A-level results, but there may be incremental validity for clinical and post-graduate performance. The AH5 had some significant correlations with outcome, but they were inconsistent. Sex and ethnicity also had predictive effects on measures of educational attainment, undergraduate, and post-graduate performance. Women performed better in assessments but were less likely to be on the Specialist Register. Non-white participants generally underperformed in undergraduate and post-graduate assessments, but were equally likely to be on the Specialist Register. There was a suggestion of smaller ethnicity effects in earlier studies.
The existence of the Academic Backbone concept is strongly supported, with attainment at secondary school predicting performance in undergraduate and post-graduate medical assessments, and the effects spanning many years. The Academic Backbone is conceptualized in terms of the development of more sophisticated underlying structures of knowledge ('cognitive capital’ and 'medical capital’). The Academic Backbone provides strong support for using measures of educational attainment, particularly A-levels, in student selection.
Academic Backbone; Secondary school attainment; Undergraduate medical education; Post-graduate medical education; Longitudinal analyses; Continuities; Medical student selection; Cognitive capital; Medical capital; Aptitude tests
A prospective study is currently being conducted of students who were freshmen at the University of British Columbia in 1961-1962. Three cohorts are being followed: 136 premedical students, 107 science students, and 136 persons comprising a sample of general arts students. At registration in 1961, the only demographic difference was that the Premedical cohort had a greater proportion of catholics and persons from upper occupational classes. Premedical students generally performed as well in high school as science students and better than general arts students, though the academic potential of science students as measured by achievement and aptitude tests was superior. Premedical students performed better on freshman examinations than the other cohorts, though a significantly larger proportion of science students (84.4%) actually passed into second year than of premedical students (75.0%). A disturbing observation was that by second year the Science cohort had recruited more than enough students to balance its losses, while the Premedical cohort had lost three times as many students as it recruited. The net effect was not much reduced by students who entered directly from senior matriculation or other universities, since these students were about twice as likely to enter the Science cohort as they were to enter the Premedical cohort.
Selection of the best medical students among applicants is debated and many different methods are used. Academic merits predict good academic performance, but students admitted by other pathways need not be less successful. The aim of this study, was to compare communication skills between students admitted to medical school through interviews or on academic merits, respectively.
A retrospective cohort study. Communication skills at a surgical OSCE in 2008 were assessed independently by two observers using an evaluative rating scale. Correlations, t-tests and multivariate analyses by logistic regressions were employed. Academic merits were defined as upper secondary school grade point average (GPA) or scores from the Swedish Scholastic Assessment Test (SweSAT).
The risk of showing unsatisfactory communicative performance was significantly lower among the students selected by interviews (OR 0.32, CI95 0.12-0.83), compared to those selected on the basis of academic merits. However, there was no significant difference in communication skills scores between the different admission groups; neither did the proportion of high performers differ. No difference in the result of the written examination was seen between groups.
Our results confirm previous experience from many medical schools that students selected in different ways achieve comparable results during the clinical semesters. However, selection through interview seems to reduce the number of students who demonstrate inferior communication skills at 4th year of medical school.
Over two-thirds of UK medical schools are augmenting their selection procedures for medical students by using the United Kingdom Clinical Aptitude Test (UKCAT), which employs tests of cognitive and non-cognitive personal qualities, but clear evidence of the tests’ predictive validity is lacking. This study explores whether academic performance and professional behaviours that are important in a health professional context can be predicted by these measures, when taken before or very early in the medical course.
This prospective cohort study follows the progress of the entire student cohort who entered Hull York Medical School in September 2007, having taken the UKCAT cognitive tests in 2006 and the non-cognitive tests a year later. This paper reports on the students’ first and second academic years of study. The main outcome measures were regular, repeated tutor assessment of individual students’ interpersonal skills and professional behaviour, and annual examination performance in the three domains of recall and application of knowledge, evaluation of data, and communication and practical clinical skills. The relationships between non-cognitive test scores, cognitive test scores, tutor assessments and examination results were explored using the Pearson product–moment correlations for each group of data; the data for students obtaining the top and bottom 20% of the summative examination results were compared using Analysis of Variance.
Personal qualities measured by non-cognitive tests showed a number of statistically significant relationships with ratings of behaviour made by tutors, with performance in each year’s objective structured clinical examinations (OSCEs), and with themed written summative examination marks in each year. Cognitive ability scores were also significantly related to each year’s examination results, but seldom to professional behaviours. The top 20% of examination achievers could be differentiated from the bottom 20% on both non-cognitive and cognitive measures.
This study shows numerous significant relationships between both cognitive and non-cognitive test scores, academic examination scores and indicators of professional behaviours in medical students. This suggests that measurement of non-cognitive personal qualities in applicants to medical school could make a useful contribution to selection and admission decisions. Further research is required in larger representative groups, and with more refined predictor measures and behavioural assessment methods, to establish beyond doubt the incremental validity of such measures over conventional cognitive assessments.
Relying solely on measures of intellectual aptitude and academic performance in university admissions can be disadvantageous to underprivileged students. The Fiji School of Medicine primarily uses such measures to evaluate and select student applicants, and the introduction of supplementary assessments could provide better access for students from disadvantaged backgrounds. This study examined the need for supplementary assessments in the admission process, types of additional assessments needed, and stakeholders’ views on a multi-entry multi-exit strategy currently in use at the Fiji School of Medicine. A survey of the key stakeholders was conducted in February and March 2012 using closed and open ended questionnaire. One hundred and twenty-two validated questionnaires were self-administered by key stakeholders from the College of Medicine, Nursing and Health Sciences (CMNHS) and Fiji Ministries of Education and Health, with a response rate of 61%. Returned questionnaires were analysed quantitatively and qualitatively. Sixty-five percent of respondents supported the introduction of supplementary assessments, 49% favoured admissions test, and 16% preferred assessing non-academic factors. Many respondents supported the School’s multi-entry multi-exit strategy as a ‘good policy’ that provided ‘flexibility’ and opportunity for students, but should be better regulated. These findings demonstrate the need for supplementary assessments in the selection process and for continued support for the use of multi-entry multi-exit strategy at the school.
Criteria, school admission; Equity; Fiji School of Medicine; Educational policy; Noncognitive factors
All medical schools must counsel poor-performing students, address their problems and assist them in developing into competent physicians. The objective of this study was to determine whether students with academic deficiencies in their M-1 year graduate more often, spend less time to complete the curriculum, and need fewer attempts at passing USMLE Step 1 and Step 2 by entering the Decompressed Program prior to failure of the M-1 year than those students who fail the M-1 year and then repeat it.
The authors reviewed the performance of M-1 students in the Decompressed Program and compared their outcomes to M-1 students who failed and fully repeated the M-1 year. To compare the groups upon admission, t-Tests comparing the Cognitive Index of students and MCAT scores from both groups were performed. Performance of the two groups after matriculation was also analyzed.
Decompressed students were 2.1 times more likely to graduate. Decompressed students were 2.5 times more likely to pass USMLE Step 1 on the first attempt than the repeat students. In addition, 46% of those in the decompressed group completed the program in five years compared to 18% of the repeat group.
Medical students who decompress their M-1 year prior to M-1 year failure outperform those who fail their first year and then repeat it. These findings indicate the need for careful monitoring of M-1 student performance and early intervention and counseling of struggling students.
Student attrition represents a waste of career opportunity and, at times, results in a holistic loss of sense of self-worth for the students involved. The aim of this study was to evaluate the nature, causes, and impact of medical student attrition in Nigeria.
A pilot analysis was undertaken using the records of students who failed at medical school as a result of inability to pass the second MBBS examination at Ebonyi State University, Abakaliki, Nigeria, between 2002 and 2007. Some of these students were interviewed using a structured questionnaire.
Data analysis showed that 58 (7.8%) of the students admitted into preclinical class withdrew from their study. Thirty-six (62.1%) were males and the rest were females. Thirteen of those withdrawn were interviewed, and 53.8% of them believed they had poor academic ability, while 15.4% attributed their withdrawal to family pressure. No record of guidance or counseling session programs was noted for these students either at the point of withdrawal from the faculty and on the choice of a new career path.
As a result of the high attrition rate due to low academic ability, efforts should be made to check students for evidence of this at the point of admission to medicine training. Also, more accommodating teaching programs should be encouraged in faculties to accommodate students with such challenges. Good guidance and counseling programs should be encouraged to handle these inevitable cases of attrition when they occur, to avoid the demoralizing low self-esteem that plagues these individuals for the rest of their lives.
medical students; attrition; medical education; Nigeria
To determine whether students' previous pharmacy-related work experience was associated with their pharmacy school performance (academic and clinical).
The following measures of student academic performance were examined: pharmacy grade point average (GPA), scores on cumulative high-stakes examinations, and advanced pharmacy practice experience (APPE) grades. The quantity and type of pharmacy-related work experience each student performed prior to matriculation was solicited through a student survey instrument. Survey responses were correlated with academic measures, and demographic-based stratified analyses were conducted.
No significant difference in academic or clinical performance between those students with prior pharmacy experience and those without was identified. Subanalyses by work setting, position type, and substantial pharmacy work experience did not reveal any association with student performance. A relationship was found, however, between age and work experience, ie, older students tended to have more work experience than younger students.
Prior pharmacy work experience did not affect students' overall academic or clinical performance in pharmacy school. The lack of significant findings may have been due to the inherent practice limitations of nonpharmacist positions, changes in pharmacy education, and the limitations of survey responses.
work experience; performance admissions; academic performance; high-stakes testing
The United Kingdom Clinical Aptitude Test (UKCAT) is a set of cognitive tests introduced in 2006, taken annually before application to medical school. The UKCAT is a test of aptitude and not acquired knowledge and as such the results give medical schools a standardised and objective tool that all schools could use to assist their decision making in selection, and so provide a fairer means of choosing future medical students.
Selection of students for UK medical schools is usually in three stages: assessment of academic qualifications, assessment of further qualities from the application form submitted via UCAS (Universities and Colleges Admissions Service) leading to invitation to interview, and then selection for offer of a place. Medical schools were informed of the psychometric qualities of the UKCAT subtests and given some guidance regarding the interpretation of results. Each school then decided how to use the results within its own selection system.
Annual retrospective key informant telephone interviews were conducted with every UKCAT Consortium medical school, using a pre-circulated structured questionnaire. The key points of the interview were transcribed, 'member checked' and a content analysis was undertaken.
Four equally popular ways of using the test results have emerged, described as Borderline, Factor, Threshold and Rescue methods. Many schools use more than one method, at different stages in their selection process. Schools have used the scores in ways that have sought to improve the fairness of selection and support widening participation. Initially great care was taken not to exclude any applicant on the basis of low UKCAT scores alone but it has been used more as confidence has grown.
There is considerable variation in how medical schools use UKCAT, so it is important that they clearly inform applicants how the test will be used so they can make best use of their limited number of applications.
The UK Clinical Aptitude Test (UKCAT) was introduced in 2006 as an additional tool for the selection of medical students. It tests mental ability in four distinct domains (Quantitative Reasoning, Verbal Reasoning, Abstract Reasoning, and Decision Analysis), and the results are available to students and admissions panels in advance of the selection process. As yet the predictive validity of the test against course performance is largely unknown.
The study objective was to determine whether UKCAT scores predict performance during the first two years of the 5-year undergraduate medical course at Nottingham.
We studied a single cohort of students, who entered Nottingham Medical School in October 2007 and had taken the UKCAT. We used linear regression analysis to identify independent predictors of marks for different parts of the 2-year preclinical course.
Data were available for 204/260 (78%) of the entry cohort. The UKCAT total score had little predictive value. Quantitative Reasoning was a significant independent predictor of course marks in Theme A ('The Cell'), (p = 0.005), and Verbal Reasoning predicted Theme C ('The Community') (p < 0.001), but otherwise the effects were slight or non-existent.
This limited study from a single entry cohort at one medical school suggests that the predictive value of the UKCAT, particularly the total score, is low. Section scores may predict success in specific types of course assessment.
The ultimate test of validity will not be available for some years, when current cohorts of students graduate. However, if this test of mental ability does not predict preclinical performance, it is arguably less likely to predict the outcome in the clinical years. Further research from medical schools with different types of curriculum and assessment is needed, with longitudinal studies throughout the course.
Since September 1961, a prospective study of premedical and science students has been conducted at the University of British Columbia. On completion of their sophomore year, after a year of changes from one group to another or withdrawal from either group, only 82 students existed in the diminished Premedical cohort while 137 students existed in the augmented Science cohort. These two groups have now become similar demographically, and their academic potential, as indicated by achievement and ability tests, has also become similar in terms of their mean test scores. In academic performance the present Premedical cohort has displayed some superiority over the Science cohort in high school, freshman and sophomore years. The sophomore premedical mean grade was 66.0% compared with 63.7% for the Science cohort. The hypothesis is developed that these findings reflect a difference in motivation, and therefore that perception of ultimate career goals will motivate and affect academic performance of students in their undergraduate years.
In Iran, admission to medical school is based solely on the results of the highly competitive, nationwide Konkoor examination. This paper examines the predictive validity of Konkoor scores, alone and in combination with high school grade point averages (hsGPAs), for the academic performance of public medical school students in Iran.
This study followed the cohort of 2003 matriculants at public medical schools in Iran from entrance through internship. The predictor variables were Konkoor total and subsection scores and hsGPAs. The outcome variables were (1) Comprehensive Basic Sciences Exam (CBSE) scores; (2) Comprehensive Pre-Internship Exam (CPIE) scores; and (3) medical school grade point averages (msGPAs) for the courses taken before internship. Pearson correlation and regression analyses were used to assess the relationships between the selection criteria and academic performance.
There were 2126 matriculants (1374 women and 752 men) in 2003. Among the outcome variables, the CBSE had the strongest association with the Konkoor total score (r = 0.473), followed by msGPA (r = 0.339) and the CPIE (r = 0.326). While adding hsGPAs to the Konkoor total score almost doubled the power to predict msGPAs (R2 = 0.225), it did not have a substantial effect on CBSE or CPIE prediction.
The Konkoor alone, and even in combination with hsGPA, is a relatively poor predictor of medical students’ academic performance, and its predictive validity declines over the academic years of medical school. Care should be taken to develop comprehensive admissions criteria, covering both cognitive and non-cognitive factors, to identify the best applicants to become "good doctors" in the future. The findings of this study can be helpful for policy makers in the medical education field.
Decrease in intrinsic motivation is a common complaint among elementary and junior high school students, and is related to poor academic performance. Since grade-dependent development of cognitive functions also influences academic performance by these students, we examined whether cognitive functions are related to the prevalence of decrease in intrinsic academic motivation.
The study group consisted of 134 elementary school students from 4th to 6th grades and 133 junior high school students from 7th to 9th grades. Participants completed a questionnaire on intrinsic academic motivation. They also performed paper-and-pencil and computerized cognitive tests to measure abilities in motor processing, spatial construction, semantic fluency, immediate memory, short-term memory, delayed memory, spatial working memory, and selective, alternative, and divided attention.
In multivariate logistic regression analyses adjusted for grade and gender, scores of none of the cognitive tests were correlated with the prevalence of decrease in intrinsic academic motivation in elementary school students. However, low digit span forward test score and score for comprehension of the story in the kana pick-out test were positively correlated with the prevalence of decrease in intrinsic academic motivation in junior high school students.
The present findings suggest that decrease in capacity for verbal memory is associated with the prevalence of decrease in intrinsic academic motivation among junior high school students.
To compare the academic performance of students who entered family medicine residency programs with that of students who entered other disciplines and discern whether or not family physicians are as academically talented as their colleagues in other specialties.
Retrospective quantitative study.
University of Calgary in Alberta.
Three graduating classes of students (2004 to 2006) from the University of Calgary medical school.
Main outcome measures
Student performance on various undergraduate certifying examinations in years 1, 2, and 3, along with third-year in-training evaluation reports and total score on the Medical Council of Canada Qualifying Examination Part I.
Complete data were available for 99% of graduates (N = 295). In the analysis, residency program (family medicine [n = 96] versus non–family medicine [n = 199]) served as the independent variable. Using a 1-way multivariate ANOVA (analysis of variance), no significant difference among any of the mean performance scores was observed (F5289 = 1.73, P > .05). Students who entered family medicine were also well represented within the top 10 rankings of the various performance measures.
The academic performance of students who pursued careers in family medicine did not differ from that of students who chose other specialties. Unfounded negativity toward family medicine has important societal implications, especially at a time when the gap between the number of family physicians and patients seeking primary care services appears to be widening.
Objective. To evaluate the characteristics of a cohort of master of pharmacy (MPharm) students upon entry into the program and examine associations between entry qualifications, type of secondary school attended, socioeconomic status, age, and academic performance in the MPharm program.
Methods. A retrospective cohort analysis was conducted of student data for graduates of the Aston University MPharm program during the 5-year period 2005-2006 through 2009-2010 (n=644).
Results. MPharm entrants were disproportionately drawn from socioeconomically deprived areas and independent (private) schools. Achievement prior to admission was related to the type of school attended but not to socioeconomic status. Performance in the program was not related to type of school or socioeconomic status but was strongly correlated with prior academic achievement.
Conclusions. Prior academic achievement was the most important predictor of performance in the MPharm program; however, the superior prior achievement of students who attended independent secondary schools was not seen at the point of graduation. These findings may have implications for admissions policies.
master of pharmacy degree; performance; pharmacy; graduate program
Objective To identify potential predictors of undergraduate students who struggle during their medical training.
Design Case-control study. Cases were students who had experienced academic or personal difficulties that affected their progression on the course (“strugglers”). Controls were selected at random from the corresponding year cohorts, using a ratio of four controls for each struggler.
Setting University of Nottingham Medical School.
Participants Students who entered the course over five consecutive years.
Main outcome measures Likelihood ratios for independent risk factors for struggling on the course
Results 10-15% of each year's student intake were identified as strugglers. Significant independent predictors of students being in this category were negative comments in the academic reference (likelihood ratio 2.25, 95% confidence intervals 1.44 to 3.50), lower mean examination grade at A level (2.19, 1.37 to 3.51), and the late offer of a place (1.98, 1.19 to 3.30). Male sex was a less significant risk factor (1.70, 1.09 to 2.65) as was a lower grade at GCSE science (2.13, 1.12 to 4.05). In UK students whose ethnicity was known, not being white was a significant predictor of struggling (2.77, 1.52 to 5.05) but the presence of negative comments was not. Age at entry to the course and the possession of a previous degree were not predictive.
Conclusions Our results support retention of existing selection practices relating to academic achievement and critical review of students' references. We plan to undertake further investigation of the reasons why some students, including men, those with late offers and those from ethnic minority backgrounds, may do less well on the Nottingham course.
This study was conducted to assess the problems related to stress encountered by Saudi dental students.
One thousand and thirty questionnaires were distributed in four dental schools to all students from 2nd year level up to the internship level. The questionnaire contained 66 items organized into four categories: personal and administrative, theoretical, preclinical and clinical. The relationship between the demographic variables and students’ academic performance with the investigated items was also assessed.
The response rate was 53%. Saudi dental students showed high levels of perceived stress. The clinical training issues imposed the highest level of stress on the students. Some significant relationships between the investigated variables and the level of the perceived stress were found.
Female students had higher mean overall problem scores compared to male students, and second-year students showed lower perceived problems compared to other students.
Stress; Dental students; Education; Clinical training
Baylor College of Medicine has conducted a summer enrichment program for minority/disadvantaged premedical students since 1969. Follow-up data on medical school application and acceptance for participants from 1980 through 1984 were analyzed in relation to selected preprogram variables--cumulative college grade point average, total Scholastic Aptitude Test score, competitiveness of undergraduate college, sex, and ethnicity. Results of univariate and multivariate analyses indicated that: 1) females were significantly less likely to apply to medical school than males, 2) females had significantly lower mean MCAT scores (5.9 vs 7.2) even though their preprogram academic performance was comparable to that of the males, and 3) after controlling for MCAT scores, none of the preprogram variables were significant in predicting medical school acceptance. These findings suggest the need for research to explain the discrepancy between male and female MCAT performance and frequency of medical school application in summer program participants. The findings also have implications for the type of counseling provided to female participants in summer enrichment programs.
We examined the relation between demographic characteristics and the career choices of medical students who entered McMaster University medical school between 1969 and 1975. In contrast to earlier work, this study found no significant differences in sex, age, marital status at the time of entry into medical school, undergraduate major, whether prerequisite premedical courses had been taken, undergraduate grade point average and academic performance between the graduates who chose primary care and those who chose a specialty. This suggests that many medical school graduates in the 1970s entered primary care by choice rather than by default.
Objective To describe the current methods used by English medical schools to identify prospective medical students for admission to the five year degree course.
Design Review study including documentary analysis and interviews with admissions tutors.
Setting All schools (n = 22) participating in the national expansion of medical schools programme in England.
Results Though there is some commonality across schools with regard to the criteria used to select future students (academic ability coupled with a “well rounded” personality demonstrated by motivation for medicine, extracurricular interests, and experience of team working and leadership skills) the processes used vary substantially. Some schools do not interview; some shortlist for interview only on predicted academic performance while those that shortlist on a wider range of non-academic criteria use various techniques and tools to do so. Some schools use information presented in the candidate's personal statement and referee's report while others ignore this because of concerns over bias. A few schools seek additional information from supplementary questionnaires filled in by the candidates. Once students are shortlisted, interviews vary in terms of length, panel composition, structure, content, and scoring methods.
Conclusion The stated criteria for admission to medical school show commonality. Universities differ greatly, however, in how they apply these criteria and in the methods used to select students. Different approaches to admissions should be developed and tested.
Measures used for medical student selection should predict future performance during training. A problem for any selection study is that predictor-outcome correlations are known only in those who have been selected, whereas selectors need to know how measures would predict in the entire pool of applicants. That problem of interpretation can be solved by calculating construct-level predictive validity, an estimate of true predictor-outcome correlation across the range of applicant abilities.
Construct-level predictive validities were calculated in six cohort studies of medical student selection and training (student entry, 1972 to 2009) for a range of predictors, including A-levels, General Certificates of Secondary Education (GCSEs)/O-levels, and aptitude tests (AH5 and UK Clinical Aptitude Test (UKCAT)). Outcomes included undergraduate basic medical science and finals assessments, as well as postgraduate measures of Membership of the Royal Colleges of Physicians of the United Kingdom (MRCP(UK)) performance and entry in the Specialist Register. Construct-level predictive validity was calculated with the method of Hunter, Schmidt and Le (2006), adapted to correct for right-censorship of examination results due to grade inflation.
Meta-regression analyzed 57 separate predictor-outcome correlations (POCs) and construct-level predictive validities (CLPVs). Mean CLPVs are substantially higher (.450) than mean POCs (.171). Mean CLPVs for first-year examinations, were high for A-levels (.809; CI: .501 to .935), and lower for GCSEs/O-levels (.332; CI: .024 to .583) and UKCAT (mean = .245; CI: .207 to .276). A-levels had higher CLPVs for all undergraduate and postgraduate assessments than did GCSEs/O-levels and intellectual aptitude tests. CLPVs of educational attainment measures decline somewhat during training, but continue to predict postgraduate performance. Intellectual aptitude tests have lower CLPVs than A-levels or GCSEs/O-levels.
Educational attainment has strong CLPVs for undergraduate and postgraduate performance, accounting for perhaps 65% of true variance in first year performance. Such CLPVs justify the use of educational attainment measure in selection, but also raise a key theoretical question concerning the remaining 35% of variance (and measurement error, range restriction and right-censorship have been taken into account). Just as in astrophysics, ‘dark matter’ and ‘dark energy’ are posited to balance various theoretical equations, so medical student selection must also have its ‘dark variance’, whose nature is not yet properly characterized, but explains a third of the variation in performance during training. Some variance probably relates to factors which are unpredictable at selection, such as illness or other life events, but some is probably also associated with factors such as personality, motivation or study skills.
Medical student selection; Undergraduate performance; Postgraduate performance; Educational attainment; Aptitude tests; Criterion-related construct validity; Range restriction; Right censorship; Grade inflation; Markov Chain Monte Carlo algorithm
Students who entered their freshman year for the first time in 1958 and in 1959, from all medical schools in Canada, and those entering the four Western schools in 1960 were studied from the time they matriculated until they either graduated or withdrew from medical school. The rate of attrition is about 15% of matriculants each year, with the lowest rate at the University of Western Ontario (1.7%) and the highest at the University of Ottawa (33.6%) over the time period studied. Attrition was classified as academic and non-academic. Significantly higher rates were found in the case of non-academic attrition for women and in the case of academic attrition for Commonwealth students. Significantly higher rates for both types of attrition were found for older students and students who had attended undergraduate colleges different from their medical school colleges. It would appear from available statistics that the factors which combine to produce attrition are the intellectual and personality characteristics of the student, school promotional policies and evaluation methods.
The UK Clinical Aptitude Test (UKCAT) was introduced in 2006 as an additional tool for the selection of medical students. It tests mental ability in four distinct domains (Verbal Reasoning, Quantitative Reasoning, Abstract Reasoning, and Decision Analysis), and the results are available to students and admission panels in advance of the selection process. Our first study showed little evidence of any predictive validity for performance in the first two years of the Nottingham undergraduate course.
The study objective was to determine whether the UKCAT scores had any predictive value for the later parts of the course, largely delivered via clinical placements.
Students entering the course in 2007 and who had taken the UKCAT were asked for permission to use their anonymised data in research. The UKCAT scores were incorporated into a database with routine pre-admission socio-demographics and subsequent course performance data. Correlation analysis was followed by hierarchical multivariate linear regression.
The original study group comprised 204/254 (80%) of the full entry cohort. With attrition over the five years of the course this fell to 185 (73%) by Year 5. The Verbal Reasoning score and the UKCAT Total score both demonstrated some univariate correlations with clinical knowledge marks, and slightly less with clinical skills. No parts of the UKCAT proved to be an independent predictor of clinical course marks, whereas prior attainment was a highly significant predictor (p <0.001).
This study of one cohort of Nottingham medical students showed that UKCAT scores at admission did not independently predict subsequent performance on the course. Whilst the test adds another dimension to the selection process, its fairness and validity in selecting promising students remains unproven, and requires wider investigation and debate by other schools.
UK medical students and doctors from ethnic minorities underperform in undergraduate and postgraduate examinations. Although it is assumed that white (W) and non-white (NW) students enter medical school with similar qualifications, neither the qualifications of NW students, nor their educational background have been looked at in detail. This study uses two large-scale databases to examine the educational attainment of W and NW students.
Attainment at GCSE and A level, and selection for medical school in relation to ethnicity, were analysed in two separate databases. The 10th cohort of the Youth Cohort Study provided data on 13,698 students taking GCSEs in 1999 in England and Wales, and their subsequent progression to A level. UCAS provided data for 1,484,650 applicants applying for admission to UK universities and colleges in 2003, 2004 and 2005, of whom 52,557 applied to medical school, and 23,443 were accepted.
NW students achieve lower grades at GCSE overall, although achievement at the highest grades was similar to that of W students. NW students have higher educational aspirations, being more likely to go on to take A levels, especially in science and particularly chemistry, despite relatively lower achievement at GCSE. As a result, NW students perform less well at A level than W students, and hence NW students applying to university also have lower A-level grades than W students, both generally, and for medical school applicants. NW medical school entrants have lower A level grades than W entrants, with an effect size of about -0.10.
The effect size for the difference between white and non-white medical school entrants is about B0.10, which would mean that for a typical medical school examination there might be about 5 NW failures for each 4 W failures. However, this effect can only explain a portion of the overall effect size found in undergraduate and postgraduate examinations of about -0.32.