The UK Clinical Aptitude Test (UKCAT) was introduced in 2006 as an additional tool for the selection of medical students. It tests mental ability in four distinct domains (Verbal Reasoning, Quantitative Reasoning, Abstract Reasoning, and Decision Analysis), and the results are available to students and admission panels in advance of the selection process. Our first study showed little evidence of any predictive validity for performance in the first two years of the Nottingham undergraduate course.
The study objective was to determine whether the UKCAT scores had any predictive value for the later parts of the course, largely delivered via clinical placements.
Students entering the course in 2007 and who had taken the UKCAT were asked for permission to use their anonymised data in research. The UKCAT scores were incorporated into a database with routine pre-admission socio-demographics and subsequent course performance data. Correlation analysis was followed by hierarchical multivariate linear regression.
The original study group comprised 204/254 (80%) of the full entry cohort. With attrition over the five years of the course this fell to 185 (73%) by Year 5. The Verbal Reasoning score and the UKCAT Total score both demonstrated some univariate correlations with clinical knowledge marks, and slightly less with clinical skills. No parts of the UKCAT proved to be an independent predictor of clinical course marks, whereas prior attainment was a highly significant predictor (p <0.001).
This study of one cohort of Nottingham medical students showed that UKCAT scores at admission did not independently predict subsequent performance on the course. Whilst the test adds another dimension to the selection process, its fairness and validity in selecting promising students remains unproven, and requires wider investigation and debate by other schools.
Most UK medical schools use aptitude tests during student selection, but large-scale studies of predictive validity are rare. This study assesses the United Kingdom Clinical Aptitude Test (UKCAT), and its four sub-scales, along with measures of educational attainment, individual and contextual socio-economic background factors, as predictors of performance in the first year of medical school training.
A prospective study of 4,811 students in 12 UK medical schools taking the UKCAT from 2006 to 2008 as a part of the medical school application, for whom first year medical school examination results were available in 2008 to 2010.
UKCAT scores and educational attainment measures (General Certificate of Education (GCE): A-levels, and so on; or Scottish Qualifications Authority (SQA): Scottish Highers, and so on) were significant predictors of outcome. UKCAT predicted outcome better in female students than male students, and better in mature than non-mature students. Incremental validity of UKCAT taking educational attainment into account was significant, but small. Medical school performance was also affected by sex (male students performing less well), ethnicity (non-White students performing less well), and a contextual measure of secondary schooling, students from secondary schools with greater average attainment at A-level (irrespective of public or private sector) performing less well. Multilevel modeling showed no differences between medical schools in predictive ability of the various measures. UKCAT sub-scales predicted similarly, except that Verbal Reasoning correlated positively with performance on Theory examinations, but negatively with Skills assessments.
This collaborative study in 12 medical schools shows the power of large-scale studies of medical education for answering previously unanswerable but important questions about medical student selection, education and training. UKCAT has predictive validity as a predictor of medical school outcome, particularly in mature applicants to medical school. UKCAT offers small but significant incremental validity which is operationally valuable where medical schools are making selection decisions based on incomplete measures of educational attainment. The study confirms the validity of using all the existing measures of educational attainment in full at the time of selection decision-making. Contextual measures provide little additional predictive value, except that students from high attaining secondary schools perform less well, an effect previously shown for UK universities in general.
Medical student selection; Educational attainment; Aptitude tests; UKCAT; Socio-economic factors; Contextual measures
Over two-thirds of UK medical schools are augmenting their selection procedures for medical students by using the United Kingdom Clinical Aptitude Test (UKCAT), which employs tests of cognitive and non-cognitive personal qualities, but clear evidence of the tests’ predictive validity is lacking. This study explores whether academic performance and professional behaviours that are important in a health professional context can be predicted by these measures, when taken before or very early in the medical course.
This prospective cohort study follows the progress of the entire student cohort who entered Hull York Medical School in September 2007, having taken the UKCAT cognitive tests in 2006 and the non-cognitive tests a year later. This paper reports on the students’ first and second academic years of study. The main outcome measures were regular, repeated tutor assessment of individual students’ interpersonal skills and professional behaviour, and annual examination performance in the three domains of recall and application of knowledge, evaluation of data, and communication and practical clinical skills. The relationships between non-cognitive test scores, cognitive test scores, tutor assessments and examination results were explored using the Pearson product–moment correlations for each group of data; the data for students obtaining the top and bottom 20% of the summative examination results were compared using Analysis of Variance.
Personal qualities measured by non-cognitive tests showed a number of statistically significant relationships with ratings of behaviour made by tutors, with performance in each year’s objective structured clinical examinations (OSCEs), and with themed written summative examination marks in each year. Cognitive ability scores were also significantly related to each year’s examination results, but seldom to professional behaviours. The top 20% of examination achievers could be differentiated from the bottom 20% on both non-cognitive and cognitive measures.
This study shows numerous significant relationships between both cognitive and non-cognitive test scores, academic examination scores and indicators of professional behaviours in medical students. This suggests that measurement of non-cognitive personal qualities in applicants to medical school could make a useful contribution to selection and admission decisions. Further research is required in larger representative groups, and with more refined predictor measures and behavioural assessment methods, to establish beyond doubt the incremental validity of such measures over conventional cognitive assessments.
Objective To determine whether the use of the UK clinical aptitude test (UKCAT) in the medical schools admissions process reduces the relative disadvantage encountered by certain sociodemographic groups.
Design Prospective cohort study.
Setting Applicants to 22 UK medical schools in 2009 that were members of the consortium of institutions utilising the UKCAT as a component of their admissions process.
Participants 8459 applicants (24 844 applications) to UKCAT consortium member medical schools where data were available on advanced qualifications and socioeconomic background.
Main outcome measures The probability of an application resulting in an offer of a place on a medicine course according to seven educational and sociodemographic variables depending on how the UKCAT was used by the medical school (in borderline cases, as a factor in admissions, or as a threshold).
Results On univariate analysis all educational and sociodemographic variables were significantly associated with the relative odds of an application being successful. The multilevel multiple logistic regression models, however, varied between medical schools according to the way that the UKCAT was used. For example, a candidate from a non-professional background was much less likely to receive a conditional offer of a place compared with an applicant from a higher social class when applying to an institution using the test only in borderline cases (odds ratio 0.51, 95% confidence interval 0.45 to 0.60). No such effect was observed for such candidates applying to medical schools using the threshold approach (1.27, 0.84 to 1.91). These differences were generally reflected in the interactions observed when the analysis was repeated, pooling the data. Notably, candidates from several under-represented groups applying to medical schools that used a threshold approach to the UKCAT were less disadvantaged than those applying to the other institutions in the consortium. These effects were partially reflected in significant differences in the absolute proportion of such candidates finally taking up places in the different types of medical schools; stronger use of the test score (as a factor or threshold) was associated with a significantly increased odds of entrants being male (1.74, 1.25 to 2.41) and from a low socioeconomic background (3.57, 1.03 to 12.39). There was a non-significant trend towards entrants being from a state (non-grammar) school (1.60, 0.97 to 2.62) where a stronger use of the test was employed. Use of the test only in borderline cases was associated with increased odds of entrants having relatively low academic attainment (5.19, 2.02 to 13.33) and English as a second language (2.15, 1.03 to 4.48).
Conclusions The use of the UKCAT may lead to more equitable provision of offers to those applying to medical school from under-represented sociodemographic groups. This may translate into higher numbers of some, but not all, relatively disadvantaged students entering the UK medical profession.
The United Kingdom Clinical Aptitude Test (UKCAT) is a set of cognitive tests introduced in 2006, taken annually before application to medical school. The UKCAT is a test of aptitude and not acquired knowledge and as such the results give medical schools a standardised and objective tool that all schools could use to assist their decision making in selection, and so provide a fairer means of choosing future medical students.
Selection of students for UK medical schools is usually in three stages: assessment of academic qualifications, assessment of further qualities from the application form submitted via UCAS (Universities and Colleges Admissions Service) leading to invitation to interview, and then selection for offer of a place. Medical schools were informed of the psychometric qualities of the UKCAT subtests and given some guidance regarding the interpretation of results. Each school then decided how to use the results within its own selection system.
Annual retrospective key informant telephone interviews were conducted with every UKCAT Consortium medical school, using a pre-circulated structured questionnaire. The key points of the interview were transcribed, 'member checked' and a content analysis was undertaken.
Four equally popular ways of using the test results have emerged, described as Borderline, Factor, Threshold and Rescue methods. Many schools use more than one method, at different stages in their selection process. Schools have used the scores in ways that have sought to improve the fairness of selection and support widening participation. Initially great care was taken not to exclude any applicant on the basis of low UKCAT scores alone but it has been used more as confidence has grown.
There is considerable variation in how medical schools use UKCAT, so it is important that they clearly inform applicants how the test will be used so they can make best use of their limited number of applications.
Measures used for medical student selection should predict future performance during training. A problem for any selection study is that predictor-outcome correlations are known only in those who have been selected, whereas selectors need to know how measures would predict in the entire pool of applicants. That problem of interpretation can be solved by calculating construct-level predictive validity, an estimate of true predictor-outcome correlation across the range of applicant abilities.
Construct-level predictive validities were calculated in six cohort studies of medical student selection and training (student entry, 1972 to 2009) for a range of predictors, including A-levels, General Certificates of Secondary Education (GCSEs)/O-levels, and aptitude tests (AH5 and UK Clinical Aptitude Test (UKCAT)). Outcomes included undergraduate basic medical science and finals assessments, as well as postgraduate measures of Membership of the Royal Colleges of Physicians of the United Kingdom (MRCP(UK)) performance and entry in the Specialist Register. Construct-level predictive validity was calculated with the method of Hunter, Schmidt and Le (2006), adapted to correct for right-censorship of examination results due to grade inflation.
Meta-regression analyzed 57 separate predictor-outcome correlations (POCs) and construct-level predictive validities (CLPVs). Mean CLPVs are substantially higher (.450) than mean POCs (.171). Mean CLPVs for first-year examinations, were high for A-levels (.809; CI: .501 to .935), and lower for GCSEs/O-levels (.332; CI: .024 to .583) and UKCAT (mean = .245; CI: .207 to .276). A-levels had higher CLPVs for all undergraduate and postgraduate assessments than did GCSEs/O-levels and intellectual aptitude tests. CLPVs of educational attainment measures decline somewhat during training, but continue to predict postgraduate performance. Intellectual aptitude tests have lower CLPVs than A-levels or GCSEs/O-levels.
Educational attainment has strong CLPVs for undergraduate and postgraduate performance, accounting for perhaps 65% of true variance in first year performance. Such CLPVs justify the use of educational attainment measure in selection, but also raise a key theoretical question concerning the remaining 35% of variance (and measurement error, range restriction and right-censorship have been taken into account). Just as in astrophysics, ‘dark matter’ and ‘dark energy’ are posited to balance various theoretical equations, so medical student selection must also have its ‘dark variance’, whose nature is not yet properly characterized, but explains a third of the variation in performance during training. Some variance probably relates to factors which are unpredictable at selection, such as illness or other life events, but some is probably also associated with factors such as personality, motivation or study skills.
Medical student selection; Undergraduate performance; Postgraduate performance; Educational attainment; Aptitude tests; Criterion-related construct validity; Range restriction; Right censorship; Grade inflation; Markov Chain Monte Carlo algorithm
Objectives To determine whether the UK Clinical Aptitude Test (UKCAT) adds value to the selection process for school leaver applicants to medical and dental school, and in particular whether UKCAT can reduce the socioeconomic bias known to affect A levels.
Design Cohort study
Setting Applicants to 23 UK medical and dental schools in 2006.
Participants 9884 applicants who took the UKCAT in the UK and who achieved at least three passes at A level in their school leaving examinations (53% of all applicants).
Main outcome measures Independent predictors of obtaining at least AAB at A level and
UKCAT scores at or above the 30th centile for the cohort, for the subsections and the entire test.
Results Independent predictors of obtaining at least AAB at A level were white ethnicity (odds ratio 1.58, 95% confidence interval 1.41 to 1.77), professional or managerial background (1.39, 1.22 to 1.59), and independent or grammar schooling (2.26, 2.02 to 2.52) (all P<0.001). Independent predictors of achieving UKCAT scores at or above the 30th centile for the whole test were male sex (odd ratio 1.48, 1.32 to 1.66), white ethnicity (2.17, 1.94 to 2.43), professional or managerial background (1.34, 1.17 to 1.54), and independent or grammar schooling (1.91, 1.70 to 2.14) (all P<0.001). One major limitation of the study was that socioeconomic status was not volunteered by approximately 30% of the applicants. Those who withheld socioeconomic status data were significantly different from those who provided that information, which may have caused bias in the analysis.
Conclusions UKCAT was introduced with a high expectation of increasing the diversity and fairness in selection for UK medical and dental schools. This study of a major subgroup of applicants in the first year of operation suggests that it has an inherent favourable bias to men and students from a higher socioeconomic class or independent or grammar schools. However, it does provide a reasonable proxy for A levels in the selection process.
Medical student selection is an important but difficult task. Three recent papers by McManus et al. in BMC Medicine have re-examined the role of tests of attainment of learning (A’ levels, GCSEs, SQA) and of aptitude (AH5, UKCAT), but on a much larger scale than previously attempted. They conclude that A’ levels are still the best predictor of future success at medical school and beyond. However, A’ levels account for only 65% of the variance in performance that is found. Therefore, more work is needed to establish relevant assessment of the other 35%.
Please see related research articles http://www.biomedcentral.com/1741-7015/11/242, http://www.biomedcentral.com/1741-7015/11/243 and http://www.biomedcentral.com/1741-7015/11/244.
Medical School Admission; Predictors of performance; Aptitude testing
Internationally, tests of general mental ability are used in the selection of medical students. Examples include the Medical College Admission Test, Undergraduate Medicine and Health Sciences Admission Test and the UK Clinical Aptitude Test. The most widely used measure of their efficacy is predictive validity.
A new tool, the Health Professions Admission Test- Ireland (HPAT-Ireland), was introduced in 2009. Traditionally, selection to Irish undergraduate medical schools relied on academic achievement. Since 2009, Irish and EU applicants are selected on a combination of their secondary school academic record (measured predominately by the Leaving Certificate Examination) and HPAT-Ireland score. This is the first study to report on the predictive validity of the HPAT-Ireland for early undergraduate assessments of communication and clinical skills.
Students enrolled at two Irish medical schools in 2009 were followed up for two years. Data collected were gender, HPAT-Ireland total and subsection scores; Leaving Certificate Examination plus HPAT-Ireland combined score, Year 1 Objective Structured Clinical Examination (OSCE) scores (Total score, communication and clinical subtest scores), Year 1 Multiple Choice Questions and Year 2 OSCE and subset scores. We report descriptive statistics, Pearson correlation coefficients and Multiple linear regression models.
Data were available for 312 students. In Year 1 none of the selection criteria were significantly related to student OSCE performance. The Leaving Certificate Examination and Leaving Certificate plus HPAT-Ireland combined scores correlated with MCQ marks.
In Year 2 a series of significant correlations emerged between the HPAT-Ireland and subsections thereof with OSCE Communication Z-scores; OSCE Clinical Z-scores; and Total OSCE Z-scores. However on multiple regression only the relationship between Total OSCE Score and the Total HPAT-Ireland score remained significant; albeit the predictive power was modest.
We found that none of our selection criteria strongly predict clinical and communication skills. The HPAT- Ireland appears to measures ability in domains different to those assessed by the Leaving Certificate Examination. While some significant associations did emerge in Year 2 between HPAT Ireland and total OSCE scores further evaluation is required to establish if this pattern continues during the senior years of the medical course.
Selection; Medical; Student; Validity; Predictive; HPAT-Ireland; Assessment; Cognitive; Ability
To compare the power of three traditional selection procedures (A levels, personal statements, and references) and one non-traditional selection procedure (personality) to predict performance over the five years of a medical degree.
Cohort study over five years.
Nottingham medical school.
Entrants in 1995.
Main outcome measures
A level grades, amounts of information contained in teacher's reference and the student's personal statement, and personality scores examined in relation to 18 different assessments.
Information in the teacher's reference did not consistently predict performance. Information in the personal statement was predictive of clinical aspects of training, whereas A level grades primarily predicted preclinical performance. The personality domain of conscientiousness was consistently the best predictor across the course. A structural model indicated that conscientiousness was positively related to A level grades and preclinical performance but was negatively related to clinical grades.
A teacher's reference is of no practical use in predicting clinical performance of medical students, in contrast to the amount of information contained in the personal statement. Therefore, simple quantification of the personal statement should aid selection. Personality factors, in particular conscientiousness, need to be considered and integrated into selection procedures.
In 1998, a new selection process which utilised an aptitude test and an interview in addition to previous academic achievement was introduced into an Australian undergraduate medical course.
To test the outcomes of the selection criteria over an 11-year period.
1174 students who entered the course from secondary school and who enrolled in the MBBS from 1999 through 2009 were studied in relation to specific course outcomes. Regression analyses using entry scores, sex and age as independent variables were tested for their relative value in predicting subsequent academic performance in the 6-year course. The main outcome measures were assessed by weighted average mark for each academic year level; together with results in specific units, defined as either ‘knowledge'-based or ‘clinically’ based.
Previous academic performance and female sex were the major independent positive predictors of performance in the course. The interview score showed positive predictive power during the latter years of the course and in a range of ‘clinically' based units. This relationship was mediated predominantly by the score for communication skills.
Results support combining prior academic achievement with the assessment of communication skills in a structured interview as selection criteria into this undergraduate medical course.
It has been suggested that studying non-science subjects at A-level should be compulsory for medical students. Our admissions criteria specify only Biology, Chemistry and one or more additional subjects. This study aimed to determine whether studying a non-science subject for A-level is an independent predictor of achievement on the undergraduate medical course.
The subjects of this retrospective cohort study were 164 students from one entry-year group (October 2000), who progressed normally on the 5-year undergraduate medical course at Nottingham. Pre-admission academic and socio-demographic data and undergraduate course marks were obtained. T-test and hierarchical multiple linear regression analyses were undertaken to identify independent predictors of five course outcomes at different stages throughout the course.
There was no evidence that the choice of science or non-science as the third or fourth A-level subject had any influence on course performance. Demographic variables (age group, sex, and fee status) had some predictive value but ethnicity did not. Pre-clinical course performance was the strongest predictor in the clinical phases (pre-clinical Themes A&B (knowledge) predicted Clinical Knowledge, p < 0.001, and pre-clinical Themes C&D (skills) predicted Clinical Skills, p = < 0.01).
This study of one year group at Nottingham Medical School provided no evidence that the admissions policy on A-level requirements should specify the choice of third or fourth subject.
Selection of medical students in the UK is still largely based on prior academic achievement, although doubts have been expressed as to whether performance in earlier life is predictive of outcomes later in medical school or post-graduate education. This study analyses data from five longitudinal studies of UK medical students and doctors from the early 1970s until the early 2000s. Two of the studies used the AH5, a group test of general intelligence (that is, intellectual aptitude). Sex and ethnic differences were also analyzed in light of the changing demographics of medical students over the past decades.
Data from five cohort studies were available: the Westminster Study (began clinical studies from 1975 to 1982), the 1980, 1985, and 1990 cohort studies (entered medical school in 1981, 1986, and 1991), and the University College London Medical School (UCLMS) Cohort Study (entered clinical studies in 2005 and 2006). Different studies had different outcome measures, but most had performance on basic medical sciences and clinical examinations at medical school, performance in Membership of the Royal Colleges of Physicians (MRCP(UK)) examinations, and being on the General Medical Council Specialist Register.
Correlation matrices and path analyses are presented. There were robust correlations across different years at medical school, and medical school performance also predicted MRCP(UK) performance and being on the GMC Specialist Register. A-levels correlated somewhat less with undergraduate and post-graduate performance, but there was restriction of range in entrants. General Certificate of Secondary Education (GCSE)/O-level results also predicted undergraduate and post-graduate outcomes, but less so than did A-level results, but there may be incremental validity for clinical and post-graduate performance. The AH5 had some significant correlations with outcome, but they were inconsistent. Sex and ethnicity also had predictive effects on measures of educational attainment, undergraduate, and post-graduate performance. Women performed better in assessments but were less likely to be on the Specialist Register. Non-white participants generally underperformed in undergraduate and post-graduate assessments, but were equally likely to be on the Specialist Register. There was a suggestion of smaller ethnicity effects in earlier studies.
The existence of the Academic Backbone concept is strongly supported, with attainment at secondary school predicting performance in undergraduate and post-graduate medical assessments, and the effects spanning many years. The Academic Backbone is conceptualized in terms of the development of more sophisticated underlying structures of knowledge ('cognitive capital’ and 'medical capital’). The Academic Backbone provides strong support for using measures of educational attainment, particularly A-levels, in student selection.
Academic Backbone; Secondary school attainment; Undergraduate medical education; Post-graduate medical education; Longitudinal analyses; Continuities; Medical student selection; Cognitive capital; Medical capital; Aptitude tests
Objective To identify potential predictors of undergraduate students who struggle during their medical training.
Design Case-control study. Cases were students who had experienced academic or personal difficulties that affected their progression on the course (“strugglers”). Controls were selected at random from the corresponding year cohorts, using a ratio of four controls for each struggler.
Setting University of Nottingham Medical School.
Participants Students who entered the course over five consecutive years.
Main outcome measures Likelihood ratios for independent risk factors for struggling on the course
Results 10-15% of each year's student intake were identified as strugglers. Significant independent predictors of students being in this category were negative comments in the academic reference (likelihood ratio 2.25, 95% confidence intervals 1.44 to 3.50), lower mean examination grade at A level (2.19, 1.37 to 3.51), and the late offer of a place (1.98, 1.19 to 3.30). Male sex was a less significant risk factor (1.70, 1.09 to 2.65) as was a lower grade at GCSE science (2.13, 1.12 to 4.05). In UK students whose ethnicity was known, not being white was a significant predictor of struggling (2.77, 1.52 to 5.05) but the presence of negative comments was not. Age at entry to the course and the possession of a previous degree were not predictive.
Conclusions Our results support retention of existing selection practices relating to academic achievement and critical review of students' references. We plan to undertake further investigation of the reasons why some students, including men, those with late offers and those from ethnic minority backgrounds, may do less well on the Nottingham course.
Although several studies have examined the relationship between minority students' admissions profiles and performance in the preclinical curriculum, there is a dearth of information about the ability of admissions variables to predict performance in the clerkships and on National Boards, Part II. Consistent with other research, a study of 59 minority students at the Albert Einstein College of Medicine found that the Medical College Aptitude Test (MCAT) chemistry score is the most consistent predictor of performance on internal examinations in years 1 and 2, and on National Boards, Part I. On the Part II examination, however, the only significant correlation is with the MCAT reading score, while the MCAT quantitative score and the recommendation of the premedical advisor are the best predictors of clerkship grades. Since students' mean MCATs and grade point averages (GPAs) are similar to those of all minority students accepted to medical schools in 1982, these findings may be generalized to that larger population.
At Nottingham University more than 95% of entrants to the traditional 5-year medical course are school leavers. Since 2003 we have admitted graduate entrants (GEM) to a shortened (4-year) course to 'widen access to students from more disadvantaged backgrounds'. We have recently shown that the GEM course widens academic and socio-demographic diversity of the medical student population. This study explored whether GEM students also bring psychological diversity and whether this could be beneficial.
We studied: a) 217 and 96 applicants to the Nottingham 5- and 4-year courses respectively, applying in the 2002-3 UCAS cycle, and, b) 246 school leavers starting the 5-year course and 39 graduate entrants to the 4-year course in October 2003. The psychological profiles of the two groups of applicants and two groups of entrants were compared using their performance in the Goldberg 'Big 5' Personality test, the Personal Qualities Assessment (PQA; measuring interpersonal traits and interpersonal values), and the Lovibond and Lovibond measure of depression, anxiety and stress. For the comparison of the Entrants we excluded the 33 school leavers and seven graduates who took the tests as Applicants.
Statistical analyses were undertaken using SPSS software (version 16.0).
Graduate applicants compared to school leaver applicants were significantly more conscientious, more confident, more self controlled, more communitarian in moral orientation and less anxious. Only one of these differences was preserved in the entrants with graduates being less anxious. However, the graduate entrants were significantly less empathetic and conscientious than the school leavers.
This study has shown that school leaver and graduate entrants to medical school differ in some psychological characteristics. However, if confirmed in other studies and if they were manifest in the extreme, not all the traits brought by graduates would be desirable for someone aiming for a medical career.
This study examines cognitive and noncognitive selection variables as predictors of multiple criteria of performance for minority and nonminority students entering the University of Missouri-Kansas City School of Medicine between 1972 and 1977. Data analysis aims at identifying characteristics associated with success in the crucial first two years of the combined BA/MD program. Pearson product moment correlations and multiple regression equations have been determined for all selection variables and three criteria measures. A number of nontraditional variables are significantly predictive of minority student performance. The cognitive variables of aptitude test and high school science/mathematics preparation are significant predictors of all three performance measures in both racial groups. The data support the position that separate equations are not necessary to predict performance of minority and nonminority students, but that admission committees, when selecting minority students, should recognize both personal attributes and academic preparation as indicators of potential success or failure.
Graduate entry medicine is a recent innovation in UK medical training. Evidence is sparse at present as to progress and attainment on these programmes. Shared clinical rotations, between an established 5-year and a new graduate entry course, provide the opportunity to compare achievement on clinical assessments. To compare completion and attainment on clinical phase assessments between students on a 4-year graduate entry course and an established 5-year undergraduate medicine course.
Overall completion rates for the 4 and 5 year courses, fails at first attempt, and scores on 14 clinical assessments, were compared between 171 graduate-entry and 450 undergraduate medical students at the University of Nottingham, comprising two graduating cohorts. Percentage assessment marks were converted to z-scores separately for each graduating year and the normalised marks then combined into a single dataset. Z-score transformed percentage marks were analysed by multivariate analysis of variance and univariate analyses of variance for each summative assessment. Numbers of fails at first attempt were analysed aggregated across all assessments initially, then separately for each assessment using χ2.
Completion rates were around 90% overall and significantly higher in the graduate entry course. Failures of assessments overall were similar, but a higher proportion of graduate entry students failed the final OSLER. Mean performance on clinical assessments showed a significant overall difference, made up of lower performance on 4 of 5 knowledge-based exams (as well as higher performance on the first exam) by the graduate entry group, but similar levels of performance on all the skills-based and attitudinal assessments.
High completion rates are encouraging. The lower performance in some knowledge-based exams may reflect lower prior educational attainment, a substantially different demographic profile (age, gender), or an artefact of the first 2 years of a new graduate entry programme.
Baylor College of Medicine has conducted a summer enrichment program for minority/disadvantaged premedical students since 1969. Follow-up data on medical school application and acceptance for participants from 1980 through 1984 were analyzed in relation to selected preprogram variables--cumulative college grade point average, total Scholastic Aptitude Test score, competitiveness of undergraduate college, sex, and ethnicity. Results of univariate and multivariate analyses indicated that: 1) females were significantly less likely to apply to medical school than males, 2) females had significantly lower mean MCAT scores (5.9 vs 7.2) even though their preprogram academic performance was comparable to that of the males, and 3) after controlling for MCAT scores, none of the preprogram variables were significant in predicting medical school acceptance. These findings suggest the need for research to explain the discrepancy between male and female MCAT performance and frequency of medical school application in summer program participants. The findings also have implications for the type of counseling provided to female participants in summer enrichment programs.
If minority students likely to score low on the Medical College Admissions Test (MCAT) can be identified in advance, they can be advised to take existing preparatory programs, or programs can be developed to meet their needs. Correlation coefficients for a number of available independent variables with MCAT scores were determined for a population of premedical students at Xavier University of Louisiana. American College Testing (ACT) and Scholastic Aptitude Test (SAT) scores were found to have similar ability to predict MCAT scores, with a correlation coefficient of 0.64 between ACT composite and MCAT total scores. Correlations of sophomore year grade point average (GPA) with MCAT scores were only slightly weaker. Use of subtest scores for the ACT and SAT, grades in science courses, and Nelson-Denny Reading Test scores did not improve prediction to any real extent, either when used alone or in multiple linear regression analysis. In contrast to some previous studies, predictions for black men were as good as those for black women. Use of only ACT composite and sophomore year GPA together gave correlations only slightly weaker than predictions using a full range of variables; data from ACT composite and sophomore year GPA can be used for calculating predictive equations on many available micro-computers. These procedures may not be applicable to minority students at majority institutions.
Prior to 1999 students entering our MBBS course were selected on academic performance alone. We have now evaluated the impact on the demographics of subsequent cohorts of our standard entry students (those entering directly from high school) of the addition to the selection process of an aptitude test (UMAT), a highly structured interview and a rural incentive program.
Students entering from 1985 to 1998, selected on academic performance alone (N = 1402), were compared to those from 1999 to 2011, selected on the basis of a combination of academic performance, interview score, and UMAT score together with the progressive introduction of a rural special entry pathway (N = 1437).
Males decreased from 57% to 45% of the cohort, students of NE or SE Asian origin decreased from 30% to 13%, students born in Oceania increased from 52% to 69%, students of rural origin from 5% to 21% and those from independent high schools from 56% to 66%. The proportion of students from high schools with relative socio-educational disadvantage remained unchanged at approximately 10%. The changes reflect in part increasing numbers of female and independent high school applicants and the increasing rural quota. However, they were also associated with higher interview scores in females vs males and lower interview scores in those of NE and SE Asian origin compared to those born in Oceania or the UK. Total UMAT scores were unrelated to gender or region of origin.
The revised selection processes had no impact on student representation from schools with relative socio-educational disadvantage. However, the introduction of special entry quotas for students of rural origin and a structured interview, but not an aptitude test, were associated with a change in gender balance and ethnicity of students in an Australian undergraduate MBBS course.
Introduction: The present study examines the question whether the selection of dental students should be based solely on average school-leaving grades (GPA) or whether it could be improved by using a subject-specific aptitude test.
Methods: The HAM-Nat Natural Sciences Test was piloted with freshmen during their first study week in 2006 and 2007. In 2009 and 2010 it was used in the dental student selection process. The sample size in the regression models varies between 32 and 55 students.
Results: Used as a supplement to the German GPA, the HAM-Nat test explained up to 12% of the variance in preclinical examination performance. We confirmed the prognostic validity of GPA reported in earlier studies in some, but not all of the individual preclinical examination results.
Conclusion: The HAM-Nat test is a reliable selection tool for dental students. Use of the HAM-Nat yielded a significant improvement in prediction of preclinical academic success in dentistry.
student selection dentistry; prediction of study success; admission test
The performance during the preclinical course of 517 students who had applied to this medical school for admission in 1981 and who had been accepted by the school or by another British medical school was analysed in relation to variables measured at the time of application to find factors that predicted success in the preclinical course, whether students chose to take an intercalated degree, and the class achieved in the intercalated degree. Thirty one of the 507 students who entered medical school withdrew from the course or failed their examinations; these students were particularly likely not to have an A level in a biological science. O level grades were of minimal predictive value for performance during the preclinical course. A level grades discriminated between successful and unsuccessful students but had too low a specificity or sensitivity to be of use in individual prediction. Mature entrants performed better overall than school leavers. Background variables accounted for only 14.2% of the variance in performance, implying that motivation and personality may be more important in determining performance. The 80 students who chose to take an intercalated degree were more likely to be men and not to be mature entrants; for a further 50 students intercalated degrees were obligatory. Performance in the intercalated degree related to performance during the preclinical course and to assessments made at the selection interview but not to achievement at O or A level.
OBJECTIVE: To assess whether the clinical experience of undergraduate medical students relates to their performance in final examinations and whether learning styles relate either to final examination performance or to the extent of clinical experience. DESIGN: Prospective, longitudinal study of two cohorts of medical students assessed by questionnaire at time of application to medical school and by questionnaire and university examination at the end of their final clinical year. SUBJECTS: Two cohorts of students who had applied to St Mary's Hospital Medical School during 1980 (n = 1478) and 1985 (n = 2399) for admission in 1981 and 1986 respectively. Students in these cohorts who entered any medical school in the United Kingdom were followed up in their final clinical year in 1986-7 and 1991-2. MAIN OUTCOME MEASURES: Student's clinical experience of a range of acute medical conditions, surgical operations, and practical procedures as assessed by questionnaire in the final year, and final examination results for the students taking their examinations at the University of London. RESULTS: Success in the final examination was not related to a student's clinical experiences. The amount of knowledge gained from clinical experience was, however, related to strategic and deep learning styles both in the final year and also at the time of application, five or six years earlier. Grades in A level examinations did not relate either to study habits or to clinical experience. Success in the final examination was also related to a strategic or deep learning style in the final year (although not at time of entry to medical school). CONCLUSIONS: The lack of correlation between examination performance and clinical experience calls into question the validity of final examinations. How much knowledge is gained from clinical experience as a student is able to be predicted from measures of study habits made at the time of application to medical school, some six years earlier, although not from results of A level examinations. Medical schools wishing to select students who will gain the most knowledge from clinical experience cannot use the results of A level examinations alone but could assess a student's learning style.
BACKGROUND AND OBJECTIVES:
The inclusion of detailed basic science courses in medical school curricula has been a concern of students. The main objective of this study was to explore the attitudes of medical students towards basic sciences courses taught to them in the preclinical years and the applicability of these courses to current clinical practice.
DESIGN AND SETTING:
A cross-sectional survey was conducted during 2008-2009 among medical students in their clinical years at King Saud University, Riyadh, Saudi Arabia.
Thirty percent of all students (n=314) were randomly selected to receive a questionnaire designed to evaluate their opinions about course load, ability to recall information, value of practical sessions, availability of references and course guidelines, and the applicability of individual courses to clinical practice.
Students identified anatomy and pathology as the courses most overloaded with content (76% and 70%, respectively). Half of the students felt they retained the most knowledge of physiology (50%), while less than a quarter of students (19%) felt they retained the most information from biochemistry coursework. The role of practical sessions in facilitating theoretical understanding was more evident in anatomy (69%). Physiology was perceived as the subject with the highest applicability to clinical practice (66%), while pathology (29%) was identified as the subject with the least practical application. Students became increasingly negative in their opinions about basic science courses as they progressed through their medical education.
Current attitudes of medical students towards their basic science courses indicate a need to reform the curricula so as to maximize the benefit of these courses.