Search tips
Search criteria

Results 1-25 (1374515)

Clipboard (0)

Related Articles

1.  The UK clinical aptitude test and clinical course performance at Nottingham: a prospective cohort study 
BMC Medical Education  2013;13:32.
The UK Clinical Aptitude Test (UKCAT) was introduced in 2006 as an additional tool for the selection of medical students. It tests mental ability in four distinct domains (Verbal Reasoning, Quantitative Reasoning, Abstract Reasoning, and Decision Analysis), and the results are available to students and admission panels in advance of the selection process. Our first study showed little evidence of any predictive validity for performance in the first two years of the Nottingham undergraduate course.
The study objective was to determine whether the UKCAT scores had any predictive value for the later parts of the course, largely delivered via clinical placements.
Students entering the course in 2007 and who had taken the UKCAT were asked for permission to use their anonymised data in research. The UKCAT scores were incorporated into a database with routine pre-admission socio-demographics and subsequent course performance data. Correlation analysis was followed by hierarchical multivariate linear regression.
The original study group comprised 204/254 (80%) of the full entry cohort. With attrition over the five years of the course this fell to 185 (73%) by Year 5. The Verbal Reasoning score and the UKCAT Total score both demonstrated some univariate correlations with clinical knowledge marks, and slightly less with clinical skills. No parts of the UKCAT proved to be an independent predictor of clinical course marks, whereas prior attainment was a highly significant predictor (p <0.001).
This study of one cohort of Nottingham medical students showed that UKCAT scores at admission did not independently predict subsequent performance on the course. Whilst the test adds another dimension to the selection process, its fairness and validity in selecting promising students remains unproven, and requires wider investigation and debate by other schools.
PMCID: PMC3621812  PMID: 23442227
2.  The UKCAT-12 study: educational attainment, aptitude test performance, demographic and socio-economic contextual factors as predictors of first year outcome in a cross-sectional collaborative study of 12 UK medical schools 
BMC Medicine  2013;11:244.
Most UK medical schools use aptitude tests during student selection, but large-scale studies of predictive validity are rare. This study assesses the United Kingdom Clinical Aptitude Test (UKCAT), and its four sub-scales, along with measures of educational attainment, individual and contextual socio-economic background factors, as predictors of performance in the first year of medical school training.
A prospective study of 4,811 students in 12 UK medical schools taking the UKCAT from 2006 to 2008 as a part of the medical school application, for whom first year medical school examination results were available in 2008 to 2010.
UKCAT scores and educational attainment measures (General Certificate of Education (GCE): A-levels, and so on; or Scottish Qualifications Authority (SQA): Scottish Highers, and so on) were significant predictors of outcome. UKCAT predicted outcome better in female students than male students, and better in mature than non-mature students. Incremental validity of UKCAT taking educational attainment into account was significant, but small. Medical school performance was also affected by sex (male students performing less well), ethnicity (non-White students performing less well), and a contextual measure of secondary schooling, students from secondary schools with greater average attainment at A-level (irrespective of public or private sector) performing less well. Multilevel modeling showed no differences between medical schools in predictive ability of the various measures. UKCAT sub-scales predicted similarly, except that Verbal Reasoning correlated positively with performance on Theory examinations, but negatively with Skills assessments.
This collaborative study in 12 medical schools shows the power of large-scale studies of medical education for answering previously unanswerable but important questions about medical student selection, education and training. UKCAT has predictive validity as a predictor of medical school outcome, particularly in mature applicants to medical school. UKCAT offers small but significant incremental validity which is operationally valuable where medical schools are making selection decisions based on incomplete measures of educational attainment. The study confirms the validity of using all the existing measures of educational attainment in full at the time of selection decision-making. Contextual measures provide little additional predictive value, except that students from high attaining secondary schools perform less well, an effect previously shown for UK universities in general.
PMCID: PMC3827332  PMID: 24229380
Medical student selection; Educational attainment; Aptitude tests; UKCAT; Socio-economic factors; Contextual measures
3.  Predictive power of UKCAT and other pre-admission measures for performance in a medical school in Glasgow: a cohort study 
BMC Medical Education  2014;14:116.
The UK Clinical Aptitude Test (UKCAT) and its four subtests are currently used by 24 Medical and Dental Schools in the UK for admissions. This longitudinal study examines the predictive validity of UKCAT for final performance in the undergraduate medical degree programme at one Medical School and compares this with the predictive validity of the selection measures available pre-UKCAT.
This was a retrospective observational study of one cohort of students, admitted to Glasgow Medical School in 2007. We examined the associations which UKCAT scores, school science grades and pre-admissions interview scores had with performance indicators, particularly final composite scores that determine students’ postgraduate training opportunities and overall ranking (Educational Performance Measure - EPM, and Honours and Commendation – H&C). Analyses were conducted both with and without adjustment for potential socio-demographic confounders (gender, age, ethnicity and area deprivation).
Despite its predictive value declining as students progress through the course, UKCAT was associated with the final composite scores. In mutually adjusted analyses (also adjusted for socio-demographic confounders), only UKCAT total showed independent relationships with both EPM (p = 0.005) and H&C (p = 0.004), school science achievements predicted EPM (p = 0.009), and pre-admissions interview score predicted neither. UKCAT showed less socio-demographic variation than did TSS.
UKCAT has a modest predictive power for overall course performance at the University of Glasgow Medical School over and above that of school science achievements or pre-admission interview score and we conclude that UKCAT is the most useful predictor of final ranking.
PMCID: PMC4063234  PMID: 24919950
UKCAT; Predictive validity; Widening participation; Socio-economic indicators; Admissions interview; School HE participation rate
4.  Predictive validity of the UK clinical aptitude test in the final years of medical school: a prospective cohort study 
BMC Medical Education  2014;14:88.
The UK Clinical Aptitude Test (UKCAT) was designed to address issues identified with traditional methods of selection. This study aims to examine the predictive validity of the UKCAT and compare this to traditional selection methods in the senior years of medical school. This was a follow-up study of two cohorts of students from two medical schools who had previously taken part in a study examining the predictive validity of the UKCAT in first year.
The sample consisted of 4th and 5th Year students who commenced their studies at the University of Aberdeen or University of Dundee medical schools in 2007. Data collected were: demographics (gender and age group), UKCAT scores; Universities and Colleges Admissions Service (UCAS) form scores; admission interview scores; Year 4 and 5 degree examination scores. Pearson’s correlations were used to examine the relationships between admissions variables, examination scores, gender and age group, and to select variables for multiple linear regression analysis to predict examination scores.
Ninety-nine and 89 students at Aberdeen medical school from Years 4 and 5 respectively, and 51 Year 4 students in Dundee, were included in the analysis. Neither UCAS form nor interview scores were statistically significant predictors of examination performance. Conversely, the UKCAT yielded statistically significant validity coefficients between .24 and .36 in four of five assessments investigated. Multiple regression analysis showed the UKCAT made a statistically significant unique contribution to variance in examination performance in the senior years.
Results suggest the UKCAT appears to predict performance better in the later years of medical school compared to earlier years and provides modest supportive evidence for the UKCAT’s role in student selection within these institutions. Further research is needed to assess the predictive validity of the UKCAT against professional and behavioural outcomes as the cohort commences working life.
PMCID: PMC4008381  PMID: 24762134
UKCAT; Predictive validity; Psychometric; Assessment; Selection; Admissions; Aptitude
5.  Comparison of A level and UKCAT performance in students applying to UK medical and dental schools in 2006: cohort study 
Objectives To determine whether the UK Clinical Aptitude Test (UKCAT) adds value to the selection process for school leaver applicants to medical and dental school, and in particular whether UKCAT can reduce the socioeconomic bias known to affect A levels.
Design Cohort study
Setting Applicants to 23 UK medical and dental schools in 2006.
Participants 9884 applicants who took the UKCAT in the UK and who achieved at least three passes at A level in their school leaving examinations (53% of all applicants).
Main outcome measures Independent predictors of obtaining at least AAB at A level and
UKCAT scores at or above the 30th centile for the cohort, for the subsections and the entire test.
Results Independent predictors of obtaining at least AAB at A level were white ethnicity (odds ratio 1.58, 95% confidence interval 1.41 to 1.77), professional or managerial background (1.39, 1.22 to 1.59), and independent or grammar schooling (2.26, 2.02 to 2.52) (all P<0.001). Independent predictors of achieving UKCAT scores at or above the 30th centile for the whole test were male sex (odd ratio 1.48, 1.32 to 1.66), white ethnicity (2.17, 1.94 to 2.43), professional or managerial background (1.34, 1.17 to 1.54), and independent or grammar schooling (1.91, 1.70 to 2.14) (all P<0.001). One major limitation of the study was that socioeconomic status was not volunteered by approximately 30% of the applicants. Those who withheld socioeconomic status data were significantly different from those who provided that information, which may have caused bias in the analysis.
Conclusions UKCAT was introduced with a high expectation of increasing the diversity and fairness in selection for UK medical and dental schools. This study of a major subgroup of applicants in the first year of operation suggests that it has an inherent favourable bias to men and students from a higher socioeconomic class or independent or grammar schools. However, it does provide a reasonable proxy for A levels in the selection process.
PMCID: PMC2824099  PMID: 20160316
6.  Can personal qualities of medical students predict in-course examination success and professional behaviour? An exploratory prospective cohort study 
BMC Medical Education  2012;12:69.
Over two-thirds of UK medical schools are augmenting their selection procedures for medical students by using the United Kingdom Clinical Aptitude Test (UKCAT), which employs tests of cognitive and non-cognitive personal qualities, but clear evidence of the tests’ predictive validity is lacking. This study explores whether academic performance and professional behaviours that are important in a health professional context can be predicted by these measures, when taken before or very early in the medical course.
This prospective cohort study follows the progress of the entire student cohort who entered Hull York Medical School in September 2007, having taken the UKCAT cognitive tests in 2006 and the non-cognitive tests a year later. This paper reports on the students’ first and second academic years of study. The main outcome measures were regular, repeated tutor assessment of individual students’ interpersonal skills and professional behaviour, and annual examination performance in the three domains of recall and application of knowledge, evaluation of data, and communication and practical clinical skills. The relationships between non-cognitive test scores, cognitive test scores, tutor assessments and examination results were explored using the Pearson product–moment correlations for each group of data; the data for students obtaining the top and bottom 20% of the summative examination results were compared using Analysis of Variance.
Personal qualities measured by non-cognitive tests showed a number of statistically significant relationships with ratings of behaviour made by tutors, with performance in each year’s objective structured clinical examinations (OSCEs), and with themed written summative examination marks in each year. Cognitive ability scores were also significantly related to each year’s examination results, but seldom to professional behaviours. The top 20% of examination achievers could be differentiated from the bottom 20% on both non-cognitive and cognitive measures.
This study shows numerous significant relationships between both cognitive and non-cognitive test scores, academic examination scores and indicators of professional behaviours in medical students. This suggests that measurement of non-cognitive personal qualities in applicants to medical school could make a useful contribution to selection and admission decisions. Further research is required in larger representative groups, and with more refined predictor measures and behavioural assessment methods, to establish beyond doubt the incremental validity of such measures over conventional cognitive assessments.
PMCID: PMC3473297  PMID: 22873571
7.  Widening access to UK medical education for under-represented socioeconomic groups: modelling the impact of the UKCAT in the 2009 cohort 
Objective To determine whether the use of the UK clinical aptitude test (UKCAT) in the medical schools admissions process reduces the relative disadvantage encountered by certain sociodemographic groups.
Design Prospective cohort study.
Setting Applicants to 22 UK medical schools in 2009 that were members of the consortium of institutions utilising the UKCAT as a component of their admissions process.
Participants 8459 applicants (24 844 applications) to UKCAT consortium member medical schools where data were available on advanced qualifications and socioeconomic background.
Main outcome measures The probability of an application resulting in an offer of a place on a medicine course according to seven educational and sociodemographic variables depending on how the UKCAT was used by the medical school (in borderline cases, as a factor in admissions, or as a threshold).
Results On univariate analysis all educational and sociodemographic variables were significantly associated with the relative odds of an application being successful. The multilevel multiple logistic regression models, however, varied between medical schools according to the way that the UKCAT was used. For example, a candidate from a non-professional background was much less likely to receive a conditional offer of a place compared with an applicant from a higher social class when applying to an institution using the test only in borderline cases (odds ratio 0.51, 95% confidence interval 0.45 to 0.60). No such effect was observed for such candidates applying to medical schools using the threshold approach (1.27, 0.84 to 1.91). These differences were generally reflected in the interactions observed when the analysis was repeated, pooling the data. Notably, candidates from several under-represented groups applying to medical schools that used a threshold approach to the UKCAT were less disadvantaged than those applying to the other institutions in the consortium. These effects were partially reflected in significant differences in the absolute proportion of such candidates finally taking up places in the different types of medical schools; stronger use of the test score (as a factor or threshold) was associated with a significantly increased odds of entrants being male (1.74, 1.25 to 2.41) and from a low socioeconomic background (3.57, 1.03 to 12.39). There was a non-significant trend towards entrants being from a state (non-grammar) school (1.60, 0.97 to 2.62) where a stronger use of the test was employed. Use of the test only in borderline cases was associated with increased odds of entrants having relatively low academic attainment (5.19, 2.02 to 13.33) and English as a second language (2.15, 1.03 to 4.48).
Conclusions The use of the UKCAT may lead to more equitable provision of offers to those applying to medical school from under-represented sociodemographic groups. This may translate into higher numbers of some, but not all, relatively disadvantaged students entering the UK medical profession.
PMCID: PMC3328544  PMID: 22511300
8.  Comparison of the sensitivity of the UKCAT and A Levels to sociodemographic characteristics: a national study 
The UK Clinical Aptitude Test (UKCAT) was introduced to facilitate widening participation in medical and dental education in the UK by providing universities with a continuous variable to aid selection; one that might be less sensitive to the sociodemographic background of candidates compared to traditional measures of educational attainment. Initial research suggested that males, candidates from more advantaged socioeconomic backgrounds and those who attended independent or grammar schools performed better on the test. The introduction of the A* grade at A level permits more detailed analysis of the relationship between UKCAT scores, secondary educational attainment and sociodemographic variables. Thus, our aim was to further assess whether the UKCAT is likely to add incremental value over A level (predicted or actual) attainment in the selection process.
Data relating to UKCAT and A level performance from 8,180 candidates applying to medicine in 2009 who had complete information relating to six key sociodemographic variables were analysed. A series of regression analyses were conducted in order to evaluate the ability of sociodemographic status to predict performance on two outcome measures: A level ‘best of three’ tariff score; and the UKCAT scores.
In this sample A level attainment was independently and positively predicted by four sociodemographic variables (independent/grammar schooling, White ethnicity, age and professional social class background). These variables also independently and positively predicted UKCAT scores. There was a suggestion that UKCAT scores were less sensitive to educational background compared to A level attainment. In contrast to A level attainment, UKCAT score was independently and positively predicted by having English as a first language and male sex.
Our findings are consistent with a previous report; most of the sociodemographic factors that predict A level attainment also predict UKCAT performance. However, compared to A levels, males and those speaking English as a first language perform better on UKCAT. Our findings suggest that UKCAT scores may be more influenced by sex and less sensitive to school type compared to A levels. These factors must be considered by institutions utilising the UKCAT as a component of the medical and dental school selection process.
PMCID: PMC3893425  PMID: 24400861
Medical student selection; Educational attainment; Aptitude tests; UKCAT; Socio-economic factors
9.  Construct-level predictive validity of educational attainment and intellectual aptitude tests in medical student selection: meta-regression of six UK longitudinal studies 
BMC Medicine  2013;11:243.
Measures used for medical student selection should predict future performance during training. A problem for any selection study is that predictor-outcome correlations are known only in those who have been selected, whereas selectors need to know how measures would predict in the entire pool of applicants. That problem of interpretation can be solved by calculating construct-level predictive validity, an estimate of true predictor-outcome correlation across the range of applicant abilities.
Construct-level predictive validities were calculated in six cohort studies of medical student selection and training (student entry, 1972 to 2009) for a range of predictors, including A-levels, General Certificates of Secondary Education (GCSEs)/O-levels, and aptitude tests (AH5 and UK Clinical Aptitude Test (UKCAT)). Outcomes included undergraduate basic medical science and finals assessments, as well as postgraduate measures of Membership of the Royal Colleges of Physicians of the United Kingdom (MRCP(UK)) performance and entry in the Specialist Register. Construct-level predictive validity was calculated with the method of Hunter, Schmidt and Le (2006), adapted to correct for right-censorship of examination results due to grade inflation.
Meta-regression analyzed 57 separate predictor-outcome correlations (POCs) and construct-level predictive validities (CLPVs). Mean CLPVs are substantially higher (.450) than mean POCs (.171). Mean CLPVs for first-year examinations, were high for A-levels (.809; CI: .501 to .935), and lower for GCSEs/O-levels (.332; CI: .024 to .583) and UKCAT (mean = .245; CI: .207 to .276). A-levels had higher CLPVs for all undergraduate and postgraduate assessments than did GCSEs/O-levels and intellectual aptitude tests. CLPVs of educational attainment measures decline somewhat during training, but continue to predict postgraduate performance. Intellectual aptitude tests have lower CLPVs than A-levels or GCSEs/O-levels.
Educational attainment has strong CLPVs for undergraduate and postgraduate performance, accounting for perhaps 65% of true variance in first year performance. Such CLPVs justify the use of educational attainment measure in selection, but also raise a key theoretical question concerning the remaining 35% of variance (and measurement error, range restriction and right-censorship have been taken into account). Just as in astrophysics, ‘dark matter’ and ‘dark energy’ are posited to balance various theoretical equations, so medical student selection must also have its ‘dark variance’, whose nature is not yet properly characterized, but explains a third of the variation in performance during training. Some variance probably relates to factors which are unpredictable at selection, such as illness or other life events, but some is probably also associated with factors such as personality, motivation or study skills.
PMCID: PMC3827328  PMID: 24229353
Medical student selection; Undergraduate performance; Postgraduate performance; Educational attainment; Aptitude tests; Criterion-related construct validity; Range restriction; Right censorship; Grade inflation; Markov Chain Monte Carlo algorithm
10.  Use of UKCAT scores in student selection by UK medical schools, 2006-2010 
BMC Medical Education  2011;11:98.
The United Kingdom Clinical Aptitude Test (UKCAT) is a set of cognitive tests introduced in 2006, taken annually before application to medical school. The UKCAT is a test of aptitude and not acquired knowledge and as such the results give medical schools a standardised and objective tool that all schools could use to assist their decision making in selection, and so provide a fairer means of choosing future medical students.
Selection of students for UK medical schools is usually in three stages: assessment of academic qualifications, assessment of further qualities from the application form submitted via UCAS (Universities and Colleges Admissions Service) leading to invitation to interview, and then selection for offer of a place. Medical schools were informed of the psychometric qualities of the UKCAT subtests and given some guidance regarding the interpretation of results. Each school then decided how to use the results within its own selection system.
Annual retrospective key informant telephone interviews were conducted with every UKCAT Consortium medical school, using a pre-circulated structured questionnaire. The key points of the interview were transcribed, 'member checked' and a content analysis was undertaken.
Four equally popular ways of using the test results have emerged, described as Borderline, Factor, Threshold and Rescue methods. Many schools use more than one method, at different stages in their selection process. Schools have used the scores in ways that have sought to improve the fairness of selection and support widening participation. Initially great care was taken not to exclude any applicant on the basis of low UKCAT scores alone but it has been used more as confidence has grown.
There is considerable variation in how medical schools use UKCAT, so it is important that they clearly inform applicants how the test will be used so they can make best use of their limited number of applications.
PMCID: PMC3248371  PMID: 22114935
11.  Should applicants to Nottingham University Medical School study a non-science A-level? A cohort study 
It has been suggested that studying non-science subjects at A-level should be compulsory for medical students. Our admissions criteria specify only Biology, Chemistry and one or more additional subjects. This study aimed to determine whether studying a non-science subject for A-level is an independent predictor of achievement on the undergraduate medical course.
The subjects of this retrospective cohort study were 164 students from one entry-year group (October 2000), who progressed normally on the 5-year undergraduate medical course at Nottingham. Pre-admission academic and socio-demographic data and undergraduate course marks were obtained. T-test and hierarchical multiple linear regression analyses were undertaken to identify independent predictors of five course outcomes at different stages throughout the course.
There was no evidence that the choice of science or non-science as the third or fourth A-level subject had any influence on course performance. Demographic variables (age group, sex, and fee status) had some predictive value but ethnicity did not. Pre-clinical course performance was the strongest predictor in the clinical phases (pre-clinical Themes A&B (knowledge) predicted Clinical Knowledge, p < 0.001, and pre-clinical Themes C&D (skills) predicted Clinical Skills, p = < 0.01).
This study of one year group at Nottingham Medical School provided no evidence that the admissions policy on A-level requirements should specify the choice of third or fourth subject.
PMCID: PMC2650695  PMID: 19159444
12.  Predicting academic outcomes in an Australian graduate entry medical programme 
BMC Medical Education  2014;14:31.
Predictive validity studies for selection criteria into graduate entry courses in Australia have been inconsistent in their outcomes. One of the reasons for this inconsistency may have been failure to have adequately considered background disciplines of the graduates as well as other potential confounding socio-demographic variables that may influence academic performance.
Graduate entrants into the MBBS at The University of Western Australia between 2005 and 2012 were studied (N = 421). They undertook a 6-month bridging course, before joining the undergraduate-entry students for Years 3 through 6 of the medical course. Students were selected using their undergraduate Grade Point Average (GPA), Graduate Australian Medical School Admissions Test scores (GAMSAT) and a score from a standardised interview. Students could apply from any background discipline and could also be selected through an alternative rural entry pathway again utilising these 3 entry scores. Entry scores, together with age, gender, discipline background, rural entry status and a socioeconomic indicator were entered into linear regression models to determine the relative influence of each predictor on subsequent academic performance in the course.
Background discipline, age, gender and selection through the rural pathway were variously related to each of the 3 entry criteria. Their subsequent inclusion in linear regression models identified GPA at entry, being from a health/allied health background and total GAMSAT score as consistent independent predictors of stronger academic performance as measured by the weighted average mark for the core units completed throughout the course. The Interview score only weakly predicted performance later in the course and mainly in clinically-based units. The association of total GAMSAT score with academic performance was predominantly dictated by the score in GAMSAT Section 3 (Reasoning in the biological and physical sciences) with Section 1 (Reasoning in the humanities and social sciences) and Section 2 (Written communication) also contributing either later or early in the course respectively. Being from a more disadvantaged socioeconomic background predicted weaker academic performance early in the course. Being an older student at entry or from a humanities background also predicted weaker academic performance.
This study confirms that both GPA at entry and the GAMSAT score together predict outcomes not only in the early stages of a graduate-entry medical programme but throughout the course. It also indicates that a comprehensive evaluation of the predictive validity of GAMSAT scores, interview scores and undergraduate academic performance as valid selection processes for graduate entry into medical school needs to simultaneously consider the potential confounding influence of graduate discipline background and other socio-demographic factors on both the initial selection parameters themselves as well as subsequent academic performance.
PMCID: PMC3931285  PMID: 24528509
13.  To what extent does the Health Professions Admission Test-Ireland predict performance in early undergraduate tests of communication and clinical skills? – An observational cohort study 
BMC Medical Education  2013;13:68.
Internationally, tests of general mental ability are used in the selection of medical students. Examples include the Medical College Admission Test, Undergraduate Medicine and Health Sciences Admission Test and the UK Clinical Aptitude Test. The most widely used measure of their efficacy is predictive validity.
A new tool, the Health Professions Admission Test- Ireland (HPAT-Ireland), was introduced in 2009. Traditionally, selection to Irish undergraduate medical schools relied on academic achievement. Since 2009, Irish and EU applicants are selected on a combination of their secondary school academic record (measured predominately by the Leaving Certificate Examination) and HPAT-Ireland score. This is the first study to report on the predictive validity of the HPAT-Ireland for early undergraduate assessments of communication and clinical skills.
Students enrolled at two Irish medical schools in 2009 were followed up for two years. Data collected were gender, HPAT-Ireland total and subsection scores; Leaving Certificate Examination plus HPAT-Ireland combined score, Year 1 Objective Structured Clinical Examination (OSCE) scores (Total score, communication and clinical subtest scores), Year 1 Multiple Choice Questions and Year 2 OSCE and subset scores. We report descriptive statistics, Pearson correlation coefficients and Multiple linear regression models.
Data were available for 312 students. In Year 1 none of the selection criteria were significantly related to student OSCE performance. The Leaving Certificate Examination and Leaving Certificate plus HPAT-Ireland combined scores correlated with MCQ marks.
In Year 2 a series of significant correlations emerged between the HPAT-Ireland and subsections thereof with OSCE Communication Z-scores; OSCE Clinical Z-scores; and Total OSCE Z-scores. However on multiple regression only the relationship between Total OSCE Score and the Total HPAT-Ireland score remained significant; albeit the predictive power was modest.
We found that none of our selection criteria strongly predict clinical and communication skills. The HPAT- Ireland appears to measures ability in domains different to those assessed by the Leaving Certificate Examination. While some significant associations did emerge in Year 2 between HPAT Ireland and total OSCE scores further evaluation is required to establish if this pattern continues during the senior years of the medical course.
PMCID: PMC3667098  PMID: 23663266
Selection; Medical; Student; Validity; Predictive; HPAT-Ireland; Assessment; Cognitive; Ability
14.  Admission selection criteria as predictors of outcomes in an undergraduate medical course: A prospective study 
Medical Teacher  2011;33(12):997-1004.
In 1998, a new selection process which utilised an aptitude test and an interview in addition to previous academic achievement was introduced into an Australian undergraduate medical course.
To test the outcomes of the selection criteria over an 11-year period.
1174 students who entered the course from secondary school and who enrolled in the MBBS from 1999 through 2009 were studied in relation to specific course outcomes. Regression analyses using entry scores, sex and age as independent variables were tested for their relative value in predicting subsequent academic performance in the 6-year course. The main outcome measures were assessed by weighted average mark for each academic year level; together with results in specific units, defined as either ‘knowledge'-based or ‘clinically’ based.
Previous academic performance and female sex were the major independent positive predictors of performance in the course. The interview score showed positive predictive power during the latter years of the course and in a range of ‘clinically' based units. This relationship was mediated predominantly by the score for communication skills.
Results support combining prior academic achievement with the assessment of communication skills in a structured interview as selection criteria into this undergraduate medical course.
PMCID: PMC3267525  PMID: 21592024
15.  Medical Students' Exposure to and Attitudes about the Pharmaceutical Industry: A Systematic Review 
PLoS Medicine  2011;8(5):e1001037.
A systematic review of published studies reveals that undergraduate medical students may experience substantial exposure to pharmaceutical marketing, and that this contact may be associated with positive attitudes about marketing.
The relationship between health professionals and the pharmaceutical industry has become a source of controversy. Physicians' attitudes towards the industry can form early in their careers, but little is known about this key stage of development.
Methods and Findings
We performed a systematic review reported according to PRISMA guidelines to determine the frequency and nature of medical students' exposure to the drug industry, as well as students' attitudes concerning pharmaceutical policy issues. We searched MEDLINE, EMBASE, Web of Science, and ERIC from the earliest available dates through May 2010, as well as bibliographies of selected studies. We sought original studies that reported quantitative or qualitative data about medical students' exposure to pharmaceutical marketing, their attitudes about marketing practices, relationships with industry, and related pharmaceutical policy issues. Studies were separated, where possible, into those that addressed preclinical versus clinical training, and were quality rated using a standard methodology. Thirty-two studies met inclusion criteria. We found that 40%–100% of medical students reported interacting with the pharmaceutical industry. A substantial proportion of students (13%–69%) were reported as believing that gifts from industry influence prescribing. Eight studies reported a correlation between frequency of contact and favorable attitudes toward industry interactions. Students were more approving of gifts to physicians or medical students than to government officials. Certain attitudes appeared to change during medical school, though a time trend was not performed; for example, clinical students (53%–71%) were more likely than preclinical students (29%–62%) to report that promotional information helps educate about new drugs.
Undergraduate medical education provides substantial contact with pharmaceutical marketing, and the extent of such contact is associated with positive attitudes about marketing and skepticism about negative implications of these interactions. These results support future research into the association between exposure and attitudes, as well as any modifiable factors that contribute to attitudinal changes during medical education.
Please see later in the article for the Editors' Summary
Editors' Summary
The complex relationship between health professionals and the pharmaceutical industry has long been a subject of discussion among physicians and policymakers. There is a growing body of evidence that suggests that physicians' interactions with pharmaceutical sales representatives may influence clinical decision making in a way that is not always in the best interests of individual patients, for example, encouraging the use of expensive treatments that have no therapeutic advantage over less costly alternatives. The pharmaceutical industry often uses physician education as a marketing tool, as in the case of Continuing Medical Education courses that are designed to drive prescribing practices.
One reason that physicians may be particularly susceptible to pharmaceutical industry marketing messages is that doctors' attitudes towards the pharmaceutical industry may form early in their careers. The socialization effect of professional schooling is strong, and plays a lasting role in shaping views and behaviors.
Why Was This Study Done?
Recently, particularly in the US, some medical schools have limited students' and faculties' contact with industry, but some have argued that these restrictions are detrimental to students' education. Given the controversy over the pharmaceutical industry's role in undergraduate medical training, consolidating current knowledge in this area may be useful for setting priorities for changes to educational practices. In this study, the researchers systematically examined studies of pharmaceutical industry interactions with medical students and whether such interactions influenced students' views on related topics.
What Did the Researchers Do and Find?
The researchers did a comprehensive literature search using appropriate search terms for all relevant quantitative and qualitative studies published before June 2010. Using strict inclusion criteria, the researchers then selected 48 articles (from 1,603 abstracts) for full review and identified 32 eligible for analysis—giving a total of approximately 9,850 medical students studying at 76 medical schools or hospitals.
Most students had some form of interaction with the pharmaceutical industry but contact increased in the clinical years, with up to 90% of all clinical students receiving some form of educational material. The highest level of exposure occurred in the US. In most studies, the majority of students in their clinical training years found it ethically permissible for medical students to accept gifts from drug manufacturers, while a smaller percentage of preclinical students reported such attitudes. Students justified their entitlement to gifts by citing financial hardship or by asserting that most other students accepted gifts. In addition, although most students believed that education from industry sources is biased, students variably reported that information obtained from industry sources was useful and a valuable part of their education.
Almost two-thirds of students reported that they were immune to bias induced by promotion, gifts, or interactions with sales representatives but also reported that fellow medical students or doctors are influenced by such encounters. Eight studies reported a relationship between exposure to the pharmaceutical industry and positive attitudes about industry interactions and marketing strategies (although not all included supportive statistical data). Finally, student opinions were split on whether physician–industry interactions should be regulated by medical schools or the government.
What Do These Findings Mean?
This analysis shows that students are frequently exposed to pharmaceutical marketing, even in the preclinical years, and that the extent of students' contact with industry is generally associated with positive attitudes about marketing and skepticism towards any negative implications of interactions with industry. Therefore, strategies to educate students about interactions with the pharmaceutical industry should directly address widely held misconceptions about the effects of marketing and other biases that can emerge from industry interactions. But education alone may be insufficient. Institutional policies, such as rules regulating industry interactions, can play an important role in shaping students' attitudes, and interventions that decrease students' contact with industry and eliminate gifts may have a positive effect on building the skills that evidence-based medical practice requires. These changes can help cultivate strong professional values and instill in students a respect for scientific principles and critical evidence review that will later inform clinical decision-making and prescribing practices.
Additional Information
Please access these Web sites via the online version of this summary at
Further information about the influence of the pharmaceutical industry on doctors and medical students can be found at the American Medical Students Association PharmFree campaign and PharmFree Scorecard, Medsin-UKs PharmAware campaign, the nonprofit organization Healthy Skepticism, and the Web site of No Free Lunch.
PMCID: PMC3101205  PMID: 21629685
16.  Predicting the “strugglers”: a case-control study of students at Nottingham University Medical School 
BMJ : British Medical Journal  2006;332(7548):1009-1013.
Objective To identify potential predictors of undergraduate students who struggle during their medical training.
Design Case-control study. Cases were students who had experienced academic or personal difficulties that affected their progression on the course (“strugglers”). Controls were selected at random from the corresponding year cohorts, using a ratio of four controls for each struggler.
Setting University of Nottingham Medical School.
Participants Students who entered the course over five consecutive years.
Main outcome measures Likelihood ratios for independent risk factors for struggling on the course
Results 10-15% of each year's student intake were identified as strugglers. Significant independent predictors of students being in this category were negative comments in the academic reference (likelihood ratio 2.25, 95% confidence intervals 1.44 to 3.50), lower mean examination grade at A level (2.19, 1.37 to 3.51), and the late offer of a place (1.98, 1.19 to 3.30). Male sex was a less significant risk factor (1.70, 1.09 to 2.65) as was a lower grade at GCSE science (2.13, 1.12 to 4.05). In UK students whose ethnicity was known, not being white was a significant predictor of struggling (2.77, 1.52 to 5.05) but the presence of negative comments was not. Age at entry to the course and the possession of a previous degree were not predictive.
Conclusions Our results support retention of existing selection practices relating to academic achievement and critical review of students' references. We plan to undertake further investigation of the reasons why some students, including men, those with late offers and those from ethnic minority backgrounds, may do less well on the Nottingham course.
PMCID: PMC1450046  PMID: 16543299
17.  Pilot study of the roles of personality, references, and personal statements in relation to performance over the five years of a medical degree 
BMJ : British Medical Journal  2003;326(7386):429-432.
To compare the power of three traditional selection procedures (A levels, personal statements, and references) and one non-traditional selection procedure (personality) to predict performance over the five years of a medical degree.
Cohort study over five years.
Nottingham medical school.
Entrants in 1995.
Main outcome measures
A level grades, amounts of information contained in teacher's reference and the student's personal statement, and personality scores examined in relation to 18 different assessments.
Information in the teacher's reference did not consistently predict performance. Information in the personal statement was predictive of clinical aspects of training, whereas A level grades primarily predicted preclinical performance. The personality domain of conscientiousness was consistently the best predictor across the course. A structural model indicated that conscientiousness was positively related to A level grades and preclinical performance but was negatively related to clinical grades.
A teacher's reference is of no practical use in predicting clinical performance of medical students, in contrast to the amount of information contained in the personal statement. Therefore, simple quantification of the personal statement should aid selection. Personality factors, in particular conscientiousness, need to be considered and integrated into selection procedures.
PMCID: PMC163931  PMID: 12595384
18.  The Academic Backbone: longitudinal continuities in educational achievement from secondary school and medical school to MRCP(UK) and the specialist register in UK medical students and doctors 
BMC Medicine  2013;11:242.
Selection of medical students in the UK is still largely based on prior academic achievement, although doubts have been expressed as to whether performance in earlier life is predictive of outcomes later in medical school or post-graduate education. This study analyses data from five longitudinal studies of UK medical students and doctors from the early 1970s until the early 2000s. Two of the studies used the AH5, a group test of general intelligence (that is, intellectual aptitude). Sex and ethnic differences were also analyzed in light of the changing demographics of medical students over the past decades.
Data from five cohort studies were available: the Westminster Study (began clinical studies from 1975 to 1982), the 1980, 1985, and 1990 cohort studies (entered medical school in 1981, 1986, and 1991), and the University College London Medical School (UCLMS) Cohort Study (entered clinical studies in 2005 and 2006). Different studies had different outcome measures, but most had performance on basic medical sciences and clinical examinations at medical school, performance in Membership of the Royal Colleges of Physicians (MRCP(UK)) examinations, and being on the General Medical Council Specialist Register.
Correlation matrices and path analyses are presented. There were robust correlations across different years at medical school, and medical school performance also predicted MRCP(UK) performance and being on the GMC Specialist Register. A-levels correlated somewhat less with undergraduate and post-graduate performance, but there was restriction of range in entrants. General Certificate of Secondary Education (GCSE)/O-level results also predicted undergraduate and post-graduate outcomes, but less so than did A-level results, but there may be incremental validity for clinical and post-graduate performance. The AH5 had some significant correlations with outcome, but they were inconsistent. Sex and ethnicity also had predictive effects on measures of educational attainment, undergraduate, and post-graduate performance. Women performed better in assessments but were less likely to be on the Specialist Register. Non-white participants generally underperformed in undergraduate and post-graduate assessments, but were equally likely to be on the Specialist Register. There was a suggestion of smaller ethnicity effects in earlier studies.
The existence of the Academic Backbone concept is strongly supported, with attainment at secondary school predicting performance in undergraduate and post-graduate medical assessments, and the effects spanning many years. The Academic Backbone is conceptualized in terms of the development of more sophisticated underlying structures of knowledge ('cognitive capital’ and 'medical capital’). The Academic Backbone provides strong support for using measures of educational attainment, particularly A-levels, in student selection.
PMCID: PMC3827330  PMID: 24229333
Academic Backbone; Secondary school attainment; Undergraduate medical education; Post-graduate medical education; Longitudinal analyses; Continuities; Medical student selection; Cognitive capital; Medical capital; Aptitude tests
19.  Selecting the right medical student 
BMC Medicine  2013;11:245.
Medical student selection is an important but difficult task. Three recent papers by McManus et al. in BMC Medicine have re-examined the role of tests of attainment of learning (A’ levels, GCSEs, SQA) and of aptitude (AH5, UKCAT), but on a much larger scale than previously attempted. They conclude that A’ levels are still the best predictor of future success at medical school and beyond. However, A’ levels account for only 65% of the variance in performance that is found. Therefore, more work is needed to establish relevant assessment of the other 35%.
Please see related research articles, and
PMCID: PMC3827327  PMID: 24229397
Medical School Admission; Predictors of performance; Aptitude testing
20.  Association of Medical Students' Reports of Interactions with the Pharmaceutical and Medical Device Industries and Medical School Policies and Characteristics: A Cross-Sectional Study 
PLoS Medicine  2014;11(10):e1001743.
Aaron Kesselheim and colleagues compared US medical students' survey responses regarding pharmaceutical company interactions with the schools' AMSA PharmFree scorecard and Institute on Medicine as a Profession's (IMAP) scores.
Please see later in the article for the Editors' Summary
Professional societies use metrics to evaluate medical schools' policies regarding interactions of students and faculty with the pharmaceutical and medical device industries. We compared these metrics and determined which US medical schools' industry interaction policies were associated with student behaviors.
Methods and Findings
Using survey responses from a national sample of 1,610 US medical students, we compared their reported industry interactions with their schools' American Medical Student Association (AMSA) PharmFree Scorecard and average Institute on Medicine as a Profession (IMAP) Conflicts of Interest Policy Database score. We used hierarchical logistic regression models to determine the association between policies and students' gift acceptance, interactions with marketing representatives, and perceived adequacy of faculty–industry separation. We adjusted for year in training, medical school size, and level of US National Institutes of Health (NIH) funding. We used LASSO regression models to identify specific policies associated with the outcomes. We found that IMAP and AMSA scores had similar median values (1.75 [interquartile range 1.50–2.00] versus 1.77 [1.50–2.18], adjusted to compare scores on the same scale). Scores on AMSA and IMAP shared policy dimensions were not closely correlated (gift policies, r = 0.28, 95% CI 0.11–0.44; marketing representative access policies, r = 0.51, 95% CI 0.36–0.63). Students from schools with the most stringent industry interaction policies were less likely to report receiving gifts (AMSA score, odds ratio [OR]: 0.37, 95% CI 0.19–0.72; IMAP score, OR 0.45, 95% CI 0.19–1.04) and less likely to interact with marketing representatives (AMSA score, OR 0.33, 95% CI 0.15–0.69; IMAP score, OR 0.37, 95% CI 0.14–0.95) than students from schools with the lowest ranked policy scores. The association became nonsignificant when fully adjusted for NIH funding level, whereas adjusting for year of education, size of school, and publicly versus privately funded school did not alter the association. Policies limiting gifts, meals, and speaking bureaus were associated with students reporting having not received gifts and having not interacted with marketing representatives. Policy dimensions reflecting the regulation of industry involvement in educational activities (e.g., continuing medical education, travel compensation, and scholarships) were associated with perceived separation between faculty and industry. The study is limited by potential for recall bias and the cross-sectional nature of the survey, as school curricula and industry interaction policies may have changed since the time of the survey administration and study analysis.
As medical schools review policies regulating medical students' industry interactions, limitations on receipt of gifts and meals and participation of faculty in speaking bureaus should be emphasized, and policy makers should pay greater attention to less research-intensive institutions.
Please see later in the article for the Editors' Summary
Editors' Summary
Making and selling prescription drugs and medical devices is big business. To promote their products, pharmaceutical and medical device companies build relationships with physicians by providing information on new drugs, by organizing educational meetings and sponsored events, and by giving gifts. Financial relationships begin early in physicians' careers, with companies providing textbooks and other gifts to first-year medical students. In medical school settings, manufacturers may help to inform trainees and physicians about developments in health care, but they also create the potential for harm to patients and health care systems. These interactions may, for example, reduce trainees' and trained physicians' skepticism about potentially misleading promotional claims and may encourage physicians to prescribe new medications, which are often more expensive than similar unbranded (generic) drugs and more likely to be recalled for safety reasons than older drugs. To address these and other concerns about the potential career-long effects of interactions between medical trainees and industry, many teaching hospitals and medical schools have introduced policies to limit such interactions. The development of these policies has been supported by expert professional groups and medical societies, some of which have created scales to evaluate the strength of the implemented industry interaction policies.
Why Was This Study Done?
The impact of policies designed to limit interactions between students and industry on student behavior is unclear, and it is not known which aspects of the policies are most predictive of student behavior. This information is needed to ensure that the policies are working and to identify ways to improve them. Here, the researchers investigate which medical school characteristics and which aspects of industry interaction policies are most predictive of students' reported behaviors and beliefs by comparing information collected in a national survey of US medical students with the strength of their schools' industry interaction policies measured on two scales—the American Medical Student Association (AMSA) PharmFree Scorecard and the Institute on Medicine as a Profession (IMAP) Conflicts of Interest Policy Database.
What Did the Researchers Do and Find?
The researchers compared information about reported gift acceptance, interactions with marketing representatives, and the perceived adequacy of faculty–industry separation collected from 1,610 medical students at 121 US medical schools with AMSA and IMAP scores for the schools evaluated a year earlier. Students at schools with the highest ranked interaction policies based on the AMSA score were 63% less likely to accept gifts as students at the lowest ranked schools. Students at the highest ranked schools based on the IMAP score were about half as likely to accept gifts as students at the lowest ranked schools, although this finding was not statistically significant (it could be a chance finding). Similarly, students at the highest ranked schools were 70% less likely to interact with sales representatives as students at the lowest ranked schools. These associations became statistically nonsignificant after controlling for the amount of research funding each school received from the US National Institutes of Health (NIH). Policies limiting gifts, meals, and being a part of speaking bureaus (where companies pay speakers to present information about the drugs for dinners and other events) were associated with students' reports of receiving no gifts and of non-interaction with sales representatives. Finally, policies regulating industry involvement in educational activities were associated with the perceived separation between faculty and industry, which was regarded as adequate by most of the students at schools with such policies.
What Do These Findings Mean?
These findings suggest that policies designed to limit industry interactions with medical students need to address multiple aspects of these interactions to achieve changes in the behavior and attitudes of trainees, but that policies limiting gifts, meals, and speaking bureaus may be particularly important. These findings also suggest that the level of NIH funding plays an important role in students' self-reported behaviors and their perceptions of industry, possibly because institutions with greater NIH funding have the resources needed to implement effective policies. The accuracy of these findings may be limited by recall bias (students may have reported their experiences inaccurately), and by the possibility that industry interaction policies may have changed in the year that elapsed between policy grading and the student survey. Nevertheless, these findings suggest that limitations on gifts should be emphasized when academic medical centers refine their policies on interactions between medical students and industry and that particular attention should be paid to the design and implementation of policies that regulate industry interactions in institutions with lower levels of NIH funding.
Additional Information
Please access these websites via the online version of this summary at
The UK General Medical Council provides guidance on financial and commercial arrangements and conflicts of interest as part of its good medical practice document, which describes what is required of all registered doctors in the UK
Information about the American Medical Student Association (AMSA) Just Medicine campaign (formerly the PharmFree campaign) and about the AMSA Scorecard is available
Information about the Institute on Medicine as a Profession (IMAP) and about its Conflicts of Interest Policy Database is also available
“Understanding and Responding to Pharmaceutical Promotion: A Practical Guide” is a manual prepared by Health Action International and the World Health Organization that medical schools can use to train students how to recognize and respond to pharmaceutical promotion
The US Institute of Medicine's report “Conflict of Interest in Medical Research, Education, and Practice” recommends steps to identify, limit, and manage conflicts of interest
The ALOSA Foundation provides evidence-based, non-industry-funded education about treating common conditions and using prescription drugs
PMCID: PMC4196737  PMID: 25314155
21.  Can the 12-item general health questionnaire be used to identify medical students who might ‘struggle’ on the medical course? A prospective study on two cohorts 
BMC Medical Education  2013;13:48.
Students who fail to thrive on the Nottingham undergraduate medical course frequently suffer from anxiety, depression or other mental health problems. These difficulties may be the cause, or the result of, academic struggling. Early detection of vulnerable students might direct pastoral care and remedial support to where it is needed. We investigated the use of the short-form General Health Questionnaire (GHQ-12) as a possible screening tool.
Two consecutive cohorts (2006 and 2007) were invited to complete the GHQ-12. The questionnaire was administered online, during the second semester (after semester 1 exams) for the 2006 cohort and during the first semester for the 2007 cohort. All data were held securely and confidentially. At the end of the course, GHQ scores were examined in relation to course progress.
251 students entered the course in 2006 and 254 in 2007; 164 (65%) and 160 (63%), respectively, completed the GHQ-12. In both cohorts, the study and non-study groups were very similar in terms of pre-admission socio-demographic characteristics and overall course marks. In the 2006 study group, the GHQ Likert score obtained part-way through the first year was negatively correlated with exam marks during Years 1 and 2, but the average exam mark in semester 1 was the sole independent predictor of marks in semester 2 and Year 2. No correlations were found for the 2007 study group but the GHQ score was a weak positive predictor of marks in semester 2, with semester 1 average exam mark again being the strongest predictor. A post-hoc moderated-mediation analysis suggested that significant negative associations of GHQ scores with semester 1 and 2 exams applied only to those who completed the GHQ after their semester 1 exams. Students who were identified as GHQ ‘cases’ in the 2006 group were statistically less likely to complete the course on time (OR = 4.74, p 0.002). There was a non-significant trend in the same direction in the 2007 group.
Results from two cohorts provide insufficient evidence to recommend the routine use of the GHQ-12 as a screening tool. The timing of administration could have a critical influence on the results, and the theoretical and practical implications of this finding are discussed. Low marks in semester 1 examinations seem be the best single indicator of students at risk for subsequent poor performance.
PMCID: PMC3616988  PMID: 23548161
22.  Same admissions tools, different outcomes: a critical perspective on predictive validity in three undergraduate medical schools 
BMC Medical Education  2013;13:173.
Admission to medical school is one of the most highly competitive entry points in higher education. Considerable investment is made by universities to develop selection processes that aim to identify the most appropriate candidates for their medical programs. This paper explores data from three undergraduate medical schools to offer a critical perspective of predictive validity in medical admissions.
This study examined 650 undergraduate medical students from three Australian universities as they progressed through the initial years of medical school (accounting for approximately 25 per cent of all commencing undergraduate medical students in Australia in 2006 and 2007). Admissions criteria (aptitude test score based on UMAT, school result and interview score) were correlated with GPA over four years of study. Standard regression of each of the three admissions variables on GPA, for each institution at each year level was also conducted.
Overall, the data found positive correlations between performance in medical school, school achievement and UMAT, but not interview. However, there were substantial differences between schools, across year levels, and within sections of UMAT exposed. Despite this, each admission variable was shown to add towards explaining course performance, net of other variables.
The findings suggest the strength of multiple admissions tools in predicting outcomes of medical students. However, they also highlight the large differences in outcomes achieved by different schools, thus emphasising the pitfalls of generalising results from predictive validity studies without recognising the diverse ways in which they are designed and the variation in the institutional contexts in which they are administered. The assumption that high-positive correlations are desirable (or even expected) in these studies is also problematised.
PMCID: PMC3880586  PMID: 24373207
Selection; Predictive validity; Admissions policy
23.  Practice effects in medical school entrance testing with the undergraduate medicine and health sciences admission test (UMAT) 
BMC Medical Education  2014;14:48.
The UMAT is widely used for selection into undergraduate medical and dental courses in Australia and New Zealand (NZ). It tests aptitudes thought to be especially relevant to medical studies and consists of 3 sections – logical reasoning and problem solving (UMAT-1), understanding people (UMAT-2) and non-verbal reasoning (UMAT-3). A substantial proportion of all candidates re-sit the UMAT. Re-sitting raises the issue as to what might be the precise magnitude and determinants of any practice effects on the UMAT and their implications for equity in subsequent selection processes.
Between 2000 and 2012, 158,909 UMAT assessments were completed. From these, 135,833 cases were identified where a candidate had sat once or more during that period with 117,505 cases (86.5%) having sat once, 14,739 having sat twice (10.9%), 2,752 thrice (2%) and 837, 4 or more times (0.6%). Subsequent analyses determined predictors of multiple re-sits as well as the magnitude and socio-demographic determinants of any practice effects.
Increased likelihood of re-sitting the UMAT twice or more was predicted by being male, of younger age, being from a non-English language speaking background and being from NZ and for Australian candidates, being urban rather than rurally based. For those who sat at least twice, the total UMAT score between a first and second attempt improved by 10.7 ± 0.2 percentiles, UMAT-1 by 8.3 ± 0.2 percentiles, UMAT-2 by 8.3 ± 0.2 percentiles and UMAT-3 by 7.7 ± 0.2 percentiles. An increase in total UMAT percentile score on re-testing was predicted by a lower initial score and being a candidate from NZ rather than from Australia while a decrease was related to increased length of time since initially sitting the test, older age and non-English language background.
Re-sitting the UMAT augments performance in each of its components together with the total UMAT percentile score. Whether this increase represents just an improvement in performance or an improvement in understanding of the variables and therefore competence needs to be further defined. If only the former, then practice effects may be introducing inequity in student selection for medical or dental schools in Australia or NZ.
PMCID: PMC4007585  PMID: 24618021
24.  Do different medical curricula influence self-assessed clinical thinking of students? 
Objectives: As a fundamental element of medical practice, clinical reasoning should be cultivated in courses of study in human medicine. To date, however, no conclusive evidence has been offered as to what forms of teaching and learning are most effective in achieving this goal. The Diagnostic Thinking Inventory (DTI) was developed as a means of measuring knowledge-unrelated components of clinical reasoning. The present pilot study examines the adequacy of this instrument in measuring differences in the clinical reasoning of students in varying stages of education in three curricula of medical studies.
Methods: The Diagnostic Thinking Inventory (DTI) comprises 41 items in two subscales (“Flexibility in Thinking” and “Structure of Knowledge in Memory”). Each item contains a statement or finding concerning clinical reasoning in the form of a stem under which a 6-point scale presents opposing conclusions. The subjects are asked to assess their clinical thinking within this range. The German-language version of the DTI was completed by 247 student volunteers from three schools and varying clinical semesters. In a quasi-experimental design, 219 subjects from traditional and model courses of study in the German state of North Rhine-Westphalia took part. Specifically, these were 5th, 6th and 8th semester students from the model course of study at Witten/Herdecke University (W/HU), from the model (7th and 9th semester) and traditional (7th semester) courses of study at the Ruhr University Bochum (RUB) and from the model course of study (9th semester) at the University of Cologne (UoC). The data retrieved were quantitatively assessed.
Results: The reliability of the questionnaire in its entirety was good (Cronbach’s alpha between 0.71 and 0.83); the reliability of the subscales ranged between 0.49 and 0.75. The different groups were compared using the Mann-Whitney test, revealing significant differences among semester cohorts within a school as well as between students from similar academic years in different schools. Among the participants from the model course of study at the W/HU, scores increased from the 5th to the 6th semester and from the 5th to the 9th semester. Among individual cohorts at RUB, no differences could be established between model and traditional courses of study or between 7th and 9th semester students in model courses of study. Comparing all participating highest semester students, the 8th semester participants from the W/HU achieved the highest scores – significantly higher than those of 9th semester RUB students or 9th semester UoC students. Scores from the RUB 9th semester participants were significantly higher than those of the 9th semester UoC participants.
Discussion: The German-language version of the DTI measures self-assessed differences in diagnostic reasoning among students from various semesters and different model and traditional courses of study with satisfactory reliability. The results can be used for discussion in the context of diverse curricula. The DTI is therefore appropriate for further research that can then be correlated with the different teaching method characteristics and outcomes of various curricula.
PMCID: PMC4027808  PMID: 24872858
clinical thinking; clinical reasoning; PBL; diagnostic thinking inventory
25.  Socio-economic predictors of performance in the Undergraduate Medicine and Health Sciences Admission Test (UMAT) 
BMC Medical Education  2013;13:155.
Entry from secondary school to Australian and New Zealand undergraduate medical schools has since the late 1990’s increasingly relied on the Undergraduate Medicine and Health Sciences Admission Test (UMAT) as one of the selection factors. The UMAT consists of 3 sections – logical reasoning and problem solving (UMAT-1), understanding people (UMAT-2) and non-verbal reasoning (UMAT-3). One of the goals of using this test has been to enhance equity in the selection of students with the anticipation of an increase in the socioeconomic diversity in student cohorts. However there has been limited assessment as to whether UMAT performance itself might be influenced by socioeconomic background.
Between 2000 and 2012, 158,909 UMAT assessments were completed. From these, 118,085 cases have been identified where an Australian candidate was sitting for the first time during that period. Predictors of the total UMAT score, UMAT-1, UMAT-2 and UMAT-3 scores were entered into regression models and included gender, age, school type, language used at home, deciles for the Index of Relative Socioeconomic Advantage and Disadvantage score, the Accessibility/Remoteness Index of Australia (ARIA), self-identification as being of Aboriginal or Torres Strait Islander origin (ATSI) and current Australian state or territory of abode.
A lower UMAT score was predicted by living in an area of relatively higher social disadvantage and lower social advantage. Other socioeconomic indicators were consistent with this observation with lower scores in those who self-identified as being of ATSI origin and higher scores evident in those from fee-paying independent school backgrounds compared to government schools. Lower scores were seen with increasing age, female gender and speaking any language other than English at home. Divergent effects of rurality were observed, with increased scores for UMAT-1 and UMAT-2, but decreasing UMAT-3 scores with increasing ARIA score. Significant state-based differences largely reflected substantial socio-demographic differences across Australian states and territories.
Better performance by Australian candidates in the UMAT is linked to an increase in socio-economic advantage and reduced disadvantage.This observation provides a firm foundation for selection processes at medical schools in Australia that have incorporated affirmative action pathways to quarantine places for students from areas of socio-economic disadvantage.
PMCID: PMC4220554  PMID: 24286571

Results 1-25 (1374515)