PMCC PMCC

Search tips
Search criteria

Advanced
Results 1-25 (1223278)

Clipboard (0)
None

Related Articles

1.  Predicting academic outcomes in an Australian graduate entry medical programme 
BMC Medical Education  2014;14:31.
Background
Predictive validity studies for selection criteria into graduate entry courses in Australia have been inconsistent in their outcomes. One of the reasons for this inconsistency may have been failure to have adequately considered background disciplines of the graduates as well as other potential confounding socio-demographic variables that may influence academic performance.
Methods
Graduate entrants into the MBBS at The University of Western Australia between 2005 and 2012 were studied (N = 421). They undertook a 6-month bridging course, before joining the undergraduate-entry students for Years 3 through 6 of the medical course. Students were selected using their undergraduate Grade Point Average (GPA), Graduate Australian Medical School Admissions Test scores (GAMSAT) and a score from a standardised interview. Students could apply from any background discipline and could also be selected through an alternative rural entry pathway again utilising these 3 entry scores. Entry scores, together with age, gender, discipline background, rural entry status and a socioeconomic indicator were entered into linear regression models to determine the relative influence of each predictor on subsequent academic performance in the course.
Results
Background discipline, age, gender and selection through the rural pathway were variously related to each of the 3 entry criteria. Their subsequent inclusion in linear regression models identified GPA at entry, being from a health/allied health background and total GAMSAT score as consistent independent predictors of stronger academic performance as measured by the weighted average mark for the core units completed throughout the course. The Interview score only weakly predicted performance later in the course and mainly in clinically-based units. The association of total GAMSAT score with academic performance was predominantly dictated by the score in GAMSAT Section 3 (Reasoning in the biological and physical sciences) with Section 1 (Reasoning in the humanities and social sciences) and Section 2 (Written communication) also contributing either later or early in the course respectively. Being from a more disadvantaged socioeconomic background predicted weaker academic performance early in the course. Being an older student at entry or from a humanities background also predicted weaker academic performance.
Conclusions
This study confirms that both GPA at entry and the GAMSAT score together predict outcomes not only in the early stages of a graduate-entry medical programme but throughout the course. It also indicates that a comprehensive evaluation of the predictive validity of GAMSAT scores, interview scores and undergraduate academic performance as valid selection processes for graduate entry into medical school needs to simultaneously consider the potential confounding influence of graduate discipline background and other socio-demographic factors on both the initial selection parameters themselves as well as subsequent academic performance.
doi:10.1186/1472-6920-14-31
PMCID: PMC3931285  PMID: 24528509
2.  To what extent does the Health Professions Admission Test-Ireland predict performance in early undergraduate tests of communication and clinical skills? – An observational cohort study 
BMC Medical Education  2013;13:68.
Background
Internationally, tests of general mental ability are used in the selection of medical students. Examples include the Medical College Admission Test, Undergraduate Medicine and Health Sciences Admission Test and the UK Clinical Aptitude Test. The most widely used measure of their efficacy is predictive validity.
A new tool, the Health Professions Admission Test- Ireland (HPAT-Ireland), was introduced in 2009. Traditionally, selection to Irish undergraduate medical schools relied on academic achievement. Since 2009, Irish and EU applicants are selected on a combination of their secondary school academic record (measured predominately by the Leaving Certificate Examination) and HPAT-Ireland score. This is the first study to report on the predictive validity of the HPAT-Ireland for early undergraduate assessments of communication and clinical skills.
Method
Students enrolled at two Irish medical schools in 2009 were followed up for two years. Data collected were gender, HPAT-Ireland total and subsection scores; Leaving Certificate Examination plus HPAT-Ireland combined score, Year 1 Objective Structured Clinical Examination (OSCE) scores (Total score, communication and clinical subtest scores), Year 1 Multiple Choice Questions and Year 2 OSCE and subset scores. We report descriptive statistics, Pearson correlation coefficients and Multiple linear regression models.
Results
Data were available for 312 students. In Year 1 none of the selection criteria were significantly related to student OSCE performance. The Leaving Certificate Examination and Leaving Certificate plus HPAT-Ireland combined scores correlated with MCQ marks.
In Year 2 a series of significant correlations emerged between the HPAT-Ireland and subsections thereof with OSCE Communication Z-scores; OSCE Clinical Z-scores; and Total OSCE Z-scores. However on multiple regression only the relationship between Total OSCE Score and the Total HPAT-Ireland score remained significant; albeit the predictive power was modest.
Conclusion
We found that none of our selection criteria strongly predict clinical and communication skills. The HPAT- Ireland appears to measures ability in domains different to those assessed by the Leaving Certificate Examination. While some significant associations did emerge in Year 2 between HPAT Ireland and total OSCE scores further evaluation is required to establish if this pattern continues during the senior years of the medical course.
doi:10.1186/1472-6920-13-68
PMCID: PMC3667098  PMID: 23663266
Selection; Medical; Student; Validity; Predictive; HPAT-Ireland; Assessment; Cognitive; Ability
3.  Should applicants to Nottingham University Medical School study a non-science A-level? A cohort study 
Background
It has been suggested that studying non-science subjects at A-level should be compulsory for medical students. Our admissions criteria specify only Biology, Chemistry and one or more additional subjects. This study aimed to determine whether studying a non-science subject for A-level is an independent predictor of achievement on the undergraduate medical course.
Methods
The subjects of this retrospective cohort study were 164 students from one entry-year group (October 2000), who progressed normally on the 5-year undergraduate medical course at Nottingham. Pre-admission academic and socio-demographic data and undergraduate course marks were obtained. T-test and hierarchical multiple linear regression analyses were undertaken to identify independent predictors of five course outcomes at different stages throughout the course.
Results
There was no evidence that the choice of science or non-science as the third or fourth A-level subject had any influence on course performance. Demographic variables (age group, sex, and fee status) had some predictive value but ethnicity did not. Pre-clinical course performance was the strongest predictor in the clinical phases (pre-clinical Themes A&B (knowledge) predicted Clinical Knowledge, p < 0.001, and pre-clinical Themes C&D (skills) predicted Clinical Skills, p = < 0.01).
Conclusion
This study of one year group at Nottingham Medical School provided no evidence that the admissions policy on A-level requirements should specify the choice of third or fourth subject.
doi:10.1186/1472-6920-9-5
PMCID: PMC2650695  PMID: 19159444
4.  Can personal qualities of medical students predict in-course examination success and professional behaviour? An exploratory prospective cohort study 
BMC Medical Education  2012;12:69.
Background
Over two-thirds of UK medical schools are augmenting their selection procedures for medical students by using the United Kingdom Clinical Aptitude Test (UKCAT), which employs tests of cognitive and non-cognitive personal qualities, but clear evidence of the tests’ predictive validity is lacking. This study explores whether academic performance and professional behaviours that are important in a health professional context can be predicted by these measures, when taken before or very early in the medical course.
Methods
This prospective cohort study follows the progress of the entire student cohort who entered Hull York Medical School in September 2007, having taken the UKCAT cognitive tests in 2006 and the non-cognitive tests a year later. This paper reports on the students’ first and second academic years of study. The main outcome measures were regular, repeated tutor assessment of individual students’ interpersonal skills and professional behaviour, and annual examination performance in the three domains of recall and application of knowledge, evaluation of data, and communication and practical clinical skills. The relationships between non-cognitive test scores, cognitive test scores, tutor assessments and examination results were explored using the Pearson product–moment correlations for each group of data; the data for students obtaining the top and bottom 20% of the summative examination results were compared using Analysis of Variance.
Results
Personal qualities measured by non-cognitive tests showed a number of statistically significant relationships with ratings of behaviour made by tutors, with performance in each year’s objective structured clinical examinations (OSCEs), and with themed written summative examination marks in each year. Cognitive ability scores were also significantly related to each year’s examination results, but seldom to professional behaviours. The top 20% of examination achievers could be differentiated from the bottom 20% on both non-cognitive and cognitive measures.
Conclusions
This study shows numerous significant relationships between both cognitive and non-cognitive test scores, academic examination scores and indicators of professional behaviours in medical students. This suggests that measurement of non-cognitive personal qualities in applicants to medical school could make a useful contribution to selection and admission decisions. Further research is required in larger representative groups, and with more refined predictor measures and behavioural assessment methods, to establish beyond doubt the incremental validity of such measures over conventional cognitive assessments.
doi:10.1186/1472-6920-12-69
PMCID: PMC3473297  PMID: 22873571
5.  The Academic Backbone: longitudinal continuities in educational achievement from secondary school and medical school to MRCP(UK) and the specialist register in UK medical students and doctors 
BMC Medicine  2013;11:242.
Background
Selection of medical students in the UK is still largely based on prior academic achievement, although doubts have been expressed as to whether performance in earlier life is predictive of outcomes later in medical school or post-graduate education. This study analyses data from five longitudinal studies of UK medical students and doctors from the early 1970s until the early 2000s. Two of the studies used the AH5, a group test of general intelligence (that is, intellectual aptitude). Sex and ethnic differences were also analyzed in light of the changing demographics of medical students over the past decades.
Methods
Data from five cohort studies were available: the Westminster Study (began clinical studies from 1975 to 1982), the 1980, 1985, and 1990 cohort studies (entered medical school in 1981, 1986, and 1991), and the University College London Medical School (UCLMS) Cohort Study (entered clinical studies in 2005 and 2006). Different studies had different outcome measures, but most had performance on basic medical sciences and clinical examinations at medical school, performance in Membership of the Royal Colleges of Physicians (MRCP(UK)) examinations, and being on the General Medical Council Specialist Register.
Results
Correlation matrices and path analyses are presented. There were robust correlations across different years at medical school, and medical school performance also predicted MRCP(UK) performance and being on the GMC Specialist Register. A-levels correlated somewhat less with undergraduate and post-graduate performance, but there was restriction of range in entrants. General Certificate of Secondary Education (GCSE)/O-level results also predicted undergraduate and post-graduate outcomes, but less so than did A-level results, but there may be incremental validity for clinical and post-graduate performance. The AH5 had some significant correlations with outcome, but they were inconsistent. Sex and ethnicity also had predictive effects on measures of educational attainment, undergraduate, and post-graduate performance. Women performed better in assessments but were less likely to be on the Specialist Register. Non-white participants generally underperformed in undergraduate and post-graduate assessments, but were equally likely to be on the Specialist Register. There was a suggestion of smaller ethnicity effects in earlier studies.
Conclusions
The existence of the Academic Backbone concept is strongly supported, with attainment at secondary school predicting performance in undergraduate and post-graduate medical assessments, and the effects spanning many years. The Academic Backbone is conceptualized in terms of the development of more sophisticated underlying structures of knowledge ('cognitive capital’ and 'medical capital’). The Academic Backbone provides strong support for using measures of educational attainment, particularly A-levels, in student selection.
doi:10.1186/1741-7015-11-242
PMCID: PMC3827330  PMID: 24229333
Academic Backbone; Secondary school attainment; Undergraduate medical education; Post-graduate medical education; Longitudinal analyses; Continuities; Medical student selection; Cognitive capital; Medical capital; Aptitude tests
6.  Predictive validity of a new integrated selection process for medical school admission 
BMC Medical Education  2014;14:86.
Background
This paper is an evaluation of an integrated selection process utilising previous academic achievement [Universities Admission Index (UAI)], a skills test [Undergraduate Medicine and Health Sciences Admission Test (UMAT)], and a structured interview, introduced (in its entirety) in 2004 as part of curriculum reform of the undergraduate Medicine Program at the University of New South Wales (UNSW), Australia. Demographic measures of gender, country of birth, educational background and rurality are considered.
Method
Admission scores and program outcomes of 318 students enrolled in 2004 and 2005 were studied. Regression analyses were undertaken to determine whether selection scores predicted overall, knowledge-based and clinical-based learning outcomes after controlling for demographics.
Results
UAI attained the highest values in predicting overall and knowledge-based outcomes. The communication dimension of the interview achieved similar predictive values as UAI for clinical-based outcomes, although predictive values were relatively low. The UMAT did not predict any performance outcome. Female gender, European/European-derived country of birth and non-rurality were significant predictors independent of UAI scores.
Conclusion
Results indicate promising validity for an integrated selection process introduced for the Medicine Program at UNSW, with UAI and interview predictive of learning outcomes. Although not predictive, UMAT may have other useful roles in an integrated selection process. Further longitudinal research is proposed to monitor and improve the validity of the integrated student selection process.
doi:10.1186/1472-6920-14-86
PMCID: PMC3999457  PMID: 24755325
Student selection; Predictive validity; Predicting; Performance; Demographics
7.  Same admissions tools, different outcomes: a critical perspective on predictive validity in three undergraduate medical schools 
BMC Medical Education  2013;13:173.
Background
Admission to medical school is one of the most highly competitive entry points in higher education. Considerable investment is made by universities to develop selection processes that aim to identify the most appropriate candidates for their medical programs. This paper explores data from three undergraduate medical schools to offer a critical perspective of predictive validity in medical admissions.
Methods
This study examined 650 undergraduate medical students from three Australian universities as they progressed through the initial years of medical school (accounting for approximately 25 per cent of all commencing undergraduate medical students in Australia in 2006 and 2007). Admissions criteria (aptitude test score based on UMAT, school result and interview score) were correlated with GPA over four years of study. Standard regression of each of the three admissions variables on GPA, for each institution at each year level was also conducted.
Results
Overall, the data found positive correlations between performance in medical school, school achievement and UMAT, but not interview. However, there were substantial differences between schools, across year levels, and within sections of UMAT exposed. Despite this, each admission variable was shown to add towards explaining course performance, net of other variables.
Conclusion
The findings suggest the strength of multiple admissions tools in predicting outcomes of medical students. However, they also highlight the large differences in outcomes achieved by different schools, thus emphasising the pitfalls of generalising results from predictive validity studies without recognising the diverse ways in which they are designed and the variation in the institutional contexts in which they are administered. The assumption that high-positive correlations are desirable (or even expected) in these studies is also problematised.
doi:10.1186/1472-6920-13-173
PMCID: PMC3880586  PMID: 24373207
Selection; Predictive validity; Admissions policy
8.  Potential influence of selection criteria on the demographic composition of students in an Australian medical school 
BMC Medical Education  2011;11:97.
Background
Prior to 1999 students entering our MBBS course were selected on academic performance alone. We have now evaluated the impact on the demographics of subsequent cohorts of our standard entry students (those entering directly from high school) of the addition to the selection process of an aptitude test (UMAT), a highly structured interview and a rural incentive program.
Methods
Students entering from 1985 to 1998, selected on academic performance alone (N = 1402), were compared to those from 1999 to 2011, selected on the basis of a combination of academic performance, interview score, and UMAT score together with the progressive introduction of a rural special entry pathway (N = 1437).
Results
Males decreased from 57% to 45% of the cohort, students of NE or SE Asian origin decreased from 30% to 13%, students born in Oceania increased from 52% to 69%, students of rural origin from 5% to 21% and those from independent high schools from 56% to 66%. The proportion of students from high schools with relative socio-educational disadvantage remained unchanged at approximately 10%. The changes reflect in part increasing numbers of female and independent high school applicants and the increasing rural quota. However, they were also associated with higher interview scores in females vs males and lower interview scores in those of NE and SE Asian origin compared to those born in Oceania or the UK. Total UMAT scores were unrelated to gender or region of origin.
Conclusions
The revised selection processes had no impact on student representation from schools with relative socio-educational disadvantage. However, the introduction of special entry quotas for students of rural origin and a structured interview, but not an aptitude test, were associated with a change in gender balance and ethnicity of students in an Australian undergraduate MBBS course.
doi:10.1186/1472-6920-11-97
PMCID: PMC3233506  PMID: 22111521
9.  The value of the UK Clinical Aptitude Test in predicting pre-clinical performance: a prospective cohort study at Nottingham Medical School 
BMC Medical Education  2010;10:55.
Background
The UK Clinical Aptitude Test (UKCAT) was introduced in 2006 as an additional tool for the selection of medical students. It tests mental ability in four distinct domains (Quantitative Reasoning, Verbal Reasoning, Abstract Reasoning, and Decision Analysis), and the results are available to students and admissions panels in advance of the selection process. As yet the predictive validity of the test against course performance is largely unknown.
The study objective was to determine whether UKCAT scores predict performance during the first two years of the 5-year undergraduate medical course at Nottingham.
Methods
We studied a single cohort of students, who entered Nottingham Medical School in October 2007 and had taken the UKCAT. We used linear regression analysis to identify independent predictors of marks for different parts of the 2-year preclinical course.
Results
Data were available for 204/260 (78%) of the entry cohort. The UKCAT total score had little predictive value. Quantitative Reasoning was a significant independent predictor of course marks in Theme A ('The Cell'), (p = 0.005), and Verbal Reasoning predicted Theme C ('The Community') (p < 0.001), but otherwise the effects were slight or non-existent.
Conclusion
This limited study from a single entry cohort at one medical school suggests that the predictive value of the UKCAT, particularly the total score, is low. Section scores may predict success in specific types of course assessment.
The ultimate test of validity will not be available for some years, when current cohorts of students graduate. However, if this test of mental ability does not predict preclinical performance, it is arguably less likely to predict the outcome in the clinical years. Further research from medical schools with different types of curriculum and assessment is needed, with longitudinal studies throughout the course.
doi:10.1186/1472-6920-10-55
PMCID: PMC2922293  PMID: 20667093
10.  Predicting the “strugglers”: a case-control study of students at Nottingham University Medical School 
BMJ : British Medical Journal  2006;332(7548):1009-1013.
Objective To identify potential predictors of undergraduate students who struggle during their medical training.
Design Case-control study. Cases were students who had experienced academic or personal difficulties that affected their progression on the course (“strugglers”). Controls were selected at random from the corresponding year cohorts, using a ratio of four controls for each struggler.
Setting University of Nottingham Medical School.
Participants Students who entered the course over five consecutive years.
Main outcome measures Likelihood ratios for independent risk factors for struggling on the course
Results 10-15% of each year's student intake were identified as strugglers. Significant independent predictors of students being in this category were negative comments in the academic reference (likelihood ratio 2.25, 95% confidence intervals 1.44 to 3.50), lower mean examination grade at A level (2.19, 1.37 to 3.51), and the late offer of a place (1.98, 1.19 to 3.30). Male sex was a less significant risk factor (1.70, 1.09 to 2.65) as was a lower grade at GCSE science (2.13, 1.12 to 4.05). In UK students whose ethnicity was known, not being white was a significant predictor of struggling (2.77, 1.52 to 5.05) but the presence of negative comments was not. Age at entry to the course and the possession of a previous degree were not predictive.
Conclusions Our results support retention of existing selection practices relating to academic achievement and critical review of students' references. We plan to undertake further investigation of the reasons why some students, including men, those with late offers and those from ethnic minority backgrounds, may do less well on the Nottingham course.
doi:10.1136/bmj.38730.678310.63
PMCID: PMC1450046  PMID: 16543299
11.  A holistic review of the medical school admission process: examining correlates of academic underperformance 
Medical Education Online  2014;19:10.3402/meo.v19.22919.
Background
Despite medical school admission committees’ best efforts, a handful of seemingly capable students invariably struggle during their first year of study. Yet, even as entrance criteria continue to broaden beyond cognitive qualifications, attention inevitably reverts back to such factors when seeking to understand these phenomena. Using a host of applicant, admission, and post-admission variables, the purpose of this inductive study, then, was to identify a constellation of student characteristics that, taken collectively, would be predictive of students at-risk of underperforming during the first year of medical school. In it, we hypothesize that a wider range of factors than previously recognized could conceivably play roles in understanding why students experience academic problems early in the medical educational continuum.
Methods
The study sample consisted of the five most recent matriculant cohorts from a large, southeastern medical school (n=537). Independent variables reflected: 1) the personal demographics of applicants (e.g., age, gender); 2) academic criteria (e.g., undergraduate grade point averages [GPA], medical college admission test); 3) selection processes (e.g., entrance track, interview scores, committee votes); and 4) other indicators of personality and professionalism (e.g., Mayer-Salovey-Caruso Emotional Intelligence Test™ emotional intelligence scores, NEO PI-R™ personality profiles, and appearances before the Professional Code Committee [PCC]). The dependent variable, first-year underperformance, was defined as ANY action (repeat, conditionally advance, or dismiss) by the college's Student Progress and Promotions Committee (SPPC) in response to predefined academic criteria. This study protocol was approved by the local medical institutional review board (IRB).
Results
Of the 537 students comprising the study sample, 61 (11.4%) met the specified criterion for academic underperformance. Significantly increased academic risks were identified among students who 1) had lower mean undergraduate science GPAs (OR=0.24, p=0.001); 2) entered medical school via an accelerated BS/MD track (OR=16.15, p=0.002); 3) were 31 years of age or older (OR=14.76, p=0.005); and 4) were non-unanimous admission committee admits (OR=0.53, p=0.042). Two dimensions of the NEO PI-R™ personality inventory, openness (+) and conscientiousness (−), were modestly but significantly correlated with academic underperformance. Only for the latter, however, were mean scores found to differ significantly between academic performers and underperformers. Finally, appearing before the college's PCC (OR=4.21, p=0.056) fell just short of statistical significance.
Conclusions
Our review of various correlates across the matriculation process highlights the heterogeneity of factors underlying students’ underperformance during the first year of medical school and challenges medical educators to understand the complexity of predicting who, among admitted matriculants, may be at future academic risk.
doi:10.3402/meo.v19.22919
PMCID: PMC3974177  PMID: 24695362
admissions; underperformance; selection; at-risk students
12.  The UK clinical aptitude test and clinical course performance at Nottingham: a prospective cohort study 
BMC Medical Education  2013;13:32.
Background
The UK Clinical Aptitude Test (UKCAT) was introduced in 2006 as an additional tool for the selection of medical students. It tests mental ability in four distinct domains (Verbal Reasoning, Quantitative Reasoning, Abstract Reasoning, and Decision Analysis), and the results are available to students and admission panels in advance of the selection process. Our first study showed little evidence of any predictive validity for performance in the first two years of the Nottingham undergraduate course.
The study objective was to determine whether the UKCAT scores had any predictive value for the later parts of the course, largely delivered via clinical placements.
Methods
Students entering the course in 2007 and who had taken the UKCAT were asked for permission to use their anonymised data in research. The UKCAT scores were incorporated into a database with routine pre-admission socio-demographics and subsequent course performance data. Correlation analysis was followed by hierarchical multivariate linear regression.
Results
The original study group comprised 204/254 (80%) of the full entry cohort. With attrition over the five years of the course this fell to 185 (73%) by Year 5. The Verbal Reasoning score and the UKCAT Total score both demonstrated some univariate correlations with clinical knowledge marks, and slightly less with clinical skills. No parts of the UKCAT proved to be an independent predictor of clinical course marks, whereas prior attainment was a highly significant predictor (p <0.001).
Conclusions
This study of one cohort of Nottingham medical students showed that UKCAT scores at admission did not independently predict subsequent performance on the course. Whilst the test adds another dimension to the selection process, its fairness and validity in selecting promising students remains unproven, and requires wider investigation and debate by other schools.
doi:10.1186/1472-6920-13-32
PMCID: PMC3621812  PMID: 23442227
13.  Comparison of communication skills between medical students admitted after interviews or on academic merits 
BMC Medical Education  2012;12:46.
Background
Selection of the best medical students among applicants is debated and many different methods are used. Academic merits predict good academic performance, but students admitted by other pathways need not be less successful. The aim of this study, was to compare communication skills between students admitted to medical school through interviews or on academic merits, respectively.
Methods
A retrospective cohort study. Communication skills at a surgical OSCE in 2008 were assessed independently by two observers using an evaluative rating scale. Correlations, t-tests and multivariate analyses by logistic regressions were employed. Academic merits were defined as upper secondary school grade point average (GPA) or scores from the Swedish Scholastic Assessment Test (SweSAT).
Results
The risk of showing unsatisfactory communicative performance was significantly lower among the students selected by interviews (OR 0.32, CI95 0.12-0.83), compared to those selected on the basis of academic merits. However, there was no significant difference in communication skills scores between the different admission groups; neither did the proportion of high performers differ. No difference in the result of the written examination was seen between groups.
Conclusions
Our results confirm previous experience from many medical schools that students selected in different ways achieve comparable results during the clinical semesters. However, selection through interview seems to reduce the number of students who demonstrate inferior communication skills at 4th year of medical school.
doi:10.1186/1472-6920-12-46
PMCID: PMC3503650  PMID: 22726308
14.  A levels and intelligence as predictors of medical careers in UK doctors: 20 year prospective study 
BMJ : British Medical Journal  2003;327(7407):139-142.
Objective To assess whether A level grades (achievement) and intelligence (ability) predict doctors' careers.
Design Prospective cohort study with follow up after 20 years by postal questionnaire.
Setting A UK medical school in London.
Participants 511 doctors who had entered Westminster Medical School as clinical students between 1975 and 1982 were followed up in January 2002.
Main outcome measures Time taken to reach different career grades in hospital or general practice, postgraduate qualifications obtained (membership/fellowships, diplomas, higher academic degrees), number of research publications, and measures of stress and burnout related to A level grades and intelligence (result of AH5 intelligence test) at entry to clinical school. General health questionnaire, Maslach burnout inventory, and questionnaire on satisfaction with career at follow up.
Results 47 (9%) doctors were no longer on the Medical Register. They had lower A level grades than those who were still on the register (P < 0.001). A levels also predicted performance in undergraduate training, performance in postregistration house officer posts, and time to achieve membership qualifications (Cox regression, P < 0.001; b=0.376, SE=0.098, exp(b)=1.457). Intelligence did not independently predict dropping off the register, career outcome, or other measures. A levels did not predict diploma or higher academic qualifications, research publications, or stress or burnout. Diplomas, higher academic degrees, and research publications did, however, significantly correlate with personality measures.
Conclusions Results of achievement tests, in this case A level grades, which are particularly used for selection of students in the United Kingdom, have long term predictive validity for undergraduate and postgraduate careers. In contrast, a test of ability or aptitude (AH5) was of little predictive validity for subsequent medical careers.
PMCID: PMC165701  PMID: 12869457
15.  Construct-level predictive validity of educational attainment and intellectual aptitude tests in medical student selection: meta-regression of six UK longitudinal studies 
BMC Medicine  2013;11:243.
Background
Measures used for medical student selection should predict future performance during training. A problem for any selection study is that predictor-outcome correlations are known only in those who have been selected, whereas selectors need to know how measures would predict in the entire pool of applicants. That problem of interpretation can be solved by calculating construct-level predictive validity, an estimate of true predictor-outcome correlation across the range of applicant abilities.
Methods
Construct-level predictive validities were calculated in six cohort studies of medical student selection and training (student entry, 1972 to 2009) for a range of predictors, including A-levels, General Certificates of Secondary Education (GCSEs)/O-levels, and aptitude tests (AH5 and UK Clinical Aptitude Test (UKCAT)). Outcomes included undergraduate basic medical science and finals assessments, as well as postgraduate measures of Membership of the Royal Colleges of Physicians of the United Kingdom (MRCP(UK)) performance and entry in the Specialist Register. Construct-level predictive validity was calculated with the method of Hunter, Schmidt and Le (2006), adapted to correct for right-censorship of examination results due to grade inflation.
Results
Meta-regression analyzed 57 separate predictor-outcome correlations (POCs) and construct-level predictive validities (CLPVs). Mean CLPVs are substantially higher (.450) than mean POCs (.171). Mean CLPVs for first-year examinations, were high for A-levels (.809; CI: .501 to .935), and lower for GCSEs/O-levels (.332; CI: .024 to .583) and UKCAT (mean = .245; CI: .207 to .276). A-levels had higher CLPVs for all undergraduate and postgraduate assessments than did GCSEs/O-levels and intellectual aptitude tests. CLPVs of educational attainment measures decline somewhat during training, but continue to predict postgraduate performance. Intellectual aptitude tests have lower CLPVs than A-levels or GCSEs/O-levels.
Conclusions
Educational attainment has strong CLPVs for undergraduate and postgraduate performance, accounting for perhaps 65% of true variance in first year performance. Such CLPVs justify the use of educational attainment measure in selection, but also raise a key theoretical question concerning the remaining 35% of variance (and measurement error, range restriction and right-censorship have been taken into account). Just as in astrophysics, ‘dark matter’ and ‘dark energy’ are posited to balance various theoretical equations, so medical student selection must also have its ‘dark variance’, whose nature is not yet properly characterized, but explains a third of the variation in performance during training. Some variance probably relates to factors which are unpredictable at selection, such as illness or other life events, but some is probably also associated with factors such as personality, motivation or study skills.
doi:10.1186/1741-7015-11-243
PMCID: PMC3827328  PMID: 24229353
Medical student selection; Undergraduate performance; Postgraduate performance; Educational attainment; Aptitude tests; Criterion-related construct validity; Range restriction; Right censorship; Grade inflation; Markov Chain Monte Carlo algorithm
16.  Predictive power of UKCAT and other pre-admission measures for performance in a medical school in Glasgow: a cohort study 
BMC Medical Education  2014;14:116.
Background
The UK Clinical Aptitude Test (UKCAT) and its four subtests are currently used by 24 Medical and Dental Schools in the UK for admissions. This longitudinal study examines the predictive validity of UKCAT for final performance in the undergraduate medical degree programme at one Medical School and compares this with the predictive validity of the selection measures available pre-UKCAT.
Methods
This was a retrospective observational study of one cohort of students, admitted to Glasgow Medical School in 2007. We examined the associations which UKCAT scores, school science grades and pre-admissions interview scores had with performance indicators, particularly final composite scores that determine students’ postgraduate training opportunities and overall ranking (Educational Performance Measure - EPM, and Honours and Commendation – H&C). Analyses were conducted both with and without adjustment for potential socio-demographic confounders (gender, age, ethnicity and area deprivation).
Results
Despite its predictive value declining as students progress through the course, UKCAT was associated with the final composite scores. In mutually adjusted analyses (also adjusted for socio-demographic confounders), only UKCAT total showed independent relationships with both EPM (p = 0.005) and H&C (p = 0.004), school science achievements predicted EPM (p = 0.009), and pre-admissions interview score predicted neither. UKCAT showed less socio-demographic variation than did TSS.
Conclusion
UKCAT has a modest predictive power for overall course performance at the University of Glasgow Medical School over and above that of school science achievements or pre-admission interview score and we conclude that UKCAT is the most useful predictor of final ranking.
doi:10.1186/1472-6920-14-116
PMCID: PMC4063234  PMID: 24919950
UKCAT; Predictive validity; Widening participation; Socio-economic indicators; Admissions interview; School HE participation rate
17.  Knowledge and skills retention following Emergency Triage, Assessment and Treatment plus Admission course for final year medical students in Rwanda: a longitudinal cohort study 
Archives of disease in childhood  2014;99(11):993-997.
Aim
To determine whether, after the Emergency Triage, Assessment and Treatment plus Admission (ETAT+) course, a comprehensive paediatric life support course, final year medical undergraduates in Rwanda would achieve a high level of knowledge and practical skills and if these were retained. To guide further course development, student feedback was obtained.
Methods
Longitudinal cohort study of knowledge and skills of all final year medical undergraduates at the University of Rwanda in academic year 2011–2012 who attended a 5-day ETAT+ course. Students completed a precourse knowledge test. Knowledge and clinical skills assessments, using standardised marking, were performed immediately postcourse and 3–9 months later. Feedback was obtained using printed questionnaires.
Results
84 students attended the course and re-evaluation. Knowledge test showed a significant improvement, from median 47% to 71% correct answers (p<0.001). For two clinical skills scenarios, 98% passed both scenarios, 37% after a retake, 2% failed both scenarios. Three to nine months later, students were re-evaluated, median score for knowledge test 67%, not significantly different from postcourse (p>0.1). For clinical skills, 74% passed, with 32% requiring a retake, 8% failed after retake, 18% failed both scenarios, a significant deterioration (p<0.0001).
Conclusions
Students performed well on knowledge and skills immediately after a comprehensive ETAT+ course. Knowledge was maintained 3–9 months later. Clinical skills, which require detailed sequential steps, declined, but most were able to perform them satisfactorily after feedback. The course was highly valued, but several short courses and more practical teaching were advocated.
doi:10.1136/archdischild-2014-306078
PMCID: PMC4198299  PMID: 24925893
18.  Knowledge and skills retention following Emergency Triage, Assessment and Treatment plus Admission course for final year medical students in Rwanda: a longitudinal cohort study 
Archives of Disease in Childhood  2014;99(11):993-997.
Aim
To determine whether, after the Emergency Triage, Assessment and Treatment plus Admission (ETAT+) course, a comprehensive paediatric life support course, final year medical undergraduates in Rwanda would achieve a high level of knowledge and practical skills and if these were retained. To guide further course development, student feedback was obtained.
Methods
Longitudinal cohort study of knowledge and skills of all final year medical undergraduates at the University of Rwanda in academic year 2011–2012 who attended a 5-day ETAT+ course. Students completed a precourse knowledge test. Knowledge and clinical skills assessments, using standardised marking, were performed immediately postcourse and 3–9 months later. Feedback was obtained using printed questionnaires.
Results
84 students attended the course and re-evaluation. Knowledge test showed a significant improvement, from median 47% to 71% correct answers (p<0.001). For two clinical skills scenarios, 98% passed both scenarios, 37% after a retake, 2% failed both scenarios. Three to nine months later, students were re-evaluated, median score for knowledge test 67%, not significantly different from postcourse (p>0.1). For clinical skills, 74% passed, with 32% requiring a retake, 8% failed after retake, 18% failed both scenarios, a significant deterioration (p<0.0001).
Conclusions
Students performed well on knowledge and skills immediately after a comprehensive ETAT+ course. Knowledge was maintained 3–9 months later. Clinical skills, which require detailed sequential steps, declined, but most were able to perform them satisfactorily after feedback. The course was highly valued, but several short courses and more practical teaching were advocated.
doi:10.1136/archdischild-2014-306078
PMCID: PMC4198299  PMID: 24925893
Medical Education; Accident & Emergency; Resuscitation; Low income populations; Rwanda
19.  The validity of Iran’s national university entrance examination (Konkoor) for predicting medical students’ academic performance 
BMC Medical Education  2012;12:60.
Background
In Iran, admission to medical school is based solely on the results of the highly competitive, nationwide Konkoor examination. This paper examines the predictive validity of Konkoor scores, alone and in combination with high school grade point averages (hsGPAs), for the academic performance of public medical school students in Iran.
Methods
This study followed the cohort of 2003 matriculants at public medical schools in Iran from entrance through internship. The predictor variables were Konkoor total and subsection scores and hsGPAs. The outcome variables were (1) Comprehensive Basic Sciences Exam (CBSE) scores; (2) Comprehensive Pre-Internship Exam (CPIE) scores; and (3) medical school grade point averages (msGPAs) for the courses taken before internship. Pearson correlation and regression analyses were used to assess the relationships between the selection criteria and academic performance.
Results
There were 2126 matriculants (1374 women and 752 men) in 2003. Among the outcome variables, the CBSE had the strongest association with the Konkoor total score (r = 0.473), followed by msGPA (r = 0.339) and the CPIE (r = 0.326). While adding hsGPAs to the Konkoor total score almost doubled the power to predict msGPAs (R2 = 0.225), it did not have a substantial effect on CBSE or CPIE prediction.
Conclusions
The Konkoor alone, and even in combination with hsGPA, is a relatively poor predictor of medical students’ academic performance, and its predictive validity declines over the academic years of medical school. Care should be taken to develop comprehensive admissions criteria, covering both cognitive and non-cognitive factors, to identify the best applicants to become "good doctors" in the future. The findings of this study can be helpful for policy makers in the medical education field.
doi:10.1186/1472-6920-12-60
PMCID: PMC3461424  PMID: 22840211
20.  Poorer verbal working memory for a second language selectively impacts academic achievement in university medical students 
PeerJ  2013;1:e22.
Working memory (WM) is often poorer for a second language (L2). In low noise conditions, people listening to a language other than their first language (L1) may have similar auditory perception skills for that L2 as native listeners, but do worse in high noise conditions, and this has been attributed to the poorer WM for L2. Given that WM is critical for academic success in children and young adults, these speech in noise effects have implications for academic performance where the language of instruction is L2 for a student. We used a well-established Speech-in-Noise task as a verbal WM (vWM) test, and developed a model correlating vWM and measures of English proficiency and/or usage to scholastic outcomes in a multi-faceted assessment medical education program. Significant differences in Speech-Noise Ratio (SNR50 ) values were observed between medical undergraduates who had learned English before or after five years of age, with the latter group doing worse in the ability to extract whole connected speech in the presence of background multi-talker babble (Student-t tests, p < 0.001). Significant negative correlations were observed between the SNR50 and seven of the nine variables of English usage, learning styles, stress, and musical abilities in a questionnaire administered to the students previously. The remaining two variables, Perceived Stress Scale (PSS) and the Age of Acquisition of English (AoAoE) were significantly positively correlated with the SNR50 , showing that those with a poorer capacity to discriminate simple English sentences from noise had learnt English later in life and had higher levels of stress – all characteristics of the international students. Local students exhibited significantly lower SNR50 scores and were significantly younger when they first learnt English. No significant correlation was detected between the SNR50 and the students’ Visual/Verbal Learning Style (r = −0.023). Standard multiple regression was carried out to assess the relationship between language proficiency and verbal working memory (SNR50 ) using 5 variables of L2 proficiency, with the results showing that the variance in SNR50 was significantly predicted by this model (r2 = 0.335). Hierarchical multiple regression was then used to test the ability of three independent variable measures (SNR50 , age of acquisition of English and English proficiency) to predict academic performance as the dependent variable in a factor analysis model which predicted significant performance differences in an assessment requiring communications skills (p = 0.008), but not on a companion assessment requiring knowledge of procedural skills, or other assessments requiring factual knowledge. Thus, impaired vWM for an L2 appears to affect specific communications-based assessments in university medical students.
doi:10.7717/peerj.22
PMCID: PMC3628612  PMID: 23638357
Medical education; International students; Working memory; Speech in noise; Assessment; OSCE; English as a second language
21.  Training of patient and consumer representatives in the basic competencies of evidence-based medicine: a feasibility study 
BMC Medical Education  2010;10:16.
Background
Evidence-based medicine (EBM) has become standard approach in medicine. Patients and health authorities increasingly claim active patient roles in decision making. Education to cope with these roles might be useful. We investigated the feasibility, acceptability and possible impact of EBM training courses for patient and consumer representatives.
Methods
We designed a generic one-week EBM course based on previous experience with EBM courses for non-medical health professionals. A course specific competence test has been developed and validated to measure EBM skills. Formative and summative evaluation of the course comprised: 1) EBM skills; 2) individual learning goals; 3) self-reported implementation after six months using semi-structured interviews; 4) group-based feedback by content analysis. EBM skills' achievement was compared to results gathered by a group of undergraduate University students of Health Sciences and Education who had attended a comparable EBM seminar.
Results
Fourteen EBM courses were conducted including 161 participants without previous EBM training (n = 54 self-help group representatives, n = 64 professional counsellors, n = 36 patient advocates, n = 7 others); 71% had a higher education degree; all but five finished the course. Most participants stated personal learning goals explicitly related to practicing EBM such as acquisition of critical appraisal skills (n = 130) or research competencies (n = 67). They rated the respective relevance of the course on average with 80% (SD 4) on a visual analogue scale ranging from 0 to 100%.
Participants passed the competence test with a mean score of 14.7 (SD 3.0, n = 123) out of 19.5 points. The comparison group of students achieved a mean score of 14.4 (SD 3.3, n = 43). Group-based feedback revealed increases of self confidence, empowerment through EBM methodology and statistical literacy, and acquisition of new concepts of patient information and counselling. Implementation of EBM skills was reported by 84 of the 129 (65%) participants available for follow-up interviews. Barriers included lack of further support, limited possibilities to exchange experiences, and feeling discouraged by negative reactions of health professionals.
Conclusions
Training in basic EBM competencies for selected patient and consumer representatives is feasible and accepted and may affect counselling and advocacy activities. Implementation of EBM skills needs support beyond the training course.
doi:10.1186/1472-6920-10-16
PMCID: PMC2843725  PMID: 20149247
22.  Predictive Value of the School-leaving Grade and Prognosis of Different Admission Groups for Academic Performance and Continuity in the Medical Course – a Longitudinal Study 
Background: The school-leaving GPA and the time since completion of secondary education are the major criteria for admission to German medical schools. However, the predictive value of the school-leaving grade and the admission delay have not been thoroughly examined since the amendment of the Medical Licensing Regulations and the introduction of reformed curricula in 2002. Detailed information on the prognosis of the different admission groups is also missing.
Aim: To examine the predictive values of the school-leaving grade and the age at enrolment for academic performance and continuity throughout the reformed medical course.
Methods: The study includes the central admission groups “GPA-best” and “delayed admission” as well as the primary and secondary local admission groups of three consecutive cohorts. The relationship between the criteria academic performance and continuity and the predictors school-leaving GPA, enrolment age, and admission group affiliation were examined up to the beginning of the final clerkship year.
Results: The academic performance and the prolongation of the pre-clinical part of undergraduate training were significantly related to the school-leaving GPA. Conversely, the dropout rate was related to age at enrolment. The students of the GPA-best group and the primary local admission group performed best and had the lowest dropout rates. The students of the delayed admission group and secondary local admission group performed significantly worse. More than 20% of these students dropped out within the pre-clinical course, half of them due to poor academic performance. However, the academic performance of all of the admission groups was highly variable and only about 35% of the students of each group reached the final clerkship year within the regular time.
Discussion: The school-leaving grade and age appear to have different prognostic implications for academic performance and continuity. Both factors have consequences for the delayed admission group. The academic prognosis of the secondary local admission group is as problematic as that of the delayed admission group. Additional admission instruments would be necessary, in order to recognise potentially able applicants independently of their school-leaving grade and to avoid the secondary admission procedure.
doi:10.3205/zma000913
PMCID: PMC4027806  PMID: 24872856
Medical studies; admission; GPA; school leaving grades
23.  Admissions processes for five year medical courses at English schools: review 
BMJ : British Medical Journal  2006;332(7548):1005-1009.
Objective To describe the current methods used by English medical schools to identify prospective medical students for admission to the five year degree course.
Design Review study including documentary analysis and interviews with admissions tutors.
Setting All schools (n = 22) participating in the national expansion of medical schools programme in England.
Results Though there is some commonality across schools with regard to the criteria used to select future students (academic ability coupled with a “well rounded” personality demonstrated by motivation for medicine, extracurricular interests, and experience of team working and leadership skills) the processes used vary substantially. Some schools do not interview; some shortlist for interview only on predicted academic performance while those that shortlist on a wider range of non-academic criteria use various techniques and tools to do so. Some schools use information presented in the candidate's personal statement and referee's report while others ignore this because of concerns over bias. A few schools seek additional information from supplementary questionnaires filled in by the candidates. Once students are shortlisted, interviews vary in terms of length, panel composition, structure, content, and scoring methods.
Conclusion The stated criteria for admission to medical school show commonality. Universities differ greatly, however, in how they apply these criteria and in the methods used to select students. Different approaches to admissions should be developed and tested.
doi:10.1136/bmj.38768.590174.55
PMCID: PMC1450044  PMID: 16543300
24.  The UKCAT-12 study: educational attainment, aptitude test performance, demographic and socio-economic contextual factors as predictors of first year outcome in a cross-sectional collaborative study of 12 UK medical schools 
BMC Medicine  2013;11:244.
Background
Most UK medical schools use aptitude tests during student selection, but large-scale studies of predictive validity are rare. This study assesses the United Kingdom Clinical Aptitude Test (UKCAT), and its four sub-scales, along with measures of educational attainment, individual and contextual socio-economic background factors, as predictors of performance in the first year of medical school training.
Methods
A prospective study of 4,811 students in 12 UK medical schools taking the UKCAT from 2006 to 2008 as a part of the medical school application, for whom first year medical school examination results were available in 2008 to 2010.
Results
UKCAT scores and educational attainment measures (General Certificate of Education (GCE): A-levels, and so on; or Scottish Qualifications Authority (SQA): Scottish Highers, and so on) were significant predictors of outcome. UKCAT predicted outcome better in female students than male students, and better in mature than non-mature students. Incremental validity of UKCAT taking educational attainment into account was significant, but small. Medical school performance was also affected by sex (male students performing less well), ethnicity (non-White students performing less well), and a contextual measure of secondary schooling, students from secondary schools with greater average attainment at A-level (irrespective of public or private sector) performing less well. Multilevel modeling showed no differences between medical schools in predictive ability of the various measures. UKCAT sub-scales predicted similarly, except that Verbal Reasoning correlated positively with performance on Theory examinations, but negatively with Skills assessments.
Conclusions
This collaborative study in 12 medical schools shows the power of large-scale studies of medical education for answering previously unanswerable but important questions about medical student selection, education and training. UKCAT has predictive validity as a predictor of medical school outcome, particularly in mature applicants to medical school. UKCAT offers small but significant incremental validity which is operationally valuable where medical schools are making selection decisions based on incomplete measures of educational attainment. The study confirms the validity of using all the existing measures of educational attainment in full at the time of selection decision-making. Contextual measures provide little additional predictive value, except that students from high attaining secondary schools perform less well, an effect previously shown for UK universities in general.
doi:10.1186/1741-7015-11-244
PMCID: PMC3827332  PMID: 24229380
Medical student selection; Educational attainment; Aptitude tests; UKCAT; Socio-economic factors; Contextual measures
25.  Requirements for admission to medical school: how many years of university study are necessary? 
OBJECTIVE: To assess whether students admitted to medical school after completing 2 years of undergraduate study performed as well as those admitted after longer periods of undergraduate study in terms of broad patient-care skills measured at the time of graduation. DESIGN: Retrospective study. SETTING: University of Alberta, Edmonton. PARTICIPANTS: Graduates of the classes of 1990 and 1991, of the 226 graduates 133 had entered medical school after 2 years of undergraduate training, 39 after 3 years and 54 after 4 or more years. Eight students had been excluded because they were either transfer students or international students. OUTCOME MEASURES: Objective and subjective assessments of the main clinical rotations (internal medicine, obstetrics and gynecology, pediatrics, psychiatry, radiology and surgery), results of the faculty's final comprehensive examination and of the Medical Council of Canada's Qualifying Examination. RESULTS: The students who had completed 2 years of undergraduate study before medical school were significantly younger than those who had completed 3 years and those who had completed 4 or more years (mean age [and standard deviation (SD)] 20.5 [2.1], 21.5 [2.4] and 25.1 [4.4] years respectively, p < 0.001). They also had a significantly higher mean grade point average (GPA) for the prerequisite courses for admission to medical school than those with 3 years and those with 4 or more years of undergraduate study (8.26 [SD 0.3], 7.95 [SD 0.3] and 7.80 [SD 0.5] respectively, p < 0.001). The overall mean GPA for the best 2 years of undergraduate study did not differ significantly between the three groups. The students with 2 years of undergraduate study had a significantly lower mean score for the pre-entry interview than those who had 4 or more years of undergraduate study (32.1 [SD 7.6] v. 38.3 [SD 8.5], p < 0.001). There were no significant differences between the three groups in the results of any of the subjective or objective outcome measures. CONCLUSION: Students who completed 2 years of undergraduate study before admission to medical school were able to achieve a satisfactory level of competency and maturity by the end of medical school. The 2-year option for entrance into medical school should be reconsidered.
PMCID: PMC1488032  PMID: 7489552

Results 1-25 (1223278)