PMCC PMCC

Search tips
Search criteria

Advanced
Results 1-3 (3)
 

Clipboard (0)
None

Select a Filter Below

Journals
Authors
more »
Year of Publication
Document Types
1.  Predictive validity of the UKCAT for medical school undergraduate performance: a national prospective cohort study 
BMC Medicine  2016;14:140.
Background
The UK Clinical Aptitude Test (UKCAT) has been shown to have a modest but statistically significant ability to predict aspects of academic performance throughout medical school. Previously, this ability has been shown to be incremental to conventional measures of educational performance for the first year of medical school. This study evaluates whether this predictive ability extends throughout the whole of undergraduate medical study and explores the potential impact of using the test as a selection screening tool.
Methods
This was an observational prospective study, linking UKCAT scores, prior educational attainment and sociodemographic variables with subsequent academic outcomes during the 5 years of UK medical undergraduate training. The participants were 6812 entrants to UK medical schools in 2007–8 using the UKCAT. The main outcome was academic performance at each year of medical school. A receiver operating characteristic (ROC) curve analysis was also conducted, treating the UKCAT as a screening test for a negative academic outcome (failing at least 1 year at first attempt).
Results
All four of the UKCAT scale scores significantly predicted performance in theory- and skills-based exams. After adjustment for prior educational achievement, the UKCAT scale scores remained significantly predictive for most years. Findings from the ROC analysis suggested that, if used as a sole screening test, with the mean applicant UKCAT score as the cut-off, the test could be used to reject candidates at high risk of failing at least 1 year at first attempt. However, the ‘number needed to reject’ value would be high (at 1.18), with roughly one candidate who would have been likely to pass all years at first sitting being rejected for every higher risk candidate potentially declined entry on this basis.
Conclusions
The UKCAT scores demonstrate a statistically significant but modest degree of incremental predictive validity throughout undergraduate training. Whilst the UKCAT could be considered a fairly crude screening tool for future academic performance, it may offer added value when used in conjunction with other selection measures. Future work should focus on the optimum role of such tests within the selection process and the prediction of post-graduate performance.
Electronic supplementary material
The online version of this article (doi:10.1186/s12916-016-0682-7) contains supplementary material, which is available to authorized users.
doi:10.1186/s12916-016-0682-7
PMCID: PMC5026770  PMID: 27638740
Predictive validity; Medical selection; Undergraduate performance; Aptitude testing
2.  Evaluating professionalism in medical undergraduates using selected response questions: findings from an item response modelling study 
BMC Medical Education  2011;11:43.
Background
Professionalism is a difficult construct to define in medical students but aspects of this concept may be important in predicting the risk of postgraduate misconduct. For this reason attempts are being made to evaluate medical students' professionalism. This study investigated the psychometric properties of Selected Response Questions (SRQs) relating to the theme of professional conduct and ethics comparing them with two sets of control items: those testing pure knowledge of anatomy, and; items evaluating the ability to integrate and apply knowledge ("skills"). The performance of students on the SRQs was also compared with two external measures estimating aspects of professionalism in students; peer ratings of professionalism and their Conscientiousness Index, an objective measure of behaviours at medical school.
Methods
Item Response Theory (IRT) was used to analyse both question and student performance for SRQs relating to knowledge of professionalism, pure anatomy and skills. The relative difficulties, discrimination and 'guessabilities' of each theme of question were compared with each other using Analysis of Variance (ANOVA). Student performance on each topic was compared with the measures of conscientiousness and professionalism using parametric and non-parametric tests as appropriate. A post-hoc analysis of power for the IRT modelling was conducted using a Monte Carlo simulation.
Results
Professionalism items were less difficult compared to the anatomy and skills SRQs, poorer at discriminating between candidates and more erratically answered when compared to anatomy questions. Moreover professionalism item performance was uncorrelated with the standardised Conscientiousness Index scores (rho = 0.009, p = 0.90). In contrast there were modest but significant correlations between standardised Conscientiousness Index scores and performance at anatomy items (rho = 0.20, p = 0.006) though not skills (rho = .11, p = .1). Likewise, students with high peer ratings for professionalism had superior performance on anatomy SRQs but not professionalism themed questions. A trend of borderline significance (p = .07) was observed for performance on skills SRQs and professionalism nomination status.
Conclusions
SRQs related to professionalism are likely to have relatively poor psychometric properties and lack associations with other constructs associated with undergraduate professional behaviour. The findings suggest that such questions should not be included in undergraduate examinations and may raise issues with the introduction of Situational Judgement Tests into Foundation Years selection.
doi:10.1186/1472-6920-11-43
PMCID: PMC3146946  PMID: 21714870
3.  Exploring reflective 'critical incident' documentation of professionalism lapses in a medical undergraduate setting 
Background
Measuring professionalism in undergraduate medical students is a difficult process, and no one method has currently emerged as the definitive means of assessment in this field. Student skills in reflection have been shown to be highly important in the development of professional behaviours. By studying student reflections on lapses in professional judgement, recorded as 'critical incidents', it is possible to explore themes which are significant for the development of professional behaviour in an undergraduate setting.
Methods
We examined critical incident reporting combined with optional written student reflection as a method for exploring professionalism in undergraduate medical students. 228 students split between Year 1 and 2 of one academic year of undergraduate medicine were studied retrospectively and a grounded theory approach to analysis was employed.
Results
This year generated 16 critical incident reports and corresponding student reflections, all of which were considered. In addition to identifying the nature of the critical incidents, 3 principal themes emerged. These were the impact and consequences of the report having been made, student reactions to the events (both positive and negative), and student responses regarding future actions.
Conclusion
This study indicates that unprofessional behaviour can be identified and challenged by both the faculty and the students involved, and suggests that positive behavioural changes might be made with the aim of preventing future occurrences. We provide a low cost approach of measuring and recording professional behaviour.
doi:10.1186/1472-6920-9-44
PMCID: PMC2714848  PMID: 19604378

Results 1-3 (3)