The 2007 entry cohort and the study group
260 students commenced the 5-year undergraduate course in October 2007. The study group comprised 204 (78%) who had taken the UKCAT and given consent for their data to be used. In the non-study group, 10 had taken the UKCAT but not given consent, and the remaining 46 had not taken the test. (30 of these students were deferred entries from 2006. Of the remaining 16, seven were enrolled automatically after completing a Foundation programme, two were re-starting the course, six were Thai students completing a parallel course, and one had been exempted for reasons not known to us). Table summarises the socio-demographic characteristics of the study and non-study groups. There were no significant differences between the two groups (Chi-square tests). Since almost all students were aged under 21, the variable for 'maturity' was not used in subsequent analyses.
Socio-demographic characteristics of the 2007 entry cohort
The UKCAT scores (mean and SD) for the study group were:
Verbal Reasoning 629 ± 72
Quantitative Reasoning 637 ± 61
Abstract Reasoning 637 ± 74
Decision Analysis 643 ± 94
Total score 2543 ± 198
Recent school examination (A-level) results were known for 193 (95%) of the study group. The remaining 11 had taken the International Baccalaureate (6), had a previous degree (2), or had other qualifications (3). Of the 193, 154 (80%) had obtained an A grade for all their subjects and therefore had an average tariff score of 120. Of the remaining 39, only 2 had an average tariff of less than 110. We therefore did not use the A-level tariff as a predictor variable in this study, since it would have had little discriminatory ability.
Full examination marks for the first two years of study were available for 195/204 students. Of the remaining nine, four had transferred out of the medical course voluntarily to study other subjects, three had transferred within the course to the BSc degree, and two had not taken all their examinations for other reasons, such as illness.
Correlation between UKCAT scores and course progress
We first examined the correlation between Theme marks in Year 1 and Year 2. The correlation matrix is shown in Table and shows a highly significant relationship (r = 0.3 - 0.8, p < 0.001) between marks for each Theme across the two years. We therefore used the overall Theme average for the remaining analysis.
Correlation matrix between Theme averages in Year 1 and Year 2
Table shows the correlation matrix between the overall Theme averages and the UKCAT scores. There were statistically significant relationships between sub-tests of the UKCAT, particularly Verbal with Quantitative Reasoning (p = 0.002), and Abstract Reasoning and Decision Analysis (p < 0.001). However, the correlation coefficient was less than 0.3 in all cases.
Correlation matrix between UKCAT scores and Theme averages for the first two years
Within the Themes alone there were stronger correlations (p < 0.001 in all cases). The coefficients were large between the knowledge-based assessments (A and B, r = 0.87) and weakest between Theme A and the OSCE (r = 0.27).
There were only three modest correlations between the UKCAT sub-tests and the Theme marks: Verbal Reasoning and Theme A and Theme C, and Quantitative Reasoning with Theme A. These were relatively weak (r = 0.32 or less). There were no significant correlations between UKCAT total score and the Themes.
Univariate analysis of socio-demographic variables against UKCAT scores and course progress
We used t-tests to examine the effects of socio-demographic variables (sex, ethnicity, domicile and schooling) on UKCAT scores and Theme averages. The Bonferroni correction for multiple comparisons would suggest that significance values larger than p = 0.01 are not of practical importance.
Table summarises the few statistically significant differences that were found. The total UKCAT score was little affected by socio-demographic variables, with a weak positive influence of Home domicile and White ethnicity. There were scattered effects on sub-scores. On the course, Theme C was the most affected, with poorer performance by males but a positive influence of White ethnicity and Home domicile.
Significant univariate effects (t-tests) of socio-demographic variables on UKCAT scores and Theme averages (Year 1 plus Year 2)
Table summarises the statistically significant results from the hierarchical multivariate linear regressions. All results significant at p < 0.05 are shown, although those with p > 0.01 are unlikely to be of practical importance as described above. As expected from the univariate analyses, there were few independent predictors of Theme scores.
Significant independent predictors of course performance (hierarchical multivariate linear regression analysis)
In the upper part of Table , UKCAT total scores are used in Block 2. It is evident that neither socio-demographic variables nor UKCAT have substantial predictive value for the Theme averages, with the exception of Theme C, in which male sex has a strong negative influence and White ethnicity a positive one. Further examination of the data showed that these differences lay primarily in the Behavioural Sciences module (year 1) and Epidemiology in Practice (year 2) respectively (p < 0.001 in both cases, data not shown). The UKCAT total score has an additional weak positive relationship with Themes A and C and adds a small amount of variance to the model.
In the lower part of Table , UKCAT sub-test scores are used in Block 2. Of note are the influences of Quantitative Reasoning in Theme A and Verbal Reasoning in Theme C, with an additional weak effect of Verbal Reasoning in Theme A. Male sex is a strong negative predictor in Theme C, but otherwise the effects of socio-demographic variables are modest.