The main aims of introducing the UKCAT were to provide schools with a tool that would offer a more objective and fairer method of discriminating between academically high achieving applicants, and support widening participation initiatives. A subsidiary, but very important aim, was to establish the predictive validity of these tests in identifying successful doctors. In the first year the schools were unfamiliar with the test and its psychometric characteristics and were concerned not to use this new tool heavy-handedly. Many schools therefore chose the borderline method because it would affect the outcome for only a small number of applicants where there was no other logical approach to use. The factor method was also seen as a fair but 'light-touch' means of using the UKCAT score as part of the selection decision.
The threshold method, like the factor method, ensured every applicant's UKCAT score contributed to the offer/reject decision. The threshold method was used in 2006 by only one school, where it was intended as a 'widening participation' tool, because the school at the same time reduced the level of academic pre-requisites. Most schools using this method by 2009 were attracted by the opportunity for speedier and more efficient selection of applicants for interview, by postponing detailed consideration of applicants' UCAS form until a later stage. This may be justifiable in the light of studies [5
] suggesting that the UCAS personal statement is a poor predictor of the subsequent medical performance of students once they have been selected. However, reducing the importance of the UCAS form represents a shift from the popularly understood use of this information in selection for medical school.
The rescue trade-off method was used to increase the pool of applicants invited to interview, for example by including those with high UKCAT scores (indicating high ability) but weaker UCAS applications (perhaps because of lack of advice in preparing their personal statement). This method thus appears the most explicit 'widening participation' use of the UKCAT score. In keeping with the schools' desire not to penalise applicants on the basis of their UKCAT score, it is noteworthy that no school employed the alternative strategy of a 'reject' trade-off, whereby a low UKCAT score would lead to the rejection of an applicant who would formerly have been selected on the basis of the school's usual assessment score.
Many schools had decided to keep their use of the UKCAT scores constant for the first few years and then review their experience. As familiarity grew with the UKCAT, schools increased the number and combinations of ways in which they used the test, or the weight the results were given. Some schools reported local analyses had indicated UKCAT scores mirrored the outcomes of their existing selection methods and consequently their confidence in the use of the test. At the same time the Consortium acted as a forum for sharing innovation and best practice, helping members to develop and change their use of the test.
Many schools reported that the UKCAT was particularly helpful as part of the assessment of non-traditional applicants such as overseas students or mature students, and in the assessment of applicants for medical courses other than the standard five year course, such as Foundation or six year programmes. Where candidates in these groups might be offering a variety of non-traditional qualifications, the UKCAT provides a standardized assessment for comparison.
An understanding of how schools have used the test and how this use has changed since UKCAT's inception is important for those researching the impact of the test on the demographics of medical admissions and the predictive validity of the test. The impact of the use of the test on widening participation in medical admissions is currently being investigated. The use of different methods may well restrict or widen the range of UKCAT scores available which is important for subsequent research into predictive validity.
The findings of this study, with 100% completion in each year, are important for informing debate on medical admissions, especially in the UK. They also highlight the considerable variation in practice which has now emerged and, therefore, the importance of transparency on the part of schools and informed decision making by applicants once they know their UKCAT score. UKCAT will remain contentious unless or until convincing evidence emerges regarding predictive validity but already some medical schools are choosing to place more reliance on the UKCAT score than on the UCAS form statements. This is because there is insufficient evidence of the predictive power of the UCAS statement [7
], because assessors have concerns that unequal levels of support provided to applicants in writing the statement, and because of the difficulties of detecting plagiarism and deception. The UKCAT Consortium has commissioned an initial study into the predictive validity of the cognitive sections of the test,