Our main finding was that the interview procedure seems to reduce the selection of students with inferior communication capability, as assessed at fourth year, in spite of the training in communication skills and other professional behaviour during medical school. However, interview selection did not prove better than traditional selection criteria at distinguishing the students with superior performance.
We did not include analyses of any association between the admission interview ratings and outcome variables, since the variation in admission ratings was minimal in this selected group.
The findings that academic results (= the written test) did not differ between groups and the moderate correlations with GPA, suggest that top GPA from upper secondary school may not be necessary for successful medical studies. This was already shown in a previous evaluation of the different selection pathways at KI
] and is in line with experience from other medical schools
Studies evaluating selection interviews have used different outcome measures, different time frames and different methods. The value of interview admissions is controversial, and seems to depend on the type of interview and level of experience among interviewers
]. Still, many medical schools use interviews to assess non-cognitive qualities in applicants. Most studies evaluating interview selection have been performed at institutions where all students are selected by the same standards, thus, within-group correlations are usually the employed analyses
]. Since interviews are not used for all admissions at KI, we were able to compare students that had been admitted as a result of their performance at a selection interview, to those who had not and, in addition, we chose to identify end-points of low and high performers, respectively, which, although it limits variance, yields a measure of certain face-validity.
Many instruments designed to evaluate communication behaviour have been published. Some of these are mainly descriptive in nature
], some are evaluative
] while others combine these two elements
]. Having carefully analysed video recordings of 16 randomly chosen students we decided to create the evaluation model described in Methods. The internal consistency of the scale was good and the inter-rater reliability was considered acceptable. The cut point chosen for under- and superior performers had not been validated. The cut point for underperforming (<10) was chosen to accord with the general pass level for summative assessments at KI. The choice may indicate some arbitrariness, which is why we tested the effect of two alternative cut points. These analyses showed that also with a more conservative cut point of 9, the effect of interview admission in reducing underperformers was significant, and even more pronounced. With a less conservative cut point of 11, there was no significant effect of interview admission. However, it is doubtful that a cut point that is so close to the norm (median 12, IQR 10.6-13.2 for interviews and 11.8 IQR 10.2-13.2 for academic merits) bears significance in defining underperformance.
There are several limitations to the study. There were significantly more women in the excluded group than in the study group, while admission type and age were similar. They were excluded because the recordings of the OSCE’s were incomplete and thus unratable; there is no reason to assume that failure of recording would be systematically related to the outcome or to gender. Moreover, since we did not find an interaction between gender and admission pathway with respect to inferior communication skills, we do not think that the exclusion affected the results in any systematic way. Also, this was a retrospective study, hence we were not able to examine the attrition rate from different admission pathways. There may thus be a selection bias, where the least able students, both regarding academic results and interpersonal skills, have already left medical school or been delayed in their studies. However, we found no effect of delay among the assessed students, neither on the communication skills score variable, nor for underperforming. Further, only one OSCE station was studied, having included more would have strengthened the reliability of performance rating. However, the other stations were of too short duration or did not include communication skills to a degree sufficient for rating. There were no additional sources available for the assessments of students’ communication skills.
The interview selection process is time-consuming and expensive
] and adequate evaluations of its merits are crucial. Communication skills are a vital part of medical professional competence and this study indicates that the proportion of students with poor communication skills can be reduced with an interview based selection process.
The ranking at the KI interview selection process is designed for selection of the best students, in accordance with the legal frame-work in Sweden, which since 2008 does not allow for universities to employ negative selection, i.e. sorting out the unwanted among applicants, which has otherwise been claimed as a purpose of admission interviews
]. In spite of this allegation, we found that interview selection rather had the effect of reducing suboptimal performance in such an important area as patient-doctor communication from 18 to 7 per cent. Further studies should evaluate whether the interview process enables a broader recruitment of students with regard to diversity of ethnicity and social economic status.