PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of jrsocmedLink to Publisher's site
 
J R Soc Med. 2002 January; 95(1): 18–22.
PMCID: PMC1279142

Can we improve on how we select medical students?

Patricia Hughes, MSc FRCPsych

Getting the right policy for admission to medical school is a balancing act: be fair to society by choosing people with the potential to be good doctors; and be fair to the applicants—that diverse group of people who for many reasons want to set out on the long road to a medical career. Selection is not an exact science but we must use what evidence we have to ensure that we do our best by all concerned. There is widespread agreement that we should select future doctors on wider criteria than scores of academic success1,2, though in practice many medical schools have valued pre-admission academic scores at the expense of other considerations3. There are recognized drawbacks to the use of school exam performance even as a measure of intellectual competence. One study has shown that a major causal determinant of A level results is social class, independent of ability4, and some would-be medical students elect to focus on sciences for their school leaving exams because very high marks are more easily achieved in the physical sciences than in the humanities5. The conviction that only exam results give valid and reliable data has been trenchantly dismissed as a ‘seductive but fallacious’ belief in the precision of quantitative tests6. We are reminded that all selective instruments depend on subjective judgments and each must be accountable to the rules of reason, fairness and public scrutiny7. However, if we decide to consider non-cognitive criteria, a legitimate concern is that the many specialties of medicine need diverse skills and they must not be too narrow. We also want to be reassured, if we include noncognitive characteristics, that we can assess them reliably and that such evaluation can predict personal character over years of practice.

While we need to maintain diversity of skills and personality, there are some characteristics which we demand in any doctor. Enough intellectual ability to do the job, plus honesty, integrity and conscientiousness, must be at the heart of good practice8. Helpfulness and willingness to cooperate come close behind8, while patients give high priority to interpersonal skills and empathy2. The personal welfare of the profession is another consideration9. Doctors are more vulnerable than comparable professional groups to alcoholism, drug abuse and suicide10,11. Burnout is well recognized, and has a high cost for the individual, for colleagues and for the quality of service that patients get12. One answer may be better support for psychologically vulnerable doctors12,13 (together with improved working conditions for all doctors), but perhaps we should try to evaluate ability to deal with stress right from the start.

ARE PERSONALITY CHARACTERISTICS STABLE OVER ADULT LIFE?

If we seek to identify the personal characteristics we want in a medical student, can we have any confidence that they tell us anything about future personality or adjustment? Studies that assessed medical undergraduates and followed them up for between 15-30 years12,14,15 indicate that doctors who are psychologically well in middle age had good self esteem as students, had an open, flexible approach to life, enjoyed a warm relationship with their parents, and had little anxiety and depression and low anger under stress. In contrast, doctors vulnerable to later substance abuse, to suicide and to burnout in middle age had significantly poorer measures of psychological health as undergraduates. Other long-term studies of stability of personality characteristics have shown that personality traits exhibit high test-retest correlations over intervals of 6 to 45 years16,17,18,19. These findings signify a substantial continuity of personality disposition in adulthood, suggesting a stable tendency to be either happy or unhappy, well or poorly adjusted.

WHAT FACTORS GENERALLY PREDICT FUTURE JOB PERFORMANCE?

There is relevant research both within medicine and outside it. Useful information comes from industry, where serious money has gone into finding out what makes a good professional20. They measure outcome in hard cash and find that the most productive people are about 40% better than average, while the least are 40% worse than average21. Is this too different from medicine to be relevant? Look around: we all know who gets the work done and keeps up to date, and who slips through life doing the minimum. These are not the only criteria for a decent doctor but they matter. There is consistent evidence that, for work involving complex tasks, the best predictor of effectiveness is some measure of mental ability or IQ, and the higher you go up the professional scale the more IQ matters. At the highest managerial level it accounts for almost 70% of performance variability22. So in demanding evidence of high IQ (even in the form of exam results) we have got something right. Predictability can be improved by including some measure of other factors. Further factors consistently found to add to prediction of performance are integrity and conscientiousness: these do not correlate with IQ23. No additional predictability comes from the number or nature of outside interests; years of education adds little to predictive validity; and the number of courses a person has been on is of no value (so much for how we measure ‘continuing professional development’). Previous job performance adds to prediction for those already in the profession, but adds nothing at entry. Some of these results are counter-intuitive: this is because IQ overlaps with other things. So a quick learner will have good performance in a previous job which will correlate so highly with IQ that it adds little to predictive validity20.

WHAT FACTORS PREDICT ACADEMIC FAILURE IN MEDICINE?

The first thing that strikes anyone exploring the work on predictors in medicine is that we are obsessed with exam results: by far the largest number of papers examines predictors of passing exams. This may be justified because of the economic and personal waste of losing students who begin a medical degree but fail to complete, with loss from schools that select at entry, both in the UK and elsewhere, generally reported between 8% and 10%24,25,26,27. However, most studies assess ‘failure’ in broad terms to include all students who re-take an examination, as well as those who are excluded from the course, so predictors should be treated with caution. Although virtually all students are high academic achievers at school, from the top 0.4%8 to the top 10%29, school and medical exam scores do correlate, with contribution to variability reported between 16%29 and 58%30. Some UK studies show that certain science A levels predict exam success, variously putting biology, chemistry or physics in prime place31,32,33, and research from outside the UK reports associations between performance in physical sciences and in medical exams34,35,36. Generally this association falls later in the course, with no difference to longer term success or failure37,38,39,40.

Non-academic factors also predict exam success or failure. Some researchers report that older students are more likely to fail exams36,38,41, but others have not found this42. Several US studies found higher failure rates among women and ethnic minority students, although most eventually graduate36,38,41, and one school reported that students admitted through affirmative action were as likely to graduate as those admitted by use of traditional criteria43. Proficiency in English is important for students for whom English is not their first language44,45, and in the US, reading skills of disadvantaged minority students have been shown to predict academic success46. Non-cognitive factors are stronger predictors for women and ethnic minority students than for white men in the US. For women, interview ratings and previous relevant experience were more predictive than previous exam scores47, while for ethnic minority students, locus of control and ability to self-evaluate were predictors48,49. One US study showed that different cognitive and non-cognitive factors correlate with academic success in different schools, so different cultures and teaching styles influence outcome50.

It has been argued that we cannot reduce loss further51, because some failure is inevitable and we cannot avoid a few students' wanting to change career. However, two medical schools have shown that careful selection and good support can have a positive impact. In Newcastle, New South Wales, for five years 50% of students were selected on academic marks alone but underwent a lengthy structured interview which was not used for selection. As a result, some students were admitted with very low interview scores. The remaining 50% were selected from a wider band of academic performance but scored high in interview. Analysis after ten years showed a significant correlation between low interview score and later drop-out but no correlation between academic score at entry and drop-out. Reasons for dropping out were academic failure or a variety of personal reasons, including lack of motivation for study or for medicine28. Another example of low drop-out comes from McMaster University in Ontario, which also invests heavily in selection and in addition offers ‘remediation’ for students having academic difficulty. In one five-year period in a class of 100 students, only one student was excluded because of academic failure, 3 changed careers, while 8% had remedial help52.

WHAT PREDICTS GOOD CLINICAL PERFORMANCE?

Investigators looking for early predictors of what makes a good clinician generally use reports from clinical clerkships and from the house officer or intern year. However, we should note that drop-out will mean that some unsatisfactory students will have left before the house officer year. Clinical performance is not generally predicted by pre-entry academic scores1,35,53,54,55,56,57: the one report of correlation between matriculation scores and clinical performance noted that matriculation scores included 50% contribution from school teacher assessment58. Neither age nor gender predicts clinical performance, nor does previous study of physical sciences, but there is evidence that previous study of English and humanities correlates with better clinical performance5,34,59. There are some reports of association between clinical performance and admission interviews55,56,60,61, although others reveal no correlation54,58. In a school that carefully evaluates applicants, empathy and motivation to be a doctor were found particularly important in predicting both clinical and academic success62.

WHAT ARE THE MOST RELIABLE PROCEDURES TO ASSESS PREDICTORS OF FUTURE PERFORMANCE?

If we can agree that there are certain characteristics that we want to select in prospective doctors, what is the best way of doing this? Research shows that, if we want to add usefully to a measure of intellectual ability in predicting later job performance, our best instrument is the structured interview. While an unstructured interview adds about 8% to prediction of subsequent performance, the structured interview adds around 24%63. Psychometric tests to measure desirable personal characteristics do predict future performance, but their validity may be compromised if they are used as a selection tool: the desired answer is not usually difficult to identify, and applicants who lack integrity are the most likely to manipulate the results64. However, some schools have applied psychometric tests at the point of entry rather than using them to select, and have found correlation between these tests and scores given in interview65,66. This suggests that a well conducted interview may give similar information and that, if constructed to assess desired characteristics such as conscientiousness or helpfulness, it will give a reasonably reliable evaluation20.

Character references from a previous employer or tutor have potential to add to prediction. However, legislative changes in the US in the 1980s meant that an employer giving an adverse report could be sued by the employee: as a result, the predictive validity of personal references in the US has fallen to almost zero20. The reliability of UCAS references in the UK may be similarly threatened. The motivation of the referee is uncertain: some tutors may feel their first loyalty to their student, others may feel compromised by recent data protection legislation that removes the confidentiality of previous years. One medical school in New Zealand has adapted the traditional reference system by writing to head teachers with specific questions, and requesting a rating of the candidate's qualities against the level the head teacher believes to be desirable in a doctor. The long-term predictive validity of this method has not been published, but the school believes it provides valid information and correlates well with other non-cognitive indices (and not at all with academic scores)35. Some schools, particularly those which do a lot of small-group work in the course, use an assessment of performance in small groups as a ‘live’ way to assess interpersonal skills29,52. Evaluation of students in this setting correlates highly with interview scores, and is reported to predict both problem-solving ability and group interaction52.

WHAT CONSTITUTES CURRENT BEST PRACTICE?

In summary, the evidence is that we need to select students with good intellectual ability and that examinations, despite limitations, have some validity. For some candidates—e.g. older applicants, or those from disadvantaged social backgrounds—we may want to look for reliable measures of intellectual ability other than the traditional A levels. We seek individuals who are conscientious and have integrity, who are empathic and motivated to become doctors, and who are psychologically robust enough to enjoy a successful medical career. Some medical schools, mainly outside the UK, have already recognized best practice and have put great care and resource into their selection procedures, with well-planned structured interviews, focused reports from schools and evaluation of interpersonal behaviour. As detailed above, there is evidence that this investment is worthwhile in terms of the suitability of students selected, and economically in terms of student loss during the course.

AND WHERE ARE WE IN THE UK?

The greatest single barrier to a more careful selection process in the UK is the amount of resource that each school has to invest. At present, would-be medical students apply to up to four medical schools. All but four of the UK's present twenty-four medical schools interview about 500 to 1000 applicants for their five or six year MB BS courses. Many interviews are still unstructured, and not all schools require their interviewers to be trained. It is unusual for the interview to be more than 15 or 20 minutes, and while brief interviews may be reliable67 the validity of a 15-minute interview is doubtful68. The fact that many candidates are interviewed four times underlines the wastefulness of our present national procedure, but the cost to individual schools to improve radically would be prohibitive. Our present system does not offer society the best practice available: at present we almost certainly turn away people who would make good doctors and accept some who will be mediocre or poor. We could probably reduce loss from the medical course, and so save money and save personal distress among those who were allowed to make an unwise choice. We could also be more just to applicants, and begin the process of education by showing that we are very serious about the kind of personal qualities that we want in a doctor.

The Civil Service, the Armed Forces, and many business corporations have had selection boards for many years: the Civil Service believe these to be money well spent, and industry has gone further and demonstrated their cost effectiveness20. Those medical schools which invest heavily in their selection procedures admit that it is not cheap: on the other hand, it is not cheap to lose students unnecessarily or to employ a poorly motivated or unhappy doctor. There is a strong argument for pooling resources so that applicants get one good assessment instead of four poor ones. This does not preclude medical schools' maintaining individuality and some degree of choice, and candidates will continue to visit schools and attend open days. However, it is time that UK medical schools got together to collaborate in setting up a first-class selection process that is fair to society and fair to all those people who hope to be the doctors of tomorrow.

References

1. Reede JY. Predictors of success in medicine. Clin Orthop Rel Res 1999;362: 72-7 [PubMed]
2. Greengross S. What patients want from their doctors. In: Allen I, Brown P, Hughes P, eds. Choosing Tomorrow's Doctors. London: Policy Studies Institute, 1997: 12-19
3. Burgess MM, Calkins V, Richards JM. The structured interview: a selection device. Psychol Rep 1972;31: 867-77 [PubMed]
4. Halsey AH, Heath AF, Ridge JM. Origins and Destinations: Family, Class and Education in Modern Britain. Oxford: Clarendon, 1980
5. Neame RLB, Powis DA, Bristow T. Should medical students be selected only from recent school-leavers who have studied sciences. Med Edu 1992;26: 433-40 [PubMed]
6. Smith MD, Hayling C. Better admission criteria. Acad Med 1998;73: 1054-5 [PubMed]
7. Collins JP, White GR, Kennedy JA. Entry to medical school: an audit of traditional selection requirements. Med Edu 1995;29: 22-28 [PubMed]
8. Marley J, Carman I. Selecting medical students: a case report of the need for change. Med Edu 1999;33: 455-9 [PubMed]
9. Allen I. What doctors want from their careers. In: Allen I, Brown P, Hughes P, eds. Choosing Tomorrow's Doctors. London: Policy Studies Institute, 1997: 20-9
10. McCawley A. The physician and burnout. In: Callan J, ed. The Physician: a Professional under Stress. Norwalk, CT: Appleton-Century-Crofts, 1983: 168-77
11. Mawardi BH. Aspects of the impaired physician. In: Farber BA, ed. Stress and Burnout in the Human Service Professions. New York: Academic Press 1983: 119-28
12. McCranie EW, Brandsma JM. Personality antecedents of burnout among middle-aged physicians. Behav Med 1998(spring): 30-6 [PubMed]
13. Hale R. How patients make their doctors ill. In: Allen I, Brown P, Hughes P, eds. Choosing Tomorrow's Doctors. London: Policy Studies Institute, 1997: 30-7
14. Vaillant GE, Sobowale NC, McArthur C. Some psychologic vulnerabilities of physicians. N Engl J Med 1972;287: 372-5 [PubMed]
15. Thomas CB. Stamina: the thread of human life. J Chron Dis 1981;34: 41-4 [PubMed]
16. Leon GR, Gillum R, Gouze M. Personality stability and change over a 30-year period—middle age to old age. J Consult Clin Psychol 1979;47: 517-24 [PubMed]
17. Conley JJ. Longitudinal consistency of adult personality: self-reported psychological characteristics across 45 years. J Pers Soc Psychol 1984;47: 1325-33 [PubMed]
18. Costa PT, McCrae RR. The influence of extraversion and neuroticism on subjective well-being: Happy and unhappy people. J Pers Soc Psychol 1980;38: 668-78 [PubMed]
19. Costa PT, McCrae RR, Norris AH. Personal adjustment to aging: Longitudinal prediction from neuroticism and extraversion. J Gerontol 1981;36: 78-85 [PubMed]
20. Schmidt FL, Hunter JE. The validity and utility of selection methods in personnel psychology: practical and theoretical implications of 85 years of research findings. Psychol Bull 1998;124: 262-74
21. Schmidt FL, Hunter JE. Individual differences in productivity: an empirical test of estimates derived from studies of selection procedure utility. J Appl Psychol 1983;68: 407-15
22. Hunter JE, Hunter RF. Validity and utility of alternative predictors of job performance. Psychol Bull 1984;96: 72-98
23. Ones DS, Viswesvaran C, Schmidt FL. Comprehensive meta-analysis of integrity test validities: findings and implications for personnel selection and theories of job performance. J Appl Psychol Monog 1993;78: 679-703
24. McManus IC. Drop out rate from medical schools seems reasonable. BMJ 1996;312: 885. [PubMed]
25. Wallace M, Berlin A. ‘Dropouts’ from London medical schools: a comparison with the rest of the United Kingdom. Health Trends 1997;29: 106-8
26. Antonovsky A. Medical student selection at the Ben Gurion University of the Negev. Isr J Med Sci 1987;23: 969-75 [PubMed]
27. Reede JY. Predictors of success in medicine. Clin Orthop Rel Res 1999;362: 72-7 [PubMed]
28. Powis D, Neame RLB, Bristow T, Murphy LB. The objective structured interview for medical student selection. BMJ 1988;296: 765-8 [PMC free article] [PubMed]
29. Collins JP. Introducing change in the process of selection. In: Allen I, Brown P, Hughes P, eds. Choosing Tomorrow's Doctors. London: Policy Studies Institute 1997: 52-9
30. Mitchell K. Traditional predictors of performance in medical school. Acad Med 1990;65: 149-58 [PubMed]
31. Tomlinson RWS, Clack GB, Pettinghale KW, Anderson J, Ryan KC. The relative role of A level chemistry, physics and biology in the medical course. Med Edu 1977;11: 103-8 [PubMed]
32. McManus IC, Richards P. Prospective study of medical students during preclinical years. BMJ 1986;293: 124-7 [PMC free article] [PubMed]
33. Monague W, Odds FC. Academic selection criteria and subsequent performance. Med Edu 1990;24: 151-7 [PubMed]
34. Lipton A, Huxham G, Hamilton D. School results as predictors of medical school achievement. Med Edu 1988;22: 381-8 [PubMed]
35. Collins JP, White GR, Petrie KJ, Willoughby EW. A structured panel interview and group exercise in the selection of medical students. Med Edu 1995;29: 332-6 [PubMed]
36. Cariaga-Lo LD, Enarson CE, Crandall SJ, Zaccaro DJ, Richards BF. Cognitive and cognitive predictors of academic difficulties and attrition. Acad Med 1997;72: S69-S71 [PubMed]
37. Herman MW, Velowski JJ. Premedical training, personal characteristics and performance in medical school. Med Edu 1981;15: 363-7 [PubMed]
38. Huff KL, Fang D. When are students most at risk of encountering academic difficulty? A study of the 1992 matriculants to US medical schools. Acad Med 1999;74: 454-60 [PubMed]
39. Koenig JA. Comparison of medical school performance and career plans of students with broad and with science focused premedical preparation. Acad Med 1992;67: 191-6 [PubMed]
40. Dickman RL, Sarnacki RE, Schimpfhauser FT, Katz LA. Medical students from natural science and nonscience undergraduate backgrounds. Similar academic performance and residency selection. JAMA 1980;243: 2506-9 [PubMed]
41. Strayhorn G, Frierson H. Assessing correlations between black and white students' perceptions of the medical school learning environment, their academic performances and their well-being. Acad Med 1989;64: 468-73 [PubMed]
42. Feil D, Kristian M, Mitchell N. Older medical students performance at McGill University. Acad Med 1998;73: 98-100 [PubMed]
43. Davidson RC, Lewis EL. Affirmative action and other special consideration admissions at the University of California Davis School of Medicine. JAMA 1997;28: 1153-8 [PubMed]
44. Hayes SC, Farnhill D. Professional training and English language proficiency. Med Edu 1993;27: 6-14 [PubMed]
45. Parker GB. On our selection: predictors of medical school success. Med J Aust 1993;158: 747-51 [PubMed]
46. McGlinn S, Jackson EW. Predicting the medical school progress of minority students who participated in a preparatory programme. Acad Med 1989;64: 164-6 [PubMed]
47. Calkins E, Arnold LM, Willoughby TL. Gender differences in predictors of performance in medical training. J Med Edu 1987;62: 682-5 [PubMed]
48. Johnson D, Sterling L, Jones R, Anderson J. Predicting academic performance at a predominantly black medical school. J Med Edu 1986;61: 629-39 [PubMed]
49. Webb C, Waugh F, Herbert J. The relationship between locus of control and performance on NMBE part I among black medical students. Psychol Rep 1993;72: 1171-7 [PubMed]
50. Webb CT, Sedlacek W, Cohen D, et al. The impact of nonacademic variables on performance at two medical schools. J Natl Med Assoc 1997;89: 173-80 [PMC free article] [PubMed]
51. Richards P, McManus C, Allen I. British doctors are not disappearing. BMJ 1997;314: 1567-8 [PMC free article] [PubMed]
52. Ferrier BM, McAuley RG, Roberts RS. Selection of medical students at McMaster University. J R Coll Physicians Lond 1978;12: 365-78 [PubMed]
53. McManus IC. From selection to qualification: how and why medical students change. In: Allen I, Brown P, Hughes P, eds. Choosing Tomorrow's Doctors. London: Policy Studies Institute, 1997: 60-79
54. Basco WT, Gilbert GE, Chessman AW, Blue AV. The ability of a medical school admission process to predict clinical performance and patients' satisfaction. Acad Med 2000;75: 743-7 [PubMed]
55. Meredity KE, Dunlap MR, Baker HH. Subjective and objective admission factors as predictors of clinical clerkship performance. J Med Edu 1987;57: 743-51 [PubMed]
56. Murden R, Gallaway GM, Reid JC, Colwill JM. Academic and personal predictors of clinical success in medical school. J Med Edu 1978;53: 711-19 [PubMed]
57. Hall FR, Regan-Smith M, Tivnan T. Relationship of medical students' admission interview scores to their deans' letter ratings. Acad Med 1992;67: 842-5 [PubMed]
58. Weiss M, Lotan I, Kedar H, Ben-Shakhar G. Selecting candidates for medical school: an evaluation of a selection model based on cognitive and personality predictors. Med Edu 1988;22: 492-7 [PubMed]
59. Rolfe IE, Pearson S, Powis DA, Smith AJ. Time for a review of admission to medical school? Lancet 1995;346: 1329-33 [PubMed]
60. Gough HG, Hall WB, Harris RE. Evaluation of performance in medical training. J Med Edu 1964;39: 959-63 [PubMed]
61. Powis DA, Rolfe I. Selection and performance of medical students at Newcastle, New South Wales. Edu Health 1998;11: 15-23
62. Powis DA, Waring TC, Bristow T, O'Connell DL. The structured interview as a tool for predicting premature withdrawal from medical school. Aust N Z J Med 1992;22: 69-698 [PubMed]
63. McDaniel MA, Whetzel DL, Schmidt FL, Mauer SD. The validity of employment interviews: a comprehensive review and meta-analysis. J Appl Psychol 1994;79: 599-616
64. Chaisson GM. Student selection: logic or lottery. J Allied Health 1976(spring): 7-16 [PubMed]
65. Tutton PJM. Medical school entrants: semi-structured interview ratings, prior scholastic achievement and personality profiles. Med Edu 1993;27: 328-36 [PubMed]
66. Rockwell DA. Medical school applicants and empathy. Proc 13 Ann Conf Res Med Edu 1974: November, 208
67. Richards P, McManus IC, Maitlis SA. Reliability of interviewing in medical student selection. BMJ 1988;296: 1520-1 [PMC free article] [PubMed]
68. Roberts GD, Porter AMW. Medical student selection—time for change: discussion paper. J R Soc Med 1989;82: 288-91 [PMC free article] [PubMed]

Articles from Journal of the Royal Society of Medicine are provided here courtesy of Royal Society of Medicine Press