The United Kingdom Clinical Aptitude Test (UKCAT) is a set of cognitive tests introduced in 2006, taken annually before application to medical school. The UKCAT is a test of aptitude and not acquired knowledge and as such the results give medical schools a standardised and objective tool that all schools could use to assist their decision making in selection, and so provide a fairer means of choosing future medical students.
Selection of students for UK medical schools is usually in three stages: assessment of academic qualifications, assessment of further qualities from the application form submitted via UCAS (Universities and Colleges Admissions Service) leading to invitation to interview, and then selection for offer of a place. Medical schools were informed of the psychometric qualities of the UKCAT subtests and given some guidance regarding the interpretation of results. Each school then decided how to use the results within its own selection system.
Annual retrospective key informant telephone interviews were conducted with every UKCAT Consortium medical school, using a pre-circulated structured questionnaire. The key points of the interview were transcribed, 'member checked' and a content analysis was undertaken.
Four equally popular ways of using the test results have emerged, described as Borderline, Factor, Threshold and Rescue methods. Many schools use more than one method, at different stages in their selection process. Schools have used the scores in ways that have sought to improve the fairness of selection and support widening participation. Initially great care was taken not to exclude any applicant on the basis of low UKCAT scores alone but it has been used more as confidence has grown.
There is considerable variation in how medical schools use UKCAT, so it is important that they clearly inform applicants how the test will be used so they can make best use of their limited number of applications.
The UK Clinical Aptitude Test (UKCAT) was introduced in 2006 as an additional tool for the selection of medical students. It tests mental ability in four distinct domains (Quantitative Reasoning, Verbal Reasoning, Abstract Reasoning, and Decision Analysis), and the results are available to students and admissions panels in advance of the selection process. As yet the predictive validity of the test against course performance is largely unknown.
The study objective was to determine whether UKCAT scores predict performance during the first two years of the 5-year undergraduate medical course at Nottingham.
We studied a single cohort of students, who entered Nottingham Medical School in October 2007 and had taken the UKCAT. We used linear regression analysis to identify independent predictors of marks for different parts of the 2-year preclinical course.
Data were available for 204/260 (78%) of the entry cohort. The UKCAT total score had little predictive value. Quantitative Reasoning was a significant independent predictor of course marks in Theme A ('The Cell'), (p = 0.005), and Verbal Reasoning predicted Theme C ('The Community') (p < 0.001), but otherwise the effects were slight or non-existent.
This limited study from a single entry cohort at one medical school suggests that the predictive value of the UKCAT, particularly the total score, is low. Section scores may predict success in specific types of course assessment.
The ultimate test of validity will not be available for some years, when current cohorts of students graduate. However, if this test of mental ability does not predict preclinical performance, it is arguably less likely to predict the outcome in the clinical years. Further research from medical schools with different types of curriculum and assessment is needed, with longitudinal studies throughout the course.
Most UK medical schools use aptitude tests during student selection, but large-scale studies of predictive validity are rare. This study assesses the United Kingdom Clinical Aptitude Test (UKCAT), and its four sub-scales, along with measures of educational attainment, individual and contextual socio-economic background factors, as predictors of performance in the first year of medical school training.
A prospective study of 4,811 students in 12 UK medical schools taking the UKCAT from 2006 to 2008 as a part of the medical school application, for whom first year medical school examination results were available in 2008 to 2010.
UKCAT scores and educational attainment measures (General Certificate of Education (GCE): A-levels, and so on; or Scottish Qualifications Authority (SQA): Scottish Highers, and so on) were significant predictors of outcome. UKCAT predicted outcome better in female students than male students, and better in mature than non-mature students. Incremental validity of UKCAT taking educational attainment into account was significant, but small. Medical school performance was also affected by sex (male students performing less well), ethnicity (non-White students performing less well), and a contextual measure of secondary schooling, students from secondary schools with greater average attainment at A-level (irrespective of public or private sector) performing less well. Multilevel modeling showed no differences between medical schools in predictive ability of the various measures. UKCAT sub-scales predicted similarly, except that Verbal Reasoning correlated positively with performance on Theory examinations, but negatively with Skills assessments.
This collaborative study in 12 medical schools shows the power of large-scale studies of medical education for answering previously unanswerable but important questions about medical student selection, education and training. UKCAT has predictive validity as a predictor of medical school outcome, particularly in mature applicants to medical school. UKCAT offers small but significant incremental validity which is operationally valuable where medical schools are making selection decisions based on incomplete measures of educational attainment. The study confirms the validity of using all the existing measures of educational attainment in full at the time of selection decision-making. Contextual measures provide little additional predictive value, except that students from high attaining secondary schools perform less well, an effect previously shown for UK universities in general.
Medical student selection; Educational attainment; Aptitude tests; UKCAT; Socio-economic factors; Contextual measures
The UK Clinical Aptitude Test (UKCAT) was introduced in 2006 as an additional tool for the selection of medical students. It tests mental ability in four distinct domains (Verbal Reasoning, Quantitative Reasoning, Abstract Reasoning, and Decision Analysis), and the results are available to students and admission panels in advance of the selection process. Our first study showed little evidence of any predictive validity for performance in the first two years of the Nottingham undergraduate course.
The study objective was to determine whether the UKCAT scores had any predictive value for the later parts of the course, largely delivered via clinical placements.
Students entering the course in 2007 and who had taken the UKCAT were asked for permission to use their anonymised data in research. The UKCAT scores were incorporated into a database with routine pre-admission socio-demographics and subsequent course performance data. Correlation analysis was followed by hierarchical multivariate linear regression.
The original study group comprised 204/254 (80%) of the full entry cohort. With attrition over the five years of the course this fell to 185 (73%) by Year 5. The Verbal Reasoning score and the UKCAT Total score both demonstrated some univariate correlations with clinical knowledge marks, and slightly less with clinical skills. No parts of the UKCAT proved to be an independent predictor of clinical course marks, whereas prior attainment was a highly significant predictor (p <0.001).
This study of one cohort of Nottingham medical students showed that UKCAT scores at admission did not independently predict subsequent performance on the course. Whilst the test adds another dimension to the selection process, its fairness and validity in selecting promising students remains unproven, and requires wider investigation and debate by other schools.
Objective To determine whether the use of the UK clinical aptitude test (UKCAT) in the medical schools admissions process reduces the relative disadvantage encountered by certain sociodemographic groups.
Design Prospective cohort study.
Setting Applicants to 22 UK medical schools in 2009 that were members of the consortium of institutions utilising the UKCAT as a component of their admissions process.
Participants 8459 applicants (24 844 applications) to UKCAT consortium member medical schools where data were available on advanced qualifications and socioeconomic background.
Main outcome measures The probability of an application resulting in an offer of a place on a medicine course according to seven educational and sociodemographic variables depending on how the UKCAT was used by the medical school (in borderline cases, as a factor in admissions, or as a threshold).
Results On univariate analysis all educational and sociodemographic variables were significantly associated with the relative odds of an application being successful. The multilevel multiple logistic regression models, however, varied between medical schools according to the way that the UKCAT was used. For example, a candidate from a non-professional background was much less likely to receive a conditional offer of a place compared with an applicant from a higher social class when applying to an institution using the test only in borderline cases (odds ratio 0.51, 95% confidence interval 0.45 to 0.60). No such effect was observed for such candidates applying to medical schools using the threshold approach (1.27, 0.84 to 1.91). These differences were generally reflected in the interactions observed when the analysis was repeated, pooling the data. Notably, candidates from several under-represented groups applying to medical schools that used a threshold approach to the UKCAT were less disadvantaged than those applying to the other institutions in the consortium. These effects were partially reflected in significant differences in the absolute proportion of such candidates finally taking up places in the different types of medical schools; stronger use of the test score (as a factor or threshold) was associated with a significantly increased odds of entrants being male (1.74, 1.25 to 2.41) and from a low socioeconomic background (3.57, 1.03 to 12.39). There was a non-significant trend towards entrants being from a state (non-grammar) school (1.60, 0.97 to 2.62) where a stronger use of the test was employed. Use of the test only in borderline cases was associated with increased odds of entrants having relatively low academic attainment (5.19, 2.02 to 13.33) and English as a second language (2.15, 1.03 to 4.48).
Conclusions The use of the UKCAT may lead to more equitable provision of offers to those applying to medical school from under-represented sociodemographic groups. This may translate into higher numbers of some, but not all, relatively disadvantaged students entering the UK medical profession.
Objectives To determine whether the UK Clinical Aptitude Test (UKCAT) adds value to the selection process for school leaver applicants to medical and dental school, and in particular whether UKCAT can reduce the socioeconomic bias known to affect A levels.
Design Cohort study
Setting Applicants to 23 UK medical and dental schools in 2006.
Participants 9884 applicants who took the UKCAT in the UK and who achieved at least three passes at A level in their school leaving examinations (53% of all applicants).
Main outcome measures Independent predictors of obtaining at least AAB at A level and
UKCAT scores at or above the 30th centile for the cohort, for the subsections and the entire test.
Results Independent predictors of obtaining at least AAB at A level were white ethnicity (odds ratio 1.58, 95% confidence interval 1.41 to 1.77), professional or managerial background (1.39, 1.22 to 1.59), and independent or grammar schooling (2.26, 2.02 to 2.52) (all P<0.001). Independent predictors of achieving UKCAT scores at or above the 30th centile for the whole test were male sex (odd ratio 1.48, 1.32 to 1.66), white ethnicity (2.17, 1.94 to 2.43), professional or managerial background (1.34, 1.17 to 1.54), and independent or grammar schooling (1.91, 1.70 to 2.14) (all P<0.001). One major limitation of the study was that socioeconomic status was not volunteered by approximately 30% of the applicants. Those who withheld socioeconomic status data were significantly different from those who provided that information, which may have caused bias in the analysis.
Conclusions UKCAT was introduced with a high expectation of increasing the diversity and fairness in selection for UK medical and dental schools. This study of a major subgroup of applicants in the first year of operation suggests that it has an inherent favourable bias to men and students from a higher socioeconomic class or independent or grammar schools. However, it does provide a reasonable proxy for A levels in the selection process.
Measures used for medical student selection should predict future performance during training. A problem for any selection study is that predictor-outcome correlations are known only in those who have been selected, whereas selectors need to know how measures would predict in the entire pool of applicants. That problem of interpretation can be solved by calculating construct-level predictive validity, an estimate of true predictor-outcome correlation across the range of applicant abilities.
Construct-level predictive validities were calculated in six cohort studies of medical student selection and training (student entry, 1972 to 2009) for a range of predictors, including A-levels, General Certificates of Secondary Education (GCSEs)/O-levels, and aptitude tests (AH5 and UK Clinical Aptitude Test (UKCAT)). Outcomes included undergraduate basic medical science and finals assessments, as well as postgraduate measures of Membership of the Royal Colleges of Physicians of the United Kingdom (MRCP(UK)) performance and entry in the Specialist Register. Construct-level predictive validity was calculated with the method of Hunter, Schmidt and Le (2006), adapted to correct for right-censorship of examination results due to grade inflation.
Meta-regression analyzed 57 separate predictor-outcome correlations (POCs) and construct-level predictive validities (CLPVs). Mean CLPVs are substantially higher (.450) than mean POCs (.171). Mean CLPVs for first-year examinations, were high for A-levels (.809; CI: .501 to .935), and lower for GCSEs/O-levels (.332; CI: .024 to .583) and UKCAT (mean = .245; CI: .207 to .276). A-levels had higher CLPVs for all undergraduate and postgraduate assessments than did GCSEs/O-levels and intellectual aptitude tests. CLPVs of educational attainment measures decline somewhat during training, but continue to predict postgraduate performance. Intellectual aptitude tests have lower CLPVs than A-levels or GCSEs/O-levels.
Educational attainment has strong CLPVs for undergraduate and postgraduate performance, accounting for perhaps 65% of true variance in first year performance. Such CLPVs justify the use of educational attainment measure in selection, but also raise a key theoretical question concerning the remaining 35% of variance (and measurement error, range restriction and right-censorship have been taken into account). Just as in astrophysics, ‘dark matter’ and ‘dark energy’ are posited to balance various theoretical equations, so medical student selection must also have its ‘dark variance’, whose nature is not yet properly characterized, but explains a third of the variation in performance during training. Some variance probably relates to factors which are unpredictable at selection, such as illness or other life events, but some is probably also associated with factors such as personality, motivation or study skills.
Medical student selection; Undergraduate performance; Postgraduate performance; Educational attainment; Aptitude tests; Criterion-related construct validity; Range restriction; Right censorship; Grade inflation; Markov Chain Monte Carlo algorithm
Objective To describe the current methods used by English medical schools to identify prospective medical students for admission to the five year degree course.
Design Review study including documentary analysis and interviews with admissions tutors.
Setting All schools (n = 22) participating in the national expansion of medical schools programme in England.
Results Though there is some commonality across schools with regard to the criteria used to select future students (academic ability coupled with a “well rounded” personality demonstrated by motivation for medicine, extracurricular interests, and experience of team working and leadership skills) the processes used vary substantially. Some schools do not interview; some shortlist for interview only on predicted academic performance while those that shortlist on a wider range of non-academic criteria use various techniques and tools to do so. Some schools use information presented in the candidate's personal statement and referee's report while others ignore this because of concerns over bias. A few schools seek additional information from supplementary questionnaires filled in by the candidates. Once students are shortlisted, interviews vary in terms of length, panel composition, structure, content, and scoring methods.
Conclusion The stated criteria for admission to medical school show commonality. Universities differ greatly, however, in how they apply these criteria and in the methods used to select students. Different approaches to admissions should be developed and tested.
A prospective study is currently being conducted of students who were freshmen at the University of British Columbia in 1961-1962. Three cohorts are being followed: 136 premedical students, 107 science students, and 136 persons comprising a sample of general arts students. At registration in 1961, the only demographic difference was that the Premedical cohort had a greater proportion of catholics and persons from upper occupational classes. Premedical students generally performed as well in high school as science students and better than general arts students, though the academic potential of science students as measured by achievement and aptitude tests was superior. Premedical students performed better on freshman examinations than the other cohorts, though a significantly larger proportion of science students (84.4%) actually passed into second year than of premedical students (75.0%). A disturbing observation was that by second year the Science cohort had recruited more than enough students to balance its losses, while the Premedical cohort had lost three times as many students as it recruited. The net effect was not much reduced by students who entered directly from senior matriculation or other universities, since these students were about twice as likely to enter the Science cohort as they were to enter the Premedical cohort.
The UK Clinical Aptitude Test (UKCAT) was introduced to facilitate widening participation in medical and dental education in the UK by providing universities with a continuous variable to aid selection; one that might be less sensitive to the sociodemographic background of candidates compared to traditional measures of educational attainment. Initial research suggested that males, candidates from more advantaged socioeconomic backgrounds and those who attended independent or grammar schools performed better on the test. The introduction of the A* grade at A level permits more detailed analysis of the relationship between UKCAT scores, secondary educational attainment and sociodemographic variables. Thus, our aim was to further assess whether the UKCAT is likely to add incremental value over A level (predicted or actual) attainment in the selection process.
Data relating to UKCAT and A level performance from 8,180 candidates applying to medicine in 2009 who had complete information relating to six key sociodemographic variables were analysed. A series of regression analyses were conducted in order to evaluate the ability of sociodemographic status to predict performance on two outcome measures: A level ‘best of three’ tariff score; and the UKCAT scores.
In this sample A level attainment was independently and positively predicted by four sociodemographic variables (independent/grammar schooling, White ethnicity, age and professional social class background). These variables also independently and positively predicted UKCAT scores. There was a suggestion that UKCAT scores were less sensitive to educational background compared to A level attainment. In contrast to A level attainment, UKCAT score was independently and positively predicted by having English as a first language and male sex.
Our findings are consistent with a previous report; most of the sociodemographic factors that predict A level attainment also predict UKCAT performance. However, compared to A levels, males and those speaking English as a first language perform better on UKCAT. Our findings suggest that UKCAT scores may be more influenced by sex and less sensitive to school type compared to A levels. These factors must be considered by institutions utilising the UKCAT as a component of the medical and dental school selection process.
Medical student selection; Educational attainment; Aptitude tests; UKCAT; Socio-economic factors
Anecdotes abound about doctors' personal illness experiences and the effect they have on their empathy and care of patients. We formally investigated the relationship between doctors' and medical students' personal illness experiences, their examination results, preparedness for clinical practice, learning and professional attitudes and behaviour towards patients.
Newly-qualified UK doctors in 2005 (n = 2062/4784), and two cohorts of students at one London medical school (n = 640/749) participated in the quantitative arm of the study. 37 Consultants, 1 Specialist Registrar, 2 Clinical Skills Tutors and 25 newly-qualified doctors participated in the qualitative arm. Newly-qualified doctors and medical students reported their personal illness experiences in a questionnaire. Doctors' experiences were correlated with self-reported preparedness for their new clinical jobs. Students' experiences were correlated with their examination results, and self-reported anxiety and depression. Interviews with clinical teachers, newly-qualified doctors and senior doctors qualitatively investigated how personal illness experiences affect learning, professional attitudes, and behaviour.
85.5% of newly-qualified doctors and 54.4% of medical students reported personal illness experiences. Newly-qualified doctors who had been ill felt less prepared for starting work (p < 0.001), but those who had only experienced illness in a relative or friend felt more prepared (p = 0.02). Clinical medical students who had been ill were more anxious (p = 0.01) and had lower examination scores (p = 0.006). Doctors felt their personal illness experiences helped them empathise and communicate with patients. Medical students with more life experience were perceived as more mature, empathetic, and better learners; but illness at medical school was recognised to impede learning.
The majority of the medical students and newly qualified doctors we studied reported personal illness experiences, and these experiences were associated with lower undergraduate examination results, higher anxiety, and lower preparedness. However reflection on such experiences may have improved professional attitudes such as empathy and compassion for patients. Future research is warranted in this area.
The demanding and intense environment of medical training can create excessive pressures on medical students that eventually lead to unfavorable consequences, either at a personal or professional level. These consequences can include poor academic performance and impaired cognitive ability. This study was designed to explore associations between pass-fail outcome and psychological health parameters (i.e. stress, anxiety, and depression symptoms).
A cross-sectional study was conducted on a cohort of first-year medical students in a Malaysian medical school. The depression anxiety stress scale 21-item assessment (DASS-21) was administered to them right after the final paper of the first-year final examination. Their final examination outcomes (i.e. pass or fail) were traced by using their student identity code (ID) through the Universiti Sains Malaysia academic office.
A total of 194 (98.0%) of medical students responded to the DASS-21. An independent t-test showed that students who passed had significantly lower stress, anxiety, and depression symptoms than those who failed the first-year final examination (P <0.05). Those who experienced moderate to high stress were at 2.43 times higher risk for failing the examination than those who experienced normal to mild stress.
Medical students who failed in the final examination had higher psychological distress than those who passed the examination. Those who experienced high stress levels were more likely to fail than those who did not. Reducing the psychological distress of medical students prior to examination may help them to perform better in the examination.
Psychological stress; Anxiety; Depression; Medical students; Student assessment; Malaysia
Selection of medical students in the UK is still largely based on prior academic achievement, although doubts have been expressed as to whether performance in earlier life is predictive of outcomes later in medical school or post-graduate education. This study analyses data from five longitudinal studies of UK medical students and doctors from the early 1970s until the early 2000s. Two of the studies used the AH5, a group test of general intelligence (that is, intellectual aptitude). Sex and ethnic differences were also analyzed in light of the changing demographics of medical students over the past decades.
Data from five cohort studies were available: the Westminster Study (began clinical studies from 1975 to 1982), the 1980, 1985, and 1990 cohort studies (entered medical school in 1981, 1986, and 1991), and the University College London Medical School (UCLMS) Cohort Study (entered clinical studies in 2005 and 2006). Different studies had different outcome measures, but most had performance on basic medical sciences and clinical examinations at medical school, performance in Membership of the Royal Colleges of Physicians (MRCP(UK)) examinations, and being on the General Medical Council Specialist Register.
Correlation matrices and path analyses are presented. There were robust correlations across different years at medical school, and medical school performance also predicted MRCP(UK) performance and being on the GMC Specialist Register. A-levels correlated somewhat less with undergraduate and post-graduate performance, but there was restriction of range in entrants. General Certificate of Secondary Education (GCSE)/O-level results also predicted undergraduate and post-graduate outcomes, but less so than did A-level results, but there may be incremental validity for clinical and post-graduate performance. The AH5 had some significant correlations with outcome, but they were inconsistent. Sex and ethnicity also had predictive effects on measures of educational attainment, undergraduate, and post-graduate performance. Women performed better in assessments but were less likely to be on the Specialist Register. Non-white participants generally underperformed in undergraduate and post-graduate assessments, but were equally likely to be on the Specialist Register. There was a suggestion of smaller ethnicity effects in earlier studies.
The existence of the Academic Backbone concept is strongly supported, with attainment at secondary school predicting performance in undergraduate and post-graduate medical assessments, and the effects spanning many years. The Academic Backbone is conceptualized in terms of the development of more sophisticated underlying structures of knowledge ('cognitive capital’ and 'medical capital’). The Academic Backbone provides strong support for using measures of educational attainment, particularly A-levels, in student selection.
Academic Backbone; Secondary school attainment; Undergraduate medical education; Post-graduate medical education; Longitudinal analyses; Continuities; Medical student selection; Cognitive capital; Medical capital; Aptitude tests
Evidence for the superiority of particular characteristics in PBL tutors in medical curricula is generally inconclusive. Most studies have investigated the effectiveness of content experts compared with that of non-experts as measured either by student satisfaction or academic achievement. A few have compared academic staff tutors with student tutors. The purpose of this study was to investigate the relationship between students' perception of overall tutor effectiveness, particular tutor behaviours, clinical qualifications and academic appointment.
A questionnaire designed to evaluate particular aspects of PBL tutoring technique, related either to subject-matter knowledge or to process-facilitation skill, as well as overall effectiveness, was distributed to students in first year of a PBL medical program at the end of each of three tutor terms. A total of 76 tutor terms were included in the study. Data analysis compared clinical with non-clinical tutors, and staff with non-staff tutors.
Clinically qualified tutors used their subject-matter knowledge significantly more than non-clinical tutors and were seen as being more empathic with their students. Staff tutors placed more emphasis on assessment than non-staff tutors and were seen as having greater skill in establishing and maintaining an environment of cooperation within their PBL groups than non-staff tutors.
These results suggest that both subject-matter knowledge and process-facilitation skills are necessary but not individually sufficient characteristics of effective tutors.
Professionalism is a difficult construct to define in medical students but aspects of this concept may be important in predicting the risk of postgraduate misconduct. For this reason attempts are being made to evaluate medical students' professionalism. This study investigated the psychometric properties of Selected Response Questions (SRQs) relating to the theme of professional conduct and ethics comparing them with two sets of control items: those testing pure knowledge of anatomy, and; items evaluating the ability to integrate and apply knowledge ("skills"). The performance of students on the SRQs was also compared with two external measures estimating aspects of professionalism in students; peer ratings of professionalism and their Conscientiousness Index, an objective measure of behaviours at medical school.
Item Response Theory (IRT) was used to analyse both question and student performance for SRQs relating to knowledge of professionalism, pure anatomy and skills. The relative difficulties, discrimination and 'guessabilities' of each theme of question were compared with each other using Analysis of Variance (ANOVA). Student performance on each topic was compared with the measures of conscientiousness and professionalism using parametric and non-parametric tests as appropriate. A post-hoc analysis of power for the IRT modelling was conducted using a Monte Carlo simulation.
Professionalism items were less difficult compared to the anatomy and skills SRQs, poorer at discriminating between candidates and more erratically answered when compared to anatomy questions. Moreover professionalism item performance was uncorrelated with the standardised Conscientiousness Index scores (rho = 0.009, p = 0.90). In contrast there were modest but significant correlations between standardised Conscientiousness Index scores and performance at anatomy items (rho = 0.20, p = 0.006) though not skills (rho = .11, p = .1). Likewise, students with high peer ratings for professionalism had superior performance on anatomy SRQs but not professionalism themed questions. A trend of borderline significance (p = .07) was observed for performance on skills SRQs and professionalism nomination status.
SRQs related to professionalism are likely to have relatively poor psychometric properties and lack associations with other constructs associated with undergraduate professional behaviour. The findings suggest that such questions should not be included in undergraduate examinations and may raise issues with the introduction of Situational Judgement Tests into Foundation Years selection.
We investigated the effects of My Teaching Partner—Secondary (MTP-S), a teacher professional development intervention, on students’ peer relationships in middle and high school classrooms. MTP-S targets increasing teachers’ positive interactions with students and sensitive instructional practices and has demonstrated improvements in students’ academic achievement and motivation. The current study tested the prediction from systems theory that effects of MTP-S on students would extend beyond the academic domain—that is, the ecology of teachers’ behaviors towards students should also influence the ecology of students’ behaviors towards one another. Participants were 88 teachers (43 randomly assigned to MTP-S and 45 assigned to a control group that received the regular professional development offerings in their school) and 1423 students in their classrooms. Observations and student self-report of classroom peer interactions were collected at the start and at the end of the course. Results indicated that in MTP-S classrooms, students were observed to show improvement in positive peer interactions, although this pattern was not found in self-report data. However, moderation analyses suggested that for students with high disruptive behavior at the start of the course, teacher participation in MTP-S mitigated a typical decline towards poorer self-reported peer relationships. The relevance of findings for the social ecology of classrooms is discussed.
This study investigated whether the introduction of professional development teaching in the first two years of a medical course improved students' observed communication skills with simulated patients. Students' observed communication skills were related to patient-centred attitudes, confidence in communicating with patients and performance in later clinical examinations.
Eighty-two medical students from two consecutive cohorts at a UK medical school completed two videoed consultations with a simulated patient: one at the beginning of year 1 and one at the end of year 2. Group 1 (n = 35) received a traditional pre-clinical curriculum. Group 2 (n = 47) received a curriculum that included communication skills training integrated into a 'professional development' vertical module. Videoed consultations were rated using the Evans Interview Rating Scale by communication skills tutors. A subset of 27% were double-coded. Inter-rater reliability is reported.
Students who had received the professional development teaching achieved higher ratings for use of silence, not interrupting the patient, and keeping the discussion relevant compared to students receiving the traditional curriculum. Patient-centred attitudes were not related to observed communication. Students who were less nervous and felt they knew how to listen were rated as better communicators. Students receiving the traditional curriculum and who had been rated as better communicators when they entered medical school performed less well in the final year clinical examination.
Students receiving the professional development training showed significant improvements in certain communication skills, but students in both cohorts improved over time. The lack of a relationship between observed communication skills and patient-centred attitudes may be a reflection of students' inexperience in working with patients, resulting in 'patient-centredness' being an abstract concept. Students in the early years of their medical course may benefit from further opportunities to practise basic communication skills on a one-to-one basis with patients.
communication skills; patient-centredness; medical student; curriculum change; video observation
In 1998, a new selection process which utilised an aptitude test and an interview in addition to previous academic achievement was introduced into an Australian undergraduate medical course.
To test the outcomes of the selection criteria over an 11-year period.
1174 students who entered the course from secondary school and who enrolled in the MBBS from 1999 through 2009 were studied in relation to specific course outcomes. Regression analyses using entry scores, sex and age as independent variables were tested for their relative value in predicting subsequent academic performance in the 6-year course. The main outcome measures were assessed by weighted average mark for each academic year level; together with results in specific units, defined as either ‘knowledge'-based or ‘clinically’ based.
Previous academic performance and female sex were the major independent positive predictors of performance in the course. The interview score showed positive predictive power during the latter years of the course and in a range of ‘clinically' based units. This relationship was mediated predominantly by the score for communication skills.
Results support combining prior academic achievement with the assessment of communication skills in a structured interview as selection criteria into this undergraduate medical course.
Internationally, tests of general mental ability are used in the selection of medical students. Examples include the Medical College Admission Test, Undergraduate Medicine and Health Sciences Admission Test and the UK Clinical Aptitude Test. The most widely used measure of their efficacy is predictive validity.
A new tool, the Health Professions Admission Test- Ireland (HPAT-Ireland), was introduced in 2009. Traditionally, selection to Irish undergraduate medical schools relied on academic achievement. Since 2009, Irish and EU applicants are selected on a combination of their secondary school academic record (measured predominately by the Leaving Certificate Examination) and HPAT-Ireland score. This is the first study to report on the predictive validity of the HPAT-Ireland for early undergraduate assessments of communication and clinical skills.
Students enrolled at two Irish medical schools in 2009 were followed up for two years. Data collected were gender, HPAT-Ireland total and subsection scores; Leaving Certificate Examination plus HPAT-Ireland combined score, Year 1 Objective Structured Clinical Examination (OSCE) scores (Total score, communication and clinical subtest scores), Year 1 Multiple Choice Questions and Year 2 OSCE and subset scores. We report descriptive statistics, Pearson correlation coefficients and Multiple linear regression models.
Data were available for 312 students. In Year 1 none of the selection criteria were significantly related to student OSCE performance. The Leaving Certificate Examination and Leaving Certificate plus HPAT-Ireland combined scores correlated with MCQ marks.
In Year 2 a series of significant correlations emerged between the HPAT-Ireland and subsections thereof with OSCE Communication Z-scores; OSCE Clinical Z-scores; and Total OSCE Z-scores. However on multiple regression only the relationship between Total OSCE Score and the Total HPAT-Ireland score remained significant; albeit the predictive power was modest.
We found that none of our selection criteria strongly predict clinical and communication skills. The HPAT- Ireland appears to measures ability in domains different to those assessed by the Leaving Certificate Examination. While some significant associations did emerge in Year 2 between HPAT Ireland and total OSCE scores further evaluation is required to establish if this pattern continues during the senior years of the medical course.
Selection; Medical; Student; Validity; Predictive; HPAT-Ireland; Assessment; Cognitive; Ability
Fatigue is a common complaint among elementary and junior high school students, and is known to be associated with reduced academic performance. Recently, we demonstrated that fatigue was correlated with decreased cognitive function in these students. However, no studies have identified cognitive predictors of fatigue. Therefore, we attempted to determine independent cognitive predictors of fatigue in these students.
We performed a prospective cohort study. One hundred and forty-two elementary and junior high school students without fatigue participated. They completed a variety of paper-and-pencil tests, including list learning and list recall tests, kana pick-out test, semantic fluency test, figure copying test, digit span forward test, and symbol digit modalities test. The participants also completed computerized cognitive tests (tasks A to E on the modified advanced trail making test). These cognitive tests were used to evaluate motor- and information-processing speed, immediate and delayed memory function, auditory and visual attention, divided and switching attention, retrieval of learned material, and spatial construction. One year after the tests, a questionnaire about fatigue (Japanese version of the Chalder Fatigue Scale) was administered to all the participants.
After the follow-up period, we confirmed 40 cases of fatigue among 118 students. In multivariate logistic regression analyses adjusted for grades and gender, poorer performance on visual information-processing speed and attention tasks was associated with increased risk of fatigue.
Reduced visual information-processing speed and poor attention are independent predictors of fatigue in elementary and junior high school students.
The anatomy course offers important opportunities to develop professionalism at an early stage in medical education. It is an academically significant course that also engenders stress in some students.
Over a three-year period, 115 of 297 students completed creative projects. Thirty-four project completers and 47 non-completers consented to participate in the study. Projects were analyzed for professionalism themes using grounded theory. A subset of project completers and non-completers were interviewed to determine their views about the stress of anatomy and medical school, as well as the value of the creative projects. We also compared test performance of project completers and non-completers.
Projects completed early in the course often expressed ambivalence about anatomy, whereas later projects showed more gratitude and sense of awe. Project completers tended to report greater stress than noncompleters, but stated that doing projects reduced stress and caused them to develop a richer appreciation for anatomy and medicine. Project completers performed significantly lower than non-completers on the first written exam (pre-project). Differences between groups on individual exams after both the first and second creative project were nonsignificant.
For some students, creative projects may offer a useful way of reflecting on various aspects of professionalism while helping them to manage stress.
Medical student selection is an important but difficult task. Three recent papers by McManus et al. in BMC Medicine have re-examined the role of tests of attainment of learning (A’ levels, GCSEs, SQA) and of aptitude (AH5, UKCAT), but on a much larger scale than previously attempted. They conclude that A’ levels are still the best predictor of future success at medical school and beyond. However, A’ levels account for only 65% of the variance in performance that is found. Therefore, more work is needed to establish relevant assessment of the other 35%.
Please see related research articles http://www.biomedcentral.com/1741-7015/11/242, http://www.biomedcentral.com/1741-7015/11/243 and http://www.biomedcentral.com/1741-7015/11/244.
Medical School Admission; Predictors of performance; Aptitude testing
Based upon student ratings of such factors as predictable work hours and personal time, medical specialties have been identified as lifestyle friendly, intermediate, or unfriendly. Lifestyle friendly programs may be more desirable, more competitive, and for students elected to the Alpha Omega Alpha (AOA) Honor Medical Society, more attainable.
The objective of this study was to evaluate whether AOA students increasingly entered lifestyle friendly residency programs and whether trends in program selection differed between AOA and non-AOA graduates.
This retrospective cohort study examined PGY-2 data from the Association of American Medical Colleges and the 12 allopathic schools in the Associated Medical Schools of New York.
Data on 1987–2006 graduates from participating schools were evaluated.
Residency program selection over the 20-year period served as the main outcome measure.
AOA graduates increasingly entered lifestyle-friendly residencies—from 12.9% in 1987 to 32.6% in 2006 (p < 0.01). There was also a significant decrease in AOA graduates entering lifestyle unfriendly residencies, from 31.6% in 1987 to 12.6% in 2006 (p < 0.01). Selection of lifestyle intermediate residencies among AOA graduates remained fairly stable at an average of 53%. Similar trends were found among non-AOA students. However, within these categories, AOA graduates increasingly selected radiology, dermatology, plastic surgery and orthopedics while non-AOA graduates increasingly selected anesthesiology and neurology.
While lifestyle factors appear to influence residency program selection, AOA graduates differentially were more likely to either choose or attain certain competitive, lifestyle-friendly specialties. Health care reform should be targeted to improve lifestyle and decrease income disparities for specialties needed to meet health manpower needs.
Electronic supplementary material
The online version of this article (doi:10.1007/s11606-011-1811-9) contains supplementary material, which is available to authorized users.
career choice; medical students; workforce; residency; lifestyle
During the evaluation process, Residency Admissions Committees typically gather data on objective and subjective measures of a medical student's performance through the Electronic Residency Application Service, including medical school grades, standardized test scores, research achievements, nonacademic accomplishments, letters of recommendation, the dean's letter, and personal statements. Using these data to identify which medical students are likely to become successful residents in an academic residency program in obstetrics and gynecology is difficult and to date, not well studied.
To determine whether objective information in medical students' applications can help predict resident success.
We performed a retrospective cohort study of all residents who matched into the Johns Hopkins University residency program in obstetrics and gynecology between 1994 and 2004 and entered the program through the National Resident Matching Program as a postgraduate year-1 resident. Residents were independently evaluated by faculty and ranked in 4 groups according to perceived level of success. Applications from residents in the highest and lowest group were abstracted. Groups were compared using the Fisher exact test and the Student t test.
Seventy-five residents met inclusion criteria and 29 residents were ranked in the highest and lowest quartiles (15 in highest, 14 in lowest). Univariate analysis identified no variables as consistent predictors of resident success.
In a program designed to train academic obstetrician-gynecologists, objective data from medical students' applications did not correlate with successful resident performance in our obstetrics-gynecology residency program. We need to continue our search for evaluation criteria that can accurately and reliably select the medical students that are best fit for our specialty.
Objective To identify potential predictors of undergraduate students who struggle during their medical training.
Design Case-control study. Cases were students who had experienced academic or personal difficulties that affected their progression on the course (“strugglers”). Controls were selected at random from the corresponding year cohorts, using a ratio of four controls for each struggler.
Setting University of Nottingham Medical School.
Participants Students who entered the course over five consecutive years.
Main outcome measures Likelihood ratios for independent risk factors for struggling on the course
Results 10-15% of each year's student intake were identified as strugglers. Significant independent predictors of students being in this category were negative comments in the academic reference (likelihood ratio 2.25, 95% confidence intervals 1.44 to 3.50), lower mean examination grade at A level (2.19, 1.37 to 3.51), and the late offer of a place (1.98, 1.19 to 3.30). Male sex was a less significant risk factor (1.70, 1.09 to 2.65) as was a lower grade at GCSE science (2.13, 1.12 to 4.05). In UK students whose ethnicity was known, not being white was a significant predictor of struggling (2.77, 1.52 to 5.05) but the presence of negative comments was not. Age at entry to the course and the possession of a previous degree were not predictive.
Conclusions Our results support retention of existing selection practices relating to academic achievement and critical review of students' references. We plan to undertake further investigation of the reasons why some students, including men, those with late offers and those from ethnic minority backgrounds, may do less well on the Nottingham course.