Search tips
Search criteria

Results 1-25 (1184098)

Clipboard (0)

Related Articles

1.  Predictive validity of the UK clinical aptitude test in the final years of medical school: a prospective cohort study 
BMC Medical Education  2014;14:88.
The UK Clinical Aptitude Test (UKCAT) was designed to address issues identified with traditional methods of selection. This study aims to examine the predictive validity of the UKCAT and compare this to traditional selection methods in the senior years of medical school. This was a follow-up study of two cohorts of students from two medical schools who had previously taken part in a study examining the predictive validity of the UKCAT in first year.
The sample consisted of 4th and 5th Year students who commenced their studies at the University of Aberdeen or University of Dundee medical schools in 2007. Data collected were: demographics (gender and age group), UKCAT scores; Universities and Colleges Admissions Service (UCAS) form scores; admission interview scores; Year 4 and 5 degree examination scores. Pearson’s correlations were used to examine the relationships between admissions variables, examination scores, gender and age group, and to select variables for multiple linear regression analysis to predict examination scores.
Ninety-nine and 89 students at Aberdeen medical school from Years 4 and 5 respectively, and 51 Year 4 students in Dundee, were included in the analysis. Neither UCAS form nor interview scores were statistically significant predictors of examination performance. Conversely, the UKCAT yielded statistically significant validity coefficients between .24 and .36 in four of five assessments investigated. Multiple regression analysis showed the UKCAT made a statistically significant unique contribution to variance in examination performance in the senior years.
Results suggest the UKCAT appears to predict performance better in the later years of medical school compared to earlier years and provides modest supportive evidence for the UKCAT’s role in student selection within these institutions. Further research is needed to assess the predictive validity of the UKCAT against professional and behavioural outcomes as the cohort commences working life.
PMCID: PMC4008381  PMID: 24762134
UKCAT; Predictive validity; Psychometric; Assessment; Selection; Admissions; Aptitude
2.  The UKCAT-12 study: educational attainment, aptitude test performance, demographic and socio-economic contextual factors as predictors of first year outcome in a cross-sectional collaborative study of 12 UK medical schools 
BMC Medicine  2013;11:244.
Most UK medical schools use aptitude tests during student selection, but large-scale studies of predictive validity are rare. This study assesses the United Kingdom Clinical Aptitude Test (UKCAT), and its four sub-scales, along with measures of educational attainment, individual and contextual socio-economic background factors, as predictors of performance in the first year of medical school training.
A prospective study of 4,811 students in 12 UK medical schools taking the UKCAT from 2006 to 2008 as a part of the medical school application, for whom first year medical school examination results were available in 2008 to 2010.
UKCAT scores and educational attainment measures (General Certificate of Education (GCE): A-levels, and so on; or Scottish Qualifications Authority (SQA): Scottish Highers, and so on) were significant predictors of outcome. UKCAT predicted outcome better in female students than male students, and better in mature than non-mature students. Incremental validity of UKCAT taking educational attainment into account was significant, but small. Medical school performance was also affected by sex (male students performing less well), ethnicity (non-White students performing less well), and a contextual measure of secondary schooling, students from secondary schools with greater average attainment at A-level (irrespective of public or private sector) performing less well. Multilevel modeling showed no differences between medical schools in predictive ability of the various measures. UKCAT sub-scales predicted similarly, except that Verbal Reasoning correlated positively with performance on Theory examinations, but negatively with Skills assessments.
This collaborative study in 12 medical schools shows the power of large-scale studies of medical education for answering previously unanswerable but important questions about medical student selection, education and training. UKCAT has predictive validity as a predictor of medical school outcome, particularly in mature applicants to medical school. UKCAT offers small but significant incremental validity which is operationally valuable where medical schools are making selection decisions based on incomplete measures of educational attainment. The study confirms the validity of using all the existing measures of educational attainment in full at the time of selection decision-making. Contextual measures provide little additional predictive value, except that students from high attaining secondary schools perform less well, an effect previously shown for UK universities in general.
PMCID: PMC3827332  PMID: 24229380
Medical student selection; Educational attainment; Aptitude tests; UKCAT; Socio-economic factors; Contextual measures
3.  Tutoring in problem-based learning medical curricula: the influence of tutor background and style on effectiveness 
Evidence for the superiority of particular characteristics in PBL tutors in medical curricula is generally inconclusive. Most studies have investigated the effectiveness of content experts compared with that of non-experts as measured either by student satisfaction or academic achievement. A few have compared academic staff tutors with student tutors. The purpose of this study was to investigate the relationship between students' perception of overall tutor effectiveness, particular tutor behaviours, clinical qualifications and academic appointment.
A questionnaire designed to evaluate particular aspects of PBL tutoring technique, related either to subject-matter knowledge or to process-facilitation skill, as well as overall effectiveness, was distributed to students in first year of a PBL medical program at the end of each of three tutor terms. A total of 76 tutor terms were included in the study. Data analysis compared clinical with non-clinical tutors, and staff with non-staff tutors.
Clinically qualified tutors used their subject-matter knowledge significantly more than non-clinical tutors and were seen as being more empathic with their students. Staff tutors placed more emphasis on assessment than non-staff tutors and were seen as having greater skill in establishing and maintaining an environment of cooperation within their PBL groups than non-staff tutors.
These results suggest that both subject-matter knowledge and process-facilitation skills are necessary but not individually sufficient characteristics of effective tutors.
PMCID: PMC1180438  PMID: 15938758
4.  Use of UKCAT scores in student selection by UK medical schools, 2006-2010 
BMC Medical Education  2011;11:98.
The United Kingdom Clinical Aptitude Test (UKCAT) is a set of cognitive tests introduced in 2006, taken annually before application to medical school. The UKCAT is a test of aptitude and not acquired knowledge and as such the results give medical schools a standardised and objective tool that all schools could use to assist their decision making in selection, and so provide a fairer means of choosing future medical students.
Selection of students for UK medical schools is usually in three stages: assessment of academic qualifications, assessment of further qualities from the application form submitted via UCAS (Universities and Colleges Admissions Service) leading to invitation to interview, and then selection for offer of a place. Medical schools were informed of the psychometric qualities of the UKCAT subtests and given some guidance regarding the interpretation of results. Each school then decided how to use the results within its own selection system.
Annual retrospective key informant telephone interviews were conducted with every UKCAT Consortium medical school, using a pre-circulated structured questionnaire. The key points of the interview were transcribed, 'member checked' and a content analysis was undertaken.
Four equally popular ways of using the test results have emerged, described as Borderline, Factor, Threshold and Rescue methods. Many schools use more than one method, at different stages in their selection process. Schools have used the scores in ways that have sought to improve the fairness of selection and support widening participation. Initially great care was taken not to exclude any applicant on the basis of low UKCAT scores alone but it has been used more as confidence has grown.
There is considerable variation in how medical schools use UKCAT, so it is important that they clearly inform applicants how the test will be used so they can make best use of their limited number of applications.
PMCID: PMC3248371  PMID: 22114935
5.  Widening access to UK medical education for under-represented socioeconomic groups: modelling the impact of the UKCAT in the 2009 cohort 
Objective To determine whether the use of the UK clinical aptitude test (UKCAT) in the medical schools admissions process reduces the relative disadvantage encountered by certain sociodemographic groups.
Design Prospective cohort study.
Setting Applicants to 22 UK medical schools in 2009 that were members of the consortium of institutions utilising the UKCAT as a component of their admissions process.
Participants 8459 applicants (24 844 applications) to UKCAT consortium member medical schools where data were available on advanced qualifications and socioeconomic background.
Main outcome measures The probability of an application resulting in an offer of a place on a medicine course according to seven educational and sociodemographic variables depending on how the UKCAT was used by the medical school (in borderline cases, as a factor in admissions, or as a threshold).
Results On univariate analysis all educational and sociodemographic variables were significantly associated with the relative odds of an application being successful. The multilevel multiple logistic regression models, however, varied between medical schools according to the way that the UKCAT was used. For example, a candidate from a non-professional background was much less likely to receive a conditional offer of a place compared with an applicant from a higher social class when applying to an institution using the test only in borderline cases (odds ratio 0.51, 95% confidence interval 0.45 to 0.60). No such effect was observed for such candidates applying to medical schools using the threshold approach (1.27, 0.84 to 1.91). These differences were generally reflected in the interactions observed when the analysis was repeated, pooling the data. Notably, candidates from several under-represented groups applying to medical schools that used a threshold approach to the UKCAT were less disadvantaged than those applying to the other institutions in the consortium. These effects were partially reflected in significant differences in the absolute proportion of such candidates finally taking up places in the different types of medical schools; stronger use of the test score (as a factor or threshold) was associated with a significantly increased odds of entrants being male (1.74, 1.25 to 2.41) and from a low socioeconomic background (3.57, 1.03 to 12.39). There was a non-significant trend towards entrants being from a state (non-grammar) school (1.60, 0.97 to 2.62) where a stronger use of the test was employed. Use of the test only in borderline cases was associated with increased odds of entrants having relatively low academic attainment (5.19, 2.02 to 13.33) and English as a second language (2.15, 1.03 to 4.48).
Conclusions The use of the UKCAT may lead to more equitable provision of offers to those applying to medical school from under-represented sociodemographic groups. This may translate into higher numbers of some, but not all, relatively disadvantaged students entering the UK medical profession.
PMCID: PMC3328544  PMID: 22511300
6.  Predictive power of UKCAT and other pre-admission measures for performance in a medical school in Glasgow: a cohort study 
BMC Medical Education  2014;14:116.
The UK Clinical Aptitude Test (UKCAT) and its four subtests are currently used by 24 Medical and Dental Schools in the UK for admissions. This longitudinal study examines the predictive validity of UKCAT for final performance in the undergraduate medical degree programme at one Medical School and compares this with the predictive validity of the selection measures available pre-UKCAT.
This was a retrospective observational study of one cohort of students, admitted to Glasgow Medical School in 2007. We examined the associations which UKCAT scores, school science grades and pre-admissions interview scores had with performance indicators, particularly final composite scores that determine students’ postgraduate training opportunities and overall ranking (Educational Performance Measure - EPM, and Honours and Commendation – H&C). Analyses were conducted both with and without adjustment for potential socio-demographic confounders (gender, age, ethnicity and area deprivation).
Despite its predictive value declining as students progress through the course, UKCAT was associated with the final composite scores. In mutually adjusted analyses (also adjusted for socio-demographic confounders), only UKCAT total showed independent relationships with both EPM (p = 0.005) and H&C (p = 0.004), school science achievements predicted EPM (p = 0.009), and pre-admissions interview score predicted neither. UKCAT showed less socio-demographic variation than did TSS.
UKCAT has a modest predictive power for overall course performance at the University of Glasgow Medical School over and above that of school science achievements or pre-admission interview score and we conclude that UKCAT is the most useful predictor of final ranking.
PMCID: PMC4063234  PMID: 24919950
UKCAT; Predictive validity; Widening participation; Socio-economic indicators; Admissions interview; School HE participation rate
7.  Role modelling of clinical tutors: a focus group study among medical students 
BMC Medical Education  2015;15:17.
Role modelling by clinicians assists in development of medical students’ professional competencies, values and attitudes. Three core characteristics of a positive role model include 1) clinical attributes, 2) teaching skills, and 3) personal qualities. This study was designed to explore medical students’ perceptions of their bedside clinical tutors as role models during the first year of a medical program.
The study was conducted with one cohort (n = 301) of students who had completed Year 1 of the Sydney Medical Program in 2013. A total of nine focus groups (n = 59) were conducted with medical students following completion of Year 1. Data were transcribed verbatim. Thematic analysis was used to code and categorise data into themes.
Students identified both positive and negative characteristics and behaviour displayed by their clinical tutors. Characteristics and behaviour that students would like to emulate as medical practitioners in the future included:
1) Clinical attributes: a good knowledge base; articulate history taking skills; the ability to explain and demonstrate skills at the appropriate level for students; and empathy, respect and genuine compassion for patients.
2) Teaching skills: development of a rapport with students; provision of time towards the growth of students academically and professionally; provision of a positive learning environment; an understanding of the student curriculum and assessment requirements; immediate and useful feedback; and provision of patient interaction.
3) Personal qualities: respectful interprofessional staff interactions; preparedness for tutorials; demonstration of a passion for teaching; and demonstration of a passion for their career choice.
Excellence in role modelling entails demonstration of excellent clinical care, teaching skills and personal characteristics. Our findings reinforce the important function of clinical bedside tutors as role models, which has implications for faculty development and recruitment.
PMCID: PMC4335700
Role modelling; Medical students; Clinical tutors
8.  Comparison of A level and UKCAT performance in students applying to UK medical and dental schools in 2006: cohort study 
Objectives To determine whether the UK Clinical Aptitude Test (UKCAT) adds value to the selection process for school leaver applicants to medical and dental school, and in particular whether UKCAT can reduce the socioeconomic bias known to affect A levels.
Design Cohort study
Setting Applicants to 23 UK medical and dental schools in 2006.
Participants 9884 applicants who took the UKCAT in the UK and who achieved at least three passes at A level in their school leaving examinations (53% of all applicants).
Main outcome measures Independent predictors of obtaining at least AAB at A level and
UKCAT scores at or above the 30th centile for the cohort, for the subsections and the entire test.
Results Independent predictors of obtaining at least AAB at A level were white ethnicity (odds ratio 1.58, 95% confidence interval 1.41 to 1.77), professional or managerial background (1.39, 1.22 to 1.59), and independent or grammar schooling (2.26, 2.02 to 2.52) (all P<0.001). Independent predictors of achieving UKCAT scores at or above the 30th centile for the whole test were male sex (odd ratio 1.48, 1.32 to 1.66), white ethnicity (2.17, 1.94 to 2.43), professional or managerial background (1.34, 1.17 to 1.54), and independent or grammar schooling (1.91, 1.70 to 2.14) (all P<0.001). One major limitation of the study was that socioeconomic status was not volunteered by approximately 30% of the applicants. Those who withheld socioeconomic status data were significantly different from those who provided that information, which may have caused bias in the analysis.
Conclusions UKCAT was introduced with a high expectation of increasing the diversity and fairness in selection for UK medical and dental schools. This study of a major subgroup of applicants in the first year of operation suggests that it has an inherent favourable bias to men and students from a higher socioeconomic class or independent or grammar schools. However, it does provide a reasonable proxy for A levels in the selection process.
PMCID: PMC2824099  PMID: 20160316
9.  Association of Medical Students' Reports of Interactions with the Pharmaceutical and Medical Device Industries and Medical School Policies and Characteristics: A Cross-Sectional Study 
PLoS Medicine  2014;11(10):e1001743.
Aaron Kesselheim and colleagues compared US medical students' survey responses regarding pharmaceutical company interactions with the schools' AMSA PharmFree scorecard and Institute on Medicine as a Profession's (IMAP) scores.
Please see later in the article for the Editors' Summary
Professional societies use metrics to evaluate medical schools' policies regarding interactions of students and faculty with the pharmaceutical and medical device industries. We compared these metrics and determined which US medical schools' industry interaction policies were associated with student behaviors.
Methods and Findings
Using survey responses from a national sample of 1,610 US medical students, we compared their reported industry interactions with their schools' American Medical Student Association (AMSA) PharmFree Scorecard and average Institute on Medicine as a Profession (IMAP) Conflicts of Interest Policy Database score. We used hierarchical logistic regression models to determine the association between policies and students' gift acceptance, interactions with marketing representatives, and perceived adequacy of faculty–industry separation. We adjusted for year in training, medical school size, and level of US National Institutes of Health (NIH) funding. We used LASSO regression models to identify specific policies associated with the outcomes. We found that IMAP and AMSA scores had similar median values (1.75 [interquartile range 1.50–2.00] versus 1.77 [1.50–2.18], adjusted to compare scores on the same scale). Scores on AMSA and IMAP shared policy dimensions were not closely correlated (gift policies, r = 0.28, 95% CI 0.11–0.44; marketing representative access policies, r = 0.51, 95% CI 0.36–0.63). Students from schools with the most stringent industry interaction policies were less likely to report receiving gifts (AMSA score, odds ratio [OR]: 0.37, 95% CI 0.19–0.72; IMAP score, OR 0.45, 95% CI 0.19–1.04) and less likely to interact with marketing representatives (AMSA score, OR 0.33, 95% CI 0.15–0.69; IMAP score, OR 0.37, 95% CI 0.14–0.95) than students from schools with the lowest ranked policy scores. The association became nonsignificant when fully adjusted for NIH funding level, whereas adjusting for year of education, size of school, and publicly versus privately funded school did not alter the association. Policies limiting gifts, meals, and speaking bureaus were associated with students reporting having not received gifts and having not interacted with marketing representatives. Policy dimensions reflecting the regulation of industry involvement in educational activities (e.g., continuing medical education, travel compensation, and scholarships) were associated with perceived separation between faculty and industry. The study is limited by potential for recall bias and the cross-sectional nature of the survey, as school curricula and industry interaction policies may have changed since the time of the survey administration and study analysis.
As medical schools review policies regulating medical students' industry interactions, limitations on receipt of gifts and meals and participation of faculty in speaking bureaus should be emphasized, and policy makers should pay greater attention to less research-intensive institutions.
Please see later in the article for the Editors' Summary
Editors' Summary
Making and selling prescription drugs and medical devices is big business. To promote their products, pharmaceutical and medical device companies build relationships with physicians by providing information on new drugs, by organizing educational meetings and sponsored events, and by giving gifts. Financial relationships begin early in physicians' careers, with companies providing textbooks and other gifts to first-year medical students. In medical school settings, manufacturers may help to inform trainees and physicians about developments in health care, but they also create the potential for harm to patients and health care systems. These interactions may, for example, reduce trainees' and trained physicians' skepticism about potentially misleading promotional claims and may encourage physicians to prescribe new medications, which are often more expensive than similar unbranded (generic) drugs and more likely to be recalled for safety reasons than older drugs. To address these and other concerns about the potential career-long effects of interactions between medical trainees and industry, many teaching hospitals and medical schools have introduced policies to limit such interactions. The development of these policies has been supported by expert professional groups and medical societies, some of which have created scales to evaluate the strength of the implemented industry interaction policies.
Why Was This Study Done?
The impact of policies designed to limit interactions between students and industry on student behavior is unclear, and it is not known which aspects of the policies are most predictive of student behavior. This information is needed to ensure that the policies are working and to identify ways to improve them. Here, the researchers investigate which medical school characteristics and which aspects of industry interaction policies are most predictive of students' reported behaviors and beliefs by comparing information collected in a national survey of US medical students with the strength of their schools' industry interaction policies measured on two scales—the American Medical Student Association (AMSA) PharmFree Scorecard and the Institute on Medicine as a Profession (IMAP) Conflicts of Interest Policy Database.
What Did the Researchers Do and Find?
The researchers compared information about reported gift acceptance, interactions with marketing representatives, and the perceived adequacy of faculty–industry separation collected from 1,610 medical students at 121 US medical schools with AMSA and IMAP scores for the schools evaluated a year earlier. Students at schools with the highest ranked interaction policies based on the AMSA score were 63% less likely to accept gifts as students at the lowest ranked schools. Students at the highest ranked schools based on the IMAP score were about half as likely to accept gifts as students at the lowest ranked schools, although this finding was not statistically significant (it could be a chance finding). Similarly, students at the highest ranked schools were 70% less likely to interact with sales representatives as students at the lowest ranked schools. These associations became statistically nonsignificant after controlling for the amount of research funding each school received from the US National Institutes of Health (NIH). Policies limiting gifts, meals, and being a part of speaking bureaus (where companies pay speakers to present information about the drugs for dinners and other events) were associated with students' reports of receiving no gifts and of non-interaction with sales representatives. Finally, policies regulating industry involvement in educational activities were associated with the perceived separation between faculty and industry, which was regarded as adequate by most of the students at schools with such policies.
What Do These Findings Mean?
These findings suggest that policies designed to limit industry interactions with medical students need to address multiple aspects of these interactions to achieve changes in the behavior and attitudes of trainees, but that policies limiting gifts, meals, and speaking bureaus may be particularly important. These findings also suggest that the level of NIH funding plays an important role in students' self-reported behaviors and their perceptions of industry, possibly because institutions with greater NIH funding have the resources needed to implement effective policies. The accuracy of these findings may be limited by recall bias (students may have reported their experiences inaccurately), and by the possibility that industry interaction policies may have changed in the year that elapsed between policy grading and the student survey. Nevertheless, these findings suggest that limitations on gifts should be emphasized when academic medical centers refine their policies on interactions between medical students and industry and that particular attention should be paid to the design and implementation of policies that regulate industry interactions in institutions with lower levels of NIH funding.
Additional Information
Please access these websites via the online version of this summary at
The UK General Medical Council provides guidance on financial and commercial arrangements and conflicts of interest as part of its good medical practice document, which describes what is required of all registered doctors in the UK
Information about the American Medical Student Association (AMSA) Just Medicine campaign (formerly the PharmFree campaign) and about the AMSA Scorecard is available
Information about the Institute on Medicine as a Profession (IMAP) and about its Conflicts of Interest Policy Database is also available
“Understanding and Responding to Pharmaceutical Promotion: A Practical Guide” is a manual prepared by Health Action International and the World Health Organization that medical schools can use to train students how to recognize and respond to pharmaceutical promotion
The US Institute of Medicine's report “Conflict of Interest in Medical Research, Education, and Practice” recommends steps to identify, limit, and manage conflicts of interest
The ALOSA Foundation provides evidence-based, non-industry-funded education about treating common conditions and using prescription drugs
PMCID: PMC4196737  PMID: 25314155
10.  Evaluating professionalism in medical undergraduates using selected response questions: findings from an item response modelling study 
BMC Medical Education  2011;11:43.
Professionalism is a difficult construct to define in medical students but aspects of this concept may be important in predicting the risk of postgraduate misconduct. For this reason attempts are being made to evaluate medical students' professionalism. This study investigated the psychometric properties of Selected Response Questions (SRQs) relating to the theme of professional conduct and ethics comparing them with two sets of control items: those testing pure knowledge of anatomy, and; items evaluating the ability to integrate and apply knowledge ("skills"). The performance of students on the SRQs was also compared with two external measures estimating aspects of professionalism in students; peer ratings of professionalism and their Conscientiousness Index, an objective measure of behaviours at medical school.
Item Response Theory (IRT) was used to analyse both question and student performance for SRQs relating to knowledge of professionalism, pure anatomy and skills. The relative difficulties, discrimination and 'guessabilities' of each theme of question were compared with each other using Analysis of Variance (ANOVA). Student performance on each topic was compared with the measures of conscientiousness and professionalism using parametric and non-parametric tests as appropriate. A post-hoc analysis of power for the IRT modelling was conducted using a Monte Carlo simulation.
Professionalism items were less difficult compared to the anatomy and skills SRQs, poorer at discriminating between candidates and more erratically answered when compared to anatomy questions. Moreover professionalism item performance was uncorrelated with the standardised Conscientiousness Index scores (rho = 0.009, p = 0.90). In contrast there were modest but significant correlations between standardised Conscientiousness Index scores and performance at anatomy items (rho = 0.20, p = 0.006) though not skills (rho = .11, p = .1). Likewise, students with high peer ratings for professionalism had superior performance on anatomy SRQs but not professionalism themed questions. A trend of borderline significance (p = .07) was observed for performance on skills SRQs and professionalism nomination status.
SRQs related to professionalism are likely to have relatively poor psychometric properties and lack associations with other constructs associated with undergraduate professional behaviour. The findings suggest that such questions should not be included in undergraduate examinations and may raise issues with the introduction of Situational Judgement Tests into Foundation Years selection.
PMCID: PMC3146946  PMID: 21714870
11.  'It gives you an understanding you can't get from any book.' The relationship between medical students' and doctors' personal illness experiences and their performance: a qualitative and quantitative study 
Anecdotes abound about doctors' personal illness experiences and the effect they have on their empathy and care of patients. We formally investigated the relationship between doctors' and medical students' personal illness experiences, their examination results, preparedness for clinical practice, learning and professional attitudes and behaviour towards patients.
Newly-qualified UK doctors in 2005 (n = 2062/4784), and two cohorts of students at one London medical school (n = 640/749) participated in the quantitative arm of the study. 37 Consultants, 1 Specialist Registrar, 2 Clinical Skills Tutors and 25 newly-qualified doctors participated in the qualitative arm. Newly-qualified doctors and medical students reported their personal illness experiences in a questionnaire. Doctors' experiences were correlated with self-reported preparedness for their new clinical jobs. Students' experiences were correlated with their examination results, and self-reported anxiety and depression. Interviews with clinical teachers, newly-qualified doctors and senior doctors qualitatively investigated how personal illness experiences affect learning, professional attitudes, and behaviour.
85.5% of newly-qualified doctors and 54.4% of medical students reported personal illness experiences. Newly-qualified doctors who had been ill felt less prepared for starting work (p < 0.001), but those who had only experienced illness in a relative or friend felt more prepared (p = 0.02). Clinical medical students who had been ill were more anxious (p = 0.01) and had lower examination scores (p = 0.006). Doctors felt their personal illness experiences helped them empathise and communicate with patients. Medical students with more life experience were perceived as more mature, empathetic, and better learners; but illness at medical school was recognised to impede learning.
The majority of the medical students and newly qualified doctors we studied reported personal illness experiences, and these experiences were associated with lower undergraduate examination results, higher anxiety, and lower preparedness. However reflection on such experiences may have improved professional attitudes such as empathy and compassion for patients. Future research is warranted in this area.
PMCID: PMC2211477  PMID: 18053231
12.  Comparison of the sensitivity of the UKCAT and A Levels to sociodemographic characteristics: a national study 
The UK Clinical Aptitude Test (UKCAT) was introduced to facilitate widening participation in medical and dental education in the UK by providing universities with a continuous variable to aid selection; one that might be less sensitive to the sociodemographic background of candidates compared to traditional measures of educational attainment. Initial research suggested that males, candidates from more advantaged socioeconomic backgrounds and those who attended independent or grammar schools performed better on the test. The introduction of the A* grade at A level permits more detailed analysis of the relationship between UKCAT scores, secondary educational attainment and sociodemographic variables. Thus, our aim was to further assess whether the UKCAT is likely to add incremental value over A level (predicted or actual) attainment in the selection process.
Data relating to UKCAT and A level performance from 8,180 candidates applying to medicine in 2009 who had complete information relating to six key sociodemographic variables were analysed. A series of regression analyses were conducted in order to evaluate the ability of sociodemographic status to predict performance on two outcome measures: A level ‘best of three’ tariff score; and the UKCAT scores.
In this sample A level attainment was independently and positively predicted by four sociodemographic variables (independent/grammar schooling, White ethnicity, age and professional social class background). These variables also independently and positively predicted UKCAT scores. There was a suggestion that UKCAT scores were less sensitive to educational background compared to A level attainment. In contrast to A level attainment, UKCAT score was independently and positively predicted by having English as a first language and male sex.
Our findings are consistent with a previous report; most of the sociodemographic factors that predict A level attainment also predict UKCAT performance. However, compared to A levels, males and those speaking English as a first language perform better on UKCAT. Our findings suggest that UKCAT scores may be more influenced by sex and less sensitive to school type compared to A levels. These factors must be considered by institutions utilising the UKCAT as a component of the medical and dental school selection process.
PMCID: PMC3893425  PMID: 24400861
Medical student selection; Educational attainment; Aptitude tests; UKCAT; Socio-economic factors
13.  A holistic review of the medical school admission process: examining correlates of academic underperformance 
Medical Education Online  2014;19:10.3402/meo.v19.22919.
Despite medical school admission committees’ best efforts, a handful of seemingly capable students invariably struggle during their first year of study. Yet, even as entrance criteria continue to broaden beyond cognitive qualifications, attention inevitably reverts back to such factors when seeking to understand these phenomena. Using a host of applicant, admission, and post-admission variables, the purpose of this inductive study, then, was to identify a constellation of student characteristics that, taken collectively, would be predictive of students at-risk of underperforming during the first year of medical school. In it, we hypothesize that a wider range of factors than previously recognized could conceivably play roles in understanding why students experience academic problems early in the medical educational continuum.
The study sample consisted of the five most recent matriculant cohorts from a large, southeastern medical school (n=537). Independent variables reflected: 1) the personal demographics of applicants (e.g., age, gender); 2) academic criteria (e.g., undergraduate grade point averages [GPA], medical college admission test); 3) selection processes (e.g., entrance track, interview scores, committee votes); and 4) other indicators of personality and professionalism (e.g., Mayer-Salovey-Caruso Emotional Intelligence Test™ emotional intelligence scores, NEO PI-R™ personality profiles, and appearances before the Professional Code Committee [PCC]). The dependent variable, first-year underperformance, was defined as ANY action (repeat, conditionally advance, or dismiss) by the college's Student Progress and Promotions Committee (SPPC) in response to predefined academic criteria. This study protocol was approved by the local medical institutional review board (IRB).
Of the 537 students comprising the study sample, 61 (11.4%) met the specified criterion for academic underperformance. Significantly increased academic risks were identified among students who 1) had lower mean undergraduate science GPAs (OR=0.24, p=0.001); 2) entered medical school via an accelerated BS/MD track (OR=16.15, p=0.002); 3) were 31 years of age or older (OR=14.76, p=0.005); and 4) were non-unanimous admission committee admits (OR=0.53, p=0.042). Two dimensions of the NEO PI-R™ personality inventory, openness (+) and conscientiousness (−), were modestly but significantly correlated with academic underperformance. Only for the latter, however, were mean scores found to differ significantly between academic performers and underperformers. Finally, appearing before the college's PCC (OR=4.21, p=0.056) fell just short of statistical significance.
Our review of various correlates across the matriculation process highlights the heterogeneity of factors underlying students’ underperformance during the first year of medical school and challenges medical educators to understand the complexity of predicting who, among admitted matriculants, may be at future academic risk.
PMCID: PMC3974177  PMID: 24695362
admissions; underperformance; selection; at-risk students
14.  Effects of participation in a cross year peer tutoring programme in clinical examination skills on volunteer tutors' skills and attitudes towards teachers and teaching 
Development of students' teaching skills is increasingly recognised as an important component of UK undergraduate medical curricula and, in consequence, there is renewed interest in the potential benefits of cross-year peer tutoring. Whilst several studies have described the use of cross-year peer tutoring in undergraduate medical courses, its use in the clinical setting is less well reported, particularly the effects of peer tutoring on volunteer tutors' views of teachers and teaching. This study explored the effects of participation in a cross-year peer tutoring programme in clinical examination skills ('OSCE tutor') on volunteer tutors' own skills and on their attitudes towards teachers and teaching.
Volunteer tutors were final year MBChB students who took part in the programme as part of a Student Selected Component (SSC). Tutees were year 3 MBChB students preparing for their end of year 'OSCE' examination. Pre and post participation questionnaires, including both Likert-type and open response questions, were used. Paired data was compared using the Wilcoxon signed-rank test. All tests were two-tailed with 5% significance level.
Tutors reflected their cohort in terms of gender but were drawn from among the more academically successful final year students. Most had previous teaching experience. They were influenced to participate in 'OSCE tutor' by a desire to improve their own teaching and associated generic skills and by contextual factors relating to the organisation or previous experience of the OSCE tutor programme. Issues relating to longer term career aspirations were less important. After the event, tutors felt that participation had enhanced their skills in various areas, including practical teaching skills, confidence in speaking to groups and communication skills; and that as a result of taking part, they were now more likely to undertake further teacher training and to make teaching a major part of their career. However, whilst a number of students reported that their views of teachers and teaching had changed as a result of participation, this did not translate into significant changes in responses to questions that explored their views of the roles and qualities required of a good clinical teacher.
Findings affirm the benefits to volunteer tutors of cross-year peer tutoring, particularly in terms of skills enhancement and reinforcement of positive attitudes towards future teaching responsibilities, and have implications for the design and organisation of such programmes.
PMCID: PMC1925072  PMID: 17598885
15.  The value of the UK Clinical Aptitude Test in predicting pre-clinical performance: a prospective cohort study at Nottingham Medical School 
BMC Medical Education  2010;10:55.
The UK Clinical Aptitude Test (UKCAT) was introduced in 2006 as an additional tool for the selection of medical students. It tests mental ability in four distinct domains (Quantitative Reasoning, Verbal Reasoning, Abstract Reasoning, and Decision Analysis), and the results are available to students and admissions panels in advance of the selection process. As yet the predictive validity of the test against course performance is largely unknown.
The study objective was to determine whether UKCAT scores predict performance during the first two years of the 5-year undergraduate medical course at Nottingham.
We studied a single cohort of students, who entered Nottingham Medical School in October 2007 and had taken the UKCAT. We used linear regression analysis to identify independent predictors of marks for different parts of the 2-year preclinical course.
Data were available for 204/260 (78%) of the entry cohort. The UKCAT total score had little predictive value. Quantitative Reasoning was a significant independent predictor of course marks in Theme A ('The Cell'), (p = 0.005), and Verbal Reasoning predicted Theme C ('The Community') (p < 0.001), but otherwise the effects were slight or non-existent.
This limited study from a single entry cohort at one medical school suggests that the predictive value of the UKCAT, particularly the total score, is low. Section scores may predict success in specific types of course assessment.
The ultimate test of validity will not be available for some years, when current cohorts of students graduate. However, if this test of mental ability does not predict preclinical performance, it is arguably less likely to predict the outcome in the clinical years. Further research from medical schools with different types of curriculum and assessment is needed, with longitudinal studies throughout the course.
PMCID: PMC2922293  PMID: 20667093
16.  First year clinical tutorials: students’ learning experience 
Bedside teaching lies at the heart of medical education. The learning environment afforded to students during clinical tutorials contributes substantially to their knowledge, thinking, and learning. Situated cognition theory posits that the depth and breadth of the students’ learning experience is dependent upon the attitude of the clinical teacher, the structure of the tutorial, and the understanding of tutorial and learning objectives. This theory provides a useful framework to conceptualize how students’ experience within their clinical tutorials impacts their knowledge, thinking, and learning.
The study was conducted with one cohort (n=301) of students who had completed year 1 of the medical program at Sydney Medical School in 2013. All students were asked to complete a three-part questionnaire regarding their perceptions of their clinical tutor’s attributes, the consistency of the tutor, and the best features of the tutorials and need for improvement. Both quantitative and qualitative data were collected and analyzed using descriptive statistics.
The response rate to the questionnaire was 88% (265/301). Students perceived that their tutors displayed good communication skills and enthusiasm, encouraged their learning, and were empathetic toward patients. Fifty-two percent of students reported having the same communications tutor for the entire year, and 28% reported having the same physical examination tutor for the entire year. Students would like increased patient contact, greater structure within their tutorials, and greater alignment of teaching with the curriculum.
Situated cognition theory provides a valuable lens to view students’ experience of learning within the clinical environment. Our findings demonstrate students’ appreciation of clinical tutors as role models, the need for consistency in feedback, the importance of structure within tutorials, and the need for tutors to have an understanding of the curriculum and learning objectives for each teaching session.
PMCID: PMC4257052  PMID: 25489253
bedside teaching; clinical tutorials; role modeling; situated cognition
17.  The UK clinical aptitude test and clinical course performance at Nottingham: a prospective cohort study 
BMC Medical Education  2013;13:32.
The UK Clinical Aptitude Test (UKCAT) was introduced in 2006 as an additional tool for the selection of medical students. It tests mental ability in four distinct domains (Verbal Reasoning, Quantitative Reasoning, Abstract Reasoning, and Decision Analysis), and the results are available to students and admission panels in advance of the selection process. Our first study showed little evidence of any predictive validity for performance in the first two years of the Nottingham undergraduate course.
The study objective was to determine whether the UKCAT scores had any predictive value for the later parts of the course, largely delivered via clinical placements.
Students entering the course in 2007 and who had taken the UKCAT were asked for permission to use their anonymised data in research. The UKCAT scores were incorporated into a database with routine pre-admission socio-demographics and subsequent course performance data. Correlation analysis was followed by hierarchical multivariate linear regression.
The original study group comprised 204/254 (80%) of the full entry cohort. With attrition over the five years of the course this fell to 185 (73%) by Year 5. The Verbal Reasoning score and the UKCAT Total score both demonstrated some univariate correlations with clinical knowledge marks, and slightly less with clinical skills. No parts of the UKCAT proved to be an independent predictor of clinical course marks, whereas prior attainment was a highly significant predictor (p <0.001).
This study of one cohort of Nottingham medical students showed that UKCAT scores at admission did not independently predict subsequent performance on the course. Whilst the test adds another dimension to the selection process, its fairness and validity in selecting promising students remains unproven, and requires wider investigation and debate by other schools.
PMCID: PMC3621812  PMID: 23442227
18.  Consumers as tutors – legitimate teachers? 
The aim of this study was to research the feasibility of training mental health consumers as tutors for 4th year medical students in psychiatry.
A partnership between a consumer network and an academic unit in Psychological Medicine was formed to jointly develop a training package for consumer tutors and a curriculum in interviewing skills for medical students. Student attitudes to mental health consumers were measured pre and post the program. All tutorial evaluation data was analysed using univariate statistics. Both tutors and students evaluated the teaching program using a 4 point rating scale. The mean scores for teaching and content for both students and tutors were compared using an independent samples t-test.
Consumer tutors were successfully trained and accredited as tutors and able to sustain delivery of tutorials over a 4 year period. The study found that whilst the medical students started with positive attitudes towards consumers prior to the program, there was a general trend towards improved attitude across all measures. Other outcomes for tutors and students (both positive and negative) are described.
Consumer tutors along with professional tutors have a place in the education of medical students, are an untapped resource and deliver largely positive outcomes for students and themselves. Further possible developments are described.
PMCID: PMC524164  PMID: 15377386
19.  Western medical ethics taught to junior medical students can cross cultural and linguistic boundaries 
BMC Medical Ethics  2004;5:4.
Little is known about teaching medical ethics across cultural and linguistic boundaries. This study examined two successive cohorts of first year medical students in a six year undergraduate MBBS program.
The objective was to investigate whether Arabic speaking students studying medicine in an Arabic country would be able to correctly identify some of the principles of Western medical ethical reasoning. This cohort study was conducted on first year students in a six-year undergraduate program studying medicine in English, their second language at a medical school in the Arabian Gulf. The ethics teaching was based on the four-principle approach (autonomy, beneficence, non-malfeasance and justice) and delivered by a non-Muslim native English speaker with no knowledge of the Arabic language. Although the course was respectful of Arabic culture and tradition, the content excluded an analysis of Islamic medical ethics and focused on Western ethical reasoning. Following two 45-minute interactive seminars, students in groups of 3 or 4 visited a primary health care centre for one morning, sitting in with an attending physician seeing his or her patients in Arabic. Each student submitted a personal report for summative assessment detailing the ethical issues they had observed.
All 62 students enrolled in these courses participated. Each student acting independently was able to correctly identify a median number of 4 different medical ethical issues (range 2–9) and correctly identify and label accurately a median of 2 different medical ethical issues (range 2–7) There were no significant correlations between their English language skills or general academic ability and the number or accuracy of ethical issues identified.
This study has demonstrated that these students could identify medical ethical issues based on Western constructs, despite learning in English, their second language, being in the third week of their medical school experience and with minimal instruction. This result was independent of their academic and English language skills suggesting that ethical principles as espoused in the four principal approach may be common to the students' Islamic religious beliefs, allowing them to access complex medical ethical reasoning skills at an early stage in the medical curriculum.
PMCID: PMC509252  PMID: 15283868
20.  Behavioural Interventions for Urinary Incontinence in Community-Dwelling Seniors 
Executive Summary
In early August 2007, the Medical Advisory Secretariat began work on the Aging in the Community project, an evidence-based review of the literature surrounding healthy aging in the community. The Health System Strategy Division at the Ministry of Health and Long-Term Care subsequently asked the secretariat to provide an evidentiary platform for the ministry’s newly released Aging at Home Strategy.
After a broad literature review and consultation with experts, the secretariat identified 4 key areas that strongly predict an elderly person’s transition from independent community living to a long-term care home. Evidence-based analyses have been prepared for each of these 4 areas: falls and fall-related injuries, urinary incontinence, dementia, and social isolation. For the first area, falls and fall-related injuries, an economic model is described in a separate report.
Please visit the Medical Advisory Secretariat Web site,, to review these titles within the Aging in the Community series.
Aging in the Community: Summary of Evidence-Based Analyses
Prevention of Falls and Fall-Related Injuries in Community-Dwelling Seniors: An Evidence-Based Analysis
Behavioural Interventions for Urinary Incontinence in Community-Dwelling Seniors: An Evidence-Based Analysis
Caregiver- and Patient-Directed Interventions for Dementia: An Evidence-Based Analysis
Social Isolation in Community-Dwelling Seniors: An Evidence-Based Analysis
The Falls/Fractures Economic Model in Ontario Residents Aged 65 Years and Over (FEMOR)
To assess the effectiveness of behavioural interventions for the treatment and management of urinary incontinence (UI) in community-dwelling seniors.
Clinical Need: Target Population and Condition
Urinary incontinence defined as “the complaint of any involuntary leakage of urine” was identified as 1 of the key predictors in a senior’s transition from independent community living to admission to a long-term care (LTC) home. Urinary incontinence is a health problem that affects a substantial proportion of Ontario’s community-dwelling seniors (and indirectly affects caregivers), impacting their health, functioning, well-being and quality of life. Based on Canadian studies, prevalence estimates range from 9% to 30% for senior men and nearly double from 19% to 55% for senior women. The direct and indirect costs associated with UI are substantial. It is estimated that the total annual costs in Canada are $1.5 billion (Cdn), and that each year a senior living at home will spend $1,000 to $1,500 on incontinence supplies.
Interventions to treat and manage UI can be classified into broad categories which include lifestyle modification, behavioural techniques, medications, devices (e.g., continence pessaries), surgical interventions and adjunctive measures (e.g., absorbent products).
The focus of this review is behavioural interventions, since they are commonly the first line of treatment considered in seniors given that they are the least invasive options with no reported side effects, do not limit future treatment options, and can be applied in combination with other therapies. In addition, many seniors would not be ideal candidates for other types of interventions involving more risk, such as surgical measures.
Note: It is recognized that the terms “senior” and “elderly” carry a range of meanings for different audiences; this report generally uses the former, but the terms are treated here as essentially interchangeable.
Description of Technology/Therapy
Behavioural interventions can be divided into 2 categories according to the target population: caregiver-dependent techniques and patient-directed techniques. Caregiver-dependent techniques (also known as toileting assistance) are targeted at medically complex, frail individuals living at home with the assistance of a caregiver, who tends to be a family member. These seniors may also have cognitive deficits and/or motor deficits. A health care professional trains the senior’s caregiver to deliver an intervention such as prompted voiding, habit retraining, or timed voiding. The health care professional who trains the caregiver is commonly a nurse or a nurse with advanced training in the management of UI, such as a nurse continence advisor (NCA) or a clinical nurse specialist (CNS).
The second category of behavioural interventions consists of patient-directed techniques targeted towards mobile, motivated seniors. Seniors in this population are cognitively able, free from any major physical deficits, and motivated to regain and/or improve their continence. A nurse or a nurse with advanced training in UI management, such as an NCA or CNS, delivers the patient-directed techniques. These are often provided as multicomponent interventions including a combination of bladder training techniques, pelvic floor muscle training (PFMT), education on bladder control strategies, and self-monitoring. Pelvic floor muscle training, defined as a program of repeated pelvic floor muscle contractions taught and supervised by a health care professional, may be employed as part of a multicomponent intervention or in isolation.
Education is a large component of both caregiver-dependent and patient-directed behavioural interventions, and patient and/or caregiver involvement as well as continued practice strongly affect the success of treatment. Incontinence products, which include a large variety of pads and devices for effective containment of urine, may be used in conjunction with behavioural techniques at any point in the patient’s management.
Evidence-Based Analysis Methods
A comprehensive search strategy was used to identify systematic reviews and randomized controlled trials that examined the effectiveness, safety, and cost-effectiveness of caregiver-dependent and patient-directed behavioural interventions for the treatment of UI in community-dwelling seniors (see Appendix 1).
Research Questions
Are caregiver-dependent behavioural interventions effective in improving UI in medically complex, frail community-dwelling seniors with/without cognitive deficits and/or motor deficits?
Are patient-directed behavioural interventions effective in improving UI in mobile, motivated community-dwelling seniors?
Are behavioural interventions delivered by NCAs or CNSs in a clinic setting effective in improving incontinence outcomes in community-dwelling seniors?
Assessment of Quality of Evidence
The quality of the evidence was assessed as high, moderate, low, or very low according to the GRADE methodology and GRADE Working Group. As per GRADE the following definitions apply:
Summary of Findings
Executive Summary Table 1 summarizes the results of the analysis.
The available evidence was limited by considerable variation in study populations and in the type and severity of UI for studies examining both caregiver-directed and patient-directed interventions. The UI literature frequently is limited to reporting subjective outcome measures such as patient observations and symptoms. The primary outcome of interest, admission to a LTC home, was not reported in the UI literature. The number of eligible studies was low, and there were limited data on long-term follow-up.
Summary of Evidence on Behavioural Interventions for the Treatment of Urinary Incontinence in Community-Dwelling Seniors
Prompted voiding
Habit retraining
Timed voiding
Bladder training
PFMT (with or without biofeedback)
Bladder control strategies
CI refers to confidence interval; CNS, clinical nurse specialist; NCA, nurse continence advisor; PFMT, pelvic floor muscle training; RCT, randomized controlled trial; WMD, weighted mean difference; UI, urinary incontinence.
Economic Analysis
A budget impact analysis was conducted to forecast costs for caregiver-dependent and patient-directed multicomponent behavioural techniques delivered by NCAs, and PFMT alone delivered by physiotherapists. All costs are reported in 2008 Canadian dollars. Based on epidemiological data, published medical literature and clinical expert opinion, the annual cost of caregiver-dependent behavioural techniques was estimated to be $9.2 M, while the annual costs of patient-directed behavioural techniques delivered by either an NCA or physiotherapist were estimated to be $25.5 M and $36.1 M, respectively. Estimates will vary if the underlying assumptions are changed.
Currently, the province of Ontario absorbs the cost of NCAs (available through the 42 Community Care Access Centres across the province) in the home setting. The 2007 Incontinence Care in the Community Report estimated that the total cost being absorbed by the public system of providing continence care in the home is $19.5 M in Ontario. This cost estimate included resources such as personnel, communication with physicians, record keeping and product costs. Clinic costs were not included in this estimation because currently these come out of the global budget of the respective hospital and very few continence clinics actually exist in the province. The budget impact analysis factored in a cost for the clinic setting, assuming that the public system would absorb the cost with this new model of community care.
Considerations for Ontario Health System
An expert panel on aging in the community met on 3 occasions from January to May 2008, and in part, discussed treatment of UI in seniors in Ontario with a focus on caregiver-dependent and patient-directed behavioural interventions. In particular, the panel discussed how treatment for UI is made available to seniors in Ontario and who provides the service. Some of the major themes arising from the discussions included:
Services/interventions that currently exist in Ontario offering behavioural interventions to treat UI are not consistent. There is a lack of consistency in how seniors access services for treatment of UI, who manages patients and what treatment patients receive.
Help-seeking behaviours are important to consider when designing optimal service delivery methods.
There is considerable social stigma associated with UI and therefore there is a need for public education and an awareness campaign.
The cost of incontinent supplies and the availability of NCAs were highlighted.
There is moderate-quality evidence that the following interventions are effective in improving UI in mobile motivated seniors:
Multicomponent behavioural interventions including a combination of bladder training techniques, PFMT (with or without biofeedback), education on bladder control strategies and self-monitoring techniques.
Pelvic floor muscle training alone.
There is moderate quality evidence that when behavioural interventions are led by NCAs or CNSs in a clinic setting, they are effective in improving UI in seniors.
There is limited low-quality evidence that prompted voiding may be effective in medically complex, frail seniors with motivated caregivers.
There is insufficient evidence for the following interventions in medically complex, frail seniors with motivated caregivers:
habit retraining, and
timed voiding.
PMCID: PMC3377527  PMID: 23074508
21.  Emotional intelligence and academic performance in first and final year medical students: a cross-sectional study 
BMC Medical Education  2013;13:44.
Research on emotional intelligence (EI) suggests that it is associated with more pro-social behavior, better academic performance and improved empathy towards patients. In medical education and clinical practice, EI has been related to higher academic achievement and improved doctor-patient relationships. This study examined the effect of EI on academic performance in first- and final-year medical students in Malaysia.
This was a cross-sectional study using an objectively-scored measure of EI, the Mayer-Salovey-Caruso Emotional Intelligence Test (MSCEIT). Academic performance of medical school students was measured using continuous assessment (CA) and final examination (FE) results. The first- and final-year students were invited to participate during their second semester. Students answered a paper-based demographic questionnaire and completed the online MSCEIT on their own. Relationships between the total MSCEIT score to academic performance were examined using multivariate analyses.
A total of 163 (84 year one and 79 year five) medical students participated (response rate of 66.0%). The gender and ethnic distribution were representative of the student population. The total EI score was a predictor of good overall CA (OR 1.01), a negative predictor of poor result in overall CA (OR 0.97), a predictor of the good overall FE result (OR 1.07) and was significantly related to the final-year FE marks (adjusted R2 = 0.43).
Medical students who were more emotionally intelligent performed better in both the continuous assessments and the final professional examination. Therefore, it is possible that emotional skill development may enhance medical students’ academic performance.
PMCID: PMC3617036  PMID: 23537129
Emotional intelligence; Educational assessments; Achievement; Medical students
22.  Development and psychometric testing of an instrument to evaluate cognitive skills of evidence based practice in student health professionals 
BMC Medical Education  2011;11:77.
Health educators need rigorously developed instruments to evaluate cognitive skills relating to evidence based practice (EBP). Previous EBP evaluation instruments have focused on the acquisition and appraisal of the evidence and are largely based in the medical profession. The aim of this study was to develop and validate an EBP evaluation instrument to assess EBP cognitive skills for entry-level health professional disciplines.
The Fresno test of competence in evidence based medicine was considered in the development of the 'Knowledge of Research Evidence Competencies' instrument (K-REC). The K-REC was reviewed for content validity. Two cohorts of entry-level students were recruited for the pilot study, those who had been exposed to EBP training (physiotherapy students, n = 24), and who had not been exposed to EBP training (human movement students, n = 76). The K-REC was administered to one cohort of students (n = 24) on two testing occasions to evaluate test-retest reliability. Two raters independently scored the first test occasion (n = 24) to evaluate the inter-rater reliability of the marking guidelines. Construct validity was assessed by comparison of the two groups, 'exposed' and 'non-exposed', and the percentage of students achieving a 'pass' score in each of these groups. Item difficulty was established.
Among the 100 participants (24 EBP 'exposed', and 76 EBP 'non-exposed' students), there was a statistically significant (p < 0.0001) difference in the total K-REC scores. The test-retest and inter-rater reliability of the individual items and total scores ranged from moderate to excellent (measured by Cohen's Kappa and ICC, range: 0.62 to perfect agreement).
The K-REC instrument is a valid and reliable evaluation instrument of cognitive skills of EBP in entry-level student health professionals. The instrument is quick to disseminate and easy to score, making it a suitable instrument for health educators to employ to evaluate students' knowledge of EBP or in the evaluation of entry-level EBP training.
PMCID: PMC3196731  PMID: 21967728
23.  Examiner and simulated patient ratings of empathy in medical student final year clinical examination: are they useful? 
BMC Medical Education  2014;14(1):199.
Many medical schools state that empathy is important and have curricular learning outcomes covering its teaching. It is thought to be useful in team-working, good bedside manner, patient perspective taking, and improved patient care. Given this, one might expect it to be measured in assessment processes. Despite this, there is relatively little literature exploring how measures of empathy in final clinical examinations in medical school map onto other examination scores. Little is known about simulated patient (actors) rating of empathy in examinations in terms of inter-rater reliability compared with clinical assessors or correlation with overall examination results.
Examiners in final year clinical assessments in one UK medical school rated 133 students on five constructs in Objective Structured Long Examination Record (OSLER) with real patients: gathering information, physical examination, problem solving, managing the diagnosis, and relationship with the patient. Scores were based on a standardized well-established penalty point system. In separate Objective Structured Clinical Examination (OSCE) stations, different examiners used the same penalty point system to score performance in both interactional and procedural stations. In the four interaction-based OSCE stations, examiners and simulated patient actors also independently rated empathy of the students.
The OSLER score, based on penalty points, had a correlation of −0.38 with independent ratings of empathy from the interactional OSCE stations. The intra-class correlation (a measure of inter-rater reliability) between the observing clinical tutor and ratings from simulated patients was 0.645 with very similar means. There was a significant difference between the empathy scores of the 94 students passing the first part of the sequential examination, based on combined OSCE and OSLER scores (which did not include the empathy scores), and 39 students with sufficient penalty points to trigger attendance for the second part (Cohen’s d = 0.81).
These findings suggest that empathy ratings are related to clinical performance as measured by independent examiners. Simulated patient actors are able to give clinically meaningful assessment scores. This gives preliminary evidence that such empathy ratings could be useful for formative learning, and bolsters the call for more research to test whether they are robust enough to be used summatively.
PMCID: PMC4261253  PMID: 25245476
Empathy; Medical students; Simulated patients; Final clinical examination; OSLER; OSCE
24.  Construct-level predictive validity of educational attainment and intellectual aptitude tests in medical student selection: meta-regression of six UK longitudinal studies 
BMC Medicine  2013;11:243.
Measures used for medical student selection should predict future performance during training. A problem for any selection study is that predictor-outcome correlations are known only in those who have been selected, whereas selectors need to know how measures would predict in the entire pool of applicants. That problem of interpretation can be solved by calculating construct-level predictive validity, an estimate of true predictor-outcome correlation across the range of applicant abilities.
Construct-level predictive validities were calculated in six cohort studies of medical student selection and training (student entry, 1972 to 2009) for a range of predictors, including A-levels, General Certificates of Secondary Education (GCSEs)/O-levels, and aptitude tests (AH5 and UK Clinical Aptitude Test (UKCAT)). Outcomes included undergraduate basic medical science and finals assessments, as well as postgraduate measures of Membership of the Royal Colleges of Physicians of the United Kingdom (MRCP(UK)) performance and entry in the Specialist Register. Construct-level predictive validity was calculated with the method of Hunter, Schmidt and Le (2006), adapted to correct for right-censorship of examination results due to grade inflation.
Meta-regression analyzed 57 separate predictor-outcome correlations (POCs) and construct-level predictive validities (CLPVs). Mean CLPVs are substantially higher (.450) than mean POCs (.171). Mean CLPVs for first-year examinations, were high for A-levels (.809; CI: .501 to .935), and lower for GCSEs/O-levels (.332; CI: .024 to .583) and UKCAT (mean = .245; CI: .207 to .276). A-levels had higher CLPVs for all undergraduate and postgraduate assessments than did GCSEs/O-levels and intellectual aptitude tests. CLPVs of educational attainment measures decline somewhat during training, but continue to predict postgraduate performance. Intellectual aptitude tests have lower CLPVs than A-levels or GCSEs/O-levels.
Educational attainment has strong CLPVs for undergraduate and postgraduate performance, accounting for perhaps 65% of true variance in first year performance. Such CLPVs justify the use of educational attainment measure in selection, but also raise a key theoretical question concerning the remaining 35% of variance (and measurement error, range restriction and right-censorship have been taken into account). Just as in astrophysics, ‘dark matter’ and ‘dark energy’ are posited to balance various theoretical equations, so medical student selection must also have its ‘dark variance’, whose nature is not yet properly characterized, but explains a third of the variation in performance during training. Some variance probably relates to factors which are unpredictable at selection, such as illness or other life events, but some is probably also associated with factors such as personality, motivation or study skills.
PMCID: PMC3827328  PMID: 24229353
Medical student selection; Undergraduate performance; Postgraduate performance; Educational attainment; Aptitude tests; Criterion-related construct validity; Range restriction; Right censorship; Grade inflation; Markov Chain Monte Carlo algorithm
25.  Associations of Pass-Fail Outcomes with Psychological Health of First-Year Medical Students in a Malaysian Medical School 
The demanding and intense environment of medical training can create excessive pressures on medical students that eventually lead to unfavorable consequences, either at a personal or professional level. These consequences can include poor academic performance and impaired cognitive ability. This study was designed to explore associations between pass-fail outcome and psychological health parameters (i.e. stress, anxiety, and depression symptoms).
A cross-sectional study was conducted on a cohort of first-year medical students in a Malaysian medical school. The depression anxiety stress scale 21-item assessment (DASS-21) was administered to them right after the final paper of the first-year final examination. Their final examination outcomes (i.e. pass or fail) were traced by using their student identity code (ID) through the Universiti Sains Malaysia academic office.
A total of 194 (98.0%) of medical students responded to the DASS-21. An independent t-test showed that students who passed had significantly lower stress, anxiety, and depression symptoms than those who failed the first-year final examination (P <0.05). Those who experienced moderate to high stress were at 2.43 times higher risk for failing the examination than those who experienced normal to mild stress.
Medical students who failed in the final examination had higher psychological distress than those who passed the examination. Those who experienced high stress levels were more likely to fail than those who did not. Reducing the psychological distress of medical students prior to examination may help them to perform better in the examination.
PMCID: PMC3616775  PMID: 23573390
Psychological stress; Anxiety; Depression; Medical students; Student assessment; Malaysia

Results 1-25 (1184098)