In 2005 a competency based curriculum was introduced in the Dutch postgraduate medical training programs. While the manager’s role is one of the seven key competencies, there is still no formal management course in most postgraduate curricula. Based on a needs assessment we conducted, several themes were identified as important for a possible management training program. We present the results of the pilot training we performed to investigate two of these themes.
The topics “knowledge of the healthcare system” and “time management” were developed from the list of suggested management training themes. Fourteen residents participated in the training and twenty-four residents served as control. The training consisted of two sessions of four hours with a homework assignment in between. 50 True/false-questions were given as pre- and post-test to both the test and control groups to assess the level of acquired knowledge among the test group as well as the impact of the intervention. We also performed a qualitative evaluation using evaluation forms and in-depth interviews.
All fourteen residents completed the training. Six residents in the control group were lost to follow up. The pre- and post-test showed improvement among the participating residents in comparison to the residents from the control group, but this improvement was not significant. The qualitative assessment showed that all residents evaluated the training positively and experienced it as a useful addition to their training in becoming a medical specialist.
Our training was evaluated positively and considered to be valuable. This study supports the need for mandatory medical management training as part of the postgraduate medical curriculum. Our training could be an example of how to teach two important themes in the broad area of medical management education.
Medical residents; Management training; Competency; Postgraduate curriculum
Medical schools struggle with large classes, which might interfere with the effectiveness of learning within small groups due to students being unfamiliar to fellow students. The aim of this study was to assess the effects of making a large class seem small on the students' collaborative learning processes.
A randomised controlled intervention study was undertaken to make a large class seem small, without the need to reduce the number of students enrolling in the medical programme. The class was divided into subsets: two small subsets (n = 50) as the intervention groups; a control group (n = 102) was mixed with the remaining students (the non-randomised group n∼100) to create one large subset.
The undergraduate curriculum of the Maastricht Medical School, applying the Problem-Based Learning principles. In this learning context, students learn mainly in tutorial groups, composed randomly from a large class every 6–10 weeks.
The formal group learning activities were organised within the subsets. Students from the intervention groups met frequently within the formal groups, in contrast to the students from the large subset who hardly enrolled with the same students in formal activities.
Main Outcome Measures
Three outcome measures assessed students' group learning processes over time: learning within formally organised small groups, learning with other students in the informal context and perceptions of the intervention.
Formal group learning processes were perceived more positive in the intervention groups from the second study year on, with a mean increase of β = 0.48. Informal group learning activities occurred almost exclusively within the subsets as defined by the intervention from the first week involved in the medical curriculum (E-I indexes>−0.69). Interviews tapped mainly positive effects and negligible negative side effects of the intervention.
Better group learning processes can be achieved in large medical schools by making large classes seem small.
Hospital clerkships are considered crucial for acquiring competencies such as diagnostic reasoning and clinical skills. The actual learning process in the hospital remains poorly understood. This study investigates how students learn clinical skills in workplaces and factors affecting this.
Six focus group sessions with 32 students in Internal Medicine rotation (4–9 students per group; sessions 80–90 minutes). Verbatim transcripts were analysed by emerging themes and coded independently by three researchers followed by constant comparison and axial coding.
Students report to learn the systematics of the physical examination, gain agility and become able to recognise pathological signs. The learning process combines working alongside others and working independently with increasing responsibility for patient care. Helpful behaviour includes making findings explicit through patient files or during observation, feedback by abnormal findings and taking initiative. Factors affecting the process negatively include lack of supervision, uncertainty about tasks and expectations, and social context such as hierarchy of learners and perceived learning environment.
Although individual student experiences vary greatly between different hospitals, it seems that proactivity and participation are central drivers for learning. These results can improve the quality of existing programmes and help design new ways to learn physical examination skills.
To understand teaching performance of individual faculty, the climate in which residents’ learning takes place, the learning climate, may be important. There is emerging evidence that specific climates do predict specific outcomes. Until now, the effect of learning climate on the performance of the individual faculty who actually do the teaching was unknown.
This study: (i) tested the hypothesis that a positive learning climate was associated with better teaching performance of individual faculty as evaluated by residents, and (ii) explored which dimensions of learning climate were associated with faculty’s teaching performance.
Methods and Materials
We conducted two cross-sectional questionnaire surveys amongst residents from 45 residency training programs and multiple specialties in 17 hospitals in the Netherlands. Residents evaluated the teaching performance of individual faculty using the robust System for Evaluating Teaching Qualities (SETQ) and evaluated the learning climate of residency programs using the Dutch Residency Educational Climate Test (D-RECT). The validated D-RECT questionnaire consisted of 11 subscales of learning climate. Main outcome measure was faculty’s overall teaching (SETQ) score. We used multivariable adjusted linear mixed models to estimate the separate associations of overall learning climate and each of its subscales with faculty’s teaching performance.
In total 451 residents completed 3569 SETQ evaluations of 502 faculty. Residents also evaluated the learning climate of 45 residency programs in 17 hospitals in the Netherlands. Overall learning climate was positively associated with faculty’s teaching performance (regression coefficient 0.54, 95% confidence interval: 0.37 to 0.71; P<0.001). Three out of 11 learning climate subscales were substantially associated with better teaching performance: ‘coaching and assessment’, ‘work is adapted to residents’ competence’, and ‘formal education’.
Individual faculty’s teaching performance evaluations are positively affected by better learning climate of residency programs.
The number of Master of Public Health (MPH) programmes in low- and middle-income countries (LMICs) is increasing, but questions have been raised regarding the relevance of their outcomes and impacts on context. Although processes for validating public health competencies have taken place in recent years in many high-income countries, validation in LMICs is needed. Furthermore, impact variables of MPH programmes in the workplace and in society have not been developed.
A set of public health competencies and impact variables in the workplace and in society was designed using the competencies and learning objectives of six participating institutions offering MPH programmes in or for LMICs, and the set of competencies of the Council on Linkages Between Academia and Public Health Practice as a reference. The resulting competencies and impact variables differ from those of the Council on Linkages in scope and emphasis on social determinants of health, context specificity and intersectoral competencies. A modified Delphi method was used in this study to validate the public health competencies and impact variables; experts and MPH alumni from China, Vietnam, South Africa, Sudan, Mexico and the Netherlands reviewed them and made recommendations.
The competencies and variables were validated across two Delphi rounds, first with public health experts (N = 31) from the six countries, then with MPH alumni (N = 30). After the first expert round, competencies and impact variables were refined based on the quantitative results and qualitative comments. Both rounds showed high consensus, more so for the competencies than the impact variables. The response rate was 100%.
This is the first time that public health competencies have been validated in LMICs across continents. It is also the first time that impact variables of MPH programmes have been proposed and validated in LMICs across continents. The high degree of consensus between experts and alumni suggests that these public health competencies and impact variables can be used to design and evaluate MPH programmes, as well as for individual and team assessment and continuous professional development in LMICs.
Public health competencies; Impact; Low- and middle-income countries; Master of Public Health
Recently competency approach in Health Professionals’ Education (HPE) has become quite popular and for an effective competency based HPE, it is important to design the curriculum around the health care needs of the population to be served and on the expected roles of the health care providers. Unfortunately, in community settings roles of health providers tend to be described less clearly, particularly at the Primary Health Care (PHC) level where a multidisciplinary and appropriately prepared health team is generally lacking. Moreover, to tailor the education on community needs there is no substantial evidence on what specific requirements the providers must be prepared for.
This study has explored specific tasks of physicians and nurses employed to work in primary or secondary health care units in a context where there is a structural scarcity of community health care providers. In-depth Interviews of 11 physicians and 06 nurses working in community settings of Pakistan were conducted along with review of their job descriptions.
At all levels of health settings, physicians’ were mostly engaged with diagnosing and prescribing medical illness of patients coming to health center and nurses depending on their employer were either providing preventive health care activities, assisting physicians or occupied in day to day management of health center. Geographical location or level of health facility did not have major effect on the roles being expected or performed, however the factors that determined the roles performed by health providers were employer expectations, preparation of health providers for providing community based care, role clarity and availability of resources including health team at health facilities.
Exploration of specific tasks of physicians and nurses working in community settings provide a useful framework to map competencies, and can help educators revisit the curricula and instructional designs accordingly. Furthermore, in community settings there are many synergies between the roles of physicians and nurses which could be simulated as learning activities; at the same time these two groups of health providers offer distinct sets of services, which must be harnessed to build effective, non-hierarchal, collaborative health teams.
Competencies; Community settings; Undergraduate curriculum; Health care providers; Tasks & functions
Community-Based Education (CBE) is an instructional approach designed and carried out in a community context and environment in which not only students, but also faculty and Health Professionals’ Education (HPE) institutions must be actively engaged throughout the educational experience. Despite the growing evidence of CBE being an effective approach for contemporary HPE, doubts about its successful implementation still exist. This study has explored HPE structure, policies and curriculum from the point of view of faculty members to gain understanding about the prevailing practices and to propose recommendations that nurtures and promotes CBE.
A purposive sample was drawn from three major cities of Pakistan- Karachi, Rawalpindi and Islamabad. Out of twelve HPE institutions present in these cities we selected six, which provided a sound representation of medical and nursing colleges around the country. At each institution we had two Focus Group Discussions; in addition we interviewed registrars of medical and nursing councils and two CBE experts.
The factors effecting implementation of CBE as perceived by study participants are categorized as: preparation of faculty members; institutional commitment and enthusiasm; curricular priorities and external milieu. Within each theme, participants recurrently described structural and curricular deficiencies, and lack of commitment and appreciation for community based teaching, service and research permeating at all levels: regulatory bodies, institutional heads and faculty members.
The factors highlighted by our study and many others suggest that CBE could not perpetuate effectively within HPE. To enhance the effectiveness of CBE approach in a way that mutually benefits local communities as well as HPE institutions and health professionals, it is important that reforms in HPE must be strategized in a holistic fashion i.e. restructuring and aligning its polices, curriculum and research priorities.
Previous work has shown that a programme that draws on a blend of theories makes a positive difference to outcomes for students who fail and repeat their first semester at medical school. Exploration of student and teacher perspectives revealed that remediation of struggling medical students can be achieved through a cognitive apprenticeship within a small community of inquiry. This community needs expert teachers capable of performing a unique combination of roles (facilitator, nurturing mentor, disciplinarian, diagnostician and role model), with high levels of teaching presence and practical wisdom. Yet, despite participants’ convergent opinions on the elements of effective remediation, significant differences were found between outcomes of students working with experienced and inexperienced teachers. The current study explores the actual practice of teachers on this remediation course, aiming to exemplify elements of our theory of remediation and explore differences between teachers.
Since it is in the classroom context that the interactions that constitute the complex process of remediation emerge, this practice-based research has focused on direct observation of classroom teaching. Nineteen hours of small group sessions were recorded and transcribed. Drawing on ethnography and sociocultural discourse analysis, selected samples of talk-in-context demonstrate how the various elements of remediation play out in practice, highlighting aspects that are most effective, and identifying differences between experienced and novice teachers.
Long-term student outcomes are strongly correlated to teacher experience (r, 0.81). Compared to inexperienced teachers, experienced teachers provide more challenging, disruptive facilitation, and take a dialogic stance that encourages more collaborative group dynamics. They are more expert at diagnosing cognitive errors, provide frequent metacognitive time-outs and make explicit links across the curriculum.
Remediation is effective in small groups where dialogue is used for collaborative knowledge construction and social regulation. This requires facilitation by experienced teachers who attend to details of both content and process, and use timely interventions to foster curiosity and the will to learn. These teachers should actively challenge students’ language use, logical inconsistencies and uncertainties, problematize their assumptions, and provide a metacognitive regulatory voice that can generate attitudinal shifts and nurture the development of independent critical thinkers.
At-risk students; Remediation; Small group teaching; Classroom discourse; Facilitation skills; Teaching experience; Pedagogical context knowledge
In fledgling areas of research, evidence supporting causal assumptions is often scarce due to the small number of empirical studies conducted. In many studies it remains unclear what impact explicit and implicit causal assumptions have on the research findings; only the primary assumptions of the researchers are often presented. This is particularly true for research on the effect of faculty’s teaching performance on their role modeling. Therefore, there is a need for robust frameworks and methods for transparent formal presentation of the underlying causal assumptions used in assessing the causal effects of teaching performance on role modeling. This study explores the effects of different (plausible) causal assumptions on research outcomes.
This study revisits a previously published study about the influence of faculty’s teaching performance on their role modeling (as teacher-supervisor, physician and person). We drew eight directed acyclic graphs (DAGs) to visually represent different plausible causal relationships between the variables under study. These DAGs were subsequently translated into corresponding statistical models, and regression analyses were performed to estimate the associations between teaching performance and role modeling.
The different causal models were compatible with major differences in the magnitude of the relationship between faculty’s teaching performance and their role modeling. Odds ratios for the associations between teaching performance and the three role model types ranged from 31.1 to 73.6 for the teacher-supervisor role, from 3.7 to 15.5 for the physician role, and from 2.8 to 13.8 for the person role.
Different sets of assumptions about causal relationships in role modeling research can be visually depicted using DAGs, which are then used to guide both statistical analysis and interpretation of results. Since study conclusions can be sensitive to different causal assumptions, results should be interpreted in the light of causal assumptions made in each study.
Currently, one of the main interventions that are widely expected to contribute to teachers’ professional development is confronting teachers with feedback from resident evaluations of their teaching performance. Receiving feedback, however, is a double edged sword. Teachers see themselves confronted with information about themselves and are, at the same time, expected to be role models in the way they respond to feedback. Knowledge about the teachers’ responses could be not only of benefit for their professional development, but also for supporting their role modeling. Therefore, research about professional development should include the way teachers respond to feedback.
We designed a qualitative study with semi-structured individual conversations about feedback reports, gained from resident evaluations. Two researchers carried out a systematic analysis using qualitative research software. The analysis focused on what happened in the conversations and structured the data in three main themes: conversation process, acceptance and coping strategies.
The result section describes the conversation patterns and atmosphere. Teachers accepted their results calmly, stating that, although they recognised some points of interest, they could not meet with every standard. Most used coping strategies were explaining the results from their personal beliefs about good teaching and attributing poor results to external factors and good results to themselves. However, some teachers admitted that they had poor results because of the fact that they were not “sharp enough” in their resident group, implying that they did not do their best.
Our study not only confirms that the effects of feedback depend first and foremost on the recipient but also enlightens the meaning and role of acceptance and being a role model. We think that the results justify the conclusion that teachers who are responsible for the day release programmes in the three departments tend to respond to the evaluation results just like human beings do and, at the time of the conversation, are initially not aware of the fact that they are role models in the way they respond to feedback.
Teachers; Professional development; Feedback; Role modeling
Despite increasing attention to providing preclinical medical students with early patient experiences, little is known about associated outcomes for students. The authors compared three early patient experiences at a large American medical school where all preclinical students complete preceptorships and weekly bedside clinical-skills training and about half complete clinical, community-based summer immersion experiences. The authors asked, what are the relative outcomes and important educational components for students?
Medical students completed surveys at end of second year 2009–2011. In 2009, students compared/contrasted two of three approaches; responses framed later survey questions. In 2010 and 2011, students rated all three experiences in relevant areas (e.g., developing comfort in clinical setting). Investigators performed qualitative and quantitative analyses.
Students rated bedside training more highly for developing comfort with clinical settings, one-on-one clinical-skills training, feedback, active clinical experience, quality of clinical training, and learning to be part of a team. They rated community clinical immersion and preceptorships more highly for understanding the life/practice of a physician and career/specialty decisions.
Preclinical students received different benefits from the different experiences. Medical schools should define objectives of early clinical experiences and offer options accordingly. A combination of experiences may help students achieve clinical and team comfort, clinical skills, an understanding of physicians’ lives/practices, and broad exposure for career decisions.
Transnational or cross-border higher education has rapidly expanded since the 1980s. Together with that expansion issues on quality assurance came to the forefront. This article aims to identify key issues regarding quality assurance of transnational higher education and discusses the quality assurance of the tropEd Network for International Health in Higher Education in relation to these key issues.
Literature review and review of documents.
From the literature the following key issues regarding transnational quality assurance were identified and explored: comparability of quality assurance frameworks, true collaboration versus erosion of national education sovereignty, accreditation agencies and transparency. The tropEd network developed a transnational quality assurance framework for the network. The network accredits modules through a rigorous process which has been accepted by major stakeholders. This process was a participatory learning process and at the same time the process worked positive for the relations between the institutions.
The development of the quality assurance framework and the process provides a potential example for others.
Quality assurance; Higher education; Cross-border; Transnational; Networks
Previous research has shown that Dutch medical residents feel inadequate in certain management areas: 85% had a need for management training and reported preferences on the format of such training. Our objective was to explore if the perceived deficiencies and needs among Dutch residents were similar to those of their peers in other countries, and if a longer duration of the incorporation of the CanMEDS competency framework into curricula as well as management training had an influence on these perceptions.
Medical residents from Denmark, Canada and Australia were approached for participation. The questionnaire used to survey the perceptions of Dutch residents was slightly modified, translated into English and sent by email to all international participants.
Response rates were; Denmark 719/2105 (34%), Canada 177/500 (35%) and Australia 194/1213 (16%) respectively. The Danish as well as the Canadian residents rated their negotiating skills poorly. In Australia the residents found their knowledge on how their specialist department was organized to be insufficient, while residents in the Netherlands rated their knowledge on how the healthcare system was organized as inadequate. In all of the countries, more than 70% of the residents reported a perceived need for management training.
A majority of the residents in all countries felt the need for specific training in developing their management competencies. The adoption of the CanMEDS framework alone seems to be insufficient in meeting residents’ perceived educational needs in this area.
Medical residents; CanMEDS; Managers role; Assessment; International comparison
Many medical schools have embraced small group learning methods in their undergraduate curricula. Given increasing financial constraints on universities, active learning groups like seminars (with 25 students a group) are gaining popularity. To enhance the understanding of seminar learning and to determine how seminar learning can be optimised it is important to investigate stakeholders’ views. In this study, we qualitatively explored the views of teachers on aspects affecting seminar learning.
Twenty-four teachers with experience in facilitating seminars in a three-year bachelor curriculum participated in semi-structured focus group interviews. Three focus groups met twice with an interval of two weeks led by one moderator. Sessions were audio taped, transcribed verbatim and independently coded by two researchers using thematic analysis. An iterative process of data reduction resulted in emerging aspects that influence seminar learning.
Teachers identified seven key aspects affecting seminar learning: the seminar teacher, students, preparation, group functioning, seminar goals and content, course coherence and schedule and facilities. Important components of these aspects were: the teachers’ role in developing seminars (‘ownership’), the amount and quality of preparation materials, a non-threatening learning climate, continuity of group composition, suitability of subjects for seminar teaching, the number and quality of seminar questions, and alignment of different course activities.
The results of this study contribute to the unravelling of the ‘the black box’ of seminar learning. Suggestions for ways to optimise active learning in seminars are made regarding curriculum development, seminar content, quality assurance and faculty development.
Seminar learning; Undergraduate (veterinary) medical education; Focus groups; Faculty development
The ‘human resources for health’ crisis has highlighted the need for more health (care) professionals and led to an increased interest in health professional education, including master’s degree programmes. The number of these programmes in low- and middle-income countries (LMIC) is increasing, but questions have been raised regarding their relevance, outcome and impact. We conducted a systematic review to evaluate the outcomes and impact of health-related master’s degree programmes.
We searched the databases Scopus, Pubmed, Embase, CINAHL, ERIC, Psychinfo and Cochrane (1999 - November 2011) and selected websites. All papers describing outcomes and impact of health-related Master programmes were included. Three reviewers, two for each article, extracted data independently. The articles were categorised by type of programme, country, defined outcomes and impact, study methods used and level of evidence, and classified according to outcomes: competencies used in practice, graduates’ career progression and impact on graduates’ workplaces and sector/society.
Of the 33 articles included in the review, most originated from the US and the UK, and only one from a low-income country. The programmes studied were in public health (8), nursing (8), physiotherapy (5), family practice (4) and other topics (8). Outcomes were defined in less than one third of the articles, and impact was not defined at all. Outcomes and impact were measured by self-reported alumni surveys and qualitative methods. Most articles reported that competencies learned during the programme were applied in the workplace and alumni reported career progression or specific job changes. Some articles reported difficulties in using newly gained competencies in the workplace. There was limited evidence of impact on the workplace. Only two articles reported impact on the sector. Most studies described learning approaches, but very few described a mechanism to ensure outcome and impact of the programme.
Evidence suggests that graduates apply newly learned competencies in the field and that they progress in their career. There is a paucity of well-designed studies assessing the outcomes and impact of health-related master’s degree programmes in low- and middle-income countries. Studies of such programmes should consider the context and define outcomes and impact.
Master’s degree programmes; Evaluation; Outcomes; Impact; Systematic review; Public health
The advantages of using simulators in skills training are generally recognized, but simulators are often too expensive for medical schools in developing countries. Cheaper locally-made models (or part-task trainers) could be the answer, especially when teachers are involved in design and production (teacher-made models, TM).
We evaluated the effectiveness of a TM in training and assessing intravenous injection skills in comparison to an available commercial model (CM) in a randomized, blind, pretest-posttest study with 144 undergraduate nursing students. All students were assessed on both the TM and the CM in the pre-test and post-test. After the post-test the students were also assessed while performing the skill on real patients.
Differences in the mean scores pre- and post-test were marked in all groups. Training with TM or CM improved student scores substantially but there was no significant difference in mean scores whether students had practiced on TM or CM. Students who practiced on TM performed better on communication with the patient than did students who practiced on CM. Decreasing the ratio of students per TM model helped to increase practice opportunities but did not improve student’s mean scores. The result of the assessment on both the TM and the CM had a low correlation with the results of the assessment on real persons.
The TM appears to be an effective alternative to CM for training students on basic IV skills, as students showed similar increases in performance scores after training on models that cost considerably less than commercially available models. These models could be produced using locally available materials in most countries, including those with limited resources to invest in medical education and skills laboratories.
Clinical skills laboratory; Teacher made models; Commercial models; Vietnam
Reflection on professional experience is increasingly accepted as a critical attribute for health care practice; however, evidence that it has a positive impact on performance remains scarce. This study investigated whether, after allowing for the effects of knowledge and consultation skills, reflection had an independent effect on students’ ability to solve problem cases.
Data was collected from 362 undergraduate medical students at Ghent University solving video cases and reflected on the experience of doing so. For knowledge and consultation skills results on a progress test and a course teaching consultation skills were used respectively. Stepwise multiple linear regression analysis was used to test the relationship between the quality of case-solving (dependent variable) and reflection skills, knowledge, and consultation skills (dependent variables).
Only students with data on all variables available (n = 270) were included for analysis. The model was significant (Anova F(3,269) = 11.00, p < 0.001, adjusted R square 0.10) with all variables significantly contributing.
Medical students’ reflection had a small but significant effect on case-solving, which supports reflection as an attribute for performance. These findings suggest that it would be worthwhile testing the effect of reflection skills training on clinical competence.
Students may encounter difficulties when they have to apply clinical skills trained in their pre-clinical studies in clerkships. Early clinical exposure in the pre-clinical phase has been recommended to reduce these transition problems. The aim of this study is to explore differences in students' experiences during the first clerkships between students exclusively trained in a skills laboratory and peers for whom part of their skills training was substituted by early clinical experiences (ECE).
Thirty pre-clinical students trained clinical skills exclusively in a skills laboratory; 30 peers received part of their skills training in PHC centers. Within half a year after commencing their clerkships all 60 students shared their experiences in focus group discussions (FGDs). Verbatim transcripts of FGDs were analyzed using Atlas-Ti software.
Clerkship students who had participated in ECE in PHC centers felt better prepared to perform their clinical skills during the first clerkships than peers who had only practiced in a skills laboratory. ECE in PHC centers impacted positively in particular on students’ confidence, clinical reasoning, and interpersonal communication.
In the Indonesian setting ECE in PHC centers reduce difficulties commonly encountered by medical students in the first clerkships.
Clinical skills training; Early clinical experiences; Clerkships
Reflection is a meta-cognitive process, characterized by: 1. Awareness of self and the situation; 2. Critical analysis and understanding of both self and the situation; 3. Development of new perspectives to inform future actions. Assessors can only access reflections indirectly through learners’ verbal and/or written expressions. Being privy to the situation that triggered reflection could place reflective materials into context. Video-cases make that possible and, coupled with a scoring rubric, offer a reliable way of assessing reflection.
Fourth and fifth year undergraduate medical students were shown two interactive video-cases and asked to reflect on this experience, guided by six standard questions. The quality of students’ reflections were scored using a specially developed Student Assessment of Reflection Scoring rubric (StARS®). Reflection scores were analyzed concerning interrater reliability and ability to discriminate between students. Further, the intra-rater reliability and case specificity were estimated by means of a generalizability study with rating and case scenario as facets.
Reflection scores of 270 students ranged widely and interrater reliability was acceptable (Krippendorff’s alpha = 0.88). The generalizability study suggested 3 or 4 cases were needed to obtain reliable ratings from 4th year students and ≥ 6 cases from 5th year students.
Use of StARS® to assess student reflections triggered by standardized video-cases had acceptable discriminative ability and reliability. We offer this practical method for assessing reflection summatively, and providing formative feedback in training situations.
Incident reporting systems (IRS) are used to identify medical errors in order to learn from mistakes and improve patient safety in hospitals. However, IRS contain only a small fraction of occurring incidents. A more comprehensive overview of medical error in hospitals may be obtained by combining information from multiple sources. The WHO has developed the International Classification for Patient Safety (ICPS) in order to enable comparison of incident reports from different sources and institutions.
The aim of this paper was to provide a more comprehensive overview of medical error in hospitals using a combination of different information sources. Incident reports collected from IRS, patient complaints and retrospective chart review in an academic acute care hospital were classified using the ICPS. The main outcome measures were distribution of incidents over the thirteen categories of the ICPS classifier “Incident type”, described as odds ratios (OR) and proportional similarity indices (PSI).
A total of 1012 incidents resulted in 1282 classified items. Large differences between data from IRS and patient complaints (PSI = 0.32) and from IRS and retrospective chart review (PSI = 0.31) were mainly attributable to behaviour (OR = 6.08), clinical administration (OR = 5.14), clinical process (OR = 6.73) and resources (OR = 2.06).
IRS do not capture all incidents in hospitals and should be combined with complementary information about diagnostic error and delayed treatment from patient complaints and retrospective chart review. Since incidents that are not recorded in IRS do not lead to remedial and preventive action in response to IRS reports, healthcare centres that have access to different incident detection methods should harness information from all sources to improve patient safety.
The drive to quality-manage medical education has created a need for valid measurement instruments. Validity evidence includes the theoretical and contextual origin of items, choice of response processes, internal structure, and interrelationship of a measure’s variables. This research set out to explore the validity and potential utility of an 11-item measurement instrument, whose theoretical and empirical origins were in an Experience Based Learning model of how medical students learn in communities of practice (COPs), and whose contextual origins were in a community-oriented, horizontally integrated, undergraduate medical programme. The objectives were to examine the psychometric properties of the scale in both hospital and community COPs and provide validity evidence to support using it to measure the quality of placements. The instrument was administered twice to students learning in both hospital and community placements and analysed using exploratory factor analysis and a generalizability analysis. 754 of a possible 902 questionnaires were returned (84% response rate), representing 168 placements. Eight items loaded onto two factors, which accounted for 78% of variance in the hospital data and 82% of variance in the community data. One factor was the placement learning environment, whose five constituent items were how learners were received at the start of the placement, people’s supportiveness, and the quality of organisation, leadership, and facilities. The other factor represented the quality of training—instruction in skills, observing students performing skills, and providing students with feedback. Alpha coefficients ranged between 0.89 and 0.93 and there were no redundant or ambiguous items. Generalisability analysis showed that between 7 and 11 raters would be needed to achieve acceptable reliability. There is validity evidence to support using the simple 8-item, mixed methods Manchester Clinical Placement Index to measure key conditions for undergraduate medical students’ experience based learning: the quality of the learning environment and the training provided within it. Its conceptual orientation is towards Communities of Practice, which is a dominant contemporary theory in undergraduate medical education.
Workplace learning; Learning environment; Evaluation; Communities of practice; Experience based learning
Reflection on experience is an increasingly critical part of professional development and lifelong learning. There is, however, continuing uncertainty about how best to put principle into practice, particularly as regards assessment. This article explores those uncertainties in order to find practical ways of assessing reflection.
We critically review four problems: 1. Inconsistent definitions of reflection; 2. Lack of standards to determine (in)adequate reflection; 3. Factors that complicate assessment; 4. Internal and external contextual factors affecting the assessment of reflection.
To address the problem of inconsistency, we identified processes that were common to a number of widely quoted theories and synthesised a model, which yielded six indicators that could be used in assessment instruments. We arrived at the conclusion that, until further progress has been made in defining standards, assessment must depend on developing and communicating local consensus between stakeholders (students, practitioners, teachers, supervisors, curriculum developers) about what is expected in exercises and formal tests. Major factors that complicate assessment are the subjective nature of reflection's content and the dependency on descriptions by persons being assessed about their reflection process, without any objective means of verification. To counter these validity threats, we suggest that assessment should focus on generic process skills rather than the subjective content of reflection and where possible to consider objective information about the triggering situation to verify described reflections. Finally, internal and external contextual factors such as motivation, instruction, character of assessment (formative or summative) and the ability of individual learning environments to stimulate reflection should be considered.
The role of deliberate practice in medical students' development from novice to expert was examined for preclinical skill training.
Students in years 1-3 completed 34 Likert type items, adapted from a questionnaire about the use of deliberate practice in cognitive learning. Exploratory factor analysis and reliability analysis were used to validate the questionnaire. Analysis of variance examined differences between years and regression analysis the relationship between deliberate practice and skill test results.
875 students participated (90%). Factor analysis yielded four factors: planning, concentration/dedication, repetition/revision, study style/self reflection. Student scores on 'Planning' increased over time, score on sub-scale 'repetition/revision' decreased. Student results on the clinical skill test correlated positively with scores on subscales 'planning' and 'concentration/dedication' in years 1 and 3, and with scores on subscale 'repetition/revision' in year 1.
The positive effects on test results suggest that the role of deliberate practice in medical education merits further study. The cross-sectional design is a limitation, the large representative sample a strength of the study. The vanishing effect of repetition/revision may be attributable to inadequate feedback. Deliberate practice advocates sustained practice to address weaknesses, identified by (self-)assessment and stimulated by feedback. Further studies should use a longitudinal prospective design and extend the scope to expertise development during residency and beyond.
When innovations are introduced in medical education, teachers often have to adapt to a new concept of what being a good teacher includes. These new concepts do not necessarily match medical teachers' own, often strong beliefs about what it means to be a good teacher.
Recently, a new competency-based description of the good teacher was developed and introduced in all the Departments of Postgraduate Medical Education for Family Physicians in the Netherlands. We compared the views reflected in the new description with the views of teachers who were required to adopt the new framework.
Qualitative study. We interviewed teachers in two Departments of Postgraduate Medical Education for Family Physicians in the Netherlands. The transcripts of the interviews were analysed independently by two researchers, who coded and categorised relevant fragments until consensus was reached on six themes. We investigated to what extent these themes matched the new description.
Comparing the teachers' views with the concepts described in the new competency-based framework is like looking into two mirrors that reflect clearly dissimilar images. At least two of the themes we found are important in relation to the implementation of new educational methods: the teachers' identification and organisational culture. The latter plays an important role in the development of teachers' ideas about good teaching.
The main finding of this study is the key role played by the teachers' feelings regarding their professional identity and by the local teaching culture in shaping teachers' views and expectations regarding their work. This suggests that in implementing a new teaching framework and in faculty development programmes, careful attention should be paid to teachers' existing identification model and the culture that fostered it.
Lack of published studies on students’ practice behaviour of physical examination skills outside timetabled training sessions inspired this study into what activities medical students undertake to improve their skills and factors influencing this. Six focus groups of a total of 52 students from Years 1–3 using a pre-established interview guide. Interviews were recorded, transcribed and analyzed using qualitative methods. The interview guide was based on questionnaire results; overall response rate for Years 1–3 was 90% (n = 875). Students report a variety of activities to improve their physical examination skills. On average, students devote 20% of self-study time to skill training with Year 1 students practising significantly more than Year 3 students. Practice patterns shift from just-in-time learning to a longitudinal selfdirected approach. Factors influencing this change are assessment methods and simulated/real patients. Learning resources used include textbooks, examination guidelines, scientific articles, the Internet, videos/DVDs and scoring forms from previous OSCEs. Practising skills on fellow students happens at university rooms or at home. Also family and friends were mentioned to help. Simulated/real patients stimulated students to practise of physical examination skills, initially causing confusion and anxiety about skill performance but leading to increased feelings of competence. Difficult or enjoyable skills stimulate students to practise. The strategies students adopt to master physical examination skills outside timetabled training sessions are self-directed. OSCE assessment does have influence, but learning takes place also when there is no upcoming assessment. Simulated and real patients provide strong incentives to work on skills. Early patient contacts make students feel more prepared for clinical practice.
Undergraduate; Medical students; Practical; Physical examination skills; Clinical skills