Aims: In most German medical faculties, credits in general practice can be earned via exams using multiple-choice questions (MCQ). Measures such as peer-reviews may help assure the quality of these exams. In order to use time and personnel intensive peer reviews effectively and efficiently, the procedures used are key. Therefore, we wanted to find out whether there are differences between group and individual reviews regarding defined parameters.
Methods: We conducted a controlled cross-over study with three GP reviewers from four different German universities. Each reviewed 80 MCQs, 40 individually and 40 within a group, including external assessments by a panel of experts. Furthermore all reviewers were asked to evaluate the review process and the time spent carrying out these reviews.
Outcomes: We found no significant differences between the reliability and the validity of individual reviews versus group reviews. On average slightly more time was spent on group reviews compared with the individual reviews. The subjective assessments of the study participants regarding their satisfaction with the process and the efficiency and effectiveness of the reviews suggest a preference for group reviews.
Conclusions: Based on this study, there are no definite recommendations for or against either approach. When choosing between the two, the specific work structures and organisation at the local faculty should be taken into account.
Medical Educatio; assessment; Multiple-Choice-Questions; Review
The purpose of this study is to describe an approach for evaluating assessments used in the first 2 years of medical school and report the results of applying this method to current first and second year medical student examinations.
Three faculty members coded all exam questions administered during the first 2 years of medical school. The reviewers discussed and compared the coded exam questions. During the bi-monthly meetings, all differences in coding were resolved with consensus as the final criterion. We applied Moore's framework to assist the review process and to align it with National Board of Medical Examiners (NBME) standards.
The first and second year medical school examinations had 0% of competence level questions. The majority, more than 50% of test questions, were at the NBME recall level.
It is essential that multiple-choice questions (MCQs) test the attitudes, skills, knowledge, and competency in medical school. Based on our findings, it is evident that our exams need to be improved to better prepare our medical students for successful completion of NBME step exams.
undergraduate medical education; assessment; course exams; NBME
International reports recommend the improvement in the amount and quality of training for mental health workers in low and middle income countries. The Scotland-Malawi Mental Health Education Project (SMMHEP) has been established to support the teaching of psychiatry to medical students in the University of Malawi. While anecdotally supportive medical educational initiatives appear of value, little quantitative evidence exists to demonstrate whether such initiatives can deliver comparable educational standards. This study aimed to assess the effectiveness of an undergraduate psychiatry course given by UK psychiatrists in Malawi by studying University of Malawi and Edinburgh University medical students' performance on an MCQ examination paper.
An undergraduate psychiatry course followed by an MCQ exam was delivered by the SMMHEP to 57 Malawi medical students. This same MCQ exam was given to 71 Edinburgh University medical students who subsequently sat their own Edinburgh University examination.
There were no significant differences between Edinburgh students' performance on the Malawi exam and their own Edinburgh University exam. (p = 0.65). This would suggest that the Malawi exam is a comparable standard to the Edinburgh exam. Malawi students marks ranged from 52.4%–84.6%. Importantly 84.4% of Malawi students scored above 60% on their exam which would equate to a hypothetical pass by UK university standards.
The support of an undergraduate course in an African setting by high income country specialists can attain a high percentage pass rate by UK standards. Although didactic teaching has been surpassed by more novel educational methods, in resource poor countries it remains an effective and cost effective method of gaining an important educational standard.
In 2004 the adoption of a modular curriculum at the medical faculty in Muenster led to the introduction of centralized examinations based on multiple-choice questions (MCQs). We report on how organizational challenges of realizing faculty-wide personalized tests were addressed by implementation of a specialized software module to automatically generate test sheets from individual test registrations and MCQ contents.
Key steps of the presented method for preparing personalized test sheets are (1) the compilation of relevant item contents and graphical media from a relational database with database queries, (2) the creation of Extensible Markup Language (XML) intermediates, and (3) the transformation into paginated documents.
The software module by use of an open source print formatter consistently produced high-quality test sheets, while the blending of vectorized textual contents and pixel graphics resulted in efficient output file sizes. Concomitantly the module permitted an individual randomization of item sequences to prevent illicit collusion.
The automatic generation of personalized MCQ test sheets is feasible using freely available open source software libraries, and can be efficiently deployed on a faculty-wide scale.
The purpose of this report was to describe the development, implementation, and outcomes from 3 complementary programs to facilitate the development of faculty members. The Faculty Development Committee (FDC) at the University of Tennessee developed 3 new complementary programs: the Individual Faculty Development Program to encourage faculty members to assess and identify their own specific developmental needs; the Seed Research Grant Program to fund scholarly activities by faculty; and the Technology Support Program to foster financial support of technology upgrades crucial for meeting the research, education, and service needs of faculty members. Eighteen faculty members participated in the Individual Faculty Development Program during the first 2 academic years and all provided positive feedback about their experiences. The Seed Research Grant Program funded 6 projects during its inaugural year. Limited outcome data from these 2 programs are extremely favorable relative to grant submissions and publications, and enhanced educational offerings and evaluations. The Technology Support Fund was initiated in the 2005-2006 academic year. The 3 faculty development programs initiated are offered as examples whereby faculty members are given a high degree of self-determination relative to identifying programs that will effectively contribute to their growth as academicians. Other colleges of pharmacy are encouraged to consider similar initiatives to foster individual faculty development at this critical period of growth within academic pharmacy.
faculty development; research; technology
Background: Multiple choice questions (MCQs) are often used in exams of medical education and need careful quality management for example by the application of review committees. This study investigates whether groups communicating virtually by email are similar to face-to-face groups concerning their review process performance and whether a facilitator has positive effects.
Methods: 16 small groups of students were examined, which had to evaluate and correct MCQs under four different conditions. In the second part of the investigation the changed questions were given to a new random sample for the judgement of the item quality.
Results: There was no significant influence of the variables “form of review committee” and “facilitation”. However, face-to-face and virtual groups clearly differed in the required treatment times. The test condition “face to face without facilitation” was generally valued most positively concerning taking over responsibility, approach to work, sense of well-being, motivation and concentration on the task.
Discussion: Face-to-face and virtual groups are equally effective in the review of MCQs but differ concerning their efficiency. The application of electronic review seems to be possible but is hardly recommendable because of the long process time and technical problems.
multiple choice questions; MCQ; face-to-fac; virtual; facilitation; review-committee
The assessment of the anesthesia course in our university comprises Objective Structured Clinical Examinations (OSCEs), in conjunction with portfolio and multiple-choice questions (MCQ). The objective of this study was to evaluate the outcome of different forms of anesthesia course assessment among 5th year medical students in our university, as well as study the influence of gender on student performance in anesthesia.
We examined the performance of 154, 5th year medical students through OSCE, portfolios, and MCQ.
The score ranges in the portfolio, OSCE, and MCQs were 16-24, 4.2-28.9, and 15.5-44.5, respectively. There was highly significant difference in scores in relation to gender in all assessments other than the written one (P=0.000 for Portfolio, OSCE, and Total exam, whereas P=0.164 for written exam). In the generated linear regression model, OSCE alone could predict 86.4% of the total mark if used alone. In addition, if the score of the written examination is added, OSCE will drop to 57.2% and the written exam will be 56.8% of the total mark.
This study demonstrates that different clinical methods used to assess medical students during their anesthesia course were consistent and integrated. The performance of female was superior to male in OSCE and portfolio. This information is the basis for improving educational and assessment standards in anesthesiology and for introducing a platform for developing modern learning media in countries with dearth of anesthesia personnel.
Anesthesiology; assessment; gender; objective structured clinical examination; portfolios and multiple-choice questions
To develop and psychometrically assess a multiple choice question (MCQ) instrument to test knowledge of depression and its treatments in patients suffering from depression.
A total of 63 depressed patients and twelve psychiatric experts participated. Based on empirical evidence from an extensive review, theoretical knowledge and in consultations with experts, 27-item MCQ knowledge of depression and its treatment test was constructed. Data collected from the psychiatry experts were used to assess evidence of content validity for the instrument.
Cronbach's alpha of the instrument was 0.68, and there was an overall 87.8% agreement (items are highly relevant) between experts about the relevance of the MCQs to test patient knowledge on depression and its treatments. There was an overall satisfactory patients' performance on the MCQs with 78.7% correct answers. Results of an item analysis indicated that most items had adequate difficulties and discriminations.
There was adequate reliability and evidence for content and convergent validity for the instrument. Future research should employ a lager and more heterogeneous sample from both psychiatrist and community samples, than did the present study. Meanwhile, the present study has resulted in psychometrically tested instruments for measuring knowledge of depression and its treatment of depressed patients.
The Advanced Trauma Operative Management (ATOM) course was first introduced into Canada in 2003 at the University of Toronto, with senior general surgery residents being the primary focus. We present an assessment of the course in this Canadian general surgery residency program.
We compared trainees' pre-and postcourse self-efficacy scores and multiple choice question (MCQ) examination results, using paired t tests and resident (n = 24) and faculty (n = 7) course ratings made according to a 10-item, 5-point Likert scale. Faculty were previously trained as ATOM instructors.
Mean pre-and postcourse self-efficacy scores were 68.9 (standard deviation [SD] 24.0) and 101.4 (SD 14.8), respectively (p < 0.001). Mean pre-and post-MCQ scores were 16.4 (SD 3.2) and 18.8 (SD 2.7), respectively (p = 0.006). On the Likert scale (1 = strongly disagree, 5 = strongly agree), all faculty and residents rated the following items as 4–5: objectives were met; knowledge, skills, clinical training, judgment and confidence improved; the live animal is a useful representation of clinical trauma; and the course should be continued but would be more appropriate for the fourth rather than the fifth year of residency. Residents rated as 1–2 the item that the human cadaver would be preferable for learning the surgical skills. Of 24 residents, 20 rated as 3 or less the item stating that the course prepares them for trauma management more adequately than their regular training program.
Self-efficacy, trauma knowledge and skills improved significantly with ATOM training. Preference was expressed for the live animal versus cadaver model, for ATOM training in the fourth rather than fifth year of residency and for the view that it complements general surgery trauma training. The data suggest that including ATOM training in Canadian general surgical residency should be considered.
After years of ‘ad hoc’ growth in the use of computers in the curriculum, the University of Minnesota Medical School in cooperation with the Bio-Medical Library and Health Sciences Computing Services developed and began implementation of a plan for integration of medical informatics into all phases of medical education.
Objectives were developed which focus on teaching skills related to:
1) accessing, retrieving, evaluating and managing medical information;
2) appropriate utilization of computer-assisted instruction lessons;
3) electronic communication with fellow students and medical faculty; and
4) fostering a lifelong commitment to effective use of computers to solve clinical problems.
Surveys assessed the status of computer expertise among faculty and entering students. The results of these surveys, lessons learned from this experience, and implications for the future of computers in medical education are discussed.
Developing and testing the cognitive skills and abstract thinking of undergraduate medical students are the main objectives of problem based learning. Modified Essay Questions (MEQ) and Multiple Choice Questions (MCQ) may both be designed to test these skills. The objectives of this study were to assess the effectiveness of both forms of questions in testing the different levels of the cognitive skills of undergraduate medical students and to detect any item writing flaws in the questions.
A total of 50 MEQs and 50 MCQs were evaluated. These questions were chosen randomly from various examinations given to different batches of undergraduate medical students taking course MED 411–412 at the Department of Medicine, Qassim University from the years 2005 to 2009. The effectiveness of the questions was determined by two assessors and was defined by the question’s ability to measure higher cognitive skills, as determined by modified Bloom’s taxonomy, and its quality as determined by the presence of item writing flaws. ‘SPSS15’ and ‘Medcalc’ programs were used to tabulate and analyze the data.
The percentage of questions testing the level III (problem solving) cognitive skills of the students was 40% for MEQs and 60% for the MCQs; the remaining questions merely assessed the recall and comprehension. No significant difference was found between MEQ and MCQ in relation to the type of questions (recall; comprehension or problem solving x2 = 5.3, p = 0.07).The agreement between the two assessors was quite high in case of MCQ (kappa=0.609; SE 0.093; 95%CI 0.426 – 0.792) but lower in case of MEQ (kappa=0.195; SE 0.073; 95%CI 0.052 – 0.338). 16% of the MEQs and 12% of the MCQs had item writing flaws.
A well constructed MCQ is superior to MEQ in testing the higher cognitive skills of undergraduate medical students in a problem based learning setup. Constructing an MEQ for assessing the cognitive skills of a student is not a simple task and is more frequently associated with item writing flaws.
Modified essay question; Multiple-choice question; Bloom’s Taxonomy; cognition
This paper assesses the utility of strategic alliances as a way of expanding and improving the quality of cancer care provided in communities with limited access to major treatment centres. Alliances provide an organizational model for future community-based cancer programs by accommodating a growing need for interdependence among organizations and providers while permitting substantial independence and autonomy. Five managerial challenges to ensuring effective and efficient delivery of cancer services are identified: to secure mutually reinforcing exchanges between and within all levels of cancer care, to develop protocols and programs relevant to the unique characteristics of patients and providers, to provide treatment and cancer control services, to involve interdisciplinary teams of providers at all levels of care and to achieve quality assurance, improvement and evaluation. In addition, the paper includes a set of guidelines to facilitate the implementation of community cancer programs as strategic alliances: reaffirm the role of community oncologists, primary care physicians and nurses as partners in the program; define the structure and culture necessary for commitment rather than simply compliance; redefine the role of management; establish data-monitoring systems; modify reward systems; and set realistic time frames and expectations.
Many North American medical schools have removed didactic surgical teaching from the nonclinical years, and there has been a trend toward shortening surgical clerkships. Of concern is that this policy has led to a decrease in surgical exposure and a diminished interest in students pursuing a surgical career. We aimed to determine the effect of curricular change on practical experiences during surgical clerkship and to evaluate overall practical clinical exposure of students during surgical clerkship.
We collected validated experience logbooks completed before (1999–2001) and after (2001–2003) the curriculum change at the University of Alberta and converted them into electronic format. The study analyzed 10 procedures and 5 patient management situations. We assessed numbers of procedures performed and student performance on the Objective Structured Clinical Exam (OSCE) and Multiple-Choice Question (MCQ) examinations before and after the curriculum change. In addition, we completed an overall survey of all 4 classes (2000, 2001, 2002, 2003), measuring clinical exposure. We reviewed a total of 428 logbooks.
There were significant gaps in clinical exposure, which was demonstrated by more than 70% of students in each class failing to complete 8 of 15 procedures or managements at least once. No significant change in practical surgical exposure resulted from the curriculum change. The curriculum change did result in a decrease in end-of-rotation MCQ score performance, which was demonstrated by a 5% decrease in the class average after the curriculum change. Students' performance on ward evaluations and their OSCE scores were unaffected.
We were encouraged that a major change in how surgical education is delivered did not have a detrimental effect on subjective and objective evaluations of student performance. However, we are concerned that a considerable number of students appeared to have not performed several inpatient procedures. Further study is warranted to determine whether this is a common problem in other schools. There is a clear need at our school, and no doubt at others, to establish skills centres and other strategies to ensure that this component of medical education is appropriately and effectively taught.
In Arab countries there are few studies on assessment methods in the field of psychiatry. The objective of this study was to assess the outcome of different forms of psychiatric course assessment among fifth year medical students at King Faisal University, Saudi Arabia.
We examined the performance of 110 fifth-year medical students through objective structured clinical examinations (OSCE), traditional oral clinical examinations (TOCE), portfolios, multiple choice questions (MCQ), and a written examination.
The score ranges in TOCE, OSCE, portfolio, and MCQ were 32–50, 7–15, 5–10 and 22–45, respectively. In regression analysis, there was a significant correlation between OSCE and all forms of psychiatry examinations, except for the MCQ marks. OSCE accounted for 65.1% of the variance in total clinical marks and 31.5% of the final marks (P = 0.001), while TOCE alone accounted for 74.5% of the variance in the clinical scores.
This study demonstrates a consistency among the students’ assessment methods used in the psychiatry course, particularly the clinical component, in an integrated manner. This information would be useful for future developments in undergraduate teaching.
Undergraduate medical students; Assessment; Psychiatry; Undergraduate; Saudi Arabia
To implement and assess the impact of a course utilizing reflective learning to explore the complex, psychosocial human issues encountered in pharmacy practice.
A 1-credit-hour elective course, The Heart of Pharmacy, was offered to all pharmacy students. The course utilized both content and reflective techniques to produce a mutual exploratory learning experience for students, staff, and faculty members. Faculty and staff facilitators observed competencies and used a single group posttest design to assess students’ attitudes. In year four, students’ written reflections for each session were added and reviewed on a continuous basis throughout the course.
Faculty and staff observations indicated that educational outcomes were achieved and student perceptions and evaluations of the course were highly positive. Three major themes were identified in the students’ qualitative responses: a recognition of communal support among student and faculty colleagues; a grounding for personal growth and professional formation; a deeper insight into and experience with the role of the pharmacist as compassionate listener and caregiver.
Faculty observations of student competencies and students’ perceptions of this course point to the need for pharmacy education to provide organized, structured reflective learning opportunities for students and faculty members to explore the deeper human issues of pharmacy practice and patient care.
active learning; emotional intelligence; psychosocial issues; compassionate care; patient care
The Marijuana Craving Questionnaire (MCQ) is a valid and reliable, 47-item self-report instrument that assesses marijuana craving along four dimensions: compulsivity, emotionality, expectancy, and purposefulness. For use in research and clinical settings, we constructed a 12-item version of the MCQ by selecting three items from each of the four factors that exhibited the greatest within-factor internal consistency (Cronbach's alpha coefficient).
Adult marijuana users (n = 490), who had made at least one serious attempt to quit marijuana use but were not seeking treatment, completed the MCQ-Short Form (MCQ-SF) in a single session.
Confirmatory factor analysis of the MCQ-SF indicated good fit with the 4-factor MCQ model, and the coefficient of congruence indicated moderate similarity in factor patterns and loadings between the MCQ and MCQ-SF. Homogeneity (unidimensionality and internal consistency) of MCQ-SF factors was also consistent with reliability values obtained in the initial validation of the MCQ.
Findings of psychometric fidelity indicate that the MCQ-SF is a reliable and valid measure of the same multidimensional aspects of marijuana craving as the MCQ in marijuana users not seeking treatment.
Marijuana craving; Cannabis; Reliability; Validity; Questionnaire
Many medical exams use 5 options for multiple choice questions (MCQs), although the literature suggests that 3 options are optimal. Previous studies on this topic have often been based on non-medical examinations, so we sought to analyse rarely selected, 'non-functional' distractors (NF-D) in high stakes medical examinations, and their detection by item authors as well as psychometric changes resulting from a reduction in the number of options.
Based on Swiss Federal MCQ examinations from 2005-2007, the frequency of NF-D (selected by <1% or <5% of the candidates) was calculated. Distractors that were chosen the least or second least were identified and candidates who chose them were allocated to the remaining options using two extreme assumptions about their hypothetical behaviour: In case rarely selected distractors were eliminated, candidates could randomly choose another option - or purposively choose the correct answer, from which they had originally been distracted. In a second step, 37 experts were asked to mark the least plausible options. The consequences of a reduction from 4 to 3 or 2 distractors - based on item statistics or on the experts' ratings - with respect to difficulty, discrimination and reliability were modelled.
About 70% of the 5-option-items had at least 1 NF-D selected by <1% of the candidates (97% for NF-Ds selected by <5%). Only a reduction to 2 distractors and assuming that candidates would switch to the correct answer in the absence of a 'non-functional' distractor led to relevant differences in reliability and difficulty (and to a lesser degree discrimination). The experts' ratings resulted in slightly greater changes compared to the statistical approach.
Based on item statistics and/or an expert panel's recommendation, the choice of a varying number of 3-4 (or partly 2) plausible distractors could be performed without marked deteriorations in psychometric characteristics.
The only realistic way to improve harmonisation of European pathology training is to define the generally accepted competencies and to test them periodically during the training programme (progress test). The European Association of Pathology Chairs and Program Directors therefore decided to implement an annual on-line test using virtual slides in addition to static jpeg images and theoretical MCQ’s. The EU supported this endeavour as EUROPALS (EUROpean Pathology Assessement & Learning System).
To address the challenges of large scale digital testing EUROPALS teamed up with i-Path Diagnostics Ltd, a company specialising in utilisation of virtual slides in histology/pathology education and examination. Specific examination software was used in the test system.
In the first 2 years we provided at five occasions progress tests, including 2 proctored tests, attracting hundreds of participants. The accessibility varied from suboptimal to good and improved with each subsequent test. It was influenced both by the hosting server capacity and the internet bandwidth at the user’s location.
On-line testing using virtual slides is possible but requires a good collaboration between the provider and the user. Both should be aware of the requirements and threads of large scale testing with hundreds of simultaneous users.
Four- or five-option multiple choice questions (MCQs) are the standard in health-science disciplines, both on certification-level examinations and on in-house developed tests. Previous research has shown, however, that few MCQs have three or four functioning distractors. The purpose of this study was to investigate non-functioning distractors in teacher-developed tests in one nursing program in an English-language university in Hong Kong.
Using item-analysis data, we assessed the proportion of non-functioning distractors on a sample of seven test papers administered to undergraduate nursing students. A total of 514 items were reviewed, including 2056 options (1542 distractors and 514 correct responses). Non-functioning options were defined as ones that were chosen by fewer than 5% of examinees and those with a positive option discrimination statistic.
The proportion of items containing 0, 1, 2, and 3 functioning distractors was 12.3%, 34.8%, 39.1%, and 13.8% respectively. Overall, items contained an average of 1.54 (SD = 0.88) functioning distractors. Only 52.2% (n = 805) of all distractors were functioning effectively and 10.2% (n = 158) had a choice frequency of 0. Items with more functioning distractors were more difficult and more discriminating.
The low frequency of items with three functioning distractors in the four-option items in this study suggests that teachers have difficulty developing plausible distractors for most MCQs. Test items should consist of as many options as is feasible given the item content and the number of plausible distractors; in most cases this would be three. Item analysis results can be used to identify and remove non-functioning distractors from MCQs that have been used in previous tests.
Objective: The OSCE (objective structured clinical examination) was put to the test in order to assess the clinical practical skills of students in medical studies in the medical faculties. For the implementation of an OSCE, a large number of personnel is necessary. In particular for subjects with limited resources, therefore, efficient cost planning is required. In the winter semester 09/10, the Department of Neurology at the Medical Faculty of the University of Ulm introduced the OSCE as a pilot project. A financial report retrospectively shows the personnel expenses. The report is intended as an example for an insight into the resources needed for the OSCE with simulated patients.
Method: Included in the calculation of the financial costs of the OSCE were: employment, status of staff involved in the OSCE, subject-matter and temporal dimension of the task. After the exam, acceptance of the examination format was reviewed by a focus group interview with the teachers and students.
Result: The total expenses for the personnel involved in the OSCE amounted to 12,468 €. The costing of the clinic’s share was calculated at 9,576 €. Tuition fees from the students have been used to the amount of 2.892 €. For conversion of total expenditure to the number of examines the sum of 86 € per student was calculated. Both students and teachers confirmed the validity of the OSCE and recognised the added value in the learning effects.
Conclusion: The high acceptance of the OSCE in neurology by both students and teachers favours maintaining the test format. Against the background of the high financial and logistical costs, however, in individual cases it should be assessed how in the long-term efficient examination procedure will be possible.
OSCE; clinical and practical examination; staff
To evaluate the effectiveness of holding weekly 60-90 minute conversation forums for faculty members to discuss, explore, and reflect on various teaching topics in a relaxed, informal, interactive format.
Weekly, 60-90 minute sessions were held for faculty members of the University of Illinois College of Pharmacy. A 15-item retrospective pretest-posttest questionnaire was developed and administered at the end of the first year of implementation to evaluate the participants’ perceived knowledge, abilities, and confidence gains relative to becoming effective educators.
Eleven faculty members completed the questionnaire. All respondents tended to agree (6/11) or agreed (5/11) that their confidence as educators improved after attending the conversation forums. In addition,7 respondents tended to agree and 4 agreed that their ability to self-assess their teaching had improved.
An ongoing weekly conversations forum provides faculty members opportunities to explore and learn about facets of teaching in a safe, informal environment.
teaching; mentoring; Rasch model; retrospective pre-post questionnaire; faculty development
This preliminary study examined how patients' defense mechanisms and psychotherapists' techniques influence early alliance formation. The authors assessed the relationships among defense mechanisms, therapist interventions, and the development of alliance in a sample of 12 patients undergoing Brief Psychodynamic Investigation (4 sessions). Alliance development occurred rapidly and was clearly established by the third session. Neither defensive functioning nor supportive or exploratory interventions alone differentiated early alliance development. However, the degree of adjustment of therapists' interventions to patients' level of defensive functioning discriminated a low alliance from both improving and high alliances. The adjustment of therapeutic interventions to patients' level of defensive functioning is a promising predictor of alliance development and should be examined further, alongside other predictors of outcome.
Therapeutic Alliance; Defense Mechanisms; Interventions
Describe the planning, implementation, and faculty perceptions of a classroom peer-review process, including an evaluation tool.
A process for peer evaluation of classroom teaching and its evaluation tool were developed and implemented by a volunteer faculty committee within our department. At the end of the year, all faculty members were asked to complete an online anonymous survey to evaluate the experience.
The majority of faculty members either agreed or strongly agreed that the overall evaluation process was beneficial for both evaluators and for those being evaluated. Some areas of improvement related to the process and its evaluation tool also were identified.
The process of developing and implementing a peer-evaluation process for classroom teaching was found to be beneficial for faculty members, and the survey results affirmed the need and continuation of such a process.
peer evaluation; classroom; teaching; survey; faculty; assessment
Introduction: Implementation of a longitudinal curriculum for training in advanced communications skills represents an unmet need in most German medical faculties, especially in the 4rth and 5th years of medical studies. The CoMeD project (communication in medical education Düsseldorf) attempted to establish an interdisciplinary program to teach and to assess communicative competence in the 4th academic year. In this paper, we describe the development of the project and report results of its evaluation by medical students.
Methods: Teaching objectives and lesson formats were developed in a multistage process. A teaching program for simulated patients (SP) was built up and continuous lecturer trainings were estabilshed. Several clinical disciplines co-operated for the purpose of integrating the communication training into the pre-existing clinical teaching curriculum. The CoMeD project was evaluated using feedback-forms after each course.
Results: Until now, six training units for especially challenging communication tasks like “dealing with aggression” or “breaking bad news” were implemented, each unit connected with a preliminary tutorial or e-learning course. An OSCE (objective structured clinical examination) with 4 stations was introduced. The students’ evaluation of the six CoMeD training units showed the top or second-best rating in more than 80% of the answers.
Discussion: Introducing an interdisciplinary communication training and a corresponding OSCE into the 4th year medical curriculum is feasible. Embedding communication teaching in a clinical context and involvement of clinicians as lecturers seem to be important factors for ensuring practical relevance and achieving high acceptance by medical students.
medical education; communication; curriculum development; simulated patients; medical interview
The postgraduate training program in psychiatry in Saudi Arabia, which was established in 1997, is a 4-year residency program. Written exams comprising of multiple choice questions (MCQs) are used as a summative assessment of residents in order to determine their eligibility for promotion from one year to the next. Test blueprints are not used in preparing examinations.
To develop test blueprints for the written examinations used in the psychiatry residency program.
Based on the guidelines of four professional bodies, documentary analysis was used to develop global and detailed test blueprints for each year of the residency program. An expert panel participated during piloting and final modification of the test blueprints. Their opinion about the content, weightage for each content domain, and proportion of test items to be sampled in each cognitive category as defined by modified Bloom’s taxonomy were elicited.
Eight global and detailed test blueprints, two for each year of the psychiatry residency program, were developed. The global test blueprints were reviewed by experts and piloted. Six experts participated in the final modification of test blueprints. Based on expert consensus, the content, total weightage for each content domain, and proportion of test items to be included in each cognitive category were determined for each global test blueprint. Experts also suggested progressively decreasing the weightage for recall test items and increasing problem solving test items in examinations, from year 1 to year 4 of the psychiatry residence program.
A systematic approach using a documentary and content analysis technique was used to develop test blueprints with additional input from an expert panel as appropriate. Test blueprinting is an important step to ensure the test validity in all residency programs.
test blueprinting; psychiatry; residency program; summative assessment; documentary and content analysis; Kingdom of Saudi Arabia