Aims: In most German medical faculties, credits in general practice can be earned via exams using multiple-choice questions (MCQ). Measures such as peer-reviews may help assure the quality of these exams. In order to use time and personnel intensive peer reviews effectively and efficiently, the procedures used are key. Therefore, we wanted to find out whether there are differences between group and individual reviews regarding defined parameters.
Methods: We conducted a controlled cross-over study with three GP reviewers from four different German universities. Each reviewed 80 MCQs, 40 individually and 40 within a group, including external assessments by a panel of experts. Furthermore all reviewers were asked to evaluate the review process and the time spent carrying out these reviews.
Outcomes: We found no significant differences between the reliability and the validity of individual reviews versus group reviews. On average slightly more time was spent on group reviews compared with the individual reviews. The subjective assessments of the study participants regarding their satisfaction with the process and the efficiency and effectiveness of the reviews suggest a preference for group reviews.
Conclusions: Based on this study, there are no definite recommendations for or against either approach. When choosing between the two, the specific work structures and organisation at the local faculty should be taken into account.
Medical Educatio; assessment; Multiple-Choice-Questions; Review
Objective: The purpose of the study was to identify technical item flaws in the multiple choice questions submitted for the final exams for the years 2009, 2010 and 2011.
Methods: This descriptive analytical study was carried out in Islamic International Medical College (IIMC). The Data was collected from the MCQ’s submitted by the faculty for the final exams for the year 2009, 2010 and 2011. The data was compiled and evaluated by a three member assessment committee. The data was analyzed for frequency and percentages the categorical data was analyzed by chi-square test.
Results: Overall percentage of flawed item was 67% for the year 2009 of which 21% were for testwiseness and 40% were for irrelevant difficulty. In year 2010 the total item flaws were 36% and 11% testwiseness and 22% were for irrelevant difficulty. The year 2011 data showed decreased overall flaws of 21%. The flaws of testwisness were 7%, irrelevant difficulty were 11%.
Conclusion: Technical item flaws are frequently encountered during MCQ construction, and the identification of flaws leads to improved quality of the single best MCQ’s.
Frequency; Item writing flaws; Testwiseness
The purpose of this study is to describe an approach for evaluating assessments used in the first 2 years of medical school and report the results of applying this method to current first and second year medical student examinations.
Three faculty members coded all exam questions administered during the first 2 years of medical school. The reviewers discussed and compared the coded exam questions. During the bi-monthly meetings, all differences in coding were resolved with consensus as the final criterion. We applied Moore's framework to assist the review process and to align it with National Board of Medical Examiners (NBME) standards.
The first and second year medical school examinations had 0% of competence level questions. The majority, more than 50% of test questions, were at the NBME recall level.
It is essential that multiple-choice questions (MCQs) test the attitudes, skills, knowledge, and competency in medical school. Based on our findings, it is evident that our exams need to be improved to better prepare our medical students for successful completion of NBME step exams.
undergraduate medical education; assessment; course exams; NBME
Background: Multiple choice questions (MCQs) are often used in exams of medical education and need careful quality management for example by the application of review committees. This study investigates whether groups communicating virtually by email are similar to face-to-face groups concerning their review process performance and whether a facilitator has positive effects.
Methods: 16 small groups of students were examined, which had to evaluate and correct MCQs under four different conditions. In the second part of the investigation the changed questions were given to a new random sample for the judgement of the item quality.
Results: There was no significant influence of the variables “form of review committee” and “facilitation”. However, face-to-face and virtual groups clearly differed in the required treatment times. The test condition “face to face without facilitation” was generally valued most positively concerning taking over responsibility, approach to work, sense of well-being, motivation and concentration on the task.
Discussion: Face-to-face and virtual groups are equally effective in the review of MCQs but differ concerning their efficiency. The application of electronic review seems to be possible but is hardly recommendable because of the long process time and technical problems.
multiple choice questions; MCQ; face-to-fac; virtual; facilitation; review-committee
The assessment of the anesthesia course in our university comprises Objective Structured Clinical Examinations (OSCEs), in conjunction with portfolio and multiple-choice questions (MCQ). The objective of this study was to evaluate the outcome of different forms of anesthesia course assessment among 5th year medical students in our university, as well as study the influence of gender on student performance in anesthesia.
We examined the performance of 154, 5th year medical students through OSCE, portfolios, and MCQ.
The score ranges in the portfolio, OSCE, and MCQs were 16-24, 4.2-28.9, and 15.5-44.5, respectively. There was highly significant difference in scores in relation to gender in all assessments other than the written one (P=0.000 for Portfolio, OSCE, and Total exam, whereas P=0.164 for written exam). In the generated linear regression model, OSCE alone could predict 86.4% of the total mark if used alone. In addition, if the score of the written examination is added, OSCE will drop to 57.2% and the written exam will be 56.8% of the total mark.
This study demonstrates that different clinical methods used to assess medical students during their anesthesia course were consistent and integrated. The performance of female was superior to male in OSCE and portfolio. This information is the basis for improving educational and assessment standards in anesthesiology and for introducing a platform for developing modern learning media in countries with dearth of anesthesia personnel.
Anesthesiology; assessment; gender; objective structured clinical examination; portfolios and multiple-choice questions
International reports recommend the improvement in the amount and quality of training for mental health workers in low and middle income countries. The Scotland-Malawi Mental Health Education Project (SMMHEP) has been established to support the teaching of psychiatry to medical students in the University of Malawi. While anecdotally supportive medical educational initiatives appear of value, little quantitative evidence exists to demonstrate whether such initiatives can deliver comparable educational standards. This study aimed to assess the effectiveness of an undergraduate psychiatry course given by UK psychiatrists in Malawi by studying University of Malawi and Edinburgh University medical students' performance on an MCQ examination paper.
An undergraduate psychiatry course followed by an MCQ exam was delivered by the SMMHEP to 57 Malawi medical students. This same MCQ exam was given to 71 Edinburgh University medical students who subsequently sat their own Edinburgh University examination.
There were no significant differences between Edinburgh students' performance on the Malawi exam and their own Edinburgh University exam. (p = 0.65). This would suggest that the Malawi exam is a comparable standard to the Edinburgh exam. Malawi students marks ranged from 52.4%–84.6%. Importantly 84.4% of Malawi students scored above 60% on their exam which would equate to a hypothetical pass by UK university standards.
The support of an undergraduate course in an African setting by high income country specialists can attain a high percentage pass rate by UK standards. Although didactic teaching has been surpassed by more novel educational methods, in resource poor countries it remains an effective and cost effective method of gaining an important educational standard.
In 2004 the adoption of a modular curriculum at the medical faculty in Muenster led to the introduction of centralized examinations based on multiple-choice questions (MCQs). We report on how organizational challenges of realizing faculty-wide personalized tests were addressed by implementation of a specialized software module to automatically generate test sheets from individual test registrations and MCQ contents.
Key steps of the presented method for preparing personalized test sheets are (1) the compilation of relevant item contents and graphical media from a relational database with database queries, (2) the creation of Extensible Markup Language (XML) intermediates, and (3) the transformation into paginated documents.
The software module by use of an open source print formatter consistently produced high-quality test sheets, while the blending of vectorized textual contents and pixel graphics resulted in efficient output file sizes. Concomitantly the module permitted an individual randomization of item sequences to prevent illicit collusion.
The automatic generation of personalized MCQ test sheets is feasible using freely available open source software libraries, and can be efficiently deployed on a faculty-wide scale.
The purpose of this report was to describe the development, implementation, and outcomes from 3 complementary programs to facilitate the development of faculty members. The Faculty Development Committee (FDC) at the University of Tennessee developed 3 new complementary programs: the Individual Faculty Development Program to encourage faculty members to assess and identify their own specific developmental needs; the Seed Research Grant Program to fund scholarly activities by faculty; and the Technology Support Program to foster financial support of technology upgrades crucial for meeting the research, education, and service needs of faculty members. Eighteen faculty members participated in the Individual Faculty Development Program during the first 2 academic years and all provided positive feedback about their experiences. The Seed Research Grant Program funded 6 projects during its inaugural year. Limited outcome data from these 2 programs are extremely favorable relative to grant submissions and publications, and enhanced educational offerings and evaluations. The Technology Support Fund was initiated in the 2005-2006 academic year. The 3 faculty development programs initiated are offered as examples whereby faculty members are given a high degree of self-determination relative to identifying programs that will effectively contribute to their growth as academicians. Other colleges of pharmacy are encouraged to consider similar initiatives to foster individual faculty development at this critical period of growth within academic pharmacy.
faculty development; research; technology
After years of ‘ad hoc’ growth in the use of computers in the curriculum, the University of Minnesota Medical School in cooperation with the Bio-Medical Library and Health Sciences Computing Services developed and began implementation of a plan for integration of medical informatics into all phases of medical education.
Objectives were developed which focus on teaching skills related to:
1) accessing, retrieving, evaluating and managing medical information;
2) appropriate utilization of computer-assisted instruction lessons;
3) electronic communication with fellow students and medical faculty; and
4) fostering a lifelong commitment to effective use of computers to solve clinical problems.
Surveys assessed the status of computer expertise among faculty and entering students. The results of these surveys, lessons learned from this experience, and implications for the future of computers in medical education are discussed.
In recent years, medical faculties at Dutch universities have implemented a legally binding study advice to students of medicine and biomedical sciences during their propaedeutic phase. Appropriate examination is essential to discriminate between poor (grade <6), moderate (grade 6–8) and excellent (grade ≥8) students. Therefore, we compared the discriminatory properties of extended matching questions (EMQs) versus multiple-choice questions (MCQs) and identified the role of sex, age and examination preference on this score. Data were collected for 452 first-year medical and biomedical science students during three distinct course examinations: one examination with EMQ only, one with MCQ only and one mixed examination (including EMQ and MCQ). Logistic regression analysis revealed that MCQ examination was 3 times better in identifying poor students compared with EMQ (RR 3.0, CI 2.0–4.5), whereas EMQ better detected excellent students (average grade ≥8) (RR 1.93, CI 1.47–2.53). Mixed examination had comparable characteristics to MCQ. Sex and examination preference did not impact the score of the student. Students ≥20 years had a 4-fold higher risk ratio of obtaining a poor grade (<6) compared with students ≤18 years old (RR 4.1, CI 2.1–8.0). Given the strong discriminative capacity of MCQ examinations to identify poor students, we recommend the use of this type of examination during the propaedeutic phase of medicine and biomedical science study programmes, in the light of the binding study advice.
Binding study advice; Medical education; Medicine; Biomedical sciences; Propaedeutic phase; Age
To evaluate the impact of two different assessment formats on the approaches to learning of final year veterinary students. The relationship between approach to learning and examination performance was also investigated.
An 18-item version of the Study Process Questionnaire (SPQ) was sent to 87 final year students. Each student responded to the questionnaire with regards to DOPS (Direct Observation of Procedural Skills) and a Multiple Choice Examination (MCQ). Semi-structured interviews were conducted with 16 of the respondents to gain a deeper insight into the students’ perception of assessment.
Students’ adopted a deeper approach to learning for DOPS and a more surface approach with MCQs. There was a positive correlation between an achieving approach to learning and examination performance. Analysis of the qualitative data revealed that deep, surface and achieving approaches were reported by the students and seven major influences on their approaches to learning were identified: motivation, purpose, consequence, acceptability, feedback, time pressure and the individual difference of the students.
The format of DOPS has a positive influence on approaches to learning. There is a conflict for students between preparing for final examinations and preparing for clinical practice.
Developing and testing the cognitive skills and abstract thinking of undergraduate medical students are the main objectives of problem based learning. Modified Essay Questions (MEQ) and Multiple Choice Questions (MCQ) may both be designed to test these skills. The objectives of this study were to assess the effectiveness of both forms of questions in testing the different levels of the cognitive skills of undergraduate medical students and to detect any item writing flaws in the questions.
A total of 50 MEQs and 50 MCQs were evaluated. These questions were chosen randomly from various examinations given to different batches of undergraduate medical students taking course MED 411–412 at the Department of Medicine, Qassim University from the years 2005 to 2009. The effectiveness of the questions was determined by two assessors and was defined by the question’s ability to measure higher cognitive skills, as determined by modified Bloom’s taxonomy, and its quality as determined by the presence of item writing flaws. ‘SPSS15’ and ‘Medcalc’ programs were used to tabulate and analyze the data.
The percentage of questions testing the level III (problem solving) cognitive skills of the students was 40% for MEQs and 60% for the MCQs; the remaining questions merely assessed the recall and comprehension. No significant difference was found between MEQ and MCQ in relation to the type of questions (recall; comprehension or problem solving x2 = 5.3, p = 0.07).The agreement between the two assessors was quite high in case of MCQ (kappa=0.609; SE 0.093; 95%CI 0.426 – 0.792) but lower in case of MEQ (kappa=0.195; SE 0.073; 95%CI 0.052 – 0.338). 16% of the MEQs and 12% of the MCQs had item writing flaws.
A well constructed MCQ is superior to MEQ in testing the higher cognitive skills of undergraduate medical students in a problem based learning setup. Constructing an MEQ for assessing the cognitive skills of a student is not a simple task and is more frequently associated with item writing flaws.
Modified essay question; Multiple-choice question; Bloom’s Taxonomy; cognition
This paper assesses the utility of strategic alliances as a way of expanding and improving the quality of cancer care provided in communities with limited access to major treatment centres. Alliances provide an organizational model for future community-based cancer programs by accommodating a growing need for interdependence among organizations and providers while permitting substantial independence and autonomy. Five managerial challenges to ensuring effective and efficient delivery of cancer services are identified: to secure mutually reinforcing exchanges between and within all levels of cancer care, to develop protocols and programs relevant to the unique characteristics of patients and providers, to provide treatment and cancer control services, to involve interdisciplinary teams of providers at all levels of care and to achieve quality assurance, improvement and evaluation. In addition, the paper includes a set of guidelines to facilitate the implementation of community cancer programs as strategic alliances: reaffirm the role of community oncologists, primary care physicians and nurses as partners in the program; define the structure and culture necessary for commitment rather than simply compliance; redefine the role of management; establish data-monitoring systems; modify reward systems; and set realistic time frames and expectations.
To develop and psychometrically assess a multiple choice question (MCQ) instrument to test knowledge of depression and its treatments in patients suffering from depression.
A total of 63 depressed patients and twelve psychiatric experts participated. Based on empirical evidence from an extensive review, theoretical knowledge and in consultations with experts, 27-item MCQ knowledge of depression and its treatment test was constructed. Data collected from the psychiatry experts were used to assess evidence of content validity for the instrument.
Cronbach's alpha of the instrument was 0.68, and there was an overall 87.8% agreement (items are highly relevant) between experts about the relevance of the MCQs to test patient knowledge on depression and its treatments. There was an overall satisfactory patients' performance on the MCQs with 78.7% correct answers. Results of an item analysis indicated that most items had adequate difficulties and discriminations.
There was adequate reliability and evidence for content and convergent validity for the instrument. Future research should employ a lager and more heterogeneous sample from both psychiatrist and community samples, than did the present study. Meanwhile, the present study has resulted in psychometrically tested instruments for measuring knowledge of depression and its treatment of depressed patients.
The Advanced Trauma Operative Management (ATOM) course was first introduced into Canada in 2003 at the University of Toronto, with senior general surgery residents being the primary focus. We present an assessment of the course in this Canadian general surgery residency program.
We compared trainees' pre-and postcourse self-efficacy scores and multiple choice question (MCQ) examination results, using paired t tests and resident (n = 24) and faculty (n = 7) course ratings made according to a 10-item, 5-point Likert scale. Faculty were previously trained as ATOM instructors.
Mean pre-and postcourse self-efficacy scores were 68.9 (standard deviation [SD] 24.0) and 101.4 (SD 14.8), respectively (p < 0.001). Mean pre-and post-MCQ scores were 16.4 (SD 3.2) and 18.8 (SD 2.7), respectively (p = 0.006). On the Likert scale (1 = strongly disagree, 5 = strongly agree), all faculty and residents rated the following items as 4–5: objectives were met; knowledge, skills, clinical training, judgment and confidence improved; the live animal is a useful representation of clinical trauma; and the course should be continued but would be more appropriate for the fourth rather than the fifth year of residency. Residents rated as 1–2 the item that the human cadaver would be preferable for learning the surgical skills. Of 24 residents, 20 rated as 3 or less the item stating that the course prepares them for trauma management more adequately than their regular training program.
Self-efficacy, trauma knowledge and skills improved significantly with ATOM training. Preference was expressed for the live animal versus cadaver model, for ATOM training in the fourth rather than fifth year of residency and for the view that it complements general surgery trauma training. The data suggest that including ATOM training in Canadian general surgical residency should be considered.
Many North American medical schools have removed didactic surgical teaching from the nonclinical years, and there has been a trend toward shortening surgical clerkships. Of concern is that this policy has led to a decrease in surgical exposure and a diminished interest in students pursuing a surgical career. We aimed to determine the effect of curricular change on practical experiences during surgical clerkship and to evaluate overall practical clinical exposure of students during surgical clerkship.
We collected validated experience logbooks completed before (1999–2001) and after (2001–2003) the curriculum change at the University of Alberta and converted them into electronic format. The study analyzed 10 procedures and 5 patient management situations. We assessed numbers of procedures performed and student performance on the Objective Structured Clinical Exam (OSCE) and Multiple-Choice Question (MCQ) examinations before and after the curriculum change. In addition, we completed an overall survey of all 4 classes (2000, 2001, 2002, 2003), measuring clinical exposure. We reviewed a total of 428 logbooks.
There were significant gaps in clinical exposure, which was demonstrated by more than 70% of students in each class failing to complete 8 of 15 procedures or managements at least once. No significant change in practical surgical exposure resulted from the curriculum change. The curriculum change did result in a decrease in end-of-rotation MCQ score performance, which was demonstrated by a 5% decrease in the class average after the curriculum change. Students' performance on ward evaluations and their OSCE scores were unaffected.
We were encouraged that a major change in how surgical education is delivered did not have a detrimental effect on subjective and objective evaluations of student performance. However, we are concerned that a considerable number of students appeared to have not performed several inpatient procedures. Further study is warranted to determine whether this is a common problem in other schools. There is a clear need at our school, and no doubt at others, to establish skills centres and other strategies to ensure that this component of medical education is appropriately and effectively taught.
To implement and assess the impact of a course utilizing reflective learning to explore the complex, psychosocial human issues encountered in pharmacy practice.
A 1-credit-hour elective course, The Heart of Pharmacy, was offered to all pharmacy students. The course utilized both content and reflective techniques to produce a mutual exploratory learning experience for students, staff, and faculty members. Faculty and staff facilitators observed competencies and used a single group posttest design to assess students’ attitudes. In year four, students’ written reflections for each session were added and reviewed on a continuous basis throughout the course.
Faculty and staff observations indicated that educational outcomes were achieved and student perceptions and evaluations of the course were highly positive. Three major themes were identified in the students’ qualitative responses: a recognition of communal support among student and faculty colleagues; a grounding for personal growth and professional formation; a deeper insight into and experience with the role of the pharmacist as compassionate listener and caregiver.
Faculty observations of student competencies and students’ perceptions of this course point to the need for pharmacy education to provide organized, structured reflective learning opportunities for students and faculty members to explore the deeper human issues of pharmacy practice and patient care.
active learning; emotional intelligence; psychosocial issues; compassionate care; patient care
Describe the planning, implementation, and faculty perceptions of a classroom peer-review process, including an evaluation tool.
A process for peer evaluation of classroom teaching and its evaluation tool were developed and implemented by a volunteer faculty committee within our department. At the end of the year, all faculty members were asked to complete an online anonymous survey to evaluate the experience.
The majority of faculty members either agreed or strongly agreed that the overall evaluation process was beneficial for both evaluators and for those being evaluated. Some areas of improvement related to the process and its evaluation tool also were identified.
The process of developing and implementing a peer-evaluation process for classroom teaching was found to be beneficial for faculty members, and the survey results affirmed the need and continuation of such a process.
peer evaluation; classroom; teaching; survey; faculty; assessment
The Accreditation Council for Graduate Medical Education requires scholarly activity for both faculty and residents in obstetrics and gynecology (Ob-Gyn). There is little evidence on the most effective method to train, recruit, and retain research faculty who can mentor resident researchers at small programs.
To address this problem, we created the “Baby Steps” program for a small university-based Ob-Gyn program.
After a thorough assessment of existing resources, a postdoctoral researcher was recruited and coupled with an established researcher to raise the standards of resident research, facilitate and coordinate resident projects, and support clinical faculty participation in research activities. Grant submissions, grants awarded, publications submitted, presentations, and awards were tracked before and after the implementation of the Baby Steps program for faculty and residents.
After 2 years the program has already begun to show an increase in scholarly activity. In a program of 12 residents, 8 made one or more presentations at regional or national meetings within the previous 24 months. Additionally, 8 of 12 clinical faculty members were engaged as mentors in resident research, compared with only 3 in past years. Further, abstract, paper, and grant submissions by faculty increased approximately 25%.
The addition of a mentored postdoctoral researcher was associated with improvements to both resident and faculty research activities. Based on this success, a sister residency program has incorporated the Baby Steps approach into its training.
The Memory Compensation Questionnaire (MCQ; de Frias & Dixon, 2005) is a psychometrically sound instrument which assesses the variety and extent to which an individual compensates for actual or perceived memory losses. Until now, only an English version of the MCQ has been psychometrically evaluated. The aim of the present study was to establish a Dutch version of the MCQ and evaluate its psychometric properties. The MCQ data of N=556 cognitively healthy adults (61.8% females) aged between 50.1 and 95.3 years (M=73.9 years, SD=8.0) were analyzed. The results showed that the factor structure of the Dutch version of the MCQ corresponded well with that of the English version of the MCQ. The reliabilities of the scales of the Dutch version of the MCQ were all high (all Cronbach’s α-values ≥ .77). Demographic variables (especially age and gender) affected most of the MCQ scale scores. Regression-based normative data which take these demographic influences into account were established, and a user-friendly computer program was provided to facilitate the scoring and norming of the MCQ.
Memory Compensation Questionnaire; construct validity; reliability; metacognition; Dutch translation; compensatory behavior
To evaluate the effectiveness of holding weekly 60-90 minute conversation forums for faculty members to discuss, explore, and reflect on various teaching topics in a relaxed, informal, interactive format.
Weekly, 60-90 minute sessions were held for faculty members of the University of Illinois College of Pharmacy. A 15-item retrospective pretest-posttest questionnaire was developed and administered at the end of the first year of implementation to evaluate the participants’ perceived knowledge, abilities, and confidence gains relative to becoming effective educators.
Eleven faculty members completed the questionnaire. All respondents tended to agree (6/11) or agreed (5/11) that their confidence as educators improved after attending the conversation forums. In addition,7 respondents tended to agree and 4 agreed that their ability to self-assess their teaching had improved.
An ongoing weekly conversations forum provides faculty members opportunities to explore and learn about facets of teaching in a safe, informal environment.
teaching; mentoring; Rasch model; retrospective pre-post questionnaire; faculty development
Describe the planning and implementation of a pilot peer review system, assess factors related to acceptance by faculty and administration, and suggest ways to increase the number of faculty members reviewed and serving as reviewers.
A faculty-driven process was used to create a model for peer review. Faculty members completed a survey instrument with open-ended responses for indicating reasons for participation or nonparticipation, components of the evaluation process that they would like to see changed, and what they found most helpful or insightful about the process of peer review.
Faculty acceptance of and satisfaction with the peer review process is attributed to the development and implementation process being faculty driven and to peer reviews not being required for promotion and tenure decisions. Faculty members who were reviewed stated that the process was helpful and insightful and would lead to better teaching and learning.
A successful faculty peer-review process was created and implemented within 6 weeks. All of the faculty members who chose to be peer reviewed or serve as reviewers reported satisfaction in gaining insights into their teaching, learning innovative approaches to their teaching, and gaining confidence in their teaching pedagogy. Techniques for achieving 100% participation in the peer review process should be addressed in the future.
peer review; faculty development; assessment
Introduction: Implementation of a longitudinal curriculum for training in advanced communications skills represents an unmet need in most German medical faculties, especially in the 4rth and 5th years of medical studies. The CoMeD project (communication in medical education Düsseldorf) attempted to establish an interdisciplinary program to teach and to assess communicative competence in the 4th academic year. In this paper, we describe the development of the project and report results of its evaluation by medical students.
Methods: Teaching objectives and lesson formats were developed in a multistage process. A teaching program for simulated patients (SP) was built up and continuous lecturer trainings were estabilshed. Several clinical disciplines co-operated for the purpose of integrating the communication training into the pre-existing clinical teaching curriculum. The CoMeD project was evaluated using feedback-forms after each course.
Results: Until now, six training units for especially challenging communication tasks like “dealing with aggression” or “breaking bad news” were implemented, each unit connected with a preliminary tutorial or e-learning course. An OSCE (objective structured clinical examination) with 4 stations was introduced. The students’ evaluation of the six CoMeD training units showed the top or second-best rating in more than 80% of the answers.
Discussion: Introducing an interdisciplinary communication training and a corresponding OSCE into the 4th year medical curriculum is feasible. Embedding communication teaching in a clinical context and involvement of clinicians as lecturers seem to be important factors for ensuring practical relevance and achieving high acceptance by medical students.
medical education; communication; curriculum development; simulated patients; medical interview
This paper describes the faculty enrichment activities and outcomes of a faculty orientation and development committee at a college of pharmacy. The committee used a continuous quality improvement (CQI) framework that included needs assessment, planning and implementation of programs and workshops, assessment of activities, and evaluation of feedback to improve future programming. Some of the programs established by the committee include a 3-month orientation process for new hires and development workshops on a broad range of topics including scholarship (eg, research methods), teaching (eg, test-item writing), and general development (mentorship). Evidence of the committee's success is reflected by high levels of faculty attendance at workshops, positive feedback on workshop evaluations, and overall high levels of satisfaction with activities. The committee has served as a role model for improving faculty orientation and retention.
faculty; faculty retention; faculty orientation; faculty development
Assessment has a powerful influence on curriculum delivery. Medical instructors must use tools which conform to educational principles, and audit them as part of curriculum review.
To generate information to support recommendations for improving curriculum delivery.
Pre-clinical and clinical departments in a College of Medicine, Saudi Arabia.
A self-administered questionnaire was used in a cross-sectional survey to see if assessment tools being used met basic standards of validity, reliability and currency, and if feedback to students was adequate. Excluded were cost, feasibility and tool combinations.
Thirty-one (out of 34) courses were evaluated. All 31 respondents used MCQs, especially one-best (28/31) and true/false (13/31). Groups of teachers selected test questions mostly. Pre-clinical departments sourced equally from “new” (10/14) and “used” (10/14) MCQs; clinical departments relied on ‘banked’ MCQs (16/17). Departments decided pass marks (28/31) and chose the College-set 60%; the timing was pre-examination in 13/17 clinical but post-examination in 5/14 pre-clinical departments. Of six essay users, five used model answers but only one did double marking. OSCE was used by 7/17 clinical departments; five provided checklist. Only 3/31 used optical reader. Post-marking review was done by 13/14 pre-clinical but 10/17 clinical departments. Difficulty and discriminating indices were determined by only 4/31 departments. Feedback was provided by 12/14 pre-clinical and 7/17 clinical departments. Only 10/31 course coordinators had copies of examination regulations.
MCQ with single-best answer, if properly constructed and adequately critiqued, is the preferred tool for assessing theory domain. However, there should be fresh questions, item analyses, comparisons with pervious results, optical reader systems and double marking. Departments should use OSCE or OSPE more often. Long essays, true/false, fill-in-the-blank-spaces and more-than-one-correct-answer can be safely abolished. Departments or teams should set test papers and collectively take decisions. Feedback rates should be improved. A Center of Medical Education, including an Examination Center is required. Fruitful future studies can be repeat audit, use of “negative questions” and the number of MCQs per test paper. Comparative audit involving other regional medical schools may be of general interest.
Assessment Technique; Curriculum review; MCQ