Search tips
Search criteria

Results 1-7 (7)

Clipboard (0)

Select a Filter Below

more »
Year of Publication
author:("niacini, John")
1.  Good exams made easy: The item management system for multiple examination formats 
BMC Medical Education  2012;12:63.
The development, implementation and evaluation of assessments require considerable resources and often cannot be carried out by a single faculty/institution. Therefore some medical faculties have founded cooperation projects which mainly focus on the exchange of multiple choice questions (MCQs).
Since these cooperation projects do not entirely support all relevant processes in terms of preparation, implementation and evaluation of assessment, in 2006 the Medical Assessment Alliance (MAA) was founded for mutual support. In addition to MCQs the MAA started to develop innovative assessment formats and facilitate content through a coordinated exchange of experiences. To support cooperation within this network, the web-based Item Management System (IMS) was developed which supports all processes of the assessment workflow as an all-in-one working platform.
At present, the Alliance has 28 partner faculties in Europe. More than 2.800 users in 750 working groups are collaborating. Currently 90.000 questions have been stored in the IMS. Since 2007, nearly 4.600 examinations have been successfully conducted.
This article describes in detail the unique features of the IMS and contrasts it with the item management systems of other associations.
PMCID: PMC3441576  PMID: 22857655
Assessment alliance; Quality control
2.  Where next with revalidation? 
BMJ : British Medical Journal  2005;330(7506):1458-1459.
PMCID: PMC558441  PMID: 15976394
3.  Use of SPRAT for peer review of paediatricians in training 
BMJ : British Medical Journal  2005;330(7502):1251-1253.
Objective To determine whether a multisource feedback questionnaire, SPRAT (Sheffield peer review assessment tool), is a feasible and reliable assessment method to inform the record of in-training assessment for paediatric senior house officers and specialist registrars.
Design Trainees' clinical performance was evaluated using SPRAT sent to clinical colleagues of their choosing. Responses were analysed to determine variables that affected ratings and their measurement characteristics.
Setting Three tertiary hospitals and five secondary hospitals across a UK deanery.
Participants 112 paediatric senior house officers and middle grades.
Main outcome measures 95% confidence intervals for mean ratings; linear and hierarchical regression to explore potential biasing factors; time needed for the process per doctor.
Results 20 middle grades and 92 senior house officers were assessed using SPRAT to inform their record of in-training assessment; 921/1120 (82%) of their proposed raters completed a SPRAT form. As a group, specialist registrars (mean 5.22, SD 0.34) scored significantly higher (t = – 4.765) than did senior house officers (mean 4.81, SD 0.35) (P < 0.001). The grade of the doctor accounted for 7.6% of the variation in the mean ratings. The hierarchical regression showed that only 3.4% of the variation in the means could be additionally attributed to three main factors (occupation of rater, length of working relationship, and environment in which the relationship took place) when the doctor's grade was controlled for (significant F change < 0.001). 93 (83%) of the doctors in this study would have needed only four raters to achieve a reliable score if the intent was to determine if they were satisfactory. The mean time taken to complete the questionnaire by a rater was six minutes. Just over an hour of administrative time is needed for each doctor.
Conclusions SPRAT seems to be a valid way of assessing large numbers of doctors to support quality assurance procedures for training programmes. The feedback from SPRAT can also be used to inform personal development planning and focus quality improvements.
PMCID: PMC558096  PMID: 15883137
5.  Work based assessment 
BMJ : British Medical Journal  2003;326(7392):753-755.
PMCID: PMC1125657  PMID: 12676847
6.  The death of the long case? 
BMJ : British Medical Journal  2002;324(7334):408-409.
PMCID: PMC65539  PMID: 11850375
7.  Recertification in the United States 
BMJ : British Medical Journal  1999;319(7218):1183-1185.
PMCID: PMC1116962  PMID: 10541514

Results 1-7 (7)