The OSCE comprised a circuit of thirteen stations, which involved completion of a number of tasks such as examination of a system, eliciting a focussed history, counselling or communicating a problem, performing a procedure and problem-solving oriented around patient and laboratory data, and photographic material (Figure ). The areas assessed included cardiovascular, respiratory, abdomen, neurological, developmental, dysmorphism and nutrition. This assessment format allowed the controlled exposure of students to a wide variety of paediatric clinical skills within a relatively short time period. Each station was 7 minutes duration with the exception of the 14-minute history-taking station. One minute was given between stations to facilitate change and the reading of instructions. With the inclusion of strategically placed rest stations, to reduce student and patient fatigue, all students completed the circuit over a 2-hour period.
A standardised technique of marking was used and student performance was assessed by criterion reference for each station. Criterion-based scoring was used, with each checklist item scored as 0 (omitted, incorrect or inadequate), or 1–2 (correct or adequate).
Face and content validity of each checklist was established by review and consensus by a core group of senior paediatricians. Stations were first selected to represent the curricular goals and objectives and to reflect authentic clinical situations. Checklists were designed to include the features thought to be most important by the development committee. Through discussions, consensus was achieved on the checklist items and structure.
The study was conducted during the period July 2001 to December 2002. Five groups of students participated in the process, during their respective clerkship rotations. Student groups had at least two briefing sessions before the OSCE, and included an orientation about the examination process (both end-of-clerkship and final MB) and a review of commonly assessed competences. They were also apprised of the valuable contribution they could make towards improving the assessment and encouraged to participate in the evaluation.
A cross-sectional survey using a 32-item self-administered questionnaire was completed at the end of each OSCE [9
]. Students were asked to evaluate the content, structure, and organization of the OSCE, rate the quality of performance and objectivity of the OSCE process, and to give their opinion about the usefulness of the OSCE as an assessment instrument compared to other forms which they had experienced (essays, multiple choice questions, long and short cases, general clerkship rating).
Participation was on a voluntary basis and students were assured that those who declined involvement in the survey would not be penalised. The Curricular Affairs Section handled the administration and analysis of the questionnaires. Ethical approval was received from the University Hospital of the West Indies/University of the West Indies Faculty of Medical Sciences Ethics Committee. Following completion of the questionnaire, an OSCE review session was conducted with the students for feedback and teaching purposes, at the end of the clerkship. Students were given the opportunity to review their individual performances at the respective stations. Examiner evaluations were also used in the feedback process.
Data were collated and descriptive and non-parametric tests applied using Stata version7 [10
]. Basic statistical analysis of the Likert items was conducted by calculating frequencies, means and standard deviations. Qualitative analysis was done through a form of content analysis by identifying themes in student responses and grouping responses according to thematic content. Two of the authors individually conducted this content analysis and identified themes and final grouping of responses were developed by consensus.