PMCC PMCC

Search tips
Search criteria

Advanced
Results 1-5 (5)
 

Clipboard (0)
None

Select a Filter Below

Journals
Authors
more »
Year of Publication
Document Types
author:("Baig, luna A")
1.  Accuracy of portrayal by standardized patients: Results from four OSCE stations conducted for high stakes examinations 
BMC Medical Education  2014;14:97.
Background
The reliability in Objective Structured Clinical Exams (OSCEs) is based on variance introduced due to examiners, stations, items, standardized patients (SP), and the interaction of one or more of these items with the candidates. The impact of SPs on the reliability has not been well studied. Accordingly, the main purpose of the present study was to assess the accuracy of portrayal by standardized patients.
Methods
Four stations from a ten station high-stakes OSCE were selected for video recording. Due to the large number of candidates to be evaluated, the OSCE was administered using four assessment tracks. Four SPs were trained for each case (n = 16). Two physician assessors were trained to assess the accuracy of SP portrayal using a station-specific instrument based on the station guidelines. For the items with disagreement a third physician was asked to review and the mode was used for analysis. Each instrument included case-specific items on verbal and physical portrayal using a 3-point rating scale (“yes”, “yes, but” and “not done”). The physician assessors also scored each SP on their overall performance based on a 5-item anchored global rating scale (“very poor”, “poor”, “ok”, “good”, and “very good”). SPs at location 1 were trained by one trainer and SPs at location 2 had another trainer. All SPs were employed in a high-stakes OSCE for at least the second time.
Results
The reliability of rating scores ranged from Cronbach’s alpha of .40 to .74. Verbal portrayal by SPs did not significantly differ for most items; however, the facial expressions of the SPs differed significantly (p < .05). An emergency management station that depended heavily on SPs physical presentation and facial expressions differed between all four SPs trained for that station.
Conclusions
Variation of trained SP portrayal of the same station across different tracks and at different times in OSCE may contribute substantial error to OSCE assessments. The training of SPs should be strengthened and constantly monitored during the exam to ensure that the examinees’ scores are a true reflection of their competency and devoid of exam errors.
doi:10.1186/1472-6920-14-97
PMCID: PMC4035823  PMID: 24884744
OSCE; Portrayal of SPs; Errors of assessments
2.  Evaluation of off-service rotations at National Guard Health Affairs: Results from a perception survey of off-service residents 
Context:
“Off-service” clinical rotations are part of the necessary requirements for many residency training programs. Because these rotations are off-service, little attention is given to their structure and quality of training. This often leads to suboptimal educational experience for the residents on these rotations.
Aims:
The aim of this study was to assess medical residents’ perceptions, opinions, and levels of satisfaction with their “off-service” rotations at a major residency training site in Saudi Arabia. It was also to evaluate the reliability and validity of a questionnaire used for quality assurance in these rotations. Improved reliability and validity of this questionnaire may help to improve the educational experience of residents in their “off-service” rotations.
Materials and Methods:
A close-ended questionnaire was developed, Pilot tested and distributed to 110 off-service residents in training programs of different specializations at King Fahad Naitonal Guard Hospital and King Abdulziz Medical City, Riyadh, Saudi Arabia, between September 2011 and December 2011.
Results:
A total of 80 out of 110 residents completed and returned the questionnaire. Only 33% of these residents had a clear set of goals and educational learning objectives before the beginning of their off-service rotations to direct their training. Surgical specializations had low satisfaction mean scores of 57.2 (11.9) compared to emergency medicine, which had 70.7 (16.2), P value (0.03). The reliability of the questionnaire was Cronbach's alpha 0.57. The factor analysis yielded a 4-factor solution (educational environment, educational balance, educational goals and objectives, and learning ability); thus, accounting for 51% variance in the data.
Conclusion:
Our data suggest that there were significant weaknesses in the curriculum for off-service clinical rotations in KAMC and that residents were not completely satisfied with their training.
doi:10.4103/2230-8229.114773
PMCID: PMC3748647  PMID: 23983565
Knowledge attitudes and perception; needs assessment; off-service rotations; residents; validity of questionnaire
3.  Temporal stability of objective structured clinical exams: a longitudinal study employing item response theory 
BMC Medical Education  2012;12:121.
Background
The objective structure clinical examination (OSCE) has been used since the early 1970s for assessing clinical competence. There are very few studies that have examined the psychometric stability of the stations that are used repeatedly with different samples. The purpose of the present study was to assess the stability of objective structured clinical exams (OSCEs) employing the same stations used over time but with a different sample of candidates, SPs, and examiners.
Methods
At Time 1, 191 candidates and at Time 2 (one year apart), 236 candidates participated in a 10-station OSCE; 6 of the same stations were used in both years. Generalizability analyses (Ep2) were conducted. Employing item response analyses, test characteristic curves (TCC) were derived for each of the 6 stations for a 2-parameter model. The TCCs were compared across the two years, Time 1 and 2.
Results
The Ep2 of the OSCEs exceeded.70. Standardized thetas (θ) and discriminations were equivalent for the same station across the two year period indicating equivalent TCCs for a 2-parameter model.
Conclusion
The 6 OSCE stations used by the AIMG program over two years have adequate internal consistency reliability, stable generalizability (Ep2) and equivalent test characteristics. The process of assessment employed for IMG’s are stable OSCE stations that may be used several times over without compromising psychometric properties.
With careful security, high-stakes OSCEs may use the same stations that have high internal consistency and generalizability repeatedly as the psychometric properties are stable over several years with different samples of candidates.
doi:10.1186/1472-6920-12-121
PMCID: PMC3552978  PMID: 23216816
Stability of OSCEs; Latent trait analyses; Ep2; Internal consistency
4.  Problems and issues in implementing innovative curriculum in the developing countries: the Pakistani experience 
BMC Medical Education  2012;12:31.
Background
The Government of Pakistan identified 4 medical Colleges for introduction of COME, one from each province. Curriculum was prepared by the faculty of these colleges and launched in 2001 and despite concerted efforts could not be implemented. The purpose of this research was to identify the reasons for delay in implementation of the COME curriculum and to assess the understanding of the stakeholders about COME.
Methods
Mixed methods study design was used for data collection. In-depth interviews, mail-in survey questionnaire, and focus group discussions were held with the representatives of federal and provincial governments, Principals of medical colleges, faculty and students of the designated colleges. Rigor was ensured through independent coding and triangulation of data.
Results
The reasons for delay in implementation differed amongst the policy makers and faculty and included thematic issues at the institutional, programmatic and curricular level. Majority (92% of the faculty) felt that COME curriculum couldn’t be implemented without adequate infrastructure. The administrators were willing to provide financial assistance, political support and better coordination and felt that COME could improve the overall health system of the country whereas the faculty did not agree to it.
Conclusion
The paper discusses the reasons of delay based on findings and identifies the strategies for curriculum change in established institutions. The key issues identified in our study included frequent transfer of faculty of the designated colleges and perceived lack of:
· Continuation at the policy making level
· Communication between the stakeholders
· Effective leadership
doi:10.1186/1472-6920-12-31
PMCID: PMC3395573  PMID: 22591729
Community-oriented medical education; Curriculum development; Educational innovation; Change management; Evaluation of curriculum; Educational leadership
5.  Assessing clinical communication skills in physicians: are the skills context specific or generalizable 
Background
Communication skills are essential for physicians to practice Medicine. Evidence for the validity and domain specificity of communication skills in physicians is equivocal and requires further research. This research was conducted to adduce evidence for content and context specificity of communication skills and to assess the usefulness of a generic instrument for assessing communication skills in International Medical Graduates (IMGs).
Methods
A psychometric design was used for identifying the reliability and validity of the communication skills instruments used for high-stakes exams for IMG's. Data were collected from 39 IMGs (19 men – 48.7%; 20 women – 51.3%; Mean age = 41 years) assessed at 14 station OSCE and subsequently in supervised clinical practice with several instruments (patient surveys; ITERs; Mini-CEX).
Results
All the instruments had adequate reliability (Cronbach's alpha: .54 – .96). There were significant correlations (r range: 0.37 – 0.70, p < .05) of communication skills assessed by examiner with standardized patients, and of mini-CEX with patient surveys, and ITERs. The intra-item reliability across all cases for the 13 items was low (Cronbach's alpha: .20 – .56). The correlations of communication skills within method (e.g., OSCE or clinical practice) were significant but were non-significant between methods (e.g., OSCE and clinical practice).
Conclusion
The results provide evidence of context specificity of communication skills, as well as convergent and criterion-related validity of communication skills. Both in OSCEs and clinical practice, communication checklists need to be case specific, designed for content validity.
doi:10.1186/1472-6920-9-22
PMCID: PMC2687440  PMID: 19445685

Results 1-5 (5)