As part of a required, 12-station comprehensive clinical performance exam, third-year medical students (n=93) completed a 15-minute OSCE where they were expected to take a focused history and provide appropriate counseling. OSCEs typically use standardized patients (SPs) trained to follow a predefined “script” in a simulated context (16
). Performance within this controlled setting can be reliably measured via trained observers, typically the SPs themselves using a behavioral checklist.
In this OSCE, a 50-year old Caucasian female (Temp 98.8° F, HR 86, RR 16, BP 110/60, Weight 130 lbs., Height 5’6”) presents to the primary care clinic complaining of feeling stressed and anxious. No references to CAM or self-care are made. After the student has completed the history-taking portion of the station, the SP is instructed to ask, without solicitation, about the stress-reducing effectiveness of l-theanine, a naturally-occurring amino acid commonly found in green tea and available as a dietary supplement in extract form. Simultaneously, the SP produces one of two forms of information regarding l-theanine: testimonials from two commercial websites (Condition A, shown in ), or two PubMed research abstracts (Condition B, shown in ). [Maintained by the U.S. National Library of Medicine, PubMed is a searchable, publicly-accessible database that includes over 16 million citations from Medline® and other sources.] Source materials were of approximately equal length, contained highlighted excerpts, and described generally encouraging results from use of the supplements. Neither type of information provided results of a scientifically valid study of l-theanine regarding decreasing stress/anxiety in a clinical population.
Condition A: Commercial Information
Immediately following each patient encounter, the SP completed a checklist for 36 observed history taking and counseling behaviors. In addition to asking general history-related items, examinees’ counseling performance was based on their abilities to: 1) acknowledge a limited familiarity with l-theanine, 2) defer judgment on safety/efficacy of l-theanine until a later time, 3) respect patients’ interest in l-theanine as a treatment for stress/anxiety, and 4) schedule a follow-up visit to discuss the use of l-theanine. Lastly, using a 5-point Likert-type scale, the SP completed a 10-item “global” ratings scale assessing the quality of students’ communication and interpersonal skills. Copies of ratings scales are available from the authors. Prior to the exam, all SPs underwent routine reliability testing to ensure their familiarity with the case and their accuracy in observing and recording key behaviors.
Following each patient encounter, student were given five minutes to rate, on a 1–10 scale: the credibility of the evidence presented (i.e., “How would you rate the evidence on l-theanine for stress/anxiety presented by this patient?”), and the proportional benefit (real versus placebo) of l-theanine to relieve stress/anxiety. Since students’ schedules for the larger CPX exam were pre-established – and the ordering of exam stations was not mutable – randomization of the script (version) occurred at the level of the SP. However, although SPs were instructed to randomly alter the version of the case presented, they were also encouraged to do so for blocks of 2–3 students in order to minimize examiner error. As a result, strict formal randomization did not occur, even though the desired results were achieved.
A chi-square test for two independent samples was used to examine differences in nominal attributes between treatment conditions. Where distributional assumptions were met, the Student’s t test for independent samples was used to examine mean differences. As a non-parametric alternative to the t test, the Mann-Whitney U test used to examine median (ranked) differences. All tests were one-tailed, with alpha set at p = < .05. Power analysis for the subjective ratings by SPs revealed a 95% power to detect a 2 point difference between groups using a one tailed test with alpha =.05. This study protocol was approved by the medical center institutional review board (IRB).