While the hypothesis was supported by only the SPIR ratings, the results from Table suggested that this was partly due to the low power of small sample sizes to test the contrast between intervention participants and controls. It should be noted that the 12 participants who lost interest while on the waiting list were not included in the above analysis as intend-to-treat participants because their inclusion as no-change controls would inflate the contrast between the two groups.
Because this study employed three methods of evaluating medical students' communication skills, the results may contribute to the discussion regarding which evaluation methods are most productive. The fact that students rated themselves less positively after the experimental manipulation suggests that either TCom was detrimental to students' ability to therapeutically communicate or that students' self-evaluation of communication skills is an ineffective measure of actual performance. The second of these conclusions is supported by the fact that the actor and self perspectives did not correlate significantly (r = .15). However, Boud and Lublin [
22] state that becoming an accurate self-evaluator and developing the ability to monitor one's own learning process represents "one of the most important processes that can occur in undergraduate education", and this may be especially true for medical students who, once working as practicing physicians, will need to rely primarily on self-evaluation to monitor their performance. Therefore, training students to be accurate self-evaluators should be viewed as an important goal of medical school education, one that may be achieved partly through programs such as TCom. Given that Boud and Falchikov [
23] identify the general trend for high-achieving students to underestimate their abilities while low-achieving students overestimate theirs, the finding that students decreased their self-evaluations as a result of TCom could itself be indicative of meaningful learning in that the students' self-evaluations became more accurate. While students were not identified as either high-achieving or low-achieving upon entering the study, making it impossible to confirm whether the experimental manipulation caused low-achieving students to lower their initially-high ratings and high-achieving students to raise their initially low ratings, the fact that all participants were first-year medical students with the majority coming from a science background suggests that most would initially be classified as low-achieving in the domain of communication, and that an overall decline in the SAICQ occurs as their self-evaluations become more accurate.
One may ask, however, whether the decrease in students' self-evaluations could correspond to feelings of discouragement, as this could decrease the likelihood that they would pursue such learning opportunities in the future. An answer to this question may be inferred from the data provided by anonymous program evaluation forms completed by TCom students during the 2000/01, 2001/02 and 2002/03 academic years. Sixty of the 81 TCom students who participated in TCom (74%) completed and returned the questionnaires. While 21 of the 254 (8%) comments made by students involved their difficulty with managing patient encounters, all except one student (98%) felt that they benefited from their experience in TCom, 78% of them substantially. Fifty-five students (92%) indicated that the time expenditure was worthwhile and fifty (83%) said that they would recommend the program to other students. Specifically, fifty students (83%) reported improvement in the way they listen and talk to patients. There were no significant differences in student responses across the three years. Because students generally rated their experiences in TCom very favourably, it is unlikely that the decrease in their self-evaluations was based on feelings of discouragement and more likely that this indicates an improvement in their accuracy as self-evaluators.
The finding that self-perceptions did not correlate significantly with actor perceptions (r = .15) also indicates the need for medical students to get intensive supervision of their clinical work early in their medical training. TCom students generally rated the supervised group format very positively. Fifty-four (92%) of the 59 students who evaluated their TCom supervisors rated them as, overall, very good to excellent. Students rated their supervisors as excellent in the areas of organization, providing a positive learning environment, and providing helpful direction and feedback.
While the students generally rated their supervisors very positively, fifty-four respondents (92%) expressed the wish for more guidance from them. However, despite this feedback, the program coordinator and supervisors elected to continue offering minimal advice, considering it essential that students discover their own interactional style and learn to give and receive collegial support and feedback. Indeed, students often reported that they became more self-aware and open with fellow students, and described the supervision group as a safe venue for discussing challenging situations and for learning from each other.
While the study's final assessment protocol was administered approximately eight months after baseline, one might still question whether the medical students' communication skills learning was maintained over a greater length of time. While the assessment of longer-term reinforcement was beyond the study's scope, other research regarding the long-term reinforcement of communication skills has suggested that communication skills may be retained over longer periods of time. Bowman et al. [
24] assessed the psychiatric interview skills of physicians 18 months after they attended a problem-based interviewing course and reported that "not only were acquired skills maintained but further changes took place during the follow-up period, change that can be seen as improvement in terms of the course model." Furthermore, a study by Maguire, Fairbairn and Fletcher [
25] found lasting improvement in communication skills into residency. Therefore, while our study does not assess the retention of communication skills over a period longer than four months, it is hoped that any learning afforded by TCom will benefit the program's medical students throughout the rest of their education and as practicing physicians.
It is also possible that, while the SPIR ratings confirmed the hypothesis, the SAICQ and ISRS failed to confirm the hypothesis specifically because they were unable to identify the factors in which students may have actually improved, such as their ability to form, develop and maintain therapeutic relationships over time. It is possible that the structure of TCom, where training occurs over several months, is ideal for the development of such skills, and that a different measure would have to be used in order to determine whether TCom is successful in attaining its goals. Another possible interfering factor might be an overestimation of student skills by the standardized patients, reflecting a wish to see the students in a positive light (i.e., to give them "the benefit of the doubt").