The discussions during the monthly Journal Club began with an obvious focus derived from the content of the Pre-test. It was expected that the Pre-test would help to identify participants who had not read or understood the impact of the articles and help to focus the discussion. The Pre-test also allowed the department to begin to track, in a sequential fashion, objective measures of performance for individual residents. The Post-test represented the group educational dynamic.
Identifying residents with problems on the Pre-test or a lack of response on the Post-test was obvious and allowed timely information to be forwarded to their academic advisors for remediation. As a bonus, tracking attendance via a turned in quiz was facilitated and this information also was forwarded to the residents academic advisor.
When critical appraisal skills among resident physicians have been formally assessed, correct responses have varied between 33 and 42 % suggesting limited integration of these concepts [8
]. This was confirmed at our institution with an overall average of 32% for incoming PGY1 residents over the preceding three years. Even after specific targeted interventions, correct responses have increased only to 67 % [9
]. The attainment of a Pre-test median score of 79.0 % at the end of the first year of our intervention was certainly comparable and significantly exceeded those results [9
Studies have also indicated that there is no correlation between self-assessed competence and actual ability in these skills [10
]. With the Pre-test there is direct feedback regarding comprehension and competence. This therefore gives us a quantitative assessment of the individual residents facility with these critical concepts, as they evolve over time.
There also appears to be little evidence that didactic Continuing Medical Education can yield any effect on performance [11
]. According to these authors the emphasis of successful educational programs is based upon a pre-course assessment of needs, opportunities to practice relevant skills and after course activities that reinforce and facilitate change. Our Pre-test assesses needs, the discussion offers opportunities to practice skills and our Post-test reinforces and facilitates change [11