|Home | About | Journals | Submit | Contact Us | Français|
Recently, a call was made for more high‐quality articles on clinical pharmacology and therapeutics (CPT) education 1. Not only is there a lack of research in this field, but also the quality of the available literature is disturbing. Most of the evaluation methods currently used in undergraduate CPT education rely too heavily on ‘soft’ endpoints, such as satisfaction with the education received, and on intermediate endpoints, such as increase in knowledge and skills instead of improvement of patient care. Moreover, measures of knowledge and skills are often not valid, because self‐assessment does not reflect actual knowledge and skills. Both issues are discussed in this letter.
First, the lack of ‘hard endpoints’ is a problem. In educational research, it is common to classify endpoints on the basis of the four levels of evaluation described in Kirkpatrick's model of impact of education (modified by Freeth) 2, 3 (Table 1). Unfortunately, studies of CPT education with hard endpoints (levels 3 & 4) are rare. For example, in four systematic reviews of undergraduate CPT education 4, 5, 6, 7, only 2 of the 63 included articles (3%) used level 3 or 4 endpoints to evaluate their educational interventions (Table 1). In fact, most of the included articles (47; 75%) used level 2 endpoints, so‐called intermediate endpoints. Improvement in intermediate endpoints (Kirkpatrick level 2) does not necessarily translate into improvement in patient care (Kirkpatrick level 4). Thus it is difficult to interpret the results of level 2 studies in terms of their value for CPT education.
Secondly, the validity of the (soft) endpoints used in educational research is sometimes dubious. A recent study showed that self‐reported confidence in prescribing skills is poorly correlated with assessed competence 8. A logical consequence of this finding is that self‐reported confidence should not be used as the primary measure of students' prescribing skills at the undergraduate level. This prompted us to question whether the same is true for pharmacology and pharmacotherapy knowledge. Is self‐evaluated knowledge also poorly correlated with objectively assessed knowledge? To evaluate this hypothesis, we reanalysed data obtained from bachelor and masters students in an earlier study by Keijsers et al. 9. In this study, students' knowledge of basic pharmacology and clinical pharmacology was assessed using a written multiple choice assessment, and students rated their confidence in applying pharmacology and pharmacotherapy knowledge in clinical practice. Correlations between self‐reported confidence and assessed basic or applied knowledge were absent or weak, with Pearson's correlations ranging from r = 0.07 to r = 0.25 (P < 0.05). This mismatch between confidence scores and knowledge scores has also been reported for residents 10.
A commonly used approach to bridge the gap between knowledge/skills assessments and clinical practice is simulation‐based education (Kirkpatrick level 3). While simulation has been shown to improve important learning outcomes, its contribution to clinical outcomes remains unclear 6. Moreover, simulation does not adequately reflect the more demanding conditions of everyday practice, such as time pressure, a high workload, and a hierarchical culture, and for this reason we think that a better approach to evidence‐based CPT education is to involve students in prescribing for real patients, such as in student‐run clinics 11, 12, student participation in medication safety programmes, or on clinical rotations 13, 14 (Kirkpatrick levels 3 & 4). Although the implementation of these ‘real context’ training programmes requires organizational effort, they are feasible and of educational value 12. More importantly, these programmes provide new opportunities for ‘hard’ outcome studies, and there is an urgent need for such studies in CPT educational research, because evidence‐based educational programmes can be designed only if there is enough supporting high‐quality research available.
All authors have completed the Unified Competing Interest form at http://www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author) and declare: no support from any organization for the submitted work, no financial relationships with any organizations that might have an interest in the submitted work in the previous 3 years and no other relationships or activities that could appear to have influenced the submitted work.
Brinkman D. J., Keijsers C. J. P. W., Tichelaar J., Richir M. C., and van Agtmael M. A. (2016) Evaluating pharmacotherapy education: urgent need for hard outcomes. Br J Clin Pharmacol, 81: 1000–1001. doi: 10.1111/bcp.12862.