PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of brjclinpharmLink to Publisher's site
 
Br J Clin Pharmacol. 2016 May; 81(5): 1000–1001.
Published online 2016 February 17. doi:  10.1111/bcp.12862
PMCID: PMC4834598

Evaluating pharmacotherapy education: urgent need for hard outcomes

Recently, a call was made for more high‐quality articles on clinical pharmacology and therapeutics (CPT) education 1. Not only is there a lack of research in this field, but also the quality of the available literature is disturbing. Most of the evaluation methods currently used in undergraduate CPT education rely too heavily on ‘soft’ endpoints, such as satisfaction with the education received, and on intermediate endpoints, such as increase in knowledge and skills instead of improvement of patient care. Moreover, measures of knowledge and skills are often not valid, because self‐assessment does not reflect actual knowledge and skills. Both issues are discussed in this letter.

First, the lack of ‘hard endpoints’ is a problem. In educational research, it is common to classify endpoints on the basis of the four levels of evaluation described in Kirkpatrick's model of impact of education (modified by Freeth) 2, 3 (Table 1). Unfortunately, studies of CPT education with hard endpoints (levels 3 & 4) are rare. For example, in four systematic reviews of undergraduate CPT education 4, 5, 6, 7, only 2 of the 63 included articles (3%) used level 3 or 4 endpoints to evaluate their educational interventions (Table 1). In fact, most of the included articles (47; 75%) used level 2 endpoints, so‐called intermediate endpoints. Improvement in intermediate endpoints (Kirkpatrick level 2) does not necessarily translate into improvement in patient care (Kirkpatrick level 4). Thus it is difficult to interpret the results of level 2 studies in terms of their value for CPT education.

Table 1
Included articles (n = 63) classified by Kirkpatrick's modified model of evaluation

Secondly, the validity of the (soft) endpoints used in educational research is sometimes dubious. A recent study showed that self‐reported confidence in prescribing skills is poorly correlated with assessed competence 8. A logical consequence of this finding is that self‐reported confidence should not be used as the primary measure of students' prescribing skills at the undergraduate level. This prompted us to question whether the same is true for pharmacology and pharmacotherapy knowledge. Is self‐evaluated knowledge also poorly correlated with objectively assessed knowledge? To evaluate this hypothesis, we reanalysed data obtained from bachelor and masters students in an earlier study by Keijsers et al. 9. In this study, students' knowledge of basic pharmacology and clinical pharmacology was assessed using a written multiple choice assessment, and students rated their confidence in applying pharmacology and pharmacotherapy knowledge in clinical practice. Correlations between self‐reported confidence and assessed basic or applied knowledge were absent or weak, with Pearson's correlations ranging from r = 0.07 to r = 0.25 (P < 0.05). This mismatch between confidence scores and knowledge scores has also been reported for residents 10.

A commonly used approach to bridge the gap between knowledge/skills assessments and clinical practice is simulation‐based education (Kirkpatrick level 3). While simulation has been shown to improve important learning outcomes, its contribution to clinical outcomes remains unclear 6. Moreover, simulation does not adequately reflect the more demanding conditions of everyday practice, such as time pressure, a high workload, and a hierarchical culture, and for this reason we think that a better approach to evidence‐based CPT education is to involve students in prescribing for real patients, such as in student‐run clinics 11, 12, student participation in medication safety programmes, or on clinical rotations 13, 14 (Kirkpatrick levels 3 & 4). Although the implementation of these ‘real context’ training programmes requires organizational effort, they are feasible and of educational value 12. More importantly, these programmes provide new opportunities for ‘hard’ outcome studies, and there is an urgent need for such studies in CPT educational research, because evidence‐based educational programmes can be designed only if there is enough supporting high‐quality research available.

Competing Interests

All authors have completed the Unified Competing Interest form at http://www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author) and declare: no support from any organization for the submitted work, no financial relationships with any organizations that might have an interest in the submitted work in the previous 3 years and no other relationships or activities that could appear to have influenced the submitted work.

Notes

Brinkman D. J., Keijsers C. J. P. W., Tichelaar J., Richir M. C., and van Agtmael M. A. (2016) Evaluating pharmacotherapy education: urgent need for hard outcomes. Br J Clin Pharmacol, 81: 1000–1001. doi: 10.1111/bcp.12862.

References

1. Keijsers CJ, Ross S. A pharmacological approach to education. Br J Clin Pharmacol 2015; 80: 329–30. [PubMed]
2. Steinert Y, Mann K, Centeno A, Dolmans D, Spencer J, Gelula M, Prideaux D. A systematic review of faculty development initiatives designed to improve teaching effectiveness in medical education: BEME Guide No. 8. Med Teach 2006; 28: 497–526. [PubMed]
3. Freeth D, Hammick M, Koppel I, Reeves S, Barr H. A Critical Review of Evaluations of Interprofessional Education. Higher Education Academy Learning and Teaching Support Network. Available at http://repos.hsap.kcl.ac.uk/content/m10123/latest/occp2.pdf (last accessed 25 August 2015).
4. Keijsers CJ, van Hensbergen L, Jacobs L, Brouwers JR, de Wildt DJ, ten Cate OT, Jansen PA. Geriatric pharmacology and pharmacotherapy education for health professionals and students: a systematic review. Br J Clin Pharmacol 2012; 74: 762–73. [PubMed]
5. Ross S, Loke YK. Do educational interventions improve prescribing by medical students and junior doctors? A systematic review. Br J Clin Pharmacol 2009; 67: 662–70. [PubMed]
6. Aura SM, Sormunen MS, Jordan SE, Tossavainen KA, Turunen HE. Learning outcomes associated with patient simulation method in pharmacotherapy education: an integrative review. Simul Healthc 2015; 10: 170–7. [PubMed]
7. Kamarudin G, Penm J, Chaar B, Moles R. Educational interventions to improve prescribing competence: a systematic review. BMJ Open 2013; 3: e003291. [PMC free article] [PubMed]
8. Brinkman DJ, Tichelaar J, Agtmael MA, de Vries TP, Richir MC. Self‐reported confidence in prescribing skills correlates poorly with assessed competence in fourth‐year medical students. J Clin Pharmacol 2015; 55: 825–30. [PubMed]
9. Keijsers CJ, Segers WS, de Wildt DJ, Brouwers JR, Keijsers L, Jansen PA. Implementation of the WHO‐6‐step in the medical curriculum to improve pharmacology knowledge and pharmacotherapy skills. Br J Clin Pharmacol 2014; 79: 897–906. [PMC free article] [PubMed]
10. Ramaswamy R, Maio V, Diamond JJ, Talati AR, Hartmann CW, Arenson C, Roehl B. Potentially inappropriate prescribing in elderly: assessing doctor knowledge, confidence and barriers. J Eval Clin Pract 2011; 17: 1153–9. [PubMed]
11. Schutte T, Tichelaar J, Dekker RS, van Agtmael MA, de Vries TP, Richir MC. Learning in student‐run clinics: a systematic review. Med Educ 2015; 49: 249–63. [PubMed]
12. Dekker RS, Schutte T, Tichelaar J, Thijs A, van Agtmael MA, de Vries TP, Richir MC. A novel approach to teaching pharmacotherapeutics – feasibility of the learner‐centered student‐run clinic. Eur J Clin Pharmacol 2015; 71: 1381–7. [PubMed]
13. van Unen RJ, Tichelaar J, Schneider AJ, Geijteman EC, Nanayakkara PW, Thijs A, Richir MC, de Vries TP. The feasibility of incorporating structured therapeutic consultations with real patients into the clinical clerkship internal medicine. Naunyn Schmiedebergs Arch Pharmacol 2012; 385: 1111–6. [PubMed]
14. Brinkman DJ, Tichelaar J, van Agtmael MA, Schotsman R, de Vries TP, Richir MC. The prescribing performance and confidence of final‐year medical students. Clin Pharmacol Ther 2014; 96: 531–3. [PubMed]

Articles from British Journal of Clinical Pharmacology are provided here courtesy of British Pharmacological Society