|Home | About | Journals | Submit | Contact Us | Français|
Objectives. To evaluate pharmacy students' drug-drug interaction (DDI) knowledge retention over 1 year and to determine whether presenting DDI vignettes increased knowledge retention.
Methods. A knowledge assessment tool was distributed to fourth-year pharmacy students before and after completing a DDI educational session. The questionnaire was re-administered after 1 year to assess knowledge retention. During the intervening year, students had the option of presenting DDI case vignettes to preceptors and other health professionals as part of their advanced pharmacy practice experiences (APPEs).
Results. Thirty-four of 78 pharmacy students completed both the post-intervention and 1-year follow-up assessments. Students’ knowledge of 4 DDI pairs improved, knowledge of 3 DDI pairs did not change, and knowledge of the remainder of DDI pairs decreased. Average scores of the 18 students who completed all tests and presented at least 1 vignette during their APPEs were higher on the 1-year follow-up assessment than students who did not, suggesting greater DDI knowledge retention (p = 0.04).
Conclusion. Although pharmacy students’ overall DDI knowledge decreased in the year following an educational session, those who presented vignettes to health professionals retained more DDI knowledge, particularly on those DDIs for which they gave presentations. Other methods to enhance pharmacy students’ retention of DDI knowledge of clinically important DDIs are needed.
Estimates of the prevalence of drug-drug interactions (DDIs) and adverse events resulting from DDIs vary widely.1-7 Co-administration of drugs that may interact appears to be relatively common, with only a small percentage of interactions resulting in adverse clinical consequences.8-11 However, harmful outcomes related to DDIs have been reported, with some cases resulting in hospitalization and even death.12-16
Even with drug dispensing safety measures such as computerized DDI checking, a pharmacist's knowledge of DDIs can facilitate appropriate clinical decision-making for patients through measures such as evaluation of risk and mitigating factors, and selection of non-interacting therapeutic alternatives. Unfortunately, pharmacists’ knowledge of DDIs and the reliability of their computer systems to detect DDIs are limited.17-21
Few assessments of healthcare providers’ DDI knowledge, their retention of information over time after being educated on DDI topics, and the amount of time they spend learning about DDIs have been reported in the literature. Understanding what pharmacists are taught regarding DDIs and how much information is retained could be useful in identifying strategies for increasing pharmacists’ knowledge about DDIs. Pharmacy students who completed an elective course to improve knowledge and skills regarding DDIs were significantly more confident in their abilities to identify and assess drug interaction information and scored significantly higher in all areas of DDI knowledge than students who did not take the course.22 Saverno and colleagues examined third- and fourth-year pharmacy students’ abilities to identify DDIs on 2 occasions and found that students could correctly identify 52% to 66% interacting drugs pairs, respectively.23 Another study that assessed DDI knowledge across healthcare professional students (ie, pharmacy, medical, and nurse practitioners) found that, while pharmacy students were more knowledgeable than medical (p < 0.001) and nurse practitioner (p < 0.0001) students prior to an educational session, the pharmacy students only were able to correctly identify 67% of DDIs on a test given before the session.24 These studies emphasize the importance of increasing DDI knowledge retention among healthcare professional students.
The main purpose of this study was to assess fourth-year doctor of pharmacy (PharmD) students’ DDI knowledge retention over 1 year. A secondary purpose was to determine whether students who presented DDI case vignettes on specific drug pairs to other students had increased knowledge retention of those DDIs.
The overall study design was a repeated measures assessment of DDI knowledge among advanced pharmacy students. Third-year students were invited to participate in an evaluation of their DDI knowledge immediately before and after a DDI educational session during their final classroom lecture-based year of study in the University of Arizona PharmD program. The learning objectives for the educational session included: (1) defining DDIs; (2) listing and identifying 11 clinically important DDIs from case scenarios; (3) explaining the difference between pharmacodynamic and pharmacokinetic mechanisms of DDIs; (4) describing management strategies for 11 clinically important DDIs; and (5) listing DDI information resources. At the time of the session, assessment of knowledge was conducted immediately prior to and after a 1½-hour lecture with case discussions. The purpose of the educational session was not only to train students in these 11 clinically significant DDIs, but also to teach general mechanisms and management of interactions so that students could apply this knowledge to all DDIs. Results from the pre- and post-intervention assessment are reported in detail in a separate publication.25
All students were given the option to present short DDI vignettes to health care professionals anytime during their fourth-year advanced pharmacy practice experiences (APPEs) as a means to reinforce DDI knowledge. If they chose to do so, they could create and present 3 DDI vignettes in place of 1 of their required drug information assignments during any APPE. Because the materials necessary to create the vignettes were provided to the students, making them easier to complete than the drug information assignments, the instructors expected several students to choose this option and present DDI vignettes during their fourth year. The clinical content of the vignette presentations included methods to identify DDIs, mechanisms that cause the interaction, the resulting consequences, potential therapeutic alternatives that could be used to avoid the DDI, or monitoring parameters for when concurrent use of interacting drugs is deemed appropriate. Each case vignette lasted approximately 5 minutes. After each presentation, students reported the following information to the researchers via a post card: (1) date of the presentation; (2) number and type of health care providers who attended the vignette (ie, nurse practitioners, pharmacists, physicians, and other healthcare professionals); (3) practice setting in which the case was presented (eg, managed care, hospital pharmacy, community pharmacy); (4) city in which the clerkship site was located; (5) DDI case that was presented and why it was selected; and (6) whether they would present the case again and why.
After completing all of their APPEs, fourth-year students completed the same DDI knowledge assessment tool administered 1 year earlier, after completion of the DDI education session. For all 3 administrations of the DDI knowledge assessment, students completed the questionnaire without using reference materials.
The DDI management strategy response categories that the students chose from included: (1) avoid combination; (2) usually avoid combination; (3) take precautions; (4) no special precautions; and (5) not sure. The “not sure” option was included to prevent guessing. Warholak and colleagues conducted a study that assessed the use of this instrument using Rasch analysis and found that it demonstrated validity and reliability in this population.26
The DDI questionnaire consisted of 15 drug pairs. Among the 15, one “avoid combination” pair of drugs was included for which concurrent use was contraindicated (ie, risk of the combination outweighs the benefits of treatment); 5 “usually avoid combination” pairs were included for which the drugs should usually not be used together except in special circumstances; 5 “take precautions” pairs of drugs for which concurrent use could be managed by increased monitoring or dose adjustment were provided; and 4 “no special precautions” pairs were included for which the drugs most likely did not interact and therefore the risk of having an adverse event was small.
Two different outcome variables were used to evaluate the main study outcomes. The first outcome was change in DDI recognition knowledge, where students were awarded credit if they correctly identified a potential DDI regardless of the level of precaution that is required. The second outcome was change in DDI management strategy knowledge, where students had to select the correct management strategy (ie, avoid, usually avoid, take precautions, or no precautions necessary) to be awarded credit. The same DDI pairs that were discussed in the educational session were included in the pre-intervention, post-intervention, and 1-year follow-up assessments. Demographic information on age, gender, and percentage of students with previous health-related degrees also were collected.
An analysis was conducted to compare those students who completed both the post-intervention and one-year follow-up assessment to those who only completed the post-intervention assessment. The investigators hypothesized that students who completed the entire process of the study would have higher overall 1-year follow-up assessment scores. Wilcoxon rank-sum test was used to determine whether significant differences existed among the sum of the students’ post-intervention test scores along with whether the age of the student impacted their post-intervention test scores. Chi-square analyses were used to determine whether gender and having a previous health-related degree impacted post-intervention test results.
Multiple-linear regression models27 were used to assess whether vignette presentations given over the course of the APPEs affected DDI recognition or DDI management strategy knowledge. The presentation of any case vignettes (ie, at least 1 case vignette) and the quantity (ie, amount of case vignettes presented varied) of case vignettes presented were both assessed for knowledge retention.
A secondary investigation was undertaken to determine whether students who provided any DDI vignette presentations were better at correctly recognizing interactions (the DDI recognition scoring strategy) or indicating the proper management of DDIs (the DDI management scoring strategy) on the 1-year post-intervention test compared to students who did not complete any case presentations. Chi-square analysis was conducted to determine whether significant differences existed between those who presented any cases versus those who presented no cases.
The investigators hypothesized that students who presented any DDI cases would be better able to recognize clinically significant DDIs and identify the appropriate management strategy and that students who presented a specific DDI would be better able to recognize and identify the appropriate management strategy for that DDI when taking the final follow-up assessment 1 year after the DDI training session.
An alpha (α) level of 0.05 was used for significance with 95% confidence intervals reported for all point estimates. Bonferroni corrections were used where appropriate. All statistics were conducted in Intercooled STATA, version 11.0 (StataCorp, College Station, TX). Absolute differences were measured among baseline test/post-intervention test; post-intervention test/1-year follow-up test; and baseline test/1-year follow-up test. Frequencies and percentages were calculated for the case presentation attendees, setting, and specific DDI selected for the vignette.
Sixty-three (81%) of the 78 students in the class completed the DDI post-intervention test that followed the educational session at the end of the third year. Thirty-four (44%) students completed both the post-intervention test and the 1-year follow-up assessment and were included in the analysis. Of these 34 students, 27 provided demographic information. The average age was 25 ± 3 years; 22 (81%) were female, and 2 (7%) held a previous health-related degree. Of the 29 who did not participate in the 1-year follow-up assessment, 20 provided demographic information. The average age of nonparticipants in the follow-up assessment was 29 ± 6 years; 11 (55%) were female, and 2 (10%) held a previous health-related degree.
Those individuals who completed the 1-year follow-up assessment (n = 34) had significantly higher DDI recognition (p = 0.021) and higher management strategy scores (p = 0.003) on the post-intervention assessment compared to those who only completed the post-intervention assessment (n = 29). Younger participants were almost 3 times more likely (p = 0.004) to complete both assessments, and female participants were 4 times more likely (p = 0.050) to complete both assessments. No significant differences existed among those who held a previous health-related degree (p = 0.75).
Tables 1 and and22 present DDI recognition and management strategy scores, respectively, on the baseline, post, and 1-year follow-up assessments. The drug pairs that received the most correct responses for DDI recognition on the post-intervention test and 1-year follow-up assessment were acetaminophen/codeine plus amoxicillin (a non-interacting combination) and nitroglycerine plus sildenafil (100% of students answered correctly for both pairs). The pair least likely to be answered correctly on the post-intervention test for DDI recognition was the pair warfarin plus digoxin (79% answered correctly; this pair has no clinically significant interaction). For the 1-year follow-up assessment, the drug pair for which students were least likely to give the correct answer was the interacting pair, warfarin plus gemfibrozil (71% answered correctly).
With respect to correctly identifying management strategies, those drug pairs that were correctly identified the least were simvastatin plus itraconazole and warfarin plus amiodarone (68% of students answered correctly) on the post-intervention test, and warfarin plus gemfibrozil (an interacting pair) was least likely to be selected correctly (32% answered correctly) on the 1-year follow-up assessment.
Results in Tables 1 and and22 report student scores at baseline, post-intervention test, and 1-year follow-up test; Table 1 includes data on DDI recognition scores and Table 2 describes management strategy scores. When DDI recognition at 1 year was compared to the post-intervention test results (the assessment given immediately after the training program), students’ scores increased on 4 of the DDI pairs, scores for 3 others did not change, and students’ scores on the remainder of DDI pairs decreased (-3% to -26% difference in percent of students answering correctly). Using the same comparisons for the management strategy scoring, no student scores improved on any DDI pairs, 2 remained the same, and the remainder of student scores decreased (-9% to -59% difference). When scores on the 1-year assessment were compared to those from the baseline assessment, however, there was a net improvement in DDI recognition for all pairs (+3% to +79% increase) and a net improvement in identification of the correct management strategy for all but one interacting pair (-9% to +35% change).
Table 3 displays the DDI knowledge of students who presented at least 1 case presentation versus those who did not present any. While the group who gave at least 1 presentation generally scored higher on recognition of the DDI pairs (7 higher, 3 the same, 1 lower), the differences were generally small. No significant differences for DDI recognition scores were noted between the groups. The presentation group scored higher on 8 DDI pairs, the same on 1, and lower on 2 for management strategy. Students who presented case vignettes scored significantly higher on 1 interaction (simvastatin plus itraconazole) (p = 0.021) on the management strategy portion of the assessment. Students presenting no case vignettes scored significantly higher for 1 DDI pair (warfarin plus naproxen) (p = 0.045). Due to the small sample size of these groups (n = 18 for those who presented case vignettes and n = 16 for those who did not present case vignettes), results should be interpreted with caution.
Of the 34 pharmacy students who completed both the post-intervention test and 1-year follow-up assessment, 18 students completed case vignettes. Results from the linear regression analysis showed a positive and significant relationship between overall (ie, aggregated) DDI recognition scores on the 1-year assessment and completing at least 1 DDI case presentation (p = 0.043) and for the number of presentations and DDI recognition score (p = 0.049). No similar significant relationships existed between scores on the management strategy portion of the assessment.
Information is shown in Table 4 for the 18 students who presented 1 or more vignettes. Pharmacists were the most common attendee at these presentations (n = 119), followed by “other health care professionals” (n = 55), physicians (n = 18), and nurse practitioners (n = 4). Warfarin was the object drug (the drug being affected by the interaction) in 89 of the 141 (63%) DDI cases presented. The majority (42%) of students stated that the reason they chose the specific DDI for their case presentation was because of the medication regimen of a particular patient at their APPE site. Students indicated that they would be willing to present their particular vignette again 99% of the time.
While student scores on the 1-year post-training assessment were less than optimal, they remained better than baseline assessment scores. Students’ DDI recognition scores improved from the post-intervention assessment to the 1-year follow-up assessment on 4 out of the 15 drug pairs, including the interacting combination digoxin plus amiodarone, and the non-interacting combinations digoxin plus sildenafil, metformin plus erythromycin, and warfarin plus digoxin. There were no improved management strategy scores between post-intervention assessment and 1-year follow-up assessment; however, 2 drug pairs, acetaminophen/codeine plus amoxicillin (non-interacting) and nitroglycerine plus sildenafil (interacting), were correctly identified by 100% of the students. These results would be expected for any type of education delivered, unless the learning is reinforced during the interim period.
Completion of the 1-year follow-up assessment was optional. Those students who chose to complete the 1-year follow-up assessment had higher overall post-intervention test scores than students who only completed the post-intervention test. As these assessments were administered in class, a potential reason for the lower completion rate may have been low attendance for that specific class period. In addition, those students who took the time to complete the 1-year follow-up assessment may have been more motivated to learn about clinically significant DDIs in general.
There are many factors that could impact DDI knowledge over time. Retention of information after a single lecture is limited, and thus, it is unreasonable to expect perfect scores on a posttest after a single educational event, particularly for topic-naive students. In addition, threats to internal validity (eg, history) had the potential to bias this study. There was a 1-year time period between the posttest and 1-year follow-up assessment, and the longer the period of time between observation 1 and observation 2, the greater the chance of extraneous variables affecting subjects. Miller and colleagues conducted a study on medical students’ knowledge retention over time and concluded that the improvement in scores on the posttest compared to the pretest was the result of students increasing their knowledge through the process of learning and retaining information during the time between test administrations and not the result of administering the same test.28
Better scores on recognition than management strategy are to be expected, given that the ability to recognize that an interaction exists (a yes or no response) is considerably easier than to recall the specifics of a management strategy with 4 options from which to choose. These pharmacy students had reasonably good recognition scores (Table 1) at the time of the pretest (all but 3 DDIs were identified correctly more than 50% of the time). This can be explained, at least in part, by the fact that many had worked in pharmacies as interns or technicians where they would be expected to have some exposure to DDI alerts, and by the fact that DDIs are discussed throughout the pharmacy curriculum. They did not perform as well on items testing management strategies (Table 2), for which only 3 DDI pairs were identified with the correct strategy more than 50% of the time. Because the management strategy scores were lower to begin with, the impact of the lecture on these scores at posttest was more dramatic (scores on 8 DDI management strategy items improved more than 50% while this level of improvement was seen on only 2 DDI recognition items).
Presenting case vignettes appeared to significantly enhance students’ knowledge retention. Students who gave at least 1 case vignette had significantly higher DDI recognition scores on their 1-year follow-up assessment (p = 0.041) compared to students who gave no case presentations. The quantity of case vignettes given by students also yielded significantly higher DDI recognition score (p = 0.049). When all case vignettes were analyzed in aggregate, no significant relationship existed for management strategy scores and case vignettes. However, when separated on a vignette basis, students who made presentations on specific DDI cases were better able to both recognize those specific DDIs and select the correct management strategy. Due to the small sample size (n = 18), these results should be interpreted with caution. Thus, the theory that learning and then teaching it to others solidifies knowledge appears to have been reinforced in this study. Presenting any vignettes had less impact on being able to identify all of the DDI pairs correctly. It is unclear why the performance on the 1-year follow-up assessment was not better. Students were not informed after completion of the post-intervention test that their DDI knowledge would be tested in the future, so students may have viewed the vignettes simply as a way to avoid writing a drug information response (which is time consuming) instead of seeing it as an opportunity to increase their DDI knowledge. Further compounding these issues is that the case vignettes were optional. Students who chose to present a vignette may have been more interested in DDIs, which may have resulted in selection bias. In addition, students were allowed to select any DDIs for their case vignettes. They could present the same vignette multiple times, resulting in less variety and reinforcement of knowledge of the same DDI pairs.
Improving the ability of pharmacy students to select appropriate strategies for managing potential DDIs is essential. Students should develop an overall approach to assessing risk, selecting non-interacting therapeutic alternatives, adjusting dosages, developing monitoring and follow-up plans, and communicating recommendations to prescribers and patients. Simply recognizing potential DDIs does not reduce risk and, therefore, methods to improve clinical management are needed. Multiple-choice questions regarding general management strategies may not be the best approach to test such knowledge. Future research should consider addressing this important aspect of clinical decision-making.
A potentially confounding factor in determining the impact of the DDI educational session and presenting vignettes on knowledge retention is that many pharmacy students worked in pharmacies while matriculating and thus may have been exposed routinely to DDI alerts. They also may have been exposed to these alerts during clerkships. Thus, students who had this exposure would have been more likely to recognize pairs of drugs that can interact than students who were not similarly exposed. The level of exposure to DDI material also would likely impact scores before and after the lecture. Additionally, DDIs were discussed in many of the courses that the pharmacy students took. If they were exposed to, and perhaps tested on, some of the DDIs used in this study, there may have been a degree of knowledge retention that certainly could have impacted the pretest scores. To the investigators knowledge, this educational session was the only formal DDI training that pharmacy students received.
The ability to identify and manage potentially harmful drug interactions is a vital component of a pharmacist's work. This study provides insight into how well pharmacy students retain their DDI knowledge over a 1-year time period and whether presenting DDI case vignettes to other health care professionals helped students with knowledge retention. While students’ scores significantly decreased between the post-educational intervention test and the 1-year follow-up assessment, knowledge appeared to be higher 1 year after the educational session than before. Overall, using case vignettes to reinforce knowledge retention appeared to modestly improve DDI knowledge. These findings suggest the need for improvement in DDI education in pharmacy curricula, in addition to methods to assist pharmacy students in retaining DDI knowledge outside the classroom.