Three hundred forty-seven of 530 (66%) students across 5 universities responded to the EPIQ questionnaire. In 4 out of 5 universities, the majority (over 96%) of students responding to the questionnaire were in their second year of the PharmD program. Respondents’ work experience ranged, on average, from 2 to 4 years. Respondents’ mean age ranged from 26 to 29 years (depending on the university), and the majority of respondents were female (approximately 65%).
Reliability and validity of the questionnaire were determined via Rasch analysis. The requirements for demonstrating proper rating scale functions were met as follows: (1) the number of observations in each category were greater than 10; (2) the average category measures increased with the rating scale categories; (3) INFIT and OUTIFT MNSQ11,12
statistics for measured steps were within acceptable range; (4) category thresholds increased with the rating scale categories; (5) category thresholds were at least 1.4 logits apart; and (6) the shape of each rating scale distribution was peaked.15
The questionnaire met the above 6 qualifications indicating that this measurement tool possessed strong reliability and validity. The group means for student ability logit measures (ie, dependent student’s t
test) was significantly different from pretest to posttest (p
The hierarchical ordering of items 1-9 as it relates to students’ perceptions of their quality improvement knowledge are shown in . The right side of shows the item hierarchy, with items at the bottom of the hierarchy being the easiest to answer positively and items at the top being the most difficult for students to endorse positively. For example, item 5, “My awareness of the impact of medication errors on patient health” was the easiest item for students to endorse positively (ie, to give oneself a high rating on). The item hierarchy shows that item 3, “Ability to implement methods to reduce medication errors” was the most difficult of the 9 items to endorse positively (ie, to assess a high level of ability).
Expected score map and student normative distributions (Items 1-9).
Student responses relative to each item are evaluated using the pretest and posttest normative distributions provided in for items 1-9. For example, the normative distribution for the pretest shows that for item 7, “My ability to improve quality in pharmacy practice,” the majority of students rated their ability as weak or fair. However, this is in contrast to the results on the interpretation of the normative distribution for this item on the posttest where it is shown that the majority of students now perceived their ability as good or very good. Results from the other 8 items can be interpreted similarly. Improvement in students’ perceived ability was reported across all 9 items.
Results from the multiple linear regression model indicated that when examining what variables significantly affected the students’ change score, the university the student attended (p = 0.02), the completion of a class project (p = 0.03), and the length of coverage (ie, number of credit hours in the program) (p = 0.01) were positively related to students’ change scores. This indicates that these variables contributed to the improvement in the students’ perceived ability across items 1-9. Gender (p = 0.57) and previous quality improvement experience (p = 0.91) were not significant.
displays the hierarchical ordering of items 10-16 as each relates to students’ perceptions about the importance of quality improvement in pharmacy education. The right side of shows the item hierarchy, with items at the bottom of the hierarchy being the easiest to answer positively and items at the top being the most difficult for students to endorse positively. For example, item 14, “Medication errors are a major issue in pharmacy” was the easiest item for students to endorse positively (ie, to agree with). The item hierarchy shows that item 16, “This class provided information that I can apply in practice” was the most difficult of the 7 items to endorse positively (ie, to agree with). However, while item 16 may have been the most difficult item to endorse, as can be viewed from the figure, the majority of responses from students in both the pretest and posttest somewhat agreed or agreed with these statements. This indicates that students’ opinions of these issues were already positive before the class. Previous quality improvement experience (p = 0.04) positively affected students’ scores on items 1-9. School (p = 0.34), completion of a class project (p = 0.25), number of credit hours (p = 0.77), and gender (p = 0.86) were not associated with students’ scores.
Expected score map and student normative distributions (Items 10-16).
Faculty Survey Results
Seven faculty respondents from colleges and schools that had implemented the EPIQ program provided feedback (). The colleges and schools varied according to the number of years of the curriculum, the school calendar, the year in which EPIQ material was taught in the curriculum, whether the EPIQ program was a required part of the curriculum, and educational methods used to teach the EPIQ curriculum. In 6 of 7 colleges and schools, EPIQ content was added to address ACPE requirements. At the time of the survey, none of the colleges or schools used EPIQ in interprofessional education, introductory pharmacy practice experiences (IPPEs), or advanced pharmacy practice experiences (APPEs).
Characteristics of Seven US Colleges and Schools of Pharmacy That Implemented the Educating Pharmacy Students to Improve Quality Curriculum (EPIQ)
In all 7 colleges and schools, EPIQ was taught using lectures and in-class activities, predominantly as part of a separate course (4 of 7 colleges and schools). A typical class session included a mini-lecture, in-class activity, debriefing, and discussion of homework. Six of 7 schools use the Warholak and Nau companion textbook because it complemented the EPIQ lecture material.16
In-class exercises were most often used as formative assessments (5 of 7 colleges and schools), while summative assessments were more varied, with 4 colleges and schools using examinations, 2 using attitudinal assessments, and 1 using a team project.
The EPIQ program was implemented differently at each institution. Coverage of the EPIQ program ranged from 2 lectures to a full 3-credit hour course, and spanned from 1 to 32 weeks. Most participating faculty members either added to or integrated their previous quality improvement materials into the EPIQ curriculum (n=6) and/or omitted topics because of time and other constraints (n=6). Content added included additional medication error identification and reduction techniques, assessment techniques from the Institute for Safe Medication Practices, postmarketing surveillance and the Science of Safety (as defined by the Food and Drug Administration), lessons from the Institute for Healthcare Improvement, medication reconciliation, drug-drug interactions, and state-specific quality improvement laws.
describes faculty respondents' opinions of EPIQ. Faculty members responded that the EPIQ content was useful in achieving their intended curricular outcomes pertaining to patient medication safety and quality improvement. All 7 colleges and schools indicated that the student-centered activities were the most helpful types of educational materials contained in the EPIQ program. Suggestions for improving EPIQ content included: adding more application opportunities (ie, using more cases), decreasing redundancy, adding materials similar to what the faculty members added (mentioned earlier), keeping it updated, changing the evaluation questions to assess higher-level objectives, and adding more real-world examples.
Opinions of Instructors at a Colleges or School of Pharmacy That Implemented the Educating Pharmacy Students to Improve Quality Curriculum (EPIQ), N = 7
When asked “What was your major challenge in teaching the EPIQ material?,” 2 respondents indicated there was too much material for the time allotted in their curriculum, and 3 responded that teaching the concepts covered in EPIQ was a challenge because many of their students and some of their colleagues did not acknowledge the importance of quality improvement in pharmacy practice. Six faculty members indicated that learning the EPIQ material would help students become better pharmacists. All respondents agreed that the EPIQ program provided information that students will use and that decreasing medication errors is a major issue.