Key Informant Interviews
One respondent replied from each of the 7 pharmacy schools in Taiwan, yielding an interview response rate of 100%. Thirty-four of the 52 US colleges and schools invited participated in the interview (response rate 65.4%). Thirty-three of the 34 interviews were completed by phone (97.1%) and 1 via e-mail. There was no significant difference between the participating and nonparticipating colleges and schools in terms of geographic region defined by the US Census Bureau (p = 0.47) or in the proportion that were public schools (p = 0.70). Participants from Taiwan were directors of the whole or part of the experiential education curriculum at their college or school.
On average, Taiwan key informants reported enrolling more students per class and more class size variability than did informants at participating US colleges and schools. There also was a difference in program duration between Taiwan and US colleges and schools ().
Colleges and Schools of Pharmacy Represented in a Survey of Experiential Education Faculty Members Regarding the Science of Safety Curriculum
Among the US colleges and schools, 3 (8.8%) did not have graduate programs. Also, 3 US participants (8.8%) reported that their college or school was new and had not graduated students yet. A higher proportion of US colleges and schools emphasized teaching as compared to research and service (US vs. Taiwan: 44.1% vs. 28.6%). On a scale of 1 to 10, the US pharmacy faculty members perceived a significantly higher degree of curricular integration at their colleges and schools than did Taiwan faculty members (US vs. Taiwan: 7.2 ± 2.0 vs. 5.0 ± 2.1, p = 0.02). Other results regarding curriculum-related comparisons between the United States and Taiwan are presented in .
Comparisons of Faculty Members and Science of Safety Curriculums Between US and Taiwan Pharmacy Schools
Key Findings Among Taiwanese Informants
Theme 1: There are SoS coverage gaps in experiential education. All of the Taiwanese faculty members identified SoS gaps during experiential education including: ethical and safety considerations; drug-drug interaction (DDI) education; the ability to gather information and make decisions during urgent safety situations; integration of basic and clinical knowledge; communication skills; the ability to read and apply knowledge from the medical literature; and comprehensiveness of drug knowledge that pharmacy students possess. However, there was no consensus as to why these gaps might exist; some thought that gaps were inherent in the curriculum (eg, providing some courses as an elective) while others thought that SoS gaps were student-based or time-dependent (ie, gaps became less obvious as students completed various practice experiences).
Theme 2: Experiential education content should be standardized to fill SoS gaps. Informants reported the need to establish SoS criteria for pharmacy students and made the following suggestions to accomplish this: transforming elective rotations (provided before the required hospital rotations in Taiwan) into required rotations; exposing students more to case studies or problem-based learning opportunities to improve students’ decision-making abilities; expanding DDI knowledge to interactions among nonprescription medications, prescription medications, and Chinese medications; and familiarizing students with health information Web sites, such as PubMed, Medline, or Micromedex.
Theme 3: Experiential rotations are focused on the postmarketing phase of the SoS. Faculty members in Taiwan reported that experiential rotations fall into the scope of patient safety or medication safety, which represents only the postmarketing aspects of the SoS. Respondents have indicated that hospitals put emphasis on patient safety, counseling, and compliance, and that patient safety is emphasized at rotation sites. In addition, one of the respondents considered issues such as recognizing patient safety, differentiating drug products, and medication error prevention to be a major focus in every rotation.
Theme 4: IPPE can enhance pharmacy students’ SoS education. All Taiwanese participants thought that mandatory IPPEs could enhance students’ SoS competencies. Respondents were aware of the overwhelming burden of tasks and skills for pharmacy students to learn during experiential education when introductory practice experiences are not available.
Common obstacles to adding IPPEs into the experiential curriculum were identified. Potential barriers stated by the participants included the number of hours available for experiential education in the curriculum and the disparity in socioeconomic status (eg, salary earned and the level of respect from the general public) between US and Taiwan pharmacists.
Key Findings Among US Informants
Theme 1: There are SoS coverage gaps in experiential education. Key informants perceived coverage gaps in US experiential SoS coverage. Respondents perceived that experiential training equips pharmacy students to safely practice at the individual level much better than it prepares students to evaluate and improve medication safety processes at an institutional level. An important cause of the gaps identified was a lack of curricular emphasis on the SoS. In order to make SoS a mandatory piece within the curriculum, one respondent indicated the need for deeper involvement and advocacy from deans of pharmacy schools. The respondent indicated that requirements to incorporate SoS in the required portion of the curriculum from the Accreditation Council for Pharmacy Education (ACPE) would also be helpful. However, approximately two-thirds of US participants indicated that, even with the coverage gaps, their colleges and schools provided sufficient SoS education during practice experiences to enable students to work in a variety of practice settings.
Theme 2: Experiential education content should be standardized to fill SoS gaps. US informants indicated that standardization of SoS teaching in experiential education is necessary. A few respondents suggested building a “safety core” within the curriculum to increase educational consistency and quality.
Theme 3: Practice experiences address the postmarketing portion of the SoS. US respondents indicated that SoS coverage usually was focused on the postmarketing phase of SoS. A few key informants mentioned offering SoS coverage in the form of elective practice experiences at the FDA, pharmaceutical companies, or research/investigational drug services.
There was no consistency concerning where in experiential education the SoS was covered. Over half of the respondents indicated that SoS topics are covered in IPPEs, about a third indicated that specific SoS education was provided as elective or institutional APPEs, and a few participants indicated the SoS was integrated throughout the curriculum and emphasized during experiential education.
Theme 4: Students should be exposed to more practice experiences during experiential education. US key informants expressed concerns about students not having enough hands-on exposure during experiential education to fully develop SoS competencies. Respondents felt that putting the students in the role of a pharmacist can educate them about the importance of accuracy and diligence. Suggested topics for practice experience included: (1) clinical pharmacokinetics; (2) continuous quality assurance processes; (3) communication of medication safety issues at practice sites; and (4) use of information obtained from the literature.
Theme 5: Health system knowledge is important for developing students’ SoS competencies. Key informants agreed that students do not fully understand the role the SoS plays throughout US health systems. Informants suggested that equipping students with knowledge related to policies of institutions would help them successfully deal with patient safety issues. Another key informant perceived the necessity for SoS to be integrated throughout the experiential portion of the curriculum to help students obtain appropriate system-based knowledge during time-limited practice experiences.
Theme 6: Preceptors and practice experience sites play a major role in experiential SoS education. Key informants were concerned about SoS coverage gaps in experiential education due to preceptor variability. Participants suggested numerous ways to enhance SoS education in practice experiences including providing preceptor SoS education such as online training classes, recruiting preceptors who focus on the SoS, and assisting preceptors in developing the syllabi for APPE courses. Some participants suggested finding or developing model sites that exemplify SoS standards. Another key informant indicated that choosing health systems in which their hospital CEOs or pharmacy directors focus on medication safety for practice experience sites also might improve SoS education.
All 7 (100%) Taiwanese experiential experts and 28 (82.4%) of 34 US faculty members completed the questionnaire. Four of the 28 (14.3%) US respondents did not provide institutional information. There was no difference between the US and Taiwan respondents in terms of proportion of pharmacy colleges and schools participating that were associated with a medical center (52.2% vs. 71.4%, p = 0.43). However, respondents indicated that US colleges and schools provided significantly more experiential hours during (1441 ± 182 vs. 640 ± 0, respectively; p < 0.01) and before APPEs (328.7 ± 56.3 vs. 34.3 ± 90.7, respectively; p < 0.01) than Taiwan schools.
Although no significant differences were found between US and Taiwan faculty perceptions regarding whether the 14 SoS topics spanning from preclinical to postmarketing surveillance topics are covered during experiential education, the proportion of colleges and schools that cover each topic was not consistent (). SoS topics pertaining to preclinical and clinical trials were covered less in both countries. There were no differences observed regarding faculty perception of the coverage adequacy between US and Taiwan pharmacy colleges and schools. In addition, most faculty members thought that the coverage was “adequate” or “somewhat adequate.”
Respondents’ Perceptions of SoS Topic Coverage in US and Taiwan Pharmacy Schools
Taiwanese respondents generally perceived that pharmacy students in Taiwan possess “poor” to “moderate” SoS skill levels before beginning rotations. Taiwanese participants perceived that students’ SoS skill levels improved during rotations such that they were “moderate” to “good” after completion of rotations. However, most Taiwanese respondents indicated that pharmacy students’ ability to identify risks related to human subject research was “poor” even after completing rotations.
US participants typically perceived that students have “moderate” or “good” SoS skill levels before APPEs. Competencies that were perceived as relatively weak before APPEs included the ability to submit a completed adverse drug event form to the FDA and to use patient databases to find factors contributing to patient illness. Students’ SoS competency levels were perceived as “good” or “very good” after APPEs. However, US pharmacy students’ ability to use patient databases and to identify risks associated with human subject research were 2 areas that were perceived as weak post-APPE.
While US participants perceived that all SoS skill levels significantly increase (p < 0.05) after APPEs, Taiwan participants perceived a significant improvement during APPEs (p < 0.05) in only half of the listed SoS skill levels (). US respondents perceived signficantly higher student SoS skill levels before and after APPE in several competencies as compared to Taiwan respondents.
Between and Within-Country Statistical Comparisons of the SoS-Related Competencies Before and After Completing Advanced Pharmacy Practice Experiencesa
When comparing the interpretation of SoS, all of the respondents in Taiwan and the United States perceived that SoS was related to medication safety. Thus, from a medication-safety standpoint, there were no differences in their perceptions regarding the definition of SoS. However, Taiwan faculty members were more likely than US faculty members to interpret the SoS using the FDA's drug development timeline approach (16.7% vs. 75.0%, respectively, p = 0.03).
Non-Response Evaluation and Preliminary Reliability and Validity Testing
Perceptions of SoS coverage in experiential education did not differ significantly between early and late US respondents. Statistical comparisons between early and late respondents in Taiwan were considered non-applicable due to the extremely small sample sizes of early (n = 3) and late (n = 4) respondents.
Only 1 of the 6 Taiwan respondents (16.7%) suggested the need to add survey questions to future survey instruments related to medication errors and drug-drug interactions. Six out of 20 US respondents (30.0%) thought additional questions may be needed, such as asking participants to provide a copy of course syllabi and an assessment of the experiential SoS topics taught in didactic courses.
In terms of construct validity, 10 out of 14 items (71.4%) had item-domain correlation coefficients of the same domain greater than 0.4. Six questions had lower item-domain correlation with different domains as compared to the same domain. Nevertheless, only 1 question (Q5) had a correlation coefficient that was more than 2 times the standard error lower than the correlation coefficient with domain 3.
In both versions, question sets of which reliability were evaluated had high internal consistency. The values of 95% confidence intervals (CI) for Cronbach's alpha overlapped between the 2 versions of the SoS questionnaire, thus providing preliminary evidence that the 2 versions do not differ significantly. Moreover, the 95% CI of alpha in both questionnaires exceeded 0.7.24