Science of Safety Topic Coverage in Experiential Education in US and Taiwan Colleges and Schools of Pharmacy

doi:10.5688/ajpe7510202

Am J Pharm Educ. 2011 December 15; 75(10): 202.

PMCID: PMC3279028

Science of Safety Topic Coverage in Experiential Education in US and Taiwan Colleges and Schools of Pharmacy

Derek H. Tang, MS,^a Terri L. Warholak, PhD,^a Marion K. Slack, PhD,^a Daniel C. Malone, PhD,^a and Churn-Shiouh Gau, PhD^b

^aThe University of Arizona College of Pharmacy

^bGraduate Institute of Clinical Pharmacy, National Taiwan University

Corresponding author.

Corresponding Author: Derek H. Tang, MS, Graduate Student, Department of Pharmacy Practice and Science, The University of Arizona College of Pharmacy, 1295 N. Martin, PO Box 210202, Tucson, AZ 85721. Tel: 520-342-7683. E-mail: dtang/at/pharmacy.arizona.edu

Received August 19, 2011; Accepted August 26, 2011.

Abstract

Objective. To compare the science of safety (SoS) topic coverage and associated student competencies in the experiential education curricula of colleges and schools of pharmacy in the United States and Taiwan.

Methods. The experiential education director, assistant director, or coordinator at a random sample of 34 US colleges and schools of pharmacy and all 7 Taiwan schools of pharmacy were interviewed and then asked to complete an Internet-based survey instrument.

Results. Faculty members in both countries perceived that experiential curricula were focused on the postmarketing phase of the SoS, and that there is a need for the pharmacy experiential curricula to be standardized in order to fill SoS coverage gaps. Inter-country differences in experiential SoS coverage were noted in topics included for safety biomarkers that signal potential for drug-induced problems and pharmacogenomics.

Conclusions. Experiential SoS topic coverage and student ability gaps were perceived within and between US and Taiwan colleges and schools of pharmacy.

Keywords: science of safety, experiential education, survey research, international

INTRODUCTION

Drug safety encompasses many activities across a range of disciplines and approaches, and pharmacists play a key role in ensuring the public's safety as a member of the healthcare team. Because of well-publicized drug safety issues, the US Food and Drug Administration (FDA) instituted a number of activities to increase awareness of and ultimately improve medication safety. In its 2008 Sentinel Initiative Report, the FDA defined the science of safety (SoS) as a broad concept that focuses on a lifecycle approach to drug safety.¹ This broad perspective on SoS allows for many opportunities for pharmacists to impact patient safety with medication use.² Using the drug lifecycle approach ensures that safety concerns raised at any point in the drug development/marketing process “can be evaluated along with relevant benefit-risk data to inform treatment choices and regulatory decision making.”¹ Analyses with respect to risk vs. benefit or safety vs. access were regarded as important areas within the SoS that the FDA seeks to improve.¹ Specifically, FDA seeks to understand how pharmacy students are educated in the SoS in order to identify how the agency can partner with pharmacy professionals to improve medication safety.

Topics related to the SoS are taught in various courses in US colleges and schools of pharmacy.² In fact, when defined by the FDA's lifecycle approach, some US faculty members pointed out that the entire pharmacy curriculum relates to the SoS.² However, the quality and quantity of curricula associated with the SoS, such as medication error instruction, varies widely among colleges and schools of pharmacy.³ Only 12% of respondents to a survey conducted by Johnson and colleagues indicated that medication error information was taught during pharmacy practice experiences (PPEs). This raises the concern that other critical safety-related teaching may be lacking during PPEs.

Although medication safety remains a global issue, there is a dearth of current studies describing curriculum devoted to the SoS in Taiwan. Moreover, fundamental differences in pharmacy education exist between the United States and Taiwan with respect to experiential education. A 4- to 5-year baccalaureate pharmacy degree program is the prevalent program offered in Taiwan. Upon graduation, baccalaureate pharmacy students are eligible to take the national pharmacist licensure examination to become a pharmacist. The required portion of pharmacy experiential education in Taiwan is composed of multiple hospital pharmacy experiences lasting for approximately half a year and are roughly analogous to the advanced pharmacy practice experiences (APPEs) offered in US doctor of pharmacy (PharmD) programs. However, Taiwanese students are not obliged to participate in community pharmacy or other experiences in other practice settings during the experiential curricula.

Investigating SoS-related curricula in Taiwan and comparing it with that taught in the United States could identify for pharmacy educators, practitioners, and administrators in the 2 countries the differences in, similarities of, and improvements needed for the current curricula to effectively provide SoS education during PPEs. This mixed-method study attempted to obtain, compare, and contrast pharmacy faculty perceptions in the United States and Taiwan on experiential education SoS curricula for pharmacy students by conducting key informant interviews and an Internet-based survey.

METHODS

For this study, the SoS was defined as: “the systematic search for knowledge about risks of exposure to drugs and methods to protect individuals from drug-related problems. The SoS encompasses the study of adverse events any place within the product life cycle from pre-clinical safety testing in animals to post-marketing surveillance in the general population.”¹ This study used a series of key informant interviews to gather qualitative data on perceptions of experiential SoS coverage and gaps, followed by an Internet-based survey instrument to gather quantitative data on SoS topic coverage in experiential education and perceptions of student SoS skills. The interviews and survey were conducted between September and October 2010. The University Institutional Review Board granted permission to conduct this study.

A list of the accredited US colleges and schools of pharmacy colleges was obtained via the American Association of Colleges of Pharmacy (AACP) listing of member organizations and was entered into Microsoft Excel. The spreadsheet was used to generate a random sample of 34 US colleges and schools of pharmacy. The sample size of 34 was determined by estimating an effect size and conducting power calculation by using the survey data from the Holdford SoS study² and G*Power, version 3.1.2 (Institute for Experimental Psychology, Dusseldorf, Germany). To detect a 65% difference, a sample size of at least 17 schools was needed (assuming a 2-sided test with alpha = 0.05 and beta = 0.80). To account for a possible nonresponse rate of 50% (nonresponse rate in the Holdford and colleagues study), the US pharmacy schools and colleges were oversampled by doubling the estimated sample size from 17 to 34. If a college or school did not respond or declined to participate, the next school on the random sample list was selected. Due to the limited number of colleges and schools of pharmacy in Taiwan, all 7 were included in the study.

Contact information for the US sample was obtained from the AACP roster. Information for the Taiwan schools was obtained from the pharmacy schools’ Web sites or by contacting their administration offices. The participants of interest were experiential education directors, assistant experiential education directors, and/or experiential education coordinators. One participant was invited from each college or school, and the college or school was the unit of analysis. If the initially invited person declined to participate in the interview, an alternate was contacted (a maximum of 3 individuals were contacted at each college or school).

Interview Script and Questionnaire Design

Adapted from the Holdford and colleagues SoS study,² the interview script and questionnaire were intended to provide general comparisons of faculty perceptions of the experiential education SoS curriculum in colleges and schools of pharmacy in the US and Taiwan. During the interview, Taiwanese participants were asked if they thought that the addition of introductory pharmacy practice experiences (IPPEs) would enhance students’ ability to manage medication safety issues and whether they thought that IPPEs would be applicable in Taiwan. The instrument described the study purpose, provided the SoS definition, and included 3 major sections: (1) the perceived coverage of SoS-related topics in experiential education; (2) the perceived impact of APPEs; and (3) the perceived definition of SoS and school demographic questions. The questionnaire was piloted via a think-aloud protocol and revised before dissemination.^4-6

Key informant interview and survey documents (including the interview script, Internet-based questionnaire, and subject disclosure forms) were translated into Chinese, and the interviews pertaining to the Taiwanese participants were processed in Chinese. The Chinese versions of the documents were translated from the original English language documents by the principal investigator. One of the authors (C.G.) from Taiwan translated the instruments back into English to ensure translation accuracy.^7,8 After the back-translation process was finalized, a Taiwanese graduate student at the University of Arizona was asked to provide input on the Chinese version in terms of script readability and usage of specific terms to ensure comparability. (The English version of the interview script and questionnaire are available upon request from the corresponding author.)

Key Informant Interviews and Internet-Based Survey Tool

Initial contacts of the key-informant interviewees and the subsequent interviews were conducted via Skype or telephone. E-mail communications were used if faculty members were not able to participate in the voice-based interview. During the invitation process, the interviewer (D.T.) introduced himself, summarized the purpose of the study, and obtained participant consent. During each interview, notes were taken, and the interview was audiotaped. Chinese interview transcripts were summarized and translated into English by the principal author.⁹ Ten percent of the translated scripts were randomly selected and retranslated by the principal author to ensure accurate interpretation.

Questionnaires were distributed via the Internet, using the online survey software SurveyMonkey (Menlo Park, CA). The survey tool was accessible from September 2010 through October 2010. For the participants who did not complete the questionnaire, subsequent e-mail reminders were sent. To decrease the likelihood of multiple responses from the same respondent, the survey was configured to allow only one response per person.¹⁰

Data Analysis

Data from interview transcripts were analyzed by the principal investigator. College and school demographic data were summarized. Categorical data were presented as percentages and frequencies; whereas mean and standard deviations were calculated for continuous data. Fisher's exact test was used to compare categorical data and the Wilcoxon rank-sum test was used for ordinal data. For theme data, data analysis included the qualitative analytic techniques of topic coding, in which responses related to each question were grouped according to topic themes and summarized, and analytical coding, in which investigators interpreted, compared, and contrasted the responses for each question and made judgments regarding the similarities and differences across colleges/schools and countries. Based on this process, major themes were identified and summarized.¹¹

Survey data were extracted from SurveyMonkey and entered in Excel 2007 for analysis. Ten percent of the data were randomly selected and reexamined by the investigators to ensure data entry accuracy. Due to the small sample size in Taiwan, nonparametric tests were conducted to identify cross-cultural differences¹²: Fisher's exact test was used for categorical data; Wilcoxon rank-sum tests were used for ordinal and non-correlated continuous data; and Wilcoxon signed-rank tests were used for paired ordinal variables.

The significance level was set at 0.05. Two statistical software packages were used due to the perceived strengths of each in analyzing particular types of data: Statistical Analysis Software (SAS), version 9.2 (Cary, NC) was used to derive reliability and validity measures, while Data Analysis and Statistical Software (STATA), version 11.0 (College Station, TX), was used to conduct the aforementioned nonparametric tests. To estimate the effect of nonresponse error, early and late responses for the questions with dichotomous response options evaluating SoS coverage in experiential education were compared. To ensure report completeness, the Checklist for Reporting Results of the Internet e-surveys (CHERRIES) was used.¹⁰

Reliability of the survey instrument was established by testing the internal consistency of the first 2 sections of the questionnaire (ie, the 28 close-ended questions). Cronbach's alpha was computed to test questionnaire reliability for each language version.^13-15 Content and construct validity were separately tested for each language version.^16-20 Content validity was tested and verified based on (1) the open-ended survey questions: “Is there anything that you think should be asked that you were not asked?” “Are there any items that you think should be revised or removed?” and (2) expert judgments by the principal investigator's thesis committee members. Construct validity was estimated based on the 14 questions concerning the adequacy of SoS coverage. Item-domain correlation was used to provide preliminary evidence of construct validity.^17,21 Reliability and validity testing were based on the assumption of equal intervals between response options.^22,23

RESULTS

Key Informant Interviews

One respondent replied from each of the 7 pharmacy schools in Taiwan, yielding an interview response rate of 100%. Thirty-four of the 52 US colleges and schools invited participated in the interview (response rate 65.4%). Thirty-three of the 34 interviews were completed by phone (97.1%) and 1 via e-mail. There was no significant difference between the participating and nonparticipating colleges and schools in terms of geographic region defined by the US Census Bureau (p = 0.47) or in the proportion that were public schools (p = 0.70). Participants from Taiwan were directors of the whole or part of the experiential education curriculum at their college or school.

On average, Taiwan key informants reported enrolling more students per class and more class size variability than did informants at participating US colleges and schools. There also was a difference in program duration between Taiwan and US colleges and schools (Table 1).

Table 1.

Colleges and Schools of Pharmacy Represented in a Survey of Experiential Education Faculty Members Regarding the Science of Safety Curriculum

Among the US colleges and schools, 3 (8.8%) did not have graduate programs. Also, 3 US participants (8.8%) reported that their college or school was new and had not graduated students yet. A higher proportion of US colleges and schools emphasized teaching as compared to research and service (US vs. Taiwan: 44.1% vs. 28.6%). On a scale of 1 to 10, the US pharmacy faculty members perceived a significantly higher degree of curricular integration at their colleges and schools than did Taiwan faculty members (US vs. Taiwan: 7.2 ± 2.0 vs. 5.0 ± 2.1, p = 0.02). Other results regarding curriculum-related comparisons between the United States and Taiwan are presented in Table 2.

Table 2.

Comparisons of Faculty Members and Science of Safety Curriculums Between US and Taiwan Pharmacy Schools

Key Findings Among Taiwanese Informants

Theme 1: There are SoS coverage gaps in experiential education. All of the Taiwanese faculty members identified SoS gaps during experiential education including: ethical and safety considerations; drug-drug interaction (DDI) education; the ability to gather information and make decisions during urgent safety situations; integration of basic and clinical knowledge; communication skills; the ability to read and apply knowledge from the medical literature; and comprehensiveness of drug knowledge that pharmacy students possess. However, there was no consensus as to why these gaps might exist; some thought that gaps were inherent in the curriculum (eg, providing some courses as an elective) while others thought that SoS gaps were student-based or time-dependent (ie, gaps became less obvious as students completed various practice experiences).

Theme 2: Experiential education content should be standardized to fill SoS gaps. Informants reported the need to establish SoS criteria for pharmacy students and made the following suggestions to accomplish this: transforming elective rotations (provided before the required hospital rotations in Taiwan) into required rotations; exposing students more to case studies or problem-based learning opportunities to improve students’ decision-making abilities; expanding DDI knowledge to interactions among nonprescription medications, prescription medications, and Chinese medications; and familiarizing students with health information Web sites, such as PubMed, Medline, or Micromedex.

Theme 3: Experiential rotations are focused on the postmarketing phase of the SoS. Faculty members in Taiwan reported that experiential rotations fall into the scope of patient safety or medication safety, which represents only the postmarketing aspects of the SoS. Respondents have indicated that hospitals put emphasis on patient safety, counseling, and compliance, and that patient safety is emphasized at rotation sites. In addition, one of the respondents considered issues such as recognizing patient safety, differentiating drug products, and medication error prevention to be a major focus in every rotation.

Theme 4: IPPE can enhance pharmacy students’ SoS education. All Taiwanese participants thought that mandatory IPPEs could enhance students’ SoS competencies. Respondents were aware of the overwhelming burden of tasks and skills for pharmacy students to learn during experiential education when introductory practice experiences are not available.

Common obstacles to adding IPPEs into the experiential curriculum were identified. Potential barriers stated by the participants included the number of hours available for experiential education in the curriculum and the disparity in socioeconomic status (eg, salary earned and the level of respect from the general public) between US and Taiwan pharmacists.

Key Findings Among US Informants

Theme 1: There are SoS coverage gaps in experiential education. Key informants perceived coverage gaps in US experiential SoS coverage. Respondents perceived that experiential training equips pharmacy students to safely practice at the individual level much better than it prepares students to evaluate and improve medication safety processes at an institutional level. An important cause of the gaps identified was a lack of curricular emphasis on the SoS. In order to make SoS a mandatory piece within the curriculum, one respondent indicated the need for deeper involvement and advocacy from deans of pharmacy schools. The respondent indicated that requirements to incorporate SoS in the required portion of the curriculum from the Accreditation Council for Pharmacy Education (ACPE) would also be helpful. However, approximately two-thirds of US participants indicated that, even with the coverage gaps, their colleges and schools provided sufficient SoS education during practice experiences to enable students to work in a variety of practice settings.

Theme 2: Experiential education content should be standardized to fill SoS gaps. US informants indicated that standardization of SoS teaching in experiential education is necessary. A few respondents suggested building a “safety core” within the curriculum to increase educational consistency and quality.

Theme 3: Practice experiences address the postmarketing portion of the SoS. US respondents indicated that SoS coverage usually was focused on the postmarketing phase of SoS. A few key informants mentioned offering SoS coverage in the form of elective practice experiences at the FDA, pharmaceutical companies, or research/investigational drug services.

There was no consistency concerning where in experiential education the SoS was covered. Over half of the respondents indicated that SoS topics are covered in IPPEs, about a third indicated that specific SoS education was provided as elective or institutional APPEs, and a few participants indicated the SoS was integrated throughout the curriculum and emphasized during experiential education.

Theme 4: Students should be exposed to more practice experiences during experiential education. US key informants expressed concerns about students not having enough hands-on exposure during experiential education to fully develop SoS competencies. Respondents felt that putting the students in the role of a pharmacist can educate them about the importance of accuracy and diligence. Suggested topics for practice experience included: (1) clinical pharmacokinetics; (2) continuous quality assurance processes; (3) communication of medication safety issues at practice sites; and (4) use of information obtained from the literature.

Theme 5: Health system knowledge is important for developing students’ SoS competencies. Key informants agreed that students do not fully understand the role the SoS plays throughout US health systems. Informants suggested that equipping students with knowledge related to policies of institutions would help them successfully deal with patient safety issues. Another key informant perceived the necessity for SoS to be integrated throughout the experiential portion of the curriculum to help students obtain appropriate system-based knowledge during time-limited practice experiences.

Theme 6: Preceptors and practice experience sites play a major role in experiential SoS education. Key informants were concerned about SoS coverage gaps in experiential education due to preceptor variability. Participants suggested numerous ways to enhance SoS education in practice experiences including providing preceptor SoS education such as online training classes, recruiting preceptors who focus on the SoS, and assisting preceptors in developing the syllabi for APPE courses. Some participants suggested finding or developing model sites that exemplify SoS standards. Another key informant indicated that choosing health systems in which their hospital CEOs or pharmacy directors focus on medication safety for practice experience sites also might improve SoS education.

Survey Findings

All 7 (100%) Taiwanese experiential experts and 28 (82.4%) of 34 US faculty members completed the questionnaire. Four of the 28 (14.3%) US respondents did not provide institutional information. There was no difference between the US and Taiwan respondents in terms of proportion of pharmacy colleges and schools participating that were associated with a medical center (52.2% vs. 71.4%, p = 0.43). However, respondents indicated that US colleges and schools provided significantly more experiential hours during (1441 ± 182 vs. 640 ± 0, respectively; p < 0.01) and before APPEs (328.7 ± 56.3 vs. 34.3 ± 90.7, respectively; p < 0.01) than Taiwan schools.

Although no significant differences were found between US and Taiwan faculty perceptions regarding whether the 14 SoS topics spanning from preclinical to postmarketing surveillance topics are covered during experiential education, the proportion of colleges and schools that cover each topic was not consistent (Table 3). SoS topics pertaining to preclinical and clinical trials were covered less in both countries. There were no differences observed regarding faculty perception of the coverage adequacy between US and Taiwan pharmacy colleges and schools. In addition, most faculty members thought that the coverage was “adequate” or “somewhat adequate.”

Table 3.

Respondents’ Perceptions of SoS Topic Coverage in US and Taiwan Pharmacy Schools

Taiwanese respondents generally perceived that pharmacy students in Taiwan possess “poor” to “moderate” SoS skill levels before beginning rotations. Taiwanese participants perceived that students’ SoS skill levels improved during rotations such that they were “moderate” to “good” after completion of rotations. However, most Taiwanese respondents indicated that pharmacy students’ ability to identify risks related to human subject research was “poor” even after completing rotations.

US participants typically perceived that students have “moderate” or “good” SoS skill levels before APPEs. Competencies that were perceived as relatively weak before APPEs included the ability to submit a completed adverse drug event form to the FDA and to use patient databases to find factors contributing to patient illness. Students’ SoS competency levels were perceived as “good” or “very good” after APPEs. However, US pharmacy students’ ability to use patient databases and to identify risks associated with human subject research were 2 areas that were perceived as weak post-APPE.

While US participants perceived that all SoS skill levels significantly increase (p < 0.05) after APPEs, Taiwan participants perceived a significant improvement during APPEs (p < 0.05) in only half of the listed SoS skill levels (Table 4). US respondents perceived signficantly higher student SoS skill levels before and after APPE in several competencies as compared to Taiwan respondents.

Table 4.

Between and Within-Country Statistical Comparisons of the SoS-Related Competencies Before and After Completing Advanced Pharmacy Practice Experiences^a

When comparing the interpretation of SoS, all of the respondents in Taiwan and the United States perceived that SoS was related to medication safety. Thus, from a medication-safety standpoint, there were no differences in their perceptions regarding the definition of SoS. However, Taiwan faculty members were more likely than US faculty members to interpret the SoS using the FDA's drug development timeline approach (16.7% vs. 75.0%, respectively, p = 0.03).

Non-Response Evaluation and Preliminary Reliability and Validity Testing

Perceptions of SoS coverage in experiential education did not differ significantly between early and late US respondents. Statistical comparisons between early and late respondents in Taiwan were considered non-applicable due to the extremely small sample sizes of early (n = 3) and late (n = 4) respondents.

Only 1 of the 6 Taiwan respondents (16.7%) suggested the need to add survey questions to future survey instruments related to medication errors and drug-drug interactions. Six out of 20 US respondents (30.0%) thought additional questions may be needed, such as asking participants to provide a copy of course syllabi and an assessment of the experiential SoS topics taught in didactic courses.

In terms of construct validity, 10 out of 14 items (71.4%) had item-domain correlation coefficients of the same domain greater than 0.4. Six questions had lower item-domain correlation with different domains as compared to the same domain. Nevertheless, only 1 question (Q5) had a correlation coefficient that was more than 2 times the standard error lower than the correlation coefficient with domain 3.

In both versions, question sets of which reliability were evaluated had high internal consistency. The values of 95% confidence intervals (CI) for Cronbach's alpha overlapped between the 2 versions of the SoS questionnaire, thus providing preliminary evidence that the 2 versions do not differ significantly. Moreover, the 95% CI of alpha in both questionnaires exceeded 0.7.²⁴

DISCUSSION

The Science of Safety (SoS) is a relatively new area defined by the FDA, thus exploratory studies are needed to describe curriculum coverage, the perceived importance of SoS, and students’ SoS competencies. This is the first study to obtain and compare faculty perceptions of SoS topic coverage and students’ SoS competencies in experiential education curriculum in colleges and schools of pharmacy in the US and Taiwan.

In general, US and Taiwan key informants agreed that the SoS coverage in the required portion of experiential education focused on the postmarketing phase: medication and patient safety. Because a significant portion of respondents in both countries perceived that their colleges and schools provided sufficient experiential education in SoS, key informants may be implying that postmarketing topics should be emphasized. Future experiential SoS curriculums may need to emphasize topics pertaining to postmarketing surveillance.

Key informants in both countries acknowledged SoS coverage gaps but identified several barriers to filling these gaps. First, some informants expressed that it was difficult to estimate SoS coverage during practice experiences because preceptors differ across and between practice experiences. This may imply that identifying gaps could be difficult. Opinions of SoS coverage during clerkships elicited from preceptors from a wide range of practice settings may be a good mechanism for identifying the breadth and depth of SoS coverage. Second, informants thought that without standards or a structured SoS experiential curriculum their impact on SoS in experiential education would be minimal because preceptors are directly responsible for teaching clerkship students. However, if a standardized SoS experiential education curriculum were to be adapted, experiential education directors could then recommend the SoS areas/topics that should be taught on each practice experience to ensure more comprehensive and consistent SoS coverage and positive student outcomes. The following suggestions from key informants may help to reduce preceptor and practice experience site differences: (1) standardize the experiential education curriculum; (2) provide preceptor education; and (3) identify and recruit preceptors and/or practice experience sites that focus on SoS. Most importantly, an implicit and explicit consensus regarding the criticality of SoS needs to be developed and systematically implemented across colleges and schools of pharmacy, directors and coordinators of experiential education, preceptors, and clinical faculty members to serve as role models for students. Developing reward systems for preceptors and clinical faculty members who are dedicated to promoting a culture of safety could be a first step towards accomplishing this goal. However, the only other SoS assessment in pharmacy education identified that curricular time limitations and lack of faculty SoS expertise may be barriers to improving SoS coverage in US colleges and schools of pharmacy.²

Key informants’ perceived that a more unified experiential SoS curricula is needed and that the gaps identified paralleled the perceptions expressed in previous investigations of US colleges and schools of pharmacy. For instance, respondents in a study that measured overall SoS topic coverage in US pharmacy curricula indicated diverse SoS coverage within and between colleges and schools of pharmacy.² Johnson and colleagues found that 12% of US pharmacy programs covered medication error prevention during experiential education.³ In another study, the proportion of students who indicated that they received adverse drug reaction reporting database instruction during practice experiences varied greatly across pharmacy colleges and schools (12%-19%).²⁵ These variations may be alleviated by regulations imposed by an accreditation organization such as the ACPE in the United States or the Clinical Pharmacy Association of Taiwan. Developing a culture in which practice-specific SoS educational techniques are exchanged among preceptors and clinical faculty members also might aid the standardization process.

Survey respondents reported that a few topics pertaining to the preclinical and clinical phases of SoS in the questionnaire (such as how first-in human safety studies are designed; or how phase I clinical trials inform safety and dosing of a product) were not perceived as a focus of practice experiences in either the US or Taiwan. These results correspond to the results obtained in the Holdford and colleagues SoS study that found that US colleges and schools of pharmacy focused on postmarketing topics of the SoS during experiential education.²

US and Taiwanese survey participants generally perceived significant improvement of SoS skills after students completed APPEs. Competencies that did not significantly improve after APPEs may have been because these skills were not emphasized and may represent skills gaps that need to be filled. For approximately two-thirds of the listed SoS skill levels, US respondents higher competency levels in pharmacy students compared to Taiwan respondents before and after APPEs. These competencies, which fall into general areas such as communicating with healthcare professionals about medication errors and adverse drug reactions, and identifying factors associated with adverse drug reactions, may reflect gaps in the pharmacy curricula of Taiwan schools. Competencies for which US respondents perceived higher skill levels before APPEs but comparable skill levels after APPEs (such as submitting a completed adverse event reporting form to the corresponding agency) may be areas that both US and Taiwan colleges and schools of pharmacy emphasize or may be areas of potential weakness in US pharmacy APPE curricula. In contrast, competencies in which Taiwan respondents perceived comparable skill levels before APPEs but lower skill levels as compared to US respondents (such as communicating the risks of medications at a patient appropriate level) may indicate that these skills are accentuated in US colleges and schools of pharmacy during APPEs, or potential required areas of improvements in Taiwan. Competencies in which no significant differences were perceived between the US and Taiwan regarding skill levels in both pre-APPE and post-APPE (such as identifying risks associated with human subject research) may imply that pharmacy students in both countries receive a comparable amount of skill training (whether sufficient or insufficient) during APPEs.

All survey participants generally perceived that SoS was related to medication safety. However, the proportion of US survey participants that had adopted the FDA's drug development timeline approach to SoS was significantly lower compared to the proportion of participants from Taiwan pharmacy schools. Thus, further explanation from the FDA may be required to clarify the boundaries and similarities between medication safety and the SoS to increase understanding and use of the SoS concept.

Despite fundamental differences between pharmacy curricula, this study is beneficial to US and Taiwan pharmacy education for several reasons. First, identification of SoS gaps between US and Taiwan colleges and schools of pharmacy provides opportunities for experiential experts in the US to scrutinize whether these gaps need to be filled. Second, individual US pharmacy colleges and schools can perform comparisons between experiential education provided at their schools and that at the national level by using the information provided in this study. Suggestions for improving SoS education provided by the informants can be implemented at the school level. This study provides SoS information from Taiwan for additional between-country comparisons that can be used for curricular growth in Taiwan. Thus, this comparison may be used to guide revisions to both classroom lecture and experiential curriculum to create a stronger emphasis on SoS.

This study had several limitations. The questionnaire solicited from only 1 experiential faculty member from each college and school and their perceptions may not truly represent their institution. However, those faculty members who were most likely to be familiar with the experiential curriculum in their college or school were recruited; thus, their responses should be sufficient for exploratory purposes. Second, language gaps may have existed between the Chinese and the English versions of the interview script and questionnaire despite completing a careful back-translation and cross-form equivalency evaluation. Ideally, bilingual translation experts should assist with the whole translation process. Third, results of the key informant interview study design was subject to limitations that are common to qualitative research, such as limited generalizability and subjectivity.²⁶ Fourth, the number of researchers summarizing the major themes was less than optimal. It would have been beneficial to have a second researcher independently derive themes from the Chinese transcripts; however, due to budgetary constraints, this was not possible.

CONCLUSIONS

Gaps in SoS coverage exist in the curricula of pharmacy colleges and schools in the United States and Taiwan; also, differences exist between the SoS curriculums in the 2 countries. Compared to pre-APPE, US respondents perceived significant improvement post-APPE in all of pharmacy students’ SoS competencies listed in the questionnaire, and Taiwan respondents perceived improvement in half of the SoS competencies. US participants perceived significantly higher skill levels compared to Taiwan in approximately two-thirds of SoS competencies before and after APPEs. Judgments pertaining to whether these perceived variations are true disparities or are regarded as acceptable depend on the existing pharmacy education culture and trends within and between the pharmacy schools and countries. Future research testing a larger sample is necessary to verify these findings. Also, research targeting individual preceptors as key informants/survey participants is needed as this would should help identify SoS coverage in specific practice experiences and investigate the extent that preceptor and site variations influence SoS coverage. Finally, future research should address how pharmacy students self-perceive SoS coverage during experiential education, and how faculty members perceive SoS coverage in individual lecture-based courses.

REFERENCES

1. US Department of Health and Human Services. Food and Drug Administration's Sentinel Initiative. The sentinal initiative 2008 report. http://www.fda.gov/downloads/Safety/FDAsSentinelInitiative/UCM124701.pdf. Accessed September 22, 2011.

2. Holdford DA, Warholak TL, West-Strum D, Bentley JP, Malone DC, Murphy JE. Teaching the Science of Safety in US colleges and schools of pharmacy. Am J Pharm Educ. 2011;(4):75. Article 77. [PMC free article] [PubMed]

3. Johnson MS, Latif DA, Gordon B. Medication error instruction in schools of pharmacy curricula: a descriptive study. Am J Pharm Educ. 2002;66(4):364–371.

4. Dillman DA. Mail and Internet Survey: The Tailored Design Method. 2^nd ed. New York: Wiley; 1999.

5. Oppenheim AN. Questionnaire Design, Interviewing and Attitude Measurement. 1^st ed. New York: Pinter Publishers; 1992.

6. Schwarz N, Sudman S. Answering Questions: Methodology for Determining Cognitive and Communication Processes in Survey Research. 1^st ed. San Francisco: Jossey-Bass; 1996.

7. Johnson TP. Approaches to equivalence in cross-cultural and cross-national survey research. ZUMA-Nachrichten Spezial. 1998;3:1–40.

8. Schaffer BS, Riordan CM. A review of cross-cultural methodologies for organizational research: a best-practices approach. Organ Res Methods. 2003;6(2):169–215.

9. Mosavel M, Simon C, Van Stade D, Buchbinder M. Community-based participatory research (CBPR) in south africa: engaging multiple constituents to shape the research question. Soc Sci Med. 2005;61(12):2577–2587. [PMC free article] [PubMed]

10. Eysenbach G. Improving the quality of Web surveys: the checklist for reporting results of Internet e-surveys (CHERRIES) J Med Internet Res. 2004;6(3):e34. [PMC free article] [PubMed]

11. Richards L. Handling Qualitative Data. California: SAGE Publication Ltd; 2005.

12. Cortina JM. What is coefficient alpha? An examination of theory and applications. J Appl Psychol. 1993;78(1):98–104.

13. Cronbach LJ. Coefficient alpha and the internal structure of tests. Psychometrika. 1951;16(3):297–334.

14. Rust RT, Cooil B. Reliability measures for qualitative data: theories and implications. J Marketing Res. 1994;31(1):1–14.

15. Bagozzi RP, Yi Y, Phillips LW. Assessing construct validity in organizational research. Admin Sci Quart. 1991;36(3):421–458.

16. Campbell DT, Fiske DW. Convergent and discriminant validation by the multitrait-multimethod matrix. Psychol Bull. 1959;56(2):81–105. [PubMed]

17. Cronbach LJ, Meehl PE. Construct validity in psychological tests. Psychol Bull. 1955;52(4):281–302. [PubMed]

18. Haynes SN, Richard DCS, Kubany ES. Content validity in psychological assessment: a functional approach to concepts and methods. Psychol Assessment. 1995;7(3):238–247.

19. Lynn MR. Determination and quantification of content validity. Nurs Res. 1986;35(6):382–385. [PubMed]

20. Nunally J, Bernstein I. Psychometric Theory. 3^rd ed. New York: McGraw Hill; 1994.

21. Wang ST, Yu ML, Wang CJ, Huang CC. Bridging the gap between the pros and cons in treating ordinal scales as interval scales from an analysis point of view. Nurs Res. 1999;48(4):226–229. [PubMed]

22. Knapp TR. Treating ordinal scales as interval scales as interval scales: an attempt to resolve the controversy. Nurs Res. 1990;39(2):121–123. [PubMed]

23. Pagano M, Gauvreau K. Principles of Biostatistics. 2^nd ed. Belmont, California: Duxbury Press; 2000.

24. Nunnally JC., Jr Psychometric Theory. 2^nd ed. New York: McGraw-Hill; 1978.

25. Sears EL, Generali JA. Adverse drug reaction and medication error reporting by pharmacy students. Ann Pharmacother. 2005;39(3):452–459. [PubMed]

26. Johnson RB, Onwuegbuzie AJ. Mixed methods research: a research paradigm whose time has come. Educ Res. 2004;33(7):14–26.

Articles from American Journal of Pharmaceutical Education are provided here courtesy of
American Association of Colleges of Pharmacy