|Home | About | Journals | Submit | Contact Us | Français|
Objectives. To implement and assess the effectiveness of a 15-week applied research elective that introduced students to secondary database analysis in clinical pharmacy.
Design. In small groups, students learned, planned, developed and completed a secondary database study to answer an original research question. During one semester, they completed a basic research proposal and Institutional Review Board application, created and analyzed a National Health and Nutrition Examination Survey (NHANES) sample dataset, and reported the results in an abstract and poster presentation.
Assessment. All deliverables resulted in high grades. Mean scores on a survey conducted following completion of the course revealed that students strongly agreed or agreed that they had high levels of confidence about performing research-related tasks. Eight student groups delivered poster presentations at professional conferences.
Conclusions. Within one semester, student pharmacists with no or little research experience completed original research projects that contributed to pharmacy practice knowledge. They felt highly confident doing research-related tasks, and successfully disseminated their studies beyond the classroom.
Research skills enhance pharmacists’ professional vitality and deal with changes in science and practice.1 The scholarly and practice communities in the biomedical fields are increasingly aware of the key role that pharmacists can play in discovery, assessing the complexities of medication use and misuse, health outcomes evaluation, evidence-based practice, and policy.2,3 While translational research progressively shows its significance in transforming health outcomes, clinical pharmacists can claim an important role in research aimed at enhancing the adoption of best practices in the community, including in the cost-effectiveness of prevention and treatment strategies.2,3 Various pathways have been defined for preparing clinical pharmacists (those with a doctor of pharmacy [PharmD] degree) to enter the competitive realm of funded research and to actively participate in translational and health outcomes research. These pathways include post-PharmD education in the form of fellowships, residencies, and advanced degrees at the masters and doctoral level.3,4 In order to increase the number of pharmacy graduates pursuing these pathways, it is important that we explore ways of enhancing research experiences in the PharmD curriculum.
Independently of the interest in a research-intensive career, graduates of pharmacy programs that offer enhanced research experiences can more successfully approach the changing nature and complex demands of the health care system. Enhanced research skills could also be instrumental in evaluating the introduction of new medical technologies and products. Professional organizations, accrediting bodies, and think tanks have highlighted the benefit of research experiences for students.4-6 For example, the American Association of Colleges of Pharmacy’s 2011-2012 Argus Commission noted that “student research skills help develop inquisitive pharmacists with attributes required for scholarly clinical practice.”7,8 The 2016 Accreditation Council for Pharmacy Education (ACPE) accreditation standards and the Center for Advancement of Pharmacy Education (CAPE) 2013 statement of educational outcomes acknowledge and encourage the inclusion of research-related knowledge and skill-building activities within the curriculum.9,10 The support for research competencies has also been recognized by international organizations.7 Considering these forces, applied research training is not yet standardized in PharmD training programs in the United States.1,11
While pursuing their PharmD degree, students should be encouraged to take advantage of the rich research environment offered by their program. Social cognitive theories have ascertained that providing research-related opportunities designed to foster student pharmacists’ self-efficacy and professional competencies can increase their interest in future involvement in research. Furthermore, these collaborations can attract and commit the students to scholarly and research-oriented activities and pathways once they graduate.12,13 There is documented evidence of the success in fostering critical thinking using different models, strategies, and activities in which student pharmacists engage in research, including summer electives, required applied projects, senior projects, practice research, and a capstone research program.14-18 However, only 14%-35% of programs offer courses in which students conduct a complete research project.1,11 A complete research project is one in which students design and conduct the research and report their results.19 Furthermore, faculty members may often find that students struggle with applying statistics knowledge as they try to interpret the methods and statistical results in published research articles during discussion sessions. Less than half of pharmacy programs in the United States require students to conduct higher-level statistical application such as selecting data analysis procedures (27.8%), analyze data (30.4%), or interpret research findings (46.8%).11
This disconnect was the initial inspiration for designing a research elective at the Nova Southeastern University College of Pharmacy (NSU COP) to teach students much needed research-related skills. The elective course, Applied Secondary Database Analysis, has been offered annually to PharmD students at the NSU COP since 2013. The goal of this course is to give students the skills and confidence to conduct research-related tasks by planning and completing a research project within one 15-week semester. This paper describes the implementation, evolution, and assessment of this research elective course. The overall goal of the course was to provide the students with skills and confidence to engage in research activities in their future postgraduate training and/or professional career. In this paper, we describe our assessment of the course including the quality of the required research assignments, student confidence in doing required research-related tasks, and course satisfaction.
A two-credit research elective course was designed for students who have a basic foundation in searching and evaluating the primary literature and in research design and statistical methods. Students eligible to register for this elective course were third-year entry-level and second-year advanced-standing students who had completed and passed the research design and biostatistics courses. (The NSU COP offers two PharmD programs: the traditional four-year entry-level program and the three-year advanced standing option for individuals with foreign pharmacy degrees.) At this point in the curriculum, students have already taken the core drug information, biomedical literature evaluation and pharmacoeconomics and outcomes research courses. Figure 1 shows the timeline of this pathway in the NSU COP curriculum.
To complete an entire research project in one semester, students had to write a research protocol for a secondary database analysis study; complete a study-specific institutional review board (IRB) application; develop the proposed study by systematically gathering, managing, analyzing, and evaluating the appropriate data from the US National Health and Nutrition Examination Survey (NHANES) database; report results in an abstract and scientific poster; and defend the project by delivering a poster presentation to a research faculty audience. Students planned and developed an original research study in groups of two to four. Due to a limited availability of research questions, the course has been continuously capped at 12 students in a class size of 230.
The course learning objectives were designed to be at the highest level of difficulty (application, analysis, synthesis, and evaluation) and add new content to the curriculum.20 The course learning objectives are listed in Table 1. Furthermore, the course was designed to meet the college’s educational outcomes, which are also listed in Table 1.
The elective course was designed so that the first half of the semester was spent planning the research project and the second half developing the study and reporting the results. A “didactic-to-laboratory” format was used in which presentation of didactic material was followed by a laboratory session(s) where the material was subsequently applied. Table 2 depicts the semester timeline and sequences. Didactic sessions in weeks one through six focused on developing the skills that students needed to write the background, objective, methods, limitations, subject safety, and expected results sections of the research protocol. Students used laboratory sessions to write sections of their protocols depending on the topics discussed in class that particular week. Week 1 was spent writing the background and study objectives for their research, while weeks 2 through 6 were spent planning and writing the methods section. Week 7 focused on writing the subject safety, limitations, and expected results sections. The course coordinator provided weekly or bi-weekly feedback to students regarding the quality of their research protocol drafts. Students were reminded to follow the plagiarism rules described in the course syllabus.
During week 7, students learned about the Nova Southeastern University Institutional Review Board (NSU IRB), its role in the conduct of ethical research involving human subjects, and its protocol review process. All students had already obtained Collaborative Institutional Training Initiative (CITI) training as part of the Research Design and Biostatistics course. Even though the NSU IRB has deemed studies using the NHANES data to be non-human subject research and outside of their purview, students were required to complete an application for a grade but did not have to submit the application to the university’s IRB. The application and the finalized research protocol were due before the week 8 session began.
Weeks 8-11 focused on the development of the studies. Students learned how to download, merge, and stack the NHANES datasets using the IBM SPSS, Inc. statistical software, how to conduct data manipulation (ie, recode into same variable, recode into different variable, compute a variable), and run basic statistical analyses (eg, chi-square, t test, one-way analysis of variance) in the respective didactic sessions, and were given laboratory time to apply the material to their studies. The role of the course coordinator at this stage was mostly to provide assistance in statistical software troubleshooting. As shown in Table 2, only one of the class periods was a didactic session, while the rest were laboratory sessions.
Weeks 12-15 focused on the reporting and dissemination of study results. Once again, only one of the class periods was a didactic session, while the rest were laboratory sessions. Students reviewed the abstract submission and poster presentation instructions of the American Society of Health-System Pharmacists (ASHP) Midyear Clinical Meeting (MCM) in the didactic session and then used them as a guide to develop their own abstract and poster during the laboratory sessions. At this point, the course coordinator closely monitored the quality of the interpretation of the studies’ results and status of abstract and poster writing. Also, the course coordinator had discussions with each student group focusing on how their study results fit into current clinical practice and how to formulate discussion and conclusion statements. For the final poster presentation sessions in week 15, each student group displayed their poster and delivered a 10 to 15 minute oral presentation to the audience. The audience was composed of NSU COP students and research faculty members. Each student group presented separately and each group member presented a separate section of the poster. Each presentation was followed by a five- to 10-minute scientific discussion moderated by the course coordinator.
Within the 15-week course, six of the nine didactic sessions and all the laboratory sessions were taught and facilitated by the course coordinator. This coordinator was a clinical pharmacist and outcomes researcher, with prior research experience conducting secondary database analysis studies using the NHANES database. Most guest lecturers teaching the remaining didactic sessions were NSU COP faculty members with PhD degrees who had research experience in the sociobehavioral and administrative science fields. An NSU librarian and IRB representative delivered their respective lectures in two didactic sessions (Weeks 1 and 7). In the 2015 semester, topic experts volunteered to assist in creating a research question for one group and provided clinical expertise through project planning and development. Along with the course coordinator, topic experts graded their group’s research protocol, abstract, poster, and poster presentation. Their role took each expert approximately four to six hours. Most topic experts were pharmacy practice faculty members and one was from the pharmaceutical sciences department.
Students were responsible for attending class, using laboratory session time to develop the required assignments, meeting assignment deadlines, and delivering a final poster presentation. One student in each group volunteered to be the study leader and was charged with submitting all group assignments. Study leaders were listed as the first investigator in the abstract and on the poster. All group members were responsible for working in all stages of the project as a team. The course coordinator and topic expert, where applicable, served as the study mentors.
Four key factors in the design of this course allowed student groups to plan and complete their research project within one semester. First, planning research questions for student groups prior to the start of the semester; second, creating “didactic-to-laboratory” topic sequences in which lecture material specifically focused on teaching skills that students could immediately apply to their projects during laboratory sessions; third, having access to existing clinical data such as that of the NHANES database; and fourth, having access to user-friendly statistical software ie, IBM SPSS, Inc.
Six pharmacoepidemiology research questions were written for the 2013 and 2014 semesters. The questions evaluated racial/ethnic medication use disparities among patients with type 2 diabetes. These questions were chosen because they were in line with the course coordinator’s research interests. In the 2015 semester, however, the course coordinator worked individually with five NSU COP faculty members who were topic experts to create a research question that had the following characteristics: (1) was an original idea, (2) addressed a current research need, (3) was drug-related, (4) could be answered by the NHANES data, and (5) could be answered within one-semester. Research questions were assigned to groups based on the students’ topic preferences.
The “didactic-to-laboratory” sequence delivery format that was used was key in giving students the skills needed to plan their research protocol and complete their study within one semester. Each didactic session ranged between 30 and 90 minutes and was followed by a one- to three-hour active learning laboratory session. For example, as shown in Table 2, the one-hour “Planning data management” didactic session in week 4 was followed by a one- and two-hour laboratory session held in weeks 4 and 5, respectively. Didactic content was streamlined so that students learned only the necessary skills that applied to completing their research project. Even though the 19 laboratory sessions offered plenty of in-class time to develop the assignments, students had to spend an additional one to six hours per week outside of class in order to complete the assignments on time. Didactic material was delivered using a live lecture format, and Blackboard was used as the learning management system. Learning materials were distributed as PowerPoint handouts. Additional materials provided throughout the semester were a research protocol outline template, and a sample research protocol, abstract, and poster that students could refer to as a guide.
The NHANES database was selected as the data source of choice for several reasons. The data is publicly available, the website and codebook are easy to follow and teach, and the survey data collection is cross-sectional in design (www.cdc.gov/nchs/nhanes). The NHANES is a program of studies designed to assess the health and nutritional status of noninstitutionalized civilian adults and children in the United States. It is unique because unlike other national surveys, it combines interviews and physical examinations. Survey contents include demographic, socioeconomic, and health-related questions including the history of chronic diseases such as diabetes and cardiovascular conditions. A list of prescribed medicines and supplements is also collected from each survey participant. The examinations consist of medical and physiological measurements as well as laboratory tests.
Lectures were delivered in a computer laboratory equipped with IBM SPSS, Inc., statistical software. This software is generally considered user friendly and easy to teach to students when conducting data management and statistical procedures. As they learned to use the software, students were encouraged to use YouTube “how to” videos as a supplement to the instruction provided in class.
Student learning was assessed through five skill-based graded assignments. The following assignments, comprised of different weights, counted toward the final grade: research protocol (25%); IRB application (10%); study dataset, data manipulation syntax, and statistical analysis output (15%), abstract (15%), and poster and oral presentation (25%). A self- and peer-evaluation were required from all students individually and were also part of the final course grade (10%). As shown in Table 2, assignment deadlines occurred at weeks 8, 12, 14 and 15. Table 3 shows the averages grades for both semesters. The research protocols, abstracts, and posters were graded by the course coordinator based on the quality of spelling, grammar, and technical writing (weight of 10%), and the scientific (70%) and clinical (20%) content. The topic expert graded the quality of spelling, grammar, and technical writing (10%) and clinical content (20%) for the 2015 cohort. After getting graded feedback on all written assignments, student groups were allowed to resubmit the protocol, abstract, and posters for a higher grade if they addressed the issues which had been identified with their respective assignment. Only the resubmitted assignments counted toward the final grade in the 2014 semester while the average of the final and resubmitted assignments was calculated into the final grade in 2015. First submission grades generally ranged between 5 and 30 percentage points lower than grades on the resubmissions. Point deductions in the initial submissions of the protocol, abstract, and posters occurred mostly due to missing information or suboptimal descriptions in the methods or results sections, where appropriate. Except for the peer evaluation and oral poster presentation, the same grade was given to all group members developing the same project.
A 35-item survey instrument was developed for the 2014 offering of the course. The objective of the survey was to evaluate the effect of the course on students in two areas: (1) confidence and confidence levels in doing research-related tasks, and (2) course satisfaction and its perceived impact on future career options. There were 28 scaled items (Table 4) and five open-ended questions. Five additional scaled items and one additional open-ended question were added to the 2015 semester survey for a total of 40 scaled items and six open-ended questions, which are summarized in this paper and presented in Table 4. The survey instruments also gathered other data including demographic student information and research experience prior to taking the course.
Scaled items evaluated confidence (17 items) and confidence levels (8 items) in performing research-related tasks. Eight course satisfaction items assessed whether the course enhanced basic research and biostatistics knowledge, if course expectations were met, the effect of research experience on any prospective postgraduate training portfolio, the likelihood of involvement in future research activities and satisfaction with course assignments. Confidence and course satisfaction items used a four-point Likert scale on which 1=strongly agree and 4=strongly disagree. The eight items assessing confidence levels in doing research-related tasks used a four-point Likert scale on which 1=very confident and 4=not confident. The survey instruments were administered to students at the end of the 2014 and 2015 semesters. Participation was voluntary.
All scaled-item responses are described as means and standard deviations. The statistical hypotheses tested whether average confidence, confidence levels, and course satisfaction were the same for the 2014 and 2015 offerings of the course. Mean score differences were compared using the independent t test (alpha= 5%). IBM SPSS, Inc Statistics was used to conduct the analysis. Qualitative data from open-ended items are described. These survey instruments and the student research protocols were granted exempt status by the NSU IRB in 2014 and 2015.
Student demographics are shown in Table 5. Over half of the student sample (54.2%) had no experience in conducting any of the listed research-related tasks prior to taking the course, while a maximum of 12.5% of students had done one or all of the mentioned tasks. The top two reasons for taking this course were to gain research experience (92%), and wanting to make a future residency application more competitive (50%). Three-quarters (75%) of students were interested in pursuing postgraduate training, half in pursuing a residency (50%).
The survey response rate in both student cohorts was 100% (n=24). As shown in Table 4, mean scores demonstrate that students felt confident performing the specified research-related tasks. Similarly, mean scores also showed high levels of confidence in conducting these tasks (Table 4). There were no significant differences in the mean scores across both semesters, which demonstrated consistency in confidence and confidence levels with different student cohorts (p>0.05 for all). Mean scores in the 2015 cohort show high confidence levels in performing the same tasks as part of a research team.
As shown in Table 4, course satisfaction results were positive. Mean scores demonstrated that students strongly agreed or agreed that this course enhanced the knowledge learned in the research design and biostatistics course and that it made them more competitive applicants if they decided to pursue postgraduate training in the future. At the time this course was offered, students were also taking their required pharmacy practice seminar course in which they conduct a primary literature review of an assigned topic and culminates as a poster and platform presentation. Students also strongly agreed or agreed that the elective course helped in this learning experience as well. There was no difference in the mean responses in scaled course satisfaction items across semesters (p>0.05 for all). Scaled item mean scores showed that the course met and exceeded student expectations. A majority of students (75%) mentioned their favorite activity was learning about IBM SPSS, Inc, conducting data manipulation, and/or running statistical analyses. Among least favorite activities were creating the study dataset, completing the IRB application, and having to correct data manipulation syntax. However, students understood these tasks to be important in the research process. Overall, students reported feeling challenged and said this process was a rewarding experience. They also mentioned that applying the lecture material in the laboratory sessions facilitated their learning. Finally, students mentioned that taking this course helped enhance their understanding of the primary literature as they were completing their pharmacy practice seminar project. This feedback was consistent across student cohorts.
Over the time this course has been offered, 11 original secondary database analyses have been completed. A total of 32 students have taken the course since 2013. To date, nine studies have been submitted and presented as posters in three ASHP MCM, one Student National Pharmaceutical Association, and one Making a Difference in Infectious Disease (2015) international conference. Several students applying for residency positions presented their research project during the interview process. Among those who applied for postgraduate year 1 (PGY1) residency programs, match rates were 100% (four out of four), 86% (six out of seven), and 62.5% (five out of eight) in the 2013, 2014, and 2015 semesters, respectively. Three additional student groups were writing their manuscripts for future publication.
An elective course titled Applied Secondary Database Analysis successfully taught 11 student groups (32 student pharmacists) how to plan and develop an original secondary database study within one semester. Students took this research elective with the intention of gaining research experience and a competitive advantage when applying for a residency position. They had none to very limited prior research experience yet consistently reported high levels of confidence in conducting all the research-related tasks learned in the course. Course satisfaction rates were also high. The majority of these research projects were presented outside of the classroom in national or international professional meetings and/or to prospective pharmacy residency programs while taking their advanced pharmacy practice experiences. A majority of students applied for and secured a residency. In addition to using the learned skills to plan and develop a study, these future graduates will know how to use a popular statistical software (IBM SPSS, Inc.) and a national, publicly available database (NHANES) during postgraduate training and in their future careers.
There were many benefits to taking this course. Enrolled students learned skills at the highest level of Bloom’s taxonomy. They applied statistical, literature evaluation, and clinical knowledge gained from prior pharmacy courses as they planned, developed, and completed their research projects. In the same process, they also analyzed data and evaluated their study results, and synthesized a research plan and new scientific information. Additionally, interpersonal skills were further developed as students learned to work with their group members and faculty mentors. Students learned how to present scientific information in written and oral formats, and got detailed feedback on their written and oral communication abilities. Most of all, they learned and experienced the scientific process including planning, development, and dissemination. They also became experts in a particular clinical topic and will most likely better serve the patient population they evaluated in a future clinical role. Faculty members benefited by having a student group answer their research question within a reasonable timeframe. Also, the pharmacy program and university got scholarly-related exposure at professional conferences.
All aspects of this course are transferable to other programs. First, many students come into pharmacy programs with an interest in having research experience. Second, pharmacy programs have faculty members with research training and expertise who can create original research questions and teach students how to develop a research protocol, build a data set, manage and analyze data, interpret statistical results, and report and disseminate research. Third, this course must be located in the curriculum at a time when students are already proficient in performing primary literature searches and can apply research design and statistics concepts to the pharmacy literature. Fourth, student pharmacists can be given access to free (or low cost) health data via national databases such as NHANES. Fifth, statistical software programs like IBM SPSS, Inc Statistics are available in many universities. Overall, the financial resources required by the institution to offer this course were reasonably low. In the future, the size of this course will be dictated by the available research questions and resources.
The Applied Secondary Database Analysis elective course successfully taught student pharmacists with limited to no research experience to plan and develop an original research project within one semester. Preselected research questions, strategic deadlines for research protocol planning and development, and access to an established comprehensive data source and user-friendly statistical software were key components in achieving the course objectives on time. Delivering material in didactic lectures that was necessary to plan and develop the research projects and subsequently having students apply what they had learned during laboratory sessions was also an essential approach used in the course. Students with no or very limited prior research experience reported high levels of confidence and satisfaction in executing the required research-related tasks. This course would be easily transferable to other pharmacy and health professional or graduate programs at a low cost.
No funding was obtained for this study. There are no conflicts of interest to report.