This study was approved as exempt research by the Institutional Review Board of Virginia Commonwealth University. The knowledge-based portion of the pre- and post-assessment consisted of 14 multiple-choice and matching questions related directly to the objectives of the activity. The confidence portion of the survey instrument consisted of items on which students rated their level of confidence in 5 areas of immunization practice. The students ranked their confidence level on a Likert scale on which 1 = completely confident, 2 = very confident, 3 = somewhat confident, 4 = not very confident, and 5 = not at all confident.
One week after completing the laboratory session, students were given a post-assessment with the same knowledge-based questions that were asked on the pre-assessment. Students were aware in advance of this assessment and completion of the assessment was required as part of their course grade. Four additional multiple-choice questions were added to the post-assessment to determine student participation in immunization activities thus far in their careers. This helped to determine in what types of activities related to immunizations the students had participated, how many patients they had immunized, and where and when they completed their immunization certificate training.
Performance on the knowledge-based questions was described using the percent of students answering each item correctly. The mean (standard deviation) and median (interquartile range) were also calculated for students’ overall score (or total percent correct) for knowledge-based questions. Students’ confidence before and after the laboratory session was described from both a continuous perspective (ie, mean and standard deviation) and a categorical perspective (ie, those responding “confident,” “very confident,” or “completely confident”). Because a student’s responses before and after the laboratory session could be linked by his/her name in Blackboard, statistical comparisons were conducted using a matched approach. Results were de-identified after matching occurred. McNemar’s test was used to compare whether a student answered an individual item correctly before and after the session. Only 9 of the 14 knowledge questions were included in the individual item analysis as the other 5 questions were matching and were unable to be easily separated based on computerized responses. A paired t test was used to compare the overall knowledge score (ie, % correct across all knowledge questions) before and after the activity. To allow for potentially non-normally distributed data, the overall knowledge score was also examined using a Wilcoxon rank sum test. Similar approaches were used for student confidence whereby McNemar’s test was used to examine confidence from a categorical perspective and a paired t test and Wilcoxon rank sum test were used for confidence from a continuous perspective. All statistical analyses were conducted using Stata/SE, release 12 (StataCorp LP, College Station, TX), with p ≤ 0.05 denoting significance.
One hundred twenty-five students completed the pre-assessment and 124 (98.4%) completed the post-assessment. The average score on the 14 knowledge-based questions on the pre-assessment was 56% (14.3%-100%) and on the post-assessment was 87.4% (50%-100%). Scores improved for 95% of students, with 4 students scoring the same and 2 students scoring lower on the post-assessment. The greatest improvement in scores between pre- and post-assessment was 78.6%.
The individual comparison of questions 1-9 is reported in . Questions 10-14 dealt with matching a vaccine to the appropriate age group, assuming the patients are otherwise healthy with no comorbid conditions. There was a positive shift in each of these 5 questions from pre- to post-assessment. A statistical analysis of these questions was not performed due to the formatting of Blackboard questions (matching). These 5 questions focused on newer influenza products, pneumococcal, and shingles vaccines.
Pharmacy Students’ Knowledge of Immunizations Assessed Before and After Completing an Active-Learning Laboratory, N=103
Students’ confidence improved in all of the 5 areas assessed. The 2 areas on the post-assessment in which students reported the greatest level of confidence were in their ability to appropriately screen and determine eligibility and appropriateness for a patient to receive the influenza vaccine (91.1%), and in their ability to document administration of a vaccine in general (95.1%). The most significant improvement in confidence was in their ability to appropriately screen and determine eligibility and appropriateness of a patient to receive the pneumococcal vaccine (+51.2%). Confidence data are reported in .
Students’ Confidence in Immunizations Before and After Completing an Active-Learning Laboratory, N=103
Approximately 96% of students had completed an immunization certificate program. The majority (51.2%) had completed the program through the school of pharmacy program. Employer-sponsored programs (31.7%) represented the second most popular method of receiving training. Sixty-nine percent of students completed this training within 6 months of the survey, while about one-quarter had completed training more than 12 months before the survey ().
Completion of Immunization Certificate Training Program
To assess how students were using their immunization training skills, the survey instrument asked questions about the types of activities in which the students had participated and the number of patients the students had personally immunized. Activities could include patient education, patient screening, and administration. Of those students who had completed certificate training, 22% had not participated in any activities related to immunizations, while approximately 60% had administered immunizations. However, 45% of all respondents reported they had not personally immunized any patients. Most students who had personally immunized patients, had immunized between 1 and 20 patients. Two students, both of whom had completed the certificate program more than 12 months prior to the survey, reported personally immunizing more than 100 patients.
Students were also asked to complete an evaluation of the laboratory session. This evaluation consisted of 8 questions: 5 questions using a Likert scale on which 1 = strongly disagree, 2 = disagree, 3 = undecided, 4 = agree, and 5 = strongly agree; 1 question regarding the overall laboratory rating using a Likert scale with 1 = very poor, 2 = poor, 3 = fair, 4 = good, and 5 = excellent; and 2 open-response questions for students to comment on what they liked and did not like about the laboratory activity. Students were asked to rate items such as organization, relevance, appropriateness, contribution to professional development, and satisfaction.
Sixty-three (51%) students completed the evaluation. The majority felt the laboratory activity was well-organized, presented at an appropriate level, and relevant to pharmacy practice. Respondents also felt it contributed to their professional development and 87.3% rated the overall activity as good or excellent. When students were asked what they liked about the activity, they said they appreciated that the most common immunizations were included and they appreciated the practicality of the activity, including the patient cases and product demonstrations. They thought it was a helpful refresher in a low-stress format with knowledgeable facilitators. Students suggested that the activity include more practice administering injections. When discussing what they did not like about the activity, some pharmacy students stated that they felt rushed and that there was some redundancy across the laboratory stations.