Overall, the test results appear to be closely related to whether students took an herbal medicine class. Results could also be influenced by 3 other factors: whether students had completed their pharmacy practical experience; anomalies in how the test was conducted at each school, and/or whether residents of different provinces have more background knowledge of herbals. Each of these points is explored in detail, along with a discussion of the applicability of the results to practice.
Whether a student had taken an herbal class was the most significant factor associated with how well they performed on the herbal medicine knowledge test. British Columbia and Alberta are the schools that had the highest enrollment in an herbal class—either because it is mandatory or a well-attended elective—and these were the schools with the highest mean herbal medicine knowledge scores. Our review of the herbal curricula of each school (Table ) helps to explain our data, showing that the schools with the highest attendance in classes in which a large amount of material on natural health products was presented scored the highest.
A key discussion point is whether a 6% difference in scores between students with or without an herbal course translates to a significant difference in practice. Given that the test evaluated the most basic knowledge of herbs, it is not surprising that there was such a small difference between the scores of those who had taken a dedicated course on herbs and those who had not. Ultimately, this may be a positive finding from the patient perspective because it means that in many pharmacy schools, even students who do not take an herbal course still graduate with basic knowledge about common herbs. Had the test been longer, and evaluated other, more complicated safety, efficacy, and regulatory issues, the gap between students with and without additional specialized training in natural health products may have been wider. What our test shows is that graduates of some pharmacy schools have modest knowledge about common herbs and that this is an issue that needs to be addressed. Students scores on an assessment of basic knowledge herbal products are generally lower than their average academic results on other topics. Our results suggest that a mandatory course with herbal content could be the answer.
Whether or not a student completed their pharmacy practical experience was also a significant factor associated with how well they performed on the herbal medicine knowledge test. It was postulated that completion of a pharmacy practicum could affect test scores because students might familiarize themselves with some of the more common herbs as part of the practicum experience, and our analysis supports this hypothesis.
Differences in curriculum designs among the pharmacy schools in the study allowed us to explore the impact of pharmacy practicum on herbal knowledge test scores. Table shows that in both British Columbia and Alberta, mean test scores for students were essentially the same regardless of whether or not students had completed their practicum. This would suggest that, contrary to our hypothesis, completion of pharmacy practicum prior to taking the herbal knowledge test did not appear to have an effect on the test score. However, in Alberta, 1 group had the herbal course and no practicum, while the other group had the practicum but no herbal course. Since our results indicate that both may have influenced the test scores, this could explain why the 2 groups of Alberta students scored so similarly. In contrast, students in the second cohort in British Columbia had completed both an herbal course and a practicum.
At the Nova Scotia school, all students who participated had already completed their practicum, and their lower scores would also seem to contradict our hypothesis. In contrast, Ontario students were given the herbal knowledge test immediately prior to beginning their pharmacy practicum, and this lack of a practicum might account for the Ontario students' relatively low scores, but there may also be other factors that we have not yet identified.
Due to the specific directives of different ethics boards, as well as various curricular constraints at each school, the test was not distributed in a uniform manner across schools, which may have affected test scores. In fact, the school with the highest score, British Columbia, was the only school where the test administration methods were executed exactly as planned.
In Alberta, the ethics board required that the tests be given out at the end of class instead of at the beginning, to give students the opportunity to leave if they did not want to participate. This may have decreased the response rate, and increased the mean, as the students who stayed to participate may have been more interested/knowledgeable in herbal medicine.
Many students at the Nova Scotia school did not fully complete the test, leaving, in some cases, up to 13 questions blank. This pattern of not completing the test was not seen at the other schools and may have been because the performance incentive (highest scoring students entered in a draw for a $100 prize) was forbidden by the ethics board in Nova Scotia, and therefore not offered at this school. However, overall participation rates in the study were comparable to other schools.
In Quebec, the tests were given out at the end of a pharmacotherapy examination, instead of at the beginning of a core class. This was due to curricular constraints, and likely considerably decreased the response rate and mean score, as the students who stayed after the examination to participate may have been tired and less focused on completing another test, especially one that did not confer any academic benefit. Due to these adverse conditions and the very low response rate, it is not possible to draw any concrete conclusions on the herbal knowledge of the Quebec students.
In Ontario, almost all students present in the class participated in the test, but only approximately 60% of students enrolled in the class were present. There is no way of knowing whether the students who missed class had a higher or lower knowledge of herbals than those who attended.
Overall, it is likely that the inconsistencies in test distribution lowered the response rates and may have affected mean test scores, perhaps with the exception of Alberta, where the students who volunteered to complete the test at the end of class may have had more interest and knowledge about herbal medicine. A key challenge for future studies of this nature is to better standardize the administration of the tests.
The culture of a province with respect to natural health product use may also have influenced the students' scores on the test. How much a population in a certain part of Canada collectively endorses the use of natural health products may affect how much general knowledge residents have simply due to their lifetime exposure to public information about herbal products.
A 2005 national survey assessing the use of natural health products in Canada found that residents of British Columbia and Alberta were the most likely to say they used a natural health product on a daily basis.1
In contrast, Atlantic Canadians and Quebeckers were more likely to be seasonal users of natural health products. Atlantic Canadians were more likely to be unfamiliar with natural health products, whereas British Columbia, Quebec, and Ontario residents were more likely to rate themselves as very familiar with natural health products. Atlantic Canadians also reported the lowest personal use of NHPs.1
The schools that scored the highest were situated in provinces with the most frequent use of natural health products, so it would make sense for students in these provinces to have more background knowledge; however, it is not clear if that made a significant difference to students' performance on our test. In addition, based on the 2005 survey finding that Ontarians identified themselves as very familiar with natural health products and yet students in Ontario produced the lowest average test scores, it would seem that provincial culture cannot predict pharmacy students' scores as well as having completed a course with herbal content.
This study was based on the assumption that pharmacy students who performed poorly on our standardized test of herbal knowledge would be unable to identify and prevent drug-related problems associated with herbal medicines. It is possible that this may be a false assumption, given the ease with which health product information on a variety of topics can be obtained from textbooks or evidence-based online databases. However, it could be argued that pharmacists still require fundamental knowledge in order to know how to evaluate the safety and effectiveness of an herbal product in a timely manner. Given the busy work settings of most pharmacists, it may not be possible to take the time to search for information to answer every patient question about herbal medicine. Thus, basic knowledge of the efficacy of common herbs and key safety issues (ie, herbs with “red flags” or specific known drug interactions) appears to be necessary in order for pharmacists to adequately counsel patients in an efficient manner. For example, St. John's wort is known to have a clinically significant interaction with warfarin and thus patients should routinely be counseled to avoid this combination.31-33
Based on our data, it is not possible to say with confidence which style of herbal instruction is the best. However, it appears that ensuring specific teaching about herbs needs to occur in a manner such that all students are exposed to the topic. Further research needs to explore the impact of different educational strategies for teaching this topic.
As alluded to above, this study suffered from several methodological limitations such as distributing the test at different times of the year for each school, the variety of venues in which the tests were distributed, and the incentives offered. Additional limitations may include a self-selection bias. Students who voluntarily completed the test may not be representative of other fourth-year students in that they may have had a greater interest in and thereby greater knowledge of herbal medicine. Furthermore, because this test did not count towards academic standing, students may not have given their best effort, potentially resulting in an inaccurate assessment of their knowledge. The performance incentive was provided to minimize this effect. Another topic for discussion is whether this test is a valid test of herbal knowledge. The results of our pilot test support the test's face validity. Moreover, the questions that were used on the test are similar to those that have already been used on examinations at 4 different Canadian universities. All of these points support the validity of the test to adequately assess pharmacy students' knowledge; however, a longer test would have been able to assess a wider range of herbal knowledge.
One additional limitation of the test was that it was comprised of only 14 questions, thus restricting our ability to test knowledge of a wide variety of products. We believe that the test is still instructive because the questions were primarily related to efficacy and safety, which are the 2 primary domains of a pharmacists' scope of practice. Thus, while the test was not long, it assessed the basic knowledge that it was designed to test. The results may not indicate that students with higher scores have sufficient knowledge, but they do signify that there probably is a gap in the herbal curricula of the schools that did not perform as well.