|Home | About | Journals | Submit | Contact Us | Français|
This study reports the secondary analysis of evaluation data from an innovative high school biology curriculum focused on infectious disease (ID) to examine the health literacy implications of teaching claims evaluation, data interpretation, and risk assessment skills in the context of 21st-Century health science. The curriculum was implemented between 2010 and 2013 in Biology II classes held in four public high schools (three in Massachusetts and one in Ohio), plus a private school in Virginia. A quasi-experimental design was used in which student participants (n = 273) were compared to an age-matched, nonparticipant, peer group (N = 125). Participants in each school setting demonstrated increases in conceptual content knowledge (Cohen’s d > 1.89) as well as in understanding how to apply scientific principles to health claims evaluation and risk assessment (Cohen’s d > 1.76) and in self-efficacy toward learning about ID (Cohen’s d > 2.27). Participants also displayed enhanced communication about ID within their social networks relative to the comparison group (p < .05). The data show that integrating the claims evaluation, data interpretation, and risk assessment skills critical for 21st-century health literacy health into high school biology classrooms is effective at fostering both the skills and self-efficacy pertinent to health literacy learning in diverse populations.
The National Assessment of Adult Literacy has concluded that 36% of US adults have limited health literacy (HL; http://nces.ed.gov/naal/). To be health literate is to possess distinct skills including the ability to “evaluate information for credibility and quality,” “interpret test results,” and “analyze relative risks and benefits” that are themselves contingent on the ability to use visual, computer, information, and numeric skills to understand current issues in health and disease (http://nnlm.gov/outreach/consumer/hlthlit). HL empowers decision making in pursuit of healthy behaviors, care seeking, and management of chronic conditions. In contrast, limited HL, which is especially prevalent in underserved communities, produces a population incapable of effectively articulating their needs and acting appropriately on subsequent recommendations (Berkman et al., 2011; Koh et al., 2012; Nielsen-Bohlman, Bohlman, Panzer, & Kindig, 2004), placing an economic burden on the U.S. taxpayer that has been estimated to be in excess of 200 billion dollars a year (Parker & Ratzan, 2010; Vernon, Trujillo, Rosenbaum, & DeBuono, 2007). There have therefore been calls to develop interventions to teach the underlying skills that will foster HL in the population (A. K. Cohen & Syme, 2013).
As can be seen from Figure 1, slightly adapted from the Agency for Healthcare Research and Quality (AHRQ) report (Berkman et al., 2011), HL is considered to encompass topic-specific content knowledge and behavioral skills. In the AHRQ model, skills in seeking, identifying, interpreting, and integrating new health information support knowledge and accurate risk perception, which in turn foster the attitudes, social norms, and self-efficacy essential for engagement in promoting healthy behavior, such as obtaining cancer screening.
Health literacy interventions are typically focused on preventing or managing specific diseases, or promoting specific behaviors, and therefore fall under the rubric of “trainings” because “we can predict … the specific situations in which people will use what they learn” (Posner, 2004, p. 70). Knowing the desired outcome a priori allows the health educator to curate the appropriate knowledge and risk information pertinent to the intervention. As a consequence, the claims evaluation and risk assessment skills needed to keep pace with rapidly evolving health care recommendations in general can fall by the wayside. These skills are highly generalizable because “we cannot predict … the situations in which people will use what they learn” (Posner, 2004, p. 70), and as such they are particularly pertinent to young people who cannot know what health questions they will face in the future. These young people need a firm foundation in inherently “generalizable” HL skills such as evaluating claims, interpreting data, and assessing risk, and acquiring these skills requires a different pedagogical approach to that employed in health education trainings. Specifically, promoting the ability to transfer skills readily among different situations requires providing opportunities to practice these skills extensively in diverse health- and disease-related contexts (Detterman & Sternberg, 1993; Haskell, 2000; Merriam & Leahy, 2005).
In Figure 2, we present our framework, a modification of the AHRQ model, that has been restructured to exemplify how (1) situation-neutral generalizable skills such as claims evaluation are a target for “preclinical’ educational interventions that could (2) affect HL attributes so that they can (3) be marshaled into trainings to influence situation-specific behaviors in health areas as disparate as cancer screening, flu shots, and low-fat diets. These generalizable HL-relevant skills align naturally with the reasoning skills critical for 21st-century science literacy already taught at the high school level using appropriate pedagogy, suggesting that the high school biology classroom may be an effective venue in which to learn HL-relevant skills too (Quinn, Schweingruber, & Keller, 2011). In fact, high school students particularly value learning about science in the context of health and disease, becoming deeply engaged with learning when they see the science behind their real-world experiences modeled in the classroom (A. K. Cohen & Syme, 2013; Jacque et al., 2013). Moreover, the high school classroom is the last venue in which entire cohorts of the U.S. population gather together to learn. Hence, incorporating health claims evaluation and risk assessment into the high school biology curriculum could be not only productive but also extremely timely (A. K. Cohen & Syme, 2013).
However, very few high schools teach biology in the context of 21st-century issues in health and disease, in large part because teachers lack cutting-edge, life-relevant health science curricula as well as access to the support needed to bring it into their classrooms, and then to teach it from an HL perspective. The Great Diseases Project has responded to this challenge by creating a collaborative learning community of content specialists from Tufts University School of Medicine and Boston area teachers, who have designed effective HL-focused biomedical and health curricula for the high school biology classroom as well as methods to support teachers implementing the curricula in their classrooms (Jacque et al., 2013, Malanson, Jacque, Faux, & Meiri, 2014). Targeted to Biology II, commonly an elective course without a prescribed syllabus, the Great Diseases curriculum focuses on four globally significant diseases (infectious disease (ID), cancer, metabolic disease, and neurological disorders). Each stand-alone module occupies about 6 weeks and teachers can implement any combination (http://sites.tufts.edu/greatdiseases/). ID, which is described here, is both globally and individually significant for high school students: ID remains the major cause of mortality in children worldwide, and in the United States, annual deaths have doubled since 1980. The increased incidence of emerging IDs (e.g., Ebola) appears largely due to changes in human behavior, suggesting that their impact going forward may critically depend on the HL capacity of the population to seek out, interpret, and act on information to reduce the impact of infection (Keesing et al., 2010; U.S. National Intelligence Council, 2000). Hence teaching HL-relevant claims evaluation, data interpretation, and risk assessment skills applicable to ID is likely to have important public health significance.
The other advantage to teaching generalizable HL skills in the high school biology classroom relates to the self-efficacy required to initiate behavior change such as health information seeking (Austin, Pinkleton, Austin, & Van de Vord, 2012; Basu & Dutta, 2008; Berkman et al., 2011; Britner & Pajares, 2006; Zeldin, Britner, & Pajares, 2008). In the AHRQ shown model shown in Figure 1, self-efficacy emerges as output after knowledge and risk assessment skills have been acquired and is directed toward establishing a healthy behavior. It is so far downstream in the instructional model that there are few opportunities to explicitly foster it. In contrast, Figure 2 emphasizes the importance of self-efficacy in acquiring basic HL skills and new health knowledge. Providing educational opportunities to learn and practice generalizable HL skills can develop self-efficacy, thereby encouraging future self-directed learning. Practicing generalizable HL skills in health-related contexts should also facilitate transfer among different domains (Figure 2; Detterman & Sternberg, 1993; Haskell, 2000; Merriam & Leahy, 2005) so the model also incorporates how the ability to transfer learning may positively influence subsequent disease-specific trainings in disparate areas (Berkman et al., 2011).
We have already shown that participating in the Great Diseases curriculum can positively influence student knowledge, attitudes, and self-efficacy toward learning about science (Jacque et al., 2013). This study is a secondary analysis of the evaluation data from several curriculum enactments of the ID module to address the hypothesis that participating in the curriculum will promote the conceptual knowledge and problem-solving skills relevant to HL that are necessary to evaluate health claims, interpret data, and generate accurate risk perception and will foster attitudes of self-efficacy towards learning about ID. We further hypothesize that the effect of the curriculum will extend beyond the classroom, with students participating in the program engaging in more conversations about health and disease than the comparison group, thereby potentially affecting subsequent intent for health behaviors.
The original study included five high schools in three states (Massachusetts, Ohio, and Virginia) and used a quasi-experimental design. Massachusetts schools included an urban college preparatory public exam school, an urban general public high school with a high proportion of minority English language learners (Hispanic), and a suburban public high school with a wide range of student abilities. In Ohio, the school was a regional public STEM (science, technology, engineering, and mathematics) high school with a diverse demographic, and in Virginia it was a private high school. The schools self-selected, and teachers volunteered to participate. The curriculum was taught as an elective Biology II course, and students self-selected. The participating students represent a diverse ethnic and social demographic of learners (see Supplemental Table 2; the online supplements are available at http://heb.sagepub.com/supplemental). A comparison group comprised age-matched peers from the same schools that completed the online surveys voluntarily. The curriculum was implemented multiple times, and data were collected between 2010 and 2013.
The curriculum aligns with the following crosscutting science education standards specifically relevant to generalizable HL skills under consideration: (a) evaluating and communicating information, (d) analyzing and interpreting data, and (c) asking questions and defining problems (Quinn et al., 2011). It embeds a rich mosaic of pedagogy in a constructivist framework to promote accessibility to different types of learners (Tanner & Allen, 2004). The ID module is built around five questions selected by ID content specialists as essential to provide a solid base of conceptual knowledge readily transferable to current and evolving real-world HL situations. Each question forms a unit comprising five to six individual lessons: Unit 1 engages the students. Units 2 to 5 comprise ID-specific crosscutting concepts forming a critical framework for evaluating ID-related health claims and facilitating future learning about ID topics. The curriculum uses active inquiry and case studies in order to promote transfer of learning between different contexts (see Supplemental Table 1; see Jacque et al., 2013, for a detailed discussion).
Successful implementation of novel curricula is challenging and requires adequate teacher support (Beyer, Delgado, Davis, & Krajcik, 2009; Garet et al., 2001). We developed comprehensive educative materials comprising a teacher text, lesson plans, and student workbooks that provide a conceptual rationale and model the pedagogical practices that facilitate implementation. A major innovation involved embedding a narrative within the lesson plans to model how classroom discussion might evolve. This narrative contextualizes the critical learning components and provides the teacher with tools to place the content in a life-relevant, health-related context. The materials are freely available online (http://sites.tufts.edu/greatdiseases/). We also pair teachers with content specialist mentors who offer support via preparative and reflective check-ins, in-person or virtually during classroom implementation (see Malanson et al., 2014, for a detailed discussion).
Between 2012 and 2013, a total of 300 students from the 11th and 12th grades completed the ID module. No students refused to participate in the study, and 273 (91%) completed both pre- and post conceptual knowledge inventory and were included in this analysis. Participating students completed the pre- and posttests during the first and last lessons, respectively. They were told that the pretest was a way for them to explain their initial understanding, and that their performance would not affect their grades. Both questions and answers were returned to the university researchers for grading. Retrospective prepost attitudinal surveys were completed online as homework. Survey items were realigned following the 2010–2011 enactments to improve reliability, so we report results only from the 2011 to 2013 enactments (N = 167). Data from the comparison group were collected using online surveys (N = 125). In all cases student responses were de-identified and confidential. The study and surveys were approved by the institutional review board of Tufts University School of Medicine.
The identical conceptual knowledge inventory pre- and posttests comprised 20 multiple-choice items and 2 short-response items designed to measure understanding of the crosscutting concepts included in Units 2 to 5, as well as a clinical case study with 6 short-response questions designed to measure claims evaluation and risk assessment skills by requiring students to draw conclusions based on evidence, create hypotheses, and extrapolate to explain new phenomena. Questions were designed to avoid a plateau effect and were reliable (Cronbach’s alpha of .86). Use of test-taking strategies was minimized by requiring several correct multiple-choice answers to be identified and by penalizing incorrect choices. Grading rubrics for short answers and the case study were established by ID content specialists, and each test was graded by two evaluators and averaged. Scoring between them had a correlation coefficient of .94. Supplemental materials can be accessed through the Health Education & Behavior website.
Self-efficacy toward learning about a topic is critical for fostering the kind of lifelong learning required to maintain health knowledge and risk perception (Basu & Dutta, 2008; Britner & Pajares, 2006; Martin & Dowson, 2009; Zeldin et al., 2008). Self-efficacy toward learning about ID was assessed using a retrospective pretest (RPT) analysis, which asks respondents to recall a previous state and then report on their current state. This method was selected because it is the most effective method of analysis when students’ lack of prior exposure to the topic (health science learning) would make it difficult for them to objectively assess their baseline in a pre–post design; RPT therefore limits Type II error (Bhanji, Gottesman, de Grave, Steinert, & Winer, 2012; Moore & Tananis, 2009; Pratt, McGuigan, & Katzev, 2000). Retrospective and current self-efficacy was measured with nine questions using a 6-point Likert-type response scale, (1 = low, 6 = high) as an element of the retrospective pre–post survey administered at the end of the module. Principal components analysis of the self-efficacy elements of the survey demonstrated a lambda value of 0.97. The reliability value (Cronbach’s alpha) was .92.
Student initiation of behavior change was determined by asking students to report the extent they discussed ID-related topics with members of their social circles outside of the classroom. This was also measured with six questions using a 6-point Likert-type response scale, (1 = low, 6 = high) as an element of the retrospective pre–post survey administered at the end of the module. Participants’ responses were also compared with the nonparticipant peer group described above. The reliability value (Cronbach’s alpha) was .847.
Differences in pre–post assessment of conceptual content knowledge and claims evaluation skills and retrospective pre–post assessment of self-efficacy were evaluated by analysis of variance (ANOVA) and paired or unpaired t tests were used to establish statistical significance. Effect size (ES), analyzed using Cohen’s d, was defined as small (ES ≥ 0.2, ≤ 0.5), medium (ES ≥ 0.5, ≤ 0.8), or large (ES ≥ 0.8; Cohen, 1988). Reliability of the measures was assessed using Cronbach’s alpha and alignment of the survey questions with specific concepts by principal components analysis.
Students were given identical pre- and posttests designed to assess both their conceptual knowledge about the ID topics (Table 1 and Figure 3) and their ability to use this information to evaluate claims about ID (Table 2 and Figure 4). Table 1 presents the aggregate M and SD of students’ pre-and post-test conceptual knowledge inventory, demonstrating a 2.8-fold change with an ES analyzed by Cohen’s d of 1.89. The pre–post differences were statistically significant (p < .001 paired t test).
The schools represent a range of demographics. To determine their individual responses, aggregate data were recalculated by school (Table 1). The average pre- and posttest scores were highly variable between schools, ranging from 3.70% to 21.88% on the pretest to 18.83% to 56.13% on the posttest (ANOVA p < .001). In addition, the pre- to posttest change also varied between school settings (ANOVA p < .001) and within schools (p < .05, paired t test). The general urban high school, which had a large Hispanic population, had the highest post–pre fold change (6.94) compared with the other schools (average 3.45 ± 0.72) reflecting a lower than average pretest score (3.70) and near average posttest performance (26.68). Plotting student data individually illustrates the variation in individual student gains (Figure 3). There was no correlation between ethnicity and gender with performance (data not shown).
Short-answer responses to a clinical case were used to determine whether students were able to transfer their knowledge and evaluation skills to solve problems similar to those they may encounter in a health management setting (Supplemental Table 3). Table 2 shows the aggregate M and SD of students’ pre-and posttest claims evaluation skills inventory and the ES (Cohen’s d of 1.76). Again, average pre- and posttest scores were highly variable between schools, ranging from 10.12% to 26.57% (pretest) to 24.03% to 56.82% (posttest). The pre–post change was also variable among schools but less than the concept knowledge inventory (ANOVA p < .002). All schools except for the private high school showed statistically significant gains in claims evaluation (p < .0001, paired t test); however, plotting individual student data by school (Figure 4) shows that if two outlier students were eliminated from the private school results, that school shows a gain as well (p < .002). As before, the general urban high school, which had a large Hispanic population, had the highest post–pre fold change (3.92) compared with the other schools (average 2.3 ± 0.55), again reflecting a lower than average pretest score (10.12) and near average posttest performance (39.71).
Table 3 shows the M and SD of students’ self-efficacy scores from the retrospective pre–post analysis. Overall gains in self-efficacy for learning about ID were normally distributed (Kolmogorov-Smirnov). The ES for the aggregated data was large (Cohen’s d of 2.27) and retrospective pre–posttest differences were statistically significant (p < .001, paired t test). Moreover the retrospective self-efficacy scores of the treatment group were not significantly different from the current self-efficacy scores in the comparison group (21.78 ± 8.31 vs. 22.8 ± 8.96).
To determine whether the schools were affected differently, aggregate data were also plotted by school (Table 3). The ES for the per-school data was large in each case: Cohen’s d (1.93–3.03). The average pretest and posttest scores ranged from 18.83 to 24.38 (pretest) to 39.25 to 41.48 (posttest) out of 54 possible points, and each school made gains. There were no differences between schools (ANOVA p = .23 and .87, respectively). The general urban high school with a high proportion of Hispanic students demonstrated the largest (2.20-fold) change in self-efficacy (ANOVA p = .032). Plotting individual student data illustrated far less variability in performance compared with the conceptual knowledge measure (Figure 5).
Participating students were more likely than the students in the comparison group to discuss ID topics outside of school (Figure 6, p < .05, independent samples t test).
Given the rapidity with which medical knowledge evolves, managing one’s health increasingly relies on generalizable HL skills in seeking, interpreting and evaluating current health-related information. The results presented here demonstrate that our health-contextualized biology curriculum was effective in developing these HL-relevant skills. The results thereby confirmed that high school biology classrooms are a viable setting in which to foster 21st-century HL skills, and suggest they should be used more extensively, as has been previously proposed (A. K. Cohen & Syme, 2013; Jacque et al., 2013).
Understanding how and when conceptual knowledge, claims evaluation, and accurate risk perception skills related to HL are most effectively acquired has been underexplored (Koh et al., 2012; Nielsen-Bohlman et al., 2004; http://www.aahperd.org/aahe/). We have shown that knowledge considered by ID experts to be essential for understanding current and future issues in ID is accessible to students at the high school level (Jacque et al., 2013). The data shown in Tables 1 and and22 demonstrate that most students could also learn the core concepts and skills required for lifelong acquisition of ID-related HL and understand how core ideas interacted. They could also leverage the skills they acquired to solve problems similar to those encountered in clinical health care settings. Hence, the curriculum provides sufficient opportunity to practice generalizable HL-relevant skills in the context of health science literacy. The greatest gains in conceptual knowledge gains occurred in a school located in a predominantly immigrant area with a majority of English language learners, indicating that it is accessible to a wide range of learners from different demographics.
Since medical advances evolve rapidly, maintaining proficient HL depends on ongoing learning. In this regard, belief in one’s capacity to learn about a topic (self-efficacy) is a critical element of an ongoing commitment to independent learning (Austin et al., 2012; Basu & Dutta, 2008; Berkman et al., 2011; Britner & Pajares, 2006; Chen & Zimmerman, 2007; Zeldin et al., 2008). However, even though self-efficacy is highly correlated with the actual capacity to learn, a definitively causal relationship has yet to be established (Austin et al., 2012; Basu & Dutta, 2008; Britner & Pajares, 2006; Chen & Zimmerman, 2007; Zeldin et al., 2008). Both the participating and comparison groups assessed their initial capacity to learn the material as low, consistent with a lack of prior exposure/practice that would build self-efficacy of learning. However, after experiencing the curriculum, participants’ perception of their capacity to learn about ID increased significantly. Comparing the student cohort as a whole, gains in self-efficacy were more consistent than gains in conceptual knowledge, consistent with research showing that self-efficacy results from perception of mastery rather than actual performance (Britner & Pajares, 2006). However, when student gains in both measures were compared on an individual basis, that trend was not sustained (Pearson’s p = .19), suggesting that there is a link between self-efficacy and capacity to learn. Again the greatest gains in both measures occurred in the school with the highest number of English language learners, a demographic that commonly demonstrates poor HL skills (Koh et al., 2012; Nielsen-Bohlman et al., 2004). These data therefore indicate that the curriculum is effective in improving student attitudes toward their capacity to learn HL-relevant claims evaluation and interpreting skills, both critical elements of the independent learning needed to sustain health knowledge and accurate risk perception in a dynamic health care landscape (Austin et al., 2012; Basu & Dutta, 2008; Britner & Pajares, 2006; Nielsen-Bohlman et al., 2004).
One major goal of improving HL is to initiate positive behavior change (Figures 1 and and2),2), and it is clear that while the capacity to use claims evaluation skills in the context of conceptual knowledge will be necessary, it is unlikely to be sufficient especially in a clinical setting (Emmons, 2000; Viswanath & Emmons, 2006). Behavior change is highly contingent on intrinsic motivation (Martin & Dowson, 2009; Renninger, 1992; Tobias, 1994); however, classroom settings mainly rely on extrinsic motivators, like grades, and are therefore not the best venue in which to measure it (Deci, Koestner, & Ryan, 2001). We therefore attempted to assess intrinsic motivation by comparing the extent to which students’ motivation to communicate about ID topics within their social circles changed as a consequence of participation in the curriculum. As might be expected, most communication about ID occurred among course participants; however, it was also significantly increased among all the different social circles investigated, including those outside school, relative to the comparison group. This secondary analysis of data obviously limits specific conclusions that can be drawn related to HL. Nonetheless, all students showed equivalent responses irrespective of school, suggesting that at the very least, interventions like this could be used to reach adult populations that are commonly affected by inadequate HL, such as the parents of the immigrant students in some of our schools. The proposition that students may act as health intermediates in this context deserves further investigation.
The number of students taking elective biology in schools offering the Great Diseases curriculum has risen exponentially since we started this study, confirming the relevance of this kind of health science to teenagers. However, we still have to show that the intervention can also positively influence non–self-selecting students. With respect to study design, this is a secondary analysis of data obtained from an education study, rather than a study explicitly designed to measure HL. In that context the RPT analysis, which asks respondents to recall a previous state and then report on their current state, was selected because it is the most effective method of analysis when students’ lack of prior exposure to the topic (in this case health science learning) would make it difficult for them to objectively assess their baseline in a pre–post design; RPT avoids Type II error (Bhanji et al., 2012; Moore & Tananis, 2009; Pratt et al., 2000). However, it can produce inflated ESs if respondents deem it more socially desirable to overestimate gains (Bhanji et al., 2012; Moore & Tananis, 2009; Pratt et al., 2000). We attempted to mitigate this effect by collecting responses online, anonymously, and by comparing participant responses with a comparison group of age-matched peer volunteers who had not participated in the ID curriculum. There was no difference in how both these groups assessed their previous state (see the Results section), supporting that in this case the RPT design has not produced bias in the participants. Two further modifications need to be made to strengthen these results: A randomized or well-matched control study designed specifically to address HL issues would allow more powerful conclusions than this comparison group. That said, current instruments to measure HL generally focus on clinical settings and may not be optimal to measure these classroom interventions (Haun, Valerio, McCormack, Sørensen, & Paasche-Orlow, 2014). However, determining the intervention’s long-term impact on behavioral change is critical and therefore may require development of new measures of HL skills specifically for this population.
Having defined the skills sets required for HL proficiency, the field is rapidly moving forward to articulate strategies that can be used to foster these skills. As these discussions evolve, it will be critical to remain vigilant that strategic planning does not formulate 20th-century responses to 21st-century problems. When we talk about a health literate population in the 21st century, we are necessarily referring to a population able to respond to new health information as it evolves rapidly. Such a population will need to be conversant in the science that underlies the generalizable HL skills we are focused on here, whether in the context of evaluating the latest nutritional information, interpreting data about the latest cancer screen, or assessing the risk of infection by an emerging contagious virus. Recent reports such as the National Action Plan for Improving Health Literacy acknowledge the importance of evaluation and risk assessment skills but have not yet paid adequate attention to how they can be most effectively acquired. Our argument, that teaching these skills requires pedagogical approaches different from those used in health education, is important, because to properly prepare students we will need a cohort of professionals well trained to teach the biology of 21st-century health and disease in an HL-related context. As yet this population barely exists. To acquire and empower this cohort will require us to discard the artificial distinction between “science” and “health” that is contributing to the low level of HL proficiency in the U.S. population and to realize that just as numerical HL may be most effectively addressed in the context of math classes, so may these critical skills be most effectively acquired in the context of the science underlying health and disease.
The authors received no financial support for the research, authorship, and/or publication of this article.
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
The online supplemental materials are available at http://heb.sagepub.com/supplemental.