|Home | About | Journals | Submit | Contact Us | Français|
Although media literacy represents an innovative venue for school-based antismoking programming, studies have not systematically compared student impressions of these and traditional programs. This study utilized data from a randomized trial comparing these two types of programs. After each program, students responded to three open-ended questions related to their assigned curriculum. Two coders, blinded to student assignments, independently coded these data. Coders had strong inter-rater agreement (kappa=0.77). Our primary measures were spontaneously noted overall assessment, enjoyment/interest and the likelihood of changing smoking behavior. Of the 531 participants, 255 (48.0%) were randomized to the intervention (media literacy) group. Intervention participants had more net positive responses [rate ratio (RR)=1.27, 95% confidence interval (CI)=1.05, 1.54], more responses rating the program as compelling (RR=1.63, 95% CI=1.16, 2.29) and fewer responses rating the program as non-compelling (RR=0.62, 95% CI=0.39, 0.97). However, the intervention group was not more likely to suggest that the curriculum was likely to change behavior positively (RR=0.57, 95% CI=0.30, 1.06). Findings suggest that although media literacy provides a compelling format for the delivery of antitobacco programming, integration of components of traditional programming may help media literacy programs achieve maximal efficacy.
Smoking is the leading cause of morbidity and mortality in the United States, killing 442000 people each year . Since the vast majority of those who will die from tobacco begin smoking at age 18 or younger , many counter-tobacco interventions appropriately focus on youth . An excellent way to reach youth is in a school-based setting since the vast majority of youth attend school . Historically, traditional school-based programs have focused on teaching the short- and long-term effects of smoking and helping young people develop skills to avoid tobacco use through counter-argument [3, 5–8]. These programs, however, are not generally compelling to youth and have failed to result in long-term reduction in tobacco use [3, 5–9]. As an example, one particularly large well-funded longitudinal school-based curriculum of 47.25 hours administered >8 years failed to result in decreased smoking rates . In that study, smoking rates were no different in the experimental and control conditions among boys, girls, high-risk individuals or low-risk individuals .
Since the development of these programs, however, the strategy of ‘media literacy’ as an antitobacco tool has been recommended by organizations such as the American Academy of Pediatrics, the Centers for Disease Control and Prevention and the Office of National Drug Control Policy [2, 10, 11]. These recommendations have been borne out of the discovery that media exposure to smoking contributes strongly to initiation of adolescent smoking, whether that exposure occurs in narrative contexts such as films [12–14] or in persuasive contexts such as advertising and promotion [15–18]. Although one method of reducing adolescent smoking would be to reduce exposure to media representation of smoking [12, 19, 20], it is not always possible and/or feasible to do this [21–25].
Thus, media literacy, often defined as the ability to understand, analyze, evaluate and create media messages in a wide variety of forms [26–28], represents a potentially powerful behavior change strategy available to public health advocates. Theoretically, media literacy may be more accessible and more effective than traditional programs. First, media literacy may be more acceptable to adolescents—who often find traditional programs to be monotonous, old-fashioned or ‘preachy’ —since it involves the use of media examples that may be compelling, interesting and relevant to young people. Once youth are ‘hooked’ by this more accessible format, they may be less likely to ‘tune out’ from the antitobacco message. Second, media literacy may be effective among adolescents because of its ability to alter smoking attitudes and smoking normative beliefs, constructs that have been closely linked to smoking behavior both by the Theory of Reasoned Action  and by empirical data [30–33]. Media literacy may be able to do this because it sensitizes young people to the manipulative and often implicit messages designed to confer pro-smoking attitudes and subjective norms. Thus, using media literacy to examine the careful construction of pro-tobacco messages and salient omissions may lead to decreased belief in the validity of these implied effects of smoking .
A recent comprehensive review of media literacy interventions shows that it has been used for a variety of purposes . Although media literacy has been used most frequently as an intervention related to eating behaviors [35–40], media literacy programs have also focused on violence, nutrition [36, 41] and substance use [42–44]. Early antismoking programs were well liked by students but have not had evaluations in the peer-reviewed literature [11, 45]. Since that time, others have evaluated antismoking media literacy programs and found that media literacy programs can help young people increase reflective thinking regarding tobacco-related media messages [43, 44]. These programs also have been successful at altering variables related to tobacco use such as normative beliefs related to smoking [43, 44]. Additionally, studies measuring media literacy in adolescents have found it to be strongly and independently associated with both reduced adolescent smoking and reduced susceptibility to future smoking [46, 47]. However, studies have yet to rigorously and directly compare students’ impressions of media literacy programs versus traditional harm-based programs.
The purpose of this study was to compare qualitatively analyzed student impressions of two different types of antismoking education: media literacy and traditional programs. Based on the theoretical arguments described above, we hypothesized that the media literacy program would be more compelling and acceptable to young people, as evidenced by more net positive overall responses (Hypothesis 1). We also expected more descriptions of media literacy as fun/interesting (Hypothesis 2) and more likely to reduce smoking behavior (Hypothesis 3). We also expected more responses from media literacy participants exhibiting critical thinking (Hypothesis 4). We expected no significant difference with regard to codes related to information (Hypothesis 5), format (Hypothesis 6) and logistics (Hypothesis 7) since we endeavored to match the curricula on these characteristics.
This study utilized data from Year 1 of a randomized controlled trial investigating the efficacy of media literacy education. Over the 2006–07 school year, all 9th grade students from two large urban public high schools were eligible for enrollment. These students represented 28 health classrooms, each of which was randomized to either the intervention (media literacy education) or control (traditional) condition. Randomization was stratified by time of day and regular classroom teacher and was conducted in blocks of four classrooms each. On all evaluation instruments, students used unique non-traceable identification numbers to keep their identities confidential. This study was approved by the University Institutional Review Board (IRB #0606146).
Each curriculum consisted of three weekly sessions consisting of one class period (~45 min) each. The experimental condition was the three-session version of the ‘AD IT UP’ curriculum—a theory-based antismoking media literacy curriculum developed and pilot tested with foundation grant funding. This curriculum teaches youth to access, analyze and evaluate mass media messages involving smoking, such as advertisements, promotions or film product placements. Each letter of the title refers to one of the six lessons contained in the curriculum, each of which is associated with a ‘core concept’ based upon a composite theoretical model of media literacy and a media literacy ‘key question’ (Table I).
This composite theoretical model combines constructs from two paradigms [28, 48]. Although they overlap substantially, there are differences in emphasis. One model emphasizes that (i) the purposes of media producers and characteristics of target audiences should be carefully considered, (ii) there are multiple complex production techniques and symbol systems used to convey meaning and (iii) there is a complex relationship between media representations and social reality . Another model emphasizes that (i) media messages are carefully constructed with the use of their own complex language, (ii) different individuals interpret messages differently, (iii) messages contain inherent values and perspectives and (iv) media messages are usually created for profit and/or power . Integrating both of these models into a comprehensive theoretical framework fully captures the construct of media literacy .
The control curriculum was based on the three-session antismoking program featured in the most widely used and commonly accepted health textbook in the region of the study . These three lessons, which form the quintessential traditional program, describe both short- and long-term effects of smoking, support students’ self-efficacy in saying ‘no’ to smoking and include a discussion of social and psychological issues involving smoking. Thus, the programs differ in that the media literacy program is more focused on developing critical thinking skills around tobacco-related media messages, whereas the control curriculum focuses on the harm of smoking and skills to avoid smoking.
The curricula were implemented by experienced health educators who were not the students’ regular high school teachers. The instructors were given appropriate training, consisting of a total of four 3-hour sessions over the course of 3 months. The first hour of each session consisted of didactic information related to the curricula and demonstrations of the curricula. Then, during the latter 2 hours, the instructors were given opportunities to practice conducting the curricula and to receive feedback from their fellow instructors. Finally, each instructor had an opportunity to practice implementation on convenience samples of high school students from other (non-study) schools before actual implementation. Each teacher remained with the same cadre of students for each of the three lessons in order to maintain teacher–student rapport and continuity.
Because of the intrinsic differences between media literacy and traditional antismoking education, the information presented was different. However, the intervention and control conditions were matched in as many other ways as possible. Each of the teachers taught both curricula. The students were given the same pre- and post-intervention surveys. The curricula featured the same format—Power Point presentation combined with activities—featuring similar numbers of slides and numbers of images in each presentation. Workbooks were also matched: both curricula featured a 16-page workbook with similar activities (although with different content). Workbooks featured activities such as open-ended thought questions, matching exercises and fill-in exercises based on the material presented in class.
Instructors self-evaluated their implementation fidelity after each teaching session. Composite implementation fidelity scores showed 98.7% fidelity to the intervention curriculum and 99.1% fidelity to the control curriculum. Thus, fidelity did not seem to be skewed by curriculum type.
All participants completed a post-test 1 week after completion of the 3-week curricula. On these post-tests, students responded to three open-ended questions assessing these curricula. In order to get an unbiased assessment, we avoided asking leading questions such as ‘Do you think this program would make kids not smoke?’ Instead, the three items for which we analyzed responses included the following: (i) ‘What did you like about this program?’, (ii) ‘What did you not like about this program, or what did you think could be improved?’ and (iii) ‘What other comments do you have about this program?’ Students typed their responses directly on to computers so they could not be identified by handwriting, and they were not required to respond for credit. We used these methods to maximize the validity of our data. Although participants also provided quantitative data assessing tobacco-related constructs that will be used for later longitudinal analyses, these data were not relevant to the current study, which aimed instead to capture students’ impressions of the programs in their own words. Overall, the post-test was completed in an average of 20 min.
We employed a quasi-statistical coding mechanism . Such ‘mixed methods’ approaches can be valuable in bringing statistical rigor to data obtained via qualitative coding [51–54]. We first developed a tentative list of codes based on our research questions and hypotheses and on prior studies, qualitatively assessing smoking-related issues in adolescents and young adults [55–60]. Four study personnel, blinded to participant group assignment (intervention or control), then individually coded the first 10% of responses using this codebook. Based on these initial reviews and on a discussion process between the four coders, we developed a final study codebook, condensing similar or redundant codes and using a system of open codes that incorporated new themes as they emerged from the text.
Two coders then independently coded data from all 531 participants using Atlas.ti, a qualitative software analysis program . Coders were blinded to participant group assignment throughout the coding process. When both coders were finished, we assessed the inter-rater reliability of our coding using Cohen’s kappa scores. The average kappa score was 0.77 which Landis and Koch  describe as a substantial level of inter-rater reliability. Finally, every discrepancy between the two coders was discussed among the team until each discrepancy was adjudicated, resulting in a final data set. Only after the list of codes was finalized were coders unblinded.
Our final codebook contained 34 relevant codes (Table II). As part of our coding scheme, we included both a single overall assessment which was applied to all cases, as well as a thematic analysis where codes were applied as topics were expressed.
As our first primary coding set, coders assigned an ordinal overall assessment code (Hypothesis 1) of ‘POSITIVE,’ ‘NEUTRAL’ or ‘NEGATIVE’ to each of the three participants’ responses (corresponding to Questions 1, 2 and 3, respectively). This method was chosen as students frequently included negative comments in the ‘most liked’ category or positive or neutral comments in the ‘least liked’ category, with the ‘other’ category having various observations, either positive, negative or neutral. Each of the three response categories completed by the students were judged to be either positive, negative or neutral (Table II). Kappa values for each of these codes were ‘nearly perfect’  at 0.88, 0.79 and 0.89, respectively. We converted this qualitative code set into a quantitative value by subtracting any negative scores from the positive scores. (Neutral codes were not included since they were given a value of zero.)
For the second primary coding set, we focused on the interest level of the course (Hypothesis 2). For this and all subsequent coding, we used a thematic analysis approach, applying codes as textual themes emerged. Here, codes related to enjoyment of and interest in the curricula were labeled ‘FUN’ and ‘INTERESTING’, and, conversely, the code labeled ‘BORING’ related to a lack of enjoyment or boredom (Table II). Coders marked text with the code labeled ‘MORE’ when participants expressed a desire for additional or longer sessions, and they used the code ‘LESS’ when the text indicated that the program was too long or contained too many sessions. A composite ‘compelling’ coding score summed instances of FUN, INTERESTING and MORE, whereas a composite ‘non-compelling’ coding score summed BORING and LESS.
Our third coding set focused on whether the curricula promoted changed behavior (Hypothesis 3). We used the code ‘CHANGE GOOD’ to mark statements that spontaneously suggested that the curriculum would be likely to change behavior in a positive way among the respondents or their peers. Although we also searched for instances of ‘CHANGE BAD’ which would indicate that the curricula might make them or their peers more likely to smoke, we did not find any instances of this code.
‘CRITICAL THINKING’ was a single code used to indicate a process of critical analysis (Hypothesis 4). This code was used if, for instance, participants used language such as ‘figured out’ or ‘learned how to look at’ a given issue in the curriculum (Table II).
Another series of secondary codes related to curriculum content (Hypothesis 5). These included ‘INFO HEALTH’, ‘INFO OTHER’, ‘INFO BAD’ and ‘NO NEW INFO’ (Table II). A total content composite score was created summing INFO HEALTH and INFO OTHER and subtracting NO NEW INFO and INFO BAD.
Codes related to the format of the curricula (Hypothesis 6) were related to activities, interaction and visual elements. ‘ACTIVITY GOOD’ and ‘ACTIVITY BAD’ were used only when the format of a specific activity was mentioned. Three codes related to interactivity during the program: ‘INTERACTION GOOD’ indicated that a participant liked the degree of interactivity, whereas ‘INTERACTION NOT ENOUGH’ indicated that it lacked sufficient interactivity. Finally, ‘VISUAL GOOD’ and ‘VISUAL BAD’ indicated whether they appreciated or disliked visual elements of the curricula.
Finally, several codes related to logistical aspects (Hypothesis 7) of the programs. For instance, ‘TEACHER GOOD’ and ‘TEACHER BAD’ indicated positive and negative comments about the program instructor and ‘WRITING GOOD’ and ‘WRITING BAD’ commented upon the workbooks that went along with each of the curricula. We coded ‘BLANK’ when a section was blank or otherwise not useful.
The frequencies of the codes were assessed using a quasi-statistical qualitative methodology . We summed the number of counts for each code among those in the intervention and the control conditions. We then computed rates by dividing each code count by the total number of participants in each study arm. Finally, we used Stata Statistical Software  to compute the rate ratio (RR) for each code, comparing intervention with control, and its 95% confidence interval (CI) since we considered statistical significance as P<0.05.
Our sample consisted of the 531 students who completed the post-test. The sample was 50.5% female with a mean age of 14.9 (SD=0.9). Of the 532 participants, 255 (48.0%) represented the 14 classrooms randomized to the intervention group and 276 (52.0%) represented the 14 classrooms randomized to the control group. Those in the intervention group were no more likely to be male than female (P = 0.09), to be non-White (P = 0.54) or to be older (P = 0.40). There were 1593 total fields to be coded (three for each individual corresponding to the three open-ended questions). Because there were a total of 242 fields without valuable data, the overall response rate was 84.8%.
Overall, intervention participants had more net positive responses [RR=1.27, 95% CI=1.05, 1.54]. Intervention participants also had more responses rating the program as compelling (RR=1.95, 95% CI=1.23, 3.13) (Table III). Similarly, intervention participants had fewer responses rating the program as non-compelling (RR=0.62, 95% CI=0.39, 0.97). However, the intervention group was not more likely to spontaneously suggest that the curriculum was likely to change behavior positively. In fact, the data exhibited a non-significant trend toward intervention participants being less likely to suggest positive change impact of the curriculum (RR=0.57, 95% CI=0.30, 1.06).
Critical thinking was more commonly found among intervention participants (RR=16.17, 95% CI=4.10, 139.66) (Table III). Our composite information score was no different in the intervention and control conditions (RR=0.92, 95% CI=0.69, 1.20). However, information specifically pertaining to health (INFO HEALTH) was less common among intervention participants (RR=0.59, 95% CI=0.38, 0.89). Intervention participants were more likely to mention specific activities in a positive way (RR=26.95, CI=4.41, 1106.64). Of the logistical codes, only TEACHER GOOD was different between the two groups; it was less likely to be mentioned by the intervention group (RR=0.40, 95% CI=0.20, 0.75).
This qualitative assessment of data from a randomized controlled trial found that students considered the media literacy intervention a more positive and compelling experience overall. However, intervention participants were no more likely to state that the intervention might change health behavior; in fact, control group participants exhibited a non-significant trend toward more such statements. While the intervention group seemed to exhibit more critical thinking and more frequently mentioned specific activities they enjoyed, the control group mentioned more health-related information they had learned.
Our finding that media literacy was compelling to students is consistent with prior research [27, 64–66]. One reason for the compelling nature of media literacy programming in adolescence may be that these programs utilize media from popular culture that are highly familiar among and relevant to students. Because they are based in critical thinking pedagogy, media literacy programs also may be more respectful of students, allowing them to make up their own minds instead of being directly told what to do by an authority figure. This ‘low-pressure’ environment may lead to a more relaxed and enjoyable learning experience.
It is interesting that intervention participants were no more likely than control participants to suggest that they or their peers might be less likely to smoke after the intervention. This may be because the control curriculum contained strong attitudinal antismoking messages, some of which can be effective [22, 67]. However, control program participants may have included more responses related to our CHANGE GOOD code simply because they were parroting the unambiguous, harm-related messages of that curriculum.
Similarly, since intervention participants experienced a curriculum emphasizing critical thinking, it is not surprising that those in the intervention group exhibited more language related to critical thinking. This is encouraging since critical thinking skills may provide longer term protection against the effects of ever-changing marketing strategies. Further studies are needed to elucidate the relationship between critical thinking as taught through media literacy education (i.e. critical thinking about media messages) and longer term protection against the ever-changing media marketing strategies for tobacco and other substances.
INFO HEALTH was more frequently coded in the control group than the intervention group. This was expected since the traditional program emphasized specific health-related information, such as how many people each year die from smoking or the names of specific chemicals in cigarette smoke.
Contrary to our hypothesis, TEACHER GOOD was more commonly coded in the control condition than the intervention condition, despite the fact that (i) the intervention had higher overall POSITIVE responses and (ii) the same teacher taught both curricula. This may have been because control group participants had fewer other memorable statements to make, so they praised the instructor instead. The fact that TEACHER BAD was equally common in both groups supports this conclusion. Another possibility may be that the teacher appeared more knowledgeable in the control condition because he was able to give unambiguous facts and figures related to smoking. TEACHER GOOD quotations such as ‘the instructor knew a lot of information and was able to answer any questions’ and ‘he was the right person to educate some one [on] this topic … [and] he knows his stuff’ lend credence to this supposition.
One limitation of any qualitative methodology is that interpretation and bias introduced by the researcher poses a threat to internal consistency . We sought to minimize this by using a team of two researchers to independently code the data and comparing their responses with reliability scores. Additionally, although the interventions were delivered at the classroom level, these analyses do not take into account nesting within classrooms. Related to this, it should be noted that since different classrooms were randomized within the same school, there was the potential for contamination (students discussing each others’ curricula after school). Another necessary limitation of our methodology was that although we blinded coders to participant assignment group, it is possible that the coders could guess intervention assignment.
Another limitation of the study was that by quantifying our results and focusing on RRs, we may have lost some of the inherent richness of the qualitative responses. However, such a methodology is valuable, when combined with rich qualitative data (Table II), to lend rigor to our conclusions. Our use of open-ended survey questions, as opposed to more commonly used focus groups or interviews [55, 58, 69–72], may have also been a limitation. This is because, without guidance from an interviewer, adolescents’ open-ended responses could generate irrelevant, ambiguous or sparse information. However, we chose this unique methodology because it allowed us to avoid the bias inherent in leading participants to certain responses. In fact, it represents an innovative method of conducting qualitative research that may be useful to future investigators.
This study is unique in that it qualitatively analyzed open-ended student responses for overall assessment as well as specific content related to the study aims. Although we found the media literacy intervention did seem to be overall a more positive and compelling experience for students, we also found that intervention participants were no more likely to state that the program might be effective. Thus, there seem to be risks and benefits to each type of antismoking education. Although media literacy seems to provide a compelling framework for antismoking education, integration of components of traditional programming into these programs may make them maximally effective.
National Cancer Institute (K07-CA114315 to B.A.P.); Robert Wood Johnson Foundation (57407 to B.A.P.); Maurice Falk Foundation (Major Grant Award to B.A.P.); Veterans Administration Health Services (Research Development Merit Review Award to S.Z.).
We gratefully acknowledge the National Institutes of Health, the Robert Wood Johnson Foundation and the Maurice Falk Foundation, and the Veterans Administration Health Services. Although each of these agencies provided financial support, they were not involved in the design and conduct of the study; the collection, management, analysis and interpretation of the data or the preparation, review or approval of the manuscript.