|Home | About | Journals | Submit | Contact Us | Français|
This study assessed the immediate and short-term outcomes of adapting a culturally-grounded middle school program, keepin’ it REAL, for elementary school students. After curriculum adaptation, 10 schools were randomly assigned to the intervention in 5th grade with follow-up boosters in 6th grade; 13 schools were randomly assigned to the control condition, implementing the school’s pre-existing substance use prevention programming. Students (n = 1,566) completed a questionnaire prior to curriculum implementation and follow-up questionnaires toward the end of 5th and 6th grade. The 5th grade kiR curriculum generally appeared no more effective than the control schools’ programming in changing students’ resistance or decision-making skills; substance use intentions, expectancies, or normative beliefs; or lifetime and recent substance use. Such findings have implications for the age appropriateness of school-based programs.
Youth who experiment with alcohol and cigarettes at a young age exhibit a higher likelihood of future substance use (Kandel, Yamaguchi, & Chen, 1992). Unfortunately, many intervention programs appear less effective for adolescents who began using alcohol, tobacco, or other drugs (ATOD) at an early age (Ellickson, Bell, & McGuigan, 1993; Murray, Hannan, Wolfinger, Baker, & Dwyer, 1998) and few prevention programs target elementary-aged youths (Finke et al., 2002). Although only a small proportion of elementary students use substances (Andrews, Tildesley, Hops, Duncan, & Severson, 2003), adolescents report an estimated average age of first-time cigarette and alcohol use between 11 and 13 years, and marijuana use between 13 and 14 years (National Institute on Drug Abuse, 2003). Waiting until middle school or high school to introduce prevention interventions can be too late, since by then adolescents may have developed entrenched substance use expectations and norms (Stipek, de la Sota, & Weishaupt, 1999).
Providing elementary school students with substance use prevention programming may prove an effective strategy for two major reasons. First, research suggests that preadolescents respond with greater willingness to practicing ATOD refusal skills (Donaldson, Graham, & Hansen, 1994). Second, the difficulty of changing established behaviors encourages the implementation of prevention before high risk behaviors emerge. Research shows that 10- to 11-year old youths are about to enter a period in their lives where they may become attracted to and occasionally partake in high-risk behaviors (Stipek et al., 1999). Physical appearances and peer acceptance become more important, thus increasing a child’s propensity to engage in substance use (Compas, Hinden, & Gerhardt, 1995; Stipek et al., 1999). Despite their young age, some preadolescents develop intentions to engage in high-risk behaviors (Andrews et al., 2003; Falco, 1992) and some start experimenting with substances (National Institute on Drug Abuse, 2003). In short, since early intervention appears potentially useful, researchers must assess whether elementary school-based prevention will effectively influence youths’ ATOD expectations, refusal and decision-making skills, normative beliefs, ATOD intentions, and current substance use, all predictors of future use. Although other prevention researchers have begun to evaluate such programs, this study extends the literature by assessing the immediate and short-term outcomes of adapting an efficacious, culturally-grounded middle school program, keepin’ it REAL (Hecht et al., 2003), for use with an ethnically diverse group of 5th and 6th grade students.
When designing an elementary school-based substance use prevention program, curriculum developers must incorporate lessons that deal with the primary motivational factors found to encourage or discourage ATOD use. Theory and empirical findings indicate that preadolescents make decisions whether to use ATOD based on several key factors including: 1) their expected outcomes of substance use; 2) their normative beliefs regarding substance use; 3) their intentions to use or refuse substances; and 4) their ability to use resistance strategies and decision-making skills (Elek, Miller-Day, & Hecht, 2006; Jobli, Dore, Werch, & Moore, 2005; MacKinnon et al., 1991; Tobler et al., 2000).
Previous prevention research reveals that youth develop influential expectations, normative beliefs, and intentions regarding substance use at a young age. For instance, youth in preschool and elementary school develop expectations of alcohol’s effects even before ever consuming alcohol (Dunn & Goldman, 1996). Preadolescents who hold positive expectancies regarding substance use appear more likely to use them compared to preadolescents with negative expectancies (Miller, Smith, & Goldman, 1990). Normative beliefs play a similarly important role in hindering substance use for this cohort. Early adolescents report less alcohol, cigarette, and marijuana use when they perceive that their parents and peers would disapprove of their substance use and when they perceive that fewer of their peers at school use substances (Elek et al., 2006; Macaulay, Griffin, Gronewold, Williams, & Botvin, 2005; Preston & Goodfellow, 2006).
As with expectancies and normative beliefs, researchers have demonstrated the importance of substance use intentions. Marcoux and Shope (1997) found that the reported intentions to drink alcohol for a sample of 5th through 8th grade students predicted their alcohol consumption. Consistent with this finding, the adolescents in Choi et al.’s (2001) study, who had high intentions to smoke at the time of the initial survey, were more likely to report smoking four years later in a follow-up survey. Andrews et al. (2003) found that among 1st through 5th grade students followed across three years, those with intentions to consume alcohol or smoke cigarettes were more likely to try these substances. In short, across several longitudinal studies, intentions to use substances predicted actual use among youths, thereby shedding light on the importance of intentions in substance use prevention research.
Although negative expectancies, anti-ATOD normative beliefs, and intentions may discourage substance use, adolescents still need to possess the appropriate skills to react to offers of alcohol, cigarettes, and other drugs. Miller, Alberts, Hecht, Trost, and Krizek (2000) found that using a variety of resistance skills or strategies appeared negatively associated with adolescent ATOD use. Stated differently, youth who learn to make thoughtful decisions about ATOD use, and who learn a diverse array of resistance skills that do not jeopardize social relationships, resist substances more successfully (Donaldson, Graham, Piccinin, & Hansen, 1995; Hecht, Warren, Wagstaff, & Elek, in press).
Given the relationship between expectancies, normative beliefs, intentions, skills, and substance use, it makes sense to evaluate the impact of a substance use intervention on all of those factors, particularly when a program purports to target those factors. Although substance use remains the primary outcome of interest, focusing on the psychosocial (expectancies, normative beliefs, and intentions) and skills (ATOD refusal and decision-making) factors appears valuable when working with preadolescents who typically exhibit relatively low rates of actual ATOD use (Andrews et al., 2003). Focusing on these factors plays a crucial role when evaluating such programs shortly after implementation, given that the effects on substance use may not yet have become apparent (Kreuter, Farrell, Olevitch, & Brennan, 2000). The following section, then, provides a review of existing elementary school-based substance use prevention programs that, with varying success, attempt to target the aforementioned predictors of substance use.
Although a large number of prevention programs address substance use, few specifically target elementary-aged students (i.e., K-6th grade) and even fewer target 5th grade students specifically; therefore, few assessments examine intervention effects on that cohort (Kam, Elek, & Hecht, 2007). In addition, in studies of elementary and middle school programs that include 5th grade students as part of a larger participant sample, prevention researchers often fail to separate the effects of the intervention for students by grade levels. Instead they combine students across grade levels into one large homogenous group for purposes of reporting results (e.g., Schinke, Tepavac, & Cole, 2000). This section briefly reviews evaluations of elementary school-based ATOD prevention programs and then explores several programs that specifically target 5th grade students.
Despite prevention science’s support for the effectiveness of school-based interventions in general (Gottfredson & Wilson, 2003; O’Donnell, Hawkins, Catalano, Abbott, & Day, 1995; Skara & Sussman, 2003), existing research provides mixed findings with regard to the effectiveness of elementary school substance use prevention programs in reducing subsequent ATOD use. In a review of such research, Kam et al. (2007) found that among 30 elementary school (6th grade and below) substance use prevention programs, only 32% of the programs significantly reduced alcohol consumption, 38% of 13 relevant programs reduced tobacco use, and 40% of 10 relevant programs reduced marijuana smoking. Instead, the programs appeared more successful in significantly influencing predictors of substance use: 75% of eight relevant programs generated more negative attitudes toward ATOD, 38% of 18 relevant programs altered youths’ norms in an anti-ATOD direction, 50% of six programs decreased intentions to use ATOD, and 57% of seven relevant programs enhanced youths’ refusal skills. These findings reveal some of the non-behavioral factors elementary school-based prevention programs target in order to succeed with their primary goal of decreasing ATOD use.
Among programs recognized by SAMHSA’s National Registry of Evidence Based Programs and Practices (NREPP; for more details, see http://nrepp.samhsa.gov), most school-based prevention programs begin at 6th grade or later (e.g., All Stars, Lions Quest Adolescence, Project Northland). When limiting consideration to programs that specifically include 5th grade students, the NREPP Website (NREPP, 2007) included only the Center on Addiction and Substance Abuse Striving Together to Achieve Rewarding Tomorrows (CASASTART), a neighborhood-school-based program targeting 8 to13 year old high-risk youth. However, the CASASTART listing did not contain peer-reviewed published manuscripts evaluating its effectiveness among 5th grade students.
When expanding consideration to elementary school-based substance use prevention programs which include 5th grade students discovered through a literature search but not recognized by SAMHSA’s NREPP, the number of programs and evaluations slightly increased. For example, Bernat, August, Hektner, and Bloomquist (2007) evaluated The Early Risers, a K-6th grade program meant to improve children’s self-regulation, social adjustment, and parent-child relationships, which in turn, were intended to decrease oppositional defiant disorder, conduct disorder, and substance use. They found no significant program effects on alcohol or tobacco use at a 6-year follow-up (they did not assess substance use effects prior to that time). In contrast, Linking the Interests of Families and Teachers (LIFT), a program for children and parents within the elementary school setting, produced delayed patterns of alcohol and marijuana use among 8th grade students who received the program in 5th grade (Eddy, Reid, & Fetrow, 2000). An outcomes analysis of the Adolescent Alcohol Prevention Trial (AAPT), which assessed a normative education, resistance skills, and information-based intervention program, found that the 5th grade participants reported enhanced resistance skills. The program also “reduced prevalence estimates and strengthened beliefs about the unacceptability” of alcohol, cigarettes, and marijuana by 7th grade (Donaldson et al., 1994, p. 211), which in turn, significantly predicted alcohol, smoking, and marijuana. In contrast, a separate implementation of the intervention’s resistance-skills training component alone did not produce significant direct or indirect effects on substance use even into 8th grade.
The aforementioned provide examples of prevention programs targeting 5th grade students in an attempt to address ATOD expectancies, correct misperceptions about peer substance use, and enhance resistance strategies and other skills. Although some of these programs demonstrated significant effects on substance use and other factors targeted by their curriculum, they differ in a few ways from the intervention considered in this article, the keepin’ it REAL (kiR) substance use prevention program. First, few of the programs have been recognized by NREPP, or were based on recognized programs, as is the case with the kiR 5th grade curriculum. Second, a number of these programs include components (e.g., neighborhood, parent) that extend beyond the school itself. Third, some programs such as AAPT were implemented and evaluated over a decade ago and may not adequately address the changing norms and attitudes toward drugs evidenced in today’s youth. Finally, most of the programs that include students in 5th grade ignore cultural factors that predict variations in substance use, even though their evaluation studies included substantial numbers of ethnic or racial minority group members in their sample (Bell, Kelley-Baker, Rider, & Ringwalt, 2005; Donaldson et al., 1994, 1995), and even though their wider dissemination could take them to schools with substantial minority populations. The following section now turns to this cultural factor, a key element in the prevention strategy adopted by kiR.
To date, the only apparent culturally-grounded program uncovered through a literature search is an elementary school-based substance use prevention program targeting Native American youth in 3rd through 5th grades. An evaluation of this program found that students who received skills training reported lower rates of smokeless tobacco, alcohol, and marijuana use over a three and a half year period (Schinke et al., 2000). The fact that this culturally-grounded elementary school-based substance use intervention demonstrated successful effects encourages further consideration of culturally-grounded elementary school-based substance use prevention programs, particularly those that involve other ethnic groups. Consistent with this notion, Resnicow et al. (2000) argue that prevention researchers should consider the cultural and ethnic distinctions of ATOD-use predictors to develop culturally sensitive and effective programs, which more general programs may not capture. Thus, the researchers behind the Drug Resistance Strategies (DRS) project designed the keepin’ it REAL (kiR) substance use prevention program as a culturally grounded middle-school intervention that considers salient variables among certain ethnic groups, in an attempt to address their unique needs.
Originally the DRS researchers designed kiR for 7th and 8th grade students where it demonstrated significant effects on substance use (Gosin, Marsiglia, & Hecht, 2003; Hecht & Miller-Day, in press; Hecht et al., 2003; Hecht, Graham, & Elek, 2006). The present study evaluates the immediate and short-term effects of a version of kiR developed expressly for 5th grade students.
The multicultural 7th grade version of kiR (distributed through ETR Publishing and Discovery Education1) consists of 10, 45-minute lessons which incorporate 5 videos and content focusing on enhancing anti-drug expectancies, normative beliefs, and refusal self-efficacy, and facilitating the development of decision making and resistance skills (Refuse, Explain, Avoid, and Leave—hence REAL). The lessons utilize videotaped scenarios, role plays, and other activities to model resistance strategies, teach risk assessment and decision making, enhance resistance efficacy, and impact expectations and normative beliefs. The program supplements the 7th grade curriculum with a selection of boosters at the 8th grade level (school assemblies, poster projects, etc.) that reinforce program content. The original curriculum development drew on the ecological risk and resiliency approach (Bogenschneider, 1996) and communication competence theory (Spitzberg & Cupach, 1984), by incorporating knowledge (narratives), motivation (norms), and skills as keys to prevention. The curriculum utilized a cultural grounding approach (Hecht & Krieger, 2006) that was based, in part, on a narrative and performance framework. Grounding was accomplished through community-based participatory research involving the integration of the students’ own stories of resisting offers of ATOD and allowing culturally similar students to tell the stories through videos and other aspects of the curriculum that model salient behaviors and beliefs, as well as by infusing the lessons with cultural values coming from the Mexican-American, European-American, and African-American communities (for greater detail, see Hecht & Krieger, 2006; Hecht & Miller-Day, in press). These techniques enhance the adolescents’ and preadolescents’ identification with the prevention messages, and led to the labeling of the standard version of kiR as “multicultural.”
Curriculum development specialists adapted the existing kiR 7th grade curriculum over a 6-month period to make the lessons developmentally appropriate for 5th grade students and to add two lessons to enhance effects (Harthun, Dustman, Reeves, Hecht, & Marsiglia, 2008). The 5th grade version uses the same basic curriculum content as the standard 7th grade multicultural version, differing primarily in communication level/format, the concreteness of the presentation of concepts, and the age-based relevance of the examples. Although the core content of the standard curriculum uses several strategies deemed successful with preadolescent children (narrative, participatory modeling, observational learning, and videos; Bandura, 1977, 1982; Botvin, Schinke, Epstein, Diaz, & Botvin, 1995; Botvin, Schinke, & Orlandi, 1995; Constantino, Malgady, & Rogler, 1988), developmental concerns necessitated simplification in language and the complexity of presentation of concepts. Limitations in the cognitive abilities of 5th grade students, specifically their more restricted ability to engage in abstract thinking, systematic reasoning, and perspective taking (Case, 1985; Piaget, 2000), encouraged changes in presentation format. In particular, the changes focused on personalizing important concepts, making concepts more concrete (e.g., by asking for and providing concrete examples), and drawing more specific conclusions. The 5th grade students’ limited experience with pressures to use drugs led the curriculum designers to ensure the concrete modeling and/or practice of each resistance strategy using other situations with which the majority of 5th grade students could relate (e.g., pressure to cheat on an exam; Polich, Ellickson, Reuter, & Kahan, 1984).
In addition to the changes described above, the curriculum development specialists specifically considered other developmentally-related instruction differences between 5th and 7th grade students. For instance, 5th grade students require greater supervision and require more direct instruction and guidance when they perform classroom assignments (Hale-Benson, 1982; Hansen, 1996), although they express more openness to group work (Case, 1985; Garmezy, 1994; Hecht & Driscoll, 1994). The 5th grade classrooms allowed for other options regarding plans for instruction (such as potential for more extended visual displays). The scenarios incorporated the more egocentric and family-centered nature of 5th grade students, along with their less sophisticated use of language (including eliminating some of the street language used in the curriculum materials), limited experience with “risky situations” (because of less mobility, freedom, and resources) and more limited future goals and understanding of consequences (Bandura, 1977). The curriculum changes focus more of the activities and homework on family rather than peer settings, accounting for the fact that generally 5th grade students rely more on their families and adults for help and express less of a peer orientation (Hansen, 1991, 1996). Finally, expanding the curriculum from 10 to 12 lessons provided more opportunities for repetition and practice.
The curriculum development specialist recruited 13 teachers from the implementation study schools to participate in 4 separate focus group sessions where they suggested curriculum modifications. These teachers commented on the language appropriateness and readability levels for the key terms. For example, they were asked to identify vocabulary words specific to that grade level and to suggest places to include those words in scenarios and worksheets. Teachers also provided feedback on the complexity and age appropriateness of the concepts and skills presented by offering suggestions of how to break down complex strategies into separate, but linked, everyday situations at school and at home. As a result, teachers could guide students to connect new ideas to prior learning and then use related learning examples from real life to bridge to the new learning focused on resistance strategies. Thus, teachers drew upon the required academic achievement standards and easily recognizable social situations that the focus group participants agreed were common to 5th graders. They offered relevant information, appropriate instructional techniques, and curriculum materials designed to guide the project personnel in adapting the curriculum materials to a new audience defined by different developmental and social characteristics. After modifying the curriculum based on focus group suggestions, study personnel then pilot-tested the curriculum in three classrooms.
Using the suggestions of focus group members and feedback provided during pilot-testing of the curriculum, DRS personnel also edited the five multicultural curriculum videos to reflect the developmental differences discussed in the preceding paragraphs and made the videos more appropriate for 5th grade students. Like the five videos used with the 7th grade curriculum, the first video used in the 5th grade curriculum introduced students to kiR and the resistance strategies and the remaining four videos showed adolescent models implementing the four resistance strategies (refuse, explain, avoid, and leave) identified in previous research (Hecht & Miller-Day, in press). While focus group teachers appreciated the multicultural composition of the videos’ casts, the video editors took their suggestions to shorten the videos to between four and eight minutes (by taking out some of the filler/non-action sections), and edited out scenes of substance use that might instruct naïve students how to use substances. Finally, the curriculum developers used the same process of incorporation of developmental considerations and teacher focus group feedback to adapt the 8th grade booster series for 6th grade students.
All of the changes focused on making the curriculum age appropriate, without degrading program effects. While some sections of the lessons were shortened (e.g., the videos), they retained coverage of all theoretically derived program components. It was anticipated that the longer videos and more complex subject material would not reach the younger audience while the modified materials would retains (and possible increase) its effectiveness through more appropriate message content and form. The addition of two lessons was expected to further enhance the dosage in response to the developmental considerations.
This article reports the immediate (approximately one to two months post-curriculum implementation) and short-term (through the end of 6th grade) effects of the 5th grade kiR curriculum. Given the short-term nature of this evaluation and the low level of use expected in this age group (Andrews et al., 2003; Kreuter et al., 2000), we anticipated that the intervention would demonstrate its clearest effects on the skills and resistance elements taught in the 12 lessons: substance use refusal efficacy, use of the four substance use resistance strategies, and active decision making. As the research on other 5th grade substance use prevention programs showed, the program also may demonstrate effects on some of its psychosocial target mediators such as intentions, normative beliefs, and expectancies. Finally, the evaluation examines program effects on substance use behaviors, although limited use and low variability in use by 5th and 6th grade students (Andrews et al., 2003) discouraged expectations of finding significant effects on those behaviors at this time. The analyses test the following hypothesis:
When compared to students in the control condition, 5th grade students receiving the new keepin’ it REAL multicultural curriculum will experience: 1) a greater increase in refusal self-efficacy and the use of resistance strategies and decision-making skills; 2) less increase in substance use intentions and more positive effects on anti-drug norms and substance use expectancies; and 3) less increase in substance use.
Self-report questionnaire data from the first three waves of an ongoing longitudinal study assessing the impact of a substance use prevention intervention for elementary and middle school students were collected over a two-year period. Students completed baseline questionnaires from September, 2004 to January, 2005 (wave 1), a second round of questionnaires from February to June, 2005 (wave 2) after intervention students participated in the adapted 12-lesson curriculum, and a third round of questionnaires from February to May, 2006 (wave 3) after intervention students participated in from three to six booster activities at their school. At baseline, the 5th grade students (n = 1,566) attended 1 of 23 public middle schools (81 homerooms) in Phoenix, Arizona. Each school was randomly assigned to the adapted multicultural curriculum intervention condition (10 schools) or the control condition (13 schools). Students in six additional schools participated in a third condition of the study, in which they received a new version of kiR which focused on acculturation issues. Since these analyses address the impact of the more general adaptation of the 7th grade substance use prevention curriculum for 5th grade students, they excluded students receiving the acculturation enhanced programming.
Lesson observations by study personnel indicated that the teachers in the multicultural condition implemented the kiR intervention with both high quality (organization, preparation, student participation, student enjoyment, etc.) and fidelity (of instruction, video presentation, student practice, and homework). Teachers implementing the kiR intervention self-reported presentation of all program lessons and activities. Personnel within the control schools implemented their existing substance use prevention programming with their students. In two schools this meant that the students did not receive any substance use prevention programming. On the other hand, project students in seven of the control schools participated in Project ALERT, a different SAMHSA middle school, evidence-based program (NREPP, 2007), during the 5th and/or 6th grades. Project ALERT also focuses heavily on providing students with the skills needed to resist alcohol, tobacco, marijuana, and inhalants, and helps establish non-use attitudes and beliefs, but does not utilize culturally-grounded materials. This program is intended to target middle school students in 7th grade and above, rather than 5th grade students with whom it was implemented in these schools (for more detail, see http://nrepp.samhsa.gov/programfulldetails.asp?PROGRAM_ID=109). Some control schools used other prevention programming such as Gonzo’s 20 Ground Rules (a local program; Communities in Schools of Arizona, 2007) and Red Ribbon Week (National Family Partnership, 2005). Several kiR intervention implementation schools also presented additional substance use intervention programming.
Parents provided informed consent a few weeks before the baseline assessment, while students provided their assent just prior to completing each wave of questionnaires. All consent/assent forms were printed in English and in Spanish. Approximately 82% of the students enrolled in the 23 study schools received parental consent; 96% of those students participated in the baseline assessment; 91% of the students who participated in the baseline assessment also participated at wave 2; and 72% of the students who participated in the baseline assessment also participated at wave 3. Schools reported students transferring out at rates of between 10% and 25% (average transfer out rate of 16%), which accounts for much of the attrition between baseline and wave 3. The previously discussed review of 30 elementary school-based prevention programs found that few of the evaluations of these programs report student participation rates (Kam et al., 2007). Among the few program evaluations that reported student attrition, those rates ranged from 1% to 12% (n = three evaluations reporting), and the evaluations encountered about three to four drop outs of entire schools (of the 6 to 130 participating schools; n = two evaluations reporting).
The present analyses were based on the data provided by 1,566 students who participated in at least one of the three waves (see Table 1). At baseline, the students ranged in age from 7 to 15 years (M = 10.4 years, SE = 0.024 years, n = 1516). Females were 49.7% of the participants; males were 50.2% (one student did not report gender). As an indicator of low socioeconomic status, 87% of the students reported taking part in the “free or reduced price lunch program.”
Approximately 75% of the students self-identified as Mexican, Mexican American, or other Latino; 9.1% self-identified as Black; 4.9% self-identified as White; 2.6% self-identified as Native American; 0.4% self-identified as Asian American; and 7.8% did not report their race/ethnicity. The fact that Mexican and Mexican Americans formed the majority of this sample contributes to the uniqueness of this study since few interventions address the needs of these populations in the United States. Individuals of Mexican heritage form an important target for substance use prevention, since by 8th grade, the lifetime prevalence rate reported by Hispanic students exceeds the rate reported by Whites and Blacks (Johnston, O’Malley, Bachman, & Schulenberg, 2005), and Mexican Americans make up 63% of the Hispanic population in the United States (Knouse, Rosenfeld, & Culbertson, 1992).
At each wave, research team members administered the questionnaire to consented students during a regular, 45-minute, classroom period. These team members emphasized the confidentiality of the data collection, and helped students who encountered any difficulties with the questionnaire items. Students marked their responses on the scannable questionnaire forms. Analyses using Lexile Analyzer (Metametrics, Inc., n.d.) placed the readability of the questionnaires at a 5th grade level.
The questionnaires consisted of 104 to 128 items (depending on the wave), which collected information on sociodemographic characteristics and substance use behaviors, as well as substance use norms, expectancies, resistance strategies, and intentions. At each wave, students had the opportunity to complete the questionnaire in Spanish or English; 8.0% completed the questionnaires in Spanish at wave 1, 7.2% at wave 2, and 4.2% at wave 3.
Sociodemographic characteristics included assessments of gender (“boy” or “girl”) and age (from 7 years to 15 years or older). Students used one forced-choice item (“Choose one best ethnic category which describes you” or later “Which ethnic group describes you best?”) to report their race/ethnicity as: “American Indian or Alaskan Native,” “African American or Black,” “Asian or Pacific Islander,” “White,” “Mexican,” “Mexican American, or Chicano,” “Other Latino/Hispanic (example: Puerto Rican, Salvadoran),” and “Other category.” To obtain an indication of socioeconomic status, students were asked “Do you get a free or reduced lunch at school?” (“Free lunch,” “Reduced cost lunch,” and “Neither”).
Refusal efficacy was assessed with three items adapted from items used by Kasen, Vaughan, and Walter (1992) and measured on a 4-point scale from Not at all sure to Very Sure. The common stem, “Are you sure you would say no if …” was completed in regard to whether “… a family member offered you alcohol?”, “… a close friend offered you marijuana?,” and “… a kid at school offered you a cigarette?” Scale scores consisted of the mean of the item scores, where increasing scores indicated greater confidence in refusing offers. The scale demonstrated a Cronbach’s alpha at each wave of .96, .96, and .93, respectively.
Substance use resistance strategies assessed students’ use of the refuse, explain, avoid and leave strategies in the last 30 days with four items adapted from Hecht et al. (2003). Students received scores of 0 on each of the items if they reported never using the strategy or if they never received offers. They received a 1 on each of the items if they reported using the strategy at least once. Scale scores were calculated by adding the four-item scores, with increasing values indicating use of a greater number of resistance strategies.
Hypothetical alcohol resistance was assessed with five items developed expressly for the parent study to reflect the fact that many of the students had not yet received substance use offers and to allow them to express their possibility of using the four resistance strategies the intervention emphasized. Students responded to the following hypothetical situation: “If your friend offered you a beer at a party, would you …” with the probability that they would “… say ‘No’ without giving a reason why?,” “… give an explanation or excuse to turn down the beer?,” “… just leave the situation without accepting the beer?,” “… avoid getting into that situation because you think beer might be offered there?,” and “… find some other way to not accept the beer?” Each item was scored on a 4-point scale from Definitely Not to Definitely. Cronbach’s alpha for the hypothetical alcohol use scale was .87, .87, and .89, in waves 1–3 respectively.
Students’ active decision-making style was assessed with three Likert-type items modified for age appropriateness from items used by Botvin, Baker, Dusenbury, Tortu, and Botvin (1990), and originally developed by Wills (1986) to assess applied coping strategies. The items, measured on a 5-point scale from Never to Always, used a common stem of “When I have an important problem to solve, I …” which was completed with “… get information to make the best choice,” “… think of different ways to solve the problem,” and “… think about what will happen for each choice before doing anything.” Scale scores consisted of the mean of the three item scores, where increasing scores indicated more use of active decision making. The scale demonstrated a Cronbach’s alpha of .78, .78, and .69, in waves 1–3 respectively.
Intentions to use substances were assessed with three items developed for this study and measured on a 4-point scale from Definitely not, for sure to Definitely yes, for sure. The common stem consisted of “If you had a chance this weekend, would you use… .” The stem was completed with “… alcohol,” “… cigarettes,” or “… marijuana.” Scale scores were calculated by taking the mean of the three item scores. Higher scores represented greater intent to use substances. The respective values of Cronbach’s alpha at each wave were .92, .88, and .86.
Parents’ anti-drug injunctive norms consisted of the mean of three items measured on a 4-point scale from Not at all angry to Very angry, and which expanded upon an item used by Hansen and Graham (1991). Higher scores represented youths’ belief that their parents would be angry if they used substances. The common stem consisted of “How angry would your parents be if they found out you… .” The stem was completed with “… drank alcohol,” “… smoked cigarettes,” or “… smoked marijuana.” The respective values of Cronbach’s alpha at each wave were .91, .78, and .86.
Friends’ anti-drug injunctive norms consisted of the mean of three items measured on a 5-point scale from Very Positively to Very Negatively and adapted from Hansen, Johnson, Flay, Graham, and Sobel (1988). Increasing scores represented youths’ belief that their friends would react negatively if they used substances. The common stem of “How would your best friend react if you …” was completed with “… got drunk,” “… smoked cigarettes,” or “… smoked marijuana.” The respective values of Cronbach’s alpha at each wave were .97, .93, and .93.
Personal anti-drug norms consisted of the mean of three items measured on a 4-point scale from Definitely OK to Definitely Not OK, where higher scores indicated a belief that youth substance use is not acceptable. The three items were adapted from Hecht et al. (2003). The common stem was “Is it OK for someone your age to… .” The stem was completed with “… drink alcohol,” “… smoke cigarettes,” or “… smoke marijuana.” The respective values of Cronbach’s alpha at each wave were .91, .88, and .87.
Descriptive norms were assessed with two items measured on a 4-point scale from Hardly any or None to All or Most and adapted from Hansen and Graham (1991). The two items were: “About how many kids in your school would you guess have used alcohol, cigarettes or marijuana at least once?” and “Now think about the friends you hang out with. How many do you think have used alcohol, cigarettes, or marijuana at least once?” Scale scores consisted of the mean of the two item scores. Increasing scores represented youths’ perception that more kids at school use substances. The respective values of Pearson’s r at each wave were .46, .55, and .46.
Substance use expectancies consisted of the mean of three items adapted from Hansen and Graham (1991) and Hecht et al. (2003) and measured on a 4-point scale from Strongly disagree to Strongly agree. The three items were “Drinking alcohol makes parties more fun,” “Smoking cigarettes makes people less nervous,” and “Smoking marijuana makes it easier to be part of a group.” Increasing values indicated the student expressed more positive substance use expectancies. The respective values of Cronbach’s alpha at each wave were .80, .77, and .74.
Lifetime prevalence was assessed with four yes/no items. Specifically, students were asked “Which of the following have you tried, even if it was only once or only a little? (Mark all that apply).” The response choices were “Alcohol (beer, wine, liquor),” “Cigarettes or tobacco,” “Marijuana (pot, weed),” “Inhalants (sniff glue or paint),” and “None of these.” Students were coded as 1 = “having ever used a substance” if they reported using any one of the substances; they were coded as 0 = “never having used one of the substances” if they had not selected any one of the four substances and they had selected “none of these.”
Past month’s prevalence was assessed with three items asking students how many “drinks of alcohol (more than a sip of beer, wine, or liquor),” “cigarettes,” or “hits of marijuana (pot, weed)” they had in the last 30 days. Since the participating 10-year-old youths reported relatively infrequent substance use in the past month each response on the item’s 7-point scale was coded with 0 indicating no use and 1 indicating any use in the past month. Then, three, wave-specific, binary indicators of any alcohol, cigarette, or marijuana use in the past month were created. Students were coded as 1 = “having ever used a substance” if they reported using any one of the substances; they were coded as 0 = “never having used one of the substances” if they had not selected any one of the three substances.
To obtain summary statistics (e.g., percentages, means, and standard errors), this study used Stata’s svy: mean and svy: tab programs for complex survey samples (Stata Corporation, 2005). These programs helped account for the fact that intact groups of students (23 middle schools) provided the data, by accounting for the intraclass correlation of students within schools (see Cornfield, 1978; Korn & Graubard, 1999). In the present study, the ICCs of the 10 psychosocial variables ranged from .006 to .033 and had a mean of .014.
We used NORM (Schafer, 2000), a freeware program, to generate 10 multiply imputed data sets. Schafer and Olsen (1998) provide a highly accessible account of multiple imputation and the concepts behind NORM. They note that three imputations can achieve an efficiency of 86% when the percent of missing information is 30% and they characterize the latter as a moderately high rate of missing information. For the cited measures, the percent of missing information increased from baseline to wave 3. It ranged from .01% to 36.3% (average 17.9%) depending on the measure and wave.
We used Stat/Transfer to convert the imputed data sets to Stata (Stata Corporation, 2005) and then used a user-contributed Stata command (Carlin, Galati, & Royston, 2008) to: a) fit a random coefficients model with fixed effects for study condition (intervention, comparison), a linear trend component, and the interaction between condition and linear time; and b) apply Rubin’s Rules (Little & Rubin, 2002). The individual estimates from the multiply-imputed data sets were used to calculate a combined estimate, standard error, test statistic, degrees of freedom for the reference distribution, and p-value. Each random coefficients model included random components that varied across participants and accounted for the serial correlation among the three measurements. Each model also included random components that varied across schools and accounted for the intraclass correlation associated with the nesting of students within schools. Generalized linear mixed models were fit to students’ reports of lifetime and past month’s substance use prevalence.
Table 1 displays selected sociodemographic characteristics of the 1,566 middle-school students who participated in one or more of the three assessments during the first 2 years of this ongoing study. As noted above, females were roughly half of the participants in both of the study conditions; Latino students were roughly 74% of the participants in each condition; and students participating in their school’s free or reduced lunch program were roughly 85% of the participants in each condition. A test of homogeneity of proportions indicated that the seven student participation patterns did not vary between the two study conditions [F(3.58, 78.82) = 0.545, p = 0.684]. Thus, there does not appear to be evidence of differential participation. Although it is possible that the students in the two conditions differed with respect to unobserved characteristics, the data presented in Table 1 suggests that they did not differ with respect to some observed characteristics that have been shown to be correlated with substance use among adolescents.
Table 2 displays the cell means and standard errors for students’ refusal efficacy, likelihood of using hypothetical alcohol resistance strategies, and use of active decision making by condition and wave. The findings from the fit of the random coefficients models indicated that the linear trend fit to the means of the multicultural and the control students did not differ for the three variables.
With respect to the number of resistance strategies used in the past month, the linear trend in the mean number of resistance strategies reported by students in the multicultural condition differed significantly from that reported by students in the control condition (t = 5.60, df = 60.7, p < 0.001). By the follow-up assessment, the multicultural students reported greater increases in their quantity of resistance strategies used, albeit to only between one and two resistance strategies on average (while the control group reported using less than one strategy).
Table 3 displays the cell means and standard errors by condition and wave for students’ substance use intentions, parents’ and friends’ anti-drug injunctive norms, descriptive norms, personal anti-drug norms, and students’ positive substance use expectancies. The linear trends fit to the means of the multicultural and the comparison students were significantly different for students’ descriptive norms (t = 4.76, df = 110.7, p < 0.001). By the follow-up assessment, the typical multicultural condition student perceived that relatively more of his or her peers were using substances than did the typical control student. The linear trends associated with the remaining five variables did not differ significantly between the multicultural and comparison students.
Table 4 displays the cell means and standard errors by condition and wave for students’ reports of lifetime and of past month’s substance use prevalence. The linear trends fit to the means of the multicultural and the comparison students were not significantly different for lifetime use (t = –0.80, df = 183.1, p = 0.80) or for recent use (t = 1.64, df = 177.8, p = 0.10).
The 5th grade version of the keepin’ it REAL substance use prevention intervention did not demonstrate a consistent, immediate or short-term effects on participating students’ substance use behaviors or on the expected mediators of the curriculum’s effect on participants’ substance use (i.e., participants’ refusal efficacy, use of resistance strategies, decision-making style, substance use norms, expectancies, or intentions). Of the two significant differences that emerged between the control and the multicultural conditions, one did not support the effectiveness of the intervention. The standard, multicultural version of the intervention does appear to contribute to greater increases in the number of resistance strategies used to respond to substance use offers and thus was successful in one of its primary pedagogical strategies. However, students receiving the intervention also increased in their perceptions of the proportion of their peers who had tried substances more than control condition students.
These findings lead to the consideration of possible explanations for the lack of demonstrated program effectiveness. While keepin’ it REAL is a normative education and resistance skills building intervention that has demonstrated efficacy for 7th grade adolescents in a randomized field trial (Hecht et al., 2003; Hecht et al., in press) including Mexican Americans (Kulis et al., 2005) and even among adolescents who had tried substances prior to the intervention (Kulis, Nieri, Yabiku, Stromwall, & Marsiglia, 2007), these effects may not generalize to the adapted form. Changes to the curriculum that addressed developmental issues such as making the examples more concrete and drawing more specific conclusions for the students may have reduced the effectiveness of the intervention by failing to stimulate their thought processes. In addition, although the adaptation preserved the core components and the lesson plans, the underlying norms and resistance skills model may be inappropriate for the younger cohort, and prevention targeted at this age group may need to address a different set of issues among preadolescents. For example, the increase in descriptive norms or perceptions of peer use in the treatment group may mean that modeling resistance strategies teaches preadolescents about use rather than about resistance. Whereas many of the middle school students have been exposed to substance use and thus were likely to focus on resistance rather then offers, among the younger population this exposure was much more limited and the offers themselves may have been novel and conferred the impression of peer use. The edits to the videos and the curriculum attempted to take this issue into consideration by providing examples of resistance in non-substance related situations. However, the remaining references to substance use may have been enough to raise the salience of such use for the students, particularly those with limited prior exposure. The fact that while perceptions of peer use increased more for the multicultural condition students, but that on average, even those students believed that only “some” of their peers used could indicate that the program enlightened them to the fact that some of their peers use as opposed to almost none of their peers. While the AAPT intervention did decrease descriptive norms in previous studies, perhaps the narrative nature of the kiR intervention manifested its effects differently for a younger audience who interpreted the stories more literally (e.g., the younger students were unaware of peer use and focused on the message that some of their peers were using substances) rather than more figurative interpretation (e.g., the older students know that some of their peers are using and focused on the resistance message).
Moreover, given the younger children’s cognitive abilities, a focus on impulse control rather than or in addition to decision making may prove more efficacious. While our studies demonstrate that preadolescents who use more active decision making styles are less likely to have used substances (Hecht et al., in press), it may be that the preponderance of youths at this age are not ready for an intervention such as keepin’ it REAL that promotes the more active decision making style because they do not have adequate impulse or self control. Without self control, preadolescents will act before engaging in any form of active decision making. The ability to inhibit or control a behavior develops across a lifespan, where the progression of impulse control may vary among children depending on their neurocognitive functioning, Riggs, Greenberg, Kusché, and Pentz (2006). Preadolescence occurs at a time when further developmental integration occurs between affect, behavior, and cognition/ language, integration vital to achieving socially competent actions (Riggs et al., 2006). Mezzacappa, Kindlon, and Earls (1999) found that among 6 to 16 year old boys, cognitive and motivational types of impulse control were positively associated with age; however, impulse control development was delayed for boys with disruptive behavior disorders or attention-deficit/hyperactivity disorder. The success of the PATHS curriculum indicates that younger children can be taught self control (Greenberg, Kusché, Cook, & Quamma, 1995). Thus, future research on substance use prevention should consider variations in youths’ developmental capacities in using impulse or self-control, which in turn, is likely to influence their decision making regarding ATOD use.
However, before drawing the conclusion that a normative education and skills building intervention implemented on students in the 5th grade is ineffective in preventing substance use, some mitigating factors need to be considered. One major contributor to the lack of findings in this analysis appears to be the youth of the sample. As seen in Table 2 through Table 4, these students demonstrated low rates of substance use (either lifetime or recent), weak substance use intentions, and strong anti-drug norms. These 5th and 6th grade students still may be too young and have uniformly anti-substance use expectancies and norms so the intervention can not demonstrate effects in the short term. Expectations of short-term results of such early interventions may be misguided or may need to focus on variables not assessed in this study. Longer term follow-up, as planned through the end of the 8th grade, will track the students through a period of increasing substance use and substance use acceptance, which may allow the program to demonstrate effectiveness in preventing these increases.
The ethnic makeup of the sample, specifically its high percentage of Mexicans, Mexican Americans, and recent immigrants bears consideration. As previous studies documented (Marsiglia, Kulis, Wagstaff, Elek, & Dran, 2005), recently immigrated Mexican youths with lower levels of acculturation experience a delayed initiation in substance use. Intervening at the 5th grade could benefit these youths if they acquired the skills needed to forego substance use, but measuring the impact of the intervention through substance use related outcomes may not be possible at this stage.
Speculation can produce various explanations of the findings; however, competition from other prevention activities in control schools and the community at large almost certainly contributed strongly to the lack of significant differences between the intervention and control conditions. For ethical and practical reasons, the design of this study accepted that control schools would want to implement their existing substance use prevention programming. Most schools implementing prevention programs must now use programs from effective program lists. In fact, most of the control schools participating in this study did use other prevention programming such as Project ALERT (Substance Abuse and Mental Health Services Administration [SAMHSA], 2007), Gonzo’s 20 Ground Rules (a local program; Communities in Schools of Arizona, 2007), and Red Ribbon Week (National Family Partnership, 2005).2 In addition, community and nationwide prevention efforts (such as the “Parents the Anti-drug” media campaign, The National Youth Anti-Drug Media Campaign, 2008), may have minimized the differences between the conditions by strengthening the anti-drug norms of all of the students.
The current findings suggest that culturally-grounded normative education and resistance skill building interventions in the 5th grade may not demonstrate an immediate effect in reducing substance use or affecting the precursors of such behaviors in populations such as the one participating in the current study. Since other programs, notably the Native American substance use prevention program (Schinke et al., 2000), found significant longer term effects, longer-term outcomes for keepin’ it REAL must be examined. Nevertheless, for interventionists seeking immediate effects, it now appears likely that targeting preadolescents in elementary schools may require a different form of intervention or different outcome measures. Future research should consider and evaluate alternative strategies while waiting for the results of longer-term studies on programs such as keepin’ it REAL.
*This publication was supported by Grant Numbers RO1 DA005629 and T32 DA017629 from the National Institute on Drug Abuse to The Pennsylvania State University (Grant Recipient), Michael Hecht, Principal Investigator, with Arizona State University as the collaborating subcontractor. Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the National Institutes of Health.
2Additional analyses compared the intervention students to students in the few control schools which did not implement other SAMHSA model substance use prevention interventions. These analyses did not alter this article’s conclusions. The low number of schools and the low power of these analyses prevent drawing strong conclusions.
Michael L. Hecht, The Pennsylvania State University, State College.
Elvira Elek, The Pennsylvania State University, State College.
David A. Wagstaff, The Pennsylvania State University, State College.
Jennifer A. Kam, The Pennsylvania State University, State College.
Flavio Marsiglia, Arizona State University, Phoenix.
Patricia Dustman, Arizona State University, Phoenix.
Leslie Reeves, Arizona State University, Phoenix.
Mary Harthun, Arizona State University, Phoenix.