|Home | About | Journals | Submit | Contact Us | Français|
This study examined how ethnic composition and linguistic acculturation within schools affected the efficacy of a youth substance use prevention model program. Data come from a randomized trial of the keepin' it REAL program, using a predominantly Mexican American sample of middle school students in Phoenix, Arizona. Schools were randomly assigned to a control group or to one of three culturally tailored intervention versions. We hypothesized that school ethnic and linguistic acculturation composition (percent Latino, percent non-English speaking at home) and individual level of linguistic acculturation jointly would moderate the efficacy of the prevention program, as indicated by students' alcohol, marijuana, and cigarette use. Using multilevel linear modeling and multiple imputation techniques to manage clustered data and attrition, results showed that desired program effects varied by the linguistic acculturation level of the school, the program version, and individual acculturation level. The Latino intervention version was more efficacious in schools with larger percentages of non-English speaking families, but only among less linguistically acculturated Latino students. There were no significant school level program effects connected to the percentage of Latino students at school, the other versions of the program, or among more linguistically acculturated students.
Drawing on prior work suggesting that some predominately Latino schools may create uniquely protective environments for Latino students, at least in terms of their academic outcomes (Morrison Institute for Public Policy, 2006; Goldsmith, 2003), the present study tests whether the same may be true for substance use. It examines whether Latino predominance and the mix of more and less acculturated students in schools enhances drug prevention program outcomes among Latino students. In addition, it explores whether the effects of school composition on program efficacy differ according to the individual student's acculturation level.
Whether an intervention focuses on individual or contextual (e.g., peer, family, community) factors, it is implemented in a specific context, the characteristics of which may influence the intervention's success. Many interventions have been shown to be efficacious in randomized controlled trials, but these results are tied to the contexts in which the trials were conducted. They do not guarantee success in alternate contexts, nor do they reveal whether the programs could have been more successful in alternate contexts. Effectiveness and replication studies aim to test program effects in alternate settings, but they leave open the possibility of program failure due to implementation in a context that undermines effectiveness, such as when there is a poor match between the intervention's design and content and the social and cultural context where it is implemented. An alternative approach to contextual efficacy is to explicitly examine contextual factors that may enhance or undermine program efficacy for a given intervention. With advance knowledge of the contexts to which interventions are best suited, organizations can select interventions that are appropriate for the target context and not merely for the individuals at risk in that context.
Recent research sheds light on the distinction between the individual and contextual levels when it comes to program efficacy. One study examined the efficacy of a universal substance use prevention program for Latino youth at different levels of risk for substance use, showing that the program achieved larger desired program outcomes for higher-risk youth, and this effect was explained in large part due to treatment floor effects among the low-risk youth (Marsiglia, Kulis, Wagstaff, Elek, & Dran, 2005). A subsequent study including neighborhood level predictors found that, for the high-risk youth, program efficacy was not moderated by neighborhood context as defined by immigrant composition (Yabiku, Kulis, Marsiglia, Lewin, Nieri, & Hussaini, 2007). The high-risk youth had similar program outcomes, regardless of the immigrant composition of the neighborhood. In contrast, the lower-risk youth reported better program outcomes when they lived in neighborhoods with higher concentrations of immigrants. If program selection were based solely on the individual level of risk in the community, communities with low levels of individual risk might overlook this program, viewing it as appropriate only for communities with a high level of individual risk. However, as the study showed, the program is efficacious for low-risk youth who live in immigrant communities. And when it comes to prevention among youth, maintaining the protections that make youth “low risk” is valuable, since it is much harder to change, than to maintain, the status quo.
To address the need for contextual efficacy research, the present study examines the influence of the school context on the efficacy of the keepin' it REAL youth substance use prevention program. keepin' it REAL is a school-based program, shown in a randomized trial to be efficacious in preventing substance use, strengthening anti-drug norms and attitudes, and increasing the use of drug resistance strategies (Hecht, Marsiglia, Elek, Wagstaff, Kulis, & Dustman, 2003; Kulis, Marsiglia, Elek, Dustman, Wagstaff, & Hecht, 2005; Marsiglia & Hecht, 2005). It is recognized by the U.S. Substance Use and Mental Health Services Administration (SAMHSA) as a model program. The intervention's name comes from an acronym of the four drug refusal skills that are stressed in the curriculum: Refuse, Explain, Avoid, and Leave. The published curriculum (Marsiglia & Hecht, 2005) was created using a participatory action research approach that promotes input and dialogue from the community as well as from youths themselves (Gosin, Dustman, Drapeau, & Harthun, 2003).
A strength of the curriculum is that it was designed, from the start, to be culturally-sensitive, making use of long-standing ethnic values that could aid youth in their attempts to avoid substance use (Castro, Proescholdbell, Abeita, & Rodriguez, 1999). Three versions of the program exist: Latino, Black/White, and Multicultural, each with a unique cultural emphasis. For example, in designing the Latino-based curriculum, program planners incorporated aspects of traditional Mexican culture into the program (Gosin, Marsiglia and Hecht, 2003). All versions entail a 10-lesson, in-school, classroom-based curriculum designed to promote drug resistance and life skills (Botvin, Griffin, Diaz, & Ifill-Williams, 2001). Overall, the curriculum's goals are to promote norms and attitudes that discourage substance use, improve students' decision-making skills, and increase their awareness of and ability to assess risks. For full details on the design of the program (including qualitative phases of development), its theoretical underpinnings, and the specific features of the local ethnic populations that influenced the program content, see Holleran, Dustman, Reeves, & Marsiglia (2002) and Gosin, Marsiglia and Hecht (2003).
Because keepin' it REAL is school based, attention to the school context in which it is implemented is warranted. Furthermore, the cultural grounding of the intervention calls for a focus on cultural characteristics of the context, hence this study's examination of school composition defined in terms of the proportion of Latino students and the proportion of less linguistically acculturated students. Because there is little theory to guide thinking specifically about the effect of contexts on interventions, we must draw on theoretical and empirical work about the relationship between contexts and individual outcomes.
Eco-developmental theory (Szapocznik & Coatsworth, 1999) posits that an understanding of adolescent outcomes requires a consideration of the social systems or contexts in which development occurs, and focuses on the ways in which the systems influence one another and the ways in which these sequences affect development. According to the theory, a school context characterized by high rates of use and pervasive pro-drug norms and attitudes might contribute to substance use at the individual level.
Findings from studies of adolescent substance use are consistent with eco-developmental theory. Ennett and colleagues (1997) found, for example, that schools that were more pro-drug use at the aggregate (or school mean) level had higher levels of lifetime and current cigarette use and lifetime alcohol use. Kumar and colleagues (2002) found that school-level disapproval of daily cigarette use, heavy drinking, and marijuana use was associated with lower probabilities of students' substance use, even among students who themselves approved of daily cigarette use. These authors argued that through a social learning process (Bandura, 1986), students within schools develop similar habits through the diffusion – or contagion – of prevailing norms (Ennett, Flewelling, Lindrooth, & Norton, 1997). Johnson and Hoffmann (2000) found that Latino students' and African American students' cigarette use decreased as the school's ethnic minority composition increased, and both groups reported less cigarette use than Whites.
Drawing on the large literature associating adherence to Latino cultural norms and practices with less drug use (Warner, Valdez, Vega, de la Rosa, Turner, & Canino, 2006; de la Rosa, 2002) and on Stanton-Salazar's (2004) work showing that youth from marginal groups in society can acquire social capital by connecting in school with like peers, Kulis and colleagues (2004) examined whether Latino ethnic composition affected individual drug use norms or behavior. They found an effect but only for less acculturated, Mexican heritage students, for whom it was associated with less substance use and less adherence to pro-drug norms. This effect was attributable to the larger presence of less acculturated Latinos in the school rather than to the proportional representation of more acculturated Latino students or Latinos as a whole. The researchers suggested that the less acculturated youth benefited from the substantial presence of like peers who not only reinforced the anti-drug social capital obtained through family and community sources but also counterbalanced the pro-drug normative influences of their more acculturated peers. This work suggests that a school context that protects against substance use at the individual level may also reinforce the prevention messages of an intervention. It also suggests that a school's Latino ethnic and acculturation composition may improve individuals' intervention outcomes by providing a normative environment that is conducive to the attitudinal and behavioral changes promoted by the intervention.
Certain characteristics of Latino communities appear to protect against substance use. For example, due to the Latino values of familismo (orientation to family) and respecto (respect for parents and other adults), Latino youth experience more adult supervision and involvement, which protect against substance use (Dishion, Burraston, & Li, 2003), and they are more strongly influenced by parental proscriptions than are youth from some other ethnic groups (Chandler, Tsai, & Wharton, 1999; Vega & Gil, 1998; Smith & Krohn, 1995). The large number of immigrants that make up many Latino communities may also provide protection. Immigrant neighborhoods often have closer knit networks of neighbors and families and thus more effectively exert control over youth (Portes, 1997; Zhou, 1997). These studies suggest that the characteristics of a Latino school population may support the aims of a substance use prevention intervention.
Research on cultural protections has focused on the advantages of low acculturation and the disadvantages of high acculturation (Warner, Valdez, Vega, de la Rosa, Turner, & Canino, 2006; de la Rosa, 2002), documenting, for example, the positive association between linguistic acculturation (greater use of English relative to the language of origin) and pro-drug norms and attitudes and actual substance use (Delva et al., 2005; Kulis, Marsiglia, Nieri, Sicotte, & Hohmann-Marriott, 2004; Epstein, Doyle, & Botvin, 2003; Marsiglia, & Waller, 2002; Unger, Cruz, Rohrbach, Ribisl, Baezconde-Garbanati, Chen et al., 2000; Wagner-Echeagaray et al., 1994). Yet, differences among Latinos by acculturation leave open the question of whether the acculturation context rather than the Latino ethnic context is the key influence. Thus, we examine here both the school Latino composition and the school acculturation composition as potential moderators of program efficacy and whether these effects differ by individual acculturation level. Among acculturation measures, we focus on linguistic acculturation because it has been shown to account for up to 65% of the variance in multidimensional acculturation measures (Rogler, Cortes, and Malgady, 1991) and is a well-established covariate of substance use (Marsiglia, Kulis, Wagstaff, Elek, and Dran, 2005; Marsiglia and Waller, 2002).
Because programs are less successful when they are culturally inappropriate for the participants (Kumpfer, Alvarado and Whiteside, 2003) and more successful when they reflect participants' cultural values and norms (Garcia-Reid, Reid and Peterson, 2005), we anticipate that the school effect will differ according to keepin' it REAL's three culturally specific versions. The Latino version is likely to interact most strongly with the school Latino and acculturation composition since it maximizes the cultural match between intervention and context. Thus, we hypothesized that the intervention would be significantly more efficacious—as reflected in changes in substance use relative to those observed in the control group—in schools with higher percentages of Latino students and higher proportions of less acculturated students, but that these desired program effects would emerge most strongly for students receiving the Latino version of the intervention.
The keepin' it REAL substance use prevention study was implemented in Fall 1998 in 35 middle schools in Phoenix, Arizona. The study was conducted in predominantly Hispanic, mostly low income neighborhoods, and a minority of wealthier schools with non-Hispanic White majorities. A detailed description of the intervention's development, multiple versions and components, and randomized control trial study design are reported in Hecht et al. (2003), along with results that established its efficacy and led to its designation as a SAMHSA model program.
Following university and school district policies for human subjects' protections, passive parental consent was obtained for their children to complete surveys that evaluated the efficacy of the intervention. University trained proctors informed students about the nature of the study, guaranteed them confidentiality, and told them they could elect to participate or not without any negative consequences. Before the prevention programming began, students completed a pre-test survey measuring their past use of substances, norms and attitudes towards use, grades and school aspirations, and family and background characteristics. Students chose whether to complete the self-administered surveys in English or Spanish. Some students were not in school on the day the survey was given due to absences, yet the overall completion rate was high: 87% of students who were listed as officially enrolled in the schools completed the pre-test survey. After the pre-test was administered, the keepin' it REAL substance use prevention program was started in 25 of the 35 study schools with the remaining 10 schools serving as controls. The program was administered by the students' regular teachers, after a one-day training by the research team, and consisted of ten 45-minute lessons spread over 10 weeks.
Whether a school received the prevention program or the control was determined by block randomization, adjusting for school size and school ethnic composition (% Latino). The schools in the control condition continued their usual substance use prevention activities, in accordance with state mandates. In spring 1999, the first follow-up, or post-test, survey was given to all students (both treatment and control conditions). This post-test survey was completed by 60% of the study participants utilized in this analysis (those self-reporting as Latino/a). This follow-up was approximately two months after the curriculum for the prevention program had been delivered in the treatment schools. The content of these surveys replicated many of the pre-test survey items, such as measures of substance use behaviors, norms and attitudes, so that differences from the first to the second survey could be compared and treatment effects of the prevention program could be estimated.
The use of three substances was measured in both the pre-test and post-test surveys: alcohol, cigarettes, and marijuana. The question about alcohol asked, “How many days in the past 30 days have you had alcohol to drink (do NOT count for religious services)?” Responses were measured on a scale from 1 to 6, where 1 was “none” and 6 was 16-30 days. Similar questions were asked of cigarettes and marijuana. Because distributions tended to be skewed toward low frequency of use, for analysis they were transformed using a logarithmic function. Once transformed, the skewness coefficients of all dependent variables were within the bounds of -3 and 3, a typical rule of thumb for variables to be considered appropriate for regression (Peat and Barton 2005). Although the use of other substances was also measured on the survey, including crack, smokeless tobacco, inhalants, uppers, downers, and LSD, the use of these substances was too infrequent to be reliably analyzed for change. As in other studies evaluating the efficacy of youth prevention programs (Botvin et al., 2001), we focus on the “gateway” substances that are used most commonly in this age group: alcohol, cigarettes, and marijuana.
Three different versions of the keepin' it REAL substance use prevention program were tested in the study. Each was a culturally sensitive intervention aimed at building upon the existing strengths in various ethnic and racial groups. The first version was a Latino version that incorporated aspects of traditional Mexican culture. Although other ethnicities fall under the broad term Latino, in the Phoenix area the overwhelming influence in the Latino community is from Mexico. The Latino version of keepin' it REAL stresses issues that build upon the strengths of Latino culture. These include an emphasis on familismo—the elevation of family issues above those in other life realms—and the general preference given to collectively expressed aims rather than an exclusive concern with individual achievement and advancement. The next version of the prevention program was an African American and European American version (or “Black/White” for short), which borrowed strategies and conventions most strongly associated with those cultures. These two cultures were combined in this version because of limitations of the sampling frame, in particular the relatively small numbers and proportions of African American students in the study schools, which made it impractical to develop, deliver, and test a separate African American version of the curriculum. Lastly, there was a multicultural version of the program. This version mixed elements from both of the first two versions of the program, thus representing Latino, African American, and European American culture. To differentiate schools that received one of the three versions of the prevention program or the control condition, we use dummy variable coding. For example, students in schools with the Latino version of the program are coded 1 on the Latino version indicator and are coded 0 on the Black/White version, Multicultural version, and control condition. In the analyses, the control condition is used as the reference group in order to assess treatment effects.
Our hypotheses predict that the different versions of the prevention program will vary in efficacy depending on the Latino composition of the school and the student body's linguistic acculturation (as measured by their use of English as opposed to Spanish). The measurement of both of these factors is from sources independent of our prevention study. The State of Arizona requires that schools report the Latino composition of their schools as well as what percent of students primarily speak Spanish at home. This information is needed so that that state can effectively distribute resources to help students learn English, such as tutors, learning materials, and translators for non-English speaking parents at parent-teacher conferences and meetings. We use both of these school-reported measures in our analyses. Note that it is advantageous that these school-level measures are collected independently of the pre-test and post-test surveys. If we were to obtain these measures through the study surveys, they would be sensitive to missing data bias due to class absences or youths incorrectly filling out the surveys. In the multivariate analyses, which involved tests of the interaction between program version and school ethnic or home linguistic composition, these measures are centered so that their mean is zero.
We use several measures from the students' pre-test survey as control variables that the literature has identified as predictors of substance use. Gender is measured with a dichotomous variable coded 1 for male, 0 for female. Students' academic performance was measured via self report with a question that asked, “What grades do you usually get in school?” Responses ranged from “mostly F's” which were coded 1 to “mostly A's” which were coded 9. We control for the students' socioeconomic status with a question that asked if they participated in a school lunch program: “Do you get a free school lunch or a reduced cost school lunch?” If a student received either a free or reduced cost lunch, the variable was coded 1, otherwise it was coded 0. Students often do not know their parents' income levels, and free or reduced lunch status has been found to be a reasonable proxy of parental socioeconomic status (Bankston & Caldas, 1996).
A final individual-level measure is the students' ethnicity and level of linguistic acculturation. In the pre-test survey, students were allowed to self-identify their racial and ethnic status in multiple categories. Students marked yes or no if they identified with each of the following categories: Mexican American/Chicano, Other Hispanic, White (Anglo), African American (Black), American Indian, or Asian/Pacific Islander. For our analysis sample, we use only students who were Latino—that is, students who marked Mexican American/Chicano or Other Hispanic. Students who were Latino and indicated other non-Latino heritages were also included. In other words, if a student marked “Mexican American/Chicano” and “American Indian,” he or she was included in the analysis. Latinos represented the majority (66%) of all students who participated in the study
Within the Latino sample, however, there was large variation in linguistic acculturation. Prior research shows that acculturation levels are associated with substance use related outcomes (Zemore, 2005; Morenoff, 2003; Escobar, 1998; Gil & Wagner, 2000; Landale, Oropesa, Llanes, & Gorman, 1999), and thus it was necessary to disaggregate this heterogeneous Latino sample. We divided the sample into Less Linguistically Acculturated and More Linguistically Acculturated groups. Students were divided based on their answers to two questions. One asked “When you talk with friends, what language do you usually speak?” The other asked about the language used with family members. Responses to both questions were measured on a five level ordinal scale that ranged from Spanish only, mostly Spanish, Spanish and English equally, mostly English, to English only. Responses to the two questions were averaged, and students who scored 3 or less were considered Less Linguistically Acculturated, and students scoring more than 3 were considered More Linguistically Acculturated.
We use multilevel linear regression to predict the level of post-test use of alcohol, cigarettes, and marijuana while controlling for the pre-test level of use. We use multilevel linear regression, as opposed to treating our outcomes as dichotomous, because even in this young sample there is variation among the users, and we do not want to ignore this important heterogeneity. Because these substances have different levels of use and patterns of initiation (Johnston, O'Malley, Bachman, & Schulenberg, 2006), we estimate models separately by substance. Our hypotheses predict differential program effects by two school-level measures: the percent school Latino and the percent of students who are non-English speaking at home. To test these hypotheses, we create interaction terms that are the products of the program indicators and the school-level measures. We also hypothesized that program effects vary by the students' levels of linguistic acculturation, and thus we estimate models separately for two groups: More Linguistically Acculturated and Less Linguistically Acculturated.
Two methodological issues complicate the models. First, because the students were sampled in schools, the data are clustered and violate the independence assumptions of ordinary least squares linear regression. Multilevel modeling methods are able to recognize the clustering in the data and avoid the added risk of Type I errors that typically occur when clustering is ignored (Raudenbush & Bryk, 2002). We use the PROC MIXED procedure in SAS 9.1 to estimate multilevel models with random intercepts. The random intercepts allow each school to have a different base level of substance use that is the result of unobserved, school-level factors (Raudenbush & Bryk, 2002). Our multilevel models thus account for the clustering in the sampling design and ensure appropriate estimates for the hypothesis tests.
The second issue is missing data. At the pre-test survey in Fall 1998, 2,894 Latino students participated. At the post-test survey in Spring 1999, there were only 1,744 Latino students. The most typical reasons for attrition at the post-test were students who moved to another school as well as student absences on the day of the survey. In addition, there was “planned missing,” i.e. item missing data in which students took the survey but did not complete all subgroups of questions; the omitted groups of questions were systematically rotated in order to accommodate more survey items without overburdening respondents. Ignoring missing data can create bias because the missing cases are often significantly different than the non-missing cases (Allison, 2002). To address missing data in our analysis, we use multiple imputation methods. These methods have been used previously in studies of program efficacy (Graham et al., 2002; Hecht et al., 2003). Although our missing data rate due to attrition was about 40% (1-1744/2894), there are no guidelines regarding how much missing data is too much (Gorelick 2004; Musil et al. 2002; Tsikriktsis 2005; Kristman, Manno, and Côté 2004). Some simulation work suggests that rates of missing data as high as 60% are not problematic, as long as methodological assumptions are met (Kristman 2004). The important assumption of this method is that the data are missing at random (MAR), conditional on the variables that have been observed. For missing data due to attrition or item non-response, this assumption is not testable, but the assumption can be strengthened by including all relevant predictors that may be related to a case being missing. Note that planned missing data, which is present in the survey, is an ideal application for multiple imputation because by design these missing data are missing completely at random (MCAR) (Little and Rubin 2002). Using the MI procedure in SAS 9.1, we created 10 imputed datasets. In the imputation model, we included all variables used in the analysis as well as variables likely to be correlated with the process leading to missing data: substance use norms, attitudes, and expectations and educational expectations. Once the data are imputed, the datasets are analyzed with complete-data methods and results from the analyses are combined using the MIANALYZE procedure in SAS to arrive at the correct estimates.
Descriptive statistics (means, standard deviations, percentages for dummy variables) for study variables are in Table 1. Because we hypothesized that the interaction between school-level variables and program effectiveness varies by linguistic acculturation, we divided the sample into two groups: More Linguistically Acculturated and Less Linguistically Acculturated Latino students. There are several notable differences in substance use patterns and background variables between these two groups. At both pre-test and post-test, the Less Linguistically Acculturated students have lower levels of recent use of alcohol, marijuana, and cigarettes. This is consistent with prior literature that has found that as Latino youth become more linguistically acculturated into English and Anglo culture, they have higher levels of substance use. Another pattern is that for both groups, substance use consistently increases from pre-test to post-test. This is also an expected finding. Middle school is a time when youth are increasingly experimenting with substances, and even over the short period from pre-test to post-test, there are increases in use. The focus of our analysis, however, is how different versions of the prevention curriculum dampen the increase in use and how this varies by school characteristics (context) and levels of linguistic acculturation.
There were differences in the environments in which the two groups of students went to school. Not surprisingly, Less Linguistically Acculturated students went to schools in which more of the student body did not speak English at home (58% versus 49% for the More linguistically Acculturated students). They also went to schools with a proportionally larger Latino student body composition (75% versus 67% for More Linguistically Acculturated students). There was no difference in the gender composition of the two groups; both were split almost evenly by gender (51% male). Grades were also similar, averaging between categories 6 (“mostly B's and C's”) and 7 (“mostly B's”) on the self-reported grades scale. The Less Linguistically Acculturated students had higher participation in the federal free or reduced lunch program (95%), compared to the More Linguistically Acculturated students (84%). This suggests an even higher level of socioeconomic disadvantage for the Less Linguistically Acculturated students. Because the study used a randomized trial, there were no large differences in the assignment of treatment conditions to the two groups. The small differences that exist are due to random variation.
Tables 2 and and33 show the multilevel linear regression results for the Less Linguistically Acculturated students. Each table has three models predicting the use of a separate substance at post-test: alcohol, cigarettes, or marijuana. Table 2 examines how program effectiveness among the Less Linguistically Acculturated students varies by the proportion of students who are non-English speaking at home. Before discussing program effects, we briefly note the effects of control variables, which are generally consistent with the past literature. As expected, pre-test substance use was significantly correlated with post-test use. Higher grades were significantly protective against increases in use of all substances. Being male increased use, but this was not significant, and free or reduced lunch status—our proxy for low socioeconomic status—was negative and only significant in predicting increased use of marijuana.
Our main concern in the multivariate models, however, is the interaction between treatment (keepin' it REAL substance use prevention program) and school-level measures. The overall efficacy of the treatment program has been shown (Hecht et al., 2003), and thus the main effects of treatment indicators are of secondary concern. In fact, significant main effects may become insignificant in interaction models due to associations between the main effect and the interaction term (Aiken & West, 1991). In these interaction models, a significant interaction coefficient between treatment and school measures indicates that program efficacy significantly varies across the school variable (Allison, 1977).
The variations in program efficacy against alcohol use are tested in model 1 of Table 2. Our focus is the three variables that create interactions between the different versions of the program (Black/White, Multicultural, and Latino) and the percent of the students in the school who do not speak English at home. The interaction terms of the Black/White version and the Multicultural version are not significant, suggesting that efficacy of these versions does not vary across the school measure. The interaction of the Latino version and the school percent non-English speaking at home, however, is significant. The coefficient is negative, which means that the program is more effective at decreasing alcohol use as the percent of students who are non-English speaking at home increases. In addition, the size of the interaction term for the Latino version, in comparison to the interaction terms for the other versions, is large: -.46 versus -.17 for the Black/White version and -.03 for the Multicultural version.
Model 2 of Table 2 estimates the same model but for cigarette use. As in model 1, the interaction terms of the Black/White and Multicultural versions are not significant. The interaction for the Latino version, however, is significant, indicating stronger program effects when the school has greater proportions of students who do not speak English at home. The size of the interaction effect is large: -.48 for the Latino version and only .04 and -.07 for the other two versions. Lastly, Model 3 examines variations in program efficacy for marijuana use. As with the prior two substances, there is evidence that the Latino version—but not other versions of the program—is significantly more efficacious in preventing marijuana use when the school has a greater percent of students who speak Spanish with their family members at home.
Table 3 continues the analysis with the Less Linguistically Acculturated students, changing the school level predictor to the percent of students who are non English speaking at home to the percent of students who are Latino. Recall that we distinguish these two measures because a measure of Latino composition in the school is an indicator of ethnicity, and the measure of non-English use at home is an indicator of acculturation. Unlike Table 2, however, the models in Table 3 show no significant interaction between any of the program versions and the percent of the school who is Latino. The interaction coefficients for the Latino version are consistently negative and have the largest coefficients, but none are significant (the closest to significance is the Latino interaction in the marijuana use model, p=.07).
The analyses in Table 4 switch to a different sub-population: the More Linguistically Acculturated Latino students. Compared to the results for the Less Linguistically Acculturated students, the More Linguistically Acculturated students do not appear to benefit from a school environment in which there are more students who are non-English speaking at home. None of the program interaction terms are significant. The Latino version interaction terms are negative, but not significant (the closest to significance is the model predicting alcohol use, p=.09). The results in Table 5 are similar. Table 5 examines how program efficacy varies by the school Latino composition. Again, Latino version interaction terms are negative but not significant, with the interaction term for the alcohol model being the closest to significance (p=.12).
This study speaks to the growing literature on program efficacy versus program effectiveness (Greenberg, 2004). The effectiveness of a program refers to how a program performs under “real-world conditions,” rather than its performance in a highly controlled and randomized trial (Greenberg, 2004). Although the data for this study came from a randomized controlled trial, the findings show that characteristics of schools, such as their ethnic or language use composition, are ecological features that impacted program efficacy and thus would be likely to enhance or detract from program effectiveness in real-world interventions.
Exploring the interaction of treatment with these ecological factors can be beneficial in different ways. First, knowing how a program may perform in specific ecological contexts can better inform stakeholders so that they can choose the best program for their needs and unique situation. Second, as these ecological factors are understood more clearly and as empirical results become consistent, these factors can be explicitly incorporated into prevention programming. As more researchers investigate how programs are modified by their ecological contexts, prevention research and outcomes for program participants are likely to improve.
The present study highlights the importance of considering social context when selecting school-based prevention interventions and assessing their efficacy. The findings expand on eco-developmental theory (Szapocznik & Coatsworth, 1999) by identifying specific school characteristics that are protective and lead to higher levels of program efficacy. Social context—in this case the school's percentage of students from non-English speaking homes, a proxy for acculturation—was associated with variations in the efficacy of the keepin'it REAL intervention. This contextual influence, however, was conditional: it emerged only for one of the culturally grounded versions of the intervention, the one focusing on Latino culture, and it applied only to the sub-group of students who were relatively less linguistically acculturated. This finding provides evidence for the argument that culturally specific interventions (Latino version) which match the social and cultural contexts of the schools can yield higher levels of efficacy than an intervention with less of a contextual match. By comparison, there was little evidence that school ethnic composition (percent Latino) influenced program efficacy. Together, the findings suggest that acculturation-related differences among Latinos are more important factors in responsiveness to this prevention program than ethnic background alone.
We did not find that school context moderated treatment effects for More Linguistically Acculturated students. It may be that these students, relative to their less linguistically acculturated peers, have broader exposure to the mainstream Anglo culture whether through more social interactions with Anglos or more consumption of English language media. This exposure may offset the potentially protective effects of a less acculturated school context. Furthermore, it may be that because these students have greater access to the wider environment, the school environment is less salient to them. If so, its particular characteristics may not influence the students' responsiveness to the intervention. Less acculturated students may rely more heavily on the school environment due to less familiarity with the broader environment or an inability to navigate that environment due to limited English proficiency.
The present findings also speak to the question about when to intervene in different contexts. The contexts where the Latino specific intervention was found to be significantly more efficacious—schools with many students from non-English speaking homes—are schools that commonly serve immigrant families, and in this sample, families likely to retain strong Mexican/Mexican American ethnic and cultural identification. This raises the question whether certain levels of cultural identification are helpful in making a culturally specific intervention work effectively. To intervene later in the youth's developmental trajectory or when students have already moved farther along the acculturation continuum may require other types of interventions that include the newly acquired values, norms and behaviors.
A limitation of these analyses is that we did not measure the actual culture that the school composition represents. For example, we did not have measures of the school norms and policies related to substance use or, for that matter, of the cultural attitudes of students, parents, teachers, and administrators. It would be important to attempt to capture these variables in future research in order test the assumption that the school composition shapes the school's cultural attitudes and norms that promote drug resistance.
Another limitation is the absence of data on specific mechanisms of contextual influences. What is it about schools with a greater proportion of non-English speaking homes that makes the less acculturated students more responsive to the prevention program? For example, perhaps these schools have greater parental involvement, or special attention to acculturated related issues that the students are navigating with their families, and these factors contribute to an environment more supportive of responsiveness to the prevention program. This possibility and others should be explored so that modifiable factors can be identified and addressed to maximize prevention efficacy.
These findings show that context matters for program efficacy. It is important to recognize not only where individual students have arrived in their acculturation journeys but also to assess the collective context of the school community. Program developers and consumers need to assess the characteristics of the environment before selecting or implementing an intervention. But just as it is nearly impossible to replicate a study in every context, isn't it nearly impossible to examine every potential contextual effect on program efficacy? Fortunately, an examination of every context may not be necessary. Program developers in concert with stakeholders can focus on potentially key contextual variables based on their understanding of the program and the cultural and social characteristics of the target population. Furthermore, researchers may not even have to collect additional data if they have the foresight to consider and collect information on contextual variables in the design of their randomized controlled trials. School social workers in partnership with other school personnel can utilize the results of school-based assessments. The environmental or contextual knowledge generated from such efforts can be applied to all aspects of school life.
Future research should focus on identifying key contextual variables that can be part of a repertoire of measures readily available from school districts when assessing the potential match or mismatch of intervention content with the social context or environment of the targeted population. Allowing the time and resources to conduct such assessment can in the long run assure higher levels of efficacy and therefore increase the cost effectiveness of the intervention.
This research was supported by National Institutes of Health/National Institute on Drug Abuse grants funding the DRS Next Generation project (R01 DA14825) and the Southwest Interdisciplinary Research Center (R24 DA13937).