|Home | About | Journals | Submit | Contact Us | Français|
The study's objective was to examine the effects of “real world,” community-based implementation of universal preventive interventions selected from a menu, including effects specific to higher- and lower-risk subsamples.
The study included twenty-eight public school districts in Iowa and Pennsylvania that were located in rural towns and small cities, ranging in size from 6975 to 44,510. Sixth and seventh graders in these school districts participated in the study.
School districts were selected on size and location and then randomly assigned to a control condition or to an experimental condition in a cohort sequential design.
Community teams were mobilized; each team implemented one of three evidence-based, family-focused interventions (5–12 sessions) and one of three evidence-based school interventions (11–15 sessions)—for 6th and 7th graders, respectively. Observations showed interventions were implemented with fidelity.
Outcomes included student reports of past month, past year, or lifetime use of alcohol, cigarettes, marijuana, methamphetamines, ecstasy, and inhalants, as well as indices of gateway and illicit substance initiation, at pretest and at a follow-up assessment 1½ years later.
Intent-to-treat analyses demonstrated significant positive effects on substance initiation (marijuana, inhalants, methamphetamines, ecstasy, gateway index, illicit use index), as well as past year use of marijuana and inhalants, with positive trends for all substances measured. For three outcomes, intervention effects were stronger for higher-risk students than lower-risk students.
Community-based implementation of brief universal interventions designed for general populations has potential for public health impact by reducing substance use among adolescents.
Substance use among adolescents is a major public health problem, with substantial health, social, and economic consequences. National surveys show continuing high rates of adolescent alcohol, tobacco, marijuana, and other illicit drug use that constitute a major cause of morbidity and mortality.1 Forty-one percent of 8th graders report lifetime alcohol use, 25.9% cigarette use, and 16.5 % marijuana use. The use of illicit “hard” drugs is especially of concern; 12.1% of eighth graders report lifetime illicit drug use, other than marijuana. Among illicit drugs, methamphetamine use continues to be a problem, with 3.1% of 8th graders reporting lifetime use. The public health-related consequences of adolescent substance use are well documented and include adverse affects on both mental and physical health,2–6 with substantial economic costs.7–8
Preventable factors and conditions that contribute to adolescent use have been treated thoroughly in the literature.9 Because so many risk and protective factors in adolescent substance use have their origin in family and school environments, a number of universal preventive interventions have been designed with components that specifically target those factors. Universal interventions (those that target general populations) are particularly suitable for addressing the public health challenge represented by adolescent substance use;10 earlier outcome research demonstrates their promise in terms of longitudinal, community-level outcomes and economic benefits.11
A number of researchers have argued that a major barrier to public health impact of universal interventions is a lack of rigorous effectiveness studies of those interventions.12 Effectiveness studies examine outcomes when interventions are implemented under conditions approximating “real world” circumstances, including community-based implementation that often utilizes community-based partnerships and teams.13–14 This stands in contrast to efficacy studies, which are typically conducted under highly controlled conditions that are designed to assure strong internal validity.15 There has been, however, only very limited experimental study of long-term substance use outcomes in the context of effectiveness trials. No effectiveness trials of evidence-based, multicomponent family and school interventions selected from a menu could be found. The present study addresses this gap.
This study reports adolescent substance use outcomes from a community-based universal intervention effectiveness trial called PROSPER (PROmoting School-community-university Partnerships to Enhance Resilience). PROSPER utilizes the land grant university outreach arm—the Cooperative Extension System—to mobilize and sustain community-based teams in local communities that implement evidence-based programs, as described in more detail below. We hypothesized that intervention group substance use rates would be significantly lower than those in the control group at 1½ years past baseline—one year following PROSPER project implementation of a family-focused intervention in 6th grade and a minimum of several weeks following PROSPER project implementation of a school-based intervention in 7th grade, as further described in the methods section. This paper examines the partnership-based intervention effects on initiation, past month, and past year substance use.
Positive results from earlier analyses of proximal mediating variables16 supported the expectation of favorable intervention-control differences on substance use rates and, moreover, suggested that respondents to survey questionnaires were not simply motivated to underreport substance-using behaviors. For example, analyses showed significantly better parenting practices by parents in intervention communities, as well as improved, more cohesive family environments. Targeted adolescent competencies also showed significant intervention effects (e.g., higher substance refusal efficacy).
In addition to examining intervention-control differences, this study examined whether students who already had initiated cigarette, alcohol, or marijuana use at pretest (the higher-risk subsample) showed stronger intervention effects than students who had not initiated use (lower-risk). Across a wide range of risk-related variables examined in earlier studies,17–19 very few significant risk moderation effects have been found, but when observed, they have indicated greater benefit to higher-risk subgroups.17–19 Risk moderation effects are defined as those occurring when one risk-related subsample of intervention group participants shows significantly stronger effects than another. Thus, it also was hypothesized that there would be either no evidence of risk moderation or, if observed, risk moderation would favor the higher-risk subsample.
The project recruited 28 school districts from Iowa and Pennsylvania and utilized a cohort sequential design involving two cohorts (designated Cohort 1 and Cohort 2). Initial eligibility criteria for communities in the study were: (a) school district enrollment between 1300 and 5200 students, and (b) at least 15% of students eligible for free or reduced cost school lunches. Communities were blocked on school district size and geographic location; they were randomly assigned to the partnership intervention and “normal programming” comparison conditions. At the beginning of the project, participants in each cohort were sixth graders and their families. Community and participant recruitment procedures are comprehensively described in an earlier published report.20 Figure 1 summarizes sample tracking over the three waves of data collected. The participating universities’ Institutional Review Boards authorized the study procedures before recruitment began.
A three-component community-university partnership model guided the implementation of evidence-based interventions (EBIs), as described in detail previously. 21, 22 The three components of the PROSPER model consist of local community teams, state-level university researchers, and a Prevention Coordinator team in the land grant university Cooperative Extension System. Prevention Coordinators served as liaisons between the community-based teams and university researchers, providing continuous, proactive technical assistance to the community teams. Community teams were comprised of a Cooperative Extension staff team leader, a public school representative co-leader, and representatives of local human service agencies, along with other local community stakeholders (e.g., youth and parents).
Following team formation, local team activities included the selection of a universal family-focused program from a menu of three EBIs. The rationale for providing teams with a menu from which to select EBIs is that such a selection process allows community-based input and decision-making and, thereby, allows them to optimize the “goodness of fit” between the EBIs and local needs or preferences for intervention implementation. Also, this is a more naturalistic process; having a range of intervention options from which to choose is common for communities.
Although they were provided with three program choices, all 14 community teams chose the seven-session Strengthening Families Program: For Parents and Youth 10–14 (SFP 10–14). They then recruited families of 6th graders into the intervention, implementing it during the first of two years of “core” program implementation for each of the two successive cohorts of students (“core” refers to the first year, basic program implementation with a given cohort, preceding elective booster sessions for each program with the same cohort in the following year). During the second year of program implementation in the project, teams were presented with a menu of three school-based EBIs (All Stars, Life Skills Training, and Project Alert) and asked to select one of them to implement with 7th graders. Life Skills Training and Project Alert were selected by four teams; All Stars was selected by six. The core school-based interventions were then administered with the first cohort of students. During this second year of program implementation intervention communities also electively implemented booster sessions for the family-focused intervention (booster sessions for school-based programs were electively conducted the year following core program implementation). The PROSPER programming model and its various functional roles are described more thoroughly in an earlier article.21
The Strengthening Families Program: For Parents and Youth 10–1423 is based upon empirically-supported family risk and protective factor models.24–26 Goals of the seven-session SFP 10–14 include the enhancement of parental skills in nurturing, limit-setting, and communication, as well as youth prosocial and peer resistance skills. Each session includes a separate, concurrent 1-hour parent and youth skills-building curriculum, followed by a 1-hour family curriculum during which parents and youth practice skills learned in their separate sessions together. A detailed description of program content can be found at www.extension.iastate.edu/sfp/.
A total of 1064 families (approximately 2650 family members) attended at least one session of SFP 10–14 in 142 groups in the 14 schools assigned to the intervention condition, representing 17% of all eligible families. Notably, attendance was predicted by level of community team functioning and technical-assistance related variables (e.g., effective collaboration with technical assistants).20 Recruitment strategies and incentives designed to encourage participation included: an SFP 10–14 promotional video and informational displays used at parent-teacher conferences, phone and mail invitations to individual families, classroom presentations, announcements in school newsletters, community distribution of promotional items, and participation incentives (e.g., $10 family gift, door prizes, a $3 youth gift for each session attended).
Of the 1064 families in attendance, 90% attended at least four sessions and 63% attended six or more sessions. Group sizes ranged from three to 15 families (an average group size of 7.5 families and an average of 20 individuals per session). Facilitators were selected by local teams; they were wide-ranging in background and included parents, teachers, counselors, and law enforcement personnel, among others. All facilitators received a 2-day training by certified SFP 10–14 trainers. Each team of program facilitators was observed two or three times to assess their adherence to the intervention protocol. Across both cohorts, coverage of the component tasks or activities described in the group leader's manual showed an average coverage of 92% in the family sessions, 88% in the parent sessions, and 91% in the youth sessions.
Each of the school-based interventions was implemented during class periods. Information on their theoretical approach, duration, and objectives follows; more detail on their implementation can be found in an earlier published report.27
Life Skills Training28,29 is a 15-lesson universal preventive intervention program with a design guided by social learning theory30 and problem behavior theory.31 The primary goals of LST are to promote skill development (e.g., peer resistance, self-management, general social skills) and to provide a knowledge base concerning the avoidance of substance use.32
Project ALERT is an 11-session program based on the social influence model of prevention.30,33,34 It has three objectives: (1) to change students’ beliefs about substance use norms, as well as the social, emotional, and physical consequences of using substances; (2) to help students identify and resist prosubstance use pressures from peers, the media, parents and others; and (3) to build resistance self-efficacy.
Like the Life Skills Training program, All Stars is based on social learning theory30 and problem behavior theory.31 It is a 13-session program with four objectives: (1) to influence students’ perceptions about substance use and violence; (2) to increase the accuracy of students’ beliefs about peer norms regarding substance use and violence; (3) to have students make a personal commitment to avoid substance use and violent behavior; and (4) to increase student school bonding.35
Trained observers monitored selected classroom lessons in the participating schools for all school-based programs. Across both cohorts, the implementation adherence rates for LST, Project Alert, and All Stars were 89%, 89%, and 91%, respectively.27
Data were collected between 2002 and 2005 via machine-scored pencil and paper questionnaires administered in school during class sessions by trained university-based data collectors. Students were assured of the confidentiality of their reports and that the individual information they provided would not be seen by parents or school administrators. Pretest assessments were conducted during the fall semester of 6th grade for each cohort of students. The post-intervention follow-up assessments for the two cohorts were conducted at 1.5 years past baseline and began several weeks after all programming during the second year of implementation had been completed. Community teams in the intervention condition implemented the family-focused intervention during the Spring semester that followed the Fall semester pretest. They implemented the chosen school-based intervention at some point during the following Fall and/or Spring semester. It is noteworthy that there was considerable variation in the timing and delivery structure of the school-based interventions (e.g., different groups of students received the program at different times of year), so that data collection was not immediately following intervention, at least for most students. On average, 88% of all eligible students completed assessments at each data collection point.
Prior studies have reported the use of similar substance use measures and indices to those in the current study.36, 37 Although self-report measures may be susceptible to social desirability biases, previous work has supported the validity of substance use self-reports.38–40 All of the individual substance use items were answered using a yes/no format and coded as “1” for yes and “0” for no.
The Substance Initiation Index-Gateway (SII-G) consists of three lifetime substance use items: (a) ”Have you ever had a drink of alcohol”; (b) ”Have you ever smoked a cigarette”; and (c) ”Have you ever smoked marijuana (grass, pot) or hashish (hash),” scored from 0–3.
The Substance Initiation Index-Illicit (SII-I) consists of five lifetime substance use items that ask “Have you ever...”: (a) “used methamphetamine (meth)” (b) “used ecstasy (MDMA)” (c) ”smoked marijuana (grass, pot) or hashish (hash)” (d) “used drugs or medications that were prescribed by a doctor for someone else” and (e) “used Vicodin, Percocet, or Oxycontin,” scored from 0–5. (Footnote: For the initiation indices and all individual lifetime use measures, students were scored “0” until the time at which they first reported use and “1” thereafter. For example, a student reporting lifetime use at the baseline assessment, but no such use at the follow-up assessment was still considered to have initiated use at baseline).
For all individual initiation measures, “new user” rates were calculated to control for baseline rates, based only on those students who did not report lifetime use of a substance at the baseline assessment. The seven new user measures assessed drinking alcohol, drunkenness, cigarette use, marijuana use, inhalant use, methamphetamine use, and ecstasy use.
There were five dichotomous measures of “current” use. These included two past month items (for substances used at relatively higher frequency rates—alcohol and cigarettes) and three past year items: drunkenness, marijuana, and inhalant use. (Footnote: The survey included past month and past year items for drunkenness and marijuana use. Due to small past month prevalence rates, however, only past year measures were analyzed in the current study. Similarly, the survey included an item assessing the use of methamphetamine in the past year, but the rate was less than 1% so this item was not analyzed).
Multilevel model (two level—school and individual) analyses of covariance using SAS PROC MIXED 9.1.3 (SAS Institute, Inc., Cary, NC, 2003) were applied to the index outcomes; the SAS GLIMMIX macro procedure was used for all dichotomous outcomes. State, cohort, block, and risk status were included as factors in the model at the school-level, as were all corresponding interaction terms. One individual-level covariate, general child management, also was included in the model. General child management addresses basic parenting practices (monitoring, consistent discipline) and was assessed by 13 items on the student questionnaires (α = 0.73), it was added because of its relevance to a wide range of child problem behaviors, including substance use. All analyses here are intent-to-treat analyses and thus include all youth on which data were collected. Although intent-to-treat analyses may not provide accurate estimates of the magnitudes of effects that a particular intervention produces on those who receive the intervention as designed,41,42 this type of analysis is still considered to be the first step in an unbiased evaluation of an intervention41 and was selected for application here for that reason. Final outcome analyses were conducted in 2006.
Pretest equivalence of the two conditions on sociodemographics and outcome measures was assessed. There was no evidence of inequivalence on any sociodemographic measure (biological parents, target gender, age, grades, school absence, race, free school lunch) or on the 14 outcome measures.
Analyses were conducted to rule out differential attrition by examining condition by dropout status interactions with the outcome variables at the pretest assessment. No significant attrition × condition interactions were found.
Young adolescents in the PROSPER intervention condition were compared with those in the control condition using the multilevel ANCOVA analyses, as summarized in Table 1. At the 7th grade follow-up, 1½ years past baseline, results showed that both the SII-G and SII-I scores for the intervention group were significantly lower than those for the control group.
Results showed that the new user rates of marijuana, methamphetamine, ecstasy, and inhalant use were significantly lower for the intervention group. In addition, drunkenness and cigarette use among intervention group adolescents were lower than in the control group, but significant only at the .10 level.
Past year marijuana and inhalant use were significantly lower in the intervention condition. Also, lower proportions of intervention-condition adolescents reported drunkenness in the past year and cigarette use in the past month; but these results only approached statistical significance (p < .10).
Risk status-related subsamples were created based on SII-G scores at pretest; those scoring ≥ “1” (those who had initiated cigarettes, alcohol, or marijuana at that time) were designated as higher-risk (a risk score of “1”), while those scoring “0” (no initiation of cigarettes, alcohol, or marijuana at that time) were designated as lower-risk (a risk score of “0”).
For each outcome, intervention-control differences for use rates in the higher-risk subgroup were greater than those for the lower-risk subgroup, but risk moderation effects did not reach statistical significance for any individual new user rates.
Results for the two substance initiation indices were significant, however, indicating a stronger intervention effect for the higher-risk subsample in both instances. For SII-G, mean scores for the intervention and control groups were 1.430 (s.e. = 0.030) and 1.552 (s.e. = 0.030), respectively, for higher risk adolescents and 0.554 (s.e. = 0.025) and 0.570 (s.e. = 0.025), respectively, for the lower risk adolescents, with a statistically significant risk moderation effect (F1,12 = 5.71, p < .05). For SII-I, mean scores for the intervention and control groups were 0.331 (s.e. = 0.024) and 0.448 (0.025), respectively, for higher risk adolescents and 0.185 (s.e. = 0.020) and 0.220 (s.e. = 0.020), respectively, for the lower risk adolescents, also with a statistically significant risk moderation effect (F1,12 = 5.71, p < .05).
Risk moderation effects were significant for one other outcome, past month cigarette use (F1,12 = 626, p < .05), also with larger intervention-control differences among the higher risk group; means for the intervention and control groups were 10.8% (s.e. = 0.012) and 16.2% (s.e. = 0.016), respectively for higher risk adolescents and 4.0% (s.e. = 0.005) and 4.1% (s.e. = .005), respectively for the lower risk adolescents.
This study examined substance use outcomes when community-based teams implement evidence-based family and school interventions. Analyses of effects showed evidence of intervention-control differences on a range of substance initiation and current use measures. Three features of the interventions producing these results are noteworthy. First, the design of these interventions followed research-based principles and met criteria for evidence of efficacy, suggested by current literature.36 Second, the community teams were required to choose a combination of one family-focused and one school-based intervention from a menu of evidence-based interventions. With this combination of interventions, a range of empirically-supported family-, school-, and peer-related etiological factors for substance use were addressed.9 Third, the interventions were implemented under “real world” conditions, consistent with an effectiveness trial.15 A critical element of this model is the provision of regular and proactive technical assistance for the community team implementation through the land grant university based Cooperative Extension System. The use of this system and active technical assistance are key features of the PROSPER model that is intended for future dissemination to other communities.21,22
The range of substances on which positive effects were observed also is noteworthy. The interventions were designed to impact substance use by intervening with etiological risk and protective factors that were expected to influence substance use generally, independent of the specific type of substance in question. Thus, intervention content was not substance specific. Nonetheless, there were statistically significant effects on a broad range of substances, from frequently used gateway substances to substances used at lower rates in the general population, like methamphetamines, inhalants, and ecstasy.
Concerning the practical significance of effects, the relative reduction rates for marijuana, methamphetamine, and ecstasy initiation are especially noteworthy. For example, the initiation rate of marijuana use was 40% lower in the intervention group than among controls. This suggests that for every 100 non-intervention, general population adolescents who begin using marijuana, only 60 intervention group adolescents would do so over the same period. In addition, effect sizes were primarily in the moderate range when calculated at the community-level, as is most consistent with the intent-to-treat analytic design and most directly relevant to the assessment of community-level public health impact.
The real world conditions under which the present effectiveness trial was conducted also are important in interpreting the practical significance of the results. Showing positive results under such conditions can be expected to be particularly difficult when conducting a rigorous assessment of community-level outcomes using intent-to-treat analysis. Under these conditions, the lower degree of researcher control over such factors as program recruitment could easily lead to weaker community-level effects. Nonetheless, in the current study, with intervention implementation driven by the local teams, intervention participation (17% of the general population families targeted, in the case of the family-focused intervention) and program fidelity were sufficiently high to produce significant reductions in adolescent substance use. In other words, these results are of greater relevance for typical community-based prevention practice than results from an efficacy trial conducted under artificially well-controlled circumstances. A number of barriers to widespread dissemination concern resource demands for sustained quality implementation of evidence-based preventive interventions, including the requisite human and financial capacity for this type of implementation. For example, implementing the SFP 10–14 for a group of 8–10 families costs approximately $3,000. The per group costs may be higher or lower, depending upon the hourly wage rate for staff, the level of participation incentives used, and what elements of the program were donated (for example, food). The PROSPER partnership model is designed to address the costs of SFP 10–14 implementation and other resource issues in two ways. First, sustainability training is central to the PROSPER model. This sustainability training is substantial, starts early on, and focuses on sustaining both effective community team functioning over time and quality program implementation, including training in local fundraising. As of this writing, all community-teams have continued to operate and each has garnered sustainability funding to continue both the family and school programs in their communities.
Second, PROSPER addressed human resource barriers to sustainability through its design as a hybrid of other types of community-based programming models, a hybrid that is grounded in a stable human resource system. As described in an earlier report20, one type of model involves no university-based research staff and another model entails paid research staff that are directly and extensively involved. The PROSPER model consists of community-based implementation by a local community team, consisting mostly of local community volunteers who have primary responsibility for implementation. As described earlier, support for community team implementation efforts is provided by technical assistants with prevention programming expertise. This technical assistance is based in a stable system (the Land Grant University Cooperative Extension System) that provides resources extending beyond research grant funding. Thereby, the PROSPER model addresses both financial and human resource barriers to widespread implementation.
Unlike the authors’ prior prevention trials, there were no effects on alcohol initiation. Earlier studies have suggested that effects on alcohol initiation are observed primarily when the intervention is implemented sufficiently early in the phase during which young adolescents are experimenting with use.43, 44 Unfortunately, in PROSPER the base rates of alcohol initiation in 6th grade were higher than expected and may have suppressed the intervention effect on that outcome. Indeed, most of the significant initiation effects were observed for substances with lower base rates. Notably, on the one alcohol intervention outcome with a relatively lower base rate (drunkenness) relatively more positive results were observed.
Consistent with earlier reports, we found comparable intervention effects among lower-and higher-risk subsamples or, in some cases, stronger effects for higher-risk youth.18,19 This pattern of findings across studies is important in that it counters the common speculation that universal interventions often benefit only those in the general population at lower risk.10 The present findings highlight how universal strategies can provide an avenue for reaching at least some higher-risk youth, while avoiding iatrogenic effects sometimes observed when high-risk youth are grouped together for intervention.45 Most importantly, results indicating comparable or stronger benefits for higher-risk youth show how a community-based approach like the one tested can address the needs of that segment of the community youth population.
The reader should be cognizant of several limitations of the current study. First, although it was conducted in two states and included diverse communities with a range of populations, the largest community was approximately 44,500 in population (with the smallest being approximately 7,000) and the participants were primarily Caucasian. It is not entirely clear to what extent the PROSPER model results would generalize to students in larger metropolitan areas and non-white populations; further research with such populations will be required to assess generalizability of the findings. Second, all of the substance use measures are based on self-report; this could introduce some bias into the reporting, although there is support in the literature for the validity of this type of data.38–40,46
Addressing the health and economic consequences of adolescent substance use noted in the introduction requires diffusion of effective community-based, empirically-supported interventions; community-based partnership approaches have been highly recommended for this purpose.17,22,47,48 The significant differences in levels of substance use across several different measures for the intervention group as compared with a control group—differences that past research17,36 suggests will increase over time—are especially noteworthy in light of this recommendation for community-university partnerships.22,47 Thus, the findings suggest that the PROSPER partnership model for sustained quality implementation of evidence-based interventions by communities has public health potential and warrants further research.
Work on this paper was supported by research grant DA 013709 from the National Institute on Drug Abuse. Request for reprints should be addressed to Richard Spoth, Director, Partnerships in Prevention Science Institute, ISU Research Park, Building 2, Suite 500, 2625 North Loop Drive, Iowa State University, Ames, Iowa 50010, U.S.A. (515-294-5383).
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
No financial conflict of interest was reported by the authors of this paper.