|Home | About | Journals | Submit | Contact Us | Français|
To examine long-term effects of a universal intervention in elementary schools in promoting positive functioning in school, work, and community, and preventing mental health problems, risky sexual behavior, substance misuse, and crime at ages 24 and 27.
Nonrandomized controlled trial followed participants to age 27, 15 years after the intervention ended. Three intervention conditions were compared: a full intervention group, assigned to intervention in grades 1 through 6; a late intervention group, assigned to intervention in grades 5 and 6 only; and a no-treatment control group.
Fifteen public elementary schools serving diverse neighborhoods including high-crime neighborhoods of Seattle.
A gender-balanced and multiethnic sample of 598 participants at ages 24 and 27 (93% of original sample in these conditions).
Teacher training in classroom instruction and management, child social and emotional skill development, and parent workshops.
Self-reports of functioning in school, work and community, mental health, sexual behavior, substance use, and crime, and court records.
A significant multivariate intervention effect across all 16 primary outcome indices was found. Specific effects included significantly better educational and economic attainment, mental health, and sexual health by age 27 (all p < .05). Hypothesized effects on substance use and crime were not found at ages 24 or 27.
A universal intervention for urban elementary school children, focused on classroom management and instruction, children’s social competence, and parenting practices, positively affected educational and economic attainment, mental health, and sexual health 15 years following the intervention’s end.
Poverty, unemployment, and neighborhood disorganization are persistent problems in America’s cities.1, 2 Mental health problems, drug use, crime, high rates of dropout and teen pregnancy plague many urban children and families.3–7 Public schools, available to all children in the United States beginning at age 5 or 6 years, are a potentially powerful setting for preventive intervention. This study examines the effects of a 3-component preventive intervention provided in public schools during the elementary grades on outcomes at ages 24 and 27, 15 years after the intervention ended. The intervention aimed to improve the skills of teachers, parents, and children themselves to increase positive functioning in school and decrease problems related to mental health, risky sexual behavior, substance use, and criminal behavior.
The mid 20s are important years for the adoption of adult roles. Engagement in education and/or occupational roles is an important predictor of future adult functioning.8–10 Civic engagement is also likely to increase during this period.11, 12 Yet, the mid 20s are also years of relatively high vulnerability to mental health problems,13, 14 sexual risk-taking,15, 16 and continued risk for substance use and crime.17–22
Little is known about the long-term effects of universal intervention in public elementary schools on these outcomes. Kellam and colleagues followed students from schools serving predominantly African American children from poor to lower-middle-class families who had been exposed to a classroom-based behavior management program in the first and second grades. By ages 19 to 21, males, particularly those who had demonstrated more aggressive or disruptive behavior in the first grade, reported significantly reduced rates of regular cigarette smoking, fewer drug and alcohol abuse or dependence disorders, and less antisocial personality disorder.23 We are aware of no other studies of universal interventions in the elementary grades that have investigated long-term effects on indicators of adult functioning.
The Seattle Social Development Project (SSDP) intervention was guided theoretically by the social development model.24, 25 We sought to identify and develop methods of management and instruction that could be used by public school teachers and adult caretakers to set children on a positive developmental course by promoting opportunities for children’s active involvement in classroom and family, developing children’s skills for participation, and encouraging reinforcement from parents and teachers for children’s effort and accomplishment. Two intervention conditions were examined: a “full” intervention condition implemented throughout grades 1 through 6; and a “late” intervention condition implemented only when children were in grades 5 and 6.26
Studies of the SSDP intervention have found significant effects in childhood and adolescence across outcomes.27–32 By age 21, the full intervention group, compared with controls, showed significantly better outcomes with respect to education, employment, and mental health, as well as reduced crime, sexual risk behavior and disease, and early pregnancy.33, 34 Some effects were found to be moderated by gender, childhood poverty, or ethnicity. 30, 31, 33, 34 This article examines the effects of the SSDP intervention at ages 24 and 27, 12 to 15 years after the intervention ended. The gender balanced and ethnically diverse sample allows investigation of possible moderators of intervention effects.
Figure 1 shows the overall design of the study. Beginning in 1981, the intervention was initiated among first-grade students in classrooms randomly assigned to condition in 8 public schools serving high-crime areas in Seattle, Washington. Students who remained in or entered the 8 schools during grades 1 through 3 (n = 377) were followed prospectively to fifth grade. The study was then expanded to include 676 fifth-grade students in 10 additional schools, and all parents were asked for consent for their child to participate in the longitudinal follow-up study. Of the population of 1053 fifth-grade students in the 18 schools, 808 (77%) were consented.
Schools were assigned nonrandomly to conditions in the fall of 1985. Thereafter, all fifth-grade students in each school received the intervention according to their school’s intervention assignment. This resulted in a nonrandomized controlled trial with 4 conditions. The full intervention group consisted of those who received at least one semester of intervention in grades 1 through 4 and at least one semester of intervention in grades 5 and 6, with an average dose of 4.13 years of intervention exposure. The late intervention group consisted of those who received the intervention during grades 5 and 6 only, with an average dose of 1.65 years of exposure. The control group received no intervention. A fourth group was offered parent training only during grades 5 and 6 and is not discussed in this article. Twenty-four participants could not be classified into any of these groups because they left participating schools before spending at least one semester there. After excluding these 24, all analyses were based on intervention assignment. All phases of the study were approved by the Human Subjects Review Committee at the University of Washington.
Retention for analysis of the full, late, and control groups averaged 93% at both ages 24 and 27, as shown in Figure 1. There were equal numbers of female and male participants at each age. Ethnic identification was 46% European American, 26% African American, 22% Asian American, and 6% Native American. As children, 56% of participants were eligible for the federal school lunch/breakfast program at some point in the fifth, sixth, or seventh grade, indicating low income status.
At both ages 24 and 27, the overall distribution of participants in the intervention conditions did not significantly differ for those lost to attrition versus the analysis sample (χ22 = 2.16, p = .34 at age 24, and χ22 = 1.41, p = .50 at age 27); nor, among those retained in the analysis sample, did the distribution of participants in the intervention conditions differ with respect to gender (χ22 = .38, p = .83, and χ22 = .59, p = .74, respectively), ethnicity (Caucasian American vs. other; χ22 = .13, p = .94, and χ22 = .02, p = .99, respectively), or childhood poverty (χ22 = .23, p = .89, and χ22 = .33, p = .85, respectively).
Given the requirement that full intervention students attended project schools at some point in grades 1 through 4 and in grades 5 and 6, whereas some control students were added to the study at grade 5, it is important to rule out differences in residential stability, a potential threat to internal validity. Analyses comparing the full intervention and control groups found no significant differences in mean number of years living in Seattle by grade 6 (F = .61, p = .44 at age 24, and F = 1.83, p = .18 at age 27), mean number of residences lived in from age 5 to 14 years (F = 1.57, p = .21, and F = 1.56, p = .21, respectively), proportion of single-parent homes during grade 5 (χ2 = .11, p = .74, and χ2 = .02, p = .89, respectively), or living in a disorganized neighborhood at age 16 years (e.g., rundown housing, crime; χ2 = .47, p = .49, and χ2 = .13, p = .72, respectively). Differential school or teacher receptivity to intervention is also an unlikely threat to internal validity. Teachers in 6 of 8 participating schools during grades 1 through 4 were randomly assigned to either intervention or control classrooms. At fifth grade, newly eligible schools were matched demographically to early experimental schools, and each agreed to serve as a control or late intervention school depending on assignment. Additionally, during the course of the intervention, the Seattle school district used mandatory busing to achieve racial equality in schools, which substantially reduced the risk that outcomes observed reflected contextual or neighborhood differences, school demographic differences, or parent school-selection effects in the populations attending different schools.
An exception to the pattern of condition equivalence was the proportion of surveyed participants who reported at age 24 that their mothers were 19 years of age or less when they were born. Nine percent of the full intervention condition, compared to 21% of the control condition reported that their mothers were teens when they were born (χ2 = 8.56, p < .01). Having a teen mother was included as a covariate in all outcome analyses in this study.
The intervention is described elsewhere.32–35 Each year during grades 1 through 6, teachers in the intervention conditions received 5 days of in-service training in instructional methods.27, 36–40 Additionally, first-grade teachers received instruction in the use of a cognitive and social skills training curriculum41, 42 and, during grade 6, a study consultant provided students with training in refusal skills.43 When children were in grades 1 through 3, parents were offered a 7-session curriculum in child behavior management skills35 and a 4-session curriculum in skills for supporting their children’s academic development.44 During grades 5 and 6, parents of participants in intervention conditions were offered a 5-session curriculum designed to strengthen skills to reduce their children’s risks for problem behaviors.45 Forty-three percent of eligible full intervention parents attended at least one parenting class during grades 1 through 3,35 and 29% of parents in eligible intervention conditions attended at least one class during grades 5 or 6, indicating that the parent training component had less reach than the teacher training and child social skills training components.
Primary outcome indices of success in each life domain were analyzed. The indices provide a limited set of comparisons for multivariate statistical tests of intervention effects. Analyses of primary outcome indices were followed, where warranted, by analyses of specific behaviors, attitudes, and events to provide descriptive detail to the findings and to allow comparison to prior reports.33, 34 Measures were participants’ self-reports of events in the past year at ages 24 and 27 unless otherwise noted, as well as court records from age 10 to age 24. Extreme values for open-ended numeric responses were limited to an appropriate maximum to limit the effects of outliers.
For school and work functioning, a median socioeconomic status (SES) attainment index was assessed by creating a dichotomous measure of those scoring at or above the median in completed education (attaining at least a high school diploma by ages 24 and 27) or household income (at least $44,000 at age 24, and at least $45,000 at age 27).40 Specific SES measures included cumulative education completed46 and income (divided by 1000 for analysis). Other measures included the degree to which students were integrated at school (time in class, participation, etc.; coded 0–4, mean reliability coefficient α at ages 24 and 27 = .37)33 and the degree of responsibility on the job for those who were employed (2 items coded 0–4, mean r = .36).33 (For purposes of comparison, the age 21 measures were replicated as closely as possible. As a result, some reliability coefficients for measures corresponding to those examined at age 21 are low.) Constructive engagement summed the average number of hours per week engaged in school and/or work (divided by 10 for analysis),33, 47 and constructive self-efficacy concerned perceived opportunities for getting a good education and a good job (coded 0–4, α = .78).33 Two additional descriptive measures, for which we did not expect intervention effects included student status in the past year and employment status in the last month, ranging from non-student or unemployed (coded 0) to full time (coded 4).33
To assess community involvement, a civic engagement index summed the average hours per month of involvement in community groups and in volunteer activities.47
Mental health problems were assessed with the Diagnostic Interview Schedule (DIS)46, 48–52 to measure DSM-IV criteria.53 A disorder criterion index (coded 0–37, α = .95) summed across the total number of DSM-IV criteria met in the past year for a generalized anxiety disorder (GAD) criterion count (coded 0–6, α = .88), a social phobia criterion count (coded 0–5, α = .92), a posttraumatic stress disorder (PTSD) criterion count (coded 0–17, α = .98), and a major depressive episode (MDE) criterion count (coded 0–9, α = .98). Specific criterion count measures for each of these disorders were also analyzed, including a separate measure of suicide thoughts (coded 0–3, α=.67).33 Additionally, a dichotomous disorder diagnosis index was computed to indicate those meeting criteria for DSM-IV diagnosis for any of the measured disorders (GAD, social phobia, PTSD, or MDE diagnosis).33
The lifetime sexually transmitted disease (STD) index was a dichotomous measure based on reports of having ever been diagnosed with an STD.34 Specific measures of sexual risk behavior included the number of lifetime sex partners and, among those not in an exclusive relationship, the number of past-year sex partners and condom use in the past year (coded as percent of time).34 We also report descriptive measures of having ever been (or gotten someone) pregnant and ever had (or fathered) a baby, for which we did not expect intervention effects. Lifetime measures were constructed accounting for affirmative responses in prior year surveys.
A substance abuse and dependence criterion index was computed as the sum of DSM-IV criteria met for alcohol and illicit drug abuse and dependence disorders (coded 0–22, α = .86), and a substance abuse and dependence diagnosis index indicated those meeting criteria for an abuse or dependence diagnosis.46 Specific behaviors related to problem substance use included high variety of substance use (identifying those above the 90th percentile in number of different substances used in the past year), and the extent of substance use interference with life (coded 0 = no use, to 4 = “very much”).33 Any past-year substance use (tobacco, alcohol, or illicit drugs) is reported for descriptive purposes.33
A past-year crime index was computed from self-reports of criminal acts (other than driving violations or illicit drug use).33 Specific crime measures included high variety of crime, identifying those above the 90th percentile in number of different types of crimes committed, as well as measures of having sold drugs and having been arrested. Also, official state and federal crime files were matched to SSDP participants (including survey non-respondents) to assess the presence of a past-year court charge or a lifetime court charge for any non-criminal, misdemeanor, or felony charge through age 24.33 b
Given the study’s design, the unit of intervention assignment consisted of the series of classrooms to which some individuals were assigned in grades 1 through 4 and the condition assignments of schools attended by all participants in grades 5 and 6. Of the 643 participants assigned to the control, late, and full conditions, over 169 different classroom/school sequences were identified, consistent with the unit of intervention assignment. On average, only 3.80 participants experienced the same units of intervention within conditions. For this reason, and to be consistent with prior reports, analyses were conducted at the individual level.
A multivariate analysis of covariance (MANCOVA) was conducted to assess overall intervention effects across multiple dependent variables, controlling for teen mother at birth. All 16 (8 each at ages 24 and 27) primary outcome indices were included in the MANCOVA. Results showed a significant overall difference between the full intervention and control groups with listwise deletion (Wilk’s lambda F [16, 276] = 1.98, p = .014). This MANCOVA was replicated across 5 datasets for which imputation procedures were used to account for missing data on some outcomes.54 Each analysis confirmed an overall group difference (Wilk’s lambda F-values [16, 319] ranged from 1.84 to 1.94, with p = .025 to .017). These results indicate a significant multivariate effect, and provide overall control for Type I error rate in the significant univariate findings presented below.55 MANCOVAs comparing the late intervention group with controls across the 16 outcomes were not significant (Wilk’s lambda F-values [16, 423] ranged from .69 to .75, p = .808 to .737).
As shown in Table 1, those in the full intervention group were significantly more likely than controls to be at or above the median in socioeconomic status (education or household income) by age 27 (93% vs. 84% in the control group). The differences in SES attainment were similar at age 24, though not significant. Specific comparisons in this domain found that those in the full intervention groups were marginally more likely to have continued their education beyond high school, with 34% having completed an associate degree by age 27, compared with 22% in the control group (p < .06). Nonsignificant trends in completion of a bachelor’s degree, and household and earned income also favored the full intervention group.
The civic engagement index indicated marginally more community involvement and volunteerism in the full intervention group at ages 24 and 27, compared to the control group. Both intervention groups reported significantly more participation in community groups at age 24, compared with controls.
It is noteworthy that across outcomes related to school, work, and community at ages 24 and 27 for which we had directional hypotheses, 27 of 28 outcomes analyzed showed directional patterns between the full intervention and control conditions consistent with prediction, and both primary outcome indices showed positive full intervention effects significant beyond p < .08 by age 27. Twenty-two of these 28 outcomes demonstrated a pattern consistent with a “dose” effect, with the late treatment group falling between the full treatment and control groups.
Analyses found significantly fewer symptoms of mental health disorders on the disorder criterion index in the full treatment condition compared with controls at both ages 24 and 27. Analyses also found significantly lower prevalence of meeting criteria for at least 1 of 4 DSM-IV diagnoses on the disorder diagnosis index in the full treatment group compared with controls at age 27, and a nonsignificant trend toward lower prevalence at age 24. Measures of specific disorder criterion counts indicated at least marginally significant (p < .10) reductions in the number of GAD, social phobia, and PTSD criteria met at age 24; and of GAD, PTSD, and MDE criteria met, as well as significantly fewer suicide thoughts, at age 27. Across all 22 mental health outcomes examined, reported problems were lower in magnitude among full intervention participants than among controls. Late intervention participants reported levels between or equal to those of the other groups for all but 3 of the 22 mental health outcomes, consistent with a dose effect.
The lifetime STD index showed significantly lower prevalence of having ever been diagnosed with a STD at ages 24 and 27 for the full intervention group compared with controls. No intervention effects on specific sexual behaviors were found.
No intervention effects on either of the substance abuse and dependence indices were found in young adulthood, nor were significant effects observed for the specific measures of high variety of substance use or substance use interference with life.
A marginally significant trend (p< .09), counter to hypotheses, for the past-year crime index at age 27 indicated a higher prevalence of having committed a crime among full intervention participants, compared to controls. This difference stemmed primarily from nonsignificant but higher rates in the full intervention group of minor theft (taken anything worth less than $50; 9% v. 4% in the control group) and drug selling (8% v. 3%, respectively) at age 27. This was the only finding counter to hypotheses that approached significance across all of the outcomes examined in Table 1. No significant intervention effects were found for the any of specific measures related to crime.
Adding interaction terms to the regression models revealed 2 significant effects of the full intervention specific to males. Full treatment group males were significantly more likely to meet median education or income criteria at age 24, as assessed with the SES attainment index, compared with their control counterparts (97%, SD = .17, vs. 85%, SD = .36, respectively, p = .030). Also, males in the full treatment group were significantly more constructively engaged at age 24 than control males; 29.9 hours per week on average (SD = 13.9) of engagement in school and/or work, versus 24.4 hours (SD = 16.4), respectively (p = .023). Neither of these outcomes showed significant intervention differences for females.
Two significant intervention effects were also found specific to those from a background of childhood poverty. Full treatment participants from poverty reported an average of nearly 9.5 hours per month of involvement in community groups at age 27, compared to 4.9 hours for controls (SDs = 13.2 and 9.2, respectively; p = .004). Those from childhood poverty in the full intervention group were also significantly less likely to meet sufficient criteria for a GAD diagnosis at age 24, compared with controls (3%, SD = .16, vs. 14%, SD = .35, respectively, p = .017).
After controlling for childhood poverty, ethnicity also significantly interacted with the intervention for some outcomes. Responsibility on the job at age 24 was significantly higher among Caucasians in the full treatment group compared with control Caucasians (M = 2.78, SD = 1.10, and M = 2.34, SD = 1.06, respectively; p = .015). African Americans in the full intervention condition compared with their control counterparts reported significantly higher household income at age 27 (M = $55,594, SD = 45,966, and M = $35,288, SD = 35,410, respectively, p = .046), fewer GAD criteria met at age 27 (M = .94, SD = 1.39, and M = 2.51, SD = 1.99, respectively; p < .001), fewer past-year sex partners at age 27 (M = 1.39, SD = 1.65, and M = 2.89, SD = 2.78, respectively; p = .026), and reduced prevalence of lifetime STD diagnosis at age 24 (12%, SD = .33 in the full intervention condition, and 55%, SD = .50 in the control condition; p = .001) and at age 27 (16%, SD = .37 in the full intervention condition, and 61%, SD = .49 in the control condition; p = .001). These results suggest that the significant main effects of the full intervention on STD diagnosis at ages 24 and 27 were due primarily to the reductions in STD diagnosis among African Americans in the full intervention group.
The SSDP intervention in the elementary grades showed a significant overall effect of the full intervention in a multivariate analysis of 16 outcome indices at ages 24 and 27, 12 and 15 years after the intervention ended. Summary indices revealed significantly better socioeconomic attainment, mental health and sexual health by age 27 among those assigned to the full SSDP intervention condition compared with controls. In contrast, effects of the intervention on substance use and crime were not found at ages 24 and 27.
In line with previous analyses32, 33 the rank of means and prevalence rates for the late intervention group, with respect to the full intervention and control groups, suggests a “dose” effect for the domains most influenced by the intervention. That is, while differences were not significant, the late intervention group reported better outcomes than the control group across the preponderance of measures for which there were directional hypotheses related to school, work, community, or mental health, but worse outcomes than the full intervention group across these measures.
The results reported here indicate that a theory-based intervention that improved classroom management and instruction, children’s social competence, and parenting practices during the elementary grades influenced some, though not all, indices of adult functioning in the mid to late 20s. The elementary grade intervention was related to greater accomplishment and engagement in school, work, and community, and fewer mental health problems by ages 24 and 27. Effects of the SSDP intervention on mental health outcomes are particularly noteworthy at this stage of young adulthood given the potentially debilitating consequences of depression and anxiety.56–59
Findings should be considered in the context of the design and analysis approach, which was based on condition assignment (rather than attendance or implementation), using controlled, 2-tailed statistical tests, and an omnibus test of multiple outcomes. Strengths of this study include the ability to detect relatively small effect sizes, comprehensive assessment capturing multiple outcome domains, delivery of the intervention package universally to all students in intervention classrooms, and the ability to investigate possible moderating effects of gender, ethnicity, and childhood poverty. As in previous reports,33 there was little evidence of serious threats to internal validity. Additionally, procedures for assigning classrooms (in the early experiment) and schools to conditions likely guarded against possible effects of differential school or teacher receptivity to intervention, and mandatory busing provided further protection against confounding neighborhood or school demographic differences or parent school-selection effects. An exception to the general pattern of group equivalence at the start of the longitudinal study was a significantly higher proportion of those in the control group who reported having had a teen mother at birth. This difference was controlled statistically throughout all analyses.
Limitations should be noted. The study was quasi-experimental and geographically limited. It relied heavily on self-reported data from study participants. Effects of the intervention on school, work, and mental health were observed, but no significant effects on substance use or crime were observed at these ages.
Intervention effects reported here indicate that universal intervention during the elementary grades to improve the management and instructional skills of urban public elementary school teachers, strengthen parenting practices in multiethnic urban families, and ensure that children have the skills to participate in the social and academic life of elementary school can positively affect attainment, functioning, and mental health in young adulthood.
This research was supported by grants #1R01DA09679-11 and #9R01DA021426-08 from the National Institute on Drug Abuse, #R24MH56587-06 from the National Institute of Mental Health, and #21548 from the Robert Wood Johnson Foundation. This article is one of a number of papers from the Seattle Social Development Project. Some descriptions of the project used in this article are similar to those used in studies published previously by the authors. All authors have contributed to the conception and design of the study, analysis and/or interpretation of data, and to drafting and/or revisions of the manuscript. Funding and data were acquired by Hawkins, Kosterman, Catalano, and Hill. Administrative and supervisory support were provided by Hawkins, Catalano, and Hill. Hawkins and Kosterman had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. Catalano is a board member of Channing Bete Company, distributor of Supporting School Success ® and Guiding Good Choices ®, programs that were tested in this study. All phases of the study were approved by the Human Subjects Review Committee at the University of Washington. Participants were informed about the nature of the interviews and provided consent prior to participation in the study at ages 24 and 27.