PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
Prev Sci. Author manuscript; available in PMC 2014 February 27.
Published in final edited form as:
PMCID: PMC3808514
NIHMSID: NIHMS460060

The Impact of the Good Behavior Game, a Universal Classroom-Based Preventive Intervention in First and Second Grades, on High-Risk Sexual Behaviors and Drug Abuse and Dependence Disorders into Young Adulthood

Abstract

The Good Behavior Game (GBG), a method of teacher classroom behavior management, was tested in first-and second-grade classrooms in 19 Baltimore City Public Schools beginning in the 1985–1986 school year. The intervention was directed at the classroom as a whole to socialize children to the student role and reduce aggressive, disruptive behaviors, confirmed antecedents of a profile of externalizing problem outcomes. This article reports on the GBG impact on the courses and interrelationships among aggressive, disruptive behavior through middle school, risky sexual behaviors, and drug abuse and dependence disorders through ages 19–21. In five poor to lower-middle class, mainly African American urban areas, classrooms within matched schools were assigned randomly to either the GBG intervention or the control condition. Balanced assignment of children to classrooms was made, and teachers were randomly assigned to intervention or control. Analyses involved multilevel growth mixture modeling. By young adulthood, significant GBG impact was found in terms of reduced high-risk sexual behaviors and drug abuse and dependence disorders among males who in first grade and through middle school were more aggressive, disruptive. A replication with the next cohort of first-grade children with the same teachers occurred during the following school year, but with minimal teacher mentoring and monitoring. Findings were not significant but generally in the predicted direction. A universal classroom-based prevention intervention in first- and second-grade classrooms can reduce drug abuse and dependence disorders and risky sexual behaviors.

Keywords: Good Behavior Game, Developmental epidemiology, Universal prevention programs, Classroom behavior management, Condom use, Age of sex onset, Number of sexual partners, High-risk sexual behaviors, Mixed effects modeling

Introduction

This article reports on the impact of the Good Behavior Game (GBG), a universal classroom behavior management program carried out in first and second grades, on young adults aged 19 to 21 (Barrish et al. 1969; Kellam et al. 2008). We examine the impact of the GBG on the course of early aggressive, disruptive behaviors from the start of first grade through middle school and on later drug abuse and dependence disorders, use of condoms, number of sexual partners, and initiation of sexual intercourse along the transition into young adulthood. Administered by the first- and second-grade teachers classroom-wide, the GBG is directed at socializing children to the role of being a student and reducing aggressive, disruptive behavior, which is a well-documented antecedent risk factor for later drug, alcohol, and antisocial behavioral disorders and other problem outcomes including high-risk sexual behaviors (see Section S1.3 available online).

The article has three goals: (a) we report the overall effectiveness and the predicted variation in effectiveness of the GBG on the use of condoms, number of sexual partners, age of initiation of vaginal sex, and on drug abuse and dependence disorders. We reported on the impact of the GBG on drug abuse and dependence disorders alone in an earlier paper (Kellam et al. 2008), and we include it here to assess the frequency of co-occurrence with high-risk sexual behaviors and to explore the GBG impact when the outcomes occur separately as well as when they co-occur; (b) we report evidence on the etiological role of early aggressive, disruptive behavior as demonstrated by directing the GBG precisely at this early antecedent and observing the impact on the theoretically predicted outcomes; and (c) we report the public health utility of the prevention strategy of aiming an early universal classroom-based intervention at a single shared antecedent of a set of later problem outcomes to reduce the risk of the entire set of outcomes. If effective, this developmental epidemiological early prevention strategy can augment other prevention strategies directed at high-risk sexual behaviors, drug abuse and dependence disorders, and other problem outcomes that share the same early risk factor.

Epidemiology of High-Risk Sexual Behaviors and Drug Use in Baltimore

Adolescents and young adults are often at increased risk for unintended consequences of sexual activity. A higher proportion of adolescents have engaged in sexual behaviors in Baltimore City as compared to the nation as a whole. In 2007, 67.1 % of Baltimore City high school students reported having sexual intercourse in their lifetime and 49.7 % were currently sexually active (Eaton et al. 2008). Over a quarter (25.9 %) did not use a condom the last time they had sexual intercourse, 29.6 % reported four or more sexual partners, and 18.6 % had sexual intercourse before age 13, more than 2.5 times the national average (Eaton et al. 2008). Similarly high rates of high-risk sexual behaviors were found in an initial analysis of the Baltimore population reported here (Ompad et al. 2006). These high rates of sexual behaviors among Baltimore adolescents are reflected in health outcomes that are generally higher than the national average (Centers for Disease Control and Prevention 2010; see Section S1.1 available online for national and Baltimore rates of sexual behavior, STIs and teen pregnancy). In the context of these high rates of risky sexual behaviors in Baltimore, we investigate whether an early classroom intervention directed at a precise antecedent risk factor can alter this developmental course.

This paper is an extension of prior analyses contained in a supplemental issue of Drug and Alcohol Dependence that was devoted to reporting the impact of the GBG intervention, implemented in first and second grade Baltimore classrooms, on a broad profile of externalizing problem outcomes by young adulthood (Drug and Alcohol Dependence 2008). Significant and meaningful impact was found by young adulthood in reducing drug and alcohol abuse and dependence disorders and regular smoking (Kellam et al. 2008), as well as antisocial personality disorder, incarceration for violent behavior (Petras et al. 2008), suicide ideation and attempts (Wilcox et al. 2008), and use of services for problems with emotions, behavior, or drugs or alcohol (Poduska et al. 2008). Except for suicidal ideation and attempts, where both genders were impacted, these results were found particularly among those males who were more aggressive, disruptive early on in the classroom (Kellam et al. 2008; Petras et al. 2008; Poduska et al. 2008; Wilcox et al. 2008). This paper extends the reporting of the GBG impact, making use of the same data base and theoretical frame to expand the profile of outcomes that stem from early aggressive, disruptive behavior to include later drug abuse and dependence disorders and high-risk sexual behaviors (Drug and Alcohol Dependence 2008).1

Developmental Epidemiology and Life Course/Social Field Theory

The Baltimore prevention intervention research and the Chicago/Woodlawn studies that preceded it are grounded in an integration of three scientific paradigms. The first paradigm is community epidemiology, which is concerned with understanding the sources of nonrandom distribution of health problems, behaviors, or related factors in a defined community. Community epidemiology provides control of selection bias and, when integrated with the second paradigm, life course development, allows the study of variation in developmental antecedents and paths in a defined community. The third paradigm is the use of a randomized experiment to test preventive interventions directed at early antecedents of later problem outcomes. Such an experiment can reveal the causal role, malleability, and mediating/moderating effects of risk factors on the course and prevention of disorders. Directing the intervention precisely at the reputed risk factor can reveal whether those with the risk factor benefit more than others, and this leads to a priori prediction regarding who should benefit.

Our research is further grounded in life course/social field theory (Kellam et al. 1975), which is focused on two dimensions of health, one societal and the other internal. The societal one concerns the levels of success and failure as defined by how successful an individual is viewed by important others, named natural raters, in specific social fields relevant to each stage of life; e.g., teachers in the classroom, peers in the peer group, parents in the home, and supervisors in the work place (Parsons 1951). The second dimension is the psychological and physical well-being of the individual. Life course/social field theory states that each stage of life occurs in meaningful social fields, each of which entails specific social task demands. In the early grades behavior that is demanded includes obeying classroom rules, including not being aggressive or disruptive, the target of the GBG. By young adulthood, the social fields relevant to this paper are the community, with its prohibitions about drug abuse, and the intimate social field, with its mandates for safe sexual practices. The theory predicts that children who display early aggressive, disruptive behavior but improve via the GBG have learned to better adapt to teachers and peers (natural raters) and will be more successful in meeting later social task demands in adolescence and adulthood. For further information on the Baltimore prevention research and life course/social field theory, see Section S1.2 available online.

Aggressive, Disruptive Behavior

Aggressive, disruptive behavior has been repeatedly shown, as early as the first grade, to be an important maladaptive classroom behavioral antecedent of many adolescent and adult externalizing behaviors including those cited above (Kellam et al. 2008). More recently, there has been increasing interest in the relationship between child and adolescent aggressive behaviors and sexual behaviors. Externalizing behaviors have been associated with increased odds of lifetime vaginal/anal sex (Brown et al. 2010). Developmental trajectories of child conduct problems (Wu et al. 2010) and youth delinquency (Miller et al. 2010) have also been associated with risky sexual activity. These results suggest that high-risk sexual behaviors are among the constellation of young adult problem behaviors predicted by early aggressive, disruptive behaviors.

The Good Behavior Game (GBG)

The GBG was developed by Barrish et al. (1969). Before the Baltimore trials reported here no randomized field trials had been conducted using the GBG as an intervention, but there had been numerous scientific papers and dissertations describing positive GBG results in fairly short-term, small studies (Mackenzie et al. 2008). The promise of these observational studies is what led to the trials in Baltimore.

In the trial reported here, first-grade classrooms received the GBG intervention over the course of 2 years. Teachers implementing the GBG initially received training and then assigned children to one of three heterogeneous teams. Within each team there were equal numbers of boys and girls, aggressive, disruptive children, and shy, socially isolated children based on baseline measurements of classroom behavior. The teacher posted basic classroom rules of student behavior, and during a particular game period all teams received a reward if they accumulated four or fewer infractions of acceptable student behavior. The GBG was played during periods of the day when the classroom environment was less structured, such as when the teacher was working with one student or a small group while the rest of the class was instructed to work on assigned tasks independently.

During the first weeks of the intervention, the GBG was played three times each week for a period of 10 minutes. The duration of the game increased so that by the end of the school year it was played for a developmentally appropriate time in each grade (i.e., about 25 minutes for first graders and about 40 minutes for second graders). Initially, the teacher announced game periods, and the rewards were delivered immediately after the game. Later, the teacher initiated the game periods without announcement, and the rewards were delayed until the end of the school day or the end of the week. Over time, the game was played at different times of the day and during different activities. In this manner, the GBG evolved from a precise procedure that was highly predictable and visible, with a number of immediate rewards, to a procedure with an unpredictable occurrence and location, with deferred rewards.

We hypothesize the GBG, directed at the interactive process of the classroom teacher's social task demands and the children's behavioral responses, will be effective for children who are failing to adapt to the classroom by displaying aggressive, disruptive behavior, the specific target of the intervention. For more elaboration see Section S1.4 available online.

Methods

Epidemiologically-Based, Randomized Field Trial Design

The multilevel randomized design encompassed a total of 19 schools, 41 classrooms, and 1196 first grade children within five urban areas. The first stage of the design involved selecting five distinctly different socio-demographic urban areas in Baltimore. Three or four schools were matched in each of the five urban areas by socioeconomic status, size of school, and ethnicity. These five urban areas, which were selected with the help of city planners and our partners in the Baltimore City Public School System (BCPSS), varied in socioeconomic status from very poor to lower-moderate income, as well as in ethnicity, including mostly African American, mixed ethnicities, and mostly white. We then randomly assigned the three to four matched schools within each urban area to serve as schools where the GBG would be tested (six), schools where a curriculum intervention would be tested independently of the GBG (seven), or schools where no intervention would be tested (six; external control schools). All schools that implemented the GBG had either two or three first-grade classrooms.

The second stage of the design involved assigning individual children to first-grade classrooms within each school so that classrooms were nearly identical before they were assigned to the intervention condition. Starting in the summer before the school year began and early into the school year, school administrators assigned all students sequentially using an alphabetized list to the different first-grade classrooms within their school. Classes within each school were checked for balance on kindergarten experience and academic and behavioral performance. Children who moved into any of these school catchment areas during the year were assigned sequentially across classrooms, with the provision that the class sizes remained comparable. These procedures produced balanced and equal-size classrooms within the schools.

The third stage of this design was random assignment of classrooms and teachers to intervention condition within each intervention school. Early in the fall of 1985, after the school year started and before the interventions began, we randomly assigned all regular (non-special education) first-grade classrooms along with their teachers to an intervention condition. Six schools across the five urban areas were assigned to the GBG intervention, and then within each school GBG classrooms were randomly selected. Our analyses here focus on the 407 students in the six GBG schools who were either randomly assigned to the eight GBG classrooms (n=238) or the six internal control classrooms (n=169). No differences between the GBG and control sample were found (Table 1; also see Section 2.4 and Tables 1 and 2 in Kellam et al. 2008 for further details on comparability between the GBG and control samples).

Table 1
Comparability of GBG and internal GBG control (N=407)

The trial involved 2 years of exposure to the GBG intervention. Children in a GBG classroom in the first grade (1985–1986) also received the GBG during second grade, during which children's first-grade classroom assignments remained the same as for the previous year although the second-grade teachers were different. Teachers of the GBG classrooms received 40 hours of training, most of which occurred at the beginning of the program, followed by supportive mentoring during the course of the first-grade school year. A comparable amount of attention was spent with control classroom teachers but without a focus on classroom behavior management to balance the amount of attention given to all teachers.

In the second school year (1986–1987) while the first cohort of children was in second grade, another cohort of first graders (n=373, with 214 in GBG classrooms and 159 in control classrooms within the six GBG schools) was assigned in the same balanced fashion to intervention condition and classroom for the first two years of elementary school, creating a second cohort. For this second cohort, the first-grade teacher remained in the same intervention condition as with the first cohort, yet the GBG first-grade teachers received little retraining, support, or further mentoring and monitoring because we assumed they would continue the intervention with fidelity. More emphasis was placed on training the second-grade teachers new to the GBG who were now teaching the first cohort children now in second-grade. For more details on the trial design see Section S2.1 available online.

Measure of Aggressive, Disruptive Behavior through Middle School

The Teacher Observation of Classroom Adaptation-Revised (TOCA-R) is a measure of each child's adequacy of performance on the core tasks in the classroom as rated by the teacher. It was developed and used in the Woodlawn studies (Kellam et al. 1975), and after modification was used as a core periodic assessment instrument for the Baltimore trials (Werthamer-Larsson et al. 1991). TOCA-R contains a multi-item scale of each social task demand; each of the ten items measuring each construct is rated on six levels. It involves a 2-h long structured interview in a private location in the school, and is administered by a trained member of the staff who initiates the session with the teacher by listening to the teacher's assessment of how the school year is going. After listening and working through trust, the interviewer asks whether the teacher is ready to rate the children. The teacher is then guided in his or her ratings of each child on each item representing the social task demand constructs. The interviewer follows a script precisely, responds in a standardized way to issues the teacher initiates, and records the teacher's ratings of the adequacy of performance of each child in the classroom. The ratings used here are those from the fall and spring of first and second grades, during the two years of intervention, and then from the spring of each year through seventh grade for a total of nine time points.

The construct of central interest here is Authority Acceptance, the maladaptive form of which is aggressive, disruptive behavior. The ten TOCA-R items comprising the aggressive, disruptive construct of TOCA-R are breaks rules, breaks things, fights, harms others, harms property, lies, stubborn, teases classmates, takes others' property, and yells at others. Psychometric work includes item-whole correlations among the ten items for each time of administration (i.e., fall and spring in first and second grades and then spring each year after through seventh grade). The range in alphas was from 0.91 to 0.95, and a correlation of 0.67 was found between the Authority Acceptance subscale and peer nominations of “gets into trouble.” In terms of predictive validity, the strength of prediction for young adult outcomes is presented in previous papers (particularly Drug and Alcohol Dependence 2008).

Young Adult Outcome Measures at Ages 19–21

A 90-min long telephone interview was carried out at ages 19–21 with the students who participated in the trial with a 75.9 % follow-up rate overall (Table 1). Young adults were asked if they would agree to be interviewed and advised that they were not obligated to answer questions if they did not wish to do so, and could terminate the interview at any time. They were offered a $50 participation incentive and were given a T-shirt inscribed with the logo of the prevention program. These procedures were approved by both the Johns Hopkins University and the American Institutes for Research institutional review boards. The interviewers were masked to the first-grade intervention condition of the respondents. The questions were organized by social fields of family of origin, school, work, intimate relationships, sexual relationships, family, and peers, followed by developmental history, current status, and psychiatric diagnoses.

Drug Abuse and Dependence Disorders

CIDI-UM (Kessler et al. 1994), modified to reflect the Diagnostic and Statistical Manual of Mental Disorders-IV (DSM-IV) diagnostic criteria (American Psychiatric Association 1994), was used to determine the lifetime, past-year, and past-month occurrence of drug abuse and dependence disorders. Diagnoses were derived in accordance with DSM-IV criteria, using a computerized scoring algorithm.

High-Risk Sexual Behaviors

Outcome data came from the same young adult interview using questions developed for the ALIVE Study (Vlahov et al. 1991). Respondents were asked a series of questions about vaginal sex (defined as “when a boy or man inserts his penis into a girl or woman's vagina”) including lifetime condom use, lifetime number of sexual partners, and age of initiation of vaginal sex. Only those who had engaged in vaginal sex were included. Next, participants were asked about sexual activity in the last 30 days. They were first asked the total number of sexual partners in the last 30 days and then asked to give the initials of up to eight sexual partners in the last 30 days. For each of these eight partners, they were asked if they had vaginal sex, the number of times they had vaginal sex and the number of times they used a condom during vaginal sex.

We report on condom use, the number of sexual partners and age of sexual initiation for vaginal sex. Condom use was described as responding “Always,” “Most of the time,” or “Often” when asked about lifetime use of condoms during vaginal sex. Not using a condom was described as responding “Never,” “Rarely,” or “Sometimes.” For analyzing condom use in the past 30 days, condom use was defined as using a condom for more than 50 % of the total vaginal sex acts and not using a condom was defined as 50 % condom use or less.

The Composite High-Risk Sexual Behavioral Score

We also created a combined measure of risk, a composite high-risk sexual behavior index, in order to examine the GBG impact on the set of high-risk sexual behaviors in combination and also separately from drug abuse and dependence disorders. This composite score was defined as meeting one of the following criteria: a) age of vaginal sex initiation at 13 or younger, b) having ten or more lifetime vaginal sexual partners, and/or c) not using condoms (i.e., responded “never,” “rarely” or “sometimes” used condoms in lifetime).

Analytic Strategy

Our examinations of the effects of the GBG on young adult outcomes utilize intent-to-treat analyses based on assignment to the GBG or control classrooms. We begin with simple cross-tabulations of young adult outcomes by intervention status. Based on previous findings that identified differential trajectories of aggressive behavior (Muthén et al. 2002; Petras et al. 2008), our primary analytic method in this paper used general growth mixture modeling (GGMM) (Brown et al. 2008; Muthén and Asparouhov 2006; Muthén et al. 2002). GGMM was used to test the impact of the GBG, delivered over the course of first and second grades, on trajectories of aggressive, disruptive behavior through middle school and then on drug abuse and dependence disorders and high-risk sexual behaviors in young adulthood. Mplus software Version 6 statistical package (Muthén and Muthén 1998–2010) was used to compare the intervention and control groups on their patterns of growth in aggressive, disruptive behavior over time and impact on the distal outcomes.

We used GGMM to test whether the GBG intervention had a long-term impact on the distal outcomes, and whether the intervention impact varied as predicted according to the individual's initial and developmental course of aggressive, disruptive behavior as represented by the estimated growth factors. To determine the number of classes we relied on the Bayesian Information Criterion (BIC, see Section S2.4 available online for BIC values) as well as likelihood ratio tests (Muthen 2006; Nylund et al. 2007). When it improved the fit, we allowed for the variances for the residuals of the growth factors indicators to differ by risk class; we also included covariates as predictors of random growth factors and risk class.

Formal tests of intervention effects within trajectory classes were based on Wald-type tests comparing model coefficients of distal outcomes by intervention status for each latent trajectory class or likelihood ratio tests when the probability estimates were near 0 or 1 (Brown et al. 2008) By adopting this GGMM approach, we were able to estimate different growth parameters across unobserved subpo-pulations or latent developmental trajectory classes and further test long term treatment impact within subpopula-tions. We fit a quadratic function to the data (Muthén et al. 2002), and our final model includes specification of a directional path from the intercept factor to the slope factor to further control for the initial status within each latent class. To test the GBG impact separately for sexual behaviors and drug abuse or dependence disorders, we extended the final model by fitting a multi-group GGMM using the known-class option in Mplus to depict the GBG and control groups. So, for a two-group (GBG verses controls) three-class GGMM, the impact on the distal outcomes in each latent class relied on posterior probability-based multiple imputations (Wang et al. 2005). For information on handling of missing data, clustering and effects see Section S2.4 available online.

Formal analyses of the GBG intervention against control classrooms were conducted as a series of parallel analyses, one for each young adult outcome and then in combinations. The assessments of the combinations were done to determine how much overlap of risk behaviors occurred in the young adult population and whether young adults who showed both drug abuse and dependence disorders and one or more high-risk sexual behaviors were impacted by the GBG differently than the young adults with single risky outcomes.

We present the results in detail for males and females in the first cohort, the original trial in which teachers of the GBG classrooms received the intended, appropriate training, mentoring, and monitoring. The identical analytical strategy was used for the second cohort analyses.

Results

We first report on the development of aggressive, disruptive behavior ratings through middle school and the impact of GBG on this trajectory. These ratings were taken in the fall and spring of first and second grades and then in the spring of each school year through seventh grade, resulting in nine time points. This analysis is followed by reporting of the GBG impact on drug abuse and dependence disorders and sexual behaviors in young adulthood overall and within each of the aggressive, disruptive classes. See Section S3 available online for values of non-significant results summarized below.

Impact of the GBG for Cohort 1 Males

Impact of the GBG on Aggressive, Disruptive Behavior from Fall of First Grade Through Seventh Grade

A three-class solution fit the aggressive, disruptive behavior data best, separating into a persistent high class, an escalating medium class, and a stable low class based on aggressive, disruptive behavior ratings from first through seventh grade (Fig. 1). The persistent high class began with the highest rates of aggressive, disruptive behavior that rose through third or fourth grade, then decreased into middle school, but remained comparatively high throughout. The escalating medium class showed medium ratings in first grade and then increasing aggressive, disruptive behavior through seventh grade, but these ratings did not reach the high levels seen initially in the persistent high class. The stable low class began and continued with low ratings of aggressive, disruptive behavior. Both linear and quadratic slopes were needed to model the course of aggressive, disruptive behavior. The quadratic slope was treated as a fixed effect and intercept and slope were allowed to co-vary. The stable low class showed a smaller intercept and residual variance compared to the other classes. Figure 1 shows the intercepts and slopes of the three classes, with times of observation along the x-axis ranging from fall of first grade through spring of seventh grade.

Fig. 1
Impact of the GBG on aggressive, disruptive behavior development for Cohort 1 Males

Among all 199 Cohort 1 males, 26.8 % (an estimated 53 males) belonged to the persistent high class, 41.0 % (82) to the escalating medium class, and 32.2 % (64) to the stable low class of aggressive, disruptive behavior (Fig. 1). Among the persistent high group, GBG males displayed a lower slope compared to control males, indicating that the GBG had an impact in lowering the growth of aggressive, disruptive behavior through seventh grade (p=0.041; Fig. 1). We did not find a significant GBG impact in the other two classes.

Impact of the GBG on Lifetime Drug Abuse and Dependence Disorders

Using simple cross-tabulations, we found overall that males who were in the GBG first-grade classrooms had lower levels of lifetime drug abuse and dependence disorders compared with those in the control classrooms (19 % for GBG versus 38 % for controls, p=0.01). Using GGMM, having lifetime drug abuse and dependence disorders was modeled as a function of class membership and intervention status (Table 2; Fig. S1 available online). Males in the persistent high class from the GBG classrooms showed a reduction in the prevalence of lifetime drug abuse and dependence disorders from 65.6 % in the control classrooms, to only 28.1 % in the GBG classrooms (p=0.050; Table 2).

Table 2
Distribution of drug abuse and dependence disorder diagnosis and sexual behaviors during vaginal sex for cohort 1 males by intervention condition and aggressive, disruptive class (N=199)

Impact of the GBG on Lifetime Condom Use During Vaginal Sex

Overall, 78.5 % of GBG males reported using condoms in their lifetime compared to 74.5 % of control males (p= 0.628). As above, lifetime condom use during vaginal sex reported in young adulthood was then modeled as a function of risk group and intervention status (Figure S2 available online). Males in GBG classrooms showed a significantly higher prevalence of lifetime condom use in the persistent high class with 89.7 % of GBG males reporting they used condoms during vaginal sex compared to 40.8 % of the control males (p00.012; Table 2). Conversely, males in the escalating medium and stable low classes demonstrated no significant difference in condom use between GBG and control classrooms. In addition to lifetime condom use, we also looked at the GBG impact on condom use during vaginal sex in the last 30 days among Cohort 1 males. In all three classes no difference was statistically significant.

Impact of the GBG on Lifetime Number of Vaginal Sex Partners

Overall, GBG males reported an average of 11.1 lifetime vaginal sex partners compared to 15.1 for control males (p=0.267). Lifetime number of vaginal sex partners was also modeled as a function of class membership and intervention status (Figure S3 available online). There were no significant differences in the number of vaginal sex partners between GBG males and controls in any of the three classes, although the trend was in the predicted direction for the persistent high class (Table 2). In the persistent high class, GBG males reported having an average of 11.8 sex partners up to the time of survey, compared to an average of 26.5 vaginal sex partners among males in the control classrooms (p00.152; See Figure S3 available online for standard errors (SE)). Similar non-significant results were obtained when the number of lifetime sex partners was dichotomized into less than 10 partners and 10 or more partners or when the number of vaginal sex partners in the last 30 days was examined.

Impact of the GBG on Age of First Vaginal Sex

Overall, GBG males reported average age of 14.4 years old for their first vaginal sex experience compared to an average age of 13.9 years old for control males (p=0.286). Age of first vaginal sex was modeled as a function of intervention status and trajectory membership (Figure S4 available online). Males in the persistent high class from the GBG classrooms initiated vaginal sex significantly later (14.4 years old) compared to similar males from the control classrooms (12.3 years old, p=0.035; Table 2; See Figure S4 available online for SE). In the escalating medium and stable low classes, the difference was not significant.

Impact of the GBG on the Lifetime High-Risk Sexual Behavior Composite Score

The composite score of lifetime high-risk sexual behavior was defined as meeting at least one of the following criteria: a) age of vaginal sexual behavior onset of 13 years old or younger, b) having ten or more lifetime vaginal sexual partners, and/or c) not using condoms (i.e., responded “never,” “rarely” or “sometimes” to lifetime condom use). Overall, 59.4 % of GBG males reported high-risk sexual behavior compared to 66.7 % of control males (p=0.432). The prevalence of this high-risk sexual behavior among males was significantly reduced in the persistent high class from 89.1 % in control classrooms to only 52.5 % in GBG classrooms (p=0.036; Table 2; Figure S5 available online). Examining the escalating medium and stable low classes revealed no significant differences.

Impact of the GBG on Co-occurrence of Lifetime Drug Abuse and Dependence Disorders and High-Risk Sexual Behaviors

The impact of the GBG on the co-occurrence of lifetime drug abuse and dependence disorders and high-risk sexual behaviors varied by the type of sexual behavior in which the males were engaged. For the high-risk sexual behavior of not using a condom, overall 3.9 % of GBG males reported co-occurrence compared to 15.1 % of control males (p=0.043). Among males in the persistent high class, the difference was also significantly lower among GBG males compared to controls (0.9 % for GBG males and 44.3 % for control males, p=0.009). The difference did not reach statistical significance in the escalating medium and stable low classes. Co-occurrence of lifetime drug abuse and dependence disorders and the high-risk sexual behavior of ten or more lifetime vaginal sex partners was also examined. Overall, 6.7 % of GBG males reported co-occurrence compared to 20 % of control males (p=0.034). No significant difference was found between males from GBG classrooms compared to control classrooms in any of the three classes alone. No significant GBG impact was found on cooccurrence of lifetime drug abuse and dependence disorders and the high-risk sexual behavior of age of first vaginal sex at 13 years or younger overall or within any of the aggressive, disruptive classes. Assessing GBG impact on the cooccurrence of overall high-risk sexual behavior (based on the composite score) and lifetime drug abuse and dependence disorders revealed a significantly lower prevalence rate in the persistent high class (22.3 % for GBG males and 68 % for control males, p=0.021) and overall (12.9 % of GBG males reported co-occurrence compared to 35.4 % of control males, p=0.007).

We examined whether males tended to engage in many or all of the high-risk behaviors reported in this paper, including drug abuse and dependence disorders and the sexual behavior outcomes. Again, we found supporting evidence of clustering among the high-risk sex behaviors outcomes and/or drug abuse and dependence disorders in the persistent high class, but no evidence for clustering in either of the other two classes.

Impact of the GBG for Cohort 1 Females

Similar to the Cohort 1 males, a three-class solution fit the data the best, separating a persistent high class (29.0 %, an estimated 59 females), an escalating medium class (40.5 %, an estimated 83 females), and a stable low class (30.5 %, and estimated 62 females) based on aggressive, disruptive behavior from first through seventh grade (Figure S6 available online). The three classes performed similarly to the three classes for Cohort 1 males, but all started from lower baseline values, reflecting overall less aggressive behavior among females than males. No significant impact of the GBG was found on the development of aggressive, disruptive behavior through middle school or on the young adult outcomes of drug abuse and dependence disorders and the reported sexual behaviors (see Sections S3.2.1 through S3.2.7 available online).

Impact of the GBG for Cohort 2

Analogous to Cohort 1, a persistent high class, an escalating medium class, and a stable low class formation matched the data. Overall, there was reduced GBG impact on aggressive, disruptive behavior development into middle school and on the distal outcomes, drug abuse and dependence disorders and high-risk sexual behaviors, among males and females in the second cohort. Significant impact was found only among Cohort 2 males overall in increased condom use (89.9 % of GBG males and 75.6 % of control males, p= 0.05) and among persistent high Cohort 2 females in the cooccurrence of drug abuse and dependence disorders and high-risk sexual behavior (p=0.05; for more details, see Sections S3.3.1 through S3.4.7 available online).

Discussion

Overview and Implications

The GBG was directed in the Baltimore trials precisely at improving aggressive, disruptive classroom behavior in first and second grades. We hypothesized that improving this early risk behavior would lead to reductions in the entire profile of later externalizing problem outcomes that share this early antecedent. Through GGMM analyses we determined that both drug abuse and risky sexual behaviors were clearly related to aggressive, disruptive behavior rated by the teacher through elementary and middle school. We also found, as hypothesized, that the GBG significantly lowered drug abuse and dependence disorders and high-risk sexual behaviors among males in the persistent high class, whose members manifested this target risk behavior during the period of the GBG intervention. The other two classes with less aggressive behavior improved less, if at all. Because the risk behavior is much lower among females, we hypothesized that there would be little impact of GBG, and we found no significant impact among females on any single or composite sex behavior measure. The results presented in this paper complement those previously reported and support an etiological role of early and continuing aggressive, disruptive behavior in the development of drug abuse and sexual risk behavior in males. They also have important implications for research strategies for the prevention of STIs including HIV and other consequences of high-risk sexual behaviors such as unintended pregnancies, as well as drug abuse and dependence disorders and the profile of externalizing behavioral outcomes reported earlier. The findings also have implications for the role of early universal programs in the design and structure of an integrated prevention and treatment services system over the life course. For more details, see Section S4.1 available online.

The Etiological Role of Early and Continuing Aggressive, Disruptive Behavior

Rather than using specific interventions for each individual long-term outcome, the GBG is an effective strategy directed at early aggressive, disruptive classroom behavior that impacts a profile of later problem behavioral outcomes. Putting this finding another way, early aggressive behavior is a non-specific antecedent of the general class of externalizing problem outcomes, not of any one outcome specifically. The GBG appears to affect this shared early antecedent, demonstrating that this single antecedent has an etiological role leading to the profile of outcomes. For further discussion see Section S4.1.1 available online.

Life Course/Social Field Theory, Life Course Development, and How the GBG Works

Life course/social field theory postulates that aggressive maladaptive behavior in the social field of the classroom will lead to long-term poor outcomes in later social fields, and that directing an intervention, such as the GBG, at the social adaptational classroom process can ameliorate such outcomes. The results reported here are consistent with this prediction, thereby lending support for researching and then implementing universal prevention interventions earlier in the social adaptational processes of classroom, family, and peer group. In addition, the GBG involves teachers determining the classmate team membership and making certain that the teams are mixed in regard to gender and behavior. Aggressive, disruptive children are not left on their own to choose classmates who share this risk behavior, but are deliberately integrated into the GBG teams that are comprised of children of both genders and variation in aggressive, disruptive behavior. The process of determining more prosocial peer affiliations for aggressive, disruptive children may have importance to developmental continuity. Witvliet et al. (2009) reported the GBG decreased antisocial behavior among high-risk youth coinciding with decreased affiliations with deviant peers and lower rates of peer rejection, indicating these factors may partly mediate the beneficial effect of the GBG. These results are consistent with the intervention theory that led us to choose the GBG as our classroom intervention. Central to the GBG is the use of group contingency and heterogeneous teams composed of children with various levels of risk (i.e., aggressive, disruptive behavior). For further discussion see Section S4.2 available online.

The results underline the vital importance of the first-grade classroom as a social field where developmental trajectories are displayed and further shaped beyond the earlier social fields of family or preschool settings. This is in no way meant to deny the importance of earlier interactions between parents and children and preschool teachers and students. Social adaptation in first grade left to itself without intervention is strongly predictive of later social adaptation, but even by first grade there is still considerable malleability—room for improvement—particularly among higher-risk males. Universal interventions in the first-grade classroom can be decisive in setting the direction for social field theory in school and beyond. The strength and clarity of the impact of this precisely directed early intervention emphasize the validity of early universal interventions in this critical social field.

Gender Differences

In this and prior papers we report clear, consistent findings in the results across these young adult outcomes among early and continuing highly aggressive, disruptive males with significant and substantial impact among all of the externalizing outcome behaviors. As we see in the analyses reported in this paper, this holds true for high-risk sexual behaviors as well. In general, however, the relationship between aggressive, disruptive behavior and drug abuse and dependence disorders and sexual risk behaviors was not strong for females, and for all outcomes the GBG impact was much stronger for males.

Overall, these GBG outcome differences among males and females suggest that early developmental processes that are salient for males may be different than those for females. The differences may be critical in understanding the gender differences in developmental and in intervention outcomes, and in directing preventive interventions at those antecedent risk factors that are more germane to females. Early aggressive, disruptive behavior may not carry the same developmental weight for females as for males (see Section S4.2 available online).

The Second Cohort

The second cohort analyses reported in this paper and in the prior papers on young adult outcomes generally yielded non-significant findings but in the same direction as Cohort 1. We hypothesize that the GBG was implemented in the second cohort with less fidelity than it was in the first cohort due to the lack of sufficient mentoring and monitoring procedures. The GBG should be carried out with precision including continuing mentoring and monitoring. In the second generation trials in the early 1990s where we measured implementation more extensively, the results revealed marked reduction in impact when the intervention (combined classroom behavior management and enhanced curriculum/instruction) was done with less precision (Ialongo et al. 1999). This problem of low sustainability without continuing mentoring and monitoring is being reported by other investigators. It is a new frontier as we move through the phases of prevention research from efficacy through effectiveness into the problem of sustainability and ultimately into system-wide fidelity as programs are disseminated (Elliott and Mihalic 2004; Hallfors and Godette 2002).

Limitations

Sexual behavior and drug abuse data were gathered via a telephone interview and are based on retrospective reporting of behavior over the students' lifetime or the last 30 days. These data may be subject to recall bias and particularly to socially desirable responding. Telephone interviews have been reported to give comparable information to face-to-face interviews (Weeks et al. 1983) and audio computer-assisted survey interviews (Ellen et al. 2002), specifically for sexual behaviors (Ellen et al. 2002; Jeannin et al. 1998). Furthermore, telephone interviews have been shown to reduce socially-desirable reporting of sensitive data (Pless and Miller 1979). Of note, the delinquency and incarceration data reported in Petras et al. (2008) were based on official court records rather than interview data and significant GBG impact was also found, supporting the meaningfulness of data reported here. See Section S4.3 available online.

Public Health Implications for High-Risk Sexual Behaviors and Drug Abuse Disorders

The GBG can be used as a universal first-stage strategy, backed-up by selective and indicated strategies over the life course. Indeed, universal preventive strategies have been used for many infectious diseases in the form of universal vaccination programs, and Embry (2002) has described the use of the GBG as such a preventive vaccination program. Typically, however, interventions aimed at preventing high-risk sexual behaviors have narrowly focused on risk factors proximal to those behaviors. According to data from the 2006–2008 National Survey of Family Growth (Martinez et al. 2010), 38 % of male and 47 % of female teenagers first received formal sex education related to birth control methods in high school, yet our results suggest initiation occurs before high school for a significant number of youth. Universal prevention interventions like the GBG are critically important in an overall prevention/treatment system. By intervening on a proven set of early antecedents to risky sexual and drug behaviors as well as other externalizing outcomes, the GBG can be a first-line intervention for preventing these behaviors. Essentially, we can implement a prevention intervention in childhood aimed at reducing high-risk sexual and drug behavior without actually talking about sexuality or drug abuse with young children. For many parents and communities this approach would be more acceptable than simply having drug and sexual education earlier. See Section S4.4 available online and Kellam et al. 2011.

The results also point to new areas for training new teachers and in-service training for more experienced ones. Teachers are not as often trained in classroom behavior management as these results strongly suggest they should be. Our data indicate that up to half of the teachers are not prepared to manage their first-grade classrooms effectively, and effective tools such as the GBG can and must be provided to teachers (Kellam et al. 1998). Providing the teacher with tools for socializing children into the role of student and managing the classroom appears to reduce the high-risk for these early aggressive, disruptive children, and demonstrates the utility of this universal intervention for maintaining such children in the mainstream classroom and helping them to develop successfully.

SAMHSA has announced grant awards totaling $11 million to 22 school districts nationwide to implement the GBG in early elementary school classrooms, and to measure the fidelity and sustainability in order to document and learn how to support this movement from research to practice (SAMHSA 2010). Translating research to practice efficiently with fidelity and sustainability is vital (see Section S4.4 available online).

Ultimately, prevention research in the public education and public health fields need to be integrated (Kellam 2012; Kellam et al. 2011). The risk factors for outcomes in each field overlap and are in many cases the same. Both require basic partnerships between scientists, research institutions and school districts. This particular study is the product of such a partnership. To develop a design and carry out randomization at multiple levels as described here required a continuing partnership with the BCPSS, just as it was in the prior Woodlawn studies (Kellam 2000). The studies reported here and elsewhere from the Baltimore partnership have been based on strong mutual self-interests of the BCPSS, the families and their children, and the researchers. We have functioned over almost three decades of our work together as a research and development arm of the BCPSS, while carrying out basic and applied prevention research.

Supplementary Material

supplement published

Acknowledgments

Funding sources During the last 27 or more years this research has been supported by NIMH Grants R01 MH 42968, P50 MH 38725, R01 MH 40859, and T32 MH018834 (with supplements from NIDA for each of the cited research grants), and NIDA grants R01 DA 019984, DA009897 and P30 DA027828.

Footnotes

1Given the same intervention, population and similar methods, much of the text describing measures and shared background in this paper is drawn strongly from the Drug and Alcohol Dependence supplemental issue, especially from Kellam et al. 2008 and Petras et al. 2008. For more details see the supplementary material available online.

Electronic supplementary material The online version of this article (doi:10.1007/s11121-012-0296-z) contains supplementary material, which is available to authorized users.

Contributor Information

Sheppard G. Kellam, Department of Mental Health, Bloomberg School of Public Health, Johns Hopkins University, 624 N. Broadway, Room 786, Baltimore, MD 21205, USA.

Wei Wang, Department of Epidemiology & Biostatistics, College of Public Health, University of South Florida, Tampa, FL, USA.

Amelia C. L. Mackenzie, Division of Reproductive Biology, Department of Biochemistry and Molecular Biology, Bloomberg School of Public Health, Johns Hopkins University, Baltimore, MD, USA.

C. Hendricks Brown, Center for Prevention Implementation Methodology, Department of Epidemiology and Public Health, Miller School of Medicine, University of Miami, Miami, FL, USA.

Danielle C. Ompad, Department of Nutrition, Food Studies and Public Health, Steinhardt School of Culture, Education, and Human Development, New York University, New York, NY, USA.

Flora Or, Department of Mental Health Bloomberg School of Public Health, Johns Hopkins University, Baltimore, MD, USA.

Nicholas S. Ialongo, Department of Mental Health, Bloomberg School of Public Health, Johns Hopkins University, 624 N. Broadway, Room 786, Baltimore, MD 21205, USA.

Jeanne M. Poduska, American Institutes for Research, Baltimore, MD, USA.

Amy Windham, American Institutes for Research, Baltimore, MD, USA.

References

  • Drug and Alcohol Dependence. 2008;95:S1–S104. [PMC free article] [PubMed]
  • American Psychiatric Association. Diagnostic and statistical manual of mental disorders: DSM-IV. 4th. Washington, DC: American Psychiatric Association; 1994.
  • Barrish HH, Saunders M, Wolf MM. Good Behavior Game: effects of individual contingencies for group consequences on disruptive behavior in a classroom. Journal of Applied Behavior Analysis. 1969;2:119–124. [PMC free article] [PubMed]
  • Brown CH, Wang W, Kellam SG, Muthen BO, Petras H, Toyinbo P, et al. Methods for testing theory and evaluating impact in randomized field trials: Intent-to-treat analyses for integrating the perspectives of person, place, and time. Drug and Alcohol Dependence. 2008;95:S74–S104. doi: 10.1016/j.drugalcdep.2007.11.013. [PMC free article] [PubMed] [Cross Ref]
  • Brown LK, Hadley W, Stewart A, Lescano C, Whiteley L, Donenberg G, et al. Psychiatric disorders and sexual risk among adolescents in mental health treatment. Journal of Consulting and Clinical Psychology. 2010;78:590–597. doi: 10.1037/a0019632. [PMC free article] [PubMed] [Cross Ref]
  • Centers for Disease Control and Prevention. Sexually Transmitted Disease Surveillance 2009. Atlanta, GA: U.S. Department of Health and Human Services; 2010.
  • Eaton DK, Kann L, Kinchen S, Shanklin S, Ross J, Hawkins J, et al. Youth risk behavior surveillance–United States, 2007. Morbidity and Mortality Weekly Report Surveillance Summary. 2008;57:1–131. [PubMed]
  • Ellen JM, Gurvey JE, Pasch L, Tschann J, Nanda JP, Catania J. A randomized comparison of A-CASI and phone interviews to assess STD/HIV-related risk behaviors in teens. Journal of Adolescent Health. 2002;31:26–30. doi: 10.1016/S1054-139X(01)00404-9. [PubMed] [Cross Ref]
  • Elliott DS, Mihalic S. Issues in disseminating and replicating effective prevention programs. Prevention Science. 2004;5:47–53. doi: 10.1023/B:PREV.0000013981.28071.52. [PubMed] [Cross Ref]
  • Embry DD. The Good Behavior Game: A best practice candidate as a universal behavioral vaccine. Clinical Child and Family Psychology Review. 2002;5:273–297. doi: 10.1023/A:1020977107086. [PubMed] [Cross Ref]
  • Hallfors D, Godette D. Will the ‘principles of effectiveness’ improve prevention practice? Early findings from a diffusion study. Health Education Research. 2002;17:461–470. doi: 10.1093/her/17.4.461. [PubMed] [Cross Ref]
  • Ialongo NS, Werthamer L, Kellam SG, Brown CH, Wang S, Lin Y. Proximal impact of two first-grade preventive interventions on the early risk behaviors for later substance abuse, depression, and antisocial behavior. American Journal of Community Psychology. 1999;27:599–641. [PubMed]
  • Jeannin A, Konings E, Dubois-Arber F, Landert C, Van Melle G. Validity and reliability in reporting sexual partners and condom use in a Swiss population survey. European Journal of Epidemiology. 1998;14:139–146. [PubMed]
  • Kellam SG. Community and institutional partnerships for school violence prevention. Preventing School Violence: Plenary Papers of the 1999 Conference on Criminal Justice Research and Evaluation—Enhancing Policy and Practice through Research. Vol. 2. Washington, D.C.: U.S. Department of Justice, Office of Justice Programs, National Institute of Justice; 2000. pp. 1–22.
  • Kellam SG. Developing and maintaining partnerships as the foundation of implementation and implementation science: reflections over a half century. Administration and Policy in Mental Health and Mental Health Services Research. 2012;28 doi: 10.1007/s10488-011-0402-8. Published online: 12 January 2012. [PubMed] [Cross Ref]
  • Kellam SG, Branch JD, Agrawal KC, Ensminger ME. Mental health and going to school: The Woodlawn program of assessment, early intervention, and evaluation. Chicago, IL: University of Chicago Press; 1975.
  • Kellam SG, Ling X, Merisca R, Brown CH, Ialongo N. The effect of the level of aggression in the first grade classroom on the course and malleability of aggressive behavior into middle school. Development and Psychopathology. 1998;10:165–185. [PubMed]
  • Kellam SG, Brown CH, Poduska JM, Ialongo NS, Wang W, Toyinbo P, et al. Effects of a universal classroom behavior management program in first and second grades on young adult behavioral, psychiatric, and social outcomes. Drug and Alcohol Dependence. 2008;95:S5–S28. doi: 10.1016/j.drugalcdep.2008.01.004. [PMC free article] [PubMed] [Cross Ref]
  • Kellam SG, Mackenzie A, Brown CH, Poduska JM, Wang W, Petras H, et al. The Good Behavior Game and the future of prevention and treatment. Addiction Science & Clinical Practice. 2011;6:73–84. [PMC free article] [PubMed]
  • Kessler RC, McGonagle KA, Zhao S, Nelson CB, Hughes M, Eshleman S, et al. Lifetime and 12-month prevalence of DSM-III-R psychiatric disorders in the United States. Results from the National Comorbidity Survey. Archives of General Psychiatry. 1994;51:8–19. [PubMed]
  • Mackenzie A, Lurye I, Kellam S. History and evoluation of the Good Behavior Game. Supplementary paper in Drug and Alcohol Dependence. 2008;95:S5–S28.
  • Martinez G, Abma J, Copen C. Educating teenagers about sex in the United States. National Center for Health Statistics Data Brief. 2010;44:1–8. [PubMed]
  • Miller S, Malone PS, Dodge KA. Conduct Problems Prevention Research Group. Developmental trajectories of boys' and girls' delinquency: Sex differences and links to later adolescent outcomes. Journal of Abnormal Child Psychology. 2010;38:1021–1032. doi: 10.1007/s10802-010-9430-1. [PMC free article] [PubMed] [Cross Ref]
  • Muthen B. Should substance use disorders be considered as categorical or dimensional? Addiction. 2006;101:6–16. doi: 10.1111/j.1360-0443.2006.01583.x. [PubMed] [Cross Ref]
  • Muthén B, Asparouhov T. Growth mixture analysis: models with non-gaussian random effects. In: Fitzmaurice G, Davidian M, Verbeke G, Molenberghs G, editors. Advances in longitudinal data analysis. London, UK: Chapman & Hall/CRC Press; 2006. pp. 143–165.
  • Muthén LK, Muthén BO. Mplus user's guide. 6th. Los Angeles, CA: Muthén & Muthén; 1998–2010.
  • Muthén B, Brown CH, Masyn K, Jo B, Khoo ST, Yang CC, et al. General growth mixture modeling for randomized preventive interventions. Biostatistics. 2002;3:459–475. doi: 10.1093/biostatistics/3.4.459. [PubMed] [Cross Ref]
  • Nylund KL, Asparouhov T, Muthén BO. Deciding on the number of classes in latent class analysis and growth mixture modeling: A Monte Carlo simulation study. Structural Equation Modeling. 2007;14:535–563.
  • Ompad DC, Strathdee SA, Celentano DD, Latkin C, Poduska JM, Kellam SG, et al. Predictors of early initiation of vaginal and oral sex among urban young adults in Baltimore, Maryland. Archives of Sexual Behavior. 2006;35:53–65. doi: 10.1007/s10508-006-8994-x. [PubMed] [Cross Ref]
  • Parsons T. The social system. Glencoe, IL: The Free Press; 1951.
  • Petras H, Kellam SG, Brown CH, Muthen BO, Ialongo NS, Poduska JM. Developmental epidemiological courses leading to antisocial personality disorder and violent and criminal behavior: Effects by young adulthood of a universal preventive intervention in first- and second-grade classrooms. Drug and Alcohol Dependence. 2008;95:S45–59. doi: 10.1016/j.drugalcdep.2007.10.015. [PMC free article] [PubMed] [Cross Ref]
  • Pless IB, Miller JR. Apparent validity of alternative survey methods. Journal of Community Health. 1979;5:22–27. [PubMed]
  • Poduska JM, Kellam SG, Wang W, Brown CH, Ialongo NS, Toyinbo P. Impact of the Good Behavior Game, a universal classroom-based behavior intervention, on young adult service use for problems with emotions, behavior, or drugs or alcohol. Drug and Alcohol Dependence. 2008;95:S29–44. doi: 10.1016/j.drugalcdep.2007.10.009. [PMC free article] [PubMed] [Cross Ref]
  • SAMHSA. SAMHSA awards up to $11 million to economically disadvantaged communities to promote behavioral health in children. Washington, DC: Substance Abuse and Mental Health Services Administration Press Office; 2010.
  • Vlahov D, Anthony JC, Munoz A, Margolick J, Nelson KE, Celentano DD, et al. The ALIVE study, a longitudinal study of HIV-1 infection in intravenous drug users: Description of methods and characteristics of participants. NIDA Research Monographs. 1991;109:75–100. [PubMed]
  • Wang CP, Brown CH, Bandeen-Roche K. Residual diagnostics for growth mixture models: Examining the impact of a preventive intervention on multiple trajectories of aggressive behavior. Journal of the American Statistical Association. 2005;100:1054–1076. doi: 10.1198/016214505000000501. [Cross Ref]
  • Weeks MF, Kulka RA, Lessler JT, Whitmore RW. Personal versus telephone surveys for collecting household health data at the local level. American Journal of Public Health. 1983;73:1389–1394. [PubMed]
  • Werthamer-Larsson L, Kellam S, Wheeler L. Effect of first-grade classroom environment on shy behavior, aggressive behavior, and concentration problems. American Journal of Community Psychology. 1991;19:585–602. [PubMed]
  • Wilcox HC, Kellam SG, Brown CH, Poduska JM, Ialongo NS, Wang W, et al. The impact of two universal randomized first- and second-grade classroom interventions on young adult suicide ideation and attempts. Drug and Alcohol Dependence. 2008;95:S60–73. doi: 10.1016/j.drugalcdep.2008.01.005. [PMC free article] [PubMed] [Cross Ref]
  • Witvliet M, van Lier PA, Cuijpers P, Koot HM. Testing links between childhood positive peer relations and externalizing outcomes through a randomized controlled intervention study. Journal of Consulting and Clinical Psychology. 2009;77:905–915. doi: 10.1037/a0014597. [PubMed] [Cross Ref]
  • Wu J, Witkiewitz K, McMahon RJ, Dodge KA, Conduct Problems Prevention Research Group A parallel process growth mixture model of conduct problems and substance use with risky sexual behavior. Drug and Alcohol Dependence. 2010;111:207–214. doi: 10.1016/j.drugalcdep.2010.04.013. [PMC free article] [PubMed] [Cross Ref]