This cluster-randomized study extends the positive findings of previous quasi-experimental studies of the Positive Action program22,23
by examining effects on student and teacher reports of student involvement in negative behaviors. Students who received the Positive Action intervention were significantly less likely to engage in substance use, violent behaviors, and sexual activity than were students who did not. The effects sizes averaged 0.73 and 0.34 for student and teacher reports, respectively, corresponding to a reduction in likelihood of having ever done the behavior ranging from 48% to 86%, compared with students who did not receive the Positive Action intervention.
The observed effects were consistent with (and sometimes stronger than) the effects reported in recent systematic reviews and meta-analyses of school-based programs targeting negative behaviors. In these studies, the average effect size was approximately .3016
for school-based substance-use programs with interactive components and ranged from 0.20 to 0.35 for programs targeting aggressive and disruptive behaviors,17
resulting in an average reduction of approximately 17.5% (range=2.3%–45.3%).44
Hence, the effect sizes (based on student reports) observed in our study fall at the upper end of the effect-size continuum,16
suggesting that the introduction of a comprehensive schoolwide social and character development program can cause substantial reductions in the prevalence of these negative behaviors during early adolescence.2,19
The reduction in the odds of students using substances and performing violent behaviors by approximately 58% and of having sex voluntarily by 76% has provided clear public health benefits for the Hawaii school district, particularly in light of the high prevalence rates of middle-school and high school youths involved in such behaviors statewide.3
The large effects observed here were likely the result of several important features of the Positive Action program. First, the Positive Action program is “interactive” in delivery: it integrates teacher–student contact and communication opportunities for the exchange of ideas, and it uses feedback and constructive criticism in a nonthreatening atmosphere. Interactive methods produce stronger beneficial program outcomes than do noninteractive delivery methods (i.e., those that are didactic in nature).16
Second, the Positive Action program is a comprehensive approach to prevention that provides the curriculum to all grades in the school at once, involving all teachers, staff, and parents. Third, the Positive Action program is a holistic approach to social and emotional development that addresses the self, emotional regulation, moral development, decision-making, skills development in these areas, and clear identification of which behaviors are positive, rather than focusing solely on the negative aspects of engaging in substance use and violence. Fourth, the program is intensive, with students receiving approximately 1 hour of exposure during a typical week. The magnitudes of the effect size differed between the student and teacher reports; this was most likely a result of teachers’ inability to observe the students’ behaviors at all times, leading to an underestimation of how well the item described the student.
The dose–response analyses clearly demonstrate that more exposure to the program decreased the number of reported negative behaviors. Those students who received 3 or more years of the Positive Action program reported 41% to 73% fewer experiences with substance use and violent behaviors and an 89% lower rate of engaging in voluntary sexual activity than did students who received less exposure to the Positive Action program. Reductions were still observed for students exposed for 1 or 2 years (although not all of the reductions were significant), suggesting that even a short exposure had a beneficial effect. Exposing youths to the program for an additional 1 to 2 years appeared to reduce the negative behaviors by half. Hence, these findings suggest that an adequate test of the intervention’s potential effectiveness could only be conducted after students had been exposed to the program for 3 or more years. This finding suggests that multiyear trials are necessary to realize the full effect of a comprehensive prevention program.
This study had some limitations that require attention. First, the reports of negative behaviors were collected only during fifth grade and only for the 2 cohorts followed in the study, and therefore may not reflect the behavior of the entire student body. This limitation was a result of the study design and of restrictions required by the institutional review board that prevented the use of sensitive questions with younger (i.e., fourth grade and below) students.
Second, only students who provided active parental consent and verbal assent responded to the negative behavior items. For the student self-report data, it is possible that some kind of selection effect led to a sample that was not typical of all the students in the schools studied. Our empirical tests for such a selection effect found no such difference in the area of negative behaviors. The negative behavior rates reported in this study are consistent with rates reported for children of similar ages across the Hawaii school district3
and are therefore likely to be representative of actual behavioral involvement.
Also, the use of a single item to assess voluntary sexual activity is unlikely to capture all the types of sexual activity that youth engage in. Moreover, the low prevalences of the negative behaviors makes it difficult to determine whether the program would have the same size of effect on older youths (i.e., middle school), when these behaviors become more prevalent. Finally, no adjustment for type-1 error rates in the analyses (as a result of multiple tests) were made, which should be considered when interpreting the significance levels of the findings.
Overall, our findings indicate that the Positive Action program can be effective in reducing multiple problem behaviors simultaneously. Programs such as Positive Action can reduce the burden on school administrators and teachers and ameliorate the demand on limited resources21
by reducing the rates of multiple problem behaviors. We are unaware of previous studies reporting the effects of prevention programs on the scale presented herein; thus, this study is likely the first to provide evidence that a comprehensive, schoolwide social and character development program can have a substantial impact on reducing problem behaviors of public health importance for more than a thousand students at a time. Although numerous school-based prevention programs exist, the Positive Action program is one of the few that has demonstrated substantial effects on multiple negative behaviors.