The Community Youth Development Study (CYDS -- Hawkins et al., 2008b
) is the first community-randomized trial of CTC. It was designed to investigate whether CTC reduces levels of risk, increases levels of protection, and reduces the incidence and prevalence of tobacco, alcohol, and other drug use and delinquency in early adolescence in communities. Communities in the CYDS were selected from 41 communities in the states of Colorado, Illinois, Kansas, Maine, Oregon, Utah, and Washington that participated in an earlier naturalistic study of the diffusion of science-based prevention strategies, called the Diffusion Project (Arthur, Glaser, & Hawkins, 2005
). The drug abuse prevention agencies in these states identified 20 of these communities that the agencies thought were trying to implement risk- and protection-focused prevention services. These 20 communities were then matched, within state, on population size, racial and ethnic diversity, economic indicators, and crime rates to comparison communities that were not thought to be using a risk and protection-focused approach, and the community pairs were recruited to participate in the Diffusion study. Following the prevention science framework for community prevention planning and Rogers' (1995)
stages of innovation diffusion, each community's stage of adoption of a science-based approach to prevention was assessed. In Stage 0, the community showed little or no awareness of prevention science concepts and their relevance to prevention programming. At stage 1, the community showed awareness of prevention science terminology and concepts including risk and protective factors, but did not use these concepts to guide prevention programming. In Stage 2, the community had adopted a science-based approach in planning prevention initiatives, but did not collect epidemiologic data to guide the selection of prevention activities in the community or use tested and effective preventive interventions. A community at Stage 3 collected epidemiologic risk and protective factor data but did not use tested and effective preventive interventions. In Stage 4, the community used tested and effective preventive interventions to address prioritized risk and protective factors based on epidemiologic data collected in the community. Finally, if a community had reached Stage 5, it used tested and effective preventive interventions and engaged in ongoing assessments to monitor implementation and effects of the interventions (Arthur et al., 2005
). Data for measuring community adoption of science-based prevention were obtained from telephone interviews conducted with 15 community leaders in each community across multiple sectors (including human services, schools, law enforcement, civic organizations, youth recreation, juvenile justice, health agencies, businesses, media, and religious organizations). In spite of states' initial assessments of these communities, neither community in 13 of the 20 pairs of communities was advanced in the use of science-based prevention to the point of Stage 4 where they selected and used tested, effective preventive interventions to address prioritized community risks during the 5 years of the Diffusion Project (Arthur et al., 2005
). These 13 pairs of communities were deemed eligible for inclusion in the CYDS study. Recruitment of communities required securing letters from the superintendent of schools, the mayor or city manager, and the lead law enforcement officer, agreeing to all data collection activities required of the project. Twelve of the 13 pairs of matched communities (24 communities total) met all recruitment criteria and were successfully recruited for the CYDS. One community from within each matched pair was assigned randomly by a coin toss to either the intervention (CTC) or control condition (Hawkins et al., 2008b
Implementation of the CTC System
CTC training and implementation began in the 12 intervention communities in the summer of 2003. Intervention communities received six CTC trainings delivered over the course of 6 to 12 months by certified CTC trainers. CYDS implementation staff provided technical assistance throughout the study via weekly phone calls, emails, and site visits to CTC communities at least once per year. The first stage of CTC began when community leaders were oriented to the CTC system and identified or created a community coalition of diverse stakeholders to implement CTC. Coalition members were trained to use data from surveys of community students collected every 2 years, beginning in 1998, to prioritize risk factors to be targeted by preventive actions in the community; to choose tested and effective prevention policies and programs that address the community's targeted risk factors; to implement these interventions with fidelity; and to monitor implementation and outcomes of newly installed prevention programs. Because the CYDS was initially funded by a 5-year grant, CTC communities in CYDS were asked to focus their prevention plans on programs for youths aged 10 to 14 years (Grades 5 through 9) and their families and schools so that possible effects on drug use and delinquency could be observed within the grant period.
Based on their unique profile of risk identified by the community-wide CTC student survey data, each CTC community in CYDS prioritized a different set of risk factors to be targeted by preventive programs. Each CTC community selected between 2 and 5 risk factors, for a total of 11 different risk factors across all 12 intervention communities (including community laws and norms favorable towards problem behavior, family management problems, family conflict, parental attitudes favorable to antisocial behavior, student attitudes favorable toward antisocial behavior, academic failure, low commitment to school, rebelliousness, antisocial peers, peer rewards for antisocial behavior, and low perceived risk of drug use among students).
To address their prioritized risk factors, CTC communities in CYDS chose programs from the CTC Prevention Strategies Guide (Substance Abuse and Mental Health Services Administration, 2005
), which provides a menu of programs that have been found to be effective in well-controlled trials in preventing tobacco, alcohol, or other drug use or delinquent behavior. The menu also identifies the risk and protective factors addressed by each intervention. Chosen programs included school-based programs (All-Stars, Life Skills Training, Lion's Quest Skills for Adolescence, Project Alert, Olweus Bullying Prevention Program
, and Program Development Evaluation Training
) as well as community-based, youth-focused programs (Participate and Learn Skills, Big Brothers/Big Sisters, Stay Smart
, and academic tutoring), and family-focused programs (Strengthening Families 10-14, Guiding Good Choices, Parents Who Care, Family Matters
, and Parenting Wisely
) (Fagan, Hanson, Hawkins, & Arthur, 2008b
; Quinby et al., 2008
). Most programs were universal in nature, designed to be implemented with all students in targeted grades, for example, or for parents of all middle school children in the community, regardless of family problems or youth involvement in problem behaviors. Tutoring programs and the Big Brothers/Big Sisters program, however, were selective interventions and targeted youth with low academic performance and those from single-parent families, respectively. During each of the 2004-2005, 2005-2006, and 2006-2007 school years, community coalitions implemented from one to five of these programs to address their prioritized risk factors, as identified through the student survey data. On average, three programs were implemented per community each year. Programs were implemented by local providers, including teachers for school programs; health and human service workers for community-based, youth-focused, and family-focused programs; and community volunteers for tutoring programs and Big Brothers/Big Sisters.
Previous analyses of CYDS data have found that the CTC system was successfully implemented with fidelity in intervention communities (Fagan, Hanson, Hawkins, & Arthur, 2009
; Quinby et al., 2008
) and that levels of adoption of science-based prevention and levels of community collaboration were significantly higher in CTC than control communities 1.5 years after initial implementation (Brown, Hawkins, Arthur, Briney, & Abbott, 2007
). Prior analyses also found that tested and effective preventive programs were selected and well implemented in the CTC communities (Fagan, Hanson, Hawkins, & Arthur, 2008a
). Hypothesized effects of CTC on risk factors targeted by the intervention communities and on the incidence of delinquent behavior among youth were observed 3 years after implementation of CTC (Hawkins et al., 2008a
). Four years after implementation of CTC, the incidences of delinquent behavior, alcohol, cigarette, and smokeless tobacco initiation between Grades 5 and 8 were found to be significantly lower in CTC than in control communities. In addition, Grade 8 prevalences of alcohol and smokeless tobacco use in the last 30 days, binge drinking in the past 2 weeks, and the number of different delinquent behaviors committed in the past year were found to be significantly lower in CTC than in control communities (Hawkins et al., 2009
Student Sample and Data Collection
Data on adolescent drug use and delinquent behavior were obtained from annual surveys of a panel of public school students who were in the fifth grade during the 2003-2004 school year in the 24 CYDS communities. The first wave of data, collected in the spring of 2004, was a pre-intervention baseline assessment. Tested prevention programs were implemented in CTC communities beginning in the summer and fall of 2004. The fourth annual wave of student data was collected in the spring of 2007 when panel students progressing normally were in Grade 8, about 2.67 years after the prevention programs chosen by CTC communities were first implemented.
Grade 6 (Wave 2) data collection included an effort to recruit students who were not surveyed in Grade 5. During Grades 5 and 6, parents of 4,420 students (76.4% of the eligible population) consented to their participation in the study. Final consent rates did not differ significantly by intervention condition. Consent rates were 76.2% for students in intervention communities and 76.7% for students in control communities. Thirteen of the 4,420 consented students were absent during scheduled dates of data collection and were not available for initial surveying. The final active longitudinal panel consisted of 4,407 students (2,194 girls, 2,213 boys; 55% from intervention communities). Students in the longitudinal panel who remained in intervention or control communities for at least one semester were tracked and surveyed at each of the following waves, even if they left the community. Ninety-six percent of students in the longitudinal panel completed the survey in Wave 4 (Grade 8).
Students completed the Youth Development Survey (YDS--Social Development Research Group, 2005-2007
) a self-administered, paper-and-pencil questionnaire designed to be completed in a 50-minute classroom period. The YDS is based on the CTC Youth Survey which has been demonstrated to have good reliability and validity (Arthur, Hawkins, Pollard, Catalano, & Baglioni, 2002
; Glaser, Van Horn, Arthur, Hawkins, & Catalano, 2005
). To ensure confidentiality, identification numbers but no names or other identifying information were included on the surveys. Parents of panel students provided written informed consent for their children's participation in the study. Students read and signed assent statements indicating that they were informed fully of their rights as research participants and agreed to participate in the study. Upon completion of the survey, students received small incentive gifts worth approximately $5 to $8. The University of Washington's Human Subjects Review Committee has approved this protocol. Additional details on recruitment and data collection can be found in Brown et al. (2009)
and Hawkins et al. (2009)
Measures of baseline risk, substance use and delinquency outcomes, and student characteristics were based on data collected with the YDS instrument. Community demographic characteristics were based on data from the National Center for Education Statistics.
At-risk youth were identified at the baseline assessment according to three criteria: engagement in delinquent behavior, lifetime substance use, and high levels of risk factors targeted by intervention communities. The baseline measure of delinquent behavior was based on student reports of four different delinquent acts (stealing, property damage, shoplifting, and attacking someone) committed in the past year. If students had engaged in any of the four behaviors in the past year, they were coded as 1 (=delinquent), otherwise as 0 (=not delinquent). Dichotomous measures of lifetime alcohol and cigarette use at baseline (1 = use and 0 = no use) were created from student-reported use of both drugs (“Have you ever had more than just a sip or two of beer, wine, or hard liquor [for example, vodka, whiskey, or gin]?
” and “Have you ever smoked a cigarette, even just a puff?
”). High targeted risk was defined as at least one standard deviation above the sample mean (coded 1, otherwise 0) on the targeted risk factor scale at baseline. The targeted risk factor score was calculated by taking the average of the community-specific set of targeted risk factors in CTC communities. Since control communities did not prioritize and target risk and protective factors using the CTC process, each control community's risk factor score was calculated based on the set of targeted risk factors in its matched CTC community. Prior analyses showed that CTC and control communities had equivalent baseline levels of targeted risk factors (Hawkins et al., 2008a
In Grade 8, students reported whether they had engaged in nine different delinquent acts (stealing, property damage, shoplifting, attacking someone, carrying a gun to school, beating up someone, stealing a vehicle, selling drugs, and being arrested) in the past year. By summing across the nine behaviors, a measure of the variety of delinquent acts was constructed ranging from 0 to 9. The prevalences (with any use dichotomized as 1 and no use as 0) of the use of alcohol, cigarettes, smokeless tobacco, and marijuana in the past month and of binge drinking (consuming 5 or more drinks in one drinking occasion) during the past 2 weeks were also measured in Grade 8 (e.g., “On how many occasions (if any) have you had beer, wine, or hard liquor during the past 30 days?”).
Student and community characteristics
Variables measuring student characteristics used as covariates in analyses included: age at time of the Grade 6 survey; gender (coded 1 = male, 0 = female); race/ethnicity (coded 1 = White or Caucasian, 0 = other); whether the student was Hispanic (coded 1 = yes, 0 = no); parental education level (ranging from 1 = grade school or less to 6 = graduate or professional degree); attendance at religious services at baseline (coded 0 = never to 4 = about once a week or more); and rebelliousness at baseline, which consisted of the mean of three items (alpha = .69): I like to see how much I can get away with; I ignore rules that get in my way; and I do the opposite of what people tell me, just to get them mad (coded from 1 = very false to 4 = very true). Variables measuring community demographic characteristics included the total population of students in the community and the percentage of students who received free or reduced price school lunches. Intervention condition was coded 1 for CTC communities and 0 for control communities.
Analysis Sample and Missing Data Procedures
Among the 4,407 students comprising the consented longitudinal panel, 26.5% were recruited in Wave 2 (Grade 6 accretion sample) and consequently did not complete a questionnaire in Wave 1 (Grade 5). Overall, 96.7% of panel students participated in at least three of four waves of data collection. A few students' data (0.7% in Grade 5 and 1.4% in Grade 8) did not meet validity criteria because they reported being honest only “some of the time” or less, having used a fictitious drug included in the survey as a validity screen, or that they had used two of three drugs (marijuana, inhalants, or other drugs) on 40 or more occasions during the past month (Hawkins et al., 2009
). If students met one or more of these validity screens in a given year, their data were deemed invalid in that year and were set to missing. Any valid information these students provided in any other year informed the imputation of the missing data (see below) and estimation of the analysis models. The proportion of students in the analysis sample who did not respond to the delinquency and drug use questions was small. Item nonresponse ranged from 0.6% (for smokeless tobacco use in Grade 5) to 2.7% (for specific delinquency items in Grade 8). Missing data were dealt with via multiple imputation (Schafer & Graham, 2002
). Using NORM version 2.03 (Schafer, 2000
), 40 separate data sets including data from all four waves were imputed separately by intervention condition (Graham, Taylor, Olchowski, & Cumsille, 2006
). Imputation models included student and community characteristics, targeted risk factors, drug use and delinquent behavior outcomes, and community membership. Imputed data sets were combined subsequently to include both intervention and control groups for analysis. There was no systematic bias due to differential accretion or differential attrition in control and intervention conditions (analyses not shown). With regard to both accretion and attrition, the methods for imputing missing data used in this study have been shown in simulations by Collins, Schafer, and Kam (2001)
and extensions by Graham (2009 – personal communication) to produce estimates of standard errors that differ little from population values.
Intervention effects on eighth-grade drug use and delinquency and their moderation by baseline risk and gender were assessed using the same models as used in the previous study of the main effects of CTC. The Generalized Linear Mixed Model (GLMM -- Breslow & Clayton, 1993
; Liang & Zeger, 1986
; Murray, 1998
) with the logit link function was used for the dichotomously coded prevalence of drug use outcomes and the Poisson link function for the count-based variety of delinquent behaviors outcome. Random-intercept models were estimated to account for variation within students, among students within communities, and communities within matched pairs of communities. All analyses were adjusted for the student- and community-level covariates (grand-mean centered) described above and were conducted using HLM version 6.0 (Raudenbush, Bryk, Cheong, & Congdon, 2004
). Results were averaged across imputed data sets using Rubin's rules (Rubin, 1987
). Approximate degrees of freedom across imputations were calculated using the formulas provided by Raudenbush et al. (Raudenbush & Bryk, 2002
; Raudenbush et al., 2004
To examine whether the effect of CTC on the prevalence of drug use and the variety of delinquent acts varied by baseline risk and gender, analyses included the community-level dichotomous indicator of intervention status (0 = control community, 1 = CTC community), the student-level dichotomous variable for baseline risk or gender, and the interaction of intervention status by baseline risk or gender. Analyses of Grade 8 alcohol use, binge drinking, and marijuana use included lifetime alcohol use at baseline as the risk variable; models predicting eighth-grade cigarette and smokeless tobacco use included lifetime cigarette use at baseline as the risk moderator. Because the 24 CYDS communities were matched in 12 pairs before randomization, the significance of the interaction effect was tested using a two-tailed critical t-value with p-1=11 degrees of freedom. Because power to detect interactions is inherently low (Brown et al., 2008
; Leon & Heo, 2009
), a Type-I error rate of .10 was used to assess the significance of the interaction effect.