PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptNIH Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
Prev Sci. Author manuscript; available in PMC Oct 7, 2008.
Published in final edited form as:
PMCID: PMC2562862
NIHMSID: NIHMS46898
Testing Communities That Care: The Rationale, Design and Behavioral Baseline Equivalence of the Community Youth Development Study
J. David Hawkins,* Richard F Catalano,* Michael W. Arthur,* Elizabeth Egan, Eric C. Brown,* Robert D. Abbott,** and David M. Murray***
* Social Development Research Group, University of Washington School of Social Work
** Educational Psychology, University of Washington
*** Division of Epidemiology, College of Medicine and Public Health, Ohio State University
Correspondence regarding this article and request for reprints may be directed to J. David Hawkins, Ph.D., Social Development Research Group, University of Washington, 9725 3rd Ave. NE, Suite 401, Seattle, WA 98115. Phone: 206-543-7655; Fax: 206-543-4507
Recent advances in prevention science provide evidence that adolescent health and behavior problems can be prevented by high-quality prevention services. However, many communities continue to use prevention strategies that have not been shown to be effective. Studying processes for promoting the dissemination and high-quality implementation of prevention strategies found to be effective in controlled research trials has become an important focus for prevention science. The Communities That Care prevention operating system provides manuals, tools, training, and technical assistance to activate communities to use advances in prevention science to plan and implement community prevention services to reduce adolescent substance use, delinquency, and related health and behavior problems. This paper describes the rationale, aims, intervention, and design of the Community Youth Development Study, a randomized controlled community trial of the Communities That Care system, and investigates the baseline comparability of the 12 intervention and 12 control communities in the study. Results indicate baseline similarity of the intervention and control communities in levels of adolescent drug use and antisocial behavior prior to the Communities That Care intervention. Strengths and limitations of the study’s design are discussed.
Adolescent health and behavior problems continue to concern parents, the public, and policy makers. Substance abuse, violence, delinquency, and risky sexual behaviors damage lives. Treatment of these problems is costly (Harwood et al. 1999).
Before 1980, there was little evidence that adolescent health and behavior problems could be reduced through preventive efforts (Berleman 1980; Hansen 1992; Tobler 1986). Over the past 25 years, this situation has changed dramatically due to two advances in prevention science. First, longitudinal studies of development have identified risk and protective factors that predict these problems (Hawkins et al. 1992). Prevention science postulates that negative health outcomes can be prevented by reducing or eliminating risk factors and enhancing protective factors in individuals and their environments during the course of development (Coie et al. 1993; Mrazek et al. 2004). Second, preventive interventions designed to reduce risk factors and bolster protective factors have been tested in controlled studies and have been found to be effective in preventing substance abuse, delinquency, and violence (Mihalic et al. 2004; Weissberg et al. 2003; Welsh and Farrington 2006).
Interventions focused on preventing the initiation of substance use during early adolescence have shown prolonged effects in reducing the use and abuse of substances later in adolescence (Botvin et al. 1995; Chou et al. 1998; Spoth et al. 1999). These results suggest that preventing the initiation of substance use during early adolescence by addressing risk and protective factors salient during this developmental period is a viable prevention approach.
Yet, many schools and communities use prevention approaches with little or no evidence of effectiveness (Ennett et al. 2003; Gottfredson et al. 2002; Wandersman and Florin 2003). Further, even when schools and communities select a tested, effective prevention approach, they often fail to implement it with fidelity to the standards delineated by program designers (Gottfredson and Gottfredson 2002; Hallfors & Godette, 2002). The development and testing of strategies for disseminating effective preventive interventions and improving the quality of implementation has emerged as a priority for prevention research (Spoth et al. 2005; Weissberg et al. 2003).
Community coalitions have been suggested as mechanisms to build capacity to mount effective prevention initiatives in communities (Butterfoss et al. 1993; Wandersman 2003). Despite support for the development of coalitions to guide community prevention efforts, as illustrated by the federal Drug Free Communities Program, several studies have found coalitions to be ineffective in reducing the prevalence of adolescent health and behavior problems in communities (Hallfors et al. 2002; Klerman et al. 2005; Merzel and D’Afflitti 2003).
Hallfors et al. (2002) evaluated 12 coalitions funded under the Robert Wood Johnson Fighting Back Against Substance Abuse initiative. They found that none of the coalitions reached their desired outcome of reduced youth or adult substance use. The editors of a special issue of the Journal of Adolescent Health, reporting findings from the Center for Disease Control’s effort to use community coalitions to prevent teen pregnancy, described “serious reservations about a coalition-based approach to teen pregnancy prevention” (Klerman et al. 2005, p. S117). They expressed concern that the coalitions studied were “comprised primarily of social service providers and … unable to reach consensus on supporting programs of proven efficacy” (Klerman et al. 2005, p. S117).
Hallfors and her colleagues (2002) suggested that community coalitions could produce better outcomes if three criteria were met. The coalitions (a) should have clearly defined, focused, and measurable goals and outcomes, with corresponding high-quality data sources to facilitate monitoring; (b) should be encouraged to use evidence-based programs, with careful attention to monitoring of both the dose and quality of programs provided; (c) should evaluate the programs implemented through coalition efforts using outcome measures meaningful to the community. At this time, it remains to be seen whether community coalitions can improve the quality of community prevention services to an extent that community goals for reducing risk, enhancing protection, and reducing the prevalence of adolescent health and behavior problems can be achieved.
Communities That Care (CTC) is an operating system that mobilizes community stakeholders to collaborate on the development and implementation of a science-based community prevention system. CTC meets and extends Hallfors’ and colleagues’ suggestions for improving coalition outcomes by providing structure, processes, and tools designed to enable coalitions to use prevention science as a basis for community prevention services. CTC provides a structure for engaging community stakeholders, a process for establishing a shared community vision, tools for assessing levels of risk and protection in communities, and processes for prioritizing risk and protective factors and setting specific, measureable community goals. CTC guides the coalition to create a strategic community prevention plan designed to address the community’s profile of risk and protection with tested, effective programs and to implement the chosen programs with fidelity. CTC instructs the coalition to monitor program implementation and to periodically reevaluate community levels of risk and protection and outcomes, and to make adjustments in prevention programming if indicated by the data (Hawkins et al. 2002). CTC is installed in communities through a series of six training events delivered over the course of 6 to 12 months by certified CTC trainers. All CTC training materials are available on the internet at http://preventionplatform.samhsa.gov.
Communities That Care is guided theoretically by the Social Development Model (SDM) (Catalano and Hawkins 1996). The SDM posits that bonding to prosocial groups and individuals and clear standards for healthy behavior are protective factors that inhibit the development of problem behaviors. The SDM hypothesizes that bonding is created when people are provided opportunities to be involved in a social group like a coalition, family, or classroom, when they have the skills to participate in the social group, and when they are recognized for their contributions to the group.
This theoretical framework is applied in CTC in two ways. First, CTC encourages community stakeholders to adopt the SDM in their daily interactions with young people as a strategy for promoting healthy development. A goal in CTC communities is to ensure that all young people are provided developmentally appropriate opportunities, skills, and recognition, as well as healthy standards for behavior, by adults and organizations in the community.
Second, the social development model guides the community mobilization and training component of CTC itself. CTC seeks to create opportunities for all interested community stakeholders to participate in developing a shared vision for positive youth development based in prevention science. Through CTC trainings, diverse community representatives develop skills to work together effectively, thus increasing the likelihood that opportunities for interaction lead to rewarding experiences. The CTC process also suggests appropriate recognition activities to enhance the reinforcement of community board members for their participation in the process.
By increasing opportunities, skills, and recognition for community stakeholders to work together toward a shared vision, CTC seeks to develop social bonds among coalition members and a strong commitment to implementing effective preventive interventions with fidelity. Strong bonds and a commitment to science-based prevention are hypothesized to lead to greater collaboration among prevention service providers and community members, and high-quality implementation of the policies and programs initiated through CTC. When installed with fidelity, the tested preventive interventions should produce positive effects on risk, protection, and the prevalence of adolescent health and behavior problems in the community.
CTC’s model of community change incorporates theories of community competence (e.g., Eng and Parker 1990), public health promotion (e.g., Bracht and Kingsbury 1990; Butterfoss et al. 1993), and prevention science (e.g., Coie et al. 1993; Mrazek et al. 2004). As shown in Figure 1, it is hypothesized that CTC training and technical assistance will mobilize a prevention coalition comprised of diverse stakeholders and will increase coalition members’ shared commitment to using a science-based approach to prevention. The coalition also is expected to stimulate widespread adoption of science-based prevention among the community’s prevention service providers. This widespread adoption of a science-based approach is expected to increase collaboration on prevention efforts as stakeholders work to achieve shared goals of reducing prioritized risk factors and increasing prioritized protective factors. CTC also emphasizes the accountability of prevention services, including appropriate choice and implementation of tested, effective programs and policies with fidelity, and monitoring progress toward measurable objectives for community risk reduction and reduction of adolescent health and behavior problems.
Figure 1
Figure 1
Communities That Care Model of Community Change
This paper describes the aims, intervention, and design of a randomized controlled community trial of the Communities That Care system and investigates the baseline comparability of the 12 intervention and 12 control communities in the study in levels and trends in adolescent drug use and delinquency.
The Community Youth Development Study
The Community Youth Development Study (CYDS) is the first community-randomized trial of CTC. The study is a joint effort of the Social Development Research Group of the University of Washington, the Illinois Division of Community Health and Prevention, the Kansas Department of Prevention and Early Intervention, the state substance abuse agencies of Colorado, Maine, Oregon, Utah, and Washington, and 24 communities within these states. The project will examine the effects of CTC on: (a) communities’ adoption of a science-based approach to guide preventive action; (b) levels of collaboration among agencies, groups, and organizations in communities; (c) community levels of risk and protection; and d) the prevalence of drug use and delinquent behavior among adolescents in communities.
Intervention
CTC mobilizes community leaders and a community prevention coalition (called the “community prevention board” in CTC) to plan and implement a set of tested interventions to reduce elevated risk factors and promote protective factors in the community. Implementation of CTC is organized into five stages, each with its own series of “benchmarks” and “milestones” to help guide and monitor implementation progress (see Quinby et al. 2008 for more details ).
Phase 1 is a Community Readiness Assessment phase in which attitudinal and organizational characteristics of community members, leaders, and organizations thought to influence the mobilization process are assessed (National Institute on Drug Abuse 1997; Oetting et al. 1995), and important individuals and organizations necessary to initiate CTC are identified.
Phase 2 introduces the community to CTC through a training event that orients key community leaders (mayor, police chief, school superintendent; and business, faith, community, social service, and media leaders) to prevention science and the community activation processes of CTC. The training defines roles and responsibilities of the key leaders and those of the community prevention board. Key leaders are expected to hold the community prevention board and staff accountable for planning and carrying out CTC and to identify and secure resources necessary to implement preventive interventions planned through the CTC process.
Key leaders then identify and invite community members to constitute the community prevention board, or, alternatively, identify an existing coalition in the community to take on the CTC prevention board functions. Key leaders are instructed to ensure that the CTC board includes members who represent the diversity of the community and who can assist in the development of linkages to resources and organizations that will support the board’s prevention work. Prevention board members attend a 2-day orientation training.
In Phase 3, the CTC board completes assessments of levels of youth problem behaviors and risk and protective factors, as well as assessments of existing community resources. Board members participate in a 2-day training on how to utilize epidemiologic data on risk and protective factors, including how to interpret survey and archival measures of risk and protective factors. CTC boards prioritize two to five risk factors for preventive action. The profiles of risk and protection provide baseline data for subsequent assessments of the community’s progress in changing levels and trends in the factors targeted by the board’s prevention plan.
Following the prioritization of risk and protective factors, CTC board members attend a 1-day resource assessment training. The goal of the resource assessment is to identify gaps in existing policies, programs, and services that address the community’s prioritized factors.
In Phase 4, the CTC board develops its community action plan. Community board members attend a 2-day Community Plan Training that reviews tested policies, programs, and actions that have demonstrated effectiveness in adequately controlled trials. The board defines measurable objectives with respect to reduction of the prioritized risk factors, enhancement of protective factors, and reductions in substance use and delinquency, and develops a plan to fill gaps in existing services through implementation of tested, effective policies and programs.
While CTC typically encompasses preventive actions from the prenatal period to young adulthood, the intervention communities in CYDS were asked to focus their plans on youths in Grades 5 – 9 and their families. It was hypothesized that, if widely implemented, tested and effective preventive interventions delivered during this time of transition from childhood to adolescence would produce measurable community-wide effects on the prevalence of drug use and delinquent behaviors during the 5-year study period of CYDS. CTC boards select policies and programs from a menu of tested preventive interventions for elementary and middle school students (shown in Table 1), a subset of those included in “Communities That Care Prevention Strategies: A Research Guide to What Works” (Hawkins and Catalano 2004). The interventions in the menu for CYDS met the following criteria. Each has shown positive effects in reducing risk factors and substance use and/or antisocial behavior in an adequately controlled experimental or quasi-experimental study. Training, technical assistance, and manuals are available to guide the implementation of the policy or program. The intervention has shown positive effects on youths in Grades 5 through 9.
Table 1
Table 1
Menu of Tested and Effective Preventive Interventions Offered to CTC Communities in the Community Youth Development Study
Communities’ action plans describe the tested prevention interventions selected and include work plans to implement these new interventions, monitor and provide feedback on implementation quality, and assess progress towards specified process and outcome goals.
Prior to funding the CTC boards’ plans, the plans are reviewed each year by the CYDS Review Panel (comprised of project staff and state substance abuse prevention officials from each of the seven collaborating states) and are revised by the community boards in light of the Review Panel’s input. This dialogue between the community prevention boards, state prevention officials, and researchers is designed to ensure feasible plans that address each community’s profile of risks and protection with appropriate tested, effective interventions. Funding for proposed interventions up to $75,000 annually is determined by the Review Panel, and is awarded and monitored by CYDS project staff.
In Phase 5, the chosen preventive interventions are implemented and implementation quality is monitored by the CTC community prevention board. Beginning in Year 2 and continuing into Year 5, those selected by the community prevention boards to implement prevention programs receive training. Developers of chosen prevention programs and CYDS project staff provide training and technical assistance to ensure high-quality implementation and monitoring of progress toward implementation and outcome goals.
At the outset of Phase 5, CTC boards receive the Community Plan Implementation Training to develop the skills and plans necessary to implement and monitor their community’s action plan and sustain the CTC effort. Monitoring of implementation is accomplished through program-specific implementation checklists completed by program providers, observation checklists completed by community board members and agency supervisors who observe 10% to 15% of program sessions, and participant pre- and posttests (see Fagan et al. in press for more details). During Phase 5, the board also engages local media to educate community members about risk and protective factors for adolescent problem behaviors, generate public support for the new preventive interventions initiated, and motivate community members to take part in the new preventive interventions.
The CTC system is expected to produce community-level changes in prevention service system characteristics, including increased collaboration among service providers and increased use of tested, effective preventive interventions that address risk and protective factors prioritized by the community. These changes in prevention services are expected to produce changes in the risk and protective factors targeted by the preventive interventions. These changes in risk and protective factors are expected, in turn, to produce changes in adolescent drug use and delinquent behaviors (See Figure 1).
Study Design
The Community Youth Development Study is built on the Diffusion Project, a prior 5-year descriptive study of prevention activities and youth outcomes in 41 communities in the seven collaborating states. At the outset of the Diffusion Project, the drug abuse prevention agencies of the seven states identified 20 communities they thought were trying to implement risk- and protection-focused prevention services. These 20 communities were then matched, within each state, on population size, racial and ethnic diversity, economic indicators, and crime rates to comparison communities that were not thought to be using a risk and protection-focused approach, and the 20 community pairs were recruited to participate in the study. In one instance, two comparison communities were identified, resulting in a total of 41 communities.
In spite of states’ initial assessments of these communities, during the 5 years of the Diffusion Project, neither community in 13 of the 20 pairs of communities advanced in the use of science-based prevention to the point of selecting and using tested, effective preventive interventions to address prioritized community risks (Arthur et al. 2005). These 13 pairs of communities were deemed eligible for inclusion in the CYDS study. Twelve of these 13 pairs of matched communities (24 communities total) were successfully recruited to participate in CYDS and comprise the study sample. Recruitment required securing letters from the superintendent of schools, the mayor or city manager, and the lead law enforcement officer, agreeing to all data collection activities required of the project. One member in each pair of eligible communities in a state was randomized to the intervention or control condition by a coin toss prior to recruitment into CYDS. The earlier matching of the intervention and control communities with regard to size, poverty, diversity, and crime indices increased the likelihood of baseline comparability between intervention and control communities in the CYDS.
The study communities are incorporated towns with an average population of 14,646 (range = 1,578 to 40,787) according to the 2000 census, and clear community names and boundaries. They are not suburbs of larger cities. These geographically distinct communities provide a suitable environment for conducting trials of community interventions, both because of the relative absence of external confounding influences that can threaten internal validity, and because of the reduced costs and complexity of assessment of relevant variables in smaller communities (Biglan 1995). On average, 89% of the students completing the CTC Youth Survey in these communities in 2002 were European American (range = 64% to 98%), 3% were African American (range = 0% to 21%), 10% were of Hispanic origin (range = 1% to 65%), and 37% were eligible for free or reduced-price lunch (range = 21% to 66%). The enrolled student population in a single grade in these communities ranged from 40 to 485, with only two communities exceeding 400 students per grade. Including communities already studied for 5 years in the Diffusion Project provides multiple baseline data points for this study.
The design of the Community Youth Development Study is shown in Figure 2. Communities That Care implementation began in the summer of 2003. Intervention communities were trained to use the baseline data collected by the Diffusion Project in 1998, 2000, and 2002 to prioritize specific risk and protective factors for attention. By April 2004, intervention communities had selected preventive interventions to address their prioritized needs and had created strategic community plans to implement these interventions.
Figure 2
Figure 2
Community Youth Development Study Design
Implementation and impact of CTC on communities’ prevention service systems are assessed using telephone interviews of community key informants and Community Resource Documentation surveys of prevention service providers in all communities as described in the following sections. Effects of the CTC intervention on risk factors, protective factors, substance use, and other problem behaviors are assessed using repeated cross-sectional and longitudinal student surveys in both intervention and control communities.
Measurement
Community Key Informant Interviews
To assess the degree to which all study communities have adopted a science-based approach to the prevention of adolescent substance abuse and delinquency, telephone interviews were conducted with key community leaders in all communities twice during the Diffusion Project in 1998 and 2001, before the CTC intervention was initiated, and are conducted again in the CYDS in 2004 and 2007. The Community Key Informant Interview (CKI in Figure 2) assesses the degree to which 10 positional community leaders and the 5 prevention leaders most frequently nominated by these positional leaders as knowledgeable about prevention report the use of epidemiological data to guide prevention planning and resource allocation decisions; the selection and use of tested, effective programs; and ongoing monitoring of implementation and changes in risk, protection, and youth outcomes. Evaluation of CTC’s impact on community prevention systems includes comparisons between the intervention and control communities on measures of adoption of a science-based prevention approach (see Arthur et al. 2005; Brown et al. 2007) and prevention collaboration (see Brown et al. 2008; Brown et al. 2007) from the CKI interview. Major changes or events in the community that might affect prevention efforts (e.g., changes in community policies, layoffs by a major employer) also are documented in the CKI.
Community Resource Documentation
The Community Resource Documentation process (CRD in Figure 2) is used to measure the type, number, and scope of prevention activities consistent with tested and effective interventions for youths in Grades 5 through 9 and their families in the 24 study communities. The CRD data collection process is a combination of telephone interviews and mail surveys for prevention service providers and program directors, school principals, and civic leaders conducted in 2001, 2004, and 2007.
The CRD process provides data on the numbers of programs and policies of each of seven types that are consistent with tested and effective preventive interventions, as well as measures of the scope (number and ages of youths served) of each type of program in the community, where applicable. The seven types of programs and policies included are school prevention curricula, school-wide organizational changes, parenting programs, tutoring programs, mentoring programs, school policies, and community policies.
Community Board Interviews
To monitor CTC board functioning in the intervention communities, 10 CTC board members in each community are surveyed annually by phone (see Figure 2). The CTC Board Survey assesses dimensions associated with developing and maintaining an effective community coalition (Gottlieb et al. 1993; Kumpfer et al. 1993; Prestby et al. 1990). This survey instrument incorporates constructs of community board functioning shown to predict higher implementation in previous studies evaluating the effectiveness of CTC boards (e.g., Feinberg et al. 2004; Feinberg et al. 2002; Greenberg et al. 1999) and other community prevention coalitions (e.g., Kegler et al. 1998). In addition, monthly reports from each community coordinator document community board, task force, and staff activities, and include meeting minutes and attendance records. Annual community action plans developed by the CTC board in each intervention community document assessment results, plans for implementing tested, effective interventions during the coming year, and evaluation plans.
Communities That Care Milestones and Benchmarks
Additional measures of CTC implementation are obtained from ratings of the CTC Milestones and Benchmarks (see Quinby et al, 2008). The CTC training materials describe “milestones” and “benchmarks” that are to be achieved during the five phases of CTC system implementation. The milestones are goals to be met by communities, and the benchmarks are the actions that community members take or conditions that must be present to achieve those goals. In CYDS, community CTC coordinators, CTC trainers, and University of Washington intervention specialists periodically rate intervention communities’ progress on the CTC milestones and benchmarks to assess implementation of the CTC system in intervention communities.
Intervention implementation measures
In the intervention communities, monitoring of preventive intervention implementation is an ongoing process that seeks to ensure that the interventions chosen by the community prevention boards achieve the fidelity, scope, and intensity necessary to achieve the community’s risk reduction and protective factor enhancement objectives. Implementation measures assist the intervention communities to monitor their progress in implementing the tested, effective interventions in their community prevention plans. The feedback provided by these measures is used to support continuous quality improvement of prevention programs during Phase 5 (see Fagan et al. in press).
Cross-sectional student surveys
In the spring of every other year since 1998 and continuing through 2008, all assenting 6th-, 8th-, 10th-, and 12th-grade public school students within the intervention and control communities complete the CTC Youth Survey (CTCYS). The CTCYS provides reliable and valid measures of an empirically derived set of community, family, school, peer, and individual risk and protective factors and drug use and antisocial behavior outcomes (Arthur et al. 2002; Glaser et al. 2005). The survey is group administered during one classroom period and takes approximately 45 minutes to complete. Parents and students have the opportunity to refuse participation in the study. Participants’ responses are anonymous. The design allows for comparisons across six cohorts for each grade over three baseline waves collected in the Diffusion Project and three waves of CTCYS data collected in CYDS.1
The effects of the elementary-grade interventions implemented in CTC communities on risk and protective factors, drug use, and delinquent behavior are assessed by comparing cross-sectional sixth-grade samples over time in CTC and control communities. The cumulative effects of the elementary and middle school interventions implemented in CTC communities are assessed by comparing eighth-grade surveys across CTC and control communities over time. The repeated cross-sectional 10h-grade surveys seek to detect changes in risk, protection, and drug use and delinquency that result from the interventions provided during the middle school grades. This paper describes the baseline equivalence of the intervention and control communities in the CYDS with regard to trends and levels in adolescent substance use and antisocial behavior reported by 6th-, 8th-, and 10th-grade students on the CTC Youth Survey from 1998 to 2002.
Longitudinal panel
A longitudinal panel of students in each community enrolled in the fifth grade in the first year of CYDS also has been constituted after obtaining written parental consent (YDS in Figure 2). The initial sample for the longitudinal panel was the population of fifth-grade public school students in the 24 participating communities in the spring of 2004.
Students in the panel who remained in the study communities through the 2004–2005 academic year were resurveyed in the spring of 2005, as were students whose parents consented to the survey for the first time in Grade 6. Students recruited into the panel sample who participated in either or both of the first two waves of data collection are tracked and resurveyed annually for the remainder of the study, even if they move away from project communities. Data obtained from the panel sample provide a second test of intervention effects (Salonen et al. 1986; Wagner et al. 1991). A full description of the panel, its baseline characteristics and analysis plans, is in preparation. The panel is not the focus of this paper.
Baseline Comparability of Intervention and Control Communities
Findings reported elsewhere (Brown et al. 2007) have documented that intervention and control communities had comparable levels of adoption of a science-based prevention approach and prevention collaboration in 2001. To evaluate the success of random assignment in creating baseline equivalence in student outcomes across the intervention and control communities, trends and levels of demographic characteristics and adolescent drug use and antisocial behavior reported on the CTC Youth Survey from 1998 to 2002 were assessed. Responses from all eligible 6th-, 8th-, and 10th-grade students in the 24 CYDS communities were used because these are the three grade levels at which effects of the CTC intervention are hypothesized during the CYDS study period. The response rates for the CTCYS samples were 79% in 1998, 72% in 2000, and 82% in 2002. A total of 2.3% of surveys were eliminated due to dishonest or invalid responding.2 Excluding dishonest respondents, the number of students providing data was 14,293 in 1998, 12,992 in 2000, and 14,910 in 2002.
Outcome measures
Substance use variables examined were lifetime alcohol, cigarette, and marijuana use; 30-day alcohol, cigarette, and marijuana use; and binge drinking. Antisocial behavior variables studied were engaging in “attacking someone with the idea of seriously hurting them,” being “drunk or high at school,” being “suspended from school,” and “selling illegal drugs” during the past year. All outcome variables were dichotomized to reflect any use or engagement in the behavior versus no use or engagement in the behavior.
Covariates
Age, sex, race, family history of substance use, frequency of attendance at religious services, and percent of students eligible for free and reduced lunch at the school level were used as covariates in the adjusted model. All items except the last were responses to the CTC Youth Survey. Percent of students eligible for free and reduced lunch at the school level came from data collected by the National Center for Education Statistics (NCES).
Analyses
Forty imputed datasets using SAS PROCMI were generated using all items on the survey to address missing data. For the dichotomized covariates and outcomes, a generalized linear mixed model using the canonical logit link and binomial error function was tested with random intercepts and linear slopes at the community level (Murray 1998). The GLIMMIX macro of the SAS procedure, MIXED, was used to run these models. For the interval-level covariates that were examined, a general linear mixed model with random intercepts and linear slopes at the community level was run in SAS PROC MIXED (Murray 1998). The intercept for all models was positioned at 2002 to reflect the most proximate time point prior to intervention. The SAS procedure MIANALYZE was used to combine the results across the 40 imputations.
Two models were examined: (a) an unadjusted model with fixed effects for condition, year, and condition X year; and (b) an adjusted model with fixed effects for condition, year, and condition X year, controlling for age, sex, dummy-coded racial categories, family history of substance use, frequency of attendance at religious services, and percent of students eligible for free or reduced lunch at the school level. These two models were run to examine slopes and 2002 intercepts for each of the three grade levels and for the 11 substance use and antisocial variables, resulting in 33 tests for each pair of tests (unadjusted and adjusted). Analyses also were conducted to examine differences in levels and trends between conditions on each of the covariates described above.
Model estimates were used to generate predicted intercepts and slopes for intervention and control conditions that were transformed back into their original scale. Differences between predicted intervention and control rates were calculated and lower and upper confidence limits around the differences were derived using standard errors based upon model t-values and degrees of freedom.
Results for the unadjusted and adjusted models were very similar. Given these findings, results from the unadjusted models are presented because they provide estimates of the observed prevalences of drug use and antisocial behaviors in the participating communities.
There were no significant differences between conditions in 2002 in any of the demographic variables examined at Grades 6, 8, and 10. The intervention and control students did not differ significantly with regard to average age; gender; proportions of Whites, African Americans, Native Americans, Latinos, or those of other ethnic or racial groups; proportion eligible for free lunch; reported religious attendance; or proportion reporting a family history of substance abuse problems. Of the trends from 1998 to 2002 examined for the 10 demographic variables, only one, the slope for eighth-grade family history of substance use, differed between conditions (t15=− 2.11, p=.047), with control (33%) and intervention (34%) communities showing similar prevalence levels in 1998, but with family history of substance abuse increasing more in control than in intervention communities over time (in 2002, 37% and 34%, respectively).
As illustrated in Table 2, there were no significant differences at any grade level between the intervention and control conditions in 2002 in the prevalence of use of any of the seven substances examined. However, slopes differed significantly between conditions for 10th-grade binge drinking (t20=2.30, p=.033), with prevalence higher in control (30%) than intervention (21%) communities in 1998 and these differences diminishing to similar prevalence levels in 2002 (24% and 23%, respectively). In addition, differences in slopes between conditions were significant for 10th-grade 30-day marijuana use (t20=2.18, p=.043). Prevalence was higher in control (23%) than in intervention (17%) communities in 1998, but these differences diminished to similar prevalence levels in 2002 (21% and 21%, respectively). No other significant differences in slopes were found across the three grades and seven drugs examined.
Table 2
Table 2
Prevalence of Substance Use and Antisocial Behaviors by Grade across Conditions in 2002
The prevalence of four antisocial behaviors were compared across communities for Grades 6, 8, and 10 in 2002. As illustrated in Table 2, only one of these comparisons revealed a significant difference. In 2002, 20% of eighth-grade students in control communities reported having attacked someone with the intention of hurting them, compared with 16% of eighth-grade students in intervention communities (t20=−2.26, p=.035). In comparing trends in these four problem behaviors over time from 1998 to 2002 across intervention and control communities, only one behavior, 10th grade students’ reports of being drunk or high at school, showed a significantly different slope across conditions. (t20=2.20, p=.040). The prevalence of this behavior among 10th grade students was higher in control (27%) than intervention (20%) communities in 1998, but by 2002 these differences had diminished to similar levels (24% and 23%, respectively).
Overall, a total of 66 tests for differences in baseline rates and trends were conducted. With this many tests, one would expect to find three tests significant by chance alone. Only one difference in the rates of drug use and antisocial behavior between intervention and control communities was significant at the 2002 baseline. Three differences in baseline trends from 1998 to 2002 between intervention and control communities were significant. These results indicate comparable levels and trends in adolescent drug use and antisocial behavior between intervention and control communities prior to initiating the Communities That Care intervention.
Analysis Plan for the CTC Youth Survey Data
Because exposure to preventive interventions will differ across grade levels, analysis of the CTC Youth Survey data will be conducted separately for each of the 6th-, 8th-, and 10th-grade cohorts. We have examined the power of various approaches to analyzing intervention effects on community levels, trends, and prevalence of youth substance use and antisocial behavior as measured by the CTCYS. Using CTCYS data from the 41 communities in the Diffusion Project, Murray et al. (2006) compared pre-post mixed-model ANCOVA models with random coefficient models, both with one-stage and two-stage versions, and found that the two-stage, pre-post mixed-model ANCOVA model, averaged within grade level across baseline survey waves, exhibited the greatest statistical power to detect intervention effects among the dichotomous drug use outcomes. Methods to model non-normally or semi-continuously distributed outcomes (e.g., frequency and quantity measures of adolescent drug use), such as two-part growth modeling (Brown et al. 2005; Olsen and Schafer 2001) and beta regression techniques (Smithson and Verkuilen 2006), will be employed as necessary. These strategies, as well as strategies to control for multiple comparisons across the alcohol, tobacco, marijuana, and antisocial behaivor outcomes measured in the CTCYS, such as global tests (Feng and Thompson 2002) and maximum individualized change score techniques (Boothroyd et al. 2004), are being assessed for their appropriateness for the CTCYS data.
Communities That Care combines a community coalition approach to prevention with tested, effective preventive interventions. The current study is the first randomized controlled trial of CTC. This paper describes the design of the study and presents data on the comparability of the intervention and control communities with regard to baseline levels and trends in alcohol, tobacco, and marijuana use and four antisocial behaviors among 6th-, 8th-, and 10th-grade public school students surveyed in 1998, 2000, and 2002. Analyses show few differences between the two study conditions across three grades over three waves of baseline data. These results, combined with findings showing comparable levels of adoption of a science-based approach to prevention and collaboration on prevention issues in 2001 (Brown et al. 2007), indicate baseline comparability on primary outcome variables between intervention and control communities in the Community Youth Development Study.
Conducting the proposed research with 12 matched pairs of communities that participated in the Diffusion Project provides advantages. These include: (a) three waves of baseline data measuring trends from student surveys conducted in each community prior to the start of the study; (b) two waves of baseline data on each community’s prevention planning approach prior to the start of the study; and (c) two waves of baseline data on the number, types, and scope of evidence-based prevention strategies within each community collected prior to the start of the study. The study design also combines repeated cross-sectional surveys of students in different grade levels with a longitudinal panel of students followed and surveyed annually for the 5 years of the CYDS. Using the two survey sampling methods, which provide complementary approaches to assessing intervention impact, addresses different threats to the internal validity of study findings (Murray 1998; Salonen et al. 1986; Wagner et al. 1991).
The study has limitations. This is an efficacy trial in which training, technical assistance, and resources for a community coordinator as well as resources to implement tested, effective interventions are provided to intervention communities. Such resources are not always available. Further, the communities studied are free-standing towns of 50,000 or fewer residents. The present trial does not include urban or suburban populations, and study findings may not generalize to larger communities. However, results from the present trial should provide the foundation needed to assess the utility and feasibility of testing CTC in larger and more complex settings. Future trials may also seek to assess the utility of the CTC system when focused on different target ages. For example, a trial of CTC focused later in adolescence might well include additional options on the menu of tested and effective preventive interventions than those included in this trial which focuses on youths aged 10 to 14. For example, while not tested for effects on children this young, policy and environmental prevention strategies developed by Wagenaar et al. (2000) and Holder et al. (2000) have shown positive effects in reducing alcohol misuse among older adolescents and young adults..
Recently, Smolkowski et al. (2006) questioned whether there is sufficient variation in drug use or antisocial behavior attributable to communities to warrant the use of student survey and archival data to help identify community priorities for prevention efforts. They expressed particular concern about the use of a single survey of a single cohort to characterize a community’s profile. We agree that communities account for relatively little of the variation in measures of drug use and antisocial behavior (Hawkins et al. 2004). We also agree that cohorts themselves account for some variation, albeit minor (see Murray et al. 2006). In Communities That Care, multiple cohorts (Grades 6, 8, 10, and 12) are measured in each community, and in the Community Youth Development Study, these cohorts were measured three times (in 1998, 2000, and 2002) prior to the initiation of the CTC system. In 2003, community boards in the CTC condition reviewed levels and trends in data from 6th, 8th, and 10th grade students over three survey waves in choosing foci for their preventive interventions.
In 2006, Communities That Care was placed in the public domain. The CTC training materials have been made available for downloading from the Center for Substance Abuse Prevention’s Prevention Platform (http://preventionplatform.samhsa.gov). This development increases the importance of this efficacy trial of CTC. As tested in the current study, CTC involves training by certified CTC trainers as well as technical assistance from intervention specialists. It is likely that variability across communities in the fidelity of implementation of the CTC system and of the programs implemented through CTC could affect outcomes. The present trial monitors and seeks to ensure high-fidelity implementation of both the CTC system and resulting prevention programs. If the CTC intervention is found to be efficacious in this study, the results would suggest that the inclusion of an adequate system of training, technical assistance, and implementation monitoring be made part of efforts to disseminate the CTC system.
Acknowledgments
This work was supported by a research grant from the National Institute on Drug Abuse (R01 DA015183-01A1) with co-funding from the National Cancer Institute, the National Institute on Child Health and Development, the National Institute on Mental Health, and the Center for Substance Abuse Prevention. The authors are grateful for the contributions of the communities participating in the Community Youth Development Study and the collaboration of the state drug abuse prevention agencies of Colorado, Illinois, Kansas, Maine, Oregon, Utah, and Washington.
Footnotes
1Twelfth-grade data currently being collected will provide multiple baselines for possible future investigations of longer term outcomes associated with CTC intervention focused on Grades 5 through 9.
2Respondents were considered dishonest if they reported responding dishonestly to the survey, endorsed frequency of monthly substance use across four substances that exceeded 120 times, reported the same frequency of high use across eight substances, reported the same frequency of high engagement across all antisocial behaviors, or answered certain items in an inconsistent or unreasonable fashion (e.g., monthly use of a fictitious drug more than once or twice).
  • Arthur MW, Glaser RR, Hawkins JD. Steps towards community-level resilience: Community adoption of science-based prevention programming. In: Peters RD, Leadbeater B, McMahon RJ, editors. Resilience in children, families, and communities: Linking context to practice and policy. New York: Kluwer Academic/Plenum Publishers; 2005. pp. 177–194.
  • Arthur MW, Hawkins JD, Pollard JA, Catalano RF, Baglioni AJ., Jr Measuring risk and protective factors for substance use, delinquency, and other adolescent problem behaviors: The Communities That Care Youth Survey. Evaluation Review. 2002;26:575–601. [PubMed]
  • Berleman WC. Reports of the National Juvenile Justice Assessment Centers. Juvenile delinquency prevention experiments: A review and analysis. Washington, DC: U.S. Department of Justice, Law Enforcement Assistance Administration, Office of Juvenile Justice and Delinquency Prevention; 1980.
  • Biglan A. Translating what we know about the context of antisocial behavior into a lower prevalence of such behavior. Journal of Applied Behavior Analysis. 1995;28:479–492. [PMC free article] [PubMed]
  • Boothroyd RA, Banks SM, Evans ME, Greenbaum PE, Brown E. Untangling the web: An approach to analyzing the impacts of individually tailored, multicomponent treatment interventions. Mental Health Services Research. 2004;6:143–153. [PubMed]
  • Botvin GJ, Baker E, Dusenbury L, Botvin EM, Diaz T. Long-term follow-up results of a randomized drug abuse prevention trial in a white middle-class population. Journal of the American Medical Association. 1995;273:1106–1112. [PubMed]
  • Bracht N, Kingsbury L. Community organization principles in health promotion: A five-stage model. In: Bracht N, editor. Health promotion at the community level. Thousand Oaks, CA: Sage; 1990. pp. 66–88.
  • Brown EC, Catalano RF, Fleming CB, Haggerty KP, Abbott RD. Adolescent substance use outcomes in the Raising Healthy Children Project: A two-part latent growth curve analysis. Journal of Consulting and Clinical Psychology. 2005;73:699–710. [PubMed]
  • Brown EC, Hawkins JD, Arthur MW, Abbott RD, Van Horn ML. Multilevel analysis of a measure of community prevention collaboration. American Journal of Community Psychology. 2008;41:115–126. [PubMed]
  • Brown EC, Hawkins JD, Arthur MW, Briney JS, Abbott RD. Effects of Communities That Care on prevention services systems: Outcomes from the Community Youth Development Study at 1.5 years. Prevention Science. 2007;8:180–191. [PubMed]
  • Butterfoss FD, Goodman RM, Wandersman A. Community coalitions for prevention and health promotion. Health Education Research. 1993;8:315–330. [PubMed]
  • Catalano RF, Hawkins JD. The social development model: A theory of antisocial behavior. In: Hawkins JD, editor. Delinquency and crime: Current theories. New York: Cambridge University Press; 1996. pp. 149–197.
  • Chou CP, Montgomery S, Pentz MA, Rohrbach LA, Johnson CA, Flay BR, et al. Effects of a community-based prevention program in decreasing drug use in high-risk adolescents. American Journal of Public Health. 1998;88:944–948. [PubMed]
  • Coie JD, Watt NF, West SG, Hawkins JD, Asarnow JR, Markman HJ, et al. The science of prevention. A conceptual framework and some directions for a national research program. American Psychologist. 1993;48:1013–1022. [PubMed]
  • Eng E, Parker E. Community competence and health: Definitional, conceptual, and measurement issues; Presented at the Annual Meeting of the American Public Health Association, Health Promotion and Health Education Section; New York. 1990.
  • Ennett ST, Ringwalt CL, Thorne J, Rohrbach LA, Vincus A, Simons-Rudolph A, et al. A comparison of current practice in school-based substance use prevention programs with meta-analysis findings. Prevention Science. 2003;4:1–14. [PubMed]
  • Fagan AA, Hanson K, Hawkins JD, Arthur MW. Bridging science to practice: Achieving prevention program implementation fidelity in the Community Youth Development Study. American Journal of Community Psychology in press. [PubMed]
  • Feinberg ME, Greenberg MT, Osgood D. Readiness, functioning, and perceived effectiveness in community prevention coalitions: A study of Communities That Care. American Journal of Community Psychology. 2004;33:163–176. [PubMed]
  • Feinberg ME, Greenberg MT, Osgood DW, Anderson A, Babinski L. The effects of training community leaders in prevention science: Communities That Care in Pennsylvania. Evaluation & Program Planning. 2002;25:245–259.
  • Feng Z, Thompson B. Some design issues in a community intervention trial. Controlled Clinical Trials. 2002;23:431–449. [PubMed]
  • Glaser RR, Van Horn ML, Arthur MW, Hawkins JD, Catalano RF. Measurement properties of the Communities That Care Youth Survey across demographic groups. Journal of Quantitative Criminology. 2005;21:73–102.
  • Gottfredson DC, Gottfredson GD. Quality of school-based prevention programs: Results from a national survey. Journal of Research in Crime and Delinquency. 2002;39:3–35.
  • Gottfredson DC, Wilson DB, Najaka SS. School-based crime prevention. In: Sherman LW, Farrington DP, Welsh BC, MacKenzie DL, editors. Evidence based crime prevention. United Kingdom: Routledge; 2002. pp. 56–164.
  • Gottlieb NHGBS, Gingiss PL. Correlates of coalition effectiveness: The Smoke Free Class of 2000 program. Health Education Research. 1993;8:375–384. [PubMed]
  • Greenberg MT, Osgood DW, Babinski L, Anderson A. Developing community readiness for prevention: Initial evaluation of the Pennsylvania Communities That Care Initiative; Presented at the Society for Prevention Research; New Orleans, LA. 1999. Jun,
  • Hallfors D, Godette D. Will the ‘Principles of Effectiveness’ improve prevention practice? Early findings from a diffusion study. Health Education Research. 2002;17:461–470. [PubMed]
  • Hallfors DD, Cho H, Livert D, Kadushin C. Fighting back against substance abuse: Are community coalitions winning? American Journal of Preventive Medicine. 2002;23:237–245. [PubMed]
  • Hansen WB. School-based substance abuse prevention: A review of the state of the art in curriculum, 1980–1990. Health Education Research. 1992;7:403–430. [PubMed]
  • Harwood HJ, Fountain D, Fountain G. Economic cost of alcohol and drug abuse in the United States, 1992: A report. Addiction. 1999;94:631–635. [PubMed]
  • Hawkins JD, Catalano RF. Communities That Care: Prevention strategies guide. South Deerfield MA: Channing Bete; 2004.
  • Hawkins JD, Catalano RF, Arthur MW. Promoting science-based prevention in communities. Addictive Behaviors. 2002;27:951–976. [PubMed]
  • Hawkins JD, Catalano RF, Miller JY. Risk and protective factors for alcohol and other drug problems in adolescence and early adulthood: Implications for substance-abuse prevention. Psychological Bulletin. 1992;112:64–105. [PubMed]
  • Hawkins JD, Van Horn ML, Arthur MW. Community variation in risk and protective factors and substance use outcomes. Prevention Science. 2004;5:213–220. [PubMed]
  • Holder HD, Gruenewald PJ, Ponicki WR, Treno AJ, Grube JW, Saltz RF, et al. Effect of community-based interventions on high-risk drinking and alcohol-related injuries. Journal of the American Medical Association. 2000;284:2341–2347. [PubMed]
  • Kegler MC, Steckler A, McLeroy K, Malek SH. Factors that contribute to effective community health promotion coalitions: A study of 10 Project ASSIST coalitions in North Carolina. Health Education and Behavior. 1998;25:338–353. [PubMed]
  • Klerman LV, Santelli JS, Klein JD. Editors’ commentary: So what have we learned? The Editors’ comments on the coalition approach to teen pregnancy. Journal of Adolescent Health. 2005;37:S115–S118. [PubMed]
  • Kumpfer KL, Shur GH, Ross JG, Bunnell KK, Librett JJ. Measurements in prevention: A manual on selecting and using instruments to evaluate prevention programs. CSAP Technical Report #8. Rockville, MD: U.S. Department of Health and Human Services; 1993.
  • Merzel C, D’Afflitti J. Reconsidering community-based health promotion: Promise, performance, and potential. American Journal of Public Health. 2003;93:557–574. [PubMed]
  • Mihalic S, Fagan A, Irwin K, Ballard D, Elliott D. Blueprints for Violence Prevention (NCJ 204274) Washington, DC: Office of Juvenile and Delinquency Prevention; 2004.
  • Mrazek PJ, Biglan A, Hawkins JD. Community-monitoring systems: Tracking and improving the well-being of America’s children and adolescents. Society for Prevention Research. 2004. Retrieved January 24, 2008, from http://www.preventionresearch.org/CMSbook.pdf.
  • Murray DM. Monographs in Epidemiology and Biostatistics. Vol. 27. New York: Oxford University Press; 1998. Design and analysis of group-randomized trials.
  • Murray DM, Van Horn ML, Hawkins JD, Arthur MW. Analysis strategies for a community trial to reduce adolescent ATOD use: A comparison of random coefficient and ANOVA/ANCOVA models. Contemporary Clinical Trials. 2006;27:188–206. [PubMed]
  • National Institute on Drug Abuse. Community readiness for drug abuse prevention: Issues, tips and tools. Rockville, MD: U.S. Department of Health and Human Services, National Institutes of Health; 1997.
  • Oetting ER, Donnermayer JF, Plested BA, Edwards RW, Kelly K, Beauvais F. Assessing community readiness for prevention. The International Journal of the Addictions. 1995;30:659–683. [PubMed]
  • Olsen MK, Schafer JL. A two-part random-effects model for semicontinuous longitudinal data. Journal of the American Statistical Association. 2001;96:730–745.
  • Prestby JE, Wandersman A, Florin P, Rich RC, Chavis D. Benefits, costs, incentive management and participation in voluntary organizations: A means to understanding and promoting empowerment. American Journal of Community Psychology. 1990;18:117–149.
  • Quinby RK, Fagan AA, Hanson K, Brooke-Weiss B, Arthur MW, Hawkins JD. Installing the Communities That Care prevention system: Implementation progress and fidelity in a randomized controlled trial. Journal of Community Psychology. 2008;36:313–332.
  • Salonen JT, Kottke TE, Jacobs DR, Hannan PJ. Analysis of community-based cardiovascular disease prevention studies--evaluation issues in the North Karelia Project and the Minnesota Heart Health Program. International Journal of Epidemiology. 1986;15:176–182. [PubMed]
  • Smithson M, Verkuilen J. A better lemon squeezer? Maximum-likelihood regression with beta-distributed dependent variables. Psychological Methods. 2006;11:54–71. [PubMed]
  • Smolkowski K, Biglan A, Dent C, Seeley J. The multilevel structure of four adolescent problems. Prevention Science. 2006;7:239–256. [PubMed]
  • Spoth R, Reyes ML, Redmond C, Shin C. Assessing a public health approach to delay onset and progression of adolescent substance use: Latent transition and loglinear analyses of longitudinal family preventive intervention outcomes. Journal of Consulting and Clinical Psychology. 1999;67:619–630. [PubMed]
  • Spoth RL, Greenberg MT. Toward a comprehensive strategy for effective practitioner-scientist partnerships and larger-scale community health and well-being. American Journal of Community Psychology. 2005;35:107–126. [PMC free article] [PubMed]
  • Tobler NS. Meta-analysis of 143 adolescent drug prevention programs: Quantitative outcome results of program participants compared to a control or comparison group. Journal of Drug Issues. 1986;16:537–567.
  • Wagenaar AC, Murray DM, Gehan JP, Wolfson M, Toomey TL, Perry CL, et al. Communities mobilizing for change on alcohol: Outcomes from a randomized community trial. Journal of Studies on Alcohol. 2000;61:85–94. [PubMed]
  • Wagner EH, Koespell TD, Anderman C, Cheadle A, Larry SG, Psaty BM, et al. The evaluation of the Henry J. Kaiser family foundation’s community health promotion grant program design. Journal of Clinical Epidemiology. 1991;44:685–699. [PubMed]
  • Wandersman A. Community science: Bridging the gap between science and practice with community-centered models. American Journal of Community Psychology. 2003;31:227–242. [PubMed]
  • Wandersman A, Florin P. Community interventions and effective prevention. American Psychologist. 2003;58:441–448. [PubMed]
  • Weissberg RP, Kumpfer KL, Seligman ME. Prevention that works for children and youth: An introduction. American Psychologist. 2003;58:425–432. [PubMed]
  • Welsh BC, Farrington DP. Evidence-based crime prevention. In: Welsh BC, Farrington DP, editors. Preventing crime: What works for children, offenders, victims and places. Dordrecht, The Netherlands: Springer; 2006. pp. 1–17.