PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of hhspaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
J Soc Social Work Res. Author manuscript; available in PMC 2017 April 19.
Published in final edited form as:
J Soc Social Work Res. 2017 spring; 8(1): 19–44.
Published online 2017 January 26. doi:  10.1086/690560
PMCID: PMC5396956
NIHMSID: NIHMS852633

Scaling Up a Multifaceted Violence Prevention Package: County-Level Impact of the North Carolina Youth Violence Prevention Center

Abstract

Objective

Multifaceted approaches to youth-violence prevention package evidence-based programs into initiatives that yield large-scale impact. This study assessed the impact of a package of evidence-based violence prevention programs, implemented as part of the North Carolina Youth Violence Prevention Center, on county-level violence indicators.

Method

Using growth-curve modeling, the target county was compared to all other counties in North Carolina and a comparison county.

Results

Results reveal downward trends on several county-level indicators (i.e., undisciplined/delinquent complaints, total delinquent complaints, juvenile arrests–aggravated assaults, and short-term suspensions) throughout the intervention period. However, statistical tests were unable to confirm that intervention-period scores on youth-violence indicators were significantly different than expected scores given the relationship between pretest and intervention-period scores in other North Carolina counties.

Conclusions

Although additional administrative data points are needed to support the hypotheses, this study provides preliminary evidence of the effectiveness of North Carolina Youth Violence Prevention Center interventions.

Keywords: youth violence, prevention, multifaceted approach, evidence-based programs, rural

Despite significant decreases in youth-violence perpetration for the past three decades (Office of Juvenile Justice and Delinquency Prevention, 2014), homicide was the third leading cause of death among youth between the ages of 15 and 24 in 2013 (Centers for Disease Control and Prevention, 2013a). Although juvenile arrests for violent crime (e.g., homicide, rape, aggravated assault) have been declining since about 1994, 53,500 youths were arrested for violent offenses in 2014 (Office of Juvenile Justice and Delinquency Prevention, 2015). In addition, according to the Centers for Disease Control and Prevention (CDC; CDC, 2013b) Youth Risk Surveillance System, 17.9% of adolescents in the United States carried a weapon at least once, and 7.1% did not go to school because they felt unsafe at least once during the 30 days preceding the survey. Further, 6.9% reported being threatened or injured with a weapon on school property, and 24.7% reported being in a physical fight in the 12 months before the survey.

In addition to the potential for injury, death, and poor psychosocial adjustment among violence-involved youth, violence also can increase health care costs, negatively impact property values, and decrease productivity at the community level (Krug, Mercy, Dahlberg, & Zwi, 2002). The high percentage of youth involved in violence (CDC, 2013b), coupled with the negative individual and community-level consequences of youth violence, suggest the need for prevention and intervention efforts. Given that risk factors for youth violence span the adolescent ecology (Herrenkohl, Aisenberg, Williams, & Jenson, 2010), multifaceted approaches to youth-violence prevention are necessary; these multifaceted approaches target risk factors for violence at each level of an adolescents’ ecology. Multifaceted approaches often involve the packaging of multiple evidence-based programs that target various levels (e.g., individual adolescents, schools, families) into a comprehensive initiative that is expected to have a substantial impact. Despite the increasing popularity of multifaceted approaches, relatively few studies have focused on the effectiveness of these comprehensive packages on large-scale levels (e.g., community or county levels). Thus, the purpose of the current study was to determine whether a package of evidence-based violence prevention programs implemented as part of the North Carolina Youth Violence Prevention Center (NC-YVPC) reduced county-level violence indicators in a rural, socioeconomically disadvantaged, racially–ethnically diverse county in North Carolina.

Multifaceted Approach to Youth Violence Prevention

The risk and protective factors for youth violence extend across multiple levels of an adolescents’ ecology. For example, common risk factors for youth violence include individual factors (e.g., mental health problems, impulsivity, low IQ), family factors (e.g., child abuse and neglect, low levels of parent–child interaction, parental substance abuse or criminality), school factors (e.g., low academic expectations, low bonding to school, poor academic performance), peer factors (e.g., delinquent or violent peers, gang membership, social rejection), community factors (e.g., neighborhood poverty, community disorganization, high level of transiency), and situational factors (e.g., substance abuse, victim–victimizer relationship; CDC, 2016; Herrenkohl et al., 2010). The protective factors that buffer against youth violence extend across the same ecological levels and include factors such as high IQ, religiosity, family connectedness, commitment to school, and relationships with non-deviant peers (CDC, 2016). Given the multiple ecological levels involved in youth violence, a multifaceted intervention approach is necessary to target risk and protective factors at each level. Each ecological level that plays a role in youth violence is a point of intervention that can be addressed using a multifaceted intervention approach (Krug et al., 2002). For example, a multifaceted approach to youth violence addresses individual risk-taking behavior; attempts to enhance prosocial relationships within and outside of the family; monitors public areas such as schools and neighborhoods in order to address problems that could result in violence; addresses gender inequality and other negative cultural attitudes and practices; and attempts to alter the larger cultural, social, and economic factors that contribute to youth violence (Krug et al., 2002).

Given the broad range of risk factors for youth violence, the CDC promotes a multifaceted approach to youth-violence prevention (CDC, 2010). This approach consists of complementary components—focused on universal and high-risk populations—that target risk factors at multiple ecological levels (i.e., individual, relationship, and community; Vivolo, Matjasko, & Massetti, 2011). To achieve a multifaceted approach, multiple evidence-based programs are often packaged into comprehensive efforts (Kingston, Bacallao, Smokowski, Sullivan, & Sutherland, 2016).

This multifaceted approach aligns with prevention science. The field of prevention science is rooted in gaining knowledge of risk and protective factors and using that knowledge to guide the development, evaluation, and dissemination of prevention programs. In addition to challenges related to widespread dissemination and adaptation, one of the major challenges facing prevention science is partner or community investment beyond initial adoption (Botvin, 2004). For interventions or multifaceted approaches to be effective, increasing community buy-in and relevance are paramount (Cherrington et al., 2008). By matching evidence-based prevention and intervention programs with specific community risk and protective factors, buy-in and relevance are maximized, and it is expected that results will yield positive effects on the prevalence of adolescent health and behavior problems in the community (Hawkins et al., 2008). The multifaceted youth-violence prevention initiative implemented by the NC-YVPC is an example of how a package of evidence-based prevention and intervention programs can be matched to specific community needs and used with the intent of decreasing youth violence.

North Carolina Youth Violence Prevention Center

Funded from 2010 to 2015, NC-YVPC was one of six centers selected and funded by the CDC to prevent youth violence in high-risk communities by using a multifaceted approach. The target county for the NC-YVPC initiative was an extraordinarily racially–ethnically diverse, rural community in south-central North Carolina. The combined population of American Indian, African American, and Latino residents comprises over 68% of the total population in this county, making it one of the 10% of U.S. counties that are majority-minority. Median household income (2009 to 2013) in the target county was one of the lowest in North Carolina at $29,806, compared to the state median of $46,334 (U.S. Census Bureau, 2015). Nearly 48.0% of children in the target county were living in poverty in 2012, compared to 25.8% of children for the state and 23.0% for the United States.

A neighboring county was selected as a no-intervention comparison county. The comparison county was chosen based on similar rates of poverty, rural lifestyle, geographic proximity, and racial–ethnic makeup. In 2012, the unemployment rate in the comparison county was 12.6%, and 35.0% of children were living in poverty. In 2014 the median household income was $34,597, and 24.3% of residents lived below the poverty line (U.S. Census Bureau, 2015). In terms of race, 39.4% of the population was African American, Hispanic/Latino, Asian, and American Indian (U.S. Census Bureau, 2015).

During a planning and partnership development year, a needs assessment was conducted with middle school students in the target county. This assessment revealed two areas of the social ecology that were in particular need: relationships with parents, and relationships with peers. Students reported high rates of perceived school danger and conflict with their parents. In addition, administrative data revealed high juvenile arrest rates: Among North Carolina’s 100 counties, the target county ranked fifth in juvenile arrest rate (7,549 per 100,000) for 2010, behind only major metropolitan areas (North Carolina State Bureau of Investigation, 2015). NC-YVPC subsequently met with community members, led them through the results of this needs assessment, and presented a menu of evidence-based program options. It was posited that working with community members in the selection of the programs would increase community buy-in and perceived intervention relevance, thereby increasing the ultimate success of the multifaceted NC-YVPC initiative.

NC-YVPC followed the CDC’s recommended multifaceted approach to prevention and, with consensus from community members, selected a package consisting of three programs: an evidence-based school curriculum (Positive Action), an evidence-based parent training program (Parenting Wisely), and a Teen Court program that the community had previously implemented and wanted to improve. Positive Action (PA) is a universal program intended for all middle school youth in the target county and focuses on risk factors at the individual and relationship levels. The program philosophy emphasizes prosocial thoughts-feelings-action and positive reciprocal interactions among students and with teachers (Flay & Allred, 2003; Flay, Allred, & Ordway, 2001). Parenting Wisely (PW) is designed for parents whose adolescents are at risk for problem behaviors and also targets the individual and relationship levels (Gordon, 2000; Kacir & Gordon, 1999). Teen Court (TC) is for high-risk adolescents (first-time juvenile offenders) and targets the individual, relationship, and community levels. This is a widely used program to divert first-time offenders away from juvenile court, thereby reducing recidivism and emphasizing restorative justice (Butts & Buck, 2000). The following section provides further information on these programs.

NC-YVPC Evidence-Based Programs

Positive Action

PA is a school-based program designed to improve academic achievement, school attendance, problem behaviors, parent-child bonding, family cohesion, and family conflict (National Registry of Evidence-Based Programs and Practices, 2014). The program consists of a series of kits that include lesson plans and materials designed for use with elementary-, middle-, or high-school aged youth. PA has been recognized as an effective program on several registries of evidence-based programs, which use stringent standards of effectiveness (Blueprints for Healthy Youth Development, 2012; Office of Justice Programs, n.d.; U.S. Department of Education, 2007).

Parenting Wisely

PW is an interactive computer-based program that is designed to increase parenting knowledge and competence, decrease adolescent problem behaviors, and improve the parent–adolescent relationship (Gordon, 2000). The program includes 10 video modules with vignettes of typical parent–adolescent conflicts, such as helping children to do housework, sharing the computer, curfew, and sibling conflict. After viewing each vignette, parents select a response strategy from a list of possible options that represent different levels of parenting effectiveness. The selected response option is then portrayed in a second video vignette and critiqued through interactive questions and answers (Kacir & Gordon, 1999). Given its effectiveness, PW has received accolades from numerous registries of evidence-based programs and other agencies (Fagan, Hanson, Hawkins, & Arthur, 2009; Office of Juvenile Justice and Delinquency Prevention, 2012; Strengthening America’s Families, 1999; Substance Abuse and Mental Health Services Administration, 2008).

Teen Court

TC is an alternative to the juvenile-justice system that aims to reintegrate first-time adolescent offenders into society through use of prosocial sanctions. Based on a model of restorative justice, TC does not establish guilt or innocence, but it is used as a sanctioning agent for youths willing to admit their guilt (Butts & Buck, 2000; Stickle, Connell, Wilson, & Gottfredson, 2008). The most common model for TC is an adult judge presiding over an adolescent defense attorney, prosecution attorney, bailiff, and jurors. After arguments are made by the defense and prosecution, the adolescent jury determines sanctions through a wide array of sentencing options with certain minimum sentences based on the severity of the offense. The aim of these sanctions is to reintegrate youthful offenders into the community by helping them connect to community members and give back to others through community service.

NC-YVPC researchers have examined the effects of each of these programs on adolescents and families in the target community. The PA program, implemented over 3 years, was found to increase self-esteem and decrease school hassles (Guo et al., 2015). PW was associated with significant decreases in parent–child conflict, adolescent externalizing behavior, and adolescent violent behavior (Cotter, Bacallao, Smokowski, & Robertson, 2013). Participation in TC was significantly associated with decreases in internalizing symptoms, externalizing behavior, violent behavior, parent–child conflict, and delinquent friends, as well as increases in self-esteem and school satisfaction (Evans, Smokowski, Barbee, Bower, & Barefoot, 2016).

Prevention scientists discuss “scaling up” of interventions as bringing effective preventive interventions to scale across a large and diverse number of schools, neighborhoods, communities, or counties. NC-YVPC scaled up the multifaceted prevention package by implementing PA, PW, and TC in diverse communities and neighborhoods within a 900-square-mile rural county. Programs were provided in 12 middle schools, dozens of churches, community centers, and multiple courthouses. The key question for this study was: Did aggregating these programs into a comprehensive package and scaling up dissemination to serve all communities within a large rural county have a measurable impact on county-level indicators of delinquency and violence?

Past Intervention Studies Examining County-Level Change

Although researchers commonly examine county-level data to assess rates of interpersonal crime and violence and the associated covariates (e.g., Kposowa, Breault, & Harrison, 1995), we were unable to identify any studies that examined how the presence of multifaceted interventions impact county-level indicators of violence. One group of researchers has developed a model for identifying county-level substance-use treatment needs with available county indictors (Herman-Stahl et al., 2001), but this approach is focused on needs assessment rather than evaluating change at the county level. In addition, Communities That Care is a multifaceted intervention approach in which communities adopt a variety of evidence-based practices to address risky adolescent behavior (Feinberg, Hones, Greenberg, Osgood, & Bon-tempo, 2009; Hawkins, Catalano, & Kuklinski, 2014). During the 20 years of research on the efficacy of Communities That Care and its implementation in over 500 communities in the United States, Australia, Canada, and Europe, researchers have mainly examined how participation in Communities That Care impacts violence at the individual or grade-cohort levels (Feinberg et al., 2009; Hawkins et al., 2008) rather than at the county level. Rather than focusing only on individual levels of violence, the current study examined if and how the multifaceted approach implemented by the NC-YVPC has impacted county levels of violence.

Hypotheses for Current Study

Based on prevention-science principles and on evidence of individual effectiveness of PA, PW, and TC, we hypothesized that (a) relative to the nonintervention counties and preintervention years, the target county (i.e., the county that received the previously described package of programs) would display significant decreases on county-level youth violence indicators throughout the intervention period (Hypothesis A); and (b) intervention-period scores on county-level youth-violence indicators would be significantly different from the scores expected given the relationship between baseline and intervention-period scores in other North Carolina counties (Hypothesis B). In sum, the purpose of this unique evaluation was to test if the aggregated package of evidenced-based programs was strong enough to impact county-level change.

Method

Research Design

The current study used an interrupted time series design to examine changes on key youth-violence indicators. Using data from several state agencies, the interrupted time-series design allowed for the examination of youth-violence trends during a preintervention period that spanned between 3 and 10 years depending on the indicator (described later) in addition to trends after the package of evidence-based programs was implemented. Specific comparisons were made between the target county, the (nonintervention) comparison county, and the other 98 urban and rural counties in the state. A series of models were estimated for eight youth-violence indicators, each of which is described in the following section.

Measures

Annual data at the county level (N = 100, or all 100 counties in North Carolina) from 2000 to 2013 were analyzed in the current study, although not all indicator variables have complete data for the entire study period (more information follows). In addition to comparing the target county with all counties in North Carolina, a neighboring county was chosen as a comparison county given geographic and socioeconomic similarities. We selected various county-level measures that captured juvenile offenses in the community as well as the school setting. The eight measures examined in the analysis come from three sources: the North Carolina Department of Public Safety, the North Carolina Department of Public Instruction, and the North Carolina State Bureau of Investigation. The state’s Department of Public Safety categorizes complaints received by the Juvenile Justice Section of the Division of Adult Correction and Juvenile Justice as undisciplined (i.e., offenses such as truancy that would not be considered a crime if committed by an adult) and delinquent (i.e., offenses that would be considered a crime if committed by an adult). Thus, North Carolina Department of Public Safety (2015) indicators in the current study included (a) total undisciplined and delinquent complaints from 2004 to 2013, (b) total delinquent complaints from 2004 to 2013 (a subset of the previous variable), and (c) school-based offenses from 2007 to 2013 (i.e., offenses that occur on school grounds, school property, at a school bus stop, or at an off-campus school-sanctioned event). Data from the North Carolina Department of Public Instruction (2015) included (a) short-term suspensions (i.e., in school or out-of-school suspensions lasting up to 10 days) from 2001 to 2013, (b) long-term suspensions (i.e., out-of-school suspensions that last between 11 days and the remainder of the school year) from 2001 to 2013, and (c) acts of crime and violence at school from 2002 to 2013 (e.g., possession of a weapon, possession of a controlled substance, assault on school personnel). Data from the North Carolina State Bureau of Investigation (2015) include information on juvenile arrests; the following indicators were included in the current study:

  • juvenile arrests—aggravated assaults (i.e., the unlawful attack by one person upon another with the intent to produce serious bodily harm, usually accompanied by use of a weapon) from 2000 to 2013; and
  • total juvenile arrests for any offense from 2000 to 2013.

Each of these variables was measured per 10,000 population aged 10 to 17. Unemployment rate and percentage of children at or below the poverty line were included as covariates. The NC-YVPC intervention began in September 2010; therefore, the interruption in a time series refers to 2011, and the intervention period refers to 2011 onward. The data we used came from state organizations and are of very high quality. County administrators are required to report this data accurately each year or their state funding is at risk; this minimized missing data and coding errors. The data are cleaned and reviewed by state agencies before being released to the public. This provided a highly standardized, longitudinal panel of data on counties across an entire state.

Analytic Plan

Graphic approaches to visualize trends

The first step of analysis comprised plotting a series of raw data in conjunction with curve smoothing. On each juvenile-indicator variable, the raw data were plotted over the study period with smoothed curves for the target county, the comparison county, urban counties, and rural counties, excluding the target and comparison counties. Curve smoothing was employed to remove random fluctuations so that the trends embedded in a time series could be visually detected. Locally weighted regression using tricube kernel, also known as lowess, was used for curve-smoothing (Fox, 2000). The graphic methods aid in understanding the patterns of change and specifically in the testing of Hypothesis A.

The linear growth-curve model

This study employs an interrupted time-series design—precisely, growth-curve modeling with random effects (i.e., hierarchical linear modeling or HLM)—to test research Hypothesis B. Known as growth-curve modeling (Lindsey, 1993; Raudenbush & Bryk, 2002), this analysis evaluates the change of time-series data to evaluate the impact of intervention or policy changes on key time-series variables.

The stationarity assumption

Following statistical methods analyzing time-series data, this study assumes stationarity; that is, each element in a time series is a random draw from a population with zero mean and constant variance. Specifically, based on Greene (2003), we assume that the stochastic process is weakly stationary or covariance stationary. Under this assumption, we specified three structures of the residual term of the growth-curve model; that is, the compound symmetric, autoregressive order one, and unstructured residual matrix. Results indicated that the unstructured matrix was best.

The growth-curve model or HLM

Following, we take the county-level measure “total undisciplined and delinquent complaints per 10,000 population aged 10–17 (complaints)” as an example to illustrate the modeling process and the test of Hypothesis B. This analysis first pools together all counties’ data on complaints from 2004 to 2013. As such, the total number of observations for the complaints model was 1,000 (or 10 years multiplied by 100 counties). Denoting the measure of interest complaints as Ytj, t is the index of time using Year 2004 as a baseline (t = 0, 1, … 9), and j the index of county (j = 1, … 100), the HLM can be expressed as follows:

Level-1:Ytj=β0j+β1j(UNEMP)tj+β2j(CHIPOV)tj+β3j(TIME)tj+β4j(TX_PERIOD)tj+etjLevel-2:β0j=γ00+γ01(TARGET)j+γ02(URBAN)j+u0j;β1j=γ10;β2j=γ20;β3j=γ30+γ31(TARGET)j+u3j;β4j=γ40+γ41(TARGET)j

In this model, (UNEMP)tj and (CHIPOV)tj are two variables measuring the jth county’s unemployment rate and percentage of children at/below the poverty line at year t, respectively. (TIME)tj is the year variable capturing the linear change of the outcome variable over time, and codes Year 2004 as baseline: (TIME)tj = 0 if year = 2004, (TIME)tj = 1 if year = 2005, … and (TIME)tj = 9 if year = 2013. (TX_PERIOD)tj is a dichotomous variable measuring the intervention period: (TX_PERIOD)tj = 1 if t is an intervention year (i.e., a year that is after 2010) for the jth county, and (TX_PERIOD)tj = 0 otherwise. (TARGET)j and (URBAN)j are two dichotomous variables indicating the type of counties: (TARGET)j = 1 if j is the target county, and (TARGET)j = 0 otherwise; (URBAN)j = 1 if j is an urban county, and (URBAN)j = 0 otherwise. As such, the reference or omitted group of county type is “rural counties excluding target county.”

By the above specifications, γ00 is the mean complaints in 2004 for all rural counties excluding target county; γ01 is the difference on mean complaints between the target county and the rest of the rural counties at any point in time; γ02 is the difference on mean complaints between the urban counties and the rural counties, excluding the target county at any point in time; γ30 is the main effect of time on complaints—precisely, γ30 indicates the linear change rate (i.e., the amount of decrease or increase per year on complaints, depending on the sign); and γ40 is the main effect of the intervention period that indicates the average difference of complaints between an intervention year and a pre-intervention year. To test Hypothesis B, we specified two cross-level interactions, γ31 and γ41; γ31 represents how the rate of change on complaints differs between the target county and the remaining counties in North Carolina, and γ41 represents the degree of difference on complaints between the target county in an intervention year and all other counties in any year, or the difference on complaints between an intervention year and a pre-intervention year for the target county. γ41 serves an important function to test Hypothesis B: Under certain conditions, if γ41 is negative, then the model suggests that complaints in the target county in the intervention period decreased more than the remaining 99 counties during the same period, or that the target county experienced an additional decrease during the intervention period compared to its preintervention period.

To obtain an overall picture of the model-predicted pattern of change, we further plotted model-based trajectories on each indicator variable using model-estimated coefficients of the growth-curve analysis. These figures take all related parameters into consideration while controlling for (UNEMP)tj and (CHIPOV)tj at the mean level. Precisely, each plot employs the intercept, the main effects of county type γ01 and γ02, the main effect of time γ30, the main effect of intervention period γ40, and the interaction effects of target county by time γ31 and target county by the intervention period γ41.

Results

Table 1 presents the linear growth-curve model for each violence indicator. Figures 14 (and figures S1S4, available online) present plots of change trajectories. There are two types of plots in each figure: one illustrates the change trajectories based on the raw data with curve smoothing that helps test Hypothesis A, and the other is a model-based prediction that helps show the general pattern and test Hypothesis B.

Figure 1
Figure 1a illustrates raw data with curve smoothing. Figure 1b illustrates model-predicted trajectories. Y = total undisciplined and delinquent complaints per 10,000 population aged 10 to 17.
Figure 4
Figure 4a illustrates raw data with curve smoothing. Figure 4b illustrates model-predicted trajectories. Y = school-based offenses per 10,000 population aged 10 to 17.
Table 1
Estimated Coefficients of Linear Change Model

Total Undisciplined and Delinquent Complaints per 10,000 Population Aged 10–17

Figure 1a illustrates the observed change trajectories on “total undisciplined and delinquent complaints” (complaints). As the figure shows, at any point in time, the target county had a level of complaints that was higher than that of the urban counties and the remaining rural counties. All four groups experienced a decrease on the measure over time; the rural counties and the comparison county experienced a faster decrease since 2009, and the target county experienced a steady decrease since 2009. Given that the county shows 2 years of a downward trend for complaints prior to the intervention, it is suggestive that the intervention county continued a downward trend in the postintervention period that had begun 2 years prior to the intervention implementation. Overall, the examination of the trends supports Hypothesis A; that is, there is a pattern of downward trends on complaints throughout the intervention period, relative to the preintervention baseline years, although the downward trend cannot be solely attributable to the intervention.

Significant findings of the linear growth-curve model (see Table 1) include:

  1. On average, complaints decreased over time at a rate of 14.0 per year (p < .001).
  2. On average, complaints in the intervention period decreased at a rate of 65.7 per year (p < .001).

The model-based prediction (Figure 1b) shows an overall pattern of change. Although the target county had a higher number of complaints than urban or rural counties, the figure indicates that the target county experienced a steady decline on the measure since the intervention. The interaction coefficient γ41 is not statistically significant, but it is negative, indicating that the change of complaints in an intervention year in the target county was faster than all other counties in any year. Alternatively, the target county’s complaints level in an intervention year is lower than its own level in any pre-intervention year. Results do not support Hypothesis B, although the intervention change in the target county is in the expected direction.

Total Delinquent Complaints per 10,000 Population Aged 10–17

Figure S1a (available online) presents the observed-change trajectories on “total delinquent complaints” (delinquents). As the figure shows, at any point in time, the target county had a level of delinquents that was higher than that of the urban counties and the remaining rural counties. All four groups experienced a decrease on the measure over time, but the comparison county experienced higher decreases than the target county since 2009, although the target county’s decrease after 2010 was steady. Overall, the examination of the trends on delinquents since 2010 supports Hypothesis A; that is, there is a pattern of downward trends on delinquents throughout the intervention period, relative to the preintervention baseline years.

Significant findings of the linear growth-curve model include:

  1. On average, delinquents decreased over time by 12.31 per year (p < .001).
  2. On average, delinquents in the intervention period decreased by 51.72 per year (p < .001).

The model-based prediction (Figure S1b, available online) shows an overall pattern of change. Although the target county had a higher number of delinquents than urban and rural counties, the figure indicates that the target county experienced a steep decline from 2010 to 2011. The interaction coefficient γ41 is not statistically significant, but it is negative, indicating that the change of delinquents in an intervention year in the target county was faster than all other counties in any year. Alternatively, the target county’s delinquents level in an intervention year is lower than its own level in any preintervention year. Results do not support Hypothesis B, although the intervention change in the target county is in the expected direction.

Long-Term Suspensions per 10,000 Population Aged 10–17

Figure S2a (available online) illustrates the observed-change trajectories on “long-term suspensions” (long suspensions). Prior to 2008, the urban counties had a higher amount of long suspensions than the target county, although the target county experienced a steady decline over time. The change in the target county after 2010 seems to be slow and leveled, and as such, the results do not support Hypothesis A.

Significant findings of the linear growth-curve model include:

  1. On average, long suspensions increased over time at a rate of 1.87 per year (p < .01).
  2. On average, long suspensions in the intervention period decreased at a rate of 25.18 per year (p < .001).

The model-based prediction (Figure S2b, available online) shows an overall pattern of change. The target county experienced a steady decline on long suspensions throughout the entire period of study, including the intervention period from 2011 to 2013. The interaction of “intervention period by target county” γ41 is positive, but the interaction of “time by target county” γ31 is negative, indicating that the target county experienced an overall downward trend in the change. Note that the model-predicted trajectories for both urban and rural counties bounced back since 2011, but the target county maintained its decreasing trend. Overall, results do not support Hypothesis B, although the intervention change in the target county is in the expected direction.

Juvenile Arrests—Aggravated Assaults per 10,000 Population Aged 10–17

Figure 2a presents the observed change trajectories on “juvenile arrests—aggravated assaults” (aggravated assaults). As the figure shows, at any point in time the target county had a level of aggravated assaults that was higher than that of the urban counties and the remaining rural counties; all four groups experienced a decrease on the measure over time, and the target county experienced a steady decrease since 2011. Overall, the examination of the trends on aggravated assaults supports Hypothesis A; that is, there was a pattern of downward trends on aggravated assaults throughout the intervention period, relative to the preintervention baseline years.

Figure 2
Figure 2a illustrates raw data with curve smoothing. Figure 2b illustrates model-predicted trajectories. Y = juvenile arrests—aggravated assault per 10,000 population aged 10 to 17.

Significant findings of the linear growth-curve model (see Table 1) include:

  1. On average and at any point in time, the target county’s aggravated assaults were higher than that of other rural counties by 27.65 (p < .05).
  2. On average and at any point in time, urban counties’ aggravated assaults were higher than that of other rural counties by 9.62 (p < .05).

The model-based prediction (Figure 2b) shows an overall pattern of change. The target county experienced a steady decline on aggravated assaults in the entire study period. It is worth noting that the decline in the target county since 2011 was steeper than that of both urban and rural counties, which could be due to the intervention package. Results do not support Hypothesis B, although the intervention change in the target county is in the expected direction.

Short-Term Suspensions per 10,000 Population Aged 10–17

Figure S3a (available online) presents the observed change trajectories on “short-term suspensions” (short suspensions). As the figure shows, at any point in time the target county had a level of short suspensions that was higher than that of the urban counties and the remaining rural counties; the comparison county experienced an increase on the measure since 2009, whereas the target county experienced a decrease since 2010. Overall, the examination of the trends on short suspensions supports Hypothesis A; that is, there is a pattern of downward trends on short suspensions throughout the intervention period, relative to the preintervention baseline years.

Significant findings of the linear growth-curve model include:

  1. On average, short suspensions increased over time at a rate of 36.45 per year (p < .05).
  2. On average, short suspensions in the intervention period decreased at a rate of 626.55 per year (p < .001). In addition, the interaction of intervention period by target county was negative (p < .05), but the interaction of time by target county was positive (p < .01); together, these two coefficients indicate that the trend in the intervention period was decreasing, which partially supports Hypothesis B.

The model-based prediction (Figure S3b, available online) shows an overall pattern of change. It is worth noting that the model-based prediction for the intervention period failed to show a downward trend manifested by the raw data (Figure S3a, available online). This is partially because the change in the target county on short suspensions is basically curvilinear, and a linear growth-curve model is unable to depict such a complicated change. Due to this limitation, the usefulness of the model-predicted figure is limited on this measure.

Acts of Crime and Violence at School per 10,000 Population Aged 10–17

Figure 3a presents the observed-change trajectories on “acts of crime and violence at school” (crime). As the figure shows, at any point in time prior to 2009, the target county had a level of crime that was higher than that of the urban counties, as well as that of the remaining rural counties; the urban counties had higher crime than the target county after 2009, and the rural counties experienced a downward trend over the entire study period. In addition, the comparison county had a steady decrease after 2010, and the target county bounced back on the measure from 2010 to 2011 and then maintained the same level of crime over the intervention period. Overall, the examination of the trends on crime do not support Hypothesis A.

Figure 3
Figure 3a illustrates raw data with curve smoothing. Figure 3b illustrates model-predicted trajectories. Y = acts of crime and violence at school per 10,000 population aged 10 to 17.

Significant findings of the linear growth-curve model (see Table 1) include:

  1. On average, crime in the intervention period decreased by 12.08 per year (p < .001).
  2. On average and at any point in time, urban counties’ crime was higher than that of the rural counties, excluding the target county, by 26.43 (p < .05).

The model-based prediction (Figure 3b) shows an overall pattern of change. The model indicates that crime had a sharp decline from 2010 to 2011 and then slightly increased. Thus, results do not support Hypothesis B.

School-Based Offenses per 10,000 Population Aged 10–17

Figure 4a illustrates the observed change trajectories on “school-based offenses” (offenses). As the figure shows, at any point in time, the target county had a level of offenses that was higher than that of the urban counties and the remaining rural counties; the comparison county experienced a steady decrease over the entire period. Because the target county bounced back from 2011 to 2012 and the change during the intervention period was minimal, results do not support Hypothesis A.

On average, offenses decreased over time at a rate of 7.35 per year (p < .001). The model-based prediction (Figure 4b) shows an overall pattern of change. Although the target county had a higher number of offenses than urban and rural counties, the figure indicates that the target county experienced a steady decline on the measure over the entire study period. Because the interaction coefficient γ41 is positive and the raw data show a leveled trend in the intervention period, the results do not support Hypothesis B.

Juvenile Arrests per 10,000 Population Aged 10–17

Figure S4a (available online) presents the observed-change trajectories on “juvenile arrests” (arrests). As the figure shows, at any point in time, the target county had a level of arrests that was higher than that of the urban counties and the remaining rural counties. Although the target county’s change was curvilinear, the other three groups experienced minimal changes over the entire study period. Overall, the trends on arrests do not show a decline during the intervention, and results do not support Hypothesis A.

Significant findings of the linear growth-curve model (see Table 1) include:

  1. On average and at any point in time, the target county’s arrests were higher than that of other rural counties by 894.07 (p < .05).
  2. On average and at any point in time, urban counties’ arrests were higher than that of other rural counties by 330.68 (p < .01). The model-based prediction (Figure S4b, (available online) shows an overall pattern of change. Results do not support Hypothesis B because the trend in the intervention period does not show change.

Discussion

Overall, the results of this study indicate that NC-YVPC’s multifaceted, comprehensive package of evidence-based programs was associated with downward trends on some county-level indicators (i.e., undisciplined and delinquent complaints, total delinquent complaints, juvenile arrests—aggravated assaults, and short-term suspensions) throughout the intervention period (i.e., Hypothesis A). However, Hypothesis B (i.e., that intervention period scores on youth violence indicators would be significantly different than expected scores given the relationship between pre-test and intervention period scores in other counties in the state) was not supported by the linear growth-curve models. Although the “intervention period × target county” interaction terms did not reach statistical significance for undisciplined and delinquent complaints, delinquent complaints, and juvenile arrests for aggravated assaults, the negative coefficients suggest that the decreases in intervention years for these indicators were faster relative to the change in all other counties, as well as relative to the target county in pre-intervention years. In sum, Hypothesis A was partially supported, and Hypothesis B was not supported, although some desirable trends emerged.

By comparing the indicators that showed desirable trends versus those that did not, a pattern emerges. First, the decreasing trends of undisciplined and delinquent complaints, delinquent complaints, and juvenile arrests for aggravated assaults suggest that the comprehensive package of programs may have decreased violent behavior and increased positive interpersonal relationships in the target county. Indeed, all three evidence-based programs (PA, PW, and TC) target problem behavior at the individual level and seek to improve interpersonal relationships. In addition, decreasing trends of short-term suspensions may indicate a combination of two factors: First, given the aforementioned decreased violent behavior and increased positive relationships, youth may be committing fewer school violations that result in short-term suspensions; second, it is possible that due specifically to the TC program, schools in the target county may not be using suspensions as frequently given the alternative option to make referrals to the TC program. Additional qualitative research, such as interviews with school personnel, could be used to explore these possibilities.

The indicators that did not show desirable trends, on the other hand, included acts of crime and violence at school, school-based offenses, total juvenile arrests, and long-term suspensions. First, the fact that reports of school-based acts of crime, violence, and offenses did not decrease could potentially be explained by the introduction of the TC program. It is possible that more offenses that occurred at school were reported because school personnel were aware of the option to refer students to the TC program. Of course, regardless of the availability of TC, certain acts that occur at school are required by law to be reported. However, state guidelines may leave some degree of subjectivity in terms of which incidents are required to be reported. It is possible that before introduction of the TC program, school personnel in the target county did not report “minor” bullying incidents, given the lack of appropriate sanctions for this type of behavior. With the introduction of the TC program, these same “minor” bullying incidents may have been reported because of the perception that TC was an appropriate consequence for this type of behavior. Of course, additional research and data points are needed to confirm this conjecture.

In terms of total juvenile arrests, looking more closely at this indicator provides some potential explanation for the lack of desirable trends in this study. Unlike juvenile arrests for aggravated assaults, which are explicitly violent crimes, total juvenile arrests comprise both violent and property crimes. Given the focus of the three programs that make up the comprehensive package on interpersonal relationships and nonviolence, it is plausible that violent offenses were impacted to a larger degree than property offenses among juveniles. Additional data points may be necessary to further reveal the impact on this indicator.

Overall, the preliminary results of the NC-YVPC’s multifaceted package of evidence-based programs are encouraging. Although additional administrative data points will provide increased confidence in the results, this study provides some preliminary evidence of the effectiveness of NC-YVPC at impacting large-scale change. Research on community-level evaluation is particularly challenging because many factors, measured and unmeasured, impact county-level change. Without the massive resources that it would take to conduct a randomized community-level trial, the observational approach with an extended baseline used in the current study is an excellent alternative for tracking community-level change. Of course, caution is warranted in attributing change solely to the intervention package; however, it was encouraging that changes during the intervention period were more pronounced in the target community relative to both other counties and to preintervention years on several outcome measures. These results provide support for the multifaceted approach to youth violence prevention endorsed by the CDC (Vivolo et al., 2011), as well as the effectiveness of packaging evidence-based programs through community partnerships to prevent youth violence (Kingston et al., 2016). Given the dearth of research on the effectiveness of “packages” of evidence-based programs, this study makes a substantial contribution to the literature.

Limitations

Given a lag in the availability of county-level administrative data, we were unable to analyze data points beyond the year 2013. We have run the same analyses for recently released 2014 data, and the results are identical to those reported here. The limited number of data points may have contributed to the nonsignificance of the main coefficients; additional data points might shed further light on the violent behavior indicator trends. In addition, given that the current study was quasi-experimental, it is possible that the differences in trends observed between the target county and the other counties were due to unmeasured county differences rather than solely due to the intervention. The study’s findings are likely tempered by unobserved heterogeneity and comparability of study sites. We must balance the limitation of having unmeasured differences among the 100 rural and urban counties in North Carolina with the ambitious goal of conducting an evaluation of youth-violence indicators across an entire state. Our inclusion of several relevant violent-behavior indicators and covariates minimizes this risk, but it remains a legitimate concern.

Implications

Now that prevention scientists have delineated numerous evidence-based programs, there is a growing need to package effective programs together to reach as many people as possible. This “scaling up” of prevention efforts using multilevel initiatives (i.e., bringing effective preventive interventions to scale across a large and diverse number of schools, neighborhoods, communities, or counties) has substantial promise to help meet public health needs. The results of this study indicate that NC-YVPC’s multifaceted, comprehensive package of evidence-based programs was associated with downward trends on county-level indicators for undisciplined and delinquent complaints, total delinquent complaints, juvenile arrests—aggravated assaults, and short-term suspensions throughout the intervention period. Although other effects were not statistically significantly different when compared in linear growth-curve models, the target county began with rates high above other counties and, during intervention, generally decreased to levels similar to other urban and rural counties. This provides initial support for the utility of scaling up prevention efforts using multilevel packages.

Supplementary Material

S1

S2

S3

S4

Acknowledgments

Funding for this research was provided through a cooperative agreement with the U.S. Centers for Disease Control and Prevention National Center for Injury Prevention and Control (5 U01 CE001948-03).

Contributor Information

Paul R. Smokowski, University of Kansas.

Katie L. Cotter, Arizona State University.

Shenyang Guo, Washington University in St. Louis.

Caroline Bill Robertson Evans, University of Kansas.

References

  • Blueprints for Healthy Youth Development. Program criteria [Web page] 2012 Retrieved from http://www.blueprintsprograms.com/programCriteria.php.
  • Botvin GJ. Advancing prevention science and practice: Challenges, critical issues, and future directions. Prevention Science. 2004;5:69–72. https://doi.org/10.1023/B:PREV.0000013984.83251.8b. [PubMed]
  • Butts JA, Buck J. Teen Courts: A focus on research [Report] The Office of Juvenile Justice and Delinquency Prevention; 2000. Retrieved from https://www.ncjrs.gov/pdffiles1/ojjdp/183472.pdf.
  • Centers for Disease Control and Prevention (CDC) Cooperative agreement program for the National Academic Centers of Excellence in Youth Violence Prevention [Web page] 2010 Retrieved from http://www.grants.gov/search/search.do?mode=VIEW&oppId=52740.
  • Centers for Disease Control and Prevention (CDC) Web-based inquiry statistics query and reporting system—United States, 2013. 2013a [Data set]. Retrieved from http://www.cdc.gov/injury/wisqars/index.html.
  • Centers for Disease Control and Prevention (CDC) Youth risk behavior surveillance—United States, 2013. 2013b [Data set]. Retrieved from https://nccd.cdc.gov/youthonline/App/Default.aspx.
  • Centers for Disease Control and Prevention (CDC) Youth violence: Risk and protective factors. Injury Prevention and Control: Division of Violence Prevention [Web page] 2016 Retrieved from http://www.cdc.gov/violenceprevention/youthviolence/riskprotectivefactors.html.
  • Cherrington A, Ayala GX, Amick H, Allison J, Corbie-Smith G, Scarinci I. Implementing the community health worker model within diabetes management: Challenges and lessons learned from programs across the United States. The Diabetes Educator. 2008;34:824–833. https://doi.org/10.1177/0145721708323643. [PubMed]
  • Cotter KL, Bacallao M, Smokowski PR, Robertson CIB. Parenting interventions implementation science: How delivery format impacts the Parenting Wisely program. Research on Social Work Practice. 2013;0(0):1–12. https://doi.org/10.1177/1049731513490811.
  • Evans CBR, Smokowski PR, Barbee J, Bower M, Barefoot S. Restorative justice programming in Teen Court: A path to improved interpersonal relationships and psychological functioning in high-risk rural youth. Journal of Rural Mental Health. 2016;40(1):15–30. https://doi.org/10.1037/rmh0000042.
  • Fagan AA, Hanson K, Hawkins JD, Arthur M. Translational research in action: Implementation of the Communities That Care prevention system in 12 communities. Journal of Community Psychology. 2009;37:809–829. https://doi.org/10.1002/jcop.20332. [PMC free article] [PubMed]
  • Feinberg MF, Jones D, Greenberg MT, Osgood DW, Bontempo D. Effects of the Communities That Care model in Pennsylvania on change in adolescent risk and problem behaviors. Prevention Science. 2009;11:163–171. https://doi.org/10.1007/s11121-009-0161-x. [PMC free article] [PubMed]
  • Flay BR, Allred CG. Long-term effects of the positive-action program. American Journal of Health Behavior. 2003;27:S6–S21. https://doi.org/10.5993/AJHB.27.1.s1.2. [PubMed]
  • Flay BR, Allred CG, Ordway N. Effects of the positive action program on achievement and discipline: Two match-control comparisons. Prevention Science. 2001;2:71–89. https://doi.org/10.1023/A:1011591613728. [PubMed]
  • Fox J. Nonparametric simple regression—smoothing scatterplots. Thousand Oaks, CA: Sage Publications; 2000.
  • Gordon DA. Parent training via CD-ROM: Using technology to disseminate effective prevention practices. The Journal of Primary Prevention. 2000;21:227–251. https://doi.org/10.1023/A:1007035320118.
  • Greene WH. Econometric analysis. 5. Upper Saddle River, NJ: Prentice Hall; 2003.
  • Guo S, Wu Q, Smokowski PR, Bacallao M, Evans CBR, Cotter KL. A longitudinal evaluation of the positive action program in a low-income, racially diverse, rural county: Effects on self-esteem, school hassles, aggression, and internalizing symptoms. Journal of Youth and Adolescence. 2015;44:2337–2358. https://doi.org/10.1007/s10964-015-0358-1. [PubMed]
  • Hawkins JD, Catalano RF, Arthur MW, Egan E, Brown EC, Abbott RD, Murray DM. Testing Communities That Care: The rationale, design, and behavioral baseline equivalence of the community youth development study. Prevention Science. 2008;9(3):178–190. https://doi.org/10.1007/s11121-008-0092-y. [PMC free article] [PubMed]
  • Hawkins JD, Catalano RF, Kuklinski MR. Communities That Care. In: Bruinsma G, Weisburd D, editors. Encyclopedia of Criminology and Criminal Justice. New York, NY: Springer Publishing; 2014. pp. 393–408.
  • Herman-Stahl M, Wiesen CA, Flewelling RL, Weimer BJ, Bray RM, Rachal JV. Using social indicators to estimate county-level substance use intervention and treatment needs. Substance Use and Misuse. 2001;36:501–521. https://doi.org/10.1081/JA-100102639. [PubMed]
  • Herrenkohl TI, Aisenberg E, Williams JH, Jenson JM. Violence in context: Current evidence on risk, protection, and prevention. New York, NY: Oxford University Press; 2010.
  • Kacir CD, Gordon DA. Parenting adolescents wisely: The effectiveness of an interactive videodisk parent training program in Appalachia. Child and Family Behavior Therapy. 1999;21:1–22. https://doi.org/10.1300/J019v21n04_01.
  • Kingston B, Bacallao M, Smokowski PR, Sullivan T, Sutherland K. Constructing “packages” of evidence-based programs to prevent youth violence: Processes and illustrative examples from the CDC’s Youth Violence Prevention Centers. The Journal of Primary Prevention. 2016;37:141–163. https://doi.org/10.1007/s10935-016-0423-x. [PMC free article] [PubMed]
  • Kposowa AJ, Breault KD, Harrison BM. Reassessing the structural covariates of violent and property crimes in the USA: A county level analysis. The British Journal of Sociology. 1995;46(1):79–105. https://doi.org/10.2307/591624.
  • Krug EG, Mercy JA, Dahlberg LL, Zwi AB. The world report on violence and health. The Lancet. 2002;360:1083–1088. https://doi.org/10.1016/S0140-6736(02)11133-0. [PubMed]
  • Lindsey JK. Models for repeated measures. New York, NY: Oxford University Press; 1993.
  • National Registry of Evidence-Based Programs and Practices. Minimum requirements [Web page] 2014 Retrieved from http://www.nrepp.samhsa.gov/ReviewSubmission.aspx.
  • North Carolina Department of Public Instruction. Research data request from North Carolina Education Research Center [Data file] Durham, NC: Author; 2015.
  • North Carolina Department of Public Safety. N.C. juvenile justice county data book. 2015 Retrieved from https://www.ncdps.gov/Index2.cfm?a=000003,002476,002487.
  • North Carolina State Bureau of Investigation. Research data request from State Bureau of Investigation Uniform Crime Reports [Data file] Raleigh, NC: Author; 2015.
  • Office of Justice Programs. Program profile: Positive Action [Web page] n.d Retrieved from http://www.crimesolutions.gov/ProgramDetails.aspx?ID=113.
  • Office of Juvenile Justice and Delinquency Prevention. OJJDP model programs guide [Website] 2012 Retrieved from http://www.ojjdp.gov/mpg/
  • Office of Juvenile Justice and Delinquency Prevention. OJJDP statistical briefing guide [Website] 2014 Retrieved from http://www.ojjdp.gov/ojstatbb/crime/JAR_Display.asp?ID=qa05201.
  • Office of Juvenile Justice and Delinquency Prevention. OJJDP statistical briefing guide [Website] 2015 Retrieved from http://www.ojjdp.gov/ojstatbb/crime/JAR_Display.asp?ID=qa05202.
  • Raudenbush SW, Bryk AS. Hierarchical linear models: Applications and data analysis methods. 2. Thousand Oaks, CA: Sage Publications; 2002.
  • Stickle WP, Connell NM, Wilson DM, Gottfredson D. An experimental evaluation of teen courts. Journal of Experimental Criminology. 2008;4:137–163. https://doi.org/10.1007/s11292-008-9050-8.
  • Strengthening America’s Families. Strengthening America’s families: Effective family programs for prevention of delinquency [Website] 1999 Retrieved from http://www.strengtheningfamilies.org/html/model_programs.html.
  • Substance Abuse and Mental Health Services Administration. SAMHSA’s national registry of evidence-based programs and practices [Website] 2008 Retrieved from http://www.nrepp.samhsagov/ViewIntervention.aspx?id¼35.
  • U.S. Census Bureau. State and county QuickFacts [Website] 2015 Retrieved from http://quickfacts.census.gov/qfd/states/37000.html.
  • U.S. Department of Education. What Works Clearinghouse [Website] 2007 Retrieved from http://ies.ed.gov/ncee/wwc/interventionreport.aspx?sid=380.
  • Vivolo AM, Matjasko JL, Massetti GM. Mobilizing communities and building capacity for youth violence prevention: The National Academic Centers of Excellence for Youth Violence Prevention. American Journal of Community Psychology. 2011;48:141–145. https://doi.org/10.1007/s10464-010-9419-5. [PubMed]