The present quasi-experiment examined the direct and indirect effects of recovery support telephone calls following adolescent substance use disorder treatment. Six-month outcome data from 202 adolescents who had received recovery support calls from primarily pre-professional (i.e., college-level social service students) volunteers was compared to 6-month outcome data from a matched comparison sample of adolescents (n = 404). Results suggested adolescents in the recovery support sample had significantly greater reductions in their recovery environment risk relative to the comparison sample (β = -.17). Path analysis also suggested that the reduction in recovery environment risk produced by recovery support calls had indirect impacts (via recovery environment risk) on reductions in social risk (β = .22), substance use (β = .23), and substance-related problems (β = .16). Finally, moderation analyses suggested the effects of recovery support calls did not differ by gender, but were significantly greater for adolescents with lower levels of treatment readiness. In addition to providing rare empirical support for the effectiveness of recovery support services, an important contribution of this study is that it provides evidence that recovery support services do not necessarily have to be “peer-based,” at least in terms of the recovery support service provider having the experiential credentials of being “in recovery.” If replicated, this latter finding may have particularly important implications for helping increase the recovery support workforce.
Adolescent; Recovery support; Substance use; Volunteers
This study evaluated a process for training raters to reliably rate clinicians delivering the Adolescent Community Reinforcement Approach (A-CRA) in a national dissemination project. The unique A-CRA coding system uses specific behavioral anchors throughout its 73 procedure components. Five randomly-selected raters each rated “passing” and “not passing” examples of the 19 A-CRA procedures. Ninety-four percent of the final ICCs were at least ‘good’ (≥.60) and 66.7% were ‘excellent’ (≥.75), and 95% of the ratings exceeded the 60% or better agreement threshold between raters and the gold standard. Raters can be trained to provide reliable A-CRA feedback for large-scale dissemination projects.
Evidence-based treatment; A-CRA; community reinforcement; rater training; coder training; treatment adherence
This study examined the extent to which changes over time in clinicians’ responses to measures of work attitude (eg, job satisfaction) and psychological climate (eg, supervisor support) could predict actual turnover and turnover intentions above and beyond absolute levels of these respective measures. Longitudinal data for this study were collected from a sample of clinicians (N = 96) being trained to implement an evidence-based treatment for adolescent substance use disorders. Supporting findings from a recent staff turnover study, we found job satisfaction change was able to predict actual turnover above and beyond average levels of job satisfaction. Representing new contributions to the staff turnover literature, we also found that change over time in several other key measures (eg, job satisfaction, role manageability, role clarity) explained a significant amount of variance in turnover intentions above and beyond the absolute level of each respective measure. A key implication of the current study is that organizations seeking to improve their ability to assess risk for staff turnover may want to consider assessing staff at multiple points in time in order to identify systematic changes in key employee attitudes like turnover intentions and job satisfaction.
turnover; retention; staff; workforce
Although evidence-based treatments are considered the gold standard for clinical practice, it is widely recognized that evidence-based treatment implementation in real world practice settings has been limited. To address this gap, the federal government provided three years of funding, training and technical assistance to 84 community-based treatment programs to deliver an evidence-based treatment called the Adolescent-Community Reinforcement Approach (A-CRA). Little is known about whether such efforts lead to long-term A-CRA sustainment after the initial funding ends.
We will use a longitudinal mixed method data analytic approach to characterize sustainment over time and to examine the factors associated with the extent to which A-CRA is sustained. We will use implementation data collected during the funding period (e.g., organizational functioning, staff certification rates and penetration) and supplement it with additional data collected during the proposed project period regarding implementation quality and the hypothesized predictors of sustainment (i.e., inner and outer contextual variables) collected over three waves from 2013 to 2015 representing program sustainment up to five years post-initial funding.
Gaining a better understanding of the factors that influence the evidence-based treatment sustainment may lead to more effective dissemination strategies and ultimately improve the quality of care being delivered in community-based addiction treatment settings.
Sustainment; Evidence-based treatments; Adolescent substance use treatment; Mixed methods; Longitudinal data analyses
The turnover of substance use disorder (SUD) treatment staff has been assumed to adversely impact treatment effectiveness, yet only limited research has empirically examined this assumption. Representing an extension of prior organizational-level analyses of the impact of staff turnover on client outcomes, this study examined the impact of SUD clinician turnover on adolescent treatment outcomes using a client perspective. Multilevel regression analysis did reveal that relative to those adolescents who did not experience clinician turnover, adolescents who experienced both direct and indirect clinician turnover reported a significantly higher percentage of days using alcohol or drugs at 6-month follow-up. However, clinician turnover was not found to have significant associations (negative or positive) with the other five treatment outcomes examined (e.g., substance-related problems, involvement in illegal activity). Thus, consistent with our prior findings, the current study provides additional evidence that turnover of SUD clinicians is not necessarily associated with adverse treatment outcomes.
adolescent; substance abuse; treatment; staff; turnover; retention
Relative to the broader industrial-organizational (I-O) psychology field, research on the turnover of substance use disorder (SUD) treatment staff is in its infancy. Despite its long and rich history, recent reviews of the turnover literature within I-O psychology have noted there remains considerable room for improvement. In particular, recommendations have been made for research that considers time in the turnover process and explores more distal causes of staff turnover. Addressing these gaps, this paper examined the temporal relationship between latent measures of psychological climate, work attitude, and staff turnover. Using data from 95 SUD treatment staff clustered within 29 treatment organizations, multilevel discrete-time survival analyses revealed that a latent measure of work attitude (e.g., job satisfaction, pay satisfaction, turnover intentions) fully mediated the temporal relationship between latent measures of psychological climate (e.g., supervisor support, coworker support, role conflict) and subsequent staff turnover.
staff; employee; turnover; retention; multilevel; mediation
The National Institutes of Health (NIH) data harmonization project on existing measures (www.phenx.org) has recommended the Global Appraisal of Individual Needs (GAIN)—Short Screener (GSS) as one of the most reliable, valid, efficient, and inexpensive general behavioral health screeners to quickly identify people with internalizing and externalizing mental health disorders, substance use disorders, and crime/violence problems. The present study examined how well the four GSS screeners and their sum predict future arrest or incarceration among individuals entering treatment for a substance use disorder. Using a cross-validation design, a diverse sample of 6,815 youth with substance use disorders was split into a development sample and a validation sample. Overall, results found the GSS’s crime and violence screener (CVScr) and the substance disorder screener (SDScr) to be the two best predictors of arrest/incarceration within the 12 months following treatment intake. Additionally, we found that these screeners could be used to categorize individuals into three groups (low risk, moderate risk, high risk) and this simplified classification had good predictive validity (Area Under the Curve = 0.601). In sum, the GSS’s predictive validity was similar to other instruments that have been developed to predict risk for recidivism; however, the GSS takes only a fraction of the time to collect (ie, approximately 2–3 minutes for just these two screeners).
juvenile justice; recidivism; substance abuse treatment; crime; violence
For adolescents, substance use disorder (SUD) treatment outcomes (e.g., abstinence, problematic behaviors) often cannot be measured soon enough to influence treatment trajectory. Although process measures (e.g., treatment engagement) can play an important role, it is essential to demonstrate their association with outcomes. This study explored the extent to which engagement in outpatient treatment was associated with outcomes and whether demographic/clinical characteristics moderated these relationships.
This is a prospective study of adolescents (N=1,491) who received outpatient treatment for SUDs at one of 28 treatment sites taking part in a national evidence-based practice implementation initiative. Information from the Global Appraisal of Individual Needs interviews at intake and six-month follow-up, as well as encounter data, were used. Adjusted hierarchical logistic models were used to estimate effects of engagement on six-month outcomes.
Sixty-one percent of adolescents engaged in outpatient treatment. Adolescents engaging in treatment had significantly lower likelihoods of reporting any substance use (OR 0.60, 95% CI 0.41, 0.87), alcohol use (OR 0.63, 95% CI 0.45, 0.87), heavy alcohol use (OR 0.53, 95% CI 0.33, 0.86), and marijuana use (OR 0.64, 95% CI 0.45, 0.93). This association of engagement with abstinence outcomes was not limited to any particular group. Treatment engagement, however, was not associated with adolescents' self-report of illegal activity or trouble controlling behavior at follow-up.
At the individual level, the Washington Circle engagement measure was a predictor of some positive outcomes for adolescents in outpatient treatment. Efforts to better engage adolescents in treatment could improve quality of care.
performance measures; outcomes; adolescent substance abuse treatment; treatment engagement
Although several costing instruments have been previously developed, few have been validated or applied systematically to the delivery of evidence-based practices (EBPs). Using data collected from 26 organizations implementing the same EBP, this paper examined the reliability, validity, and applicability of the brief Treatment Cost Analysis Tool (TCAT-Lite). The TCAT-Lite demonstrated good reliability—correlations between replications averaged 0.61. Validity also was high, with correlation of treated episodes per $100,000 between the TCAT-Lite and independent data of 0.57. In terms of applicability, cost calculations found that if all organizations had operated at optimal scale (124 client episodes per year), existing funds could have supported 64% more clients.
treatment; adolescent; cost; efficiency; cost function; reliability; validity
This study examined client and program factors predicting initiation and engagement for 2,191 adolescents at 28 outpatient substance abuse treatment sites implementing evidence-based treatments. Using Washington Circle criteria for treatment initiation and engagement, 76% of the sample initiated, with 59% engaging in treatment. Analyses used a 2-stage Heckman probit regression, accounting for within-site clustering, to identify factors predictive of initiation and engagement. Adolescents treated in a pay-for-performance (P4P) group were more likely to initiate, whereas adolescents in the race/ethnicity category labeled other (Native American, Asian, Pacific Islander, Native Alaskan, Native Hawaiian, mixed race/ethnicity), or who reported high truancy, were less likely to initiate. Race/ethnicity groups other than Latinos were equally likely to engage. Among White adolescents, each additional day from first treatment to next treatment reduced likelihood of engagement. Although relatively high initiation and engagement rates were achieved, the results suggest that attention to program and client factors may further improve compliance with these performance indicators.
Adolescent substance abuse treatment; Performance measures; Treatment engagement; Evidence-based practices; Pay-for-performance
Within the context of an initiative to implement evidence-based practices (EBPs) for adolescents with substance use disorders, this study examined the extent to which staff factors measured at an initial EBP training workshop were predictive of EBP-competence and turnover status of staff (N=121) measured 6, 9, and 12 months post-training. By the final assessment point, 52.3% of staff transitioned to the Employed/EBP-Competent category, 26.6% transitioned to the Not Employed/Not EBP-Competent category, 4.6% transitioned to the Not Employed/EBP-Competent category, and 16.5% had not transitioned out of the initial category. Multilevel multinomial regression analysis identified several measures that were significant predictors of staff transitions to the Not Employed/Not EBP-Competent category (e.g., program needs, job satisfaction, burnout) and transitions to the Employed/EBP-Competent category (e.g., months in position, pressures for change, influence). Findings have implications for the development and testing of strategies to train and retain staff to deliver EBPs in practice settings.
Adolescent; treatment; training; implementation; turnover; fidelity; competence
High staff turnover has been described as a problem for the substance use disorder treatment field. This assertion is based primarily on the assumption that staff turnover adversely impacts treatment delivery and effectiveness. This assumption, however, has not been empirically tested. In this study, we computed annualized rates of turnover for treatment staff (n=249) participating in an evidence-based practice implementation initiative and examined the association between organizational-level rates of staff turnover and client-level outcomes. Annualized rates of staff turnover were 31% for clinicians and 19% for clinical supervisors. Additionally, multilevel analyses did not reveal the expected relationship between staff turnover and poorer client-level outcomes. Rather, organizational-level rates of staff turnover were found to have a significant positive association with two measures of treatment effectiveness: less involvement in illegal activity and lower social risk. Possible explanations for these findings are discussed.
substance abuse counselors; clinical supervisors; turnover; retention; workforce
This paper examined the extent to which assignment to a pay-for-performance (P4P) experimental condition impacted therapists’ intentions to deliver high quality treatment and the extent to which therapists’ intentions could be explained by the theory of planned behavior. Data were collected from 95 therapists who agreed to participate in a P4P experiment related to their implementation of an evidence-based treatment (EBT) for adolescents with substance use problems. Relative to those in the control condition, therapists in the P4P condition reported significantly greater intentions to achieve monthly competence (B = 1.41, p < .001) and deliver a targeted threshold level of treatment to clients (B = 1.31, p < .001). Additionally, therapists’ intentions could be partially explained by the theory of planned behavior. Meta-analyses have found intentions to be one of the best predictors of behavior, thus these findings provide initial support for using P4P approaches as a method of increasing the quality of substance use treatment.
adolescents; intentions; pay-for-performance; substance use treatment
Multiple evidence-based treatments for adolescents with substance use disorders are available; however, the diffusion of these treatments in practice remains minimal. A dissemination and implementation model incorporating research-based training components for simultaneous implementation across 33 dispersed sites and over 200 clinical staff is described. Key elements for the diffusion of the Adolescent Community Reinforcement Approach and Assertive Continuing Care were: (a) three years of funding to support local implementation; (b) comprehensive training, including a 3.5 day workshop, bi-weekly coaching calls, and ongoing performance feedback facilitated by a web tool; (c) a clinician certification process; (d) a supervisor certification process to promote long-term sustainability; and (e) random fidelity reviews after certification. Process data are summarized for 167 clinicians and 64 supervisors.
technology transfer; clinical training; substance abuse treatment; adolescent; A-CRA; ACC
This study evaluated the effectiveness and cost-effectiveness of two types of outpatient treatment with and without Assertive Continuing Care (ACC) for 320 adolescents with substance use disorders. Study participants were randomly assigned to one of four conditions: (a) Chestnut’s Bloomington Outpatient Treatment (CBOP) without ACC; (b) CBOP with ACC; (c) Motivational Enhancement Therapy/Cognitive Behavior Therapy-7 session model (MET/CBT7) without ACC; and (d) MET/CBT7 with ACC. All study conditions attained high rates of participant engagement and retention. Follow-up interviews were completed with over 90% of the adolescents at three, six, nine, and twelve months after treatment admission. There was a significant time by condition effect over 12 months, with CBOP having a slight advantage for average percentage of days abstinent. Unlike previous findings that ACC provided incremental effectiveness following residential treatment, there were no statistically significant findings with regard to the incremental effectiveness of ACC following outpatient treatment. Analysis of the costs of each intervention combined with its outcomes revealed that the most cost-effective condition was MET/CBT7 without ACC.
substance abuse; adolescent outpatient treatment; continuing care; outcomes
Ensuring evidence-based treatments are delivered with a high degree of fidelity is an important aspect of transporting these practices to community-based treatment providers. Just as training is critical for clinicians who plan to deliver the Adolescent Community Reinforcement Approach (A-CRA), a process of training and vetting individuals who assess sessions for fidelity also is critical. This article describes the training process for session raters who assess fidelity of A-CRA during the clinician training and certification process. A-CRA is currently being implemented in 32 independent community-based agencies as part of a large initiative funded by the Center for Substance Abuse Treatment.
training; treatment fidelity; adolescent; treatment; A-CRA
The current study examined the predictive validity of the Washington Circle (WC) continuity of care after long-term residential treatment performance measure, as well as the impact of assertive continuing care interventions on achieving continuity of care. This measure is a process measure that focuses on timely delivery of a minimal floor of services that are necessary to provide sufficient quality of treatment but should not be construed to be the optimal continuity of care after residential treatment for any specific adolescent. Participants included 342 adolescents who were admitted to long-term residential treatment and randomly assigned to either standard continuing care or an assertive continuing care condition. Overall, results provide initial support for the WC continuity of care after residential treatment performance measure as a useful predictor of 3-month recovery status. Additionally, assignment to an assertive continuing care condition was found to significantly increase the likelihood of achieving continuity of care.
adolescent; continuing care; substance use treatment; performance measures; Washington Circle
Data from 399 adolescents, who participated in one of four randomly controlled trials of the A-CRA intervention, were used to examine the extent to which exposure to Adolescent Community Reinforcement Approach (A-CRA) procedures mediated the relationship between treatment retention and outcomes. Although zero-order correlations indicated that retention in treatment was a significant predictor of AOD use (r = -.18, p < .001), this relationship was reduced to non-significance (p = .39) when exposure to A-CRA procedures was included in the model. Overall, the final model evidenced a very good fit (RMSEA=.00; NFI = .99; CFI = 1.00), and explained 29% and 43% of the variance in adolescents’ post-treatment AOD use and AOD-related problems, respectively. Additionally, CHAID analysis was used to derive a target level of A-CRA exposure, which was found to be significantly related to being in recovery at follow-up. The current findings are useful, as little research to date has identified significant mediators of the relationship between retention and treatment outcomes or identified target thresholds of treatment exposure.
adolescent; treatment; exposure; fidelity; substance use
Rewarding provider performance has been recommended by the Institute of Medicine as an approach to improve the quality of treatment, yet little empirical research currently exists that has examined the effectiveness and cost-effectiveness of such approaches. The aim of this study is to test the effectiveness and cost-effectiveness of providing monetary incentives directly to therapists as a method to improve substance abuse treatment service delivery and subsequent client treatment outcomes.
Using a cluster randomized design, substance abuse treatment therapists from across 29 sites were assigned by site to either an implementation as usual (IAU) or pay-for-performance (P4P) condition.
Substance abuse treatment therapists participating in a large dissemination and implementation initiative funded by the Center for Substance Abuse Treatment.
Therapists in both conditions received comprehensive training and ongoing monitoring, coaching, and feedback. However, those in the P4P condition also were given the opportunity to earn monetary incentives for achieving two sets of measurable behaviors related to quality implementation of the treatment.
Effectiveness outcomes will focus on the impact of the monetary incentives to increase the proportion of adolescents who receive a targeted threshold level of treatment, months that therapists demonstrate monthly competency, and adolescents who are in recovery following treatment. Similarly, cost-effectiveness outcomes will focus on cost per adolescent receiving targeted threshold level of treatment, cost per month of demonstrated competence, and cost per adolescent in recovery.
Trial Registration Number: NCT01016704