|Home | About | Journals | Submit | Contact Us | Français|
To determine the effectiveness of brief strategic family therapy (BSFT; an evidence-based family therapy) compared to treatment as usual (TAU) as provided in community-based adolescent outpatient drug abuse programs.
A randomized effectiveness trial in the National Drug Abuse Treatment Clinical Trials Network compared BSFT to TAU with a multiethnic sample of adolescents (213 Hispanic, 148 White, and 110 Black) referred for drug abuse treatment at 8 community treatment agencies nationwide. Randomization encompassed both adolescents’ families (n = 480) and the agency therapists (n = 49) who provided either TAU or BSFT services. The primary outcome was adolescent drug use, assessed monthly via adolescent self-report and urinalysis for up to 1 year post randomization. Secondary outcomes included treatment engagement (≥2 sessions), retention (≥8 sessions), and participants’ reports of family functioning 4, 8, and 12 months following randomization.
No overall differences between conditions were observed in the trajectories of self-reports of adolescent drug use. However, the median number of days of self-reported drug use was significantly higher, χ2(1) = 5.40, p < .02, in TAU (Mdn = 3.5, interquartile range [IQR] = 11) than BSFT (Mdn = 2, IQR = 9) at the final observation point. BSFT was significantly more effective than TAU in engaging, χ2(1) = 11.33, p < .001, and retaining, χ2(1) = 5.66, p < .02, family members in treatment and in improving parent reports of family functioning, χ2(2) = 9.10, p < .011.
We discuss challenges in treatment implementation in community settings and provide recommendations for further research.
Family interventions have been shown to be efficacious with drug using youth (for reviews see Sexton, Robbins, Holliman, Mease, & Mayorga, 2003; Stanton & Shadish, 1997; Waldron & Turner, 2008). The positive effects of family therapy have been remarkably consistent across both studies and clinical models with respect to engagement and retention in treatment, adolescent drug use, and family functioning (Ozechowski & Liddle, 2000; Stanton & Shadish, 1997; Waldron & Turner, 2008; Williams & Chang, 2000).
Based on this body of research, family interventions have been disseminated widely into drug abuse, mental health, and juvenile justice treatment settings. The promise of the effectiveness of empirically based family interventions in real-world settings is supported by research (Dennis et al., 2004) in which family therapy performed at least as well as other empirically based treatments, such as individual and group motivational interviewing and cognitive behavioral interventions. However, few studies have examined the effectiveness of family therapy in real-world settings, and more studies are necessary to determine whether empirically based family therapy is significantly more effective than the standard services that adolescents receive in community settings.
The focus of this study was to determine if an empirically based family therapy, the brief strategic family therapy (BSFT) model, is more effective than treatment as provided in community-based adolescent outpatient drug abuse programs. This study was carried out within the context of the National Institute on Drug Abuse’s National Drug Abuse Treatment Clinical Trials Network (CTN). The research design and plan of analyses were reviewed and approved by the CTN’s independent protocol review and data safety and monitoring committees and National Institutes of Health CTN staff. Below, we specifically note the planned design and analyses, deviations from the research plan, and additional analyses that were conducted to examine relevant clinical outcomes. Planned analyses were independently replicated by the sponsor’s statistical coordinating center.
The BSFT model (Szapocznik, Hervis, & Schwartz, 2003; Szapocznik & Kurtines, 1989) has been shown to be an efficacious intervention for adolescents with drug use and related behavior problems. The primary goal of the BSFT intervention is to reduce adolescent behavior problems by improving within-family relationships and relationships between family members and other important systems (e.g., school, peers) that influence the youth’s behaviors and to widen the reach of services by increasing engagement rates. Three decades of research have supported the efficacy of the BSFT model with minority families in Miami, Florida. This research has demonstrated the positive effects of the BSFT model in engaging and retaining adolescents/family members in treatment (Coatsworth, Santisteban, McBride, & Szapocznik, 2001; Santisteban et al., 1996; Szapocznik et al., 1988), reducing adolescent drug use (Santisteban et al., 2003), and improving family functioning (Santisteban et al., 2003; Szapocznik et al., 1989).
In prior efficacy studies, the BSFT intervention was conducted by therapists who were rigorously selected, and members of the research team managed the therapists’ time and provided clinical supervision. Family participants were almost exclusively of Hispanic descent. Thus, the effectiveness of the BSFT model in real-world settings with an essentially unselected sample of therapists from community agencies and with a racially and ethnically diverse sample of adolescents and family members has not yet been established. The purpose of this study was to extend research on the BSFT intervention by examining the effectiveness of the BSFT intervention, compared to treatment as usual (TAU), in community drug abuse treatment agencies with a racially/ethnically diverse sample of adolescent drug users.
On the basis of the results of prior efficacy studies, we hypothesized that the BSFT model would be significantly more effective than TAU in (a) engaging and retaining adolescents in treatment, (b) reducing adolescent drug use, and (c) improving family functioning.
Adolescents and their parents were enrolled in a randomized trial comparing the BSFT model to TAU at each of eight community treatment providers. Services in both conditions were delivered by therapists at community agencies. Therapists were randomly assigned to treatment condition and thus nested within condition. Adolescent drug use was assessed at baseline and at 12 monthly follow-up assessments. All additional adolescent and family assessments were completed at baseline and at 4, 8, and 12 months post-randomization.
The BSFT intervention is an integrative model that combines structural and strategic family therapy theory and intervention techniques to address systemic/relational (primarily family) interactions that are associated with adolescent substance use and related behavior problems. The structural components of the BSFT treatment draw on the work of Minuchin (Minuchin, 1974; Minuchin & Fishman, 1981), and the strategic aspects of BSFT treatment were influenced by Haley (1976) and Madanes (1981). The BSFT intervention is a problem-focused, directive, and practical approach, following a prescribed format. Interventions are organized into four domains to facilitate change and are planned and delivered in treatment phases to achieve specific goals at different times during treatment. Early sessions are characterized by joining interventions that are intended to establish a therapeutic alliance with each family member and with the family as a whole. This requires that the therapist demonstrate acceptance of and respect toward each individual family member as well as the way in which the family as a whole is organized. Early sessions also include tracking and diagnostic enactment interventions that are designed to systematically identify family strengths and weaknesses and develop a treatment plan. Reframing interventions are used to reduce family conflict and create a motivational context (e.g., sense of hope or possibility) for positive change. Over the course of treatment, therapists are expected to continue to maintain an effective working relationship with family members (joining), to facilitate within-family interactions (tracking and diagnostic enactment), and to directly address negative affect/beliefs and family interactions. However, the focus of treatment shifts to implementing restructuring strategies to transform family relations from problematic to effective and mutually supportive, and these include (a) directing, redirecting, or blocking communication; (b) shifting family alliances; (c) helping families develop conflict resolution skills; (d) developing effective behavior management skills; and (e) fostering parenting and parental leadership skills.
The BSFT intervention was expected to include 12 to 16 sessions over a 4-month period. The length of service is based on the therapist’s ability to achieve necessary improvements in specific behavioral criteria (e.g., drug use) and the severity of family problems. Sessions were approximately 1 hr and were expected to involve multiple family members. Whenever appropriate, other systems were addressed either as content within the sessions or were included in the session. For example, parents were coached on how to communicate with school personnel or a probation officer. Participation in generally available agency-based ancillary services (e.g., case management, Alcoholics Anonymous, etc.) was also permitted. However, in the BSFT condition, over 97% of the sessions were classified as family therapy. The BSFT sessions including the adolescent and (a) one family member represented 22% of sessions; (b) two family members, 24%; (c) three, 22%; (d) four, 18%; and, (e) five, 14%. Finally, location of services was flexible to ensure that location was not an obstacle to the delivery of the BSFT intervention. Locations included home (52.2%) and clinic (45.3%), as well as school, work, or other (2.5%).
By including a TAU comparison condition, this study was designed to examine the public health question of whether the BSFT model is more effective than standard agency services in reducing adolescent drug use. TAU varied across participating community agencies and included individual and/or group therapy, parent training groups, non-manualized family therapy, and case management. All agencies were expected to provide at least one intervention session per week. Participation in ancillary services (e.g., case management, Alcoholics Anonymous, etc.) was typical. No agencies were excluded because they met the manualized family therapy exclusion criterion.
According to the participating agencies, booster sessions were a common aspect of clinical practice, which reflected the reality that many youth leave and reenter the treatment agency within a 1-year period. To allow both conditions to be approximately parallel in sessions allowed, both conditions permitted booster sessions. During the study, 6% (i.e., 32 [17 in BSFT; 15 in TAU]) of youth/families received a booster session, with more than half of these cases (18 of 32) receiving only one or two booster sessions. On average, the number and timing of booster sessions were not significantly different across the two conditions. The average number of sessions was 2.7 in BSFT (ranging from 1 to 11) and 3.2 in TAU (ranging from 1 to 9).
The study was designed to ensure that participants in TAU were expected to receive at a minimum at least as many sessions as participants in the BSFT condition. A prerequisite for participation was that program managers expected TAU to include at least 12–16 scheduled sessions over a 3- to 4-month period.
Randomization (therapist and family participants) occurred at eight outpatient community treatment providers: La Frontera (Tucson, Arizona); The Crossroads Center (Cincinnati, Ohio); The Village (Miami, Florida); Gateway Community Services (Jacksonville, Florida), Administración de Servicios de Salud Mental y Contra la Adicción (ASSMCA; Bayamón, Puerto Rico); Daymark Recovery Services (Salisbury, North Carolina); Tarzana Treatment Centers (Tarzana, California); and Arapahoe House (Denver, Colorado). Sites were volunteers from within the National Drug Abuse Treatment Clinical Trials Network (CTN). Inclusion criteria included having sufficient numbers of participants within the agency (established through review of agency records for 2 or more years prior to study participation) to enroll between 30 and 90 participants within 12 months and at least four therapist volunteers initially to ensure that two could be randomly assigned to each treatment condition.
At each agency, the typical therapist participants were four volunteer counselors from the total pool of individuals who provided clinical services. Therapists had to provide informed consent and be willing to be randomized to the BSFT condition or TAU. Although a selection process was designed to evaluate basic counseling skills (i.e., openness to learning a new intervention; the ability to convey understanding, acceptance, and respect to all family members; and directness and clarity of communication), because of a paucity of therapists at many of the community agencies, only two out of 75 therapists who consented were excluded from the study. In both instances, the therapists were excluded due to concerns about the therapists’ willingness to learn a new intervention.
Within each site, therapist randomization was conducted within therapist pairs that were balanced to the extent possible on academic degree and years of clinical experience. At two sites, therapists were also balanced on language (i.e., Spanish) to include Spanish-speaking participants in each condition.
We required that at least two therapists per condition be randomized at each site. We did this for two reasons. First, we were concerned that not all therapists would be certified to provide study services in BSFT. Second, we wanted to minimize the extent to which therapist effects were solely responsible for outcomes at sites. Therapist turnover, however, did create some challenges over the course of the study. Of the 49 therapists who provided study services, 5 of 20 (25%) in BSFT and 7 of 29 (24%) in TAU left the agency prior to the conclusion of the study and thus dropped out of the study. When a therapist dropped out of the study, we consented and randomized another pair of therapists. Note that in some circumstances, one therapist in the replacement pair was randomized out of the study, while the other was randomized to the treatment condition of the dropout therapist. This procedure was used in all instances in which only one replacement therapist was needed, irrespective of treatment condition, and was implemented to avoid having to train a therapist in the BSFT model (which was time intensive) when it was not necessary. Therapists randomized into a study condition were included in the 49 who were included in the analyses.
Participants were recruited from all new 13- to 17-year-old referrals to the agency. Typically, intake staff members were trained to identify potential participants and to provide a basic overview of the study to parents and youth. Whenever possible, a research assistant was immediately available to present a more thorough review of the study to interested parents. For example, we encouraged considerable communication between research staff and agency staff to schedule research assistants to be available at times when agency intakes were scheduled.
To enroll in the study, adolescent participants had to self-report use of illicit drugs (other than alcohol and tobacco) within the 30-day period that preceded baseline or had to have been referred from an institution (e.g., detention, residential treatment) for the treatment of drug abuse. Hence, many participants were not actively using drugs within the 30-day period prior to randomization. Adolescents and parents had to provide assent/consent. Further eligibility criteria included living with a family (defined to include any parental/adult guardian, except foster) and in the same geographical area as the agency (because home-based services were expected). Adolescents with current or pending severe criminal offences that would likely result in incarceration were excluded from participation to ensure availability for follow-up. The study was approved for prisoner involvement by the University of Miami institutional review board and the institutional review boards at participating sites and universities.
Families were randomized within site to the BSFT or TAU conditions following the completion of the baseline assessment using an urn randomization procedure (Wei & Lachin, 1988). The urn randomization approach was used to increase the probability that participants in the treatment conditions would be balanced on ethnicity/race (Hispanic, African American/Black, and White, non-Hispanic plus other) and level of drug use (any vs. no drug abuse/dependence diagnosis) at baseline. Research assistants performed the randomization through an automated telephone system programmed and run by the Veteran’s Administration.
A demographic questionnaire was used to gather information about household composition, household income, age, race and ethnicity, gender, and age. Multiple endorsements on racial/ethnic categories were allowed. For the purpose of analysis, if adolescents endorsed Hispanic they were classified as Hispanic, and if they endorsed Black but not Hispanic they were classified as Black, non-Hispanic. An adolescent was classified as White, non-Hispanic only if White was the only race/ethnicity endorsed. All remaining adolescents were classified as other.
To ensure privacy and confidentiality of information collected by the agency, dose was tracked through monthly therapists’ interviews. Therapists reported, using the participant’s clinical charts and billing data, on recommended dose, the clinical status of cases, and the number of sessions that had actually been delivered since the last interview. Total dose was constructed as the sum of all therapy sessions (individual, group, and family) conducted by any therapist (not just the study therapist) at the agency, including booster sessions. In all analyses and discussion, dose refers to total dose.
For planned analyses, failure to engage and failure to retain in therapy were constructed as binary variables, based on the adolescent’s attendance in therapy sessions of any type. The criteria for each variable were consistent across treatment conditions. A participant was classified as a failure to engage if there was zero or one session with the adolescent. A participant was classified as failure to retain if there were seven or fewer sessions with the adolescent. The number of sessions defining engagement and retention were obtained from prior research (Robbins et al., 2008; Santisteban et al., 1996).
The Timeline Follow-Back (TLFB; Sobell & Sobell, 1992) was used to assess adolescent drug use. The TLFB yields consistently high test–retest correlations (Mason, Cauce, Gonzales, Hiraga, & Grove, 1994). The TLFB was administered at 13 points (baseline and 12 monthly follow-up assessments) to establish a pretreatment rate of use and 365 continuous days of data on daily drug use after randomization. The TLFB was administered by research assistants who were blind to treatment condition. Research assistants were requested to indicate if the blind was broken at each of the 12 follow-up assessments. Only with 1.2% of participants did research assistants note that the blind was broken.
The dependent variable for the first analysis was the percentage of drug use days in the 28-day periods. If there were acceptable data on at least 14 days of reporting for a 28-day period, the observed percentage of drug use days out of the observed number of days was used as the dependent variable. If there were more than 14 days missing from any 28-day period, the percentage of drug use days was set to missing. A planned secondary analysis examined the median days of use for the final 28-day period.
Urine drug screens were conducted at baseline and all monthly follow-up assessments using the SureStep Drug Screen Card 10A and urine cups, which included temperature-controlled monitoring and detection of adulterants. Urine drug screens were administered immediately prior to the administration of the TLFB to improve the chances of accurate reporting of days of drug use. Urine screens, like all other data collected by the research assistants, were used for research purposes only. Neither the results of these screens, nor any other data, were shared with therapists, adolescents, or family members.
The computerized generic Diagnostic Interview Schedule for Children (DISC) was used to diagnose drug abuse or dependence (Shaffer et al., 1996) in the past year. The DISC is a highly structured diagnostic interview designed for use by non-clinicians to assess mental health diagnosis. Test–retest reliability and validity of the DISC 2.3 have been moderate to good in multiple samples (Schwab-Stone et al., 1996). The DISC 2.3 has been shown to be in high agreement (ranging from 0.69 to 0.99) with the International Statistical Classification of Diseases and Related Health Problems (10th ed., ICD–10), the Diagnostic and Statistical Manual of Mental Disorders (3rd ed., rev.; DSM–III–R), and the Diagnostic and Statistical Manual of Mental Disorders (4th ed.; DSM–IV) (Hasin, Van Rossem, McCloud, & Endicott, 1997). The presence or absence of any drug abuse or dependence diagnoses was used as a stratification variable in the urn randomization procedure.
Two BSFT theoretically relevant measures of family functioning were obtained. First, the Parenting Practices Questionnaire from the Chicago Youth Development Study was used to identify parenting practices (Gorman-Smith, Tolan, Zelli, & Huesmann, 1996). Adolescent and parent responses to 47 items were used to identify positive and negative parenting behaviors in four domains: (a) positive parenting, (b) discipline effectiveness, (c) avoidance of discipline, and (d) monitoring. Discipline effectiveness and avoidance of discipline were assessed only of the parents. Internal consistency reliabilities of each of the subscales ranged from .68 to .81. Second, the Family Environmental Scale (Moos & Moos, 1986) was used to measure (a) cohesion and (b) conflict. Internal consistency reliability estimates for the subscales ranged from .61 to .78. The Conflict and Cohesion subscales were administered to both parents and adolescents.
The dependent measure in analyses of family functioning was a composite scale created from the Parenting Practices Questionnaire and the Family Environment Scale based on the results of a confirmatory factor analysis of a larger measurement model (Feaster et al., 2010). Feaster et al. (2010) conducted a confirmatory factor analysis of the adolescent-reported and parent-reported indicators of the overall family functioning composites that fit the data well (comparative fit index [CFI] = .95, root-mean-square error of approximation [RMSEA] = .07), while the confirmatory factor analysis that separated out the two constructs of family functioning, family relations and parenting, did not fit the data well (CFI = .664, RMSEA = .174). The individual subscales associated with the domains reported above were converted to z scores and summed. The resulting composites (one each for adolescent and parent report) were normalized to have standard deviations of 1 (across condition) at baseline and had good reliability for youth (α =.90) and parents (α = .91).
Before working with study cases, therapists assigned to the BSFT condition received approximately 96 hr of training delivered in four 3-day workshops, attended 48 hr of group weekly supervision sessions over a 5-month period, and typically had two to four pilot cases before starting to see study cases. For two of the five therapists in BSFT, it was necessary to accelerate the delivery of the workshops to avoid delays in recruitment at the site. In both instances, the materials were delivered to the therapist in two workshops over a 1-month period. However, the actual time spent in training was comparable to the training provided to other therapists who received the four 3-day workshops over several months. These two therapists were still required to work with four pilot cases and to achieve certification-level criteria, as established by an expert group of raters, before providing services to study cases.
During study implementation, therapists participated in a 3-hr weekly group supervision session with an expert clinical supervisor. All supervision sessions were conducted by telephone. All BSFT therapy sessions were recorded. Copies of recordings were sent to the University of Miami for adherence rating and clinical supervision. Every week, randomly selected sessions from each therapist were rated by independent raters. A total of 905 ratings were completed on study cases (which corresponds to approximately 37% of the sessions in BSFT). Adherence ratings captured the frequency with which therapists engaged in 20 prescribed interventions. Ratings were completed using a 5-point scale, ranging from 1 ( poor) to 5 (excellent), with a score of 3 representing a minimally acceptable level of adherence. Scores on the scale were generated primarily on the frequency with which interventions were present in the session, with lower scores indicating that interventions were rarely or inconsistently present and higher scores indicating that interventions were consistently or always present. Interrater reliability was .83, ranging from .81 to .85 across the four domains (joining, tracking and diagnostic enactment, reframing, and restructuring).
Recordings were not conducted in TAU to avoid altering existing practices at participating community agencies. For the same reason, data were not collected from agency clinical supervisors about therapist adherence.
Analyses were grouped into three sections: (a) engagement and retention, (b) adolescent drug use, and (c) family functioning. For primary assessment of treatment effects for each of these sections, the statistical model included random effects for site and therapists (nested within site) to account for these two levels of nesting. These models allow the inclusion of all observed data for individuals and are robust to data that are missing at random (Little & Rubin, 2002). Robustness to data missing at random means that observed data such as baseline predictors or levels of the outcome measure from prior times may predict the probability of missingness but do not need to be included in the statistical model to achieve unbiased estimates of effects. Planned analyses were conducted testing difference in intervention effects by race/ethnicity and adolescent gender. In addition, for race/ethnicity, analyses tested whether intervention effects differed across three groups, non-Hispanic White, non-Hispanic Black, and Hispanic.
The first analysis examined differences in engagement and retention using a logistic regression specification in Mplus 6 (Muthén & Muthén, 1998–2007) to predict failure to engage and failure to retain by treatment condition. This model used the two-level and complex options to incorporate the two levels of nesting (site and therapist within site). In a second analysis, contingency table methods were used to calculate the percentages of failure to engage and failure to retain in each condition as well as the risk ratio (RR), with a 95% confidence interval (CI) of being engaged or retained for the three different racial/ethnic groups and two genders. In a third analysis, the Wilcoxon rank-sum test was used to compare the conditions on actual number of therapy sessions attended and the length of time to deliver study interventions. Finally, an exploratory analysis examined the effect of gender on failure to engage and failure to retain.
It is difficult to uncover the impact of therapy dose on treatment outcomes unless participants are randomized to differing amounts of therapy (Feaster, Newman, & Rice, 2003). For example, in most psychotherapeutic interventions, dose is frequently driven by the severity of problems that clients present. As such, the most difficult cases typically require more sessions, and yet may be the most difficult to change. For this reason, instead of examining simple relationships between dose and outcome, we have included tests of differential treatment effects by engagement and retention status to determine if there are differential effects of treatment for those that get a predefined (Robbins et al., 2008; Santisteban et al., 1996) full minimal dose of treatment.
The first analysis examined the primary hypothesis that the BSFT intervention is significantly more effective than TAU in reducing self-reported adolescent drug use.1 This analysis used SAS Proc Mixed to examine differences in self-reported drug use at 4 months post-randomization (the growth curve intercept) and linear and quadratic trajectories of adolescent drug use over thirteen 28-day follow-up assessments as determined by the TLFB. In addition to the random effects associated with the components of the growth curve (intercept, linear, and quadratic) that account for the nesting of individual observations over time, additional random effects were estimated for site and therapist (within site) to account for these two additional levels of nesting. Four months post-randomization was selected a priori as the point at which to center the intercept based on the expectation that the majority of study interventions in both conditions would be delivered by this time. This analysis included planned recentering of the intercept at 12 months post-randomization to represent follow-up and maintenance of intervention effects. Due to significant skew in adolescent drug use, a modified logit transformation (log[(p + c)/(1 − p + c)] with c set at .0001; Piepho, 2003) was used to create a more normally distributed variable for analyses. The logit transformation was chosen because it normalized the residuals from the drug use model. The initial outcome analysis included planned covariates.2 Differences in trajectories were tested using likelihood ratio tests (comparing the likelihood when the BSFT interaction terms with the intercept, linear, and quadratic terms were dropped from the model and when they were included). Degrees of freedom for particular parameter estimates were obtained using a Satterthwaite approximation (Satterthwaite, 1946).
A second planned analysis examined differences between the BSFT intervention and TAU in the level of adolescent drug use at 12 months post-randomization. Given the significant skew that was observed in adolescent drug use, the median was used for this comparison. An additional post hoc analysis examined differences between the BSFT intervention and TAU in the probability of a positive urine drug screen. This analysis was estimated using generalized estimating equations (GEE) in SAS and an autoregressive within-subjects error structure. GEE models use a sandwich estimator for variances that is robust to misspecification of the within-subjects error structure. A score test is used to assess the restriction that the BSFT intervention interactions with the components of the growth trajectory all be zero.
The same principles of the analyses of adolescent drug use were used to examine family functioning, but the dependent variable was collected at baseline and at 4, 8, and 12 months post-randomization. The baseline value was not included as a covariate but rather as part of the repeated measures. In these analyses, orthonormal polynomial growth curves were used, and only linear and quadratic differences in trajectories were tested because the intercept in an orthonormal polynomial model is the between-groups effect across time. Planned analyses were also conducted to examine differences in family functioning by minority status (White or minority), race/ethnicity (African American, Hispanic, non-Hispanic White), and adolescent gender.
Power for risk difference of failure to engage and retain differs by the base rate of these variables. There is over 80% power to uncover a risk difference of 8.9% to 12.7% as the base rate varies from 10% to 50%. The risk difference range for which there is over 80% power varies from 14.5% to 19.1% in the Hispanic subgroup and from 20.8% to 26.7% in the African American subgroup. Power for the analysis on days of drug use was calculated using Optimal Design (Raudenbush & Liu, 2000), which is appropriate to calculate power in the presence of a random site by treatment interaction. Results showed that a sample of 480 would have over 80% power for a standardized difference across groups of .45. Because the preliminary specification analysis showed no evidence of a random site by treatment interaction, power is considerably higher. Using software described by Hedeker, Gibbons, and Waternaux (1999), the power for the model (without random site by treatment interaction) is over 80%, for a standardized difference at the end of treatment of .28. In the Hispanic and African American subgroups, there is over 80% power for a standardized difference of .34 and .45, respectively.
These calculations also apply to the analysis of family functioning.
Family participants were 480 adolescents and their family members. Figure 1 shows the flow of participants from screening through follow-up. There were no differences between conditions in the rates of those lost to follow-up. As shown in Table 1, adolescents were predominately male (n = 377 vs. 103 female). Based on adolescent self-reports, the sample included 213 Hispanics/Latinos (Hispanic), 148 non-Hispanic Whites, 110 non-Hispanic Blacks (African Americans), 5 American Indians/Alaskans, 2 Japanese/Whites, 1 Persian, and 1 Lebanese. Seventy-two percent of adolescents were referred for treatment from the juvenile justice system. No differences between the BSFT condition and TAU were observed on any of the variables listed in Table 1.
Therapist demographic information is presented for the 49 therapists who were randomized (BSFT, n = 20; TAU, n = 29) and provided clinical services to study participants. Therapists were 37 women and 12 men, with a mean age of 40.37 years (SD = 10.78). Therapists included non-Hispanic Whites (n = 27), non-Hispanic Blacks (n = 9), Hispanics (n = 11), Asian/Pacific Islanders (n = 1), and other, not specified (n = 1). Therapists reported an average of 8.29 years (SD = 7.42) of clinical experience. The majority of therapists had a master’s degree (n = 34), followed by those who had a bachelor’s degree (n = 8) or doctorate (n = 5) or were a certified addiction counselor (n = 1) or high school graduate (n = 1). No statistically significant differences in demographic variables were observed between therapists assigned to the BSFT condition and TAU.
Analysis of the 905 independent ratings of therapy sessions indicated that the mean of prescribed interventions in the BSFT model (M = 3.49, SD = 0.52) was above the minimum adherence threshold (3 = fair). Therapists’ mean level of adherence rating ranged from 3.12 to 4.11, suggesting minimal acceptable to adequate mean levels of adherence to BSFT across all therapists. Families’ mean level of adherence ranged from 2.00 to 4.36, with 90% of families over the minimum adherence threshold. Intraclass correlations (ICCs) showed that 6.2% to 16.3% of the variance in adherence ratings was associated with families and 1.8% to 9.8% of the variance in adherence ratings was associated with therapists across the domains rated. In these calculations, the residual variance, time within families, is the largest component, ranging from 77.3% to 88.3% of variability.
The first planned analysis revealed that participants in the BSFT condition had lower rates of failure to engage, χ2(1) = 11.33, p < .001, and failure to retain in treatment, χ2(1) = 5.66, p < .02, than TAU. These results paralleled contingency table methods that showed that the BSFT treatment was 0.43 (95% CI [0.28, 0.64]) times as likely as TAU to fail to engage a participant into therapy, and 0.71 (95% CI [0.58, 0.85]) times as likely to fail to retain participants in therapy for at least eight sessions. The rate of failure to engage in the BSFT condition was 11.4% (28/245), relative to 26.8% (63/235) in TAU. The rate of failure to retain in the BSFT condition was 40.0% (98/245), relative 56.6% (133/235) in TAU. The results for retention were mirrored in rates of unplanned terminations. BSFT treatment had 48.6% of cases with unplanned terminations and TAU had 70.2%, χ2(1) = 23.25, p < .0001.
There were no significant differences in engagement and retention between the BSFT condition and TAU by race/ethnicity (i.e., BSFT had significantly higher rates of engagement and retention within each racial/ethnic group; see Table 2). However, there were significant main effects of race/ethnicity on engagement, χ2(2) = 19.93, p < .001, and retention, χ2(2) = 7.60, p < .023, independent of treatment assignment. The contingency table analysis showed that when compared with Whites, African Americans were more likely to fail to engage (RR = 1.78, 95% CI [1.18, 2.69]) and Hispanic were less likely to fail to engage (RR = 0.58, 95% CI [0.35, 0.95]). African Americans were also more likely than Whites to fail to retain (RR = 1.30, 95% CI [1.05, 1.61]). There was not a significant difference in the rates of failure to retain between Whites and Hispanics. There were no differences in engagement or retention by gender.
Planned secondary analysis of the actual number of sessions attended also showed that the BSFT condition had significantly higher levels of attendance, χ2(1) = 6.48, p < .02. The median number of sessions attended in the BSFT condition was 9.5 (first quartile [Q1] = 3.5, third quartile [Q3] = 14) and in TAU was 6 (Q1 = 1, Q3 = 15). The pattern of median differences was consistent across all race/ethnicity groups. However, there were different patterns across gender. Boys in the BSFT condition had a significantly higher median dose than in TAU, χ2(1) = 9.25, p < .003. The median for boys in the BSFT treatment was 10 sessions (Q1 = 4, Q3 = 14) and in TAU was 5.5 sessions (Q1 = 1, Q3 = 14). There was not a significant dose difference between conditions for girls, χ2(1) = 0.11, p < .74. The median for girls in BSFT was 8 sessions (Q1 = 2, Q3 = 16) and in TAU was 7 sessions (Q1 = 3, Q3 = 17).
In an exploratory analysis, although there was a significant difference in median number of sessions by condition for boys and not for girls, there were no significant treatment differences by adolescent gender, χ2(1) = 3.83, p < .051.
Treatment duration lasted much longer than the expected 4 months. The median month of last treatment session for those participants who retained in treatment was the 8th month for each condition. However, even at the last month (12 months post-baseline) there were participants still in active treatment in the BSFT condition (10.9%) and in TAU (18.6%).
In the first planned analysis, there were no overall significant differences of treatment on the trajectories of adolescent self-reported drug use days across 28-day periods, χ2(3) = 2.1, p < .56. The linear component of the overall trajectory of drug use was not statistically significant (β = 0.05, SE = 0.05, 95% CI [−0.04, 0.14], p < .27), but the quadratic component was (β = 0.014, SE = 0.007, 95% CI [0.0003, 0.03], p < .05). There were 18 participants in TAU and 13 in the BSFT condition who were excluded from the analysis because they did not have any follow-up drug use data (see Figure 1). Baseline self-reports of drug use (β = 0.39, SE = 0.04, 95% CI [0.32, 0.47], p < .001) and having a diagnosis of drug abuse or dependence (β = 1.58, SE = 0.35, 95% CI [0.89, 2.26], p < .001) were both associated with higher self-reported drug use across time. Baseline family functioning (β = −0.08, SE = 0.03, 95% CI [−0.13, −0.02], p < .004) and living with both biological parents (β = −0.74, SE = 0.36, 95% CI [−1.45, −0.02], p < .043) were associated with lower self-reported drug use across time. There were no statistically significant differences in treatment effects within the engaged, χ2(3) = 2.00, p < .573, or the retained subgroups, χ2(3) = 4.00, p < .262. In addition, no statistically significant differences on adolescent self-reported drug use days were observed by race/ethnicity or gender. Table 3 includes the median levels of drug use days by condition across the study.
The median number of self-reported drug use days at 12 months was significantly higher in TAU (Mdn = 3.5, interquartile range [IQR] = 11) than in the BSFT condition (Mdn = 2, IQR = 9), χ2(1) = 5.40, p < .02 (see Figure 2). There were no significant differences in Median Self-Reported Drug Use Days by racial/ethnic group or gender either overall or within treatment groups.
Table 4 contains the percentage of positive urines by condition at each assessment. The growth curve model for the probability of a positive drug use screen showed no significant differences between conditions, χ2(3) = 4.29, p < .23. There were no significant race/ethnicity or gender main effects or interactions with treatment.
There were significant treatment differences in the trajectories of parent-reported family functioning, χ2(2) = 9.10, p < .011. Examination of the individual components of the trajectory showed that parents in TAU had a significant orthonormal linear component (β = 0.12, SE = 0.05, 95% CI [0.03, 0.22], p < .015), with the BSFT condition having a statistically significantly more positive orthonormal linear trajectory over time than parents in TAU (increment to linear trajectory: β = 0.17, SE = 0.07, 95% CI [0.04, 0.31], p < .014). The orthonormal quadratic component for parents in TAU was also significant (β = − 0.11, SE = 0.04, 95% CI [−0.19, −0.03], p < .006). The increment to the quadratic component for BSFT was not significant (β = 0.07, SE = 0.06, 95% CI [−0.04, 0.18], p < .207). The family functioning composite was decomposed into the subscales associated with parenting and the sub-scales associated with family environment to assess whether the results were consistent across subcomponents. The linear increment to change for BSFT was significant in both the parenting practices subcomponent (β = 0.17, SE = 0.07, 95% CI [0.02, 0.31], p < .023) and the family environment subcomponent (β = 0.15, SE = 0.06, 95% CI [0.01, 0.29], p < .033), as was the case for the full composite.
The orthonormal coding of time represents the average linear component across the times of assessment (Biesanz, Deeb-Sossa, Papadakis, Bollen, & Curran, 2004). When time is transformed to 4-month increments and the model is estimated with varying times of centering, the linear increment associated with BSFT relative TAU was significant at 8 (β = 0.11, SE = 0.04, 95% CI [0.03, 0.19], p < .006) and 12 months (β = 0.18, SE = 0.04, 95% CI [0.01, 0.36], p < .037) but not at baseline and 4 months. There was not a significant difference in the intercept (the predicted value of family functioning) at any time. The means and standard deviations of family functioning at each time by treatment condition are presented in Table 5. The model-predicted trajectories of family functioning are presented in Figure 3. There was not a significant difference in the incremental trajectories associated with BSFT relative to TAU across race/ethnicity, χ2(4) = 5.80, p < .215, or gender, χ2(4) = 0.10, p < .999. There were also no significant differences in the incremental trajectories by engaged, χ2(2) = 2.50, p < .494, or retained status, χ2(2) = 2.40, p < .494.
Adolescents in both conditions reported significant improvements over time in family functioning (linear β = 0.11, p < .005); however, no statistically significant differences between treatment conditions or treatment by race/ethnicity or gender were observed.
There was only evidence of significant impact of therapist for engagement and retention. These models used Mplus to estimate a two-level model to correct for nesting within therapist and a correction for complex sampling to correct for nesting within site. We used p < .10 to test for variance components (due to the truncation of the parameter space at zero). The variance components associated with both therapist and site were significant for both engagement, σ2 = .472, χ2(1) = 3.28, p < .070, and retention, σ2 = .605, χ2(1) = 3.52, p < .061. These correspond to ICCs for therapist and site of .180 for engagement and .220 for retention, suggesting differences in engagement and retention across sites/therapists. In an additional analysis we reestimated this model as a within-site model (by including fixed effects for site) to isolate the impact of therapist alone. The ICCs for therapist alone were .099 for engagement and .110 for retention. For self-reported drug use and family functioning, the variance components for therapist and site were estimated separately in SAS 9.1.3. The variance components for therapist were not significant in the models for self-reported drug use or for the adolescent and parent reports of family functioning. The variance component for site was statistically significant only in the self-reported drug use model, σ2 = .575, χ2(1) = 2.05, p < .076. The variance component for site in the family functioning model was so close to zero that it had to be fixed to zero.
The purpose of this study was to extend research on the BSFT intervention from controlled efficacy trials (e.g., Santisteban et al., 2003) to examine the effectiveness of this intervention in community treatment settings. The results demonstrated that the BSFT intervention was significantly more effective than TAU in engaging and retaining adolescents into treatment and improving parent-reported family functioning. However, differences between conditions were weak with respect to adolescent self-reported drug use trajectories, with differences observed only in the median level of adolescent self-reported drug use at the final observation period.
The BSFT intervention was highly effective in engaging and retaining youth in treatment. In this study, adolescents in TAU were 2.5 times more likely to fail to engage and 1.41 times more likely to fail to retain in treatment than adolescents in the BSFT condition. The BSFT treatment was significantly more effective than TAU in engaging and retaining adolescents across all racial/ethnic groups. There were, however, important main effect differences between racial/ethnic groups across the conditions in the patterns of engagement and retention. For example, consistent with prior research (Shillington & Clapp, 2003), across conditions African American adolescents showed the lowest rates of engagement and retention, with only 25% of African American youth retained in treatment in TAU and 50% in the BSFT condition.
Analyses of adolescent drug use did not reveal significant differences between the BSFT and TAU interventions in reducing the trajectories of mean days of self-reported drug use, but youth in TAU had significantly higher median levels of drug use than did those in the BSFT condition during the final (12-month) observation period. No significant median differences were observed at other assessment points; however, the number of days reflected in the median difference (3.5 in TAU vs. 2.0 in BSFT) at the 12-month assessment was small due to floor effects, raising questions about the clinical significance of this result.
While both conditions showed improvement over time for parent report of family functioning, the BSFT intervention was shown to be significantly more effective, although with a small effect, than TAU in improving parent report of family functioning. Adolescents in both conditions reported significant improvements in family functioning, with no differences observed between conditions. These findings provide support for the impact of BSFT on parent-reported family functioning but suggest that TAU may also be having a positive impact on both parent and adolescent reports of family functioning.
One limitation was the low rates in self-reported drug use at baseline and over the course of the study. Part of this may have arisen because the full range of adolescents referred for drug abuse treatment at community agencies were included in the study to enhance the potential generalizability of the study findings. In doing so, drug use at baseline varied substantially, with many youth reporting no use, which severely limited the ability to identify improvements in percentage of days used. Ironically, many of the youth who reported no drug use at baseline may have had the most severe history of behavior problems, including drug use. For example, 25% of the youth entered study treatments directly from residential treatment programs, jails/detention centers, or group homes where they had been placed for extended periods of time and where they had fewer opportunities to use drugs. These youth were nearly 5 times more likely to report no drug use (33.0%) at baseline than youth who were not referred from an institution (7.0%).
The low rates of use at baseline created a floor effect that permitted only the examination of prevention of relapse or escalation of drug use. In retrospect, different methods, albeit less reliable ones, could have been used to establish a more appropriate “baseline” estimate of drug use for these youth, for example, asking about the patterns of use in the 30 days that preceded their enrollment in a restricted environment. Unfortunately, it is difficult to make comparisons across our prior efficacy study and the current study because, for the youth in our prior study, zero use in the past 30 days of reported use really meant zero use. However, in the current sample, the zero report was due to other factors (such as number of days incarcerated or in residential treatment prior to the baseline assessment). Also, because drug use was being directly monitored by the juvenile justice system (e.g., urine screens and reporting to the court) while the adolescent was enrolled in treatment (72% were referred by the juvenile justice system), drug use may have been suppressed during the active “intervention” phase of the study. It is possible that once treatment ended and surveillance was reduced or stopped, potential differences between treatments began to emerge.
Finally, randomization of therapists within provider agencies helped to ensure similarities in therapist characteristics and a high rate of therapist inclusion (only two excluded), which strengthened the potential generalizability of study findings to the universe of drug abuse treatment providers. However, our inability to exclude therapists on the basis of their competencies occurred because the pool of available therapists was limited within site. The inclusion of a mix of therapists with varying levels of clinical sophistication, however, may have resulted in difficulties in implementing the BSFT model. Perhaps more stringent therapist selection criteria would have yielded different results. This is particularly relevant because the pattern of variability in adherence over time across families was predictive of engagement and retention in treatment, improvements in family functioning, and adolescent self-reported drug use (Robbins et al., 2011). In real-world applications, when disseminating/transporting evidence-based family systemic treatment programs to community agencies (such as the BSFT model, functional family therapy, and multisystemic therapy), considerable time and resources are dedicated to selecting therapists for training. Moreover, each of these models requires that therapists work in a dedicated unit where therapists spend a significant portion of their time implementing only the evidence-based intervention. Thus, it is possible that selecting therapists with little or no experience in family therapy, and occasionally no experience with adolescents, variable clinical sophistication, and allowing BSFT therapists to have multiple roles beyond BSFT treatment may have negatively impacted the effectiveness of the BSFT intervention. It should be noted that the initial design of the study called for randomizing units to BSFT or TAU. However, the larger number of sites that would have been needed made that design unfeasible. However, such a design would have better modeled how family-based interventions are disseminated in real-world settings.
The delivery of services in both conditions took much longer than anticipated. It was expected that the BSFT and TAU interventions would be delivered in 4 months, allowing sufficient post-treatment follow-up through 12 months post-randomization. However, the median length of treatment in both conditions was 8 months, and 15% of youth were still enrolled in services at the 12-month follow-up assessment. Although there was insufficient power to conduct these post hoc analyses, there were no significant baseline or treatment differences between the 15% of cases that were still in treatment at the end of the study and those who were not. Although our sample of community providers may not be fully representative of the universe of community agencies, it is possible that the results of this study might shed some light on the trajectory of sessions delivered in outpatient adolescent treatment settings.
In light of our difficulties in establishing a baseline level of use that accurately represented the severity of problems in the sample and in delivering treatment within the expected timeframe, we conducted a number of exploratory analyses. With respect to drug use, the following additional analyses were conducted: (a) survival analyses to examine time to first use (post-randomization) using both urine drug screens and TLFB measures, (b) comparison of rates of abstinence over the entire follow-up period, and (c) examination of differences in trajectories with the intercept recentered at the final assessment point. The results of these analyses were consistent with our primary tests, with no significant differences observed between BSFT and TAU, suggesting that the lack of differences may not be due merely to the low levels of use reported at baseline.
During the trial, we collected adherence ratings for more than 900 sessions that were randomly sampled from therapists’ caseloads each week. Adherence captured the frequency with which therapists implemented specific techniques, as identified by graduate student raters. The results of adherence ratings indicated that therapists were able to implement the clinical techniques associated with all domains of the BSFT model, with all therapists meeting on average the minimum acceptable adherence threshold of 3 to adequate (4). Moreover, both the level of adherence and trajectories of adherence to core domains of BSFT were related to engagement and retention in treatment, improvements in family functioning, and adolescent drug use at the final assessment point (Robbins et al., 2011). For example, with respect to predicting clinical outcomes, the overall level of therapist joining interventions across all phases of treatment was associated with improvements in family functioning. However, the trajectories of change in joining and restructuring were related to improvements in adolescent drug use. Specifically, positive drug use outcomes were predicted by less sharp declines in joining and a sharper increase in restructuring interventions over the course of treatment. It was not the overall level of joining (over time or early in treatment) that was critical, but rather therapists’ ability to maintain their level of joining. And, with respect to restructuring, it was the increase in directive, restructuring interventions over time that was critical. However, it is also possible that the maintenance of higher levels of joining interventions influenced other unobserved factors, such as family member alliances with the therapist, that created a context in which family members were able to benefit from the more directive, behavioral interventions that are characteristic of the restructuring process. It should be noted that while tracking and reframing were not directly related to clinical outcomes, these interventions were associated with engagement and retention in treatment. Hence, it is possible that all four sets of techniques are necessary to engage and retain individuals and thereby achieve outcomes.
The current findings provide support for the impact of the BSFT intervention on engagement, retention, and parent-reported family functioning with adolescents from diverse racial/ethnic groups. However, the weak effects on drug use outcomes, combined with the difficulties in establishing optimal implementation of the BSFT model, raise concerns about how to most successfully transport the BSFT model into community settings. Although a significant strength of the study was that the participating community sites had a high level of commitment to providing and evaluating empirically based treatments, there were numerous challenges that sites encountered in implementing study services (Robbins et al., 2010). For example, an initial challenge for sites was in identifying a pool of potential therapists available to provide services and creating space in therapists’ workload to meet study requirements. Therapists in both conditions typically had responsibilities beyond study participation, and they often carried large caseloads and had multiple roles or responsibilities in their agencies, both of which may have negatively influenced the quality of clinical services. The challenges of large caseloads were particularly burdensome for BSFT therapists, who were also responsible for attending additional training and supervision sessions, completing study-related paperwork, and conducting home visits.
The challenge community-based organizations face is that training staff to fidelity in the BSFT model or other evidence-based family treatments is a long commitment, and these organizations need to know that there is a return on this investment, both in terms of retaining therapists and making sure they are appropriately reimbursed for the services provided. The former has been addressed by BSFT, functional family therapy, and multisystemic family therapy by certifying units rather than individual therapists. The latter has been solved by managed care reimbursement schemes that provide a single fee for managing a case, rather than reimbursement on a fee-for-service basis. As with all manualized family therapy approaches, proper third-party reimbursement is an essential component of implementation and sustainability. For example, while TAU families may have had as many individual sessions as BSFT family therapy sessions, there were many more group sessions in TAU. Because groups are more profitable than family sessions, most organizations are not financially incentivized to use non-group evidence-based treatments.
It is particularly disconcerting that only one of every four African American adolescents in TAU and one of every two in the BSFT condition received at least eight sessions. These rates of engagement and retention highlight the critical need to improve African American adolescent participation in drug abuse treatment. Future research is needed to understand how to more effectively select, train, and supervise therapists and to identify the characteristics of drug treatment systems that are necessary to support high-quality implementation with this population. Clearly, BSFT was highly effective in doubling rates of engagement. Additional research is needed to identify the processes of treatment that may bring about further improvements with this population. Finally, without further research on how best to implement manualized family therapies when therapists are the unit of randomization, future outcome studies should consider testing family therapy treatment units as is typically done by functional family therapy, brief strategic family therapy, multidimensional family therapy, and multisystemic family therapy in their implementation initiatives.
This work was supported by National Institute on Drug Abuse Grant U10 DA 13720 (José Szapocznik, principal investigator). The University of Miami holds the copyright and trademark for Brief Strategic Family Therapy (BSFT). José Szapocznik is the developer of this method. The University and José Szapocznik have the potential for financial benefit from future commercialization of this method. Michael S. Robbins, who served as the University of Miami lead investigator and supervised implementation of the BSFT intervention, is now the Director of Research for Functional Family Therapy, Inc., and has the potential to benefit from the implications of this research on the future commercialization of Functional Family Therapy.
1The analytic plan was developed in collaboration with an independent group, Duke Clinical Research Institute, and was approved by the sponsor prior to the authors being able to match randomized participants to follow-up data. The Duke Clinical Research Institute team also confirmed the results of the analyses separately.
2Planned covariates were recommended by the protocol review board and included the following variables: baseline drug use, adolescent drug of choice, drug abuse or dependence diagnosis, race/ethnicity, age, gender, peer delinquency, internalizing and externalizing, family functioning, family composition, and referral from an institution (e.g., juvenile justice). Details of the covariates are available by request.
Michael S. Robbins, Miller School of Medicine, University of Miami.
Daniel J. Feaster, Miller School of Medicine, University of Miami.
Viviana E. Horigian, Miller School of Medicine, University of Miami.
Michael Rohrbaugh, Department of Psychology, University of Arizona.
Varda Shoham, Department of Psychology, University of Arizona.
Ken Bachrach, Tarzana Treatment Centers, Tarzana, California.
Michael Miller, The Village, Inc., Miami, Florida.
Kathleen A. Burlew, Department of Psychology, University of Cincinnati.
Candy Hodgkins, Gateway Community Services, Inc., Jacksonville, Florida.
Ibis Carrion, Institute of Research, Education, and Services in Addiction (IRESA), Universidad Central del Caribe, Bayamón, Puerto Rico.
Nancy Vandermark, Arapahoe House, Denver, Colorado.
Eric Schindler, Child and Family Resources, Inc., Tucson, Arizona.
Robert Werstlein, Daymark Recovery Services, Inc., Salisbury, North Carolina.
José Szapocznik, Miller School of Medicine, University of Miami.