We conducted an initial effectiveness study in which the outcomes generated by an evidence-based treatment for youth anxiety (i.e., Coping Cat) provided by therapists who received training and supervision in the program were compared to the outcomes of UC in several public mental health clinics. We tested for treatment effects across a variety of outcomes. Youths in both the CBT and UC conditions exhibited improved diagnostic outcomes: more than half the ITT sample and more than two-thirds of children for whom we had post-treatment data (66.7% in CBT and 73.7% in UC) no longer met diagnostic criteria for their primary anxiety disorder. The proportion of youth in the Coping Cat condition who no longer met diagnostic criteria for their primary diagnosis compares favorably to the proportions in past Coping Cat efficacy studies5, 11, 13
and in the Barrington et al. trial. 8
Similarly, the proportion of youth in the UC group who no longer met diagnostic criteria for their primary diagnosis was comparable to that found in CBT groups in prior Coping Cat trials and in the Barrington et al. trial. Symptom outcomes were similar to diagnostic outcomes in pattern. Youths in both conditions exhibited improvement from pre- to post-treatment but there were no statistically significant differences between the CBT and UC groups. The pattern of no treatment group differences held across all but one secondary outcome, including duration of treatment, estimated cost of services, and impact of treatment on comorbid disorders. The one significant group difference was that CBT youth used significantly fewer additional services during treatment compared to the UC group, a finding we discuss shortly. Overall, our data suggest that the CBT intervention achieved outcomes within the range of previous trials using Coping Cat, but that it did not outperform UC.
There are a few key findings relevant to the potency of the CBT condition in the present study. First, therapists newly trained and supervised to provide CBT in public mental health clinics to clients with primary anxiety disorders and complex clinical pictures were able to (a) obtain good adherence scores, and (b) produce outcomes at levels similar to those in efficacy trials with expert CBT therapists. These results are similar to those reported by Weisz et al.30
in a study of CBT for youth depression and suggest that community therapists can be trained to deliver CBT effectively.
We found that youth in UC received significantly more additional mental health services (e.g., additional therapy, group therapy) than youth in the CBT condition: 41.0% vs. 0.0%, a result similar to that of a recent effectiveness trial of CBT with depressed youth.30
Because most community-based studies8–10
have not reported such systems-level data, it is difficult to place our findings in context. If future work replicates the finding, it may be reasonable to suggest that CBT is a simpler prescription, or is better able to produce effects without supplemental services, than is UC.
A principal goal of effectiveness research is to assess how well specific interventions fare under representative clinical care conditions, conditions that often include less-than-complete delivery of treatment protocols. Such a pattern was evident in our study, with only 52% of youth in the CBT group receiving the full 16 sessions and only 59% receiving exposure sessions. It is possible that CBT outcomes would have been better if all of the youth receiving CBT had received the full dose, including exposures. On the other hand, it is important to note that CBT outcomes in the present sample—despite cases of less-than-complete treatment—were quite comparable to those of prior efficacy trials using Coping Cat5, 11, 13
. The non-significant CBT-UC differences we found reflected, not poor CBT outcomes, but instead the positive outcomes of youngsters in the UC condition, outcomes comparable to those of Coping Cat in prior efficacy trials. Thus, the fact that our community clinic youths in the CBT condition did not all complete full Coping Cat protocols did not prevent them from showing treatment gains characteristic of previous efficacy trials.
What then did account for our findings? Experts have noted that multiple levels of the ecology can influence the success of an EBT when transported to a new setting.1
For example, added client complexity (e.g., higher levels of comorbidity, increased levels of ethnic diversity) in community service settings, supported by several studies,33, 34
may complicate the delivery of EBTs. These considerations are relevant to our findings for at least two reasons. First, we did not modify the treatment program, so CBT therapists maintained a focus on anxiety whereas UC therapists were free to attend to and address non-anxiety problems. This is particularly relevant given the high rate of comorbid disorders in our sample. It is possible that the flexibility and latitude of UC afforded an advantage, and it is possible that adapting EBTs to permit focus on multiple problems could yield even better outcomes.3, 35
Second, our design included random assignment of therapists to treatment condition and training and ongoing supervision by experts in the model—procedures that are consistent with efficacy trials; however, therapists in community settings are likely to vary in their training, experience, interest, and comfort in CBT—factors that could have impacted the outcomes of the study.36, 37
By contrast, UC therapists used procedures of their own choice and with which they were presumably experienced; this may have enhanced their confidence and thereby the effectiveness of their treatment. And as our analyses demonstrated, UC therapists apparently included some CBT strategies, suggesting that the CBT approach has been diffused somewhat into usual care.
The study was underpowered to detect anything other than large effects. At the outset of the trial, the early success of Coping Cat11
led us to anticipate medium to large effects, but our actual effects proved similar to those that were later evident in recent comparisons of EBTs to UC comparison groups8, 30
. Completing this trial lays the foundation for future double-randomized trials across multiple community clinics, training in-house therapists, and treating community-referred clients, with larger samples.
Two additional issues warrant attention in future research. First, our sample of clients and therapists was quite heterogeneous, increasing error variance and possibly making the delivery of an anxiety-focused CBT program more challenging. Of course, a limitation of this kind that may weaken internal validity may also be viewed as a strength when it amplifies a study’s external validity.38
Studies that focus on the impact of client and therapist heterogeneity on outcome could shed important light on the tradeoffs between efficacy and effectiveness designs. Second, the field lacks relevant measures of competence in the use of CBT with children; such measures would have permitted us to examine how well the CBT program was delivered, beyond adherence to the protocol. Future work developing competence assessment methods could be valuable to the field.39
Considering the study’s blend of strengths and limitations, this appears to be the most rigorous randomized effectiveness trial to date testing an EBT for childhood anxiety disorders compared to an active UC alternative. Our results contribute to a growing body of literature identifying the challenges in documenting differential effects between manual-based and UC treatments. Indeed, a recent meta-analysis of RCTs comparing youth EBTs to UC40
for a variety of clinical problems (e.g., internalizing and externalizing disorders) found only a modest mean effect size favoring EBTs, and in many of the studies there was no significant EBT vs. UC difference in outcome. Interestingly, no RCTs were found for that meta-analysis that involved anxiety treatment and met methodological standards for inclusion. With completion of the present study, there is now one fully randomized trial of CBT vs. UC in the youth anxiety domain and another partially randomized trial;7
both trials have shown no significant difference in outcome between CBT and UC. This suggests a need for considerable research in the days ahead, to determine whether there are conditions under which CBT for youth anxiety can, in fact, outperform the treatments that are routinely provided by clinicians in everyday clinical care.