The current study is unique in conducting a tri-level (patient, counselor, and program) analysis of the alliance-outcome relationship. The primary finding of the study was that alliance variability at the patient and program level, but not the counselor level, was significantly related to presence/absence of any drug and alcohol use during the past week. These findings were strongest for patients who had been in treatment for a relatively shorter period of time (less than 3 months). This general pattern of results was replicated using a second sample of patients that participated at a later point in time in the randomized trial from which these data were drawn. However, in the replication sample there was no evidence for differential alliance-outcome relationship depending on length of treatment. An additional finding was that certain program characteristics predicted treatment outcome, but these relationships were not mediated by the alliance.
The lack of finding at the counselor level is discrepant from recent studies (Baldwin et al., 2007
; Crits-Christoph et al., 2009
) that found that clinician variability in alliance was related to outcome. In the context of the current study, it might be hypothesized that counselors experiencing burn-out at poorly functioning programs that have difficult-to-treat patient populations would have poorer alliance and outcomes compared to counselors at relatively better functioning programs, thus producing a confound between the program level and the counselor level. In fact, we did find significant overall mean differences between counselors in their average alliance, indicating that some counselors typically form better alliances than do other counselors. This counselor variability in alliance was not related to outcome, with the exception of a finding of the alliance in relation to alcohol use. This effect, however, was small and was not found in the replication sample at Week 12. Moreover, the lack of counselor effect in alliance-outcome relationship was consistent across programs (i.e., no significant counselor by program cross-level interaction), suggesting that even at the better functioning programs there was no evidence for a counselor level effect in the alliance-outcome relationship. One of the previous studies (Crits-Christoph et al., 2009
) that did find a counselor-level effect in the alliance-outcome relationship also used a sample from community-based substance abuse treatment clinics, so the differences between that study and the current one are not likely due to patient population or clinician differences. It may be that the methodology used here, including the use of a single assessment of alliance and drug/alcohol use, was not sensitive enough to pick up on potential counselor effects. Alternatively, the lack of modeling program effects in the Crits-Christoph et al. (2009)
study may have generated apparent counselor effects if the counselor effect was confounded with program in that study.
Another difference between the current study and previous multilevel studies of the alliance-outcome relationship was our finding that patient variability (within counselor) in alliance was related to outcome. The large sample size used here (N
=1613) and associated high statistical power compared to other studies may be in part responsible for the current study detecting the patient level effects. However, the effect sizes for the relationship between alliance and any alcohol use, any drug use, and any drug or alcohol use at the patient level (converting Cohen's d
, these were r
's of .30, .20, and .30, respectively) were similar in magnitude, or higher, than that reported for the alliance-outcome relationship in meta-analyses (i.e., r
= .22 from Martin et al., 2000
). Thus, it may be more perplexing why the Baldwin et al. (2007)
and Crits-Christoph et al. (2009)
studies failed to find that patient variability in alliance predicted outcome than it is perplexing why the current study did find such a relationship. From a clinical point of view, differences between patients in their experience of the alliance, within a clinicians' caseload, has long been thought to be a determinant of outcome. Thus, the current results are more consistent with clinical expectations.
The findings in the current study, namely that certain aspects of organizational functioning of programs are associated with ongoing patient-reported drug/alcohol use, replicates previous research in substance abuse treatment programs (Moos & Moos, 1998
; Lehman et al., 2002
; Greener et al., 2007
). In particular, Lehman, Greener, and Simpson (2002)
reported that ORC scales of Staffing, Influence, Mission, Cohesion, Autonomy, Communication, and Openness to Change were related to treatment satisfaction and/or counselor rapport at the program level. The current study extends these results by documenting that Staffing, Influence, Cohesion, and Communication also predict drug/alcohol use outcomes at the program level. Assuming a causal connection between these program characteristics and drug/alcohol use outcomes, clinical treatment programs could make use of such research findings by assessing and intervening at the organizational level to improve treatment outcomes. Furthermore, accrediting agencies, policy makers, and third-party payers should also reflect on such results when considering how best to accredit, structure, and finance the treatment of substance use problems in order to develop a service delivery system that achieves better treatment outcomes. From a research point of view, intervention studies should incorporate assessment of the treatment context and existing program infrastructure to fully understand treatment outcomes. This is especially relevant for multi-center trials in the treatment of substance use problems.
Of special interest in the current study are the findings that (1) programs differ in average alliances, and (2) variability in the average alliance for programs is related to the average program outcomes. The mean differences in alliance between programs suggests that patients are able to relate better to their counselors in some programs compared to other programs (independent of any impact of the particular counselors working in these program). It may be that patients are picking up expectations about treatment and about counselors from the program environment and these expectations impact alliance. Expectations about treatment have been found to be associated with the alliance in previous investigations (Gibbons et al., 2003
; Joyce & Piper, 1998
). Another possibility is that there is some patient variable that is highly confounded with program and is also correlated with alliance, such as severity of drug/alcohol dependence.
Although variability in programs in their average alliances was found to be related to variability in average program drug and alcohol use outcomes, alliance was not a mediator of the relationship between program functioning variables and drug and alcohol use outcomes. Thus, the data do not support the hypothesis offered by Simpson (2004)
that alliance mediates the relationship between program factors and outcome in the treatment of substance abuse. If poor organizational functioning has a causal impact on outcome, the current data suggest that it does so through a different mechanism than does the alliance. One implication of these findings is that, assuming causality based on these correlations, better outcomes could be achieved by both improving the organizational functioning of substance abuse treatment programs and improving the alliance of counselors with their patients.
Several limitations of the current study are important to keep in mind. One issue that limits the potential generalizability of our results is that many patients (both with drug/alcohol problems and other disorders) are not treated within a program setting. In the context of office practice (private practice) treatment of substance use problems, clinician differences may have a relatively greater influence on the alliance-outcome relationship. A second limitation was that the alliance measure was very brief (4 items). Stronger relationships may be apparent with an alliance measure with more items, although internal consistency reliability of the brief alliance measure was good (.87) in the current study. Similarly, the outcome measures were single items of recent drug and alcohol use. Again, multi-item outcome measures would more likely show stronger relationships with alliance.
A further limitation of the current study was that assessments were conducted as a “snapshot” of all patients within a program in a given week. Patients therefore varied on the duration of their treatment, with almost half of the patient sample having been in treatment for 3 months or longer. Because many patients report no current alcohol or drug use when they begin a treatment episode for substance use problems (they have already stopping using drugs and/or alcohol prior to their intake visit), the level of ongoing use throughout the course of treatment reflects the extent to which they have “slipped” or “relapsed” during their recovery. From this perspective, the level of reported use is an appropriate short-term outcome measure. However, the amount of use over a longer period of time than one week would be a more reliable index of treatment outcome.
Another limitation was that alliance and drug/alcohol use were assessed at the same point in time. Thus, the current study does not document that alliance predicts further change beyond the time point at which alliance was assessed. Additional studies will be needed to examine whether the program level, and patient level, effects in the alliance-outcome relationship found here are also evident when outcome is measured longitudinally for each patient beginning with their intake assessment. Such longitudinal assessments would be especially useful in understanding the causal direction of influence among these variables.
In summary, the current study found that variability in alliance at the program level and variability in alliance at the patient level were both related to drug and alcohol use outcomes. Alliance, however, was not found to be a mediator of the relationship between program characteristics and drug and alcohol use during treatment.