Prior to examining our primary hypotheses, we examined whether there were significant differences in competence ratings across the six study therapists. We first calculated the average CTS scores for each patient. Therapists differed significantly in these mean ratings (F (5, 54) = 7.74, R2 = .42, p < .0001). Mean scores for the six therapists were: 49.9 (range of 41.3 to 56.6), 42.1 (range of 30.6 to 49.3), 40.3 (range of 27.1 to 49.1), 39.9 (range of 32.7 to 47.5), 34.0 (range 17.8 to 45.9), and 31.1 (range of 11.3 to 40.1). Therapists did not differ on ratings of patient difficulty (F (5,54) = 1.40, R2 = .11, p = .24). Because two therapists had less experience in CT and obtained additional training during the trial, we also examined whether these two therapists were rated as less competent than the therapists more experienced in CT. Although differences were in the expected direction, they were not significant (F(1, 58) = 2.75, R2 = .05, p = .10; high experience M = 41.1, SD = 9.5; low experience M = 37.0, SD = 8.1). The overall average of competence scores for each therapist-patient dyad was 39.7 (SD = 9.2).
Competence and Session-to-Session Symptom Change
We examined competence as a predictor of session-to-session symptom change across the first four sessions of CT. As shown in , competence significantly predicted subsequent symptom change in this model. For ease of interpretation, signs have been adjusted so that a positive relationship indicates that higher competence ratings predict positive outcomes in these and all subsequently reported analyses. We then conducted exploratory analyses, using the same statistical approach, in which each CTS item served as a predictor of session-to-session symptom change (see ). ICC estimates of inter-rater reliability for individual items are also reported in the table. The largest effects were for the following items: Agenda, Focusing on Key Cognitions or Behaviors, Pacing, Homework, and Application of Cognitive-Behavioral Techniques.
Cognitive Therapy Scale Total and Item Scores as Predictors of Session-to-Session Symptom Change
We then conducted analyses of therapist differences and differences in patient difficulty as they might explain variation in early session-to-session symptom change. For the model examining therapist, site was not entered as a covariate (as therapists were nested within site). Therapist was not a significant predictor of session-to-session symptom change across the first four sessions (p = .3). Higher patient difficulty ratings were predictive of less session-to-session symptom change (r = −.33, t = −2.41, p = .02). Interestingly, these difficulty ratings (completed at session 1) were not strongly related to concurrent ratings of competence at session 1 (r = −.15, p = .25). We also examined whether competence ratings predicted session-to-session symptom change after controlling for patient difficulty. In this model, the effect of competence was reduced to a non-significant trend (r = .28, t = 1.97, p = .06).
Competence and Long-term Symptom Change Following Early Sessions
In the model predicting subsequent change in HRSD severity through the end of treatment, higher competence ratings were predictive of lower HRSD scores at post treatment (r
= .33, t
= 2.45, p
= .02). In the parallel model using BDI as the index of symptom severity, this relationship was reflected by a non-significant trend (r
= .24, t
= 1.72, p
We then examined therapist (in place of competence ratings) as a predictor in the models described above. Therapist was a significant predictor in the HRSD model (F(5, 53) = 5.07, p
= .0007), but not in the model for BDI (F(5, 52) = 1.61, p
= .17). When we examined therapist as a covariate (rather than site), competence remained a significant predictor in the models for HRSD (r
= −.37, t
= −2.93, p
= .005) and was now significant in the model for the BDI (r
= −.34, t
= −2.55, p
= .01). Although there were significant differences among therapists in mean competence ratings given for each therapist-patient dyad (as noted previously), these differences did not appear to correspond to the observed differences in HRSD or BDI severity at the end of treatment. Therapist remained a significant predictor of end of treatment HRSD symptom severity after controlling for competence ratings (F(5, 53) = 5.03, p
= .0001). Thus, in the long-term models, competence ratings predicted subsequent symptom change on the HRSD (with and without therapist covaried), but competence ratings only predicted subsequent symptom change on the BDI when therapist was covaried. In addition, where therapist differences on outcome were evident (i.e., on the HRSD), these differences were largely not accounted for by competence ratings.
Neither of two analyses of the relation between ratings of patient difficulty and symptom improvement (as indexed by the HRSD) yielded a significant effect (for the analysis of post-treatment scores: r = .18, t = 1.27, p = .2; for the analysis of the slope of change: r = .14, t = .96, p = .3). However, significant effects were obtained in both kinds of analyses when the BDI was the indicator of depressive symptoms (post-treatment symptom severity: r = .31, t = 2.41, p = .02; slope of change: r = .28, t = 2.04, p = .046). We examined competence as a predictor in the models that used the HRSD and BDI scores, respectively, with both patient difficulty and the patient difficulty by time interaction entered as additional covariates. In the model for HRSD, competence ratings remained a significant predictor subsequent symptom change through the post-treatment assessment (r = −.31, t = −2.34, p = .02). In the model for BDI, competence ratings remained a non-significant predictor of subsequent symptom change through the post-treatment assessment (r = −.23, t = −1.62, p = .11).
Only nine patients discontinued treatment prematurely, which limited power to detect differences between completers and dropouts. However, we did compare mean CTS scores of these two patient groups. The means did not differ significantly (t(58) = .94, p = .4, d = .3, completers: M = 40.2, SD = 8.8, CI = 37.7, 42.6; drop-outs: M = 37.0, SD = 11.3, CI =28.3, 45.8). We then used logistic regression to examine whether the average competence rating for each patient predicted risk of dropout, after controlling for site and HRSD scores at intake. CTS scores were unrelated to risk of drop-out in this model (β = −.32, SE = .37, Wald = .74, p = .4, OR = .73, CI = .35, 1.50).
Moderators of Competence and Session-to-Session Symptom Change
We examined four potential moderators using our primary analytic strategy focused on session-to-session models of the early portion of CT. However, we also explored whether these variables served as moderators of the relationship between competence and outcome in the longer-term models using HRSD and BDI.
As shown on the left side of , in the session-to-session analyses, significant interactions between competence and the potential moderators emerged for two of the four variables (i.e., age of onset and anxiety), as did a non-significant trend for the interaction of chronicity (i.e., dysthymia or chronic depression) and competence. Competence did not predict outcome differentially between dyads in which the patient was versus was not given a personality disorder diagnosis.4
The significant and trend level interactions were each obtained in the context of significant main effects of the potential moderators on BDI-II scores in the next session (anxiety: r
= .30, t
= 2.37, p
= .02; age of onset: r
= −.27, t
= −2.12, p
= .04; and chronic/dysthymic: r
= −.28, t
= −2.20, p
= .03). Although there was no evidence of moderation by personality disorder status, there was a trend for personality disorder status to predict a reduced magnitude of session-to-session symptom change (r
= −.26, t
= −2.00, p
Interactions between Patient Characteristics and Therapist Competence in Predicting Session-to-Session Symptom Change and End of Treatment Depressive Symptom Severity
To better understand the significant interaction effects, models of competence as a predictor of symptom change were examined separately for those with and without chronicity; for the continuous moderators, median splits were used to divide the sample into high and low groups. As depicted in , the two significant interactions, as well as the trend-level interaction, were driven by competence being more predictive of outcome among patients who exhibited factors that were expected to require more competently delivered CT. That is, competence predicted session-to-session symptom change more strongly for patients with higher levels of comorbid anxiety, a younger age of onset and, at the level of a non-significant trend, a more chronic course of depressive symptoms.
Relation between Competence Ratings and Subsequent Session-to-Session Symptom Change by Pre-Treatment Patient Characteristics
The evidence was less robust that these patient characteristics interact with competence to predict more distal outcomes (i.e., post-treatment symptom severity). As shown on the right side of , the only significant interaction to emerge from these models was that of anxiety and competence in predicting HRSD post-treatment symptom severity. That same interaction yielded a non-significant trend in predicting end of treatment BDI symptom severity.5
Using a median split on anxiety to probe this interaction in the model for HSRD, competence was more strongly related to outcome among patients with higher levels of anxiety (r
= .40, t
= 2.22, p
= .04) as compared to those with lower levels of anxiety (r
= .29, t
= 1.38, p
= .18). The non-significant trend in predicting end of treatment BDI symptom severity was driven by a less striking effect of the same pattern (high anxiety: r
= .25, t
= 1.32, p
= .2; low anxiety: r
= .19, t
= .91, p
= .4). Thus, while these analyses were exploratory and should be interpreted with caution, it is noteworthy that the most consistent evidence of moderation of the relationship between competence and subsequent outcome was observed for pre-treatment severity of anxiety. Other evidence of moderators of the relationship between competence and outcome was limited to the session-to-session analyses focused on patients’ early responses to treatment.