|Home | About | Journals | Submit | Contact Us | Français|
Evaluate a new 5-step method for testing mediators hypothesized to account for the effects of depression prevention programs.
In this indicated prevention trial at-risk teens with elevated depressive symptoms were randomized to a group cognitive-behavioral (CB) intervention, group supportive expressive intervention, CB bibliotherapy, or assessment-only control condition.
The group CB intervention reduced depressive symptoms and reduced negative cognitions and increased pleasant activities, change in these mediators predicted change in depression, and intervention effects became weaker controlling for change in the mediators; yet, change in depression appeared to typically occur before change in the mediators. The supportive expressive intervention reduced depressive symptoms but affected only one of two mediators (emotional expression, but not loneliness), change in emotional expression did not correlate with change in depression, and change in depression usually occurred before change in the mediators. Bibliotherapy did not significantly affect depressive symptoms or the ostensive mediators (negative cognitions and pleasant activities) and change in depression usually occurred before change in the mediators.
Results imply that this procedure provides a sensitive test of mediation, but yielded limited support for the hypothesized mediators, suggesting that non-specific factors may play an important meditational role.
Cognitive behavioral (CB) depression prevention programs, which focus on reducing negative cognitions and increasing pleasant activities, have reduced depressive symptoms and risk for future onset of major depression relative to assessment-only control conditions in randomized trials (Stice, Shaw, Bohon, Marti, & Rohde, 2009). Yet, few trials have investigated mediators that account for these effects, which provides a test of the theoretical mechanisms that putatively underlie program effects and can implicate components that are most effective, potentially aiding in program refinement (Kazdin & Nock, 2003).
Although procedures for examining mediators that account for intervention effects have been proposed, they do not provided a rigorous test of all of the components necessary for mediation, particularly the temporal sequencing between changes in the mediator and the outcome. It has been noted it is vital to document that change in the mediator temporally precedes change in the outcome (Kazdin & Nock, 2003; Weersing & Weisz, 2002). Because a specific procedure for testing this temporal sequencing of change has not been offered, we proposed a 5-step procedure for testing mediation with data from randomized trials (Stice, Presnell, Gau, & Shaw, 2007) and suggested that the strongest case for mediation would be made when five conditions are satisfied: (1) intervention participants show greater decreases on the outcome than controls, (2) intervention participants show greater decreases on the mediator than controls, (3) change in the mediator predicts change in the outcome in the intervention condition,1 (4) the predictive effect of the intervention on change in the outcome is significantly reduced (for partial mediation) or eliminated (for full mediation) controlling for change in the mediator, and (5) meaningful change in the mediator occurs before meaningful change in the outcome more frequently than would be expected by chance in the intervention condition. We think this 5-step procedure provides a rigorous approach for testing mediation in randomized trials, yet this heuristic model may require refinement (e.g., the fifth criterion offers but one method for testing whether the mediator changes before the outcome) and these criteria involve assumptions that should be verified.
Although trials have evaluated mediators that account for the effects of depression prevention programs, the most rigorous ones have only examined criteria 1 though 4 (Jaycox, Reivich, Gillham, & Seligman, 1994; Yu & Seligman, 2002), with mixed results. The purpose of the present report was to evaluate the 5-step method for testing the hypothesized mediators of depression prevention programs. In this indicated depression prevention program at-risk youth with elevated depressive symptoms were randomized to a CB group, a supportive expressive group, CB bibliotherapy, or an assessment-only control condition. We hypothesized that reductions in negative cognitions and increases in pleasant activities would mediate the effects of the group and bibliotherapy CB programs and that increases in emotional expression and reductions in loneliness would mediate the effects of the supportive expressive program on reductions in depressive symptoms. We tested for mediation during intervention delivery because this is typically when the most pronounced intervention effects occur and change is often linear during this period (Stice et al., 2009), making it easier to model change. As suggested by Kazdin and Nock (2003), we assessed changes in the mediator and outcome during the 6-week intervention delivery; at baseline, mid-way through, and after termination. Although we considered using weekly surveys, participants complained about the assessment burden in our prior trial that took this approach (Stice et al., 2007). A trial evaluating multiple programs also affords an opportunity to examine the specificity of intervention effects. Kazdin and Nock (2003) argued that it is important to test whether interventions affect mediators specific to that intervention, but not mediators specific to other interventions. Thus, we also tested whether each prevention program affects the theoretically specific mediators, but not mediators specific to the other programs.
Participants were 341 high school students (M age = 15.6, SD = 1.2) who reported elevated depressive symptoms. Participants were 2% Asian, 9% African American, 46% Caucasian, 33% Hispanic, and 10% with mixed heritage, which was more ethnically diverse than the population. Stice, Rohde, Seeley, and Gau (2008) provides details regarding recruitment, intervention content, supervision, therapist fidelity and competence, and attrition.
Depressive symptoms were assessed with the 21-item Beck Depression Inventory (BDI; Beck, Steer, & Garbin, 1988). The BDI has shown internal consistency (α = .73 - .95), test-retest reliability (r = .60 - .90), and convergent validity with clinician ratings of depressive symptoms in adults (M r = .75; Beck et al., 1988). The BDI showed internal consistency (α = .89), 3-week test-retest in the control condition (r = .76), and correlated with functional impairment (r = .54; assessed by the Social Adjustment Scale-Self Report for Youth [Weissman, Orvaschel, & Padian, 1980]), and self-esteem (r = -.53; assessed with the Rosenberg  scale).
Negative cognitions were assessed with 12 items from the Automatic Thoughts Questionnaire (ATQ; Hollon & Kendall, 1980). The original 30-item ATQ scale correlated more strongly with depressive versus anxiety symptoms in adolescent inpatients (Jolly & Wiesner, 1996). The 12 items from the personal maladjustment/desire for change and negative self-concept/expectations subscales were used because they correlated .98 with the total ATQ score in a past adolescent depression treatment trial (Rohde et al., 2004).2 Pilot testing with teens for this trial (N = 44) indicated that a 12-items short form of the ATQ showed internal consistency (α = .92) and 1-week test-retest reliability (r = .74) and correlated with BDI scores (r = .62). This scale showed internal consistency (α = .93), 3-week test-retest reliability (r = .75), and correlated with BDI scores (r = .65), functional impairment (r = .53) and self-esteem (r = -.59).
Pleasant activities were assessed with 12 items from the Pleasant Events Schedule (PES; MacPhillamy & Lewinsohn, 1982) commonly endorsed by teens. The original 320-item PES showed 72% agreement with peer ratings of pleasant activities, predictive validity for future self-reported pleasant activities (M r = .59), and discriminated between depressed and healthy adults (MacPhillamy & Lewinsohn, 1982). This 12-item scale correlated .84 with the total PES score in a past adolescent depression treatment trial (Rohde et al., 2004). Pilot testing with teens (N = 44) indicated that the 12-item form of the PES showed internal consistency (α = .79), 1-week test-retest reliability (r = .88), and correlated with BDI scores (r = -.45). This scale showed internal consistency (α = .73), 3-week test-retest reliability (r = .62), and correlated with BDI scores (r = -.37), functional impairment (r = -.36) and self-esteem (r = .36).
Emotional expression was assessed with 9 items developed for this study (e.g., I was able to discuss my feelings in a helpful way) because we were unable to locate a scale developed for teens that assessed this construct. This scale showed internal consistency (α = .82), 3-week test-retest reliability (r = .66), and correlated with BDI scores (r = -.29), functional impairment (r = -.29) and self-esteem (r = .39).
Loneliness was assessed with an 8-item Loneliness Scale adapted by Lewinsohn et al., (1994) from Russell (1996). The full 20-item Loneliness Scale correlated with other loneliness scales (M r = .69) and inversely with self-reported social support (M r = -.52) among adults (Russell, 1996). The 8-item scale correlated .92 with the full 20-item scale in a second pilot study of teens (n = 25). It showed internal consistency (α = .83), 3-week test-retest reliability (r = .74), and correlated with BDI scores (r = .49), functional impairment (r = .53) and self-esteem (r = -.51).
Random coefficient unconditional growth models confirmed the a priori assumption that linear trends were appropriate for modeling change over time for the mediators and outcome (see Table 1 for cell means across assessment points). Stice et al. (2007) reports details regarding the analytic approach, which utilized hierarchical linear modeling (HLM) because it produces Empirical Bayes estimates used in our procedure for testing mediation.3
We tested whether intervention condition predicted the slope parameter for depressive symptoms while controlling for the depression intercept. CB group (Table 2) and supportive expressive participants (Table 4), but not bibliotherapy participants (Table 3), showed significantly greater reductions in depression than assessment only controls.
We tested whether intervention condition predicted the slope parameter for each mediator controlling for the intercept of the respective mediator. CB group participants (Table 2), but not bibliotherapy participants (Table 3), showed significantly greater decreases in negative cognitions and greater increases in pleasant activities than controls (Table 2). Supportive expressive participants showed significantly greater increases in emotional expression relative to controls, but there were no effects for loneliness (Table 4).
We also tested whether the intervention effects on the mediators were specific. Relative to controls, group CB participants showed greater decreases in loneliness (B = -1.86, SE = 0.44, t = -4.22, p < .001) and increases in emotional expression (B = 0.31, SE = 0.06, t = 5.00, p < .001). Bibliotherapy participants did not show differential change in loneliness (B = -0.22, SE = 0.40, t = -0.57, p = .572) or emotional expression (B = 0.03, SE = 0.06, t = 0.47, p = .637). Supportive expressive participants showed greater increases in pleasant activities (B = 1.31, SE = 0.60, t = 2.18, p = .030), but not negative cognitions (B = -0.09, SE = 0.06, t = -1.53, p = .128).
We tested whether the slope of the mediator predicted change in depressive symptoms controlling for the depression intercept. Change in negative cognitions and pleasant activities predicted change in depressive symptoms for the CB group (Table 2) and bibliotherapy condition (Table 3). Change in loneliness, but not emotional expression, predicted change in depressive symptoms for the supportive expressive condition (Table 4).
We tested whether the predictive effect of treatment on change in depressive symptoms was significant when controlling for change in the mediator (4a) and whether the predictive effect of treatment on change in depression was significantly reduced when controlling for change in the mediator (4b). For the CB group (Table 2), treatment no longer predicted change in depression controlling for change in negative cognitions, but treatment predicted change in depression controlling for change in pleasant activities (4a). The effect of the CB group on depression was significantly reduced when change in each of these mediators was controlled (4b). For bibliotherapy (Table 3), treatment showed a significant relation to change in depression when change in negative cognitions and change in pleasant activities were controlled (4a) and there was a significant change in the effect of treatment on change in depression when change in the mediators was controlled (4b). Although these results appear to provide support for criterion 4 for bibliotherapy, there was no main effect of bibliotherapy on change in depression, indicating that these results do not provide support for criterion 4. The effect of the supportive expressive intervention on depression became nonsignifiant when change in loneliness and emotional expression were controlled (4a). Yet, the effect of treatment on change in depression was significantly weaker only when change in loneliness was controlled (Table 4).
We tested whether meaningful reduction in the mediator typically occurred before meaningful reduction in the outcome more frequently than expected based on chance. We defined meaningful change as a .33 SD reduction in the variables because most participants showed this degree of change in the mediators and outcomes. Although somewhat arbitrary decision, the important point is that we used the same criterion for mediators and the outcome that adjust for the fact that the measures had different scaling. Participants who showed a .33 SD reduction in the mediator before they showed a .33 SD reduction in the outcome received a score of 1. Those who showed a .33 SD decrease in the mediator after they showed a .33 SD decrease in the outcome, did not show a .33 SD decrease in the outcome, or did not show a .33 SD decrease in the mediator received a score of 0. We used a non-parametric binomial test to determine whether the proportion of participants who showed a meaningful change in the mediator before showing a meaningful change in the outcome was greater than .50 among intervention participants. In the CB group condition 75% showed a .33 SD decrease in depression, but only 8% showed a decrease in negative cognitions before a .33 SD reduction in depression and only 9% showed an increase in pleasant activities before a .33 SD reduction in depression (Table 2). In the bibliotherapy condition 55% showed a .33 SD decrease in depression, but only 6% showed a decrease in negative cognitions before a .33 SD in reduction of depression and only 5% showed an increase in pleasant activities before a .33 SD reduction in depression (Table 3). In the supportive expressive condition 69% showed a .33 SD decrease in depression, but only 15% showed a decrease in loneliness before a .33 SD in reduction of depression and only 39% showed an increase in emotional expression before a .33 SD reduction in depression (Table 4). None of these proportions were significantly greater than .50. For reference, 65% of controls showed a .33 SD decrease in depression; between 1% and 16% showed a .33 SD decrease in mediators before showing a .33 SD decrease in depression.
Results provided partial support for the intervention theory for the group CB. Participants showed greater reductions in depressive symptoms and greater change in negative cognitions and pleasant activities than controls, with effect sizes that were clinically significant, and change in the mediators correlated with change in depression. The intervention effect became nonsignifiant when change in negative cognitions was controlled, but remained significant when change in pleasant activities was controlled, with the effects of the intervention on change in depression being significantly reduced. Yet results suggested that only 8% of participants showed meaningful change in the mediators before showing meaningful change in depression, which was significantly lower than expected by chance. Thus, results do not appear to support the theory that changes in negative cognitions and pleasant activities mediate the intervention effects because the outcome typically changed before the mediators. Given that the most rigorous tests of mediators of CB depression prevention programs only examined the first 4 criteria (Jaycox et al., 1994; Yu & Seligman, 2002), it is possible that change in depression may have preceded change in the putative mediator in those trials too.
Results provided little support for the intervention theory for bibliotherapy. Relative to controls, bibliotherapy participants did not show greater improvements in depressive symptoms, negative cognitions, or pleasant activities. Although change in the mediators correlated with change in depression, there was no main effect of intervention on depression, so it is not relevant to test whether the main effects of treatment on the outcome became weaker when change in the mediator was controlled. Further, results imply that only 6% of participants showed a decrease in negative cognitions before showing a reduction in depression and only 5% showed an increase in pleasant activities before showing a reduction in depression. Both of these percentages were significantly lower than expected by chance. Although 72% reported reading at least some of the book, it may take teens longer than 6 weeks to read this book and show symptom reduction. Indeed, bibliotherapy participants did show significantly greater reductions in depressive symptoms than controls by 6-month follow-up (Stice et al., 2008).
Results provided only partial support for the intervention theory for the supportive expressive program. Relative to controls, supportive expressive participants did show clinically significant reductions in depressive symptoms and greater change in one mediator (emotional expression), but not the other mediator (loneliness). Ironically, change in loneliness, but not emotional expression, correlated with change in depression. Although the intervention effect on depression became nonsignifiant controlling for change in both of the mediators, the change in the main effect was only significantly different controlling for change in loneliness. Data imply that only 15% showed a decrease in loneliness before showing a reduction in depression and only 39% showed an increase in emotional expression before showing a reduction in depression (which were both significantly less then expected based on chance).
There was mixed support for the specificity of the mediators. Group CB participants showed significant reductions in loneliness and improvements in emotional expression, in addition to the expected effects for negative cognitions and pleasant activities, suggesting that this program affects both specific and non-specific factors. Supportive expressive group participants showed increases in pleasant activities, but no change in negative cognitions. The former finding might suggest that reducing depression increases involvement in pleasant activities. Bibliotherapy did not reduce loneliness or emotional expression.
It is important to note that this 5-step approach of testing mediators is not so stringent that it always fails to support intervention theory. In the first paper to apply this approach (Stice et al., 2007), we found that all 5 criteria were satisfied for a dissonance-based eating disorder prevention program. Indeed, the four other programs for which we have applied this test of mediation each failed for distinct reasons, implying that this approach for testing mediation is sensitive to detecting violations of any of the five criteria implied by the mediational model.
Because change in depressive symptoms typically occurred before change in the mediator for all three prevention programs examined herein, it is important to consider alternative explanations for these findings. First, it is possible that the use of shortened versions of the mediator measures, but not the outcome, reduced the sensitivity of the former to detect intervention effects relative to the latter. Yet, the average pre to post intervention effects (semi-partial r’s) was .17 for the mediators and .18 for the BDI and the average 3-week test-retest reliability among controls was similar for the mediators (r = .69) and the BDI (r = .76). Further, Stice et al. (2007) that found full support for the meditational model of an eating disorder prevention program using an even briefer measure to assess the mediator.
Second, it may have been difficult to model the precise timing of change in this trial because we only assessed the mediators and outcome three times during the intervention. Although we could have modeled the timing of change more precisely with more frequent assessments, this factor cannot logically explain why results suggested that change in depression occurred before change in the mediators significantly more frequently than expected by chance for all three programs, as both mediators and the outcome were assessed three times. Further, if the estimated timing of changes were unreliable, one would expect to see about only half the sample showing changes in depression before showing changes in the mediators (the null hypothesis).
Third, it is possible that using slopes to test the temporal order of change in the mediators relative to the outcome is not optimal and that an alternative approach would provide evidence that the mediators change before symptoms. As an alternative approach, we calculated the proportion of participants in each condition who showed greater change in the mediator than in depression by the mid-intervention assessment. In the CB condition the proportion of participants showing greater change in the mediator before change in the outcome in the expected direction was 17% for negative cognitions and 19% for pleasant activities. In bibliotherapy the proportion of participants showing greater change in the mediator before change in the outcome was 13% for negative cognitions and 14% for pleasant activities. In the supportive expressive condition the proportion of participants showing greater change in the mediator before change in the outcome was 7% for emotional expression and 14% for loneliness. The fact that both approaches for modeling the relative timing of change produced similar results, despite different assumptions, implies the findings are robust.
Fourth, it is possible that regression to the mean resulted in more rapid reductions in depression than in the mediators, given that participants were initially selected because of elevated depression. However, the average effect for depression (r = .18) was similar to the average effect for the mediators (r = .17), suggesting this alternative explanation is implausible.
The finding that change in depressive symptoms occurred more rapidly than change in all of the mediators may imply that non-specific factors, such as the installation of hope, normalization of the person’s experience, the creation of a therapeutic alliance, motivation and a willingness to change, and the introduction to a theoretical model for change, drove symptom reductions. Yet, if non-specific factors alone produced rapid reductions in depressive symptoms, it is difficult to explain why the group CB intervention outperformed the other interventions. From pre to post, CB group participants showed significantly greater reductions in BDI scores than supportive expressive and bibliotherapy participants (Stice et al., 2008). Perhaps the group CB intervention was more credible, which produced stronger non-specific effects. Further, the CB group was the only program that resulted in significant improvements in all four mediators, which may suggest that the effects are larger for this program because there are more mediators or because different mediators are operating for different subsets of participants. Because depression reductions often occur early, future trials testing for mediators of intervention effects should assess non-specific factors in the first few sessions, as they may play an important role in generating intervention effects. This study has several limitations, including the reliance on self-report data, the infrequent measurement of mediators and the outcome, and the fact that we did not assess non-specific factors. Future trials should investigate additional mediators that may account for depression prevention effects, such as non-specific factors, should attempt to assess mediators and the outcome more frequently during the intervention, and should extend these procedures for testing meditational processes over longer follow-up periods, during which change may not be linear, as continued research on the mediators that produce intervention effects may result in more efficacious prevention programs.
This study was supported by a research grants MH067183 and MH080760 from the National Institute of Health.
Thanks go to project research assistants, Courtney Byrd, Kathryn Fischer, Amy Folmer, Cassie Goodin, Jacob Mase, and Emily Wade, a multitude of undergraduate volunteers, the Austin Independent School District, and the participants who made this study possible.
1We acknowledge that change in the mediator may predict change in the outcome in the control condition. Yet, because the change in the mediator in the control condition could not be caused by the intervention, we assert that the correlation between the mediator and outcome among controls reflects etiologic or maintenance processes and is not germane to testing mediators that account for intervention effects.
2We report the correlations between the short forms of the negative cognitions, pleasant activities, and loneliness scales used herein and the original full-length forms in the measures section because these correlations provide an indication of how well the short forms serve as a proxy for the longer forms. In this footnote, we report the correlations between the short forms of these scales and the items from the full-length forms not included in the short forms of these scales, which is akin to the split-half reliability of these scales, as this may be of interest to some readers (negative cognitions r = .96; pleasant activities r = .78; loneliness r = .76).
3We noted errors in the equations provided in Stice et al. (2007). Each of the composite multilevel models have the time parameter omitted from the rate of change random effect, the person level random effect is represented with the Greek letter zeta rather than the more standard “e”, and the composite model for criterion 4 incorrectly identifies γ12 as a group by change in mediator parameter. Important to note is the fact that errors were only in the description of the equations, the models used to generate the parameters estimates reported in the manuscript are correct. More specifically, on page 24, column two, lines 7-8 the equation should read Yij = [γ00 + γ10 (timeij) + γ01 (groupi) + γ11 (Groupi X Timeij)] + [ζ0i + ζ1i (timeij) + eij]; the parenthetical (ζij) on line 15 should read (eij); the equation on lines 18-20 should read Yij = [γ00 + γ10 (timeij) + γ01 (Δmediatori) + γ11 (ΔMediatori X Timeij)] + [ζ0i + ζ1i (timeij) + eij]; the parenthetical (ζij) on line 27 should read (eij); the equation on lines 30-32 should read Yij = [γ00 + γ10 (timeij) + γ01 (groupi) + γ02 (Δmediatori) + γ11 (Groupi X Timeij]) + γ12 (ΔMediatori X Timeij)] + [ζ0i + ζ1i (timeij) + eij]; the phrase “effect of condition on the change in the mediator” on lines 36-37 should read “effect of change in the mediator on change in the outcome.”; and the parenthetical (ζij) on line 41 should read (eij).