LINKING SESSION FOCUS TO TREATMENT OUTCOME IN EVIDENCE-BASED TREATMENTS FOR ADOLESCENT SUBSTANCE ABUSE

doi:10.1037/0033-3204.41.2.83

Psychotherapy (Chic). Author manuscript; available in PMC 2010 May 14.

Published in final edited form as:

Psychotherapy (Chic). 2004 SUMMER; 41(2): 83–96.

doi: 10.1037/0033-3204.41.2.83

PMCID: PMC2871030

NIHMSID: NIHMS184780

LINKING SESSION FOCUS TO TREATMENT OUTCOME IN EVIDENCE-BASED TREATMENTS FOR ADOLESCENT SUBSTANCE ABUSE

AARON HOGUE, SARAH DAUBER, HOWARD A. LIDDLE, and JESSICA SAMUOLIS

AARON HOGUE, National Center on Addiction and Substance Abuse at Columbia University;

Contributor Information.

Correspondence regarding this article should be addressed to Aaron Hogue, PhD, National Center on Addiction and Substance Abuse at Columbia University, 633 Third Avenue, 19th Floor, New York, NY 10017. Email: ahogue/at/casacolumbia.org

The publisher's final edited version of this article is available at Psychotherapy (Chic)

See other articles in PMC that cite the published article.

Abstract

The relation between specific therapy techniques and treatment outcome was examined for 2 empirically supported treatments for adolescent substance abuse: individual cognitive–behavioral therapy and multidimensional family therapy. Participants were 51 inner-city, substance-abusing adolescents receiving outpatient psychotherapy within a larger randomized trial. One session per case was evaluated using a 17-item observational measure of model-specific techniques and therapeutic foci. Exploratory factor analysis identified 2 subscales, Adolescent Focus and Family Focus, with strong interrater reliability and internal consistency. Process–outcome analyses revealed that family focus, but not adolescent focus, predicted posttreatment improvement in drug use, externalizing symptoms, and internalizing symptoms within both study conditions. Implications for the implementation and dissemination of individual-based and family-based approaches for adolescent drug use are discussed.

Psychotherapy process research plays an integral role in the development of empirically based treatments. Treatment development refers to systematic efforts to test, critique, and revise the theoretical underpinnings and technical ingredients of intervention models in connection with an accumulating research base (Kazdin, 1994; Rounsaville, Carroll, & Onken, 2001). Treatment development relies on process research that can elucidate the mechanisms of change responsible for observed outcome effects: How does a treatment work and what features are essential for its success? (Kazdin, 1999).

Process research is also poised to make a substantial contribution to treatment dissemination efforts. The imposing gap between efficacy research (testing therapies under highly controlled conditions to maximize internal validity) and effectiveness research (testing therapies under standard practice conditions to maximize external and ecological validity) has prompted demand for research on transporting treatment models from the lab to the clinic (Nathan, Stuart, & Dolan, 2000; Weisz, Donenberg, Han, & Weiss, 1995). One key to successful transportation will be identifying the specific aspects of efficacious models that are linked with good outcomes—knowing what, exactly, should be transported (Kazdin, 2001). This is particularly true for multicomponent, flexibly delivered models whose clinical look may vary from case to case as therapists attempt to meet the unique needs of each client (Gaston & Gagnon, 1996).

The current study investigated specific therapy processes for two empirically supported outpatient treatments for adolescent substance abuse: cognitive–behavioral therapy and family therapy. National surveys and household probability studies conducted within the past decade reveal that adolescent drug use remains a prevalent and serious problem (Department of Health and Human Services, 2000; Gfroerer, 1995; Johnston, O’Malley, & Bachman, 1995; Kilpatrick et al., 2000). To date, family therapy has generated the largest evidence base in the treatment of adolescent drug use and cooccurring symptoms (Stanton & Shadish, 1997; Williams, Chang, & ACARG, 2000). Rigorous empirical studies have shown that family-based therapy can produce engagement and retention of drug users and their families in treatment (Henggeler et al., 1991); reduction or elimination of drug use (Liddle et al., 2001; Waldron, Slesnick, Brody, Turner, & Peterson, 2001); decreased involvement in delinquent activities (Henggeler, Melton, Smith, Schoenwald, & Hanley, 1993); improvement in multiple domains of psychosocial functioning such as school grades, school attendance, and family functioning (Liddle et al., 2000); and increased quality of parenting behavior (Mann, Borduin, Henggeler, & Blaske, 1990; Schmidt, Liddle, & Dakof, 1996). There is also evidence that therapeutic gains maintain at long-term follow-up (Liddle et al., 2001) and that family-based approaches are cost-effective in comparison to treatment as usual (Schoenwald, Ward, Henggeler, Pickrel, & Patel, 1996).

A second highly regarded treatment approach for adolescent drug abuse is cognitive–behavioral therapy (Bukstein, 1995; Weinberg, Rahdert, Colliver, & Glantz, 1998). Cognitive–behavioral approaches have demonstrated efficacy in reducing adolescent substance abuse (Waldron et al., 2001; Winters, Latimer, & Stinchfield, 1999) and comorbid psychiatric problems (Kaminer, Blitz, Burleson, & Sussman, 1998). Cognitive–behavioral approaches are also widely practiced with adolescent drug users (Bukstein, 1995), making them a critical target for further empirical validation.

This study investigated two manualized treatments for adolescent substance use: individual cognitive–behavioral therapy (CBT) and multidimensional family therapy (MDFT). These models were previously tested in a randomized controlled trial with inner-city, primarily ethnic minority adolescent drug abusers (Liddle & Hogue, 2001). Results of that study indicated that both treatments were effective in reducing marijuana use, externalizing symptoms, and internalizing symptoms at posttreatment and up to 1 year later, with MDFT showing some superiority in producing gains more rapidly and maintaining posttreatment gains at follow-up.

The main goals of the current study were to identify differences between CBT and MDFT in the use of specific therapy techniques and to link these process elements to outcomes found in the parent randomized trial. Specific therapy techniques are the technical aspects of a treatment model that derive directly from its fundamental theory- and practice-based principles (Elkin, Pilkonis, Docherty, & Sotsky, 1988). Advances in the technology of psychotherapy process research, including specification of clinically meaningful process variables and use of dimensional scales to measure therapist behavior (Greenberg, 1986; Schaffer, 1982; Sechrest, 1994), have spurred efforts to identify specific techniques that predict treatment outcome. A recent meta-analysis found that the effects of specific therapy components exceeded those of non-specific and facilitative factors, particularly for clients with more severe problems (Stevens, Hynan, & Allen, 2000). Nevertheless, the therapeutic potency of specific techniques (Messer & Wampold, 2002; Wampold et al., 1997), and the theoretical and methodological feasibility of discovering process–outcome correlations involving discrete technique variables (Stiles & Shapiro, 1994), are still in question.

This study examined the degree to which individual-focused and family-focused techniques within CBT and MDFT promote change in core behavioral symptoms related to adolescent substance abuse. Both approaches provide for some degree of therapist flexibility in targeting both individual and family functioning in order to achieve treatment goals. CBT focuses on changing the behaviors and cognitions of individual adolescents and works predominantly with teens alone in session. However, CBT therapists are trained to discuss salient issues pertaining to the youth’s relationships with parents and family members and, also, to meet periodically with care-givers. In the same vein, whereas MDFT targets family interactions directly and works predominantly with caregivers and other family participants in session, MDFT therapists also routinely hold individual sessions and work on the personal attitudes and behaviors of teens.

There are two primary study hypotheses: Greater use of adolescent-focused intervention techniques will predict improvement in CBT, whereas greater use of family-focused techniques will predict improvement in MDFT. Adolescent outcomes in drug use, externalizing symptoms, and internalizing symptoms were measured at pre- and posttreatment. Therapy techniques were measured using observational scales from a psychotherapy process instrument developed in a previous study on this sample (Hogue et al., 1998). The current study extended the Hogue et al. 1998 study by adding new items to the observational scales, conducting new exploratory factor analyses of the expanded scale, increasing the number of participants, and conducting process–outcome analyses.

Method

Participants

Clients

The sample was made up of 51 substance-abusing adolescents (67% male youths) and their families. The ethnic composition was 65% African American, 25% European American, and 10% Hispanic American. Adolescent characteristics included the following: mean age was 15.2 years (SD = 1.3); 84% of the adolescents were enrolled in school at intake, 63% were on juvenile probation, 22% were court ordered into treatment, and 16% attended previous drug counseling; 61% were living in single-parent households, 10% were living with both biological parents, and 29% had various other family compositions; and 37% had a yearly household income less than $10,000. Family characteristics included the following: 73% of mothers and 80% of fathers completed at least a high school education, 57% of mothers and 86% of fathers were employed at full- or part-time jobs, and 53% of the sample had a family member with previous criminal involvement. Adolescent substance use diagnoses were 75% marijuana dependence, 12% marijuana abuse, 16% alcohol dependence, 12% dependence on other substances; comorbid diagnoses were 69% conduct disorder, 61% oppositional defiant disorder, and 61% at least one mood disorder.

Therapists

The 12 therapists who delivered the treatments, 6 in each condition, ranged in age from 29 to 54 years (M = 40). Each condition had 3 men and 3 women, and each had 3 African American and 3 European American therapists. In the MDFT condition, 4 therapists had master’s degrees and 2 doctorates, with an average of 7.7 years (SD = 4.5) postgraduate experience in family therapy. In the CBT condition, 3 therapists had master’s and 3 doctorates, with an average of 3.5 years (SD = 1.7) postgraduate experience in cognitive–behavioral therapy.

Raters

Observational coding was completed by 11 undergraduate students (9 women): 2 African Americans, 5 European Americans, and 4 Hispanic Americans. Raters had no prior experience in process coding or with the treatment models and were naïve to treatment condition during coding.

Treatments

Both treatments were designed for office-based, weekly sessions conducted over 16–24 weeks. Therapists in both conditions completed 6 months of training prior to receiving study cases, were supervised directly by model developers in weekly individual meetings (both live and with videotape) and monthly group meetings, and showed strong treatment adherence and differentiation in a previous treatment integrity study (reported in Hogue et al., 1998).

Multidimensional family therapy (MDFT)

MDFT (Liddle, 2002) is a multicomponent, developmental–ecological treatment for adolescent drug abuse and related problems that seeks to reduce symptoms and enhance developmental functioning by facilitating change in several behavioral domains. Treatment targets within-family interactions as well as interactions between the family and relevant social systems, and particular intervention outcomes (e.g., emotional reconnection of caregivers to their adolescents) are understood to be the platforms from which other, more complex outcomes are attempted (e.g., changes in parenting practices). MDFT has four interdependent modules that target multiple aspects of adolescent and family functioning: (a) the adolescent module addresses developmental issues such as identity formation, peer relations, prosocial involvement, and drug use consequences; (b) the parent module enhances parenting skills in the areas of monitoring and limit setting, rebuilding emotional bonds with the adolescent and participating in the teen’s life outside of the family; (c) the interactional module facilitates change in family relationship patterns by helping families develop the motivation and skills to revitalize attachments and interact in more adaptive ways; and (d) the extrafamilial module seeks to establish collaborative relationships among all social systems in which the adolescent participates (i.e., family, school, peer, recreational, and juvenile justice). MDFT has been tested in a variety of clinical settings and subjected to numerous process studies; see Liddle and Hogue (2001) for a summary.

Individual cognitive–behavioral therapy

The CBT model for multiproblem adolescent substance abusers used in this study is based on a broadly defined cognitive–behavioral framework (Turner, 1992, 1993) that emphasizes adolescent coping skills and a harm-reduction approach to substance use. Treatment is divided into three stages. Treatment planning focuses on identifying and prioritizing adolescent problems and making a treatment contract in conjunction with both adolescent and caregiver. Parents, or their surrogates, participate in the first two sessions to facilitate support for the adolescent’s participation in treatment and to get parents’ perspectives on the youth’s strengths and problematic behaviors. Problems described by the adolescent and parents, in addition to problems reported by school and juvenile court, are used to develop a treatment plan. Intensive CBT program aims to increase coping competence and reduce problematic behavior, with intervention selection based on clinical need from multiple therapeutic modules. Typical therapeutic modules include the following: drug education, contingency contracting, coping and relaxation skills, communication and problem-solving skills, self-monitoring and cognitive distortions, and increasing prosocial activities. Specifically regarding substance abuse, harm reduction (Marlatt & Tapert, 1993), not abstinence, is the primary goal. Clients are taught to recognize behavioral and cognitive cues for cravings and drug use and to increase behavioral self-control. Termination focuses on treatment termination issues and relapse prevention. The goal is to enhance clients’ long-term self-management skills. Role rehearsal and problem solving are used to strengthen adolescents’ ability to resist against peer pressure to use drugs and engage in delinquent behavior.

Process Measure

Therapist Behavior Rating Scale (TBRS)

The TBRS (Hogue et al., 1998) is an observational rating system that assesses core theory-based intervention strategies prescribed by CBT and MDFT, including both individual and family techniques. This observer-based methodology for evaluating the implementation of model-specific techniques within manualized treatments has been widely adopted (Carroll et al., 1998; Hill, O’Grady, & Elkin, 1992; Morgenstern, Morgan, McCrady, Keller, & Carroll, 2001), and the psychometric properties of the TBRS are sound (Hogue et al., 1998). Two kinds of interventions were coded for this study sample (see Table 1): 12 therapist technique items that are scored based upon therapist behavior only, and 5 session focus items that are scored based on the content of therapist–client discussions. The session focus items were not used in the Hogue et al. study. For both kinds of items, raters estimate the extent to which items are observed during an entire session using a 7-point Likert scale ranging from 1 (not at all) to 7 (extensively). Both thoroughness and frequency are considered in making each rating. Thoroughness refers to the depth, complexity, or persistence with which the intervention is pursued. Frequency refers to the number of times throughout the session that a given intervention appears (regardless of thoroughness in any particular segment). Raters are instructed that complex interventions may be characterized by more than one scale item, although each item is theoretically independent of all others.

TABLE 1

Interrater Reliability and Item Content and Factor Loadings for the Two-Factor Solution for TBRS Items

Outcome Measures

Timeline Follow-Back Interview

The Time-line Follow-Back (TLFB) interview (Sobell & Sobell, 1996) measures quantity and frequency of daily consumption of drugs using a calendar and other memory aids to gather retrospective estimates. The TLFB is reliable and valid for the measurement of alcohol consumption and cigarette and cannabis use (Brandon, Copeland, & Saper, 1995; Breslin, Sobell, & Sobell, 1996; Sobell & Sobell, 1996). The TLFB has shown high temporal stability for measurement of alcohol consumption, with most test–retest correlations exceeding .85 (Fals-Stewart, O’Farrell, Freitas, McFarlin, & Rutigliano, 2000). Criterion validity has been established by comparing self- and collateral reports, as well as self-reports and records of verifiable events such as hospitalizations and jail stays (Fals-Stewart et al., 2000). For this study, the variable measuring the number of days out of the previous 30 during which the adolescent smoked marijuana was used.

Child Behavior Checklist (CBCL): Externalizing and Internalizing dimensions

The Revised Child Behavior Checklist (Achenbach & Edelbrock, 1983) is a parent self-report measure that assesses children’s behavioral problems and social competencies. The CBCL contains groupings of Externalizing (delinquent and aggressive) and Internalizing (withdrawn, anxious/depressed, and somatic complaints) symptoms (Achenbach & Edelbrock, 1983). One-week test–retest reliability of .93, and interparent reliability of .66 for Internalizing and .80 for Externalizing, have been shown (Achenbach, 1991). Content and criterion validity are supported by the ability of CBCL items to discriminate between matched referred and nonreferred youth (Achenbach, 1991). The CBCL also has excellent internal consistency, construct validity, and discriminant validity properties (Achenbach, 1991), and it has proven useful for assessing changes in behavior following psychotherapy (Webster-Stratton, 1984).

Procedure

Sampling design

Of the 51 cases included in this study, 26 received CBT and 25 received MDFT. Cases averaged a total of 16.1 (SD = 8.1) sessions across conditions; there was no difference in length of case between conditions. For the purposes of selecting sessions that were representative of the entire course of treatment, study cases were divided into three phases according to the following scheme: Sessions 1–5 (beginning phase), Sessions 6–14 (middle phase), Sessions 15+ (end phase). Fourteen percent of the study sample (n = 7) had participated in only Phase 1 of treatment, 27% (n = 14) had participated in Phases 1 and 2, and 59% (n = 30) had participated in all three phases. One session only from each case was randomly selected for coding. If a given case had only one videotaped session, that session was used. The final study sample included 10 cases (20%) with a Phase 1 session, 22 cases (43%) with a Phase 2 session, and 19 cases (37%) with a Phase 3 session.

Training raters

Raters were trained in weekly 2-hr meetings over a period of 5 months. Training consisted of didactic instruction and discussion of the coding manual, in-group coding and review of practice tapes, and exercises designed to increase understanding of the scale items. Raters demonstrated acceptable reliability for each TBRS item (intraclass correlation coefficient [ICC] > .60) before beginning to code study tapes. They continued to meet weekly for the duration of the study for supportive training and to prevent rater drift.

Ratings

Raters were naive to the study goals and to the fact that two different treatments were being evaluated. They were instructed that participant configuration in each session would vary according to the contingencies of each case and were informed that each intervention may or may not arise in any given session. Raters coded entire videotaped therapy sessions, which ranged from 30 to 75 min and averaged approximately 60 min per session. Raters were unaware of the treatment condition, therapist, and session number of the videotapes they coded. Two raters coded each videotape; raters were assigned to tapes using a randomized block design (Fleiss, 1981), such that coders were randomly paired with one another across the sample of tapes.

Plan of Analysis

The Results section contains four sequential parts. In Part 1, the 17 items of the TBRS are subjected to an exploratory factor analysis in order to derive an optimal set of core therapy process dimensions that characterize the current sample. In Part 2, preliminary analyses of the process scales derived in Part 1 are conducted to examine (a) basic therapist adherence to each treatment model, (b) differences between therapists in each condition with regard to utilization of model techniques, and (c) potential impact of pretreatment levels of client symptomatology on therapist use of core techniques. Part 3 then examines the main process–outcome hypotheses of the study for each condition. Part 4 further explores the unexpected process–outcome findings for one process scale.

Results

Part 1: Exploratory Factor Analysis of the TBRS

To examine the dimensionality of the TBRS, a principal-components analysis using maximum likelihood extraction and direct oblimin rotation (δ = 0; see Fabrigar, Wegener, MacCallum, & Strahan, 1999) was conducted on the average scores (mean of two raters per session) of all 17 TBRS items for the entire study sample.¹

The initial exploratory analysis extracted four factors with eigenvalues above one that accounted for 61% of the total variance; however, the pattern of item loadings failed to converge. One-, two-, and three-factor solutions were then extracted to determine the best solution. The three-factor solution also failed to converge. The one-factor solution accounted for only 29% of the total variance, while the two-factor solution accounted for 44% of the variance. Examination of the scree plot revealed a substantial drop in the magnitude of eigenvalues between the second and third factors. The two-factor solution was also the most theoretically viable, matching the theoretical factor structure almost exactly, and was therefore retained. The Kaiser–Meyer–Olkin measure of sampling adequacy was .68, indicating that correlations within the factor matrix were sufficient to support the procedure. Eigenvalues were 4.88 for Factor 1 and 2.54 for Factor 2.

The items composing each subscale, their reliability coefficients, and their factor loadings are contained in Table 1. Interrater reliability (ICC_(1,2); Shrout & Fleiss, 1979) for the TBRS items ranged from .31 to .88. For this type of data, an ICC of .60 is considered adequate, .70 is robust, and .80 is excellent. One item, Adolescent Investment, fell below the adequacy standard (ICC = .31) but was retained because of its theoretical importance and because it did not detract significantly from the overall reliability of its factor score. Following Grice (2001), factor-based subscales were created by interpreting the pattern matrix, setting a minimal factor loading threshold of .30, allowing items to load onto one factor only, and using a unit weighting method. Two items, Encourages Affect and Focus on Antisocial, fell below the loading threshold and were dropped from further analysis; one item, Focus on Family, was permitted to load onto both factors because of its extremely high loading on each and its status as a theoretical touchstone within this study. Items with negative loadings were reverse coded, and the mean of the item scores was calculated so that subscales would retain the scaling properties of the original items.

The first subscale, Family Focus, explained 29% of TBRS variance. It contains eight items representing direct interventions with families in session and therapeutic focus on family relationship issues (see Table 1). Factor loadings ranged from .43 to .80 and were highest for Family Attachment, Focus on Family, and Core Relational Themes. Interrater reliability (ICC = .87) and internal consistency (Cronbach’s α = .81) were strong. The second subscale, Adolescent Focus, explained 15% of scale variance. It contains eight items related to exploring the adolescent’s extra-familial functioning and avoiding parent- and family-centered interventions. Factor loadings ranged from .41 to .79 and were highest for Adolescent Ecosystem and Focus on Peers. Interrater reliability (ICC = .92) and internal consistency (Cronbach’s α = .76) were strong as well. The correlation between subscales was significant, Pearson’s r(51) = −.45, p < .001, and moderate, indicating that the use of one roster of interventions during any given session attenuated, but did not preclude, use of interventions from the other roster.

Part 2: Preliminary Analyses of Process Effects

Model differentiation

Independent samples t tests (correcting degrees of freedom to adjust for unequal variances) were conducted on the two process scales to determine the extent to which therapists in each condition adhered to their respective treatments (see Table 2). Consistent with the therapeutic principles of each model, MDFT had a higher score on Family Focus, t(41) = 4.99, p < .001, whereas CBT had a higher score on Adolescent Focus, t(33) = −5.97, p < .001. Analysis of session composition showed that for MDFT cases, 16% of sessions included the adolescent alone and 84% included family members. For CBT, 92% of sessions were with the adolescent alone and 8% with family members.

TABLE 2

Differences in Use of Therapist Techniques by Treatment Condition

Therapist main effects

Therapist main effects refer to potential mean-level differences (i.e., heterogeneity) among multiple therapists in a given study with respect to implementing treatment models or producing client outcomes. It has been argued that individual therapists’ contributions to client outcomes are a primary, or even predominant, factor in demonstrating differences between competing treatments (Crits-Christoph & Mintz, 1991). It is, therefore, critical to diagnose and control for possible therapist effects when examining clinical trial data with multiple therapists. This study examined therapist effects on both the process and outcome variables. First, therapist differences in utilization of adolescent-and family-focused techniques were examined for each condition in four separate analyses of variance (ANOVAs); Therapist was entered as a Fixed-Factor 4, and the process variable was entered as the dependent variable. Second, therapist differences in outcomes were examined in six separate analyses of covariance (ANCOVAs), one for each of three outcomes within the two conditions; Therapist was again entered as a Fixed-Factor 4, pretreatment score on the given outcome (drug use, externalizing, or internalizing) as a covariate, and posttreatment outcome score as the dependent variable. This method of examining fixed-factor therapist main effects is common in contemporary treatment research (e.g., Huppert, Bufka, Barlow, Gorman, Shear, & Woods, 2001; Project MATCH Research Group, 1998).² Also, conducting separate ANOVAs and ANCOVAs for each variable maximizes the likelihood of detecting significant effects, especially for small sample sizes. Across the 10 therapist effects analyses, only one significant effect was found: Therapist predicted Adolescent Focus within the MDFT condition, F(5, 24) = 3.06, p < .05, indicating substantial variability among MDFT therapists in their emphasis on adolescent-centered techniques.

Correlations between baseline symptoms and process scales

Before examining the impact of treatment implementation on client outcome, it is important to determine whether baseline client functioning affects how therapists implement treatment. First, bivariate correlations were computed for pretreatment levels of all outcome (drug use, externalizing, and internalizing) and process (adolescent focus and family focus) variables. No significant correlations were found between the three baseline variables and the two process variables. In addition, as expected, externalizing behavior was associated with internalizing behavior, r(51) = .60, p < .001; however, neither was correlated with drug use. Second, hierarchical linear regressions were conducted to investigate whether pretreatment symptoms predicted therapist implementation while controlling for treatment condition. Six separate regressions were conducted, with the given baseline outcome variable and treatment model (MDFT and CBT) entered as predictors in Step 1, their interaction entered in Step 2 (predictor variables were centered around the mean to reduce multicollinearity; Aiken & West, 1991), and each TBRS process scale entered as the dependent variable. No significant interaction effects were found. Thus, overall there was no evidence that client baseline functioning affected how therapists in either condition utilized adolescent- and family-focused techniques.

Part 3: Process–Outcome Analyses

Six hierarchical regressions were conducted to investigate whether therapeutic focus predicted treatment gains separately for each outcome: drug use, externalizing, and internalizing. In all regression equations, pretreatment symptom level (drug use, externalizing, and internalizing) was entered in Step 1, TBRS subscale score (Family Focus and Adolescent Focus) in Step 2, treatment model (MDFT and CBT) in Step 3, and the interaction between TBRS score and treatment model in Step 4; posttreatment symptom level was the dependent variable. This order of entry permits strong inference about how well the given process variable (Step 2) and the interaction term (Step 3) predict change in the given outcome. The process variable term (Step 2) revealed the effect of therapist intervention focus on adolescent outcome across both conditions. The interaction term (Step 3) revealed whether intervention focus affected outcomes differently for MDFT versus CBT cases. See Table 3, which presents results of the three regressions that include Family Focus.

TABLE 3

Hierarchical Regression Analyses of Family Focus Predicting Adolescent Outcomes

The process variables were tested in separate equations, rather than simultaneously in one equation for each outcome, for two reasons intended to promote the interpretability of results: (a) avoiding multicollinearity problems stemming from the moderate correlation (r = −.45) between Family Focus and Adolescent Focus across conditions and (b) maintaining a reasonable ratio of sample size to predictors (roughly 13:1), thereby limiting threats to statistical validity. Also, the outcome variables were tested as dependent variables in separate regressions, rather than in a single canonical correlation, because of a priori interest in how therapeutic focus might differentially affect each kind of symptom, and because the pattern of intercorrelations among them (nonsignificant or moderate-size correlations; see Part 2 in the Results section) suggested that each offered unique information about client well-being.

The small sample size in this study heightened concern about the potential influence of outliers. For each regression, multivariate outlier diagnostics—studentized residuals, leverage, Cook’s D, and standardized DfBeta—were examined (Tabachnick & Fidell, 2001). Cases that were above the critical value on at least two of these four indices were considered outliers. Each equation produced one or two outlier values, all belonging to different cases. In response, all regressions were then run twice, with and without the outliers. No differences were found for drug use and externalizing. However, removing outliers changed the results for both equations that predicted internalizing symptoms; we report only those results with outliers removed.

Neither Adolescent Focus nor the Adolescent Focus × Treatment Condition interaction term predicted any of the three outcomes. Results for Family Focus are depicted in Table 3. Note that Cohen (1988) identified d = .50 as a moderate effect and d = .80 as a large effect. For drug use, a trend-level main effect was found for Family Focus (β = −.22, p = .10, Cohen’s d = .47), with higher levels of Family Focus predicting greater reduction in drug use. For externalizing symptoms, there was a main effect for Family Focus (β = −.23, p < .05, d = .62) and a trend for Family Focus × Treatment Condition (β = −.17, p = .10). This interaction was probed by reconducting the regressions separately for each treatment condition (Aiken & West, 1991). A substantive effect for Family Focus emerged only for the CBT condition (β = −.26, p < .10, d = .83), indicating that greater use of family-focused techniques predicted decline in externalizing behavior specifically for CBT cases. For internalizing symptoms, there was a main effect for Family Focus (β = −.26, p < .05, d = .74) and a trend for Family Focus × Treatment Condition (β = −.21, p < .10). Analysis of the interaction revealed effects for Family Focus within both CBT (β = −.31, p < .05, d = 1.10) and MDFT (β = −.37, p < .10, d = .81). These findings reveal that family-focused interventions facilitated improvement in internalizing symptoms for both conditions, with somewhat larger gains in CBT.

Part 4: Exploratory Analysis of the Family Focus Subscale

To further understand the role of family-focused interventions in predicting outcomes, particularly within the individually based CBT condition, the eight items of the Family Focus subscale were divided into two clusters that represent a logical division of family-focused therapeutic methods. The Family Content cluster contains five items representing discussion of family-related treatment themes with the adolescent or other family members: Family Attachment, Focus on Family, Core Relational Themes, Normative Development, Parental Monitoring. The Family Interaction cluster contains three items that involve direct intervention with other family members in session: Coaches Interactions, Prepares for Interactions, and Targets Others. Table 4 presents basic psychometric properties and between-conditions mean comparisons for both clusters. Interrater reliability and internal consistency were sound for both clusters, with one exception. As expected, Family Interaction techniques were virtually nonexistent in the CBT condition and their internal consistency was negligible; recall that only 8% of CBT sessions included any family members. The clusters were positively correlated for both MDFT, r(25) = .53, p < .01, and CBT, r(26) = .34, p < .10, and MDFT therapists had significantly higher scores than CBT therapists for both clusters.

TABLE 4

Basic Psychometric Properties and Between-Condition Mean Comparisons of the Two Family Focus Clusters

Hierarchical regressions identical to those described earlier were conducted on each cluster separately within condition. Based on results presented previously, all three treatment outcomes were examined, excluding externalizing symptoms for MDFT cases. No effects in either condition were found for drug use. For CBT, Family Content only predicted decreases in both externalizing (β = −.26, p < .05, d = .85) and internalizing (β = −.33, p < .01, d = .1.19), suggesting that CBT therapists’ focus on family-related therapeutic content led to improved outcomes even in the absence of direct contact with family members. Similarly, for MDFT cases Family Content only predicted decline in internalizing symptoms (β = −.42, p < .05, d = .97).

Discussion

The main finding of this study is that success in treating adolescent drug abuse and cooccurring symptoms was related to in-session focus on family-related treatment themes. Moreover, the benefits of focusing on family-related content and themes were as strong within individual cognitive–behavioral therapy as within multidimensional family therapy. These findings are in accord with the consensus that family conflict, parent–child detachment, and deficient parenting skills are primary etiologic factors for adolescent substance use (Repetti, Taylor, & Seeman, 2002) and are thus logical targets of treatment. Overall, results underscore the critical importance of attending to family risk and protective factors in the treatment of adolescent drug use, regardless of the particular theoretical orientation of the therapist.

Results also suggest a partial yet intriguing explanation for the relative superiority of family-based approaches in treating adolescent drug abuse (Williams et al., 2000): They are, at the very least, sure to make family-related issues central to the treatment agenda. It is true that family interaction techniques, a specific type of family therapy technique that engages family members as cotargets of therapeutic activity during conjoint sessions, are the traditional foundation of most family therapy models. In this study, however, interventions that targeted family themes, but not those that required family member participation in session, predicted treatment gains. This was true in the family condition as well as the individual condition. These findings do not imply that interaction techniques can be deemed expendable or second-class features of family therapies for teen drug problems. Rather, they contribute to the sparse literature on parent–family involvement in therapy with adolescents (Weisz & Hawley, 2002) and underscore the fact that family involvement in childhood treatments is a continuum that can vary in scope and intensity, depending on the nature of the disorder and the treatment model being applied (Fauber & Long, 1991).

It was surprising that family-focused interventions predicted outcomes across the board in individual therapy, particularly given that CBT therapists used them much less extensively than adolescent-focused interventions. Specifically, CBT sessions with the highest ratings for family content techniques (one standard deviation above the mean) registered on average below mid-point of the scale, falling roughly at the “Somewhat” anchor. Apparently, though used sparingly, they were used to great effect. Note that exploring family-related themes in session, especially at a moderate level, was permitted by the CBT protocol and was not an adherence violation. Results do not imply that family-focused techniques were the primary therapeutic agent in these CBT cases, only that, all other things being equal, cases with more family focus tended to have better outcomes. Also, these findings do not diminish the theoretical and clinical significance of well-articulated adolescent-focused interventions for effectively implementing both CBT and MDFT. In fact, it may well be true that a certain threshold of adolescent-focused techniques, in addition to focus on family themes, is required for treatment success in both models. Certainly, therapists in both conditions used greater mean levels of individual techniques than family techniques, even though use of individual techniques did not vary with treatment outcome.

This study attempts to advance knowledge about implementing evidence-based treatments for adolescent drug abuse, a pervasive psychological disorder and public health problem. Both study conditions were manualized treatments with excellent fidelity. There were few differences between therapists within conditions in utilizing specific techniques and no differences in outcome success; also, there was no evidence that therapists adjusted their utilization in response to pretreatment symptom levels. In these respects, the study demonstrated high internal validity. Some have argued that tight control of the treatment variable in randomized trials reduces the possibility of finding process–outcomes correlations because variance in implementation is stripped away (Gaston & Gagnon, 1996). Even so, medium-to-large effects were observed.

Given the small sample size, care was taken to identify and remove multivariate outliers that unduly influence observed relations among variables. Nonetheless, the study design was limited by sample size, primarily in the need to conduct exploratory rather than confirmatory factor analysis of the empirical factor structure of the TBRS. Also, liberties were taken in interpreting trend-level effects (ps < .10), which was considered justifiable in light of the sample size, the medium-to-large effect sizes for all process–outcome correlations, and the inherently conservative nature of using multiple regression to model interaction effects for field data (McClelland & Judd, 1993). Because the design was not purely experimental—the extent of adolescent focus and family focus was not randomly assigned to cases—it cannot be ruled out that those clients who evoked more family focus in session were also those more inclined to improve. Finally, results must be deemed preliminary until confirmed with larger samples and long-term outcome assessments.

Because the study did not measure therapeutic alliance, therapist competence, or other psychotherapy common factors, the study cannot directly enter the debate about the relative importance of specific versus common elements (for recent position papers, see Beutler, 2002, and Messer & Wampold, 2002), nor can we rule out the possibility that common factors were a third-variable influence working behind the scenes to bolster observed technique–outcome relations. Still, results favor the contention that specific therapy techniques can directly facilitate client improvement within the context of theory driven, flexibly applied treatments for specified populations (Beutler, 2002; Sechrest, 1994), perhaps especially for clients with more severe impairment (Stevens et al., 2000). More important, the findings suggest a tangible option for real-world therapists who prefer working alone with adolescents and face insurmountable barriers to including family members in treatment for adolescent drug problems: incorporate work on family themes into treatment plans. This simple directive, if verified by future studies, may prove to be a valuable common ingredient in the training protocols of research groups who hope to disseminate empirically supported treatments and train front-line clinicians to deliver either individual-based or family-based approaches for adolescent drug abuse.

Acknowledgments

Preparation of this article was supported by a grant from the National Institute on Drug Abuse (R03 DA12993). Portions of this article were presented in August 2000 at the 108th Annual Meeting of the American Psychological Association in Washington, DC. We are grateful for the diligent work of the observational raters: Hector Benitez, Tania Capparelli, Maria Delvecchio, Carolina Gil, Alexis Kramer, Jennifer Manzetti, Tatiana Rich, Maria Rivera, Christa Sanfilippo, Melissa Sanfilippo, and Clifford Vital.

Footnotes

¹The 3:1 ratio of sample size (N = 51) to items factored (17 scale items) is lower than the 5:1 minimum ratio typically recommended (Tabachnick & Fidell, 2001). However, because Kaiser’s measure of sampling adequacy was robust (KMO = .68), and because results strongly conformed to the theoretical factor structure of the scale, the factor analysis was considered sound for supporting the final composition of the subscales (see Fabrigar et al., 1999).

²The nested designs of most randomized clinical trials—multiple therapists practicing within each treatment condition—gives rise to a second concern as well: therapist clustering effects. Clustering effects relate to the fact that clients who share the same therapist may have outcomes more highly correlated to one another than to clients treated by different therapists. That is, the error terms of clients within therapist are correlated, thus violating the assumption of independent observations (Wampold & Serlin, 2000). Ignoring clustering effects can lead to biased standard errors of parameter estimates and inflated Type I error rates when using ordinary least squares regression (Norton, Bieler, Ennett, & Zarkin, 1996; Zucker, 1990). Procedures are available for diagnosing clustering effects and adjusting significance tests accordingly (e.g., Hedeker, Gibbons, & Flay, 1994; Wampold & Serlin, 2000). However, the current study does not contain a sufficient number of average clients per therapist to calculate reliable intraclass correlation coefficients (Hedeker et al., 1994; Norton et al., 1996), a necessary first step in correctly diagnosing significant therapist clustering.

Contributor Information

AARON HOGUE, National Center on Addiction and Substance Abuse at Columbia University.

SARAH DAUBER, National Center on Addiction and Substance Abuse at Columbia University.

HOWARD A. LIDDLE, Center for Treatment Research on Adolescent Drug Abuse, University of Miami School of Medicine.

JESSICA SAMUOLIS, Mailman School of Public Health, Columbia University College of Physicians and Surgeons.

References

Achenbach TM. Manual for the Child Behavior Checklist/4–18 and 1991 Profile. Burlington, VT: University of Vermont; 1991.
Achenbach TM, Edelbrock C. Manual for the Child Behavior Checklist and Revised Child Behavior Profile. Burlington, VT: University of Vermont; 1983.
Aiken LS, West SG. Multiple regression: Testing and interpreting interactions. Newbury Park, CA: Sage; 1991.
Beutler LE. The dodo bird is extinct. Clinical Psychology: Science and Practice. 2002;9:30–34.
Brandon TH, Copeland AL, Saper ZL. Programmed therapeutic messages as a smoking treatment adjunct: Reducing the impact of negative affect. Health Psychology. 1995;14:41–47. [PubMed]
Breslin C, Sobell LC, Sobell MB. Aftercare telephone contacts with problem drinkers can serve a clinical and research function. Addiction. 1996;91:1359–1364. [PubMed]
Bukstein OG. Adolescent substance abuse: Assessment, prevention, and treatment. New York: Wiley; 1995.
Carroll KM, Connors GJ, Cooney NL, Di-Clemente CC, Donovan DM, Kadden RR, et al. Internal validity of the Project MATCH treatments: Discriminability and integrity. Journal of Consulting and Clinical Psychology. 1998;66:290–303. [PubMed]
Cohen J. Statistical power analysis for the behavioral sciences. 2. Hillsdale, NJ: Erlbaum; 1988.
Crits-Christoph P, Mintz J. Implications of therapist effects for the design and analysis of comparative studies of psychotherapies. Journal of Consulting and Clinical Psychology. 1991;59:20–26. [PubMed]
Department of Health and Human Services. Substance abuse—A national challenge: Prevention, treatment, and research at HHS. Washington, DC: Author; 2000.
Elkin I, Pilkonis P, Docherty J, Sotsky S. Conceptual and methodological issues in comparative studies of psychotherapy and pharmacotherapy. I: Active ingredients and mechanisms of change. American Journal of Psychiatry. 1988;145:909–917. [PubMed]
Fabrigar LR, Wegener DT, MacCallum RC, Strahan EJ. Evaluating the use of exploratory factor analysis in psychological research. Psychological Methods. 1999;4:272–299.
Fals-Stewart W, O’Farrell TJ, Freitas TT, McFarlin SK, Rutigliano P. The Timeline Follow-Back reports of psychoactive substance use by drug-abusing patients: Psychometric properties. Journal of Consulting and Clinical Psychology. 2000;68:134–144. [PubMed]
Fauber RL, Long N. Children in context: The role of family in child psychotherapy. Journal of Consulting and Clinical Psychology. 1991;59:813–820. [PubMed]
Fleiss JL. Balanced incomplete block designs for inter-rater reliability studies. Applied Psychological Measurement. 1981;5:105–112.
Gaston L, Gagnon R. The role of process research in manual development. Clinical Psychology: Science and Practice. 1996;3:13–24.
Gfroerer J. 1994 National Household Survey of Drug Abuse (Advance Report No. 10) Washington, DC: Office of Applied Studies, Substance Abuse and Mental Health Services Administration; 1995.
Greenberg LS. Change process research. Journal of Consulting and Clinical Psychology. 1986;54:4–9. [PubMed]
Grice JW. A comparison of factor scores under conditions of factor obliquity. Psychological Methods. 2001;6:67–83. [PubMed]
Hedeker D, Gibbons RD, Flay BR. Random-effects regression models for clustered data with an example from smoking prevention research. Journal of Consulting and Clinical Psychology. 1994;62(4):757–765. [PubMed]
Henggeler SW, Borduin CM, Melton GB, Mann BJ, Smith LA, et al. Effects of multisystemic therapy on drug use and abuse in serious juvenile offenders: A progress report from two outcome studies. Family Dynamics of Addiction Quarterly. 1991;1(3):40–51.
Henggeler SW, Melton GB, Smith LA, Schoenwald SK, Hanley JH. Family preservation using multisystemic treatment: Long-term follow-up to a clinical trial with serious juvenile offenders. Journal of Child and Family Studies. 1993;2:283–293.
Hill CE, O’Grady KE, Elkin I. Applying the Collaborative Study Psychotherapy Rating Scale to rate therapist adherence in cognitive–behavioral therapy, interpersonal therapy, and clinical management. Journal of Consulting and Clinical Psychology. 1992;60:73–79. [PubMed]
Hogue A, Liddle HA, Rowe C, Turner RM, Dakof GA, LaPann K. Treatment adherence and differentiation in individual versus family therapy for adolescent drug abuse. Journal of Counseling Psychology. 1998;45:104–114.
Huppert JD, Bufka LF, Barlow DH, Gorman JM, Shear MK, Woods SW. Therapists, therapist variables, and cognitive–behavioral therapy outcome in a multicenter trial for panic disorder. Journal of Consulting and Clinical Psychology. 2001;69(5):747–755. [PubMed]
Johnston LD, O’Malley PM, Bachman JG. National survey results on drug use from the Monitoring the Future Study 1975–1993. Washington, DC: National Institute on Drug Abuse; 1995.
Kaminer Y, Blitz C, Burleson JA, Sussman J. The Teen Treatment Services Review (T-TSR) Journal of Substance Abuse Treatment. 1998;15:291–300. [PubMed]
Kazdin AE. Psychotherapy for children and adolescents. In: Bergin AE, Garfield SL, editors. Handbook of psychotherapy and behavior change. 4. New York: Wiley; 1994. pp. 543–594.
Kazdin AE. Current (lack of) status of theory in child and adolescent psychotherapy research. Journal of Clinical Child Psychology. 1999;28:533–543. [PubMed]
Kazdin AE. Bridging the enormous gaps of theory with therapy research and practice. Journal of Clinical Child Psychology. 2001;30:59–66. [PubMed]
Kilpatrick DG, Acierno R, Saunders B, Resnick HS, Best CL, Schnurr PP. Risk factors for adolescent substance abuse and dependence: Data from a national sample. Journal of Consulting and Clinical Psychology. 2000;68:19–30. [PubMed]
Liddle HA. Cannabis Youth Treatment [CYT] manual series. Vol. 5. Rockville, MD: Center for Substance Abuse Treatment, Substance Abuse and Mental Health Services Administration; 2002. Multidimensional Family Therapy (MDFT) treatment for adolescent cannabis users; p. 244. Information available at www.samhsa.gov/csat/csat.htm.
Liddle HA, Dakof GA, Parker K, Diamond GS, Barrett K, Tejeda M. Multidimensional family therapy for adolescent drug abuse: Results of a randomized clinical trial. American Journal of Drug and Alcohol Abuse. 2001;27:651–688. [PubMed]
Liddle HA, Hogue A. Multidimensional family therapy for adolescent substance abuse. In: Wagner EF, Waldron HB, editors. Innovations in adolescent substance abuse interventions. Amsterdam, The Netherlands: Pergamon/Elsevier Science; 2001. pp. 229–261.
Mann BJ, Borduin CM, Henggeler SW, Blaske DM. An investigation of systemic conceptualizations of parent–child coalitions and symptom change. Journal of Consulting and Clinical Psychology. 1990;58:336–344. [PubMed]
Marlatt GA, Tapert SF. Harm reduction: Reducing the risk of addictive behavior. In: Baer JS, Marlatt GA, McMahon B, editors. Addictive behaviors across the lifespan. Newbury Park, CA: Sage; 1993. pp. 243–273.
McClelland GH, Judd CM. Statistical difficulties of detecting interactions and moderator effects. Psychological Bulletin. 1993;114:376–390. [PubMed]
Messer SB, Wampold BE. Let’s face facts: Common factors are more potent than specific therapy ingredients. Clinical Psychology: Science & Practice. 2002;9:21–25.
Morgenstern J, Morgan TJ, McCrady BS, Keller DS, Carroll KM. Manual-guided cognitive–behavioral therapy training: A promising method for disseminating empirically supported substance abuse treatments to the practice community. Psychology of Addictive Behaviors. 2001;15:83–88. [PubMed]
Nathan PE, Stuart SP, Dolan SL. Research on psychotherapy efficacy and effectiveness: Between Scylla and Charybdis? Psychological Bulletin. 2000;126:964–981. [PubMed]
Norton EC, Bieler GS, Ennett ST, Zarkin GA. Analysis of prevention program effectiveness with clustered data using generalized estimating equations. Journal of Consulting and Clinical Psychology. 1996;64(5):919–926. [PubMed]
Project MATCH Research Group. Therapist effects in three treatments for alcohol problems. Psychotherapy Research. 1998;8(4):455–474.
Repetti RL, Taylor SE, Seeman TE. Risky families: Family social environments and the mental and physical health of offspring. Psychological Bulletin. 2002;128:330–366. [PubMed]
Rounsaville BJ, Carroll KM, Onken LS. A stage model of behavioral therapies research: Getting started and moving on from Stage I. Clinical Psychology: Science and Practice. 2001;8:133–142.
Schaffer ND. Multidimensional measures of therapist behavior as predictors of outcome. Psychological Bulletin. 1982;92:670–681. [PubMed]
Schmidt SE, Liddle HA, Dakof GA. Changes in parenting practices and adolescent drug abuse during multidimensional family therapy. Journal of Family Psychology. 1996;10:12–27.
Schoenwald SK, Ward DM, Henggeler SW, Pickrel SG, Patel H. MST treatment of substance abusing or dependent adolescent offenders: Costs of reducing incarceration, inpatient, and residential placement. Journal of Child and Family Studies. 1996;5:431–444.
Sechrest L. Recipes for psychotherapy. Journal of Consulting and Clinical Psychology. 1994;62:952–954. [PubMed]
Shrout PE, Fleiss JL. Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin. 1979;86:420–428. [PubMed]
Sobell LC, Sobell MB. Timeline Followback user’s guide: A calendar method for assessing alcohol and drug use. Toronto, Ontario, Canada: Addiction Research Foundation; 1996.
Stanton MD, Shadish WR. Outcome, attrition, and family–couples treatment for drug abuse: A meta-analysis and review of the controlled, comparative studies. Psychological Bulletin. 1997;122:170–191. [PubMed]
Stevens SE, Hynan MT, Allen M. A meta-analysis of common factor and specific treatment effects across the outcome domains of the phase model of psychotherapy. Clinical Psychology: Science and Practice. 2000;7:273–290.
Stiles WB, Shapiro DA. Disabuse of the drug metaphor: Psychotherapy process–outcome correlations. Journal of Consulting and Clinical Psychology. 1994;63:942–948. [PubMed]
Tabachnick BG, Fidell LS. Using multivariate statistics. 4. Boston, MA: Allyn & Bacon; 2001.
Turner RM. Launching cognitive–behavioral therapy for adolescent depression and drug abuse. In: Budman S, Hoyt M, Friedman S, editors. Casebook of brief therapy. New York: Guilford Press; 1992. pp. 135–156.
Turner RM. Dynamic cognitive–behavioral therapy. In: Giles T, editor. Handbook of effective psychotherapy. New York: Plenum; 1993. pp. 437–454.
Waldron HB, Slesnick N, Brody JL, Turner CW, Peterson TR. Treatment outcomes for adolescent substance abuse at 4- and 7-month assessments. Journal of Consulting and Clinical Psychology. 2001;69:802–813. [PubMed]
Wampold BE, Mondin GW, Moody M, Stich F, Benson K, Ahn H. A meta-analysis of outcome studies comparing bona fide psychotherapies: Empirically, “All must have prizes. Psychological Bulletin. 1997;122:203–215.
Wampold BE, Serlin RC. The consequence of ignoring a nested factor on measures of effect size in analysis of variance. Psychological Methods. 2000;5(4):425–433. [PubMed]
Webster-Stratton C. Randomized trial of two parent-training programs for families with conduct-disordered children. Journal of Consulting and Clinical Psychology. 1984;52:666–667. [PubMed]
Weinberg NZ, Rahdert E, Colliver JD, Glantz MD. Adolescent substance abuse: A review of the past 10 years. Journal of the American Academy of Child and Adolescent Psychiatry. 1998;37:252–261. [PubMed]
Weisz JR, Donenberg GR, Han SS, Weiss B. Bridging the gap between laboratory and clinic in child and adolescent psychotherapy. Journal of Consulting and Clinical Psychology. 1995;63:688–701. [PubMed]
Weisz JR, Hawley KM. Developmental factors in the treatment of adolescents. Journal of Consulting and Clinical Psychology. 2002;70:21–43. [PubMed]
Williams RJ, Chang SY. ACARG. A comprehensive and comparative review of adolescent substance abuse treatment outcome. Clinical Psychology: Science and Practice. 2000;7:138–166.
Winters KC, Latimer WL, Stinchfield RD. Adolescent treatment for alcohol and other drug abuse. In: Tarter T, Ammerman RT, Ott PJ, editors. Source book on substance abuse: Etiology, methodology, and intervention. Boston: Allyn & Bacon; 1999. pp. 350–361.
Zucker DM. An analysis of variance pitfall: The fixed effects analysis in a nested design. Educational and Psychological Measurement. 1990;50:731–738.