|Home | About | Journals | Submit | Contact Us | Français|
To evaluate the utility of the Child Behavior Check list (CBCL) for identifying children of parents with panic disorder or major depression at high-risk for future psychopathology.
Baseline Internalizing and Externalizing CBCL T-scores were used to predict subsequent depressive, anxiety, and disruptive behavior disorders at a five-year follow-up in children of parents with panic disorder, major depression, or neither disorder.
The Internalizing scale predicted subsequent agoraphobia, generalized anxiety disorder, separation anxiety disorder, and social phobia. In contrast, the Externalizing scale predicted subsequent disruptive behavior disorders and major depression.
The convergence of these results with previous findings based on structured diagnostic interviews suggests that the CBCL broad-band scales can inexpensively and efficiently help identify children at high risk for future psychopathology within a population of children already at risk by virtue of parental psychopathology.
In a series of cross-sectional (Biederman, Rosenbaum, Bolduc, Faraone, & Hirshfeld, 1991) and longitudinal studies (Biederman, Petty, Faraone et al., 2006; Biederman, Petty, Hirshfeld-Becker et al., 2006) of children at high and low risk for panic disorder and major depression, our group documented a divergent pattern of risk for psychopathology in children at risk. These studies documented a divergent pattern of risk in which parental panic disorder selectively increased the risk for anxiety disorders in the offspring while parental major depression selectively increased the risk for major depression and disruptive behavior disorders in the offspring.
Although these longitudinal findings clarified and extended previous cross-sectional studies providing support for the transmission of disorders from parents to their high risk offspring (Biederman et al., 2001; Biederman et al., 2004; Biederman, Rosenbaum, Bolduc, Faraone, & Hirshfeld, 1991; Crowe, Noyes, Pauls, & Slymen, 1983; Faraone & Biederman, 1997; Goldstein et al., 1994; Hettema, Prescott, & Kendler, 2001; Noyes et al., 1986; Weissman, Leckman, Merikangas, Gammon, & Prusoff, 1984), these studies relied upon structured diagnostic interviews, a very time intensive and expensive methodology. Thus, simpler and more cost effective approaches to identifying offspring at high risk for psychopathology are needed.
One such approach is the Child Behavior Checklist (CBCL). The CBCL is a paper and pencil instrument with excellent psychometric properties. Good convergence between structured interview-derived diagnostic categories and syndrome-congruent CBCL scales has been well documented (Biederman et al., 1993; Biederman, Monuteaux, Kendrick, Klein, & Faraone, 2005; Biederman et al., 1995; Chen, Faraone, Biederman, & Tsuang, 1994; Edelbrock & Costello, 1988; Faraone, Althoff, Hudziak, Monuteaux, & Biederman, 2005; Geller, Warner, Williams, & Zimerman, 1998; Geller et al., 2006; Kazdin & Heidish, 1984). Findings from Koot and Verhulst (Koot & Verhulst, 1992) and Verhulst and Vander Ende (Verhulst, Koot, & Van der Ende, 1994) showed that internalizing and externalizing problems predicted referral to mental health services in 4-year and 6-year follow-ups of a population-based sample, respectively. Kroes et al. (Kroes et al., 2002) found that externalizing problems predicted ADHD, conduct disorders, and elimination disorders while internalizing problems predicted mood and anxiety disorders 1.5 years later in 5- to 6-year-old children from the general population. Mesman and Koot (Mesman & Koot, 2001) showed that internalizing problems were predictive of at least one DSM-IV anxiety or affective disorder 8 years later and externalizing problems were predictive of at least one disruptive behavior disorder 8 years later.
Because the CBCL contains broad-band Internalizing and Externalizing scales that largely correspond to mood and anxiety disorders and disruptive behavior disorders, respectively, they can be particularly suitable for the investigation of whether they can predict divergent patterns of risk from early childhood to adolescent years in children at risk. That is, broad behavioral conditions in early childhood may be predictors of later, more specific psychopathology. If effective in predicting psychopathological outcomes, the CBCL would be an inexpensive method for identifying at-risk children in need of preventative or early intervention strategies.
In this prospective analysis, we evaluated the use of the Internalizing and Externalizing broad-band scales of the CBCL for identifying children at risk for mood, anxiety, and disruptive behavior disorders. We sought to expand on the existing longitudinal literature by using outcomes of major depression and individual anxiety disorders rather than the previously studied larger categories of mood, anxiety, or internalizing disorders. Based on the literature, we hypothesized that the Internalizing scale would identify children at risk for mood and anxiety disorders and the Externalizing scale would identify children at risk for disruptive behavior disorders.
As previously described (Biederman et al., 2001; Biederman, Petty, Hirshfeld-Becker et al., 2006), parents with panic disorder and major depression were recruited from clinical referrals and advertising in the local media. Comparison parents were recruited through advertisements to hospital personnel and in community newspapers. Control proband selection was guided by contemporary epidemiologic methodology, which dictates that the sampling of controls should be drawn from the exposure distribution of the source population that gave rise to the cases (Miettinen, 1985). Comparison parents were free of major anxiety disorders or mood disorders. Parents with PD and/or MD were selected on the basis of having been treated for these disorders. There were no constraints on treatment in offspring. For this analysis, we used children with CBCL data at baseline that had psychiatric assessments at follow-up (N = 248). Offspring of PD + MD Parents (N = 107) came from 70 families, Offspring of PD Parents (N = 17) from 11 families, Offspring of MD Parents (N = 46) from 29 families, and Offspring of Controls (N = 78) from 40 families. Both parents were assessed in all families. The institutional review board at Massachusetts General Hospital approved this study protocol. All parents signed written consent and children assented to study procedures.
Data were collected at two time points. The first was the baseline assessment, conducted when the sample was originally recruited, between 1993 and 1998. The second was conducted approximately five years later, between 1999 and 2004. Because some families were easier to reach at follow-up than others, there was variability in the follow-up period (mean = 4.8 years, SD = 1.5).
Psychiatric assessments of the children at baseline relied on the DSM-III-R criteria and were previously described (Biederman et al., 2001). Assessments at follow-up were based on the DSM-IV Schedule of Affective Disorders and Schizophrenia for school-aged children –Epidemiological Version (K-SADS-E) (Orvaschel, 1994) for subjects < 18 years and the Structured Clinical Interview for DSM-IV (SCID) (First, Spitzer, Gibbon, & Williams, 1997) (supplemented with modules from the K-SADS-E to assess childhood diagnoses) for subjects ≥18 years. We conducted indirect interviews with the mothers for all subjects and direct interviews with subjects ≥12 years. We considered a disorder positive if DSM-IV diagnostic criteria were unequivocally met in either interview. A committee of board-certified child and adult psychiatrists and licensed psychologists who were blind to the subject’s ascertainment status, referral source, and all other data resolved diagnostic uncertainties. Diagnoses presented for review were considered positive only if a consensus was achieved that criteria were met to a degree that would be considered clinically meaningful. The principal investigators supervised the interviewers throughout the study.
The raters had undergraduate degrees in psychology and were trained to high levels of inter-rater reliability. Kappa coefficients of agreement were computed between the interviewers and board-certified psychiatrists who listened to the audiotape interviews. Based on 500 assessments from interviews of children and adults, the median kappa coefficient was 0.98. Kappa coefficients for individual diagnoses included: ADHD (0.88), conduct disorder (CD; 1.0), oppositional defiant disorder (ODD; 0.90), MD (1.0), mania (0.95), separation anxiety (1.0), agoraphobia (1.0), panic (0.95), obsessive compulsive disorder (OCD) (1.0), generalized anxiety disorder (GAD; 0.95), specific phobia (0.95), and social phobia (1.0). These measures indicated excellent reliability between ratings made by the non-clinician raters and experienced clinicians. In addition, to estimate the reliability of the diagnostic review process we computed kappa coefficients of agreement between clinician reviewers. For these clinical diagnoses, the median reliability between individual clinicians and the diagnoses assigned by the review committee was 0.87. Kappa coefficients for individual diagnoses included: ADHD (1.0), CD (1.0), ODD (0.90), MD (1.0), mania (0.78), separation anxiety (0.89), agoraphobia (0.80), panic (0.77), OCD (0.73), GAD (0.90), specific phobia (0.85), and social phobia (0.90).
At baseline, dimensional measures of psychopathology in children age 2 and older were collected by having mothers complete the Child Behavior Checklist in separate versions for preschoolers (2–3 years old; N = 88) and for school-age children (4–18 years old; N = 160) (Achenbach, 1991; Achenbach, 1992). For the purpose of this analysis, we combined data from the corresponding scales in the two versions of the Child Behavior Checklist.
A family was defined as intact if the parents were married and living together at the time of assessment. The family was considered not intact if the parents were divorced, separated, or never married. Socioeconomic status (SES) was assessed with the Hollingshead four-factor scale (Hollingshead, 1975).
Logistic regression was used to model psychiatric disorders at follow-up assessment as a function of baseline CBCL Internalizing scale T-score, baseline CBCL Externalizing scale T-score, and age at follow-up. Subjects who met full criteria for a disorder at baseline were not included in the analysis of that particular disorder. Excluding these subjects ensured a true longitudinal test of whether the CBCL scales could predict the onset of new disorders later in time. Multiple members of a single family (i.e., members of the same sibship) cannot be considered independent of one another because they share genetic, cultural, and social risk factors. Therefore, we used Huber-White robust estimates of variance so that p-values would be accurately estimated. For each psychiatric disorder found to have a significant relationship with the Internalizing or Externalizing scales, Receiver Operating Characteristic (ROC) analyses were performed and curves plotted. All tests were two-tailed with alpha set at 0.05.
Demographic features of the sample are presented in Table 1. Although the age of the sample at baseline ranged from 2 to 17 years of age, only 9% (23/248) of the subjects were older than 8 years. Likewise, the sample at follow-up ranged from 6 to 25 years of age but only 9% (23/248) were older than 14 years. There were no significant differences between the offspring who returned at follow-up (N = 248) and offspring lost to follow-up (N = 77) on baseline age, gender, ethnicity, family intactness, socioeconomic status, and lifetime GAF score (all p > 0.05).
Rates of psychiatric disorders at follow-up included 6% (13/230) agoraphobia, 6% panic disorder (14/243), 8% social phobia (20/237), 9% (21/242) major depression, 10% (21/207) specific phobia, 13% (30/228) generalized anxiety disorder, 13% (29/216) separation anxiety disorder, 17% (36/211) disruptive behavior disorders (ADHD, oppositional defiant disorder, or conduct disorder). These rates indicate the emergence of new disorders during the follow-up period among subjects who did not have the given disorder at baseline. Of the total 248 subjects, 149 (60%) had no new disorders at follow-up, 53 (21%) had one new disorder, and 46 (19%) had 2 or more new disorders at follow-up. Of the 36 subjects with new disruptive behavior disorders at follow-up, 8 had ADHD only, 16 had oppositional defiant disorder only, 8 had ADHD and oppositional defiant disorder, and 4 had conduct disorder (1 with comorbid ADHD, 1 with comorbid oppositional defiant disorder, and 2 with comorbid ADHD and oppositional defiant disorder).
Table 2 summarizes the utility of the baseline Internalizing and Externalizing CBCL scales to predict subsequent psychiatric disorders in subjects who did not have the outcome disorder at baseline. The Externalizing scale significantly predicted subsequent disruptive behavior disorders and major depressive disorder while the Internalizing scale significantly predicted subsequent agoraphobia, generalized anxiety disorder, separation anxiety disorder, and social phobia. Neither CBCL scale significantly predicted panic disorder or specific phobia.
ROC curves for disorders that were significantly predicted by the Internalizing scale are depicted in Figure 1. Areas under the ROC curves ranged from 0.63 for social phobia to 0.74 for separation anxiety disorder. The area under the ROC curve indicates, for example, that there is a 63% chance that the Internalizing T-score of a randomly selected offspring who later developed social phobia will be greater than the Internalizing T-score of a randomly selected offspring who did not develop social phobia. Internalizing T-scores that yielded sensitivities of approximately 25%, 50%, and 75% are included as a reference. The sensitivity is the percentage of subjects who develop the disorder who were correctly identified by an Internalizing T-score equal to or above the cutoff score. The corresponding specificity indicates the percentage of subjects who did not develop the disorder who were correctly identified by an Internalizing T-score below the cutoff score.
ROC curves for disorders that were significantly predicted by the Externalizing scale are depicted in Figure 2. Areas under the ROC curves were 0.77 for disruptive behavior disorders and 0.70 for major depressive disorder. Externalizing T-scores that yielded sensitivities of approximately 25%, 50%, and 75% are included as a reference.
Table 3 provides sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) results using cutoffs of 60 and 70 for Externalizing T-scores predicting disruptive behavior disorders and major depressive disorder and Internalizing T-scores predicting agoraphobia, generalized anxiety disorder, separation anxiety disorder, and social phobia. Sensitivity ranged from 25–34%, specificity from 88–96%, PPV from 12–59%, and NPV from 87–95% using a cutoff T-score of 60. Using a cutoff T-score of 70, sensitivity ranged from 8–20%, specificity from 95–100%, PPV from 15–100%, and NPV from 84–95%.
Because offspring were ascertained by different parental diagnoses, it is possible that our results may have varied by family ascertainment status. To test this hypothesis, we repeated our logistic regression models (Table 2) using parental major depression and parental panic disorder as predictors of the offspring’s psychiatric disorders. All previous statistically significant findings of Internalizing and Externalizing scales remained significant, and parental major depression and parental panic disorder were not statistically significant predictors of any psychiatric disorders independent of the effects of Internalizing and Externalizing T-scores and age.
In a longitudinal study of children at risk for major depression and panic disorder, logistic regression and ROC analyses indicated that the Internalizing broad-band scale selectively predicted subsequent agoraphobia, generalized anxiety disorder, separation anxiety disorder, and social phobia. In contrast, the Externalizing broad-band scale selectively predicted disruptive behavior disorders and major depression. These prospective findings suggest that the CBCL broad-band scales can be useful and cost-effective tools for identifying children at high risk for subsequent psychiatric disorders within a population of children already at high risk by virtue of parental psychopathology.
The prospective ability of the Internalizing broad-band scale of the CBCL to selectively predict a wide range of anxiety disorders in a sample of children of parents with panic disorder and depression is consistent with findings reported in cross-sectional studies that examined individual scales of the CBCL that are included in the Internalizing broad-band scale. For example, Aschenbrand et al. (Aschenbrand, Angelosante, & Kendall, 2005) recently reported that the Anxious/Depressed subscale was associated with severity of generalized anxiety disorder, while the Withdrawn subscale was associated with social phobia. We previously reported (Biederman, Monuteaux, Kendrick, Klein, & Faraone, 2005) a good correspondence between the Anxious/Depressed scale and multiple (≥2) anxiety disorders in children with ADHD.
The finding that the Externalizing scale predicts disruptive behavior disorders is not surprising considering that it is a composite of the Delinquent Behavior and Aggressive Behavior syndrome scales. Previous work documented excellent convergence between the Delinquent Behavior scale of the CBCL with the structured diagnostic interview diagnosis of conduct disorder (Biederman et al., 1993; Biederman, Monuteaux, Kendrick, Klein, & Faraone, 2005; Edelbrock & Costello, 1988; Kazdin & Heidish, 1984).
Less intuitive than the previously discussed findings are the results that the Externalizing scale, and not the Internalizing scale, predicted subsequent major depression. However, this is consistent with longitudinal findings from Fischer et al. (Fischer, Rolf, Hasazi, & Cummings, 1984) and Pihlakoski et al. (Pihlakoski et al., 2006) showing that preschool externalizing symptoms were predictive of later externalizing and internalizing symptoms. Keiley et al.’s (Keiley, Lofthouse, Bates, Dodge, & Pettit, 2003) analysis of childhood externalizing and internalizing behavior suggests that peer rejection may be the mediating factor in the development of depression from externalizing symptoms. That is, externalizing children are rejected by their peers and internal symptoms emerge as they struggle with the effects of their disruptive or aggressive behavior. The relationship between externalizing behavior and depression is also supported by previous findings showing a familial link between depression and disruptive behavior disorders (Biederman, Petty, Hirshfeld-Becker et al., 2006) and evidence that approximately 45% of the observed co-variation between depressive and antisocial symptoms could be explained by a common genetic liability (O’Connor, McGuire, Reiss, Hetherington, & Plomin, 1998).
Our analysis of various cutoff scores (Table 3) shows that the CBCL is much better at classifying children who will not develop psychopathology (i.e., it has high specificity) than it is at classifying those who do develop psychopathology (i.e., it has low sensitivity). For example, using a T-score cutoff of 60 on the Internalizing or Externalizing broad-band scales will correctly identify from one quarter to one third of subjects who later developed a DSM-IV disorder while correctly identifying nearly all (88% to 96%) of subjects who did not develop a disorder. A cutoff of 70 identified at best 20% and at worst only 8% of subjects who developed a disorder, results consistent with previous findings (Edelbrock & Costello, 1988; Shekim et al., 1986) that showed a cutoff of 70 to be too stringent. However, a child with a T-score of 70 or more may be at particularly high risk. For example, all four subjects with an Externalizing T-score of at least 70 went on to develop major depressive disorder.
Our findings should be viewed in the context of some methodological limitations. Because most subjects were Caucasian, our findings may not generalize to minorities. Although the subjects were not referred, their parents were, so the findings generalize to children with family risk for panic disorder or depression and may not generalize to community samples. The low rates of disorders in the control offspring made it infeasible to conduct analyses in this group only. In addition, because the subjects’ parents were referred for different diagnoses, findings may have been influenced or varied by parental diagnosis. However, when controlling for parental diagnosis, our findings remained the same. Although our statistical models adjusted for age, our subjects were still young and any future development of psychiatric disorders might reveal different results. The assessment of psychopathology in children younger than twelve years of age was based on interviews with the mothers, which may have led to underestimates of the true rates of psychopathology in offspring. Mothers completed the CBCL for their children so reports by teachers or fathers might yield different results. Because the CBCL cutoff scores identified at best one third of the subjects who would later develop a particular disorder, other risk factors should be considered when targeting children for intervention. On the other hand, the high specificity found with each cutoff score suggests that false positives would be limited.
Despite these limitations, our longitudinal findings support the use of broad-band CBCL scales to identify children at risk for subsequent diagnoses of depression, anxiety disorders, and disruptive behavior disorders. The broad-band scales predicted divergent patterns of risk in children of parents with panic disorder and depression similar to those identified using structured diagnostic interviews (Biederman, Petty, Hirshfeld-Becker et al., 2006).
This study is supported by NIMH grant MH-47077-05.
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.