|Home | About | Journals | Submit | Contact Us | Français|
We have previously published on a pattern of behavior called the Child Behavior Checklist Dysregulation Profile (DP)(Ayer et al., 2009). This profile consists of elevations on the attention problems, aggressive behavior, and anxious/depressed scales on the Child Behavior Checklist (CBCL) and has also been referred to in the literature as the CBCL Juvenile Bipolar Disorder profile or the CBCL Pediatric Bipolar Disorder profile. This profile has been described by many groups as a potential marker for severe childhood psychopathologies (Biederman et al., 1995; Faraone et al., 2005; Holtmann et al., 2007), or as a predictor of negative adult outcomes (Meyer et al., 2009). Our group and others have reported this profile to be genetically independent of Attention Deficit/Hyperactivity Disorder, Oppositional Defiant Disorder, and anxious-depression and to be stable across development and gender. It has separable behavioral and molecular genetics from Attention Deficit/Hyperactivity Disorder, and Oppositional Defiant Disorder (Hudziak et al., 2005; McGough et al., 2008) and children with the CBCL Juvenile Bipolar Disorder phenotype perform differently under tryptophan depletion conditions than children with Attention Deficit/Hyperactivity Disorder alone (Zepf et al., 2008). Despite its previous label, however, accumulating evidence suggests that while these children have a severe, debilitating pathology (Meyer et al., 2009), it is not synonymous with DSM defined pediatric bipolar disorder (Kahana et al., 2003; Youngstrom et al., 2005; Althoff et al., 2006; Volk and Todd, 2007). We have demonstrated that the profile measures a similar latent construct as another measure of severe impairment in self-regulation (Ayer et al., 2009). Because of this and our intention to not further muddle the study of severely impaired children by equating them only with juvenile bipolar disorder, our group has argued that the profile should be named the Dysregulation Profile (DP) to reflect its distinctiveness from Attention Deficit/Hyperactivity Disorder, Oppositional Defiant Disorder and bipolar disorder and its conceptualization as a broad disorder of self-regulation (Althoff et al., 2008; Ayer et al., 2009).
All findings to date on the Dysregulation Profile have used parental report to define the profile. Here, we sought to determine whether or not the profile would be empirically identified by other informants and whether different informants would identify the same individuals. Additionally, several groups have now demonstrated the critical finding that children with the Dysregulation Profile have an increase in parental reported suicidal behavior (Althoff et al., 2006; Volk and Todd, 2007), but an association with self-reported suicidal behavior has not been demonstrated. Because self-reported suicidal thoughts and behaviors do not often agree with parental reports (Foley et al., 2006) an examination of this important issue seemed crucial. Previous literature has shown that informant agreement is generally low for child and adolescent behavioral and emotional problems (Achenbach et al., 1987b; De Los Reyes and Kazdin, 2005). For example, Achenbach and colleagues (1987) identified mean correlation coefficients of 0.22 for subjects and other informants (e.g., parents, teachers) and 0.28 for different types of observer-informants (e.g., parents, teachers) in a meta-analysis. This disagreement has been found to be associated with several factors, including child characteristics such as gender (Duhig et al., 2000) and age (Achenbach et al., 1987a; Grills and Ollendick, 2003), and parent characteristics such as depression (Chilcoat and Breslau, 1997) and anxiety (Youngstrom et al., 1999).
Because different groups have used various T-score cutpoints to determine eligibility for the Dysregulation Profile, we opted to use Latent Class Analysis (LCA) rather than a cutpoint approach to determine the presence of the Dysregulation Profile. LCA is a statistical procedure that allows the investigator to test empirically for the existence of discrete groups who endorse similar patterns of symptoms (Hudziak et al., 1998). In a general population Dutch twin sample, we have conducted LCA on maternal reports of the attention problems, aggressive behavior, and anxious/depressed subscales of the CBCL (Althoff et al., 2006). A class that approximated the Dysregulation Profile was the least common - about 4–5% of the boys and girls by maternal report. To the best of our knowledge, determination of a Dysregulation Profile using self or teacher report has not been performed previously.
Thus, the three-fold purpose of this study was to determine: (1) the presence of the Dysregulation Profile by self and teacher report, (2) the agreement across those informants, and (3) the association between the Dysregulation Profile and self-reported suicidality. We predicted that the class structure for parent reports on the CBCL would be similar to what was observed previously. We further predicted that a Dysregulation Profile would emerge from the LCA regardless of informant. Because parent-child agreement is often better than parent-teacher agreement, (Achenbach et al., 1987a) we expected better cross-informant agreement between parent-child Dysregulation Profile comparisons than between parent-teacher or youth-teacher comparisons. Finally, we expected that children who were placed into a Dysregulation Profile class would have a higher prevalence of suicidal thoughts and behavior than children placed into other classes.
Participants were American nonhandicapped, clinically-referred and nonhandicapped, nonclinically referred children and adolescents from a national sample first collected in 1999 (Achenbach and Rescorla, 2001). This sample has been described in detail elsewhere (Achenbach et al., 2002). Briefly, data were obtained from home interview surveys with the parents of participants chosen to be representative of the contiguous 48 states. These surveys included the CBCL for parent or guardian report, Youth Self Report (YSR), Teacher Report Form (TRF) and other questions regarding demographics and the participant's mental health and special education history. The sample consisted of 2031 children of which 276 had been referred for mental health services in the preceding 12 months. 1073 (53%) participants were male. All children ranged in age from 6–18. The mean age of boys was 11.94 (SD = 3.56) and the mean age of girls was 12.03 (SD = 3.50). YSR data were available only on subjects older than 11. For the purposes of identifying the Dysregulation Profile latent class, data were analyzed with all participants. Full cross-informant data from CBCL, YSR, and TRF was available on 687 participants (n=359, 52% male) out of the 1321 participants aged 11–18. See Figure 1 for more detail on the sample. On the basis of the Hollingshead scale (Hollingshead, 1975), participants with data from all three informants had higher socioeconomic status (SES) than participants who were missing one or more informant (mean standardized SES 53.70 for participants with one or more missing informant versus 57.16 for participants with complete data, F = 10.12, p<0.002). All participants provided informed consent.
Problem behavior was measured with the CBCL, YSR and TRF (Achenbach and Rescorla, 2001). These questionnaires consist of 112 (YSR) or 113 (TRF, CBCL) items developed to measure problem behavior in children 6 to 18 years old. Parents, youth, and teachers were asked to rate the behavior of the child over the preceding 6 months on a 3-point scale. Items on the attention problems, anxious/depressed, and aggressive behavior scales differ among the CBCL, TRF, and YSR; therefore. Therefore, we limited the analysis to the 38 items which the three surveys had in common which eliminated three items from the TRF anxious/depressed scale, seventeen items from the TRF attention problems scale, and one item from the YSR aggressive behavior scale, two items from the CBCL aggressive behavior scale, and four items from the TRF aggressive behavior scale. The reason for removing these non-overlapping items was to allow for the latent classes to be compared using Euclidean distance (see below). Items are scored by the informant on a scale from 0–2. For LCA, which assumes dichotomous variables, we collapsed scores of 1 (“somewhat true”) and 2 (“often true”) as positive responses and 0 (“not true”) as a negative response as per previous recommendations (Dumenci et al., 2004). This truncation is standard for placing CBCL items into LCA (Hudziak et al., 1996; Hudziak et al., 1999; Wadsworth et al., 2001; Althoff et al., 2006).
LCA is a form of categorical data analysis which attempts to identify a small number of mutually exclusive classes to account for the observed symptom (or item) endorsement profiles. LCA presupposes the existence of discrete latent categories or classes. This feature distinguishes it from factor analysis (which assumes continuous latent variables are present). If, however, the underlying latent variable in an LCA is dimensional rather than categorical, the analysis will often reveal differences in severity rather than discretely different patterns of responding. The measures that emerge from LCA are (1) probabilities of class membership assignment for each individual and (2) symptom endorsement probabilities for each class. Each class is assumed to be functionally independent from the others within a particular solution (Goodman, 1974).
Latent class models were fitted by means of an Expectation Maximization (EM) algorithm (Dempster et al., 1977; Heinen, 1996) using the program Latent Gold (Vermunt and Magidson, 2000). Models estimating 1-class through 10-class solutions were compared. LCA was performed separately on CBCL, YSR, and TRF data using all of the completed reports available for the given informant. It is important to note that Latent Gold will interpolate missing data and will attempt to classify participants with the data available. If there are no data for a participant for an informant, they will be placed into the class with the lowest overall item endorsement probability, which is usually a class with no or few symptoms in general population samples. Thus, using all data as was done here is a conservative approach as participants with missing data may have symptoms, but will be placed into a “no or few symptoms” class.
To calculate the best fitting model, we compared the change in the Bayesian Information Criterion (BIC) when moving from one class to the next. The BIC is a goodness-of-fit index that considers the rule of parsimony. Other log-likelihood-based estimates are biased in an analytic environment with large samples and sparse data matrices, although other indices such as the Consistent Akaike Information Criterion (CAIC) were also examined. Once the best fitting model was found with sex and age included as covariates, these covariates were then dropped and the BIC was examined again.
Once the most parsimonious class solution was determined, we created common classes by equating classes with the lowest Euclidean distance between item endorsement probabilities (Rasmussen et al., 2004). The Dysregulation Profile latent class was determined to be the class with elevations on items from the attention problems, anxious/depressed, and aggressive behavior scales. We computed chi-square statistics and Cramer's V to measure the significance and strength of the nominal-to-nominal associations across informants and Cohen's Kappa was measured as an index of agreement. These statistics were performed using SPSS 15.0 (SPSS Inc., 2006)
Finally, we computed the average scores and proportion of either the youth or the parents endorsing YSR or CBCL items #18 (“deliberately harms self or attempts suicide”) and #91 (“talks about killing self”) as an indication of suicidal thought or behavior present in each of the classes. Four additional subjects were removed at this stage because of lack of information on one of the 2 suicidal items by either informant. Average scores on the suicidal items alone and in combination were compared between classes using the General Linear Modeling subroutine of SPSS.
The best fitting models were a 7-class solution for CBCL, a 6 class solution for TRF, and a 5 class solution for YSR data. As the number of classes increased, improvements were made in the goodness of fit demonstrated by a decrease in the BIC. Additional classes failed to result in better-fitting solutions. Dropping the age covariate did not change model fit as indexed by the BIC. Dropping the sex covariate did and so it was retained. Model fitting results are available on request.
The Dysregulation Profile latent class was identified by all informants. Approximately 10% of boys and 6% of girls were placed into this class by parent report. For self-report, the prevalence of the JBD class was 8% of boys and 9% of girls and for teacher report the prevalence was 8% of boys and 5% of girls. The Dysregulation Profile latent class along with the overall proportions placed into the class for all informants is shown in Figure 2. There was also a class similar to the Dysregulation Profile latent class that was identified by parent and youth report. This class had elevations on all three scales, but without the more “violent” aggressive items on aggressive behavior (items including being cruel or bullying, threatening, destroying property, fighting and attacking), and was much more common in females in the covariate analysis.
The nominal associations across informants were significant across all comparisons via chi square with the exception of TRF compared to YSR for girls. Chi square and Cramer's V statistics across the three informants are provided in Table 1. Kappa is also provided in Table 1 for the Dysregulation Profile latent class. Kappa for the Dysregulation Profile latent class ranged from a low of 0.139 for TRF compared to CBCL in boys to 0.279 for TRF compared to YSR in boys. Agreement across informants, therefore, is significant and in the slight to mild range (Landis and Koch, 1977). While all three types of informants empirically identified a small group of children and adolescents with the Dysregulation Profile, agreement was much more modest when trying to identify the particular individuals in this class. With the exception of girls assessed with the YSR and TRF, agreements were statistically significant. The chance-corrected kappa values, however, were found to range between 0.14 and 0.28 and suggest many disagreements across informants. While generally low, these kappa values are similar to those found across informants for the component subscales where Pearson correlations ranged from 0.16 to 0.52 (Achenbach and Rescorla, 2001) and those found when computing agreement between diagnoses generated from clinical evaluations to those generated from structured or semi-structured interview (Rettew et al., 2009).
Table 2 gives the number of youth in Dysregulation Profile latent class who self-reported suicidal ideation or behavior (SI) by informant. The total number of individuals with parental or self-reported SI was about 3.8% of children by parental report, 7.6% by self-report, and only 1.5% by teacher report. As has been previously demonstrated (Joffe et al., 1988; Velez and Cohen, 1988; Kashani et al., 1989; Breton et al., 2002) the correspondence between parental report of their child's suicidal thoughts or behavior and self report was a statistically significant. Kappa statistic was significant, but in the low range for boys (Kappa = 0.159, t=3.083, p<0.01) and was higher for girls (Kappa =0.344, t=6.881, p<0.001), indicating that there was significant, albeit low agreement across parental and self-reported suicidal ideation or behavior. These values are slightly higher, but consistent with previous literature demonstrating that youth and parent agreement about suicidal behavior is typically poor (Velez and Cohen, 1988; Breton et al., 2002). The Kappa for teacher was low when compared to youth reports (Kappa=0.075, t=2.70, p<0.01) and to parental report (Kappa=0.037, t=1.067, p=0.286). Examining SI across informants and within classes demonstrated that there was significantly higher reporting of suicidal behavior in the Dysregulation Profile latent class with chi-square tests demonstrating significance for self-reported suicidality for parental and self-reports in girls and for self-report in boys (Table 2). In the 34 children where parental reports and self-reports both classified the children as Dysregulation Profile, 42% of boys and 67% of girls reported suicidal thoughts or behavior. This is in contrast to children placed into the No or Few Symptoms class across parent and self reports where very few reported suicidal thoughts. In fact, of the 136 children where parents and children agreed on the No or Few Symptoms class, only 1 boy and no girls reported suicidal thoughts or behavior. For teachers, however, 5% of the boys and 10% of the girls placed into the No or Few Symptoms reported SI. Looked at another way, 37% of adolescent boys and 58% of adolescent girls who did self-report suicidal thoughts or behavior were placed into the teacher's No or Few Symptoms class.
The data presented here support previous findings of a consistent, replicable Dysregulation Profile identified using a questionnaire approach. The current work extends these previous findings on parental reporting by demonstrating the presence of a similar profile in self and teacher reports. Similarly, previous findings of high associations between parent Dysregulation Profile and suicidality is replicated here and extended to demonstrate that the Dysregulation Profile is associated with increased risk of self-reported suicidal thoughts and behaviors. When parent and self reports both place children into a Dysregulation Profile latent class, the association with self-reported suicidal thoughts and behavior is astonishingly high. Thus, without consideration of whether these children with the so-called “CBCL-Juvenile Bipolar Disorder” phenotype meet DSM-IV criteria for bipolar disorder, the data presented here and elsewhere suggest that they are at significant risk for expressing suicidal thoughts and behaviors. Further, they are at increased risk to develop substance abuse, mood disorders, and personality disorders in adulthood (Meyer et al., 2009). Indeed, this lack of diagnostic specificity coupled with such significant impairment and morbidity underscore the need for further study of this high risk group. These findings further underscore the need to consider this profile not as a proxy for bipolar disorder, but as a broad disorder of self-regulation.
It is possible that some will interpret the low to moderate informant agreement on the Dysregulation Profile latent class as evidence that this syndrome's validity and reliability are questionable. However, previous research on informant (dis)agreement indicates that the extent of informant agreement found here for the Dysregulation Profile latent class are comparable to those found for other child and adolescent psychiatric problems (for a review, see De Los Reyes & Kazdin, 2005). Further, informant agreement has been shown to vary widely between studies depending on several factors, such as sample, disorder, gender, age, and parental psychopathology (De Los Reyes and Kazdin, 2005). For example, (Karver, 2006) found kappas ranging from −0.01 to 0.81 when he examined parent-child agreement about 59 child behaviors in a sample of 675 parent-child dyads. De Los Reyes and Kazdin (2005) have suggested that such informant discrepancies result not only from parent and child characteristics (e.g., gender, age, psychopathology), but also from sociocognitive processes typically described in social and cognitive psychology literature (e.g., the actor-observer phenomenon (Jones and Nisbett, 1972). Therefore, the low to moderate informant agreement between teachers, parents and children when assessing the Dysregulation Profile should not be interpreted as indicating that the syndrome is unreliable or inconsequential. Instead, the emergence of a distinct latent class of dysregulation symptoms that is relatively stable across informants suggests that this profile at least warrants future research with other samples to better understand its replicability and validity
Additional information provided here again underscores the relatively poor agreement between SI across youth and teacher reports, despite the identification of a Dysregulation Profile latent class in teacher reports. Consistent with literature that underscores lower teacher to youth agreement on internalizing versus externalizing symptoms, these data demonstrate that teachers are often unaware of the severe impairment and thoughts of self-injury that some of their students experience. It should be mentioned that more of the TRF items were removed from the analysis because there were not comparable items on the YSR and CBCL reports. However, the majority of these were removed from the attention problems scale. It's hard to imagine a mechanism by which the addition of these additional items (which aren't present on the YSR or CBCL) would generate improved agreement between youth and teacher on suicidal items.
Data used in this examination were based on an epidemiological sample of referred and non-referred children. These children were not examined specifically for this study, and thus we cannot present data on the DSM diagnoses within these latent classes. At this time, it is difficult to imagine a way in which LCA could be used practically in a clinical setting, especially with multiple informants. However, the evidence that children, and parents agree on the phenotype is encouraging for further etiological studies taking a similar bottom-up approach to the assessment of children with profound dysregulatory problems. The clinical implications of these findings are that children who are dysregulated in multiple domains, especially when determined by both parental and youth report, are at increased risk for suicidal thoughts and behavior. Further, these data reify the common supposition inherent to child psychiatry that information from multiple informants is necessary for a complete picture of the child. For broad dysregulation, parents, teachers, and youth do not fully agree on the problem domains, suggesting that all three informants are providing some common but some unique information about the functioning of the youth. On the other hand, there are a substantial number of children examined here where informants have vastly different opinions. Especially in the area of suicidality, the finding that the three informants frequently disagree again highlights the need for multi-informant assessment and for assuming that all informants see a particular, but not necessarily complete, picture of the child. Moreover, there is the possibility of identifying at risk teens on the basis of parental responding. For example, teens whose parents' responses placed them into the dysregulation profile class, 9% of boys and 35% of girls self-reported suicidality. For research purposes, because there may be different characteristics of the three component phenotypes of the Dysregulation Profile (attention problems, aggressive behavior, and anxious-depression), it might be useful to apply factor mixture models that could allow for both latent classes and factors. These types of models are currently being examined. Finally, these data also suggest the presence of other potentially important classes of children that have yet to be studied as a group. In focusing on Dysregulation Profile, we have not presented these other classes in detail. While the Dysregulation Profile class may warrant particular attention given its association with suicidal behavior, further exploration into the other classes identified may similarly yield clearer results in future research by examining types of symptoms that tend to cluster naturally within the population.
In summary, these data support the presence of a replicable profile of attention problems, aggressive behavior, and anxious-depressed symptoms that cluster together using parent, teacher, or self-report. When it occurs in both child and parent report this profile indexes a particularly severe form of psychopathology. Causes of and treatments for this highly impairing profile should be the focus of further study.
This work was partially supported by a grant from the American Foundation for Suicide Prevention (R. Althoff, PI) and NIMH Grants K08MH082116 (R. Althoff, PI) and R01MH058799 (J. Hudziak, PI). Neither the NIMH nor the AFSP had any further role in study design; collection, analysis or interpretation of data; in writing of the report; or in the decision to submit the paper for publication. This work was presented in part as a poster at the 2008 Annual Meeting of the American Psychopathological Association. The authors would like to thank Dr. Thomas Achenbach for his generosity in allowing the use of these data.
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.