Search tips
Search criteria 


Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
J Am Acad Child Adolesc Psychiatry. Author manuscript; available in PMC 2014 February 1.
Published in final edited form as:
PMCID: PMC3558689

Separating the Domains of Oppositional Behavior: Comparing Latent Models of the Conners’ Oppositional Subscale



Although Oppositional Defiant Disorder (ODD) is usually considered the mildest of the disruptive behavior disorders, it is a key factor in predicting young adult anxiety and depression and is distinguishable from normal childhood behavior. In an effort to understand possible subsets of oppositional defiant behavior (ODB) which may differentially predict outcome, we used Latent Class Analysis (LCA) of mother’s report on the Conners’ Parent Rating Scales Revised Short Forms (CPRS-R:S).


Data were obtained from mother’s report for Dutch twins (7 year-old [n = 7,597], 10 year-old [n = 6,548], and 12 year-old [n = 5,717]) from the Netherlands Twin Registry. Samples partially overlapped at ages 7 and 10 (19% overlapping) and at ages 10 and 12 (30% overlapping), but not at ages 7 and 12. Oppositional defiant behavior was measured using the 6-item Oppositional subscale of the CPRS-R:S. Multilevel LCA with robust standard error estimates was performed using Latent Gold to control for twin-twin dependence in the data. Class assignment across ages was determined and an estimate of heritability for each class was calculated. Comparisons to maternal report Child Behavior Checklist (CBCL) scores were examined using linear mixed models at each age, corrected for multiple comparisons.


The LCA identified an optimal solution of 4-classes across age groups: Class 1 was associated with no or low symptom endorsement (69–75% of the children), class 2 was characterized by defiance (11–12%), class 3 was characterized by irritability (9–11%), and class 4 was associated with elevated scores on all symptoms (5–8%). Odds ratios for twins being in the same class at each successive age point were higher within classes across ages than between classes. Heritability within the two “intermediate” classes was nearly as high as for the class with all symptoms, except for boys at age 12. Children in the Irritable Class were more likely to have mood symptoms on the CBCL scales than children in the Defiant Class while demonstrating similar scores on aggression and externalizing scales. Children in the All Symptoms Class were higher in both internalizing and externalizing scales and subscales.


The LCA indicates 4 distinct latent classes of oppositional defiant behavior, where the distinguishing feature between the two intermediate classes (classes 2 and 3) is level of irritability and defiance. Implications for the longitudinal course of these symptoms, association with other disorders, and genetics are discussed.

Keywords: oppositional defiant disorder, twin, latent class analysis

Along with Conduct Disorder (CD) and attention-deficit/hyperactivity disorder (ADHD), Oppositional Defiant Disorder (ODD) is one of the leading reasons for referral to youth mental health services. 1 In contrast to CD, which is seen as a severe and inflexible condition,2 ODD has often been considered a fairly mild condition3 possibly because some of the behaviors associated with it approximate normative child development (e.g., losing one’s temper, arguing with adults). This thought has persisted, despite evidence that ODD is in fact distinguishable from normal childhood behavior 2,4 and is present in up to 2% of girls and nearly 5% of boys 5. Despite differences between CD and ODD, research on these disorders has typically combined the two, collapsing them into a single construct.6, 7, 8 In doing so, many studies involving ODD and CD fail to consider the two disorders distinctly and often ODD is excluded altogether. Because ODD and CD are often studied in concert, the specific environmental and genetic contributants to ODD remain elusive. It is often assumed that ODD is due to poor parenting or environmental causes, yet research demonstrates that, separate from CD, there is support for specific genetic factors associated with ODD.9 While studies have found that ODD and CD are correlated, the symptoms appear to represent distinct processes.10 As researchers have begun to separate the disruptive behavior disorders and to examine ODD individually, it has become clear that ODD may not be as benign as previously thought. Instead of serving as prodrome for CD, the ODD diagnosis may in fact play a significant role on its own in the development of a wide range of child psychopathology, including, depression, anxiety, CD, and later the development of antisocial personality disorder. 11

Subtyping of the ODD diagnosis may be especially important if we hope to understand its association with later development of psychopathology (e.g., more defiant behavior predicting something distinct from ODD with more irritable or reactive features), as well as its association with service use and prescribed treatment adherence. Copeland et al.12 found that ODD emerges as a key disorder in predicting young adult anxiety and depression. Earlier age at onset of ODD symptoms generally results in a poorer prognosis in terms of progression to CD and ultimately antisocial personality disorder. In fact, it has been estimated that approximately 30% of children who have an early onset of ODD later progress to develop CD.13, 1 However, it may be important to differentiate between boys and girls as findings have been mixed. In one study, ODD in girls was found to be associated with increased risk of depression, anxiety, and later ODD but not associated with increased risk for later development of CD.14 In examining the course of the disorder, preschool children with ODD are likely to exhibit additional disorders several years later, and with increasing age, comorbidity with ADHD, anxiety, or mood disorders begins to appear.15 In fact, ODD as a long-term predictor of many other disorders holds in childhood and adolescence even when controlling for other disorders. 12 Furthermore, the distinction among ADHD, ODD, and CD seems to be supported by research, but findings have again been mixed.16, 13, 17, 4 Similar to CD, the association of ODD and ADHD appears to indicate more severe psychopathology. Compared to children with ADHD only, children with ODD and ADHD tend to be more aggressive, show a greater range and persistence of problem behaviors, are rejected at higher rates by peers, and underachieve more severely in the academic domain. Children and adolescents with ODD not only appear to have significantly higher rates of comorbid psychiatric disorders, but they also seem to have significantly greater family and social dysfunction relative to other youths with psychopathology.13,18 Understanding the subtypes of ODD that might predict differential outcomes seems prudent.

A study by Stringaris and Goodman19 attempted to subtype ODD using three distinct a-priori derived dimensions of oppositionality: (1) irritable, (2) hurtful, and (3) headstrong. This study found that all three dimensions were associated with differing manifestations of CD; therefore the authors concluded that distinct subtypes of oppositionality likely do not exist. Furthermore, they concluded that the three dimensions may suggest differing origins and trajectories to oppositionality, based on the cross-sectional and longitudinal associations that they had seen. This has been followed by studies from Aebi et al.20 who demonstrated similar dimensions, new work demonstrating similar dimensions in preschoolers, 21,22 and Rowe et al.23 who demonstrated that there were few cases of “pure” headstrong. They examined differential prediction of the dimensions and showed that the headstrong dimension was associated with substance disorder and irritability was associated with later anxiety disorder. Similarly, Kolko and Pardini 24 studied dimensions of treatment-resistance and showed that irritability was associated with treatment-resistant ODD while hurtfulness was associated with later treatment-resistant CD. We questioned whether defining subtypes using a bottom-up approach, rather than using a-priori dimensions might produce a slightly different result. Specifically, we questioned whether latent class analysis (LCA) could be used to refine the ODD phenotype. LCA is a form of person-centered categorical data analysis that allows one to identify latent classes that account for the distribution of cases that have similar categorical response variables 25. By the nature of the analysis, these classes are mutually exclusive with each having its own particular pattern of item endorsement. LCA presupposes the existence of discrete latent categories of responding and groups individuals, distinguishing it from factor analysis, which assumes continuous latent variables that group symptoms. LCA results in two metrics: (1) the probability of class membership for each individual and (2) symptom endorsement probabilities for each class. The class that is most probable for a particular individual or the posterior probability of class assignment can then be used in subsequent analyses. The advantage to this approach is that it is free of preconceived notions about which items should go together and thus allows for a manner of classifying individuals empirically using a bottom-up approach. This approach has been used to study classes of ADHD,26, 27, 28 OCD,29 juvenile bipolar disorder,30 tic disorders,31 and alcohol use disorders,32 among others.

The objective of the current analysis was to determine if specific ODB subclasses could be identified using a LCA of mother’s report on the Conners’ Parent Rating Scales Revised Short Forms (CPRS-R:S). Given that the CPRS-R:S does not have hurtful items, we hypothesized that subjects would differ on their levels of headstrong (or defiant) and irritable symptoms, based on the previous literature. With this in mind, we hypothesized that a person-centered, latent class analysis would reveal 4 latent classes of individual responding: a class with no or few symptoms, a class with mainly irritable symptoms, and class with mainly defiant symptoms, and a class with high levels of all symptoms. Given findings of the stability of the heritability of ODB over time, we expected that the same latent structure would hold at ages 7, 10, and 12.


This study proceeded in three steps. First, latent class analysis was performed within each age group and heritability estimated. Next, the across-age stability of these classes was tested by comparing across ages 7 to 10 and 10 to 12. Third, a comparison of concurrent validity was performed within each age group. This can be seen graphically in Figure 1.

Figure 1
Analysis work flow demonstrating the cross-sectional nature of the latent class analysis (LCA) and mixed models with partial longitudinal analysis between ages 7 and 10 for class membership. Note: CBCL = Child Behavior Checklist; LMM = linear mixed models; ...

Subjects and Procedure

Data was obtained using mother’s report for Dutch twin pairs from the Netherlands Twin Registry, kept by the Department of Biological Psychology at the Free University in Amsterdam. 3335 Starting in 1987, families with twins were recruited a few months after birth. Currently, 40%–50% of all multiple births are registered by the Netherlands Twin Registry. The data of the present study are derived from a large ongoing longitudinal study that examines the genetic and environmental influences on the development of problem behavior in families with twin’s ages 3 to 12-years-old.33,34 Information from the Conners’ forms used here were introduced later in the data collection using a cohort-based data collection (see 33 for full details). The data from the original 7-year-old cohort are only now turning 12; therefore, there are no individuals with full longitudinal data from ages 7 – 12. The final sample for LCA consisted of Conners’ forms for 14,844 children. 5,018 children were sampled more than once (2,214 sampled at both age 7 and 10; 2,804 sampled at both age 10 and 12; there were no children sampled at both age 7 and age 12). Thus, 19,862 total observations were entered into the latent class analysis. 7,597 children had data at age 7 (38.2% of total observations), 6,548 children had data at age 10 (33.0% of total observations), and 5,717 children had data at age 12 (28.8% of total observations). For examining the concurrent validity, Child Behavior Checklist (CBCL) data were included using maternal report.

For the present study, data of mother report for 7, 10, and 12-year-old twin pairs was examined separately for each age group. Mothers of twins were asked to fill out questionnaires about problem behavior separately. After 2 months, a reminder was sent to the non-responders, and, when finances permitted, families who had not responded after 4 months were telephoned. Families who did not participate at a certain age were subsequently contacted and allowed to participate in the next scheduled study contact. The overall participation rate for the age groups used in the present study is 66% at age 7, 64% at age 10, and 64% at age 12 (this includes all registered families with a twin pair at a particular age). Previous work on this sample has demonstrated that attrition was random with respect to childhood psychopathology. 33 This study was approved by the institutional review boards of the Free University, Amsterdam, and the University of Vermont.


Mothers of participants completed the Conners’ Parent Rating Scales Revised Short Form (CPRS-R:S). The questionnaire consists of 27 items rated on a four-point Likert scale for symptom severity (i.e., 0 = not true at all, 1 = just a little true, 2 = pretty much true, 3 = very much true). The items are summarized on four scales: Oppositional, Cognitive Problems/Inattention, Hyperactivity, and the ADHD Index. Three of these scales, Oppositional, Cognitive Problems/Inattention, and Hyperactivity, were originally derived from the Conners’ Rating Form: Long Form. To provide brief versions of these scales, only items loading the highest (loadings 0.40) from an exploratory factor analysis of the factor scale items on the long form were used.36 This study specifically used the Oppositional subscale, which consists of six items (Table 1). The internal consistency coefficient for both scales was greater than 0.80 for males and females and the test-retest reliability coefficients for scales were between 0.63 and 0.85 during a period of 6 to 8 weeks.36

Table 1
Comparison of Connors’ Subscale Questions and DSM-IV Oppositional Defiant Disorder (ODD) Items

For the purpose of the LCA, items on the oppositional subscale were recoded such that 0 and 1 were recoded to be 0. Items scored 2 and 3 were recoded to be 1. This approach has been used in the analysis of the ADHD Index on the same scale and the use of truncation strategies did not change the overall pattern, only the number of children placed into each class.27 Prior to using this truncation strategy on these data, we compared and contrasted three possible truncation strategies. Dichotomizing data with 0 and 1 responses grouped together and 2 and 3 responses grouped together resulted in lower residuals and higher explained variance and with model fits that were, quite similar. With all truncation strategies, if the best fitting model was actually a 3- or 5-class model, these models were essentially equivocal with the 4-class model. This information, along with detailed information about the model fitting, is available in the Supplement 1, available online.

For examining the concurrent validity, information from the scales of the Child Behavior Checklist/4–18 37 were used. The CBCL is a questionnaire of 118 items developed to measure problem behavior in 4 to 18 years old children. Mothers were asked to rate the behavior of the child of the preceding 6 months on a 3-point scale. Eight syndrome scales plus two broadband scales (internalizing and externalizing) were composed according to the Dutch scales for the 1991 version, which are the same as the American scales 38.

Data Analysis

The data analytic workflow is shown graphically in Figure 1. Latent class models were fitted by means of an Expectation Maximization (EM) algorithm39 with the program Latent Gold40 to control for twin-dependence, a multilevel model was used with family number as a grouping variable and standard errors adjusted using the robust (Sandwich) standard error estimator. Models estimating 1-class through 5-class solutions were compared. Changes in the Bayesian Information Criterion (BIC; a goodness-of-fit index that considers the rule of parsimony) were primarily used, although other metrics were considered as was a factor-mixture model of the data which yielded results consistent with those reported from the LCA (see Supplement 1, available online). LCA proceeded in 5 steps for each age group. First, models were fit without any restrictions, then bivariate residuals were reduced by allowing for direct effects, the role of the sex covariate was considered, then significance of the model was examined using nonparametric bootstrapping, and finally the fits with models with one additional or one fewer class were examined (see Supplement 1 and Tables S1, S2, and S3, available online).

To examine heritability of the latent classes, the posterior probability of class membership for each latent class for each twin was compared to the posterior probability of class membership for that same latent class in their co-twin. This was performed using intraclass correlations in SPSS. To calculate a rough estimate of the heritability, Falconers formula 41 was used by calculating 2 times the difference of MZ intraclass correlation and DZ class correlation [2*(ICCMZ−ICCDZ)]. In situations where genetic dominance might be evident (i.e. the MZ correlation was more than twice the DZ correlation), the MZ correlation itself was taken as the estimate of heritability.

Logistic regression was used to predict stability of class membership by examining the likelihood that being in a particular class at one age predicted the categorical outcome of being in all other classes at the next age.

Finally, a set of linear mixed models were performed to examine the relations between the classes and CBCL scales. We controlled for family-clustering by choosing 1 random MZ twin from any MZ twin pair and including a family clustering variable for the DZ pairs. These models used CBCL scale as the dependent variable, family as a random factor, and latent class, sex, and sex × latent class interaction as categorical fixed effects. Each latent class was compared to the all symptoms class in the model and a p-value criterion of p<.005 was set for the significance value for each test to control for multiple comparisons. For comparisons within a fixed effect (e.g. for comparing between latent classes), we examined the confidence interval around the estimate, using the 99.5% confidence interval, again, to be conservative with multiple comparisons.


The LCA identified an optimal solution of 4-classes across age groups co-varying for sex (Figures 24). The best model had control for twin-dependence, did not include sex as a covariate, and included direct effects (except age 10) to account for significant bivariate residuals. Distributions of the groups are shown in Table 2. The across-twin intraclass correlations and estimated heritabilities are provided in Table 3. On the whole, twin correlations within a particular class were higher for MZ twins than for DZ twins, indicating the role of genetics. Estimated heritabilities of the latent classes ranged from 0.13 (12-year-old males in the No Symptoms class) to 0.59 (7 year-old males in the Defiant Class). Heritability estimates for males generally decreased in each class from 7 through 12, while estimates were equivalent or increased for females from 7 through 12. At each age, the estimates for the Defiant and Irritable classes were in the same general range as the estimates for the All Symptoms class, with the exception of age 10 where correlations were generally lower for the Irritable class. A one-way analysis of variance (ANOVA) did not demonstrate Bonferroni-corrected differences among the classes in terms of heritability.

Figure 2
4-Class solution for 7 year old Conners’ Parent Rating Scale: Revised-Short Latent Class Analysis.
Figure 4
4-Class solution for 12 year old Conners’ Parent Rating Scale: Revised-Short Latent Class Analysis.
Table 2
Item Endorsement Probabilities and Class Membership for 4-Class Latent Class Solution
Table 3
Intraclass Correlations and Estimated Heritabilities Between Twins Within Latent Classes at Ages 7, 10, and 12

The results of the logistic regression are shown in Table 4, which presents the ratio of the odds of being in a particular class versus the odds of being in any other class. On the whole, odds ratios were significantly higher between age groups (on the basis of non-overlapping confidence intervals) for comparisons within a particular latent than across latent classes. Additionally, being in Class 2 at age 7 did predict being in either Class 2 or Class 4 at age 10 and being in Class 3 at age 10 did predict being in either Class 3 or Class 4 at age 12. However, there was no significant crossover in switching between Class 2 and 3.

Table 4
Across Age Comparison: Odds Ratios and 95% Confidence Intervals

At all ages, linear mixed models demonstrated a significant effect of latent class on all CBCL scales. Controlling for multiple comparisons, children in the Irritable class had significantly higher mean scores on the anxious-depressed subscale than children in the Defiant class at all ages, and higher mean scores for both withdrawn behavior and the internalizing problems at age 7. While children in the Defiant class had higher mean scores on aggressive behavior and externalizing problems than the children in the Low or No Symptoms class, they had equivalent scores on these scales to the children in the Irritable class and lower scores, at all ages, on aggressive behavior and externalizing than children in the High symptoms class. It was only at age 12 that children in the Defiant class began to separate statistically from the No symptoms class in terms of rule-breaking behavior. Full model results are provided in Table S4, available online.


The current findings indicate 4 distinct latent classes of ODB. As expected, the majority of children had low or no symptom endorsement. This should be expected in a general population sample of children. Furthermore, consistent with the literature, which suggest a decrease in ODD diagnoses at 3-year follow-up,13,1 approximately 75% of children were in the low symptom class by age 12 (compared to 69% at both age 7 and 10). The level of either irritability or defiance was the distinguishing difference between class 2 and class 3 in the LCA (Table 2). Specifically, these findings may indicate some ability to separate children who present with oppositional behavior into different patterns of behavior. Children classified into class 2 by mother’s report were more likely to argue with adults and to be actively defiant, however, this same class of children was not likely to be rated as irritable or hot-tempered. Moreover, this class was also unlikely to demonstrate more internalizing symptoms than the low symptoms class. This finding may suggest that these children’s low level of irritability and higher rates of defiance are indicative of children with lower levels of prosocial behavior and more anti-social-like behaviors. This is in contrast to class 3 in the LCA, which includes children whose mother’s endorsed items related to very high levels of irritability (e.g., anger, resentment, and hot-temperedness) accompanied by low levels of defiance. In fact, unlike children in class 4 (the high symptoms class), children in class 3 were not any more defiant than children with low or no symptom endorsement. This finding may indicate a pattern of behavior more associated with the later development of mood disorders, consistent with higher levels of internalizing symptoms on the CBCL in the this class compared to classes 1 or 2. This distinction between “irritable non-defiance” and “defiant non-irritability” is consistent with findings in the literature of the distinction between Reactive-affective-defensive-impulsive (RADI) vs. Proactive-instrumental-planned-predatory (PIPP). RADI refers to aggression that is unplanned and accompanied by negative emotions such as anger, irritability, or fear. PIPP aggression, on the other hand, is associated with positive emotions and is willfully planned and executed.42 This is the first example, however, that we are aware of where these distinctions have been reified within an oppositionality scale using a person-centered approach.

The results of the logistic regression (Table 3) done on the LCA classes suggest that class membership is relatively stable. In fact, at all ages there was a significant likelihood of homotypic continuity. The only class with significant drift regarding class membership was the 10-year-old group in class 4 (high symptom class); although these children were likely to maintain membership in the high symptom class, they were also likely to shift to class 2 or class 3. This finding is in line with previous studies which suggest that a significant portion of kids with ODD, exit the diagnosis by the age of 12.

There are some limitations that need to be acknowledged. First, despite the fact that the ODD checklist began with all eight items, and factor analyses done by Conners36 yielded evidence that the six items retained were the most highly loaded, the Conners’ Oppositional scale does not include all eight of the DSM-IV ODD criteria. Thus the scale used for this analysis assesses oppositional defiant behavior rather than the ODD diagnosis specifically, and the items relating to the hurtful dimension was not included in these analyses. Second, having only one informant means that we cannot be sure whether the results would be different if teachers, other caregivers, or the children themselves provided information. This is work that we are continuing to explore. Moreover, using an all Dutch sample means that we cannot be sure whether these results are generalizable to other groups of children, although these children have been demonstrated to be similar to the Dutch general population 33 and the overall levels of psychopathology in children in The Netherlands has been demonstrably similar to U.S. populations 43 and, while attrition for general psychopathology was demonstrably missing at random, this might not necessarily hold for latent class assignment. Similarly, it is not completely clear that these model fits would generalize to another sample, although here, because ages 7 and 12 contain completely different children, with absolutely no overlap, the fact that these two models are so similar represents a large replication in a separate sample. Further, we have conducted a second study on an entirely different sample using a different instrument and demonstrated similar latent structure and have demonstrated that these classes have predictive validity (Althoff et al., manuscript in preparation). Additionally, the limits of odds ratios need to be acknowledged especially when numbers in classes get small. However, Pearson correlations were also performed for probability of class membership in each class and the results were essentially the same. Finally, these data were from a mixed cross-sectional/longitudinal sample yielding no individuals with full longitudinal data from ages 7–12, full longitudinal data will be available when all waves reach age 12 at which time the full longitudinal genetic model for these classes can also be fit.

An understanding of distinct differences between classes is necessary if clinicians and researchers wish to tease apart the specific contributions of environmental and genetic factors to ODD. The assumption that ODB in general and ODD in particular are entirely due to poor parenting or environmental causes, has not been supported by research. Future research must evaluate the complex etiology of ODD apart from CD, which may allow for a more accurate and complete picture of presenting oppositional defiant behaviors in both research and clinical settings. The current findings suggest that there are subsets of ODB in the population that may have differential presentation and course. These findings are consistent with recent proposed changes to the ODD diagnosis by the American Psychiatric Association (APA) DSM-5 committee. Specifically, a proposed change in the reorganization of ODD: “Recommendation 3. Organize symptoms in the criteria for ODD to distinguish emotional and behavioral symptoms.” In examining possible changes, the committee found that while behavioral and emotional symptoms both predicted disruptive behavior disorders, mood and anxiety disorders were predicted independently by emotional symptoms.44 This recommendation is supported by the results presented here that person-centered analyses can distinguish between children with “irritable non-defiance” and “defiant non-irritability. We would predict that children with irritable, non-defiance would be more at risk for later mood disorders versus children with defiant non-irritability who would be more at risk for conduct disorders. New work performed in our laboratory using a similar construct has suggested that this is the case (Kuny, unpublished doctoral thesis; Althoff et al., manuscript in preparation) with children in the defiant non-irritable group demonstrating higher levels of criminal behavior in adulthood, compared to children in the irritable but not defiant group showing a higher rate of mood disorders in adulthood.

Figure 3
4-Class solution for 10 year old Conners’ Parent Rating Scale: Revised-Short Latent Class Analysis.

Supplementary Material


Table S1. Steps in Analysis of Age 7 Latent Class Analysis.

Table S2. Steps in Analysis of Age 10 Latent Class Analysis.

Table S3. Steps in Analysis of Age 12 Latent Class Analysis.

Table S4. Mean Child Behavior Checklist Scale (CBCL) Scores by Latent Class for Each Age.


This work was supported by National Institute of Mental Health (NIMH) grants MH082116 and MH58799, and the Netherlands Organisation for Scientific Research grants 575-25-006, 575-25-012, and 904-57-94.

The authors would like to thank Dorret Boomsma of Vrije Universiteit Amsterdam, the Netherlands, for allowing the use of Netherlands Twin Registry data.


Supplemental material cited in this article is available online.

Disclosure: Dr. Althoff has received grant or research support from NIMH and the Klingenstein Third Generation Foundation. Dr. Copeland has received grant or research support from NIMH and the Brain and Behavior Research Foundation. Dr. Bartels has received funding from the National Institute of Diabetes and Digestive and Kidney Disease (NIDDK). Dr. Hudziak has received funding from NIMH and NIDDK. His primary appointment is with the University of Vermont. He has additional appointments with Erasmus University in Rotterdam, Vrije University in Amsterdam, Dartmouth Medical School in Hanover, New Hampshire, and the Avera Institute of Human Behavioral Genetics in Sioux Falls, South Dakota. Ms. Kuny, Dr. Van Beijesterveldt, and Ms. Baer report no biomedical financial interests or potential conflicts of interest.

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.


1. Loeber R, Burke JD, Lahey BB, Winters A, Zera M. Oppositional defiant and conduct disorder: a review of the past 10 years, part I. J Am Acad Child Adolesc Psychiatry. 2000 Dec;39(12):1468–1484. [PubMed]
2. Keenan K, Shaw DS. Starting at the beginning: Exploring the etiology of antisocial behavior in the first years of life. In: Lahey BB, Moffitt TE, Caspi A, editors. Causes of Conduct Disorder and Juvenile Delinquency. New York, NY, US: Guilford Press; 2003. pp. 153–181.
3. Rey JM, Bashir MR, Schwarz M, Richards IN, Plapp JM, Stewart GW. Oppositional disorder: fact or fiction? J Am Acad Child Adolesc Psychiatry. 1988 Mar;27(2):157–162. [PubMed]
4. Rutter M, Giller H, Hagell A. Antisocial behavior by young people. J Am Acad Child Adolesc Psychiatry. 1999;38:1320–1321.
5. Maughan B, Rowe R, Messer J, Goodman R, Meltzer H. Conduct disorder and oppositional defiant disorder in a national sample: developmental epidemiology. J Child Psychol Psychiatry. 2004 Mar;45(3):609–621. [PubMed]
6. Burke JD. Relationship between conduct disorder and oppositional defiant disorder and their continuity with antisocial behaviors: evidence from longitudinal clinical studies. In: Shaffer D, Leibenluft E, Rohde LA, editors. Externalizing Disorders of Childhood: Refining the Research Agenda for DSM-V. Arlington, VA: Am. Psychiatr. Assoc; 2009.
7. Greene RW, Biederman J, Zerwas S, Monuteaux MC, Goring JC, Faraone SV. Psychiatric comorbidity, family dysfunction, and social impairment in referred youth with oppositional defiant disorder. The American journal of psychiatry. 2002 Jul;159(7):1214–1224. [PubMed]
8. Kuhne M, Schachar R, Tannock R. Impact of comorbid oppositional or conduct problems on attention-deficit hyperactivity disorder. Journal of the American Academy of Child and Adolescent Psychiatry. 1997 Dec;36(12):1715–1725. [PubMed]
9. Hudziak JJ, Derks EM, Althoff RR, Copeland W, Boomsma DI. The genetic and environmental contributions to oppositional defiant behavior: a multi-informant twin study. J Am Acad Child Adolesc Psychiatry. 2005 Sep;44(9):907–914. [PubMed]
10. Frick PJ, Lahey BB, Loeber R, Tannenbaum L. Oppositional defiant disorder and conduct disorder: A meta-analytic review of factor analyses and cross-validation in a clinic sample. Clin Psychol Rev. 1993;13:319–340.
11. Loeber R, Burke JD, Pardini DA. Development and etiology of disruptive and delinquent behavior. Annu Rev Clin Psychol. 2009;5:291–310. [PubMed]
12. Copeland WE, Shanahan L, Costello EJ, Angold A. Childhood and adolescent psychiatric disorders as predictors of young adult disorders. Arch Gen Psychiatry. 2009 Jul;66(7):764–772. [PMC free article] [PubMed]
13. Connor DF. Aggression and Antisocial Behavior in Children and Adolescents: Research and Treatment. New York: Guilford; 2002.
14. Rowe R, Maughan B, Pickles A, Costello EJ, Angold A. The relationship between DSM-IV oppositional defiant disorder and conduct disorder: findings from the Great Smoky Mountains Study. J Child Psychol Psychiatry. 2002 Mar;43(3):365–373. [PubMed]
15. Lavigne JV, Cicchetti C, Gibbons RD, Binns HJ, Larsen L, DeVito C. Oppositional defiant disorder with onset in preschool years: longitudinal stability and pathways to other disorders. J Am Acad Child Adolesc Psychiatry. 2001 Dec;40(12):1393–1400. [PubMed]
16. Burke JD, Loeber R, Birmaher B. Oppositional defiant disorder and conduct disorder: a review of the past 10 years, part II. Journal of the American Academy of Child and Adolescent Psychiatry. 2002 Nov;41(11):1275–1293. [PubMed]
17. Hinshaw SP, Anderson CA. Conduct and oppositional defiant disorders. In: Mash EJ, Barkley RA, editors. Child Psychopathology. New York, NY, US: Guilford Press; 1996. pp. 113–149.
18. Burke JD, Hipwell AE, Loeber R. Dimensions of oppositional defiant disorder as predictors of depression and conduct disorder in preadolescent girls. J Am Acad Child Adolesc Psychiatry. 2010 May;49(5):484–492. [PMC free article] [PubMed]
19. Stringaris A, Goodman R. Three dimensions of oppositionality in youth. J Child Psychol Psychiatry. 2009 Mar;50(3):216–223. [PubMed]
20. Aebi M, Muller UC, Asherson P, et al. Predictability of oppositional defiant disorder and symptom dimensions in children and adolescents with ADHD combined type. Psychol Med. 2010 Dec;40(12):2089–2100. [PubMed]
21. Ezpeleta L, Granero R, de la Osa N, Penelo E, Domènech JM. Dimensions of oppositional defiant disorder in 3-year-old preschoolers [published online March 12, 2012] Journal of Child Psychology and Psychiatry. doi: 10.1111/j.1469-7610.2012.02545.x. [PubMed] [Cross Ref]
22. Wakschlag LS, Henry DB, Tolan PH, Carter AS, Burns JL, Briggs-Gowan MJ. Putting theory to the test: modeling a multidimensional, developmentally-based approach to preschool disruptive behavior. J Am Acad Child Adolesc Psychiatry. 2012 Jun;51(6):593–604. e594. [PMC free article] [PubMed]
23. Rowe R, Costello EJ, Angold A, Copeland WE, Maughan B. Developmental pathways in oppositional defiant disorder and conduct disorder. J Abnorm Psychol. 2010 Nov;119(4):726–738. [PMC free article] [PubMed]
24. Kolko DJ, Pardini DA. ODD dimensions, ADHD, and callous-unemotional traits as predictors of treatment response in children with disruptive behavior disorders. J Abnorm Psychol. 2010 Nov;119(4):713–725. [PubMed]
25. McCutcheon AL. Latent class analysis. Newbury Park: Sage Publications; 1987.
26. Neuman RJ, Heath A, Reich W, et al. Latent class analysis of ADHD and comorbid symptoms in a population sample of adolescent female twins. J Child Psychol Psychiatry. 2001 Oct;42(7):933–942. [PubMed]
27. Althoff RR, Copeland WE, Stanger C, et al. The latent class structure of ADHD is stable across informants. Twin Res Hum Genet. 2006 Aug;9(4):507–522. [PubMed]
28. Todd RD, Lobos EA, Sun LW, Neuman RJ. Mutational analysis of the nicotinic acetylcholine receptor alpha 4 subunit gene in attention deficit/hyperactivity disorder: evidence for association of an intronic polymorphism with attention problems. Mol Psychiatry. 2003 Jan;8(1):103–108. [PubMed]
29. Althoff RR, Rettew DC, Boomsma DI, Hudziak JJ. Latent class analysis of the Child Behavior Checklist Obsessive-Compulsive Scale. Compr Psychiatry. 2009;50(6):584–592. [PMC free article] [PubMed]
30. Althoff RR, Rettew DC, Faraone SV, Boomsma DI, Hudziak JJ. Latent class analysis shows strong heritability of the child behavior checklist-juvenile bipolar phenotype. Biol Psychiatry. 2006 Nov 1;60(9):903–911. [PubMed]
31. Nestadt G, Addington A, Samuels J, et al. The identification of OCD-related subgroups based on comorbidity. Biol Psychiatry. 2003 May 15;53(10):914–920. [PubMed]
32. Rindskopf D. Heavy alcohol use in the “fighting back” survey sample: Separating individual and community level influences using multilevel latent class analysis. Journal of Drug Issues Special Issue: The “Fighting Back” program. 2006;36:441–462.
33. Bartels M, van Beijsterveldt CE, Derks EM, et al. Young Netherlands Twin Register (Y-NTR): a longitudinal multiple informant study of problem behavior. Twin Res Hum Genet. 2007 Feb;10(1):3–11. [PubMed]
34. Boomsma DI, de Geus EJ, Vink JM, et al. Netherlands Twin Register: from twins to twin families. Twin Res Hum Genet. 2006 Dec;9(6):849–857. [PubMed]
35. Boomsma DI, Vink JM, van Beijsterveldt TC, et al. Netherlands Twin Register: a focus on longitudinal research. Twin Res 2002. 2002 Oct;5(5):401–406. [PubMed]
36. Conners CK. Conners Rating Scales-Revised. New York/Toronto: Multi-Health Systems; 2001.
37. Achenbach TM. Manual for the Child Behavior Checklist/4-18 and 1991 Profile. Burlington, VT: University of Vermont Department of Psychiatry; 1991.
38. Verhulst FC, van der Ende J, Koot HM. Handleiding voor de CBCL/4-18. Rotterdam: Sophia Kinderziekenhuis/Academisch Ziekenhuis Rotterdam/Erasmus Universiteit Rotterdam, Afdeling Kinder-en jeugdpsychiatrie; 1996.
39. Dempster A, Laird N, Rubin D. Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society, Series B. 1977;39(1):1–38.
40. Vermunt JK, Magidson J. Latent GOLD User’s Manual. Belmont, MA: Statistical Innovations Inc; 2000.
41. Falconer D, MacKay T. Introduction to Quantitative Genetics. Harlow, Essex, UK: Longmans Green; 1996.
42. Steiner H, Saxena K, Chang K. Psychopharmacologic strategies for the treatment of aggression in juveniles. CNS Spectrums. 2003;8:298–308. [PubMed]
43. Ivanova MY, Achenbach TM, Dumenci L, et al. Testing the 8-syndrome structure of the child behavior checklist in 30 societies. Journal of Clinical Child and Adolescent Psychology. 2007;36:405–417. [PubMed]
44. Association AP. [Accessed 12 July 2011];DSM-5 Development. 2011