Search tips
Search criteria 


Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
J Abnorm Psychol. Author manuscript; available in PMC 2008 February 13.
Published in final edited form as:
PMCID: PMC2242352

Externalizing Psychopathology in Adulthood: A Dimensional-Spectrum Conceptualization and Its Implications for DSM–V


Mental disorders involving antisocial behavior and substance use are genetically linked and vary continuously. The authors present a review and integrative conceptualization of these observations in terms of a dimensional and hierarchically organized externalizing spectrum. As a foundation for this conceptualization, the authors introduce a quantitative, model-based approach to comparing categorical and continuous conceptions of psychopathology and apply this approach in an empirical study of patterns of comorbidity among externalizing disorders as defined in the Diagnostic and Statistical Manual of Mental Disorders. The authors present evidence that comorbidity among externalizing disorders is best modeled by an underlying normally distributed continuum of risk for multiple disorders within the externalizing spectrum. The authors conclude by discussing implications of the externalizing spectrum conceptualization for classification of disorders in the upcoming 5th edition of the Diagnostic and Statistical Manual of Mental Disorders.

Keywords: substance abuse/dependence, antisocial behavior, impulsivity, aggression, classification

Problems with antisocial behavior and substance use and personality traits such as aggression and impulsivity commonly co-occur. In the Diagnostic and Statistical Manual of Mental Disorders (4th ed.; DSM–IV; American Psychiatric Association, 1994), these problems constitute the symptoms of a variety of specific mental disorders, and these mental disorders are described in a variety of distinct sections of the manual. In this article, we argue that in the upcoming 5th edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM–V), these problems should be classified together under the rubric of externalizing disorders. We argue that etiologic commonalities should be a major consideration in efforts to classify psychopathological entities (cf. Gottesman, 2002; Skinner, 1981; Tsuang, Stone, & Faraone, 2000; Westen, Heim, Morrison, Patterson, & Campbell, 2002). Moreover, classification of entities in science in general tends to proceed from being more initially descriptive to being based more on underlying principles as these underlying principles become more clearly articulated (Hempel, 1965). Current evidence, reviewed herein, indicates that various externalizing syndromes are linked to the same underlying etiologic factors.

We also consider the issue of whether externalizing problems are better conceived of in terms of dimensions or categories. We evaluate this issue both within specific syndromes and in terms of the co-occurrence of syndromes within the externalizing spectrum, and we conclude by endorsing a dimensional conceptualization on the basis of the existing evidence. However, even if externalizing syndromes and their patterns of co-occurrence are dimensional in nature, this does not preclude categorical decision making for clinical purposes. The key is to make clear the distinction between the empirical structure of these syndromes and the ways in which knowledge of that structure can be brought to bear on categorical decisions that need to be made in clinical practice (Widiger & Clark, 2000). We begin by reviewing evidence for the existence of an etiologically coherent spectrum of externalizing problems encompassing substance dependence and antisocial behaviors and personality traits. Although our focus is primarily on adulthood, we also refer to some literature on adolescence to the extent that it pertains to these problems and is relevant to the development of a dimensional-spectrum conceptualization of the externalizing spectrum.

Evidence for a Coherent Externalizing Spectrum of Psychopathology

Although the idea of a group of disorders or problems united by a common disposition to act out is not new (Achenbach & Edelbrock, 1984; Donovan, Jessor, & Costa, 1988; Gorenstein & Newman, 1980; Zuckerman, 1989), the application of this idea to the study of adult psychopathology via formal quantitative models of patterns of co-occurrence among DSM-defined mental disorders is a newer development. Another development pertains to biometrical models of disorder co-occurrence that take into account biological relationships among research participants in delineating the genetic and environmental etiology of co-occurrence. We first describe the evidence for a coherent observed (or phenotypic) externalizing spectrum, and we then turn to biometrical evidence delineating the genetic underpinnings of the externalizing spectrum.

Phenotypic Modeling of Co-occurrence Among Externalizing Syndromes

Extensive evidence indicates that specific externalizing syndromes are often comorbid (see, e.g., Armstrong & Costello, 2002; Morgenstern, Langenbucher, Labouvie, & Miller, 1997; Newcomb, Galaif, & Locke, 2001, for recent evidence). In fact, externalizing syndromes are correlated with each other; meeting criteria for one externalizing syndrome is a good predictor of meeting criteria for another. This phenomenon of systematic co-occurrence can be accommodated by invoking the idea of a spectrum (cf. Faraone, Tsuang, & Tsuang, 1999; Maser & Patterson, 2002). Essentially, the idea is that specific externalizing syndromes are linked at a more basic level, such that they can be conceived of as specific instantiations of a coherent underlying domain of human variation.

At the same time, the spectrum conceptualization also allows for unique aspects of syndromes within the spectrum. The model is hierarchical: It allows for both general influences on all syndromes within the spectrum as well as specific influences that differentiate syndromes within the spectrum (Krueger & Piasecki, 2002; Lilienfeld, 2003). The general propensity toward externalizing psychopathology can be shaped and molded into different expressions by other genetic and environmental forces, separate from those that contribute to the general propensity. For example, as we describe and document below, specific genetic factors contribute to risk for problems with specific substances, above and beyond the general genetic risk for externalizing problems.

A first line of evidence relevant to the spectrum proposition relates to the phenotypic structure of externalizing syndromes. Statistical models can be fit to data on patterns of co-occurrence among externalizing syndromes to determine whether they are, in fact, indicators of a coherent underlying domain. Stated somewhat differently, if the spectrum notion makes sense, then a model stating that distinct externalizing syndromes are indicators of a coherent underlying domain should fit the data.

A number of specific empirical studies of DSM-defined externalizing psychopathology in adults support this phenotypic conceptualization. Krueger, Caspi, Moffitt, and Silva (1998) modeled comorbidity among 10 common Diagnostic and Statistical Manual of Mental Disorders (3rd ed., rev.; DSM–III–R; American Psychiatric Association, 1987) mental disorders, assessed at ages 18 and 21 in a longitudinal-epidemiological study of a birth cohort from Dunedin, New Zealand. At both ages, diagnoses of conduct disorder (age 18) or antisocial personality disorder (age 21), marijuana dependence, and alcohol dependence formed a coherent single externalizing factor that was clearly distinguished from a separate internalizing factor formed by the other diagnoses studied (major depression, dysthymia, generalized anxiety disorder, agoraphobia, social phobia, simple phobia, and obsessive–compulsive disorder). Krueger (1999) replicated this finding in the National Comorbidity Survey, a national survey of noninstitutionalized persons aged 15–54 in the United States, showing that alcohol dependence, drug dependence, and antisocial personality disorder formed a single externalizing factor, again distinguished from a separate factor representing internalizing disorders. Kendler, Davis, and Kessler (1997) also report evidence from a subset of the National Comorbidity Survey for an externalizing factor in respondents’ reports regarding their parents, correlated with an externalizing factor in the respondents’ reports regarding themselves and, therefore, suggesting parent-to-child transmission of the externalizing propensity. Krueger, McGue, and Iacono (2001) again found a single externalizing factor in a sample of middle-aged parents of twins, indicated by DSM–III–R adult antisocial behavior (the adult criteria from the antisocial personality disorder diagnosis), conduct disorder, alcohol dependence, and drug dependence. Vollebergh et al. (2001) replicated and extended these findings in the Netherlands Mental Health Survey and Incidence Study, in which it was found, across two waves of an epidemiological survey of Dutch adults, that alcohol and drug dependence formed a coherent factor that was distinguished from the internalizing factor. Krueger, Chentsova-Dutton, Markon, Goldberg, and Ormel (2003) also showed that alcohol problems could not be encompassed by the internalizing factor in data from primary care patients in 14 different countries. Thus, the phenotypic aspect of the spectrum proposition has been supported across multiple nations and research groups. Diagnoses pertaining to substance dependence and antisocial behavior form a coherent, unified domain.

Biometrical Modeling of Co-Occurrence Among Externalizing Syndromes

The phenotypic structure of externalizing syndromes is consistent with the idea of a coherent externalizing spectrum of psychopathology. Nevertheless, models of phenotypic correlations among externalizing syndromes are not able to discern the underlying etiologic bases of these correlations. If the idea of a coherent spectrum of externalizing disorders makes sense, then these disorders also should be linked at an etiologic level.

A number of biometrical studies have examined the etiologic bases of correlations among specific externalizing syndromes traversing the spectrum, that is, pertaining to both antisocial behavior and substance use problems, as well as related personality traits. Grove et al. (1990) reported substantial genetic correlations among indices of alcohol abuse/dependence, drug abuse/dependence, and the child and adult aspects of antisocial personality disorder in a sample of identical twins reared apart. Pickens, Svikis, McGue, and LaBuda (1995) presented evidence for genetic contributions to the co-occurrence of alcohol dependence and antisocial personality in male twins. Fu et al. (2002) reported evidence for shared genetic risk for antisocial personality disorder and alcohol dependence and marijuana dependence in a twin sample of Vietnam-era male military veterans (but see also True et al., 1999, who reported evidence for nongenetic, family-environmental contributions to the co-occurrence of conduct disorder, alcohol dependence, and marijuana dependence in this sample). Slutske et al. (1998), however, examined the association between retrospectively reported conduct disorder and alcohol dependence in an Australian adult twin sample and found that common genetic risk factors accounted for most of the correlation between these disorders. Slutske et al. (2002) extended this report to demonstrate that genetic influences on the personality trait of behavioral undercontrol accounted for the majority (90%) of the genetic risk factors common to conduct disorder and alcohol dependence. Mustanski, Viken, Kaprio, and Rose (2003) also presented significant genetic correlations among the personality traits of social deviance and excitement seeking, and alcohol consumption and problems, in young adult Finnish twins. Similarly, Jang, Vernon, and Livesley (2000) reported significant genetic correlations linking alcohol misuse and a variety of personality scales indexing dissocial tendencies.

A handful of recent studies have moved toward explicitly conceptualizing these shared etiologic mechanisms in terms of a genetically coherent externalizing spectrum in late adolescence and adulthood. Krueger et al. (2002) sought to test this conceptualization in a sample of male and female 17-year-old twins. Antisocial behavior (the adult criteria of the antisocial personality disorder diagnosis), conduct disorder, alcohol dependence, drug dependence, and an unconstrained personality style were all connected to a single, highly heritable (81%) externalizing factor. Genetic contributions to this spectrum of disorders and traits were mostly in common, mediated through the heritable externalizing factor connecting these disorders and traits. Hicks, Krueger, Iacono, McGue, and Patrick (2004) extended this model to include the twins’ parents and found that resemblance between parents and their twin offspring could be accounted for entirely by transmission of the general externalizing propensity, as opposed to residual transmission of specific risks for specific externalizing outcomes. Young, Stallings, Corley, Krauter, and Hewitt (2000) presented related findings of substantial genetic contributions (85%) to an externalizing factor linking conduct disorder, substance experimentation, attention-deficit/hyperactivity disorder, and the personality trait of novelty seeking in a sample somewhat younger and more diverse in age (12–18 years). Kendler, Prescott, Myers, and Neale (2003) extended these findings to adults and also to delineation of the externalizing spectrum from the internalizing spectrum at an etiologic level. Alcohol dependence, other drug abuse or dependence, adult antisocial behavior, and conduct disorder formed a genetically coherent externalizing factor. Moreover, genetic influences on this factor were found to be separate from genetic influences on the internalizing factor.


Across distinct samples and research groups, extensive evidence now exists to support the externalizing spectrum conceptualization. DSM disorders involving substance use and antisocial behavior form a coherent, unified domain. Moreover, this phenotypic coherence is undergirded by genetic coherence: Multivariate biometric studies of the spectrum conceptualization indicate that the reason these disorders co-occur is because they share a common underlying genetic vulnerability.

Nevertheless, it is also important to emphasize the hierarchical component of this conceptualization. For example, evidence exists for specific genetic contributions to alcohol and drug abuse/dependence, independent of the broader genetic contribution to all disorders within the externalizing spectrum (Kendler et al., 2003; Krueger et al., 2002). In this regard, the model accommodates both lumping and splitting perspectives because it includes a general (mostly genetic) risk for psychopathology in this domain, along with other specific etiologic influences that shape how this general risk is expressed (e.g., genetic influences on the precise substances abused, perhaps mediated through individual differences in the metabolism of specific drugs).

This emerging empirical literature on the externalizing spectrum has clear implications for classification of mental disorders in the DSM–V. The literature indicates that personality and psychopathology are intimately intertwined; the distinction between syndrome, or Axis I, disorders and personality, or Axis II, disorders is difficult to maintain, given the way in which the spectrum transcends Axis I, Axis II, and normal-range personality variation (see Krueger & Tackett, 2003). Externalizing syndromes form a genetically coherent spectrum of disorders and traits, and the organization of the DSM–V should reflect this etiologic coherence. Nevertheless, for practical reasons (e.g., in determining need for treatment), we need to articulate an approach to differentiating between more normative and more pathological expressions of externalizing tendencies. Thus, we turn next to evidence concerning continuous versus categorical conceptions of the structural organization of the externalizing spectrum.

Continuous Versus Categorical Structure of Externalizing Syndromes

The evidence reviewed above indicates that problems with antisocial behavior, substance use, and impulsivity co-occur because they are elements within a genetically coherent, hierarchically organized externalizing spectrum. But how is this spectrum structured? Do specific symptoms within this spectrum cohere into specific, discrete classes of psychopathology? Or are the symptoms within this spectrum better thought of as indicators of more continuous dimensions that cohere within a broader, overarching domain representing the overall severity and extent of externalizing behavior?

This is a fundamental issue with crucial implications for both research and practice. Yet, the categorical versus dimensional nature of psychopathology often has been adjudicated on the basis of a priori preferences for a specific conception as opposed to empirical evidence (Klein & Riso, 1993). Our view is that this issue is fundamentally empirical, albeit challenging from both conceptual and methodological standpoints. In constructing a diagnostic system—and, hence, a way of thinking about psychopathology in research and practice—we must rely on data. Moreover, empiricism is an important element of the DSM construction process; each successive edition of the DSM has been accompanied by more extensive documentation and empirical support (Nathan & Langenbucher, 1999; Widiger & Clark, 2000). Hence, we describe below how a model-fitting approach to data on externalizing problems can inform our understanding of the structure of externalizing phenomena.

Although data are important in constructing the diagnostic system, when we enter the clinic, data on the dimensional-categorical issue also intersect with practical issues. For example, the evidence reviewed below indicates that alcohol problems are arrayed along a dimension of severity in nature (as opposed to indicating discrete classes of people). Nevertheless, clinicians are often faced with the task of rendering a specific categorical decision about a specific person, such as, “Is inpatient detoxification appropriate and necessary, or not?” Hence, in addition to reviewing evidence pertinent to the dimensional versus categorical nature of externalizing psychopathology, we also address issues at the intersection of data and practical clinical issues.

Latent Structural Modeling of Externalizing Syndromes

Ideas about the nature of externalizing problems as continuous versus categorical are limited to the realm of speculation and a priori preferences until they encounter data. How can we map theoretical ideas about the continuous versus categorical nature of externalizing problems onto data? We can accomplish this by formalizing the empirical implications of our theories as specific statistical models and evaluating how well these models fit the pertinent data. A hypothesis such as “alcohol dependence is a distinct psychopathological category” is a statement about what one would expect to discover in analyzing data on the signs and symptoms of alcohol dependence. Similarly, a hypothesis such as “alcohol dependence refers to severe problems along a graded continuum encompassing diverse alcohol problems, ranging from mild to severe” entails a different prediction about the behavior of data on the signs and symptoms of alcohol dependence.

The kinds of statistical models relevant to this discussion fall under the rubric of latent structure models (Bollen, 2002). A plethora of models of varying degrees of similarity and difference fall under this rubric, but there are two basic sorts of models that are especially pertinent to the current discussion: latent class models and latent trait models (often also referred to as item response theory). These models are the most pertinent statistical models because both are applicable to the same observed data (such as presence vs. absence of specific symptoms or diagnoses). However, they differ in their conceptions of the underlying structures that gave rise to the observed data (see Heinen, 1996, for an excellent comparative review of these models).

Latent class models flow from the idea that the best way to explain the patterns of co-occurrence among a set of symptoms is with reference to a specific number of mutually exclusive underlying classes or categories. Moreover, in the typical latent class model, people within a specific class have the same probability of having the symptoms. For example, a latent class model applied to externalizing symptoms or diagnoses might, in theory, reveal a class of persons with a high probability of antisocial behavior and a low probability of substance use problems and a complementary class with a low probability of antisocial behavior and a high probability of substance use problems. Such findings would suggest that a compelling way to think about the externalizing spectrum is in terms of distinct, homogenous groups of people, that is, separate and distinct categories of antisocial and substance-related psychopathology.

Nevertheless, a latent class model also could show a class with low probabilities of both antisocial behavior and substance use problems and a complementary class with high probabilities of both antisocial behavior and substance use problems. This might suggest that a more compelling account of this domain is in terms of an underlying continuum linking antisocial and substance-related pathology and, therefore, that a latent trait model might be a more appropriate model of the domain. That is, in contrast to latent class models, latent trait models flow from the idea that the best way to explain the patterns of co-occurrence among a set of symptoms or diagnoses is with reference to an underlying continuous dimension.

In a typical latent trait model, specific symptoms have two properties: severity and discrimination. The severity of a symptom pertains to where that symptom is located along the continuum relative to other symptoms, that is, a latent trait model models symptoms along a continuum, from milder symptoms to more rare and severe symptoms. In addition, symptoms vary in their discrimination—their ability to discern the differences between people at nearby levels of severity. Returning to the example of antisocial behavior and substance use problems, a latent trait model might reveal that these problems can be located along a single dimension, with milder symptoms (e.g., abusive drinking) located at lower levels of the continuum than more severe symptoms (e.g., persistent antisocial behavior) and with some symptoms showing better discrimination than others (e.g., having used an illegal substance might be an excellent discriminator between people who have only modest externalizing tendencies and those who are more severely externalizing).

Technically speaking, these descriptions of latent class models and latent trait models do not consider all of the possible extensions of these models that have been described in the methodological literature (e.g., variations on the latent class model that relax the assumption of equal probabilities of symptoms within all classes, latent trait models for multiple underlying continuous dimensions). Nevertheless, these descriptions capture the basic properties of the sorts of latent class models and latent trait models that typically have been fit to data on externalizing psychopathology. We turn now to review this literature and to consider its implications for the optimal conceptualization of the externalizing spectrum.1 We first consider analyses of symptoms within specific externalizing syndromes. We then turn to present new data analyses relevant to adjudicating between continuous and categorical conceptions of the co-occurrence of externalizing syndromes.

Alcohol Problems

A number of latent class model studies of alcohol problems have appeared in the literature. We find it interesting that even though these studies have involved fitting latent class models to data on alcohol problems, the classes that have emerged from these analyses appear to represent degrees of severity on an underlying continuum of alcohol problems.

Bucholz et al. (1996) applied the latent class model to 37 interview-assessed alcohol problems reported by adult relatives of alcoholics. They identified four classes of persons who were differentiated by their probability of having multiple alcohol problems, and the four classes were clearly ranked from least to most severely affected. There was little evidence for distinct types of problems associated with distinct latent classes, and the authors therefore interpreted their results as more consistent with the idea of a continuum of severity underlying alcohol problems. Heath et al. (1994) also reported evidence for latent alcohol problem classes ranked by severity in a sample of male Australian twins. Consistent with the conclusions reached by Bucholz et al., Heath et al. (1994) concluded from their latent class model results that “it appears that there is a continuum of severity of alcohol-related problems” (p. 299).

Nelson, Heath, and Kessler (1998) reported similar findings in the National Comorbidity Survey, an epidemiological survey of noninstitutionalized U.S. civilians. They applied the latent class model to lifetime reports of the nine Criterion A symptoms of DSM–III–R-defined alcohol dependence (American Psychiatric Association, 1987) and found evidence for four classes that were clearly ranked by severity. Bucholz, Heath, and Madden (2000) extended their report on the relatives of alcoholics to a sample of adolescent female twins. They applied the latent class model to 15 alcohol problems and identified five classes of twins that, again, could be arrayed from least to most severe. Kendler, Karkowski, Prescott, and Pedersen (1998) applied the latent class model to temperance board data on Swedish male twins born between 1902 and 1949. The data submitted to the latent class model included age at first registration with the temperance board, number of registrations, and reasons for registration. The authors presented evidence for five classes that could be “roughly ranked by severity” (p. 811). Finally, Chung and Martin (2001) applied the latent class model to data on DSM–IV alcohol abuse and dependence symptoms observed in a sample of adolescents recruited from clinical settings. Three classes were found that, again, could be ranked by severity rather than by unique profiles of problems.

Krueger et al. (2004) noted the consistency of these latent class model findings and suggested, therefore, that the latent trait model approach might be a useful alternate model for the alcohol problems domain. Specifically, Krueger et al. applied the latent trait model to data on a wide range of interview-assessed alcohol problems (110 specific symptoms) in a large general-population sample of adult men. Krueger et al. found that the latent trait model provided an excellent fit to these data. That is, the 110 problems could be clearly scaled along a single continuum of severity, spanning heavy and abusive drinking at the low end, through tolerance and withdrawal, to serious complications of alcoholism at the high end, such as being unable to work without a drink. Kahler, Strong, Hayaki, Ramsey, and Brown (2003) and Kahler, Strong, Stuart, Moore, and Ramsey (2003) used related methods (a nonparametric form of item response theory) in a sample of treatment-seeking alcoholics and a sample of persons mandated to a domestic violence intervention program and also found evidence for a continuum of alcohol-related problems arranged by severity. In addition, Kahler, Strong, Read, Palfai, and Wood (2004) used Rasch modeling (a latent trait model with a severity parameter but not a parameter for discrimination) to map a continuum of severity of alcohol problems in college students (see also Cornel, Knibbe, van Zutphen, & Drop, 1994, for related evidence of the applicability of the Rasch model to alcohol problems in the general practice setting). Similarly, Muthén (1996) reported evidence for “a large degree of ‘severity’ ordering” (p. 106) among DSM-derived alcohol abuse and dependence criteria, using a sophisticated approach to factor analysis (essentially a multidimensional extension of the latent trait model). In sum, both latent class model and latent trait model approaches to the analysis of alcohol problems are consistent in indicating that alcohol problems compose a continuum of severity.

Drug Problems

Drug problems also appear to represent degrees of severity on underlying continua. Schafer and Caetano (1996) presented a particularly relevant analysis of the DSM–IV construct of cocaine dependence. Their analysis is particularly relevant because they fit both latent class models and latent trait models to their data on a mixed ethnicity sample of men admitted to substance abuse treatment programs. Both analyses were interpreted as supporting the unidimensionality of the cocaine-dependence construct. The latent trait model analysis revealed a single continuum of severity linking the criteria, and the latent class model also revealed classes arranged in order of severity. In addition, Kan, Breteler, van der Ven, and Zitman (1998) used Rasch modeling to refine criteria for benzodiazepine dependence and demonstrated severity ordering among the refined criteria. Specific aspects of substance dependence, such as withdrawal, also appear continuous in nature. For example, Madden et al. (1997) used a latent class model to identify three major classes of nicotine withdrawal that could be ordered from most to least severe.

Latent class model analyses of multiple drugs of abuse also indicate a continuum of severity. Pedersen and Skrondal (1999) used the latent class model to show a continuum of drug involvement in a large general population sample of adolescents, ranging from more licit substances (alcohol and cigarettes), through cannabis, to illicit substances (amphetamines, ecstasy, heroin). Latent class model analyses presented by McCutcheon and Thomas (1995) and Mitchell and Plunkett (2000) also support a severity continuum of drug involvement in adolescence. Kirisci, Vanyukov, Dunn, and Tarter (2002) applied this conceptualization to adults via the latent trait model. Specifically, these authors used the latent trait model to reveal a single continuous trait of substance involvement linking 10 licit and illicit substances. Thus, as with alcohol problems, latent class model and latent trait model studies of drug problems also indicate that such problems can be meaningfully modeled in terms of severity continua.

Antisocial Personality and Behavior

A number of recent studies have fit latent class models to data on antisocial behavior. Bucholz, Hesselbrock, Heath, Kramer, and Schuckit (2000) fit latent class models to data on 38 symptoms pertaining to child and adult aspects of DSM–III–R antisocial personality disorder, obtained from a large sample of alcoholic probands, their relatives, and control participants. The classes that emerged from the analysis in both men and women represented grades of severity on an underlying continuum. That is, rather than different probabilities of different symptoms distinguishing the classes, the latent class model revealed increasing probabilities of symptoms across classes.

Kovac et al. (2002) fit latent class models to data on conduct disorder and adult antisocial symptoms (i.e., they separated the adult and child criteria in antisocial personality disorder) from a multinational study of men who met DSM–IV criteria for alcohol dependence. For the most part, the probability of the symptoms increased systematically across the latent classes (especially for the conduct disorder symptoms), but the authors also noted some more abrupt increases in symptom probabilities across classes (specifically, for job-related and aggressive aspects of the adult criteria), suggesting possible discreteness of some aspects of the underlying adult behavior classes. Related to this analysis, Muthén (2002) presented a latent class model of nine diverse items pertaining to substance problems, aggression, and delinquency in a sample of 16–23 year olds. Two of the four emergent classes were characterized by high and low probabilities of displaying each of these behaviors, but the two other classes could be distinguished by more overt, aggressive behaviors (e.g., fights) and more covert, delinquent behaviors (e.g., shoplifts; also, see Eaves et al., 1993; Fergusson, Horwood, & Lynskey, 1994, and Muthén & Muthén, 2000, for related latent class model findings of both continuous and discrete variation distinguishing latent classes of antisocial behavior). Nevertheless, Brownfield and Sorenson (1987) presented latent class model evidence for three delinquency classes ordered by severity. In addition, Osgood, McMorris, and Potenza (2002) fit a latent trait model to delinquency data and found evidence for a continuum of severity. Finally, David Cooke and his colleagues (Cooke, Kosson, & Michie, 2001; Cooke & Michie, 1997, 1999; Cooke, Michie, Hart, & Hare, 1999) have pursued a successful program of research demonstrating the applicability of latent trait models to aspects of psychopathic personality.

In sum, findings from this domain are broadly consistent with a continuum of severity. However, some studies also suggest additional heterogeneity (e.g., the presence of latent classes distinguished by more than a single continuum of severity), relative to alcohol and drug problems. This is likely because the variables in these analyses sometimes represent multiple domains within the overarching externalizing spectrum. We recently evaluated this conceptualization specifically with reference to DSM–IV-defined conduct disorder in boys (Tackett, Krueger, Sawyer, & Graetz, 2003). Tackett et al. (2003) found that the DSM–IV conduct disorder criteria formed two correlated subdimensions rather than one purely unitary dimension. The first dimension was characterized more by aggressive content (e.g., bullies, threatens, or intimidates), whereas the second dimension was characterized more by delinquent content (e.g., stolen without confrontation). In a typical latent class model, this type of detailed, hierarchical structure is not modeled directly and likely would emerge as classes distinguished by a mixture of continuous and discrete variation, as was sometimes seen in the studies reviewed above. A more complete and explicit evaluation of the antisocial domain would, therefore, involve refinement of fine-grained groups of coherent symptoms (or facets) and statistical modeling of symptoms within these facets and of the relations among the specific facets.

Thus, although many findings from this domain are broadly consistent with a continuum of severity, it is important to begin to develop a conceptualization, and associated measurement technology, for refined facets of antisocial behavior that constitute the basic elements of variation within this part of the externalizing spectrum (e.g., aggression and impulsivity). In addition, across each of the externalizing domains reviewed, methodological work is needed that aims to bring together various latent structure modeling techniques under a more unified conception. We have begun pursuing this kind of work, and in the next section, we introduce a novel quantitative approach to directly comparing latent class models and latent trait models of comorbidity among externalizing syndromes.

Comparing Latent Class and Latent Trait Models of Comorbidity Among Externalizing Syndromes: An Empirical Investigation

We are not aware of a study that has specifically sought to model comorbidity among DSM-defined externalizing syndromes using both the latent class model and latent trait model in a large-scale, population-based sample of adults, with the goal of directly comparing the results from both models. We therefore undertook such a study. Specifically, we compared the fit of both latent class and latent trait models with data on the comorbidity among diagnoses of conduct disorder, adult antisocial behavior (the adult criteria from antisocial personality disorder), alcohol dependence, marijuana dependence, and drug dependence in a large-scale, population-based sample of adults. The goal of this analysis was to determine whether the comorbidity among these externalizing disorders was better modeled in terms of an overarching externalizing continuum or in terms of a series of discrete latent classes.

Because this investigation entailed grappling directly with technical aspects of comparing continuous and discrete conceptions of latent structure, our presentation is, by necessity, somewhat more technical than other sections of this article. However, throughout this presentation, we endeavor to explain technical concepts in an accessible manner. In addition, technical issues are part and parcel of bringing data to bear on comparing discrete and continuous conceptions of psychopathology. As we described earlier, each edition of the DSM has been accompanied by increasing emphasis on the role of data (Nathan & Langenbucher, 1999; Widiger & Clark, 2000). This necessitates formal modeling of data pertinent to the continuous versus categorical nature of psychopathology as we move toward the DSM–V.


Research Participants

The sample for this investigation comprised parents of male and female twins participating in the Minnesota Twin Family Study (MTFS), an ongoing longitudinal study of psychopathology, with a particular focus on substance-related problems. Detailed descriptions of MTFS have been provided elsewhere (Iacono & McGue, 2002); families participating in MTFS were ascertained by the identification of all twins born in Minnesota during specific time periods, by use of public birth records. Potential participants were not included if they lived farther than a 1-day drive from the study location in Minneapolis or if either twin had a physical or intellectual disability that prevented participation in a day-long assessment. Previous research has suggested that the MTFS sample is well representative of the Minnesota population in demographic and psychological characteristics (Holdcraft & Iacono, 2004). The current sample at intake comprised 2,859 adult parents of twins; 24 individuals with missing data could not be included, resulting in a final sample of 2,835 participants (49% fathers and 51% mothers).

Assessment of Externalizing Syndromes

All participants were interviewed to assess lifetime symptoms of psychopathology according to DSM–III–R criteria (American Psychiatric Association, 1987), which was the diagnostic system in use at intake. Interviewers were trained extensively and had either a bachelor’s or master’s degree in psychology. Following the interview, interview data were reviewed in a clinical case conference by at least two graduate students with advanced training in descriptive psychopathology and differential diagnosis. Symptoms were deemed present or absent on the basis of consensus between the two diagnosticians.

Consistent with existing conceptualizations of the externalizing spectrum (see Krueger et al., 2002), five diagnoses were examined in the current analyses: conduct disorder, adult antisocial behavior, alcohol dependence, marijuana dependence, and drug dependence. Adult antisocial behavior comprised Criterion C (adult) symptoms of antisocial personality disorder; the threshold for the diagnosis was four symptoms, consistent with Criterion C of the antisocial personality disorder diagnosis in DSM–III–R. As has been supported by previous work on externalizing (Krueger et al., 2002), we distinguished between child (conduct disorder) and adult (adult antisocial behavior) symptoms of antisocial personality disorder. We did not assess conduct disorder Symptom 9 (“has forced someone into sexual activity with him or her”) to avoid potential mandated reporting. Included in the drug dependence diagnosis were amphetamines, cocaine, hallucinogens, inhalants, opioids, PCP, and sedatives. Individuals were considered to have met criteria for drug dependence if they met substance dependence criteria for at least one of these substances. Further details regarding the assessment protocol can be found in Iacono, Carlson, Taylor, Elkins, & McGue (1999).

Statistical Models

As the goal of the analyses was to compare discrete versus continuous conceptualizations of externalizing phenomena, two models were compared in their ability to model comorbidity among the five diagnoses. These two models were the latent class model and a latent trait model with the two parameters we described earlier, that is, severity and discrimination (this latent trait model has typically been referred to as a two-parameter partial credit model in the methodological literature, where “partial credit” refers to the idea that the model can handle variables with multiple ordered categories of response; see Heinen, 1996; Muraki, 1992). Thus, in this latent trait model, each diagnosis was modeled in terms of how severe it was, within the externalizing spectrum, and how discriminating it was (i.e., the ability of the diagnosis to discriminate between people at nearby levels of externalizing severity). We formulated each of these models in the same mathematical framework (a logistic framework) to allow for more direct comparisons between the two models (Heinen, 1996; Vermunt, 2001).

The latent class model represents comorbidity among diagnoses in terms of discrete latent groups or classes. Diagnoses are assumed to be conditionally independent given the latent classes—that is, the latent classes are assumed to completely account for comorbidity among diagnoses. Although there are many formulations of the unrestricted latent class model, one particularly useful formulation is in terms of the conditional probability of being assigned a particular diagnosis given that one is in a particular latent class. This formulation of the unrestricted latent class model is given by


where P(Xij |θk) is the conditional probability of having diagnostic status j on diagnosis i, given that one is in latent class k. The conditional probability is accounted for by two components: a component associated with latent status and a component associated with assessed diagnostic status. The parameter uik, appearing in the exponential term, represents that component of the conditional probability associated with one being in latent class k. The other parameter of the exponential term, uij, similarly represents that component of the conditional probability associated with diagnostic status j on diagnosis i. In the equation above, j is not an estimated parameter but a diagnostic status multiplier equal to the diagnostic status index (i.e., 0 … J for J categories of diagnostic status—two categories, absent or 0 vs. present or 1, in our case). This multiplier accounts for the ordinal nature of the diagnostic variables, as categories of diagnostic status are ordered in severity (absence of a diagnosis is less severe than presence of a diagnosis).

In contrast to the latent class model, which accounts for comorbidity in terms of discrete underlying types, the two-parameter latent trait model accounts for comorbidity in terms of continuous underlying traits. Again, diagnoses are assumed to be conditionally independent given the latent trait—that is, the latent trait is assumed to completely account for comorbidity among diagnoses. The general conditional probability formulation of the latent trait model we used is given by


where P(Xij |θk) is the conditional probability of having diagnostic status j on diagnosis i, given that one has latent trait value k. The parameter ui is a discrimination parameter reflecting how closely the diagnosis reflects the underlying latent trait, and the parameter θk reflects the latent trait value k. The parameter again represents that component of the conditional uij probability associated with having diagnostic status j on diagnosis i. As in the first equation, j is not a parameter but a diagnostic status multiplier equal to the diagnostic status index (i.e., 0 vs. 1, corresponding to absence and presence of the diagnosis, respectively).

Comparisons between the latent class model (in Equation 1) and the latent trait model (in Equation 2) reveal that discrete and continuous latent variable models can be treated within a single framework (cf. Heinen, 1996). Both can be formulated in a conditional probability framework, where the conditional probability of an observed diagnosis is modeled by two terms: a term representing the relationship between observed status and latent status (i.e., uik j in the latent class model and uiθk j in the latent trait model) and a term representing observed status alone (i.e., ui j in both the latent class model and latent trait model). Through these two terms, both models include a description of how the latent and observed variables are related to one another and how the observed variables function beyond their relationships with the latent variables.

The latent class model differs from the latent trait model in the way that the terms representing the relationship between observed status and latent status are modeled. In the latent class model, the term representing relationships between observed status and latent status is not assumed to have any specific form; there are as many terms as there are independent combinations of diagnoses and latent values. In the latent trait model, in contrast, the term representing relationships between observed status and latent status is assumed to have a particular form. It is assumed, in particular, that the relationship can be described multiplicatively as the product of a discrimination parameter and a latent trait value.

In this way, the latent trait model can be seen as a restricted form of the unrestricted latent class model, where the uik j term in Equation 1 is instead assumed to have the form uiθk j of Equation 2. As a restricted form of the latent class model, the latent trait model can be treated as a comparison model. Specifically, this comparison represents a test that the effects of each additional latent value—for example, each additional value of the latent trait or each additional latent class—are graduated rather than abrupt. The latent trait model, in this way, can be thought of as a restricted latent class model in which the effects of each latent class are eliminated because they are assumed to have a certain form.

Fitting these models to data involves a procedure known as maximum-likelihood estimation. The goal is to use the data from the sample to identify the estimates of the parameters of both the latent class model and latent trait model that have the greatest likelihood of being the actual parameters in the population from which the sample was drawn. A key point along these lines is that the likelihood of the data can be written in the same form for both the latent class model and the latent trait model, facilitating direct comparison. The likelihood is expressed as a log-likelihood of the data and is given by


Where nx is the frequency of response pattern x (i.e., the frequency with which a specific pattern of diagnostic status on the five externalizing diagnoses was observed). The log-likelihood is thus equal to the product of the frequency of response pattern x and the log of its expected probability. The expected probability of response pattern x can be written, in turn, as the product of the conditional probability of x given θ and the probability of θ.

The probability of each latent value θk, P(θk), can be estimated and thus represents another set of parameters. Alternatively, P(θk) can be assumed to have a certain form, such as a normal distribution, and fixed a priori. Thus, in addition to testing whether the latent variables are nominal or interval in nature, the distribution of the latent variables, if interval in nature, can be evaluated. In the present analyses, we both estimated and fixed the probabilities of the latent values to those of a normal distribution, when appropriate, to compare different conceptions of the latent structure of the externalizing spectrum. We used the software package LEM (Vermunt, 1993, 1997) to fit both latent class and latent trait models to our data on patterns of comorbidity among externalizing disorders in adults.

Model Selection Criteria

We used an information-theoretic approach to identify the best fitting model for our data. Information-theoretic methods emphasize minimizing the amount of information required to express the data and the model, either in an absolute or relative sense. Information-theoretic methods thus result in selection of models that are the both efficient and accurate representations of observed data. Information-theoretic methods have a number of desirable properties not shared by classical likelihood-based inference. For example, in cases of nested model comparisons, information-theoretic methods coincide with likelihood-based inference, making the two methods analogous for certain problems. However, unlike classical likelihood-based methods, information-theoretic methods are appropriate for nonnested model comparisons as well, such as comparisons between discrete and continuous representations of latent structure.

The information-theoretic fit criterion used in current analyses was the Bayesian information criterion (BIC). BIC is enjoying widespread use and has been examined extensively in the context of latent variable modeling (e.g., Celeux & Soromenho, 1996; Fishler, Grosmann, & Messer, 2002; Lin & Dayton, 1997; Yang, 1998) and elsewhere (e.g., Zhang, 1993). Simulation studies have demonstrated that BIC is suitable for comparisons between latent class models (Yang, 1998) and is theoretically certain to select true models provided the sample size is large enough (Barron & Cover, 1991). BIC is given by the equation


where −ln(L) is the negative log-likelihood, k is the number of parameters in the model, and N is the sample size. Smaller values of BIC indicate better fitting models.

Although BIC is often explained with Bayesian theory (e.g., Raftery, 1993; Schwarz, 1978), it has another interpretation that more directly relates to issues of model parsimony. In addition to estimating Bayesian quantities, BIC also provides an estimate of the normalized maximum-likelihood (NML) or minimum description length criterion, originally described by Rissanen (1983, 1989, 1996, 2001). The NML, which is prohibitively challenging to compute directly, quantifies the amount of information (e.g., in bits) required to describe the data given a model of interest. In the selection of models through the minimization of the NML, complexity is minimized, thereby maximizing parsimony. As BIC estimates the NML in large samples, minimizing BIC thus directly corresponds to maximizing parsimony in an information-theoretic sense. Hence, BIC was used as a means of adjudicating among latent class and latent trait conceptions of the externalizing spectrum, because BIC identifies the model that most parsimoniously and accurately models patterns of comorbidity.


Model Fit

Model comparisons are given in Table 1. We present the results collapsed across gender because analyses of the data separated by gender revealed no evidence of gender differences in terms of the best fitting model for the data. As can be seen in Table 1, the best fitting model for the data (i.e., the model with the smallest BIC value) was the latent trait model in which the latent distribution was continuous and normally distributed. As described earlier, ln(L) and k cannot be interpreted directly as indices of model fit but are reported for the sake of completeness in description of the quantities that enter into the computation of BIC.

Table 1
Summary of the Fit of Latent Class and Latent Trait Models to Patterns of Comorbidity Among Externalizing Diagnoses in Adults

BIC differences between two models also can be expressed as the posterior odds of one model over the other, given the data, where the prior odds indicate no a priori preference between the two models (1:1). This is done through exponentiation of the BIC difference (i.e., posterior odds = eBIC difference). As can be seen in Table 1, the closest competitor to the normal distribution latent trait model was the latent trait model with three values. The BIC difference between these two models was 8.712; hence, the posterior odds of the normal distribution latent trait model, compared with the latent trait model with three values, were 6,075.381:1. Before we turn to further interpret the normal distribution latent trait model, we also note some interesting additional aspects of the information presented in Table 1.

First, the fit of the two-class latent class model and the two-value latent trait model are equivalent. This has been noted in other work (e.g., Bartholomew & Knott, 1999, p. 135; Haertel, 1990) and is intuitively reasonable—when only two classes are modeled, it must be possible to order them on an interval. Nevertheless, other models in Table 1 are not equivalent, a situation that allows for comparisons to be made among these models.

Along these lines, in general, the latent class models have less negative likelihood values, closer to 0. However, these likelihoods are linked to greater model complexity, at least as is indexed by the number of parameters: The latent class models have the greatest number of parameters of the models considered. The latent class models are relatively complex and do not specify a form of relationship between latent variables and observed variables. The latent trait models, in contrast, specify that the relationship between the latent and observed variables has a specific form. The additional complexity of the latent class models might, in theory, be needed to capture aspects of the data that cannot be captured by a latent trait model. This would be reflected in a notably greater likelihood for a latent class model, in spite of its greater number of parameters and, hence, in a better fit (i.e., a smaller BIC). However, this is not what we observed in the results displayed in Table 1. Instead, the best fitting model was the normal distribution latent trait model.

Interpretation of Parameter Estimates From the Best Fitting Model

The results indicate that the best fitting model for the externalizing spectrum conceived of the spectrum in terms of a continuous latent trait rather than as a set of discontinuous latent classes. These results further indicate that this latent trait or liability to externalizing psychopathology can be assumed to have a normal distribution. In sum, the results indicate that comorbidity among externalizing disorders is best accounted for by an underlying normally distributed latent continuum of risk for multiple forms of externalizing psychopathology.

Given this observation, it is important to ask how externalizing disorders are related to the latent continuum in the sense of what information each diagnosis provides about the latent continuum. In this context, information has a specific statistical meaning: The information provided by a diagnosis refers to how discriminating that diagnosis is at specific levels of externalizing severity. Greater information refers to greater discriminating power (the ability of the diagnosis to discriminate among people at different levels of externalizing severity), and the information provided by a specific diagnosis varies as one traverses the externalizing spectrum, from lower to higher levels of severity. The location of the peak of an information curve corresponds to how severe the corresponding diagnosis is (how much externalizing tendency a person has to have to be more likely than not to meet criteria for the diagnosis), and the height of the curve corresponds to the discriminating power of the diagnosis.

Parameter estimates corresponding to the normal distribution latent trait model are provided in Table 2 for each diagnosis, and Figure 1 presents plots of the information provided by each diagnosis, derived from the estimates in Table 2. Inspection of Table 2 and Figure 1 reveals that each externalizing diagnosis can be considered in two ways: how much information it provides about an individual’s position on the latent externalizing continuum and where on that continuum it provides information. The overall amount of information provided by each diagnosis is proportional to the height of the corresponding curve on Figure 1 and to the discrimination parameter ui in Table 2. Thus, diagnoses with larger ui values and taller information curves provide more information overall across the latent continuum. Where each diagnosis provides maximum information, in contrast, is proportional to the quantity −(ui1/ui), which can be thought of as a threshold or severity value (ui1 values are also given in Table 2 because these are the estimates that correspond to Equation 2; however, they do not have as direct an interpretation as threshold values). The quantity −(ui1/ui) represents the point on the latent continuum (scaled in standard deviation units and with a mean of 0, i.e., in a z-score metric) where an individual is more likely to receive the diagnosis than not; note that the quantity −(ui1/ui) corresponds with the point on the horizontal axis of Figure 1 where the information curve is highest along the vertical axis. Diagnoses with larger threshold values can be thought of as being more severe, in the sense that someone has to be more externalizing to be more likely to receive a diagnosis than not.

Figure 1
Information refers to how discriminating a diagnosis is at specific levels of externalizing severity. Greater information indicates greater discriminating power. AAB = adult antisocial behavior; MD = marijuana dependence; DD = drug dependence; CD = conduct ...
Table 2
Parameter Estimates From the Normal Distribution Latent Trait Model of Comorbidity Among Externalizing Diagnoses in Adults

Diagnostic status with regard to adult antisocial behavior, drug dependence, and marijuana dependence, for example, composed the three greatest sources of information about someone’s externalizing tendencies (i.e., they had the largest ui values). Similarly, the parameter estimates reflected in Table 2 and Figure 1 indicate that the substance dependence diagnoses can be arrayed in severity, with drug dependence representing the most severe diagnosis, marijuana dependence being slightly less severe, and alcohol dependence being the least severe substance dependence diagnosis; note the −(ui1/ui) threshold values for these diagnoses in Table 2, as well as the locations of the peaks of their information curves on Figure 1. This finding is intuitively appealing and consistent with the aforementioned literature documenting a continuum of drug involvement ranging from more licit to more illicit substances (see Kirisci, et al., 2002; McCutcheon & Thomas, 1995; Mitchell & Plunkett, 2000; Pedersen & Skrondal, 1999). Figure 1 also indicates that adult antisocial behavior provides greater information about externalizing tendencies than do other disorders. This is consistent with the relatively broad content subsumed by the adult antisocial behavior criteria. Adult antisocial behavior also covers a longer developmental period than do the conduct disorder criteria, which may explain why conduct disorder is more limited in how informative it is regarding overall extent of externalizing psychopathology in adults. That is, conduct disorder may be less informative about the latent externalizing trait in adults because, in many cases, it represents an adolescence-limited expression of antisocial tendencies (Moffitt, 1993). It may also be less informative in the sense that in adults, its assessment is based on retrospective recall.

Also notable is the observation that even though we used dichotomous DSM–III–R diagnoses in this work, the results indicate that those dichotomies are linked by a latent continuum. That is, dichotomous DSM diagnoses of externalizing disorders are indicative of a person’s level on a continuum of risk for multiple forms of externalizing psychopathology rather than of membership in discrete psychopathological classes. In particular, the diagnoses reflect higher levels of the externalizing continuum (note how the peaks of the information curves on Figure 1 are shifted to the right). This is consistent with the fact that DSM diagnoses are designed to represent more extreme, less normative behaviors in the general population. Nevertheless, the continuum could, in theory, be better captured along its entire range. That is, Figure 1 suggests that an important goal for future research on externalizing phenomena will be to understand and model the entire breadth of the spectrum, from its more normative to its more pathological expressions. This will involve assessing constructs at a lower level of aggregation than the DSM diagnoses and across a wide range (e.g., specific varieties of aggression, from more verbal manifestations to physical manifestations that would be grounds for arrest). The key point of these analyses, however, is that externalizing mental disorders, described and diagnosed as categories per the DSM, appear to represent an extreme form of a broad continuum of externalizing tendencies.

Moving Toward DSM–V: Future Directions and Practical Considerations at the Interface of the Externalizing Spectrum and Clinical Practice

Our literature review and analysis of data support a dimensional-spectrum conceptualization of externalizing psychopathology in adulthood. A review of the pertinent literature, as well as our data analysis, shows that externalizing phenomena are well conceived in terms of a broad but coherent group of disorders that vary continuously both within and among syndromes. A number of specific recommendations for placing DSM–V on solid empirical footing follow from these observations.

First, the coherence of the externalizing spectrum should be explicitly recognized in DSM–V. A logical way to recognize this coherence is to organize externalizing syndromes into a specific chapter of the manual. In addition, the organization of these syndromes calls for a reevaluation of the putative distinction between Axis I (syndrome disorders) and Axis II (personality disorders). This distinction is not well supported by data on the externalizing spectrum, which transcends Axes I and II (cf. Krueger & Tackett, 2003).

Along these lines, the hierarchical–dimensional externalizing spectrum model should frame the relevant section of DSM–V, because the fit of the model to the data leads to improved understanding and practical application in comparison with the existing prototype categorization model. For example, from a research standpoint, the current perspective is more efficient in terms of exploring etiology. A coordinated theory of several disorders can be formulated, rather than separate theories for each disorder, and research efforts coordinated accordingly. From a treatment standpoint, the model leads to a focus on generalized interventions for disorders within the spectrum, as well as variegated strategies for treatment of specific syndromes within the spectrum (e.g., specific forms of substance dependence) on the basis of the relative role of the broad externalizing vulnerability in different cases.

This latter point is of particular importance. Although the evidence reviewed herein clearly points to the coherence of the externalizing spectrum, the literature also points to the importance of specific genetic and environmental factors in shaping the broad genetic liability linking all externalizing syndromes. To date, such evidence mostly has pertained to extant DSM syndromes. The possibility of further subdivisions within the DSM syndromes has received less attention. Nevertheless, this is an important area for future research because emerging evidence points toward meaningful variants within some syndromes. For example, as we described earlier, conduct disorder consists of at least two distinguishable subdimensions pertaining to more aggressive and more rule-breaking behaviors (Tackett et al., 2003). The development of an enriched hierarchical model that encompasses levels ranging from these very fine-grained distinctions up to the level of the overarching externalizing dimension is an important goal for future research.

Second, research should continue to develop our understanding of the breadth of the externalizing spectrum. A number of disorders described in DSM–IV may be good candidates for inclusion within the spectrum. For example, antisocial personality disorder shares significant genetic variance with pathological gambling, suggesting that this disorder might also be conceptualized as an externalizing spectrum disorder (Slutske et al., 2001). Similarly, disorders that have received less research attention (e.g., intermittent explosive disorder; McElroy, 1999) might be reconceptualized as configurations of fundamental elements within the externalizing spectrum, such as aggression and impulsivity (Coccaro, Kavoussi, Berman, & Lish, 1998). In addition, the methods and conceptual approach developed here also could be applied to disorders that may transcend spectra (e.g., borderline personality disorder, which includes features that appear to be both internalizing and externalizing in nature; Sanislow et al., 2002) as well as to putatively distinct psychopathology spectra (e.g., the schizophrenia spectrum; Wolf et al., 1988).

Third, given that the current evidence supports a dimensional conceptualization, DSM–V should explicitly recognize that externalizing syndromes are dimensional in nature. Along these lines, the inclusion of specific symptoms as indicators of specific externalizing syndromes should be based on empirical models of the relevant data. That is, symptoms should be selected because research shows them to be informative with reference to specific syndromes. In particular, the approach we have outlined in this article, which involves explicit quantitative modeling of psychopathological data, should prove useful in identifying optimal conceptualizations and informative symptoms. In theory, such research might also identify syndromes in which a categorical conceptualization provides a better account of the data. Although this seems unlikely with reference to the externalizing syndromes reviewed herein (given the current state of the evidence), the cardinal point is that the process of identifying the optimal conceptualization of psychopathology must be based on data and close collaboration between methodological and substantive experts, not on a priori preferences.

In addition, it is also important to note that models of psychopathological phenomena can accommodate data from any modality. In this sense, external correlates also can enter into the process of adjudicating among various models. For example, reduced amplitude of the P3 event-related potential appears to be a promising indicator of genetic risk for externalizing spectrum disorders (Iacono, Malone, & McGue, 2003). This putative endophenotype has been associated with familial risk for a variety of externalizing disorders, including substance use and antisocial behavior disorders (Iacono, Carlson, Malone, & McGue, 2002). Moreover, Iacono et al. (2002) showed that mentally healthy adolescents at low familial risk for developing externalizing psychopathology who nevertheless did so by young adulthood had reduced P3 amplitudes as adolescents, thus illustrating that this endophenotype can identify high-risk individuals in the absence of other common externalizing risk indicators. Other promising biological markers of risk for psychopathology (e.g., specific genotypes that appear to confer risk for externalizing psychopathology, such as monoamine oxidase A; Caspi et al., 2002) can and should be modeled along with data pertinent to specific syndromes in constructing the diagnostic system.

Finally, we note that conceptualizing externalizing syndromes as continuous in nature does not in any way mean that thresholds cannot or should not be identified for clinical and practical purposes (cf. Widiger & Clark, 2000). The key is to understand the fundamental distinction between psychopathology in and of itself and the implications of psychopathology for society and the individual. That is, even if externalizing phenomena vary continuously in the population, beyond a certain level they become a problem for the person and/or for other people. What it means for something to “become a problem” is partly a value judgment, not something that can be decided solely with reference to data (cf. Kendler, 1990). Nevertheless, empirical research on how the probability of adverse consequences is linked to variation within the externalizing spectrum can help professionals decide where to place cutoffs relevant to specific clinical decisions. This situation is exactly akin to other situations in clinical medicine, for example, the need to place cutoffs on continuous variables such as cholesterol count that correspond to unacceptable risk for adverse consequences. In preparation for DSM–V, therefore, research should be pursued that examines and models the nature of the relationship between externalizing phenomena and social consequences such as economic impact (e.g., days lost at work, missed educational opportunities), personal impact (e.g., family strife, child abuse and neglect), and impact on society (e.g., involvement with the criminal justice system). The results of such research should inform professional decisions about how DSM–V will demarcate the distinction between societally sanctioned and societally problematic externalizing behavior.


This work was supported in part by United States Public Health Service Grants AA00175, AA09367, DA05147, and MH65137.


1We do not review literature using an alternative approach to addressing the categorical vs. continuous nature of psychopathology, the taxometric methods developed by Paul Meehl (see Waller & Meehl, 1998, for an account of these methods). With regard to externalizing syndromes, we are not aware of work applying these methods to alcohol or drug problems; however, a handful of taxometric studies of antisocial behavior have been reported in the literature (e.g., Harris, Rice, & Quinsey, 1994; Skilling, Harris, Rice, and Quinsey, 2002; Skilling, Quinsey, & Craig, 2001). These studies are intriguing, but they are difficult to integrate with latent class model and latent trait model studies. Taxometric methods are quite distinct when compared with the statistical methods embodied by the latent class model and latent trait model, because much of the statistical infrastructure behind latent variable modeling is not used in taxometrics. Instead, taxometric methods rely primarily on computation of quantities in the observed sample data and visual examination of plots of these quantities to adjudicate between continuous and categorical conceptions of the latent structures that gave rise to the observed data. Statistical approaches to latent variable analysis, in contrast, involve fitting explicit mathematical models to sample data by use of well-characterized estimators of population parameters (e.g., maximum likelihood) and evaluating the fit of these models by use of quantitative indices of fit. Along these lines, compelling methodological and conceptual concerns have been raised regarding taxometric methods (e.g., Beauchaine & Waters, 2003; Miller, 1996; Widiger, 2001), and some putative taxa identified using these methods appear to have weak construct validity (e.g., weak temporal stability of membership in a putative pathological dissociation taxon; see Watson, 2003). In sum, along with others (e.g., Waldman & Lilienfeld, 2001), we feel that methodological work that helps to link taxometrics with other latent variable models is needed to more fully evaluate taxometric research, in addition to continued research directed at establishing the construct validity of putative taxa identified with these methods.


  • Achenbach TM, Edelbrock CS. Psychopathology of childhood. Annual Review of Psychology. 1984;35:227–256.
  • American Psychiatric Association. Diagnostic and statistical manual of mental disorders. 3. Washington, DC: Author; 1987.
  • American Psychiatric Association. Diagnostic and statistical manual of mental disorders. 4. Washington, DC: Author; 1994.
  • Armstrong TD, Costello EJ. Community studies on adolescent substance use, abuse, or dependence and psychiatric comorbidity. Journal of Consulting and Clinical Psychology. 2002;70:1224–1239. [PubMed]
  • Barron AR, Cover TM. Minimum complexity density estimation. IEEE Transactions on Information Theory. 1991;37:1034–1054.
  • Bartholomew DJ, Knott M. Latent variable models and factor analysis. London: Arnold; 1999.
  • Beauchaine TP, Waters E. Pseudotaxonicity in MAMBAC and MAXCOV analyses of rating-scale data: Turning continua into classes by manipulating observer’s expectations. Psychological Methods. 2003;8:3–15. [PubMed]
  • Bollen KA. Latent variables in psychology and the social sciences. Annual Review of Psychology. 2002;53:605–634.
  • Brownfield D, Sorenson AM. A latent structure analysis of delinquency. Journal of Quantitative Criminology. 1987;3:103–124.
  • Bucholz KK, Heath AC, Madden PAF. Transitions in drinking in adolescent females: Evidence from the Missouri Adolescent Female Twin Study. Alcoholism: Clinical and Experimental Research. 2000;24:914–923.
  • Bucholz KK, Heath AC, Reich T, Hesselbrock VM, Kramer JR, Nurnberger JI, Jr, Schuckit MA. Can we subtype alcoholism? A latent class analysis of data from relatives of alcoholics in a multicenter family study of alcoholism. Alcoholism: Clinical and Experimental Research. 1996;20:1462–1471.
  • Bucholz KK, Hesselbrock VM, Heath AC, Kramer JR, Schuckit MA. A latent class analysis of antisocial personality disorder symptom data from a multi-centre family study of alcoholism. Addiction. 2000;95:553–567. [PubMed]
  • Caspi A, McClay J, Moffitt T, Mill J, Martin J, Craig IW, et al. Role of genotype in the cycle of violence in maltreated children. Science. 2002 August 2;297:851–854. [PubMed]
  • Celeux G, Soromenho G. An entropy criterion for assessing the number of clusters in a mixture model. Journal of Classification. 1996;13:195–212.
  • Chung T, Martin CS. Classification and course of alcohol problems among adolescents in addictions treatment programs. Alcoholism: Clinical and Experimental Research. 2001;25:1734–1742.
  • Coccaro EF, Kavoussi RJ, Berman ME, Lish JD. Intermittent explosive disorder-revised: Development, reliability, and validity of research criteria. Comprehensive Psychiatry. 1998;39:368–376. [PubMed]
  • Cooke DJ, Kosson DS, Michie C. Psychopathy and ethnicity: Structural, item, and test generalizability of the Psychopathy Checklist—revised (PCL-R) in Caucasian and African American participants. Psychological Assessment. 2001;13:531–542. [PubMed]
  • Cooke DJ, Michie C. An item response theory analysis of the Hare Psychopathy Checklist—revised. Psychological Assessment. 1997;9:3–14.
  • Cooke DJ, Michie C. Psychopathy across cultures: North America and Scotland compared. Journal of Abnormal Psychology. 1999;108:58–68. [PubMed]
  • Cooke DJ, Michie C, Hart SD, Hare RD. Evaluating the screening version of the Hare Psychopathy Checklist—revised (PCL: SV): An item response theory analysis. Psychological Assessment. 1999;11:3–13.
  • Cornel M, Knibbe RA, van Zutphen WM, Drop MJ. Problem drinking in a general practice population: The construction of an interval scale for severity of problem drinking. Journal of Studies on Alcohol. 1994;55:466–470. [PubMed]
  • Donovan JE, Jessor R, Costa FM. Syndrome of problem behavior in adolescence: A replication. Journal of Consulting and Clinical Psychology. 1988;56:762–765. [PubMed]
  • Eaves LJ, Silberg JL, Hewitt JK, Rutter M, Meyer JM, Neale MC, Pickles A. Analyzing twin resemblance in multisymptom data: Genetic applications of a latent class model for symptoms of conduct disorder in juvenile boys. Behavior Genetics. 1993;23:5–19. [PubMed]
  • Faraone SV, Tsuang MT, Tsuang DW. Genetics of mental disorders. New York: Guilford Press; 1999.
  • Fergusson DM, Horwood LJ, Lynskey MT. The comorbidities of adolescent problem behaviors: A latent class model. Journal of Abnormal Child Psychology. 1994;22:339–354. [PubMed]
  • Fishler E, Grosmann M, Messer H. Detection of signals by information theoretic criteria: General asymptotic performance analysis. IEEE Transactions on Signal Processing. 2002;50:1027–1036.
  • Fu Q, Heath AC, Bucholz KK, Nelson E, Goldberg J, Lyons MJ, et al. Shared genetic risk of major depression, alcohol dependence, and marijuana dependence: Contribution of antisocial personality disorder in men. Archives of General Psychiatry. 2002;59:1125–1132. [PubMed]
  • Gorenstein EE, Newman JP. Disinhibitory psychopathology: A new perspective and a model for research. Psychological Review. 1980;87:303–315.
  • Gottesman II. Defining genetically informed phenotypes for the DSM–V. In: Sadler JZ, editor. Descriptions and prescriptions: Values, mental disorders, and the DSMs. Baltimore: Johns Hopkins University Press; 2002. pp. 291–300.
  • Grove WM, Eckert ED, Heston L, Bouchard TJ, Jr, Segal N, Lykken DT. Heritability of substance abuse and antisocial behavior: A study of monozygotic twins reared apart. Biological Psychiatry. 1990;27:1293–1304. [PubMed]
  • Haertel EH. Continuous and discrete latent structure models for item response data. Psychometrika. 1990;55:477–494.
  • Harris GT, Rice ME, Quinsey VL. Psychopathy as a taxon: Evidence that psychopaths are a discrete class. Journal of Consulting and Clinical Psychology. 1994;62:387–397. [PubMed]
  • Heath AC, Bucholz KK, Slutske WS, Madden PAF, Dinwiddie SH, Dunne MP, et al. The assessment of alcoholism in surveys of the general community: What are we measuring? Some insights from the Australian twin panel interview survey. International Review of Psychiatry. 1994;6:295–307.
  • Heinen T. Latent class and discrete latent trait models: Similarities and differences. Thousand Oaks, CA: Sage; 1996.
  • Hempel CG. Aspects of scientific explanation and other essays in the philosophy of science. New York: Free Press; 1965.
  • Hicks BM, Krueger RF, Iacono WG, McGue M, Patrick CJ. The family transmission and heritability of externalizing disorders: A twin-family study. Archives of General Psychiatry. 2004;61:922–928. [PubMed]
  • Holdcraft LC, Iacono WG. Cross-generational effects on gender differences in psychoactive drug abuse and dependence. Drug and Alcohol Dependence. 2004;74:147–158. [PubMed]
  • Iacono WG, Carlson SR, Malone SM, McGue M. P3 event-related potential amplitude and the risk for disinhibitory disorders in adolescent boys. Archives of General Psychiatry. 2002;59:750–757. [PubMed]
  • Iacono WG, Carlson SR, Taylor J, Elkins IJ, McGue M. Behavioral disinhibition and the development of substance-use disorders: Findings from the Minnesota Twin Family Study. Development and Psychopathology. 1999;11:869–900. [PubMed]
  • Iacono WG, Malone SM, McGue M. Substance use disorders, externalizing psychopathology, and P300 event-related potential amplitude. International Journal of Psychophysiology. 2003;48:147–178. [PubMed]
  • Iacono WG, McGue M. Minnesota Twin Family Study. Twin Research. 2002;5:482–487. [PubMed]
  • Jang KJ, Vernon PA, Livesley WJ. Personality disorder traits, family environment, and alcohol misuse: A multivariate behavioural genetic analysis. Addiction. 2000;95:873–888. [PubMed]
  • Kahler CW, Strong DR, Hayaki J, Ramsey SE, Brown RA. An item response analysis of the alcohol dependence scale in treatment-seeking alcoholics. Journal of Studies on Alcohol. 2003;64:127–136. [PubMed]
  • Kahler CW, Strong DR, Read JP, Palfai TP, Wood MD. Mapping the continuum of alcohol problems in college students: A Rasch model analysis. Psychology of Addictive Behaviors. 2004;18:322–333. [PubMed]
  • Kahler CW, Strong DR, Stuart GL, Moore TM, Ramsey SE. Item functioning of the alcohol dependence scale in a high-risk sample. Drug and Alcohol Dependence. 2003;72:183–192. [PubMed]
  • Kan CC, Breteler MHM, van der Ven AHGS, Zitman FG. An evaluation of DSM–III–R and ICD-10 benzodiazepine dependence criteria using Rasch modeling. Addiction. 1998;93:349–359. [PubMed]
  • Kendler KS. Toward a scientific psychiatric nosology: Strengths and limitations. Archives of General Psychiatry. 1990;47:969–973. [PubMed]
  • Kendler KS, Davis CG, Kessler RC. The familial aggregation of common psychiatric and substance use disorders in the National Comorbidity Survey: A family history study. British Journal of Psychiatry. 1997;170:541–548. [PubMed]
  • Kendler KS, Karkowski LM, Prescott CA, Pedersen NL. Latent class analysis of temperance board registrations in Swedish male-male twin pairs born 1902 to 1949: Searching for subtypes of alcoholism. Psychological Medicine. 1998;28:803–813. [PubMed]
  • Kendler KS, Prescott CA, Myers J, Neale MC. The structure of genetic and environmental risk factors for common psychiatric and substance use disorders in men and women. Archives of General Psychiatry. 2003;60:929–937. [PubMed]
  • Kirisci L, Vanyukov M, Dunn M, Tarter R. Item response theory modeling of substance use: An index based on 10 drug categories. Psychology of Addictive Behaviors. 2002;16:290–298. [PubMed]
  • Klein DN, Riso LP. Psychiatric disorders: Problems of boundaries and comorbidity. In: Costello CG, editor. Basic issues in psychopathology. New York: Guilford Press; 1993. pp. 19–66.
  • Kovac I, Merette C, Legault L, Dongier M, Palmour RM. the WHO/ISBRA Study on State and Trait Markers of Alcohol Use and Dependence Investigators. Evidence in an international sample of alcohol-dependent subjects of subgroups with specific symptom patterns of antisocial personality disorder. Alcoholism: Clinical and Experimental Research. 2002;26:1088–1096.
  • Krueger RF. The structure of common mental disorders. Archives of General Psychiatry. 1999;56:921–926. [PubMed]
  • Krueger RF, Caspi A, Moffitt TE, Silva PA. The structure and stability of common mental disorders (DSM–III–R): A longitudinal-epidemiological study. Journal of Abnormal Psychology. 1998;107:216–227. [PubMed]
  • Krueger RF, Chentsova-Dutton YE, Markon KE, Goldberg D, Ormel J. A cross-cultural study of the structure of comorbidity among common psychopathological syndromes in the general health care setting. Journal of Abnormal Psychology. 2003;112:437–447. [PubMed]
  • Krueger RF, Hicks BM, Patrick CJ, Carlson SR, Iacono WG, McGue M. Etiologic connections among substance dependence, antisocial behavior, and personality: Modeling the externalizing spectrum. Journal of Abnormal Psychology. 2002;111:411–424. [PubMed]
  • Krueger RF, McGue M, Iacono WG. The higher-order structure of common DSM mental disorders: Internalization, externalization, and their connections to personality. Personality and Individual Differences. 2001;30:1245–1259.
  • Krueger RF, Nichol PE, Hicks BM, Markon KE, Patrick CJ, Iacono WG, McGue M. Using latent trait modeling to conceptualize an alcohol problems continuum. Psychological Assessment. 2004;16:107–119. [PubMed]
  • Krueger RF, Piasecki TM. Toward a dimensional and psychometrically-informed approach to conceptualizing psychopathology. Behaviour Research and Therapy. 2002;40:485–499. [PubMed]
  • Krueger RF, Tackett JL. Personality and psychopathology: Working toward the bigger picture. Journal of Personality Disorders. 2003;17:109–128. [PubMed]
  • Lilienfeld SO. Comorbidity between and within childhood externalizing and internalizing disorders: Reflections and directions. Journal of Abnormal Child Psychology. 2003;31:285–291. [PubMed]
  • Lin TH, Dayton CM. Model selection information criteria for non-nested latent class models. Journal of Educational and Behavioral Statistics. 1997;22:249–264.
  • Madden PAF, Bucholz KK, Dinwiddie SH, Slutske WS, Bierut LJ, Statham DJ, et al. Nicotine withdrawal in women. Addiction. 1997;92:889–902. [PubMed]
  • Maser JD, Patterson T. Spectrum and nosology: Implications for. DSM–V Psychiatric Clinics of North America. 2002;25:855–885.
  • McCutcheon AL, Thomas G. Patterns of drug use among White institutionalized delinquents in Georgia: Evidence from a latent class analysis. Journal of Drug Education. 1995;25:61–71. [PubMed]
  • McElroy SL. Recognition and treatment of DSM–IV intermittent explosive disorder. Journal of Clinical Psychiatry. 1999;60:12–16. [PubMed]
  • Miller MB. Limitations of Meehl’s MAXCOV-HITMAX procedure. American Psychologist. 1996;51:554–556.
  • Mitchell CM, Plunkett M. The latent structure of substance use among American Indian adolescents: An example using categorical variables. American Journal of Community Psychology. 2000;28:105–125. [PubMed]
  • Moffitt TE. Adolescent-limited and life-course-persistent antisocial behavior: A developmental taxonomy. Psychological Review. 1993;100:674–701. [PubMed]
  • Morgenstern J, Langenbucher J, Labouvie E, Miller KJ. The comorbidity of alcoholism and personality disorders in a clinical population: Prevalence rates and relation to alcohol typology variables. Journal of Abnormal Psychology. 1997;106:74–84. [PubMed]
  • Muraki E. A generalized partial credit model: Application of an EM algorithm. Applied Psychological Measurement. 1992;16:159–177.
  • Mustanski BS, Viken RJ, Kaprio J, Rose RJ. Genetic influences on the association between personality risk factors and alcohol use and abuse. Journal of Abnormal Psychology. 2003;112:282–289. [PubMed]
  • Muthén BO. Psychometric evaluation of diagnostic criteria: Application to a two-dimensional model of alcohol abuse and dependence. Drug and Alcohol Dependence. 1996;41:101–112. [PubMed]
  • Beyond SEM, Beyond SEM. General latent variable modeling. Behaviormetrika. 2002;29:81–117.
  • Muthén B, Muthén LK. Integrating person-centered and variable-centered analyses: Growth mixture modeling with latent trajectory classes. Alcoholism: Clinical and Experimental Research. 2000;24:882–891.
  • Nathan PE, Langenbucher JW. Psychopathology: Description and classification. Annual Review of Psychology. 1999;50:79–107.
  • Nelson CB, Heath AC, Kessler RC. Temporal progression of alcohol dependence symptoms in the U.S. household population: Results from the National Comorbidity Survey. Journal of Consulting and Clinical Psychology. 1998;66:474–483. [PubMed]
  • Newcomb MD, Galaif ER, Locke TF. Substance use diagnoses within a community sample of adults: Distinction, comorbidity, and progression over time. Professional Psychology: Research and Practice. 2001;32:239–247.
  • Osgood DW, McMorris BJ, Potenza MT. Analyzing multiple-item measures of crime and deviance: I. Item response theory scaling. Journal of Quantitative Criminology. 2002;18:267–296.
  • Pedersen W, Skrondal A. Ecstasy and new patterns of drug use: A normal population study. Addiction. 1999;94:1695–1706. [PubMed]
  • Pickens RW, Svikis DS, McGue M, LaBuda MC. Common genetic mechanisms in alcohol, drug, and mental disorder comorbidity. Drug and Alcohol Dependence. 1995;39:129–138. [PubMed]
  • Raftery AE. Bayesian model selection in structural equation models. In: Bollen KA, Long JS, editors. Testing structural equation models. Newbury Park, CA: Sage; 1993. pp. 163–180.
  • Rissanen J. A universal prior for integers and estimation by minimum description length. Annals of Statistics. 1983;11:416–431.
  • Rissanen J. Stochastic complexity and statistical inquiry. Singapore: World Scientific; 1989.
  • Rissanen J. Fisher information and stochastic complexity. IEEE Transactions on Information Theory. 1996;42:40–47.
  • Rissanen J. Strong optimality of the normalized ML models as universal codes and information in data. IEEE Transactions on Information Theory. 2001;47:1712–1717.
  • Sanislow CA, Grilo CM, Morey LC, Bender DS, Skodol AE, Gunderson JG, et al. Confirmatory factor analysis of DSM–IV criteria for borderline personality disorder: Findings from the collaborative longitudinal personality disorders study. American Journal of Psychiatry. 2002;159:284–290. [PubMed]
  • Schafer J, Caetano R. The DSM–IV construct of cocaine dependence in a treatment sample of black, Mexican American, and White men. Psychological Assessment. 1996;8:304–311.
  • Schwarz G. Estimating the dimension of a model. Annals of Statistics. 1978;6:461–464.
  • Skilling TA, Harris GT, Rice ME, Quinsey VL. Identifying persistently antisocial offenders using the Hare Psychopathy Checklist and DSM antisocial personality disorder criteria. Psychological Assessment. 2002;14:27–38. [PubMed]
  • Skilling TA, Quinsey VL, Craig WM. Serious antisocial behavior in boys: Evidence of an underlying taxon. Criminal Justice and Behavior. 2001;28:450–470.
  • Skinner HA. Toward the integration of classification theory and methods. Journal of Abnormal Psychology. 1981;90:68–87. [PubMed]
  • Slutske WS, Eisen S, Xian H, True WR, Lyons MJ, Goldberg J, Tsuang M. A twin study of the association between pathological gambling and antisocial personality disorder. Journal of Abnormal Psychology. 2001;110:297–308. [PubMed]
  • Slutske WS, Heath AC, Dinwiddie SH, Madden PAF, Bucholz KK, Dunne MP, et al. Common genetic risk factors for conduct disorder and alcohol dependence. Journal of Abnormal Psychology. 1998;107:363–374. [PubMed]
  • Slutske WS, Heath AC, Madden PAF, Bucholz KK, Statham DJ, Martin NG. Personality and the genetic risk for alcohol dependence. Journal of Abnormal Psychology. 2002;111:124–133. [PubMed]
  • Tackett JL, Krueger RF, Sawyer MG, Graetz BW. Subfactors of DSM–IV conduct disorder: Evidence and connections with syndromes from the child behavior checklist. Journal of Abnormal Child Psychology. 2003;31:647–654. [PubMed]
  • True WR, Heath AC, Scherrer JF, Xian H, Lin N, Eisen SA, et al. Interrelationship of genetic and environmental influences on conduct disorder and alcohol and marijuana dependence symptoms. American Journal of Medical Genetics (Neuropsychiatric Genetics) 1999;88:391–397. [PubMed]
  • Tsuang MT, Stone WS, Faraone SV. Toward reformulating the diagnosis of schizophrenia. American Journal of Psychiatry. 2000;157:1041–1050. [PubMed]
  • Vermunt JK. LEM: Log-linear and event history analysis with missing data using the EM algorithm (WORC Paper No. 93.09.015/7) Tilburg, the Netherlands: Tilburg University; 1993.
  • Vermunt JK. LEM: A general program for the analysis of categorical data [Computer software] Tilburg, the Netherlands: Tilburg University; 1997.
  • Vermunt JK. The use of restricted latent class models for defining and testing nonparametric and parametric item response theory models. Applied Psychological Measurement. 2001;25:283–294.
  • Vollebergh WAM, Iedema J, Bijl RV, de Graaf R, Smit F, Ormel J. The structure and stability of common mental disorders: The NEMESIS study. Archives of General Psychiatry. 2001;58:597–603. [PubMed]
  • Waldman ID, Lilienfeld SO. Applications of taxometric methods to problems of comorbidity: Perspectives and challenges. Clinical Psychology: Science and Practice. 2001;8:520–527.
  • Waller NG, Meehl PE. Multivariate taxometric procedures: Distinguishing types from continua. Thousand Oaks, CA: Sage; 1998.
  • Watson D. Investigating the construct validity of the dissociative taxon: Stability analyses of normal and pathological dissociation. Journal of Abnormal Psychology. 2003;112:298–305. [PubMed]
  • Westen D, Heim AK, Morrison K, Patterson M, Campbell L. Simplifying diagnosis using a prototype-matching approach: Implications for the next edition of the DSM. In: Beutler LE, Malik ML, editors. Rethinking the DSM: A psychological perspective. Washington, DC: American Psychological Association; 2002. pp. 221–250.
  • Widiger TA. What can be learned from taxometric analyses. Clinical Psychology: Science and Practice. 2001;8:528–533.
  • Widiger TA, Clark LA. Toward DSM–V and the classification of psychopathology. Psychological Bulletin. 2000;126:946–963. [PubMed]
  • Wolf AW, Schubert DS, Patterson MB, Grande TP, Brocco KJ, Pendleton L. Associations among major psychiatric diagnoses. Journal of Consulting and Clinical Psychology. 1988;56:292–294. [PubMed]
  • Yang C. Finite mixture model selection with psychometric applications (Doctoral dissertation, University of California, Los Angeles 1996) Dissertation Abstracts International. 1998;59:3421B.
  • Young SE, Stallings MC, Corley RP, Krauter KS, Hewitt JK. Genetic and environmental influences on behavioral disinhibition. American Journal of Medical Genetics (Neuropsychiatric Genetics) 2000;96:684–695. [PubMed]
  • Zhang P. On the convergence of model selection criteria. Communications in Statistics—Theory and Methods. 1993;22:2765–2775.
  • Zuckerman M. Personality in the third dimension: A psychobiological approach. Personality and Individual Differences. 1989;10:391–418.