Search tips
Search criteria 


Logo of schbulOxford JournalsContact UsMy BasketMy AccountSchizophrenia BulletinAbout this JournalContact this JournalSubscriptionsCurrent IssueArchiveSearch
Schizophr Bull. 2011 November; 37(6): 1168–1178.
Published online 2010 March 31. doi:  10.1093/schbul/sbq024
PMCID: PMC3196945

Schizophrenia in the Internalizing-Externalizing Framework: A Third Dimension?


Background: Prior studies of common disorders in community-dwelling adults identified internalizing and externalizing spectra of mental illness. We investigated the placement of schizophrenia and schizotypal personality disorder in this framework and tested the validity of the resulting organization in a clinical population. Methods: The data came from the Suffolk County Mental Health Project cohort (N = 628), which consists of first-admission patients with psychosis recruited from inpatient units throughout Suffolk County, NY (72% response rate). The sample was reassessed multiple times over the following 10 years. Complete diagnostic data were available for 469 participants. Mental health professionals diagnosed 11 target conditions based on semistructured clinical interviews, review of medical records, and reports of significant others. Two validators were included: family history of schizophrenia and 10-year illness course. Results: Confirmatory factor analysis revealed that the The Diagnostic and Statistical Manual of Mental Disorders-IV grouping of conditions fit the data poorly. The best alternative classification consisted of three clusters: internalizing, externalizing, and schizophrenic. Both validators supported the coherence and distinctiveness of the schizophrenic cluster. Conclusions: We replicated internalizing and externalizing spectra in a clinical population, identified a schizophrenic spectrum, and provided initial evidence of its validity. These findings suggest that schizotypal personality disorder may be better placed with schizophrenia, antisocial conditions with substance use disorders, and major depression with anxiety disorders.

Keywords: nosology, classification, DSM, confirmatory factor analysis, validation


The Diagnostic and Statistical Manual of Mental Disorders (DSM), 4th edition,1 and the International Classification of Diseases (ICD), 10th edition,2 have been criticized for assigning psychiatric disorders to classes based on the apparent similarity of syndromes rather than empirical links among them.3 The DSM-IV/ICD-10 approach may misrepresent relations among disorders, thereby limiting the value of a formal diagnosis for clinical decision-making and for research.4,5 One alternative is to group disorders according to their patterns of covariation (ie, comorbidity). Indeed, it has been argued that analyses of comorbidity can reveal the natural classification of mental illness because conditions that belong to the same underlying class tend to co-occur more frequently.3,6,7 The resulting quantitative organization could aid research on genetic and neurobiological underpinnings of psychopathology because clustering of syndromes is likely to reflect shared etiologies.3,8,9 This system could also facilitate the development of interventions to target risk factors and pathological processes common to related disorders.10

Such quantitative classification can be developed with the aid of factor analysis because this method examines associations among variables and group-related (ie, comorbid) variables together. Moreover, confirmatory factor analysis is able to test competing arrangements of disorders by comparing their fit to the data; that is, how well different models explain the observed comorbidity patterns.

In fact, there is a long tradition of factor analytically derived classification systems, especially in child psychiatry.1113 This work led to the identification of 2 fundamental dimensions of mental illness, the internalizing and externalizing spectra. Recent studies extended these ideas to adult samples and compared alternative classifications of psychiatric diagnoses in large community surveys.7,14 Krueger and Markon15 performed a meta-analysis of this literature and applied confirmatory factor analysis to pooled data on 11 common mental disorders and replicated the 2 general dimensions. Internalizing conditions included all depressive and anxiety disorders. The externalizing cluster was composed of substance use disorders, conduct disorder, and adult antisocial behavior. This organization has been replicated across many cultures.16

Initial validation studies produced encouraging evidence in support of the 2 clusters. For instance, twin studies suggest that the genetic architecture of common disorders follows the internalizing-externalizing pattern.1719 Two recent reviews evaluated data on 11 major validators20,21; the reviews concluded that disorders within each cluster exhibit many notable similarities on these validators and argued that DSM-V and ICD-11 should recognize these spectra.

In sum, the internalizing and externalizing clusters are now reasonably established. However, this 2-spectrum framework is far from comprehensive. Factor analytic studies of adults have been conducted only with community-dwelling participants and were limited to conditions that are relatively common in the general population. Studies of severely ill populations can extend the emerging quantitative classification to less common disorders. Most notably, prior investigations have not considered schizophrenia or schizotypal personality disorder, and it is unknown how they fit within this framework. There is some indirect evidence to suggest a link between these conditions and externalizing psychopathology.22,23 Other data argue for placing them in the internalizing cluster.24 Still other studies suggest that schizophrenia and schizotypal personality disorder define a separate spectrum.25,26 These alternatives have not been tested rigorously in a head-to-head comparison.

Factor analyses of schizophrenia and schizotypal personality disorder have not been attempted in large part because a hierarchical exclusion rule precludes the diagnosis of schizotypal personality disorder in individuals who experienced frank psychosis, which creates an artificial disjunction between these conditions. In fact, this rule is the main obstacle to classifying these disorders together.27 Thus, in the present study, we sought to relax the hierarchical rules and investigate the natural links among target conditions. This approach is consistent with previous factor analytic studies.15 Indeed, meaningful modeling is not possible unless hierarchical rules are removed because these definitional assumptions would dictate the results otherwise. By relaxing these rules, we were able to obtain some of the first factor analytic evidence regarding the placement of schizophrenia and schizotypal personality disorder in the internalizing-externalizing framework.

Factor analysis is only a part of the larger strategy for establishing a valid classification system.10 Carpenter et al27 reviewed data on the validity of a distinct cluster defined primarily by schizophrenia and schizotypal personality disorder. They found the 2 conditions to be similar with regard to family history, temperamental antecedents, certain biomarkers, and cognitive impairment. The existing studies simply reported correlates of schizotypal personality disorder and in some cases compared them with correlates of schizophrenia,28,29 but the proposed schizophrenic cluster has not been directly validated. Furthermore, validation by illness course has been relatively neglected in the literature. To help address these gaps, the present investigation examined the validity of alternative classifications with regard to long-term illness course as well as family history, another key validator.30

Hence, the aim of the present study was to extend classification research by evaluating the placement of schizophrenia and schizotypal personality syndromes in the emerging quantitative nosology and by providing initial evidence about the validity of the resulting organization. With regard to the latter, we examined data on family history of schizophrenia and long-term illness course. To achieve study goals, we employed an epidemiological cohort of inpatients whose initial symptoms included psychosis. This cohort was chosen to ensure sufficient prevalence of the target conditions and to take advantage of rigorous diagnoses ascertained by mental health professional as well as detailed data on 2 important validators. The present study also tested the generalizability of the externalizing-internalizing framework to a severely ill psychiatric population and thus complimented past research, which had been conducted in community-dwelling adults.


Sample and Procedure

Data for this study came from a county-wide cohort with first-admission psychosis, described in detail elsewhere.3133 The participants were recruited from all 12 psychiatric inpatient units of a large county (Suffolk County, NY; population 1.3 million) between 1989 and 1995 for a naturalistic longitudinal investigation of illness course (Suffolk County Mental Health Project; SCMHP). Participating facilities represent a wide range of settings, including state hospitals, private clinics, a Veterans Administration hospital, and an academic medical center. The SCMHP is one of the few North American samples broadly representative of incident psychotic disorders. The inclusion criteria were age 15–60, first admission either concurrent or during the 6 months prior to index admission, clinical evidence of psychosis, ability to understand the assessment procedures in English, and capacity to provide written informed consent. The procedures for obtaining informed consent were approved annually by the Institutional Review Boards at Stony Brook University and all hospitals where respondents were recruited. For participants aged 15–17, written consent of parents was also required. The response rate was 72%. Respondents completed face-to-face interviews at baseline, month 6, year 2, year 4, and year 10.

Overall, 675 respondents completed baseline interview, but 47 were found to be ineligible, primarily because it became clear that they never suffered from psychosis. Of 628 eligible participants, 469 had complete data at 6-month follow-up and comprised the analysis sample. This follow-up was required because several disorders were not assessed at baseline. Of the 159 excluded participants, 2 died before the 6-month point, 30 could not be traced, 62 declined to be interviewed, and 65 provided incomplete data. There were no differences between the analysis sample (N = 469) and excluded participants (N = 159) on any of the baseline diagnoses described below except for obsessive-compulsive syndrome, which was less prevalent in the excluded group (6% vs 13%, P < .01). There also were no differences on the validators. Hence, attrition was largely unrelated to study variables and likely had little systematic impact on the results. Data on illness course beyond the 6-month point were available for the 97% of the analysis sample (453/469), and family history data were available for everyone.


Diagnostic ratings were made using the Structured Clinical Interview for DSM-III-R (SCID)34 and the Structured Interview for Schizotypy (SIS).35 These ratings were made by study interviewers—all trained master’s level mental health professionals—based on interviews with participants, reports of significant others, and reviews of medical records. The SCID was administered both at baseline and at month 6. Baseline interview had lifetime time frame, and 6-month assessment covered the interval since baseline. The SIS was administered only at month 6 and assessed lifetime symptoms. Interrater reliability of diagnostic ratings was good to excellent with kappas ranging from 0.68 to 0.88.32,36

It was necessary to relax the hierarchical exclusion rules imbedded in the DSM because they prohibit certain combinations of diagnoses a priori and would dictate the structure if employed. Hence, our focus was on syndromes rather than on full diagnoses. We considered the following 11 syndromes: major depressive episode, panic attack, social anxiety, obsessive-compulsive symptoms, alcohol use disorder, cannabis use disorder, other drug use disorder, conduct problems, antisocial personality, schizotypal personality, and schizophrenia syndrome (table 1). Other anxiety disorders could not be analyzed because exclusion rules precluded their assessment in individuals with active psychotic disorders, and thus, even syndrome-level information was not available for many participants. All variables were scored dichotomously: 1 = had the syndrome at any point from childhood through month 6 of the study and 0 = did not have the syndrome.

Table 1.
Assessment and Prevalence of Target Syndromes

The SCID provided ratings of major depressive episode, panic attack, obsessive-compulsive symptoms, alcohol use disorder, cannabis use disorder, and other drug use disorder (table 1). The latter 3 variables were defined as presence of dependence or abuse according to DSM-III-R criteria.37 Dependence and abuse were combined into a single category because of the hierarchical exclusion between them.

The SIS provided ratings of conduct problems, antisocial personality, social anxiety, and schizotypal personality. The SIS is a comprehensive measure of schizotypal personality disorder and related characteristics developed to provide a reliable and comprehensive assessment of these pathological traits.38 Each characteristic is probed with a series of questions, which are then summarized by a global 7-point rating (1 = marked to 7 = absent). We dichotomized each rating, with score of 4 or less considered positive. This cutoff ensured that positive scores indicated at least moderate severity, and it fits well with characteristics of our data. The conduct problems variable was thus derived from the summary rating of antisocial behavior in childhood and adolescence. The antisocial personality variable was derived from the summary rating of antisocial traits in adulthood. The SIS assesses the 9 symptoms of schizotypal personality disorder individually. However, the ninth rating taps general social anxiety (eg, discomfort in social situations, worries about possible scrutiny, and concern about negative evaluation) rather than the specific form of anxiety associated with schizotypal personality disorder, namely “excessive social anxiety that does not diminish with familiarity and tends to be associated with paranoid fears rather than negative judgments about self.”1(p701) Hence, the ninth summary rating was used to indicate social anxiety. The schizotypal personality variable was based on the other 8 ratings (0 = less than 5 positive ratings, 1 = 5–8 positive ratings).

Schizophrenia syndrome was derived from symptom ratings. Schizophrenia diagnosis was assessed with the SCID. However, due to a hierarchical exclusion rule, diagnostic criteria were not rated when mood disturbance was concurrent with psychotic symptoms. This rule affected 39.4% of the sample, and it was not possible to determine if they met other criteria for schizophrenia. Symptom ratings were available for the whole sample, however, irrespective of mood disturbance. These symptoms were assessed with the Psychosis Screening module of the SCID, the Brief Psychiatric Rating Scale (BPRS),39 and the Scale for the Assessment of Negative Symptoms (SANS).40 We selected 4 interview ratings that had the strongest associations with the schizophrenia diagnosis and dichotomized them at the most discriminating cut points: auditory hallucinations (rated 3 = “above threshold” on the SCID), bizarre delusions (rated at least 2 = “subthreshold” on the SCID), flat affect (corresponding global rating of at least 2 = “mild” on the SANS), and overall dysfunction (illness severity score of at least 4 = “moderately ill” on the BPRS). The presence of 2 of these 4 markers was indicative of the SCID schizophrenia diagnosis, with the positive predictive value of 80% and the negative predictive value of 76% in 284 cases not affected by the mood disturbance rule.

Two validators were examined. Family history of schizophrenia was assessed with the Family History RDC41 interview administered to the participant and a significant other (usually a family member). Relevant items asked whether any first-degree relative was ever diagnosed with schizophrenia or “heard voices, saw things that were not there, or thought others were against them.” In addition, family history was coded as positive if the medical record clearly described schizophrenia diagnosis in a first-degree relative. Long-term illness course was operationalized with the consensus rating of study psychiatrists on the World Health Organization Course of Illness Scale.42 This measure is an 8-point rating designed to summarize the course. It ranges from “single episode with full remission of symptoms” to “continuous illness.” For this analysis, the scale was dichotomized into continuous illness vs any remission. For this variable, we used the last available course rating, which for 79% was made at year 10, for 15% at year 4, for 3% at year 2, and 3% were missing all follow-ups.

Data Analysis

Four classifications of mental disorders were compared using confirmatory factor analysis, namely 5-factor, 3-factor, and two 2-factor models. The first was based on the DSM and grouped variables into 5 clusters: anxiety (panic attack, obsessive-compulsive symptoms, and social anxiety), mood (major depressive episode), schizophrenia (schizophrenia syndrome), substance use (alcohol, cannabis, and other drug use disorders), and personality disorder (schizotypal personality, antisocial personality, and conduct problems). The conduct problems variable was assigned to the latter group because it is a part of the antisocial personality disorder trajectory. The 2-factor A model arranged diagnoses into internalizing (mood and anxiety syndromes) and externalizing (personality and substance use conditions) clusters. Schizophrenia syndrome and schizotypal personality were assigned to the externalizing group as suggested by some recent evidence.22,23 The 2-factor B model located these 2 conditions on the internalizing spectrum instead following Verona et al24 The 3-factor model assigned the 2 syndromes to a separate schizophrenic cluster—in light of evidence suggesting a distinct spectrum26,27—and all other conditions remained in their original positions in the internalizing and externalizing clusters. The 2-factor A and the 3-factor models also included a correlated error term between social anxiety and schizotypal personality to reflect shared method variance because social anxiety was measured in the context of schizotypal personality. The correlated error was unnecessary in the 2-factor B model because the 2 variables were allowed to load on the same factor, which captured shared variance.

After the best-fitting model was identified, validator variables were added to it in order to examine relations between spectra and validators. First, validators were allowed to correlate with all factors. Next, we constrained nonsignificant associations to zero. The models were tested in Mplus version 5 using MLR (robust maximum likelihood) estimator, which can handle nonnormal distributions.43

In comparing these models, we considered 7 fit indices: the overall model chi-square test statistic, the Comparative Fit Index (CFI), the Tucker-Lewis Index (TLI), the root-mean-square error of approximation (RMSEA), the Akaike Information Criterion (AIC), the Bayesian Information Criterion (BIC), and the sample size–adjusted BIC (ABIC).4446 Although there are no strict criteria for evaluating these fit indices, conventional guidelines suggest that a fit is acceptable if (a) CFI and TLI are .90 or greater and (b) RMSEA is .08 or less.46 In interpreting these results, we considered TLI and CFI values of .90 or greater to indicate an adequate fit and values of .95 or greater to represent an excellent fit. RMSEA values of less than .08 were viewed as reflecting an adequate fit, with values less than .06 representing an excellent fit. There are no absolute cutoffs on AIC, BIC, and ABIC, but these indices can be used to compare models, with lower values representing better fit.47,48 Conventional guidelines suggest that a difference of <6 is small, 6–10 is substantial, and >10 is very substantial.48


Alternative Classifications of Syndromes

All target syndromes were sufficiently common in our sample to be included in the analyses as their prevalences ranged from 15.8% to 57.3% (table 1). Tetrachoric correlations (table 2, bottom triangle) revealed a pattern of strong associations among externalizing conditions (antisocial personality, conduct problems, and substance use disorders). Correlations among other variables were much lower, indicating considerable heterogeneity and likely multidimensionality. Anxiety and depressive conditions were moderately correlated, consistent with arguments for the internalizing cluster. The link between schizophrenia syndrome and schizotypal personality was fairly strong (r = .45). This association also was specific because it was the strongest correlation for both variables.

Table 2.
Tetrachoric Correlations Among Syndromes (Bottom Triangle) and With Full Schizophrenia Diagnosis Instead of Schizophrenia Syndrome (Top Triangle)a

To evaluate the effect of employing schizophrenia syndrome rather than full schizophrenia diagnosis in these analyses, we also computed tetrachoric correlations using the SCID schizophrenia diagnosis (table 2, top triangle). The correlations remained the same or very similar except that the full diagnosis had a strong negative association with major depressive episode (r = −.35 vs .01 for the syndrome), and its link with panic attack was more negative (r = −.14 vs −.03). These results indicate the distorting effect of the mood disturbance exclusion rule embedded in the schizophrenia diagnosis. This certainly is problematic for studying relations among target conditions. It was reassuring that the syndrome mirrored full diagnosis with regard to all other correlations. Hence, subsequent analyses used the schizophrenia syndrome variable.

Confirmatory factor analysis indicated that the DSM organization fits the data poorly and was the worst model by far on all fit indices (table 3). Both 2-factor models represented the data adequately, with fit indices in the acceptable range. In fact, the arrangement that placed schizophrenia and schizotypal syndromes on the internalizing spectrum (2-factor B) was on the border between adequate and excellent fit. The 3-factor model, however, performed even better with the CFI, TLI, and RMSEA indices, suggesting excellent fit. Moreover, the 3-factor organization was highly superior to other models on the AIC and ABIC, although it was not different from the 2-factor B model on the BIC. Hence, the 3-factor organization was selected as the best representation of the data (figure 1, top). All disorders had substantial loadings in this model and thus appear to have been appropriately assigned. Correlations among the factors were modest and positive.

Table 3.
Fit Indices for Confirmatory Factor Analysesa
Fig. 1.
Best-fitting Models. The top diagram shows the best-fitting (3-factor) model of relations among syndromes. The bottom diagram shows associations between the resulting factor structure and validators (nonsignificant paths are constrained to zero).

Validation of the Schizophrenic Cluster

To evaluate the validity of this organization, we added family history and illness course variables to the 3-factor model. Validators were allowed to correlate with each factor, which enabled us to examine specificity of these effects (ie, whether validators were significantly linked to schizophrenic but not to other factors). The resulting model showed excellent fit on the CFI and RMSEA and very good fit on the TLI (table 3). Only the schizophrenic factor correlated with the validators (both P < .01); all other correlations were small (rs < .07) and nonsignificant. Hence, we constrained the nonsignificant relations to zero. This model had excellent fit on all indices. The schizophrenic factor correlated moderately with family history and very strongly with illness course (figure 1, bottom), which is clear and specific evidence of validity of this new cluster. The 2 validators were moderately associated with each other (r = .27).

To investigate validity further, we aggregated conditions into the 3 clusters, which were scored as dichotomous (not mutually exclusive) variables so that having even 1 relevant syndrome qualified for membership in a given cluster. Thus, 71% of the sample had internalizing, 60% schizophrenic, and 57% externalizing psychopathology. We then related the 11 syndromes and the 3 clusters to family history of schizophrenia. Among syndromes, only the schizophrenia variable showed a significant association with the validator; however, schizotypal personality was a close second, and all other effects were appreciably weaker (table 4). This pattern argues for kinship between schizophrenia and schizotypal syndromes and their distinctiveness from the other 9 conditions. On the level of clusters, only the schizophrenic group was significantly related to family history. This effect was even stronger than the association observed for schizophrenia syndrome alone.

Table 4.
Associations of Diagnostic Variables with Validatorsa

Next, we related the syndromes and clusters to the long-term course validator. Schizophrenia syndrome was the strongest predictor of poor course, schizotypal personality was somewhat weaker, and other syndromes were not predictive except for social anxiety, which showed a modest effect that was barely significant (table 4). These data again set schizophrenia and schizotypal syndromes apart from the other variables and suggested the existence of a coherent schizophrenic cluster. In analysis of the 3 spectra, only the schizophrenic cluster was significant; it showed a very large effect that was stronger than the predictive power of schizophrenia syndrome.


This study extended research on the quantitative reorganization of psychiatric nosology in several ways. First, it confirmed the basic internalizing-externalizing framework in a severely ill cohort. These spectra appear to be robust across ages and cultures, and we now observed them in an inpatient sample. Second, we sought to incorporate schizophrenia and schizotypal personality disorder in the quantitative framework and found that these conditions do not belong to the internalizing or externalizing domains but define a separate spectrum. Third, we examined the validity of the resulting cluster with regard to family history and long-term course. The schizophrenic spectrum was coherent and clearly distinct from the internalizing and externalizing conditions on both validators. Our results agree well with prior research that argued for a separate schizophrenic cluster2527 and provide some of the first direct evidence in support of this proposal. They also confirm the decision of framers of ICD-10 to place schizotypal disorder with schizophrenia rather than with personality disorders.

Our findings need to be considered against the limitations of the present investigation. Although broader than many prior studies, our analysis was still limited to only 11 syndromes. Future investigations need to include many more conditions to explicate a comprehensive quantitative classification. Another limitation is that only a partial assessment of anxiety disorders was possible, which prevented us from testing distinctions within the internalizing cluster (ie, fear and distress subclusters). Also, the syndromes were based on DSM-III-R rather than on DSM-IV criteria. Fortunately, the majority of syndromes considered were identical in the 2 manuals, and others showed only minor differences. In addition, schizotypal personality was assessed after onset of psychosis. Although personality ratings were made by experienced interviewers under explicit instructions of capturing long-standing characteristics and were based on information from multiple sources collected soon after first admission, the assessment may have been affected by behavioral changes that occurred since illness onset. Furthermore, long-term course and family history ratings were made by interviewers who were not blind to diagnoses. Validity data were usually collected many years after the 6-month wave, and syndromes used in this study are not full DSM diagnoses that were accessible to interviewers, but some confounding may have occurred nevertheless. Furthermore, the present study was focused on a specific population (individuals with a history of hospitalization for psychosis), and a replication in a less selective patient sample is necessary. On balance, the nature of this cohort is a strength because employment of this epidemiological first-admission sample gave us a rare opportunity to study the diagnostic picture that was free from distortions associated with chronic illness and long history of medication use. Moreover, the patients were drawn from a broad range of inpatient facilities that covered an entire county, thus reducing selection biases. Furthermore, this cohort was followed for 10 years with multiple interim assessments, which allowed us to examine the predictive validity of the proposed clusters.

Despite the limitations noted above, the present study provided important new information on the placement of schizophrenia and schizotypal personality disorder in the quantitative framework. This knowledge can be valuable because the emerging classification promises to improve our understanding of psychopathology in several respects. First, factor analytically derived spectra appear to be directly tied to common genetic liabilities and can facilitate etiologic research. Twin studies indicated that the internalizing and externalizing spectra capture much of the genetic vulnerability to common mental disorders.1719 Similarly, there is evidence for a distinct and coherent genetic factor underlying the schizophrenic cluster.49,50 In fact, molecular genetic studies are beginning to uncover specific genes contributing to the 2 established spectra,5153 and we hope that our findings will stimulate parallel research on the schizophrenic spectrum.

Second, the clusters may reflect certain fundamental neurobiological abnormalities. For instance, the internalizing spectrum appears to be linked to amygdala hyperactivity because it has been implicated in both anxiety and depression.5456 The externalizing domain may be associated with functioning of ventromedial prefrontal cortex, which seems to be involved both in substance abuse and in antisocial behavior.5759 The schizophrenic cluster may be linked with gray matter volume deficits, especially in the temporal lobe and thalamus because these abnormalities have been observed both in schizophrenia and in schizotypal personality disorders.28,29 Delineation of this cluster’s boundaries may lead to more powerful designs for investigating pathophysiology of psychosis.29

Third, the quantitative organization may explain and predict efficacy of psychotropic medications. For example, selective serotonin reuptake inhibitors had been regarded as antidepressants but were found to be efficacious in treating anxiety disorders as well.60 In contrast, their efficacy has not been established in schizophrenia.27,60 These observations are consistent with the distinction between internalizing and schizophrenic spectra, which supports the contention that the quantitative organization can inform intervention research. Finally, the proposed arrangement arguably increases the coherence of the classification system and facilitates differential diagnosis by grouping disorders on the basis of empirical association rather than surface similarity.

It is important to acknowledge that the present study evaluated only 2 candidates for membership in the schizophrenic cluster. A number of other candidates have been suggested by factor analytic investigations of psychiatric symptoms. In the most comprehensive study of this kind, Markon61 analyzed various Axis I and II symptoms and found a spectrum defined by features of schizotypal, schizoid, paranoid, and obsessive-compulsive personality disorders as well as frank psychosis. Markon's results largely agree with factor analytic studies of Axis II conditions. A quantitative review of this literature concluded that schizotypal, schizoid, and paranoid personality disorders cluster together, although obsessive-compulsive personality disorder belongs to a different spectrum.62 Other research identified strong links between schizotypal personality and dissociation.63,64 There is also evidence that obsessive-compulsive disorder and bipolar disorder are related to schizophrenic disorders, but the supporting evidence is rather mixed.27,61,63 In the present study, obsessive-compulsive disorder fell in the internalizing rather than the schizophrenic cluster, which is consistent with prior factor analytic investigations.61,65 We did not evaluate bipolar disorder, but recent literature reviews concluded that support for its inclusion in the schizophrenic spectrum is insufficient.27 In sum, the leading candidates for addition to the schizophrenic cluster are dissociative disorders as well as schizoid and paranoid personality disorders, and they need to be evaluated in future studies.

Several other observations warrant comment. Importantly, the DSM-IV organization failed to represent the data adequately, despite being the most elaborate of the models considered. Indeed, quantitative organizations were clearly superior to it. It also was apparent that the existing nosology cannot account for pronounced similarities between schizophrenia and schizotypal syndromes with regard to family history and illness course. Although these conditions are allocated to different axes presently, we found that they were much more similar to each other than either was to other Axis I and Axis II variables considered. This observation is consistent with prior research on familial aggregation of schizophrenia and schizotypal personality disorder25,27 and extends it by validating the schizophrenic cluster directly. Also, our study provided novel evidence for the validity of this cluster with regard to long-term illness course. Finally, we should note that when the hierarchical exclusion for mood disturbance was relaxed, the correlation between schizophrenia syndrome and major depression was essentially zero (table 2), which indicates that patients with schizophrenia syndrome were no less likely to experience lifetime depression than participants with other severe illnesses. Thus, our data suggest that concomitant depression should not be an exclusion for a schizophrenia diagnosis. This conclusion is consistent with prior research, which argued that these 2 conditions are not mutually exclusive.6668

In conclusion, our study provided early direct evidence for a separate schizophrenic cluster. It also confirmed the internalizing and externalizing spectra in a new population. These findings underscore the need to reorganize the diagnostic system, especially because the DSM model fit the data very poorly and was not aligned with patterns of family history and long-term course. Specifically, our findings and results of prior research imply that unipolar depression and some anxiety disorders should be placed together in the internalizing cluster.10,69 Schizotypal and antisocial personality disorders may be better placed on Axis I with schizophrenia and substance use disorders, respectively. In fact, among other proposals, the DSM-V Personality Disorders Work Group is considering relocation of personality disorders to Axis I.70 It also appears that conduct disorder would fit well with antisocial personality and substance use disorders. Of note, ICD-10 already places schizophrenia and schizotypal disorder together, but it still splits internalizing conditions and externalizing conditions across different rubrics. Beyond these initial recommendations, the outline of the quantitative classification remains rough, but it is almost certain that development of nosology that captures natural patterns of mental illness will greatly enhance the validity and practical utility of psychiatric diagnosis. The present study brings us 1 step closer to a truly empirical classification of mental illness.


National Institutes of Health (MH-44801 to E.J.B.); Stony Brook University (Clinical Research Scholar Award to R.K.).


The authors thank the mental health professionals in Suffolk County, the project psychiatrists and staff, and most of all, the study participants and their families and friends. Special thanks to Camilo Ruggero, Kristian Markon, and David Watson who commented on drafts of this manuscript. Conflict of interest: None.


1. American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders. 4th ed, Text Revision. Washington, DC: American Psychiatric Association; 2000.
2. World Health Organization. The ICD-10 Classification of Mental Disorders. Geneva, Switzerland: World Health Organization; 1992.
3. Watson D. Rethinking the mood and anxiety disorders: a quantitative hierarchical model for DSM-V. J Abnorm Psychol. 2005;114:522–536. [PubMed]
4. Beutler LE, Malik ML. Diagnostic and Statistical Manual of Mental Disorders. Washington, DC: American Psychological Association; 2002.
5. Gould TD, Gottesman II. Psychiatric endophenotypes and the development of valid animal models. Genes Brain Behavr. 2006;5:113–119. [PubMed]
6. Brown TA, Barlow DH. Comorbidity among anxiety disorders: implications for treatment and DSM-IV. J Consult Clin Psychol. 1992;60:835–844. [PubMed]
7. Krueger RF. The structure of common mental disorders. Arch Gen Psychiatry. 1999;56:921–926. [PubMed]
8. Andrews G, Slade T, Issakidis C. Deconstructing current comorbidity: data from the Australian National Survey of Mental Health and Well-being. Br J Psychiatry. 2002;181:306–314. [PubMed]
9. Markon KE, Krueger RF. Categorical and continuous models of liability to externalizing disorders: a direct comparison in NESARC. Arch Gen Psychiatry. 2005;62:1352–1359. [PMC free article] [PubMed]
10. Andrews G, Goldberg DP, Krueger RF, et al. Exploring the feasibility of a meta-structure for DSM-V and ICD-11: could it improve utility and validity? Psychol Med. 2009;39:1993–2000. [PubMed]
11. Achenbach TM, Rescorla LA. Manual for the ASEBA School-Age Forms & Profiles. Burlington, VT: University of Vermont, Research Center for Children, Youth and Families; 2001.
12. Lahey BB, Applegate B, Waldman ID, Loft JD, Hankin BL, Rick J. The structure of child and adolescent psychopathology: generating new hypotheses. J Abnorm Psychol. 2004;113:358–385. [PubMed]
13. Lahey BB, Rathouz PJ, Van Hulle C, et al. Testing structural models of DSM-IV symptoms of common forms of child and adolescent psychopathology. J Abnorm Psychol. 2008;36:187–206. [PubMed]
14. Vollebergh WAM, Iedema J, Bijl RV, de Graaf R, Smit F, Ormel J. The structure and stability of common mental disorders: the NEMESIS study. Arch Gen Psychiatry. 2001;58:597–603. [PubMed]
15. Krueger RF, Markon KE. Reinterpreting comorbidity: a model-based approach to understanding and classifying psychopathology. Annu Rev Clin Psychol. 2006;2:111–133. [PMC free article] [PubMed]
16. Krueger RF, Chentsova-Dutton YE, Markon KE, Goldberg D, Ormel J. A cross-cultural study of the structure of comorbidity among common psychopathological syndromes in the general health care setting. J Abnorm Psychol. 2003;112:437–447. [PubMed]
17. Hicks BM, Krueger RF, Iacono WG, McGue M, Patrick CJ. Family transmission and heritability of externalizing disorders: a twin-family study. Arch Gen Psychiatry. 2004;61:922–928. [PubMed]
18. Kendler KS, Prescott CA, Myers J, Neale MC. The structure of genetic and environmental risk factors for common psychiatric and substance use disorders in men and women. Arch Gen Psychiatry. 2003;60:929–937. [PubMed]
19. Krueger RF, Hicks BM, Patrick CJ, Carlson SR, Iacono WG, McGue M. Etiologic connections among substance dependence, antisocial behavior, and personality: modeling the externalizing spectrum. J Abnorm Psychol. 2002;111:411–424. [PubMed]
20. Goldberg DP, Krueger RF, Andrews G, Hobbs MJ. Emotional disorders: cluster 4 of the proposed meta-structure for DSM-V and ICD-11. Psychol Med. 2009;39:2043–2059. [PubMed]
21. Krueger RF, South SC. Externalizing disorders: cluster 5 of the proposed meta-structure for DSM-V and ICD-11. Psychol Med. 2009;39:2061–2070. [PubMed]
22. Keshavan M, Montrose DM, Rajarethinam R, Diwadkar V, Prasad K, Sweeney JA. Psychopathology among offspring of parents with schizophrenia: relationship to premorbid impairments. Schizophr Res. 2008;103:114–120. [PMC free article] [PubMed]
23. Rubino IA, Frank E, Croce NR, Pozzi D, Lanza di Scalea T, Siracusano A. A comparative study of axis I antecedents before age 18 of unipolar depression, bipolar disorder and schizophrenia. Psychopathology. 2009;42:325–332. [PubMed]
24. Verona E, Sachs-Ericsson N, Joiner TE., Jr Suicide attempts associated with externalizing psychopathology in an epidemiological sample. Am J Psychiatry. 2004;161:444–451. [PubMed]
25. Kendler KS, Gardner CO. The risk for psychiatric disorders in relatives of schizophrenic and control probands: a comparison of three independent studies. Psychol Med. 1997;27:411–419. [PubMed]
26. Wolf AW, Schubert DSP, Patterson MB, Grande TP, Brocco KJ, Pendleton L. Associations among major psychiatric diagnoses. J Consult Clin Psychol. 1988;56:292–294. [PubMed]
27. Carpenter WT, Jr., Bustillo JR, Thaker GK, van Os J, Krueger RF, Green MJ. The psychoses: cluster 3 of the proposed meta-structure for DSM-V and ICD-11. Psychol Med. 2009;39:2025–2042. [PubMed]
28. Raine A. Schizotypal personality: neurodevelopmental and psychosocial trajectories. Annu Rev Clin Psychol. 2006;2:291–326. [PubMed]
29. Siever LJ, Davis KL. The pathophysiology of schizophrenia disorders: perspectives from the spectrum. Am J Psychiatry. 2004;161:398–413. [PubMed]
30. Robins E, Guze SB. Establishment of diagnostic validity in psychiatric illness: its application to schizophrenia. Am J Psychiatry. 1970;126:983–987. [PubMed]
31. Bromet EJ, Bushra N, Fochtmann LJ, Carlson GA, Tanenberg-Karant M. Long-term diagnostic stability and outcome in recent first-episode cohort studies of schizophrenia. Schizophr Bull. 2005;31:639–649. [PubMed]
32. Bromet EJ, Schwartz JE, Fennig S, et al. The epidemiology of psychosis: the Suffolk County Mental Health Project. Schizophr Bull. 1992;18:243–255. [PubMed]
33. Schwartz JE, Fennig S, Tanenberg-Karant M, et al. Congruence of diagnoses 2 years after a first-admission diagnosis of psychosis. Arch Gen Psychiatry. 2000;57:593–600. [PubMed]
34. Spitzer RL, Williams J, Gibbon M, First MB. The Structured Clinical Interview for DSM-III-R (SCID). I: history, rational, and description. Arch Gen Psychiatry. 1992;49:624–629. [PubMed]
35. Kendler KS, Lieberman JA, Walsh D. The Structured Interview for Schizotypy (SIS): a preliminary report. Schizophr Bull. 1989;15:559–571. [PubMed]
36. Brown AS, Susser ES, Jandorf L, Bromet EJ. Social class of origin and cardinal symptoms of schizophrenic disorders over the early illness course. Soc Psychiatry Psychiatr Epidemiol. 2000;35:53–60. [PubMed]
37. American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders. 3rd ed, revised. Washington, DC: American Psychiatric Association; 1987.
38. Kendler KS, McGuire M, Gruenberg AM, Walsh D. Schizotypal symptoms and signs in the Roscommon Family Study. Arch Gen Psychiatry. 1995;52:296–303. [PubMed]
39. Woerner M, Manuzza S, Kane J. Anchoring the BPRS: an aid to improved reliability. Psychopharmacol Bull. 1988;24:112–124. [PubMed]
40. Andreasen NC. The Scale for the Assessment of Negative Symptoms (SANS) Iowa City, IA: The University of Iowa; 1983.
41. Endicott J, Andreasen N, Spitzer RL. Family History—Research Diagnostic Criteria. New York, NY: Biometrics Research, New York State Psychiatric Institute; 1985.
42. Jablensky A, Sartorius N, Ernberg G, et al. Schizophrenia: manifestations, incidence and course in different cultures: a World Health Organization ten-country study. Psychol Med Monogr. 1992;20:l–97. [PubMed]
43. Muthen LK, Muthen BO. Mplus. Statistical Analyses with Latent Variables. User’s Guide. 5th ed. Los Angeles, CA: Muthen & Muthen; 2007.
44. Hu L, Bentler PM. Fit indices in covariance structure modeling: sensitivity to underparametrized model misspecification. Psychol Methods. 1998;3:424–453.
45. Hu L, Bentler PM. Cutoff criteria for fit indices in covariance structure analysis: conventional criteria versus new alternatives. Struct Equ Modeling. 1999;9:1–55.
46. Marsh HW, Hau KT, Wen Z. In search of golden rules: comment on hypothesis-testing approaches to setting cutoff values for fit indices and dangers in overgeneralizing Hu and Bentler’s (1999) findings. Struct Equ Modeling. 2004;11:320–341.
47. Akaike H. A new look at the statistical mode identification. IEEE Trans Automat Contr. 1974;19:716–723.
48. Burham KP, Anderson DR. Model Selection and Multimodel Inference: A Practical Information-Theoretic Approach. 2nd ed. New York, NY: Springer-Verlag; 2002.
49. Cardno AG, Rijsdijk FV, Sham PC, Murray RM, McGuffin P. A twin study of genetic relationships between psychotic symptoms. Am J Psychiatry. 2002;159:539–545. [PubMed]
50. Kendler KS, McGuire M, Gruenberg AM, O’Hare A, Spellman M, Walsh D. The Roscommon family study: affective illness, anxiety disorder, and alcoholism in relatives. Arch Gen Psychiatry. 1993;50:952–960. [PubMed]
51. Dick DM, Aliev F, Wang JC, et al. Using dimensional models of externalizing psychopathology to aid in gene identification. Arch Gen Psychiatry. 2008;65:310–318. [PubMed]
52. Hettema JM, An SS, Bukszar J, et al. Association between glutamic acid decarboxylase genes and anxiety disorders, major depression, and neuroticism. Mol Psychiatry. 2006;11:752–762. [PubMed]
53. Hettema JM, An SS, Bukszar J, et al. Catechol-O-methyltransferase contributes to genetic susceptibility shared among anxiety spectrum phenotypes. Biol Psychiatry. 2008;64:302–310. [PMC free article] [PubMed]
54. Cannistraro PA, Rauch SL. Neural circuitry of anxiety: evidence from structural and functional neuroimaging studies. Psychopharmacol Bull. 2003;37:8–25. [PubMed]
55. Davidson RJ, Pizzagalli D, Nitschke JB, Putnam KM. Depression: perspectives from affective neuroscience. Annu Rev Psychol. 2002;53:545–574. [PubMed]
56. Siegle GJ, Steinhauer SR, Thase ME, Stenger VA, Carter CS. Can’t shake that feeling: event-related fMRI assessment of sustained amygdala activity in response to emotional information in depressed individuals. Biol Psychiatry. 2002;51:693–707. [PubMed]
57. Anderson SW, Barrash J, Bechara A, Tranel D. Impairments of emotion and real-world complex behavior following childhood- or adult-onset damage to ventromedial prefrontal cortex. J Int Neuropsychol Soc. 2006;12:224–235. [PubMed]
58. Dom G, Sabbe B, Hulstijn W, Van Den Brink W. Substance use disorders and the orbitofrontal cortex: systematic review of behavioural decision-making and neuroimaging studies. Br J Psychiatry. 2005;187:209–220. [PubMed]
59. Yang Y, Raine A, Lencz T, Bihrle S, LaCasse L, Colletti P. Volume reduction in prefrontal gray matter in unsuccessful criminal psychopaths. Biol Psychiatry. 2005;57:103–1118. [PubMed]
60. Martinez M, Marangell LB, Martinez JM. Psychopharmacology. In: Hales RE, Yudofsky SC, Gabbard GO, editors. The American Psychiatric Publishing Textbook of Psychiatry. 5th ed. Arlington, VA: American Psychiatric Publishing; 2008. pp. 1053–1133.
61. Markon KE. Modeling psychopathology structure: a symptom-level analysis of Axis I and II disorders. Psychol Med. 2010;40:273–288. [PubMed]
62. O’Connor BP. A search for consensus on the dimensional structure of personality disorders. J Clin Psychol. 2005;61:323–345. [PubMed]
63. Chmielewski M, Watson D. The heterogeneous structure of schizotypal personality disorder: item-level factors of the schizotypal personality questionnaire and their associations with obsessive-compulsive disorder symptoms, dissociative tendencies, and normal personality. J Abnorm Psychol. 2008;117:364–376. [PubMed]
64. Watson D. Dissociations of the night: individual differences in sleep-related experiences and their relation to dissociation and schizotypy. J Abnorm Psychol. 2001;110:526–535. [PubMed]
65. Slade T, Watson D. The structure of common DSM-IV and ICD-10 mental disorders in the Australian general population. Psychol Med. 2006;36:1593–1600. [PubMed]
66. Bressan RA, Chaves AC, Pilowsky LS, Shirakawa I, Mari JJ. Depressive episodes in stable schizophrenia: critical evaluation of the DSM-IV and ICD-10 diagnostic criteria. Psychiatry Res. 2003;117:47–56. [PubMed]
67. Sands J, Harrow M. Depression during the longitudinal course of schizophrenia. Schizophr Bull. 1999;25:157–171. [PubMed]
68. Siris SG. Depression in schizophrenia: perspective in the era of ‘atypical’ antipsychotic agents. Am J Psychiatry. 2000;157:1379–1389. [PubMed]
69. Clark LA, Watson D. Distress and fear disorders: an alternative empirically based taxonomy of the ‘mood’ and ‘anxiety’ disorders. Br J Psychiatry. 2006;189:481–483. [PubMed]
70. Clark LA. Assessment and diagnosis of personality disorder: perennial issues and an emerging reconceptualization. Annu Rev Psychol. 2007;58:227–257. [PubMed]

Articles from Schizophrenia Bulletin are provided here courtesy of Oxford University Press