PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
Behav Res Ther. Author manuscript; available in PMC 2010 July 1.
Published in final edited form as:
PMCID: PMC2692625
NIHMSID: NIHMS106540

When social anxiety disorder co-exists with risk-prone, approach behavior: Investigating a neglected, meaningful subset of people in the National Comorbidity Survey-Replication

Abstract

Little is known about people with social anxiety disorder (SAD) who are not behaviorally inhibited. To advance knowledge on phenomenology, functional impairment, and treatment seeking, we investigated whether engaging in risk-prone behaviors accounts for heterogeneous outcomes in people with SAD. Using the National Comorbidity Survey-Replication (NCS-R) dataset, our analyses focused on people with current (N = 679) or lifetime (N = 1143) SAD diagnoses. Using latent class analysis on NCS-R risk-prone behavior items, results supported two SAD classes: (1) a pattern of behavioral inhibition and risk aversion and (2) an atypical pattern of high anger and aggression, and moderate/high sexual impulsivity and substance use problems. An atypical pattern of risk-prone behaviors was associated with greater functional impairment, less education and income, younger age, and particular psychiatric comorbidities. Results could not be subsumed by the severity, type, or number of social fears, or comorbid anxiety or mood disorders. Conclusions about the nature, course, and treatment of SAD may be compromised by not attending to heterogeneity in behavior patterns.

Keywords: social phobia, impulsivity, self-regulation, comorbidity, classification, epidemiology

Social anxiety disorder (SAD) is defined by a marked fear and avoidance of social situations where there is potential for evaluation or rejection by others (American Psychiatric Association, 2000). It is one of the most common psychiatric disorders with a lifetime prevalence rate approaching 12% (Kessler, Chiu, Demler, & Walters, 2005; Kessler et al., 1994). Since SAD was officially recognized as a disorder in 1980, researchers found evidence for distinct subgroups. Some people fear and avoid a circumscribed set of social situations, typically performance situations, whereas others fear and avoid a broad array of situations spanning formal performances and informal interactions (Kessler, Stein, & Berglund, 1998). Yet, there is no consensus for how SAD can be classified according to a symptom-based approach as other researchers reached different conclusions (Eng, Heimberg, Coles, Schneier, & Liebowitz, 2000; Furmark, Tillfors, Stattin, Ekselius, & Fredrikson, 2000). Recent epidemiological results support a linear relation for the number and type of feared social situations, failing to find evidence for qualitatively distinct subgroups (Kollman, Brown, Liverant, & Hofmann, 2006; Ruscio et al., 2008). Despite over 25 years of research on the nature of SAD, little research has explored the validity and clinical utility of defining people on variables other than the number and types of feared social situations (see Hofmann, Heinrichs, & Moscovitch, 2004, for a review).

The prototypical person with SAD is characterized as shy, submissive, behaviorally inhibited, and risk aversive (Gilbert, 2001; Leary, 2001). At the core is an underlying motivation to avoid social threat and danger leading to avoidance behaviors such as escape and withdrawal from social situations (APA, 2000; Liebowitz, 1987). Yet, there are many theoretically important variables with no relevance to diagnostics and assessment as currently practiced. Some evidence suggests that, at least for some people, SAD is related to risk-prone activities including hostile impulses and interpersonal behavior, unsafe sexual practices, and novelty seeking in response to impulsive decision-making (Erwin, Heimberg, Schneier, & Liebowitz, 2003; Kachin, Newman, & Pincus, 2001; Kashdan, Elhai, & Breen, 2008; Kashdan & Hofmann, 2008). These omitted variables reflect a relatively atypical manifestation of SAD that is rarely discussed in theoretical models of social anxiety disorder or accounted for in clinical practice. However, some insights can be gleaned from social-cognitive theories of personality.

Self-Regulatory Patterns and Social Anxiety Disorder

The notion of multifinality (Cicchetti & Rogosch, 1996; Rutter, 2005) can be applied to the various ways that people with SAD avoid being rejected and improve their ability to gain acceptance by others. Basically, subgroups of people with similar problems (i.e., SAD) generate different (i.e., risk averse versus prone) behaviors and strategies that in turn, lead to different outcomes. The fundamental issue is how to best capture this heterogeneity in people with social anxiety problems.

The presence or strength of social anxiety fears fails to provide information on which strategies will be chosen. An individual’s actual behavior depends on their goals for a particular situation (e.g., Elliot, Gable, & Mapes, 2006; Elliot & Thrash, 2002). When deeply ingrained social fears and goals are viewed as separable constructs, the potential for heterogeneity in self-regulatory patterns among people with SAD becomes clear.

As an illustration, consider two people with SAD worried about attending at an upcoming party. Alexis decides to stay at home, refusing to answer the phone because she is relatively submissive and does not want to be persuaded to go. Jessica on the other hand goes to the party and is controlling and dominant during social interactions. Before someone can talk or ask her questions, she is quick to direct the conversation, judgmental of others, and the first to move on to another topic or leave to talk to someone else. Her aggressive behavior is an effective way to manage the situation such that she decides who to accept or reject before other people can do the same. Alexis adopts avoidance goals and Jessica adopts approach goals, both in the service of underlying avoidance motivation. Avoidance goals and behaviors can be expected to be more probable but there is evidence to suggest a subset of people with SAD with an odd mixture of approach goals and behaviors.

Most of the work on SAD has focused on over-regulated, risk-averse responses to perceived negative evaluation. However, there are studies providing initial evidence for a non-obvious subset of people with social anxiety problems engaging in an array of risk-prone activity (Erwin et al., 2003; Kachin et al., 2001; Vohs, Baumeister, & Ciarocco, 2005). As an example, prior research has documented a link between rejection (real or perceived) from significant others and hostile, aggressive reactions (Mabel, 1994; Leary, Twenge, & Quinlivan, 2006). For people with social anxiety problems, already expecting rejection, they are primed to view ambiguous reactions from other people as evidence of social threat and being negatively evaluated (e.g., Amir, Foa, & Coles, 1998; Downey & Feldman, 1996). Having their goal of being accepted thwarted, often leads to angry feelings (besides anxiety) that for some, translates into the desire to hurt perpetrators and for a smaller minority, actual aggressive behavior (Ayduk, Downey, Testa, & Yen, 1999; Williams, 2001). In these studies of laboratory aggression and responses to rejection in the real-world, anger and aggression is a retaliatory response. However, other work suggests that repeated rejection experiences can lead to pre-emptive anger and aggression. After all, these particular risk-taking behaviors can showcase dominance, earn acceptance and respect, and prevent relational devaluation or loss in social status from unexpected acts of rejection by other people (Fournier, Moskowitz, & Zuroff, 2002; Gilbert, 2001; Leary et al., 2006). A perfectly reasonable strategy for someone with SAD is to attack and reject other people before they get a chance to do the same to them.

These seemingly paradoxical approach behaviors differ in form from over-regulated avoidance behaviors but the functional goal might be the same: temporarily avoid the unwanted experience of anxiety or probability of rejection. In three independent studies, with clinical (SAD diagnoses; Kachin et al., 2001; Kashdan & Hofmann, 2008) and non-clinical (elevated trait social anxiety; Kashdan et al., 2008) samples, person-centered data analytic procedures (cluster analysis) were used to evaluate the presence of qualitatively different groups of people with elevated social anxiety or SAD were identified. Cluster analyses were conducted on social anxiety scores and risk-taking behavior patterns (in one study, cost and benefit appraisals of risk-taking behavior were used). In these studies, the majority of people with social anxiety problems were behaviorally inhibited, overregulated, and risk-averse. A smaller subset of people with social anxiety problems evidenced impulsive, risk-prone activity and aggressive social behavior. In one study, compared with the more inhibited group, the risk-prone group was characterized by greater difficulties regulating emotions, fewer social resources (social support, satisfying relationships), and less psychological flexibility. In addition, upon using a time-line follow-back method over a 3-month summer period of assessment, the risk-prone group reported more frequent unsafe sexual practices, aggression, and substance abuse (Kashdan et al., 2008). In another study, people with SAD with impulsive novelty seeking tendencies showed evidence of greater comorbid substance use problems compared with people with SAD and low novelty seeking (Kashdan & Hofmann, 2008). Interestingly, in both samples of people diagnosed with SAD (Kachin et al., 2001; Kashdan & Hofmann, 2008), social anxiety severity and functional impairment failed to differentiate groups predisposed toward risk-prone and risk-averse behaviors.

Taken together, there is initial support for a critical set of behaviors and self-regulatory strategies omitted from most descriptions of SAD. Impulsive, risk-prone, approach behaviors can serve an adaptive role in temporarily managing anxiety symptoms by circumventing some of the effortful and disruptive cognitive processes initiated by situational social anxiety (e.g., Rapee & Heimberg, 1997). A subset of people with social anxiety problems recognize the benefits of being aggressive, engaging in unsafe sexual practices, and experimenting with recreational drugs (Kashdan, Collins, & Elhai, 2006), yet, are conflicted by the additional recognition of potential costs for the same behaviors (Kashdan et al., 2008). Beyond internal approach-avoidance conflicts, there are important maladaptive consequences when impulse control problems exist within SAD. People who are impulsive have been shown to exhibit less rational decision making, poor relationship functioning, compromised quality of life (Baumeister, 2002; Muraven & Baumeister, 2000) and shorter life expectancy (Friedman et al., 1995). This behavior pattern is not suitable for determining the best general course of action as circumstances change from one situation to the next and a rigid behavioral pattern (insensitive to context) would lead to less effective adaptation to rewarding opportunities and challenges. Thus, the presence of SAD and a high frequency of risk-prone, impulsive, approach behaviors might characterize a particularly impaired subset of people that might account for meaningful variability in treatment outcomes.

To date, there is one published study showing that adults with SAD in cognitive-behavioral group therapy with greater anger and aggression problems are less likely to complete treatment and of those that do, exhibit worse outcomes (Erwin et al., 2003). At best, therapy was minimally effective in reducing anger, aggression, and impulsivity (Cohen’s d ranging from.11 to.27). Thus, there is preliminary evidence that people with SAD who are particularly angry and aggressive might require a modified treatment protocol to improve life disturbances.

Modeling the Heterogeneity of Social Anxiety Disorder

The current study is part of a program of research focused on uncovering meaningful but omitted variables in describing and understanding SAD. Preliminary empirical evidence for the utility of these variables has been found with three independent clinical (Kachin et al., 2001; Kashdan & Hofmann, 2008) and non-clinical (Kashdan et al., 2008) samples. These studies, however, were limited by small samples, unstable data analytic procedures (relying on cluster analyses), self-report questionnaires, and the failure to test alternative explanations.

The purpose of the present investigation was to examine the reliability and validity of using risk-prone behavior as an approach to classify people with SAD into various subgroups. To access a large sample crossing the diagnostic threshold of SAD, we conducted our analyses on the National Comorbidity Survey-Replication (NCS-R) database (Kessler et al., 2004). Several questions were addressed. First, is it possible to classify people with SAD based on their engagement in risk-prone behavior compared with a more prototypical pattern of behavioral inhibition? Second, does this risk-prone behavior approach to SAD subgroups offer anything beyond the dominant DSM-IV strategy of describing SAD according to severity, type, and number of feared social situations? That is, we examined whether there were any differences in group membership between strategies to subgroup people with SAD based on symptoms versus risk-prone behaviors. To meet these aims, we used latent class analyses to determine the existence of qualitatively distinct subsets of people based on risk-prone behavior and SAD symptoms. Finally, to evaluate the validity of any latent classes, comparisons were made on theoretically relevant sociodemographic, functional impairment, and comorbidity variables.

The combination of SAD and tendencies to engage in risk-prone behaviors1 was expected to be a maladaptive configuration in terms of mental and physical health.

Methods

Participants

The data come from the National Comorbidity Survey-Replication NCS-R (Kessler et al., 2004) – a nationally representative, face-to-face household survey conducted in the United States. The survey used a multistage clustered-area design whereby participants were randomly recruited (response rate 70.9%) from geographic regions. Survey respondents were first recruited via a letter and then provided detailed instructions and verbal informed consent. All respondents completed the first part of the survey. Only a subset of the full sample meeting lifetime criteria for any disorder in the first part or sampled randomly according to a probability routine completed the second part. Further details on research protocols and sampling weighting procedures are detailed elsewhere (Kessler et al., 2004). All analyses reported in this article used the weighted values for the part two sample—conforming to the NCS-R instructions; thus rendering our sample to be representative of the US population. For our purposes, cases were selected if they met DSM-IV criteria for SAD over the past 12 months (N=679, age=39.9 (14.3), male=37%) or in their lifetime (N=1143, age=41.5 (15.0), male=39%) as coded in the NCS-R database (from a total of N=9,282). We did conduct a single test of specificity with the entire NCS-R database. The reason was to evaluate whether our categorical model of risk-prone behavior was unique to people with SAD or generalizable to everyone sampled.

Measures

Risk-prone behaviors

We compiled NCS-R items that might relate to risk taking behavior and impulsive thoughts. Using an iterative process, we subsequently removed items circumscribed to childhood and adolescence, vague or abstract such that the purpose was unclear (e.g., “hard to stay out of trouble”), reflective of common (e.g., “never annoyed when others cut in line”) or extremely uncommon (e.g., “threatened someone with gun in past year”) behaviors, or redundant with items already selected. A total of 20 binary items relating to the four general risk-prone domains of aggression, anger, substance use, and sexuality were selected from the NCS-R (Table 1). These items were then added to form composite variables (i.e., sum-score frequencies of item endorsements) reflecting the five domains. We coded non-administered items as “no” or “0” since that reflected the most conservative imputation method (McKnight, McKnight, Sidani, and Figueredo, 2007)

Table 1
Items Representing Risk-Prone Behaviors from the NCS-R for People with Current Social Anxiety Disorder

Social anxiety disorder severity

We identified NCS-R items (N=18) relevant to the severity and pervasiveness of SAD (Table 2). Similar to the risk-prone behavior items, we grouped binary severity items into six composite scores to reduce inter-item dependencies. The six scores reflected social interaction fears (N=4), social observation fears (N=5), panic symptoms in social situations (N=4), performance fears (N=2), humiliating behavior fears (N=2), and assertiveness fears (N=1). The SAD severity variables were used to test hypotheses that risk-prone behavior patterns provide additional information beyond prior approaches to classifying SAD by type and number of feared social situations.

Table 2
Selected Items Representing Social Anxiety Disorder Severity from the NCS-R

Summary of risk-prone behavior and SAD severity items

The final dataset after abstraction and data reduction consisted of 20 risk-prone behavior items leading to four aggregate risk-prone behavior variables (anger, aggression, anger, substance use, and sexuality), and 18 SAD severity items leading to six aggregate SAD severity variables.

Validity items

To assess the validity of any new latent classes, we included: five socio-demographic variables (sex, cohabitation with romantic partners, income, education, and age), a visual analogue scale of physical and mental health comparable to the Global Assessment of Functioning, and three relevant variables from the SAD module of the NCS-R: age of SAD onset, a binary variable of SAD treatment history, and functional impairment.

Finally, we included binary variables reflecting the presence of comorbid disorders (based on DSM-IV criteria) in the categories of impulse control (Attention-Deficit Hyperactivity Disorder, Conduct Disorder, Intermittent Explosive Disorder, and Oppositional Defiant Disorder), substance use (Alcohol Abuse, Alcohol Dependence, Drug Abuse, and Drug Dependence), anxiety (Agoraphobia, Generalized Anxiety Disorder, Panic Disorder, Post-Traumatic Stress Disorder, and Specific Phobia), and mood (Bipolar I, Bipolar II, Bipolar Subthreshold, Dysthymic Disorder, and Major Depressive Disorder).

Data Analytic Plan

The data analyses consisted of 1) separate latent class analyses for risk-prone behavior and SAD severity items, 2) comparison between these two latent class results, 3) tests of the validity of these latent classes. We address these analyses in order below.

Latent Class Analysis

The risk-prone behavior and SAD severity composite scores were analyzed using a latent class analysis (LCA) method to determine the number of sub-types that underlie the general classification of SAD. Our analyses used composite scores instead of raw items to reduce the inter-item covariance. Parameter estimates and standard errors were derived from a modified EM algorithm (estimated in the R statistical package poLCA; Linzer & Lewis, 2008) and, as indicated previously, sample weights were used as covariates to render the results more appropriate for population estimates. All goodness of fit estimates and nested-model comparisons were conducted using both theory and empirical measures of fit. Specifically, we used the Bayesian information criterion (BIC), Akaike’s Information Criterion (AIC), and ML residual estimates to assess fit between the latent class model results and the data. Discrepant results among the indicators lead us to favor the BIC since that fit statistic tends to perform better with larger sample sizes (Yang, 2006). To avoid the problems of local maxima, five hundred iterations were performed on each of the final models. Results from both risk-prone behaviors and SAD severity composite scores were compared to determine whether the classification differed by either set of variables.

Validity Analysis

Latent class results were analyzed for both the risk-prone behavior and SAD severity items in subsequent logistic regression models. Predictors of latent class membership included the previously discussed socio-demographic variables, SAD characteristics (age of onset, treatment history, and impairment), health, and psychiatric comorbidities. All models were weighted by the sampling weights included in the NCS-R database as specified by Kessler et al. (2004). To control for alpha inflation and potential multicollinearity, groups of predictors were used in three separate models for each of the latent class results risk-prone behaviors and SAD severity items. Furthermore, alpha inflation was controlled by a Bonferroni correction – six models were run for three separate item groups representing socio-demographics, SAD relevant variables, and comorbid mental health diagnoses. The critical alpha level given the six models was .008 (.05/6). The purpose of separating the validity analyses from the latent class analyses was to ensure that the sampling weights could be applied properly using standard statistical tools.

Results

Latent Class Analysis

Risk-prone Behavior

The latent class models from both the current and lifetime SAD subsamples indicated that a 2-class solution fit best (see Table 3) for the risk-prone behavior items. There were no differences between the two samples for either solution so the remainder of the results focused exclusively on the current SAD sample due to more complete data and the better reliability associated with recent reporting. The two classes generated by the risk-prone behavior model were theoretically meaningful. Class 1 represented 79% of the SAD sample, characterized by low aggression (37% endorsed no items), anger (60% endorsed only one item), sexual impulsivity (86% endorsed no items), and substance use (69% endorsed no items) (see Table 4). Class 2 represented the remaining 21% of the SAD sample, characterized by moderate-to-high aggression (57% endorsed four or more items), high anger (66% endorsed all items), low-to-moderate sexual impulsivity (45% endorsed one or more item) and low-to-moderate substance use (51% endorsed at least one item). We refer to the first latent class as the prototypical inhibited, risk-averse subgroup and the second class as the atypical, risk-prone subgroup. 2

Table 3
Latent Class Analysis Model Comparisons for Risk Prone Behavior Items
Table 4
Item Endorsement Rates by Latent Class

SAD severity

A two-class solution was the best fit for the SAD severity items. The two classes generated by the SAD severity items were theoretically meaningful. Class 1 represented 54% of the SAD sample, characterized by relatively lower fears of social interaction (less than 40% endorsed high levels), being observed (less than 12% endorsed high levels), and assertiveness (60% endorsed no fear) as well as slightly lower performance fears (approximately 30% did not endorse the highest levels) and embarrassing or humiliating actions (approximately 40% did not endorse the highest levels). Class 2 represented the remaining 46% of the SAD sample, characterized by relatively higher fears of social interaction (almost 80% endorsed most items), being observed (almost 70% endorsed most items), performance situations (92% endorsed all items), embarrassing or humiliating actions (84% endorsed all items), and assertiveness (almost 90% endorsed the single item). These latent class results indicate that the SAD sample was differentiated with the first class representing the less severe cases and, the second class representing the most severe cases. Table 5 reports individual item endorsements by latent class.

Table 5
Item Endorsement by Latent Classification Based on Social Anxiety Disorder Severity

Comparison between Risk-prone Behavior and Severity Latent Class Results

The relationship between the latent classes identified by risk-prone behavior and SAD severity items was significant (χ2 (1) =19.94, p <.0001) but small (Cohen’s Kappa =.15). As shown in Table 6, latent class 2 from the risk-prone behavior items (i.e., the atypical SAD group) was more likely to be classified in latent class 2 from the severity items (i.e., the more severe group). However, the atypical group was not redundant with the severe SAD group; only 63% of the atypical group and 42% of the prototypical SAD group was classified in the severe SAD group. The relative risk—an index of the likelihood of classification—of the risk-prone behavior group being classified as more severe was 1.50 (95% CI: 1.28, 1.76).

Table 6
Membership in Social Anxiety Disorder Latent Classes Based on Risk-Prone Behaviors and Severity

Validity Analysis Results for SAD Latent Classes

Validity analyses for latent classes are presented in Table 7. We begin by comparing people classified in the risk-prone behavior class with those in the prototypical class. As an alternative classification scheme, we examined people in the two SAD severity classes.

Table 7
Demographic Correlates and 12-Month Psychiatric Comorbidity of Latent Class

Socio-Demographic Characteristics

People in the atypical SAD class were more likely to be younger in age at the time of interview, reported poorer global health, and there was a trend for being a male with lower income compared with people in the prototypical SAD class. In contrast, people in the more severe SAD class were more likely to be women with lower income and younger in age. With the exception of gender, the demographic variables related to membership in the atypical SAD class also predicted membership in the more severe SAD class.

SAD Characteristics

We tested the extent to which SAD class membership was predicted by the number of feared social situations, functional impairment, treatment seeking, and age of SAD onset. People in the atypical SAD class reported greater feared social situations, functional impairment, and an earlier age of onset; despite greater impairment, there was no relation to treatment seeking. In contrast, people in the more severe SAD class reported a greater feared social situations and a greater probability of treatment seeking; however, there were no differences between severity classes on impairment or age of onset. Thus, latent classes based on risk-prone behavior and severity showed distinct relations to SAD variables.

Comorbidity

We tested whether the latent SAD class of risk-prone behaviors affected the risk for various comorbid disorders during the past 12-months. Being in the atypical SAD class led to a greater risk for most impulse control and bipolar disorders; there were no significant relations with comorbid anxiety, depressive, or substance use disorders. In contrast, several anxiety and mood disorders functionally tied to risk-aversion and behavioral inhibition predicted membership in the more severe SAD class.

Discussion

With new epidemiological data from the NCS-R, we used a latent class model to describe and classify people with SAD using risk-prone behavior items. Our latent class analysis provided evidence for critical heterogeneity in SAD based on aggression, anger, sexuality, and substance use behavior patterns. The majority of people with SAD reported a prototypical pattern of behavioral inhibition and risk aversion. A smaller subset of people reported elevated aggression and moderate levels of sexual impulsivity that are rarely described in reference to SAD (as well as severe substance use problems). Similar profiles emerged for people with current and lifetime SAD diagnoses. By moving beyond items used to diagnose SAD, the current study extends prior attempts to target variables that account for different levels of impairment and treatment seeking (Ruscio et al., 2008).

Our two classes were not redundant with and could not be accounted for by SAD variables reflecting severity, and type and number of feared social situations. Despite reflecting two variants of pathology, we found a weak relation between risk-prone behaviors and SAD severity (r =.19) and large minority of people with SAD and risk-prone behaviors were categorized as low in SAD severity (37%). As further evidence of specificity, a dissimilar three latent class solution emerged for the entire NCS-R sample (see footnote 2). That is, the classification scheme that emerged in people with SAD did not hold for the general population.

There is reason to be confident that the current findings provide new, reliable, and valid information. Compared with inhibited peers, people with SAD and a pattern of risk-prone behaviors reported lower education, lower financially security, younger age (at the time of assessment), worse physical and mental health, greater impairments from SAD symptoms, and greater risk for a range of mental health disorders. Despite greater impairment, these people were no more likely to seek treatment for SAD.

The comorbidity data provide evidence for the validity of this atypical subset of people. The most substantial comorbidity rates were found with impulse control disorders and the least with anxiety disorders (particularly generalized anxiety disorder) and major depressive disorder. Impulse control disorders suggest disruptions in self-control ability even when serious life impairments might result. Major depressive disorder and generalized anxiety disorder are characterized by features that run counter to spontaneity and poor impulse control. These features include extreme fatigue, conservation of energy expenditure, intolerance of uncertainty, and over-analytical processing (brooding, worrying) (APA, 2000; Davey & Wells, 2006). Some theorists suggest that depression is an evolutionarily adaptive mechanism leading to disengagement from unattainable goals and submission to other people after apparent defeats by rivals to prevent further loss of energy, status, and power (Neese, 2000). Thus, the comorbidity patterns distinguishing our SAD classes converge with prior work.

Theoretical Speculation to Explain Heterogeneity in Social Anxiety Disorder

Sufficient data are unavailable in this study to address questions about etiologic paths that might lead to people with SAD who show a tendency toward risk-prone behavior. However, we offer several competing theoretical models that might explain this subset of individuals.

First, atypical SAD might be explained by research suggesting that a person’s ability to inhibit impulses is weakened by frequent acts of self-control (Muraven & Baumeister, 2000). People with SAD believe that others hold unrealistically high social standards and the inability to meet these standards is highly probable and costly (Rapee & Heimberg, 1997). In response, people devote considerable attention and effort to alter undesired internal states such as anxiety and behaviors observable to others (Hayes, Wilson, Gifford, Follette, & Strosahl, 1996). Social anxiety, however, does not only occur during social situations; it can occur during any situation eliciting beliefs about low or declining social value to other people (Leary, 2001). Frequent acts of self-control can deplete the finite personal resources available to a person at any given point in time, leading to compromised executive functioning (Baumeister, 2002). When a person manages fears prior, during, or after social situations the effort often occurs at the expense of acting toward other desired goals (Kashdan & Steger, 2006; Vohs et al., 2005).

Second, there are biological and environmental influences leading to impulsivity. These influences might account for high comorbidity rates with impulse control, bipolar, and substance use disorders (e.g., Barkley, 1997; Zuckerman, 1991). Third, there is the possibility that impulsivity problems lead to SAD. Engaging in impulsive acts can lead to embarrassing and humiliating consequences. Waking up in bed with a prostitute and doing inappropriate things when intoxicated can lead to painful feelings of regret and a fear of being evaluated by other people and for some, this response pattern might contribute to the development of disorder.

Fourth, the subset of people reporting elevated risk-prone behaviors might be better characterized by general distress. Although his parsimonious explanation is appealing, the atypical and prototypical groups did not meaningfully differ on all types of comorbidities. Fifth, our atypical subset of people with SAD might be better described by their position in society. These individuals are characterized by less education, household income, general physical health, and taken together, are probably burdened by less access to social and economic opportunities. Many seemingly impulsive behaviors such as substance abuse might be a form of self-medication to cope with maladaptive life circumstances. If they have a higher probability of living in impoverished environments, substance abuse and aggression might reflect socially accepted behaviors and they may have never acquired alternative, more adaptive coping skills. Some people might have been predisposed to their psychological problems as a result of living conditions whereas others might have experienced a downward social drift.

The five competing explanations mentioned above allude to how little is known about the heterogeneity of SAD and the multiple paths leading to and from the onset of SAD. We look forward to future research evaluating these competing theoretical models and explanations.

Future Directions

The potential implications of our findings are far-ranging. Primary care physicians, often the first and only professionals with an opportunity to evaluate and recommend treatment for mental health problems, routinely fail to detect the presence of anxiety disorders (e.g., Fifer, Mathias, Patrick, & Mazonson, 1994). The inherent difficulties of detecting SAD can be amplified when clients present with atypical patterns of aggression, sexual impulsivity, and substance abuse. We suspect that a number of people are misdiagnosed as a result of superficial assessments of approach and avoidance processes. To understand how people with SAD respond to underlying avoidance motivation, we need to examine the goals they develop and the specific strategies employed to make progress toward them (e.g., Elliot et al., 2006). These strategies can be avoidance focused (e.g., saying very little in a conversation ensures nothing embarrassing is revealed), or approach focused (e.g., being aggressive and hostile toward other people to reject them before they have a chance to do so first). Both higher and lower levels of the hierarchy require consideration in the diagnostic and treatment planning process. Future investigations can examine whether this atypical, risk-prone oriented SAD pattern is over-represented in undetected or misdiagnosed cases by clinicians. It is also worth examining how clinicians differentially respond in terms of treatment planning when examining clients with these varying profiles.

There are other clinical reasons to examine heterogeneity in self-regulatory behavior patterns in people with SAD. Only about 20–50% of patients with SAD achieve at least moderate end state functioning following pharmacological and psychological interventions (Blanco et al., 2003; Fedoroff & Taylor, 2001; Turner et al., 1996). There is a need to better understand which patients fail to respond to standard treatment protocols, and how interventions can be tailored to enhance the magnitude of treatment efficacy. People with SAD suffering from impulse control problems and hostile interpersonal patterns that are not directly targeted by existing interventions may be overly represented as treatment non-responders (Erwin et al., 2003).

Our findings also have potential implications for basic research on the phenomenology of SAD. Scientists studying the neurobiological underpinnings of SAD have found evidence of reduced activation in brain regions associated with reward sensitivity and positive affect (e.g., dopaminergic pathways; left pre-frontal cortex) (Mathew, Coplan, & Gorman, 2001; Sareen et al., 2007; Schneier et al., 2000). Yet, relations between SAD and specific neurobiological mechanisms are often modest and inconsistent (Kennedy et al., 2001). One explanation is the presence of meaningful individual differences in biological heterogeneity. Since impulsive tendencies are independent from behavioral inhibition at biological levels of analysis (e.g., Bechara, 2005; Markon, Krueger, & Watson, 2005; Zuckerman, 1991), addressing subsets of people with SAD based on risk-prone behavior patterns might increase the precision of neuroscience advances.

It might seem obvious to find that people with SAD engaging in risk-prone behaviors appear more impaired compared with behaviorally inhibited individuals. However, the only studies examining SAD and externalizing behavior problems in adults has been limited to comorbidity rates. Of the few child and adolescent studies, it is unclear whether externalizing behavior problems compounds existing anxiety problems (Biederman et al., 1991; Jensen, Martin, & Cantwell, 1997; Livingson, Dykman, & Ackerman, 1990) or anxiety symptoms attenuate impairments linked to risk-prone behaviors (Pine, Cohen, Cohen, & Brook, 2000; O’Brien & Frick, 1996; Pliszka, 1989). With the exception of Pine et al. (2000), these studies addressed anxiety conditions other than SAD. As a result, it was important to establish that a pattern of risk-prone behaviors was more maladaptive than a pattern of behavioral inhibition in SAD.

Caveats and Limitations

Despite the heuristic value of this study, caveats and limitations require consideration. The subset of individuals with SAD and risk-prone behavior patterns were more severely impaired and at greater risk for comorbid psychiatric conditions than the prototypical group. These results raise the issue of whether the two classes are only identifying diagnostic errors. Despite evidence on the diagnostic precision of the structured interviews used to collect NCS-R data (Kessler, Chiu et al., 2005; Reed et al., 1998; Wittchen, Lachner, Wunderlich, & Pfister, 1998), it remains unclear how well interviewers distinguished the presence of SAD from social anxiety better accounted for by other general medical or mental conditions.

The subset of people with risk-prone behavior patterns might possess comorbid conditions perhaps more comorbid conditions than the typical group but those comorbidities do not account for the complete effects that we observed. Furthermore, our intent was not to estimate the likelihood of these two classes of SAD but rather to highlight the fact that there may be a small subset of people who suffer from SAD but do not behave in ways that clinicians have come to expect. We want to emphasize this point because treatment planning and outcomes might improve if these atypical behaviors are considered in the context of a functional analysis instead of standard rule outs for standard treatment.

Another issue was that NCS-R data were collected using cross-sectional interviews. We were unable to disentangle temporal relations with comorbid conditions. On average, SAD tends to precede the onset of other psychiatric conditions (Brown, Campbell, Lehman, Grisham, & Mancill, 2001). The study of developmental trajectories, however, remains an area in need of research as few studies examine changes from childhood into adulthood in a single sample.

Finally, we chose the most conservative method for coding and analyzing the NCS-R items and, as a result, our estimates of the relative rates of latent classes might be biased against finding the atypical class. Specifically, items related to risky sexual behavior were only administered to respondents who endorsed feeling excited or irritable but we coded the responses as “no” for those who were never administered the risky items. Thus, fewer respondents with SAD had responded to those risky items and might introduce greater error – and perhaps bias - into our latent class results.

Summary

With over 25 years of research on the nature of SAD, clinicians and researchers should be skeptical of any new model of splitting clients into subgroups. The sine qua non of any data reduction technique is obtaining a theoretically meaningful solution that can be replicated in multiple samples using various methods. Our latent class solution is theoretically meaningful, converging with recent work on social anxiety and self-regulation (Maner, DeWall, Baumeister, & Schaller, 2007; Nezlek & Leary, 2002; Rodebaugh, 2007).

This study and prior work with smaller samples provide evidence for a neglected, atypical manifestation of SAD. The present findings could not be explained completely by the DSM-IV classification system for the severity, type, and number of social anxiety symptoms. There was evidence for construct specificity above and beyond related conditions such as major depressive disorder. Evidence of robustness was shown in subsamples of people with current and lifetime diagnoses of SAD. Additionally, the profile found in our SAD subsamples was not found in the full NCS-R sample. The presence of SAD and an atypical pattern of risk-prone behaviors were associated with particularly compromised health and functioning compared with individuals endorsing a prototypical, behaviorally inhibited pattern. Focusing on within-disorder diagnostic profiles can uncover relations masked by presumed homogeneity.

Acknowledgments

The contributions of the first two authors are equal. This work was supported by National Institute of Mental Health grant MH-73937 to Todd B. Kashdan. Stefan G. Hofmann is a paid consultant of Organon and supported by NIMH grants MH-078308 and MH-081116.

Footnotes

1For the remainder of the paper, we use the term risk-prone behaviors as a concise term for the risk-prone, approach, impulsive behaviors under study.

2We examined whether a similar 2-class solution would be found if we conducted our analyses on the entire NCS-R sample. The best-fitting solution was a 3-class solution (BIC = 54188.62; ML log-likelihood = −26824.80) compared with a 2-class (BIC = 55598.95; ML log-likelihood = −27621.33) and 4-class (BIC = 54223.01; ML log-likelihood = −26750.64) solutions. Although these solutions were dissimilar, it was still possible that all of the individuals with SAD in the atypical class fit into one of the three latent classes in the full NCS-R sample. However, upon comparing these two classification schemes, we found evidence for their independence. In the full NCS-R sample, latent class one was comprised of 68% of individuals in the prototypical SAD class, latent class two was comprised of 15% of individuals in the prototypical SAD class, and latent class three was comprised of 16% of individuals in the prototypical SAD class. In contrast, 100% of individuals in the atypical SAD class were classified in latent class three of the full NCS-R sample. These data show that of the individuals with SAD classified in latent class three of the full NCS-R sample, only 62% are in the atypical SAD class. These results provide evidence for the specificity of our latent class findings to individuals with SAD compared to all individuals in the NCS-R.

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

Contributor Information

Todd B. Kashdan, George Mason University.

Patrick E. McKnight, George Mason University.

J. Anthony Richey, Boston University.

Stefan G. Hofmann, Boston University.

References

  • American Psychiatric Association. Diagnostic and statistical manual of mental disorders. 4. Washington, DC: American Psychiatric Association; 2000.
  • Amir, Foa, Coles Negative interpretation bias in social phobia. Behaviour Research and Therapy. 1998;36:945–957. [PubMed]
  • Ayduk O, Downey G, Testa A, Yen Y. Does rejection elicit hostility in rejection sensitive women? Social Cognition. 1999;17:245–271.
  • Baer RA. Mindfulness training as a clinical intervention: A conceptual and empirical review. Clinical Psychology: Science and Practice. 2003;10:125–143.
  • Barkley RA. ADHD and the nature of self-control. New York: Guilford Press; 1997.
  • Baumeister RF. Ego depletion and self-control failure: An energy model of the self’s executive function. Self and Identity. 2002;1:129–136.
  • Bechera A. Decision making, impulse control and loss of willpower to resist drugs: a neurocognitive perspective. Nature Neuroscience. 2005;8:1458–1463. [PubMed]
  • Biederman J, Faraone SV, Keenan K, Steingard R, Tsuang MT. Familial association between attention deficit disorder and anxiety disorders. American Journal of Psychiatry. 1991;148:251–256. [PubMed]
  • Blanco C, Schneier FR, Schmidt A, Blanco-Jerez CR, Marshall RD, Sánchez-Laca A, Liebowitz MR. Pharmacological treatment of social anxiety disorder: A meta-analysis. Depression and Anxiety. 2003;18:29–40. [PubMed]
  • Brown KW, Ryan RM, Creswell JD. Mindfulness: Theoretical foundations and evidence for its salutary effects. Psychological Inquiry. 2007;18:1–26.
  • Brown TA, Campbell LA, Lehman CL, Grisham JR, Mancill RB. Current and lifetime comorbidity of the DSM-IV anxiety and mood disorders in a large clinical sample. Journal of Abnormal Psychology. 2001;110:585–599. [PubMed]
  • Cicchetti D, Rogosch FA. Equifinality and multifinality in developmental psychopathology. Development and Psychopathology. 1996;8:597–600.
  • Crocker L, Algina J. Introduction to classical and modern test theory. Fort Worth, TX: Harcourt Brace Jovanovich; 1986.
  • Dalrymple KL, Herbert JD. Acceptance and commitment therapy for generalized social anxiety disorder: A pilot study. Behavior Modification. 2007;31:543–568. [PubMed]
  • Davey GCL, Wells A. Worry and rumination: Styles of persistent negative thinking in anxiety and depression. Hoboken, NJ, US: Wiley Publishing; 2006.
  • Downey G, Feldman SI. Implications of rejection sensitivity for intimate relationships. Journal of Personality and Social Psychology. 1996;70:1327–1343. [PubMed]
  • Elliot AJ, Gable SL, Mapes RR. Approach and avoidance motivation in the social domain. Personality and Social Psychology Bulletin. 2006;32:378–391. [PubMed]
  • Elliot AJ, Thrash TM. Approach-avoidance motivation in personality: Approach and avoidance temperaments and goals. Journal of Personality and Social Psychology. 2002;82:804–818. [PubMed]
  • Eng W, Heimberg RG, Coles ME, Schneier FR, Liebowitz MR. An empirical approach to subtype identification in individuals with social phobia. Psychological Medicine. 2000;30:1345–1357. [PubMed]
  • Erwin BA, Heimberg RG, Schneier FR, Liebowitz MR. Anger experience and expression in social anxiety disorder: Pretreatment profile and predictors of attrition and response to cognitive-behavioral treatment. Behavior Therapy. 2003;34:331–350.
  • Eysenck HJ. The biological basis of personality. Springfield, IL: Charles C. Thomas; 1967.
  • Fedoroff IC, Taylor S. Psychological and pharmacological treatments of social phobia: a meta-analysis. Journal of Clinical Psychopharmacology. 2001;21:311–324. [PubMed]
  • Fifer SK, Mathias SD, Patrick DL, Mazonson PD. Untreated anxiety among adult primary care patients in a health maintenance organization. Archives of General Psychiatry. 1994;51:740–750. [PubMed]
  • Fournier MA, Moskowitz DS, Zuroff DC. Social rank strategies in hierarchical relationships. Journal of Personality and Social Psychology. 2002;83:425–433. [PubMed]
  • Friedman HS, Tucker J, Schwartz JE, Martin LR, Tomlinson-Keasey C, Wingard D, et al. Childhood conscientiousness and longevity: Health behaviors and cause of death. Journal of Personality and Social Psychology. 1995;68:696–703. [PubMed]
  • Furmark T, Tillfors M, Stattin H, Ekselius L, Fredrikson M. Social phobia subtypes in the general population revealed by cluster analysis. Psychological Medicine. 2000;30:1335–1344. [PubMed]
  • Gilbert P. Evolution and social anxiety: The role of attraction, social competition, and social hierarchies. Psychiatric Clinics of North America. 2001;24:723–751. [PubMed]
  • Hayes SC, Luoma J, Bond F, Masuda A, Lillis J. Acceptance and Commitment Therapy: Model, processes, and outcomes. Behaviour Research and Therapy. 2006;44:1–25. [PubMed]
  • Hayes SC, Wilson KG, Gifford EV, Follette VM, Strosahl K. Experiential avoidance and behavioral disorders: A functional dimensional approach to diagnosis and treatment. Journal of Consulting and Clinical Psychology. 1996;64:1152–1168. [PubMed]
  • Hofmann SG, Heinrichs N, Moscovitch DA. The nature and expression of social phobia: Toward a new classification. Clinical Psychology Review. 2004;24:769–797. [PubMed]
  • Jensen PS, Martin D, Cantwell DP. Comorbidity in ADHD: Implications for research, practice, and DSM-V. Journal of the American Academy of Child & Adolescent Psychiatry. 1997;36:1065–l079. [PubMed]
  • Kachin KE, Newman MG, Pincus AL. An interpersonal problem approach to the division of social phobia subtypes. Behavior Therapy. 2001;32:479–501.
  • Kashdan TB, Collins RL, Elhai JD. Social anxiety, positive outcome expectancies, and risk-taking behavioral intentions. Cognitive Therapy and Research. 2006;30:749–761.
  • Kashdan TB, Elhai JD, Breen WE. Social anxiety and disinhibition: An analysis of curiosity and social rank appraisals, approach-avoidance conflicts, and disruptive risk-taking behavior. Journal of Anxiety Disorders. 2008;22:925–939. [PMC free article] [PubMed]
  • Kashdan TB, Hofmann SG. The high novelty seeking, impulsive subtype of generalized social anxiety disorder. Depression and Anxiety. 2008;25:535–541. [PubMed]
  • Kashdan TB, Steger MF. Expanding the topography of social anxiety: An experience sampling assessment of positive emotions and events, and emotion suppression. Psychological Science. 2006;17:120–128. [PubMed]
  • Kennedy JL, Neves-Pereira M, King N, Lizak MV, Basile VS, Chartier MJ, Stein MB. Dopamine system genes not linked to social phobia. Psychiatric Genetics. 2001;11:213–217. [PubMed]
  • Kessler RC, Berglund P, Chiu WT, Demler O, Heeringa S, Hiripi E, Jin R, Pennell BE, Walters EE, Zaslavsky A, Zheng H. The US National Comorbidity Survey Replication (NCS-R): Design and field procedures. International Journal of Methods in Psychiatric Research. 2004;13:69–92. [PubMed]
  • Kessler RC, Chiu WT, Demler O, Walters EE. Prevalence, severity, and comorbidity of 12-month DSM-IV disorders in the national comorbidity survey replication. Archives of General Psychiatry. 2005;62:617–627. [PMC free article] [PubMed]
  • Kessler RC, McGonagle KA, Zhao S, Nelson CB, Hughes M, Eshelman S, et al. Lifetime and 12-month prevalence of DSM-III-R psychiatric disorders in the United States. Archives of General Psychiatry. 1994;51:8–19. [PubMed]
  • Kessler RC, Stein MB, Berglund P. Social phobia subtypes in the national comorbidity survey. American Journal of Psychiatry. 1998;155:613–619. [PubMed]
  • Kollman DM, Brown TA, Liverant GI, Hofmann SG. A taxometric investigation of the latent structure of social anxiety disorder in outpatients with anxiety and mood disorders. Depression and Anxiety. 2006;23:190–199. [PubMed]
  • Leary MR. Social anxiety as an early warning system: A refinement and extension of the self-presentation theory of social anxiety. In: Hofmann SG, DiBartolo PM, editors. From social anxiety to social phobia: Multiple perspectives. Needham Heights, MA: Allyn & Bacon; 2001. pp. 321–334.
  • Leary MR, Twenge JM, Quinlivan E. Interpersonal rejection as a determinant of anger and aggression. Personality and Social Psychology Review. 2006;10:111–132. [PubMed]
  • Liebowitz MR. Social phobia. Modern Problems in Pharmopsychiatry. 1987;22:141–173. [PubMed]
  • Linzer DA, Lewis J. poLCA: Polytomous Variable Latent Class Analysis: R package version 1.1. 2008.
  • Little RJA. A test of missing completely at random for multivariate data with missing values. Journal of the American Statistical Association. 1988;83:1198–1202.
  • Livingston R, Dykman R, Ackerman P. The frequency and significance of additional self-reported psychiatric diagnoses in children with ADD. Journal of Abnormal Child Psychology. 1990;18:465–478. [PubMed]
  • Mabel S. Empirical determination of anger provoking characteristics intrinsic to anger provoking circumstances. Journal of Social and Clinical Psychology. 1994;13:174–188.
  • Maner JK, DeWall CN, Baumeister RF, Schaller M. Does social exclusion motivate interpersonal reconnection? Resolving the “porcupine problem. Journal of Personality and Social Psychology. 2007;92:42–55. [PubMed]
  • Markon KE, Krueger RF, Watson D. Delineating the structure of normal and abnormal personality: An integrative hierarchical approach. Journal of Personality and Social Psychology. 2005;88:139–157. [PMC free article] [PubMed]
  • Mathew SJ, Coplan JD, Gorman JM. Neurobiological mechanisms of social anxiety disorder. American Journal of Psychiatry. 2001;158:1558–1567. [PubMed]
  • McKnight PE, McKnight KM, Sidani S, Figueredo AJ. Missing data: A gentle introduction. New York: Guilford Press; 2007.
  • Muraven M, Baumeister RF. Self-regulation and depletion of limited resources: Does self-control resemble a muscle? Psychological Bulletin. 2000;126:247–259. [PubMed]
  • Muraven M, Baumeister RF, Tice DM. Longitudinal improvement of self-regulation through practice: Building self-control strength through repeated exercise. Journal of Social Psychology. 1999;139:446–457. [PubMed]
  • Nesse RM. Is depression an adaptation? Archives of General Psychiatry. 2000;57:14–20. [PubMed]
  • Nezlek J, Leary MR. Individual differences in self-presentational motives in daily social interaction. Personality and Social Psychology Bulletin. 2002;28:211–223.
  • O’Brien BS, Frick PJ. Reward dominance: Associations with anxiety, conduct problems, and psychopathy in children. Journal of Abnormal Child Psychology. 1996;24:223–240. [PubMed]
  • Pine DS, Cohen EC, Cohen P, Brook JS. Social phobia and the persistence of conduct problems. Journal of Child Psychology and Psychiatry and Allied Disciplines. 2000;41:657–665. [PubMed]
  • Pliszka S. Effect of anxiety on cognition, behavior, and stimulant response in ADHD. Journal of the American Academy of Child and Adolescent Psychiatry. 1989;28:882–887. [PubMed]
  • Rapee RM, Heimberg RG. A cognitive-behavioral model of anxiety in social phobia. Behaviour Research and Therapy. 1997;35:741–756. [PubMed]
  • Reed V, Gander F, Pfister H, Steiger A, Sonntag H, Trenkwalder C, et al. To what degree does the Composite International Diagnostic Interview (CIDI) correctly identify DSM-IV disorders? Testing validity issues in a clinical sample. International Journal of Methods in Psychiatric Research. 1998;7:142–155.
  • Rodebaugh TL. The effects of different types of goal pursuit on experience and performance during a stressful social task. Behaviour Research and Therapy. 2007;45:951–963. [PubMed]
  • Ruscio AM, Brown TA, Chiu WT, Sareen J, Stein MB, Kessler RC. Social fears and social phobia in the USA: Results from the National Comorbidity Survey Replication. Psychological Medicine. 2008;38:15–28. [PMC free article] [PubMed]
  • Rutter M. Multiple meanings of a developmental perspective on psychopathology. European Journal of Developmental Psychology. 2005;2(3):221–252.
  • Sareen J, Campbell DW, Leslie WD, Malisza KL, Stein MB, Paulus MP, et al. Striatal function in generalized social phobia: A functional magnetic resonance imaging study. Biological Psychiatry. 2007;61:396–404. [PubMed]
  • Schneier FR, Liebowitz MR, Abi-Dargham A, Zea-Ponce Y, Shu-Hsing L, Laruelle M. Low dopamine D2receptor binding potential in social phobia. American Journal of Psychiatry. 2000;157:457–459. [PubMed]
  • Stein MB, Torgrud LJ, Walker JR. Social phobia symptoms, subtypes, and severity: Findings from a community survey. Archives of General Psychiatry. 2000;57:1046–1052. [PubMed]
  • Turner SM, Beidel DC, Wolff PL, Spaulding S, Jacob RG. Clinical features affecting treatment outcome in social phobia. Behaviour Research and Therapy. 1996;34:795–804. [PubMed]
  • Vohs KD, Baumeister RF, Ciarocco N. Self-regulation and self-presentation: Regulatory resource depletion impairs impression management and effortful self-presentation depletes regulatory resources. Journal of Personality and Social Psychology. 2005;88:632–657. [PubMed]
  • Williams KD. Ostracism: The power of silence. New York: Guilford; 2001.
  • Wittchen HU, Lachner G, Wunderlich U, Pfister H. Test-retest reliability of the computerized DSM-IV version of the Munich-Composite International Diagnostic Interview (M-CIDI) Social Psychiatry and Psychiatric Epidemiology. 1998;33:568–578. [PubMed]
  • Yang CC. Evaluating latent class analysis models in qualitative phenotype identification. Computational Statistics and Data Analysis. 2006;50:1090–1104.
  • Zuckerman M. Psychobiology of personality. Cambridge: Cambridge University Press; 1991.