|Home | About | Journals | Submit | Contact Us | Français|
Personality psychology is concerned with affect (A), behavior (B), cognition (C) and desire (D), and personality traits have been defined conceptually as abstractions used to either explain or summarize coherent ABC (and sometimes D) patterns over time and space. However, this conceptual definition of traits has not been reflected in their operationalization, possibly resulting in theoretical and practical limitations to current trait inventories. Thus, the goal of this project was to determine the affective, behavioral, cognitive and desire (ABCD) components of Big-Five personality traits. The first study assessed the ABCD content of items measuring Big-Five traits in order to determine the ABCD composition of traits and identify items measuring relatively high amounts of only one ABCD content. The second study examined the correlational structure of scales constructed from items assessing ABCD content via a large, web-based study. An assessment of Big-Five traits that delineates ABCD components of each trait is presented, and the discussion focuses on how this assessment builds upon current approaches of assessing personality.
Personality psychology has long been concerned with identifying the fundamental ways in which people differ from each other. For over 2000 years, numerous taxonomies of personality traits have been developed in order to catalogue individual differences in personality traits (Ashton, Lee, Perugini, Szarota, de Vries, Di Blas, Boies, & De Raad, 2004; Cattell, 1946; Costa & McCrae, 1992b; DeYoung, Quilty, & Peterson, 2007; Eysenck & Himmelweit, 1947; Eysenck & Eysenck, 1964; Eysenck, 1959, 1992; Galton, 1884; Goldberg, 1992; Hofstee, de Raad, & Goldberg, 1992; Jebb, 1909; John, Donahue, & Kentle, 1991; Norman, 1963; Tellegen, 1982; Wundt, 1897). There is a growing consensus that taxonomies assess at least five broad trait dimensions (the “Big-Five” traits): extraversion, agreeableness, conscientiousness, emotional stability/neuroticism, and openness/intellect1. The Big-Five traits have been useful in predicting myriad important outcomes such as mental health, mental disorders, job success, marriage satisfaction, and even mortality (Ozer & Benet-Martinez, 2006; Roberts, Kuncel, Shiner, Caspi, & Goldberg, 2007). Thus, trait psychology has been one of the most successful enterprises of personality for predicting and understanding healthy psychological functioning. However, there may be important flaws in the assessment of the Big-Five that limit both the understanding of traits and their relationships to mental health.
Personality trait psychology, at its heart, seeks to understand variation in how people feel, act, and think, and want (Allport, 1937; Emmons, 1989; Johnson, 1997; Winter, John, Stewart, Klohnen, & Duncan, 1998). That is, personality is concerned with affect (A), behavior (B), cognition (C) and desire (D) (Revelle, 2008). Personality traits have been defined conceptually as abstractions used to either explain or summarize coherent ABC (and sometimes D) patterns over time and space (Ortony, Norman, & Revelle, 2005). However, this conceptual definition of traits has not been reflected in their operationalization. Specifically, the most widely used Big-Five trait inventories (Costa & McCrae, 1992b; Goldberg, 1992) do not delineate traits according to ABCD components. Rather, ABCD content is emphasized differentially across traits. Each of the Big 5 traits represents one content domain to the neglect of others: neuroticism is typically assessed with items emphasizing affective content; extraversion, agreeableness, and conscientiousness emphasize behavioral content; and openness is represented primarily by cognitive content (Pytlik Zillig, Hemenover, & Dienstbier, 2002; Wilt & Revelle, 2009).
This research seeks to construct a taxonomy of personality traits that assesses ABCD components of each trait. We aim to accomplish this goal by first determining the ABCD content of items assessing each Big 5 domain and then selecting items for our inventory that represent relatively homogenous reflections of just one A, B, C, or D content. For example, our ideal measure of extraversion would include a number of items that reflect the A content of extraversion, a different set of items reflecting the B content, and so on. It is hoped that describing Big Five items by their ABCD content and constructing such a measure situates our Big Five trait assessment within a framework of basic and universal psychological constructs. The ABCD approach may not only reveal a more fundamental composition of traits, but it also may aid in the understanding of processes by which traits are related to healthy psychological functioning.
We will refer to the terms “affect”, “behavior”, “cognition”, and “desire” throughout this paper. Although psychologists may feel at ease with these concepts, there have been a number of efforts to define each domain more precisely. It is likely that psychologists feel most comfortable with behavior. Specific behaviors can be observed, classified, and even manipulated. Despite its ostensibly intuitive nature, there have been inconsistencies in the way the term behavior has been applied in the field of personality. It is therefore important to provide a definition of behavior so as not to create undue confusion about how the term is applied in this paper. We adopt the definition of behavior offered by (i) Furr (2009) - “behaviour may be defined as verbal utterances (excluding verbal reports in psychological assessment contexts) or movements that are potentially available to careful observers using normal sensory processes” (p. 372) - and add to it (ii) Ortony et al. (2005)’s observation that behavior encompasses physical actions that may not be observable through normal sensory processes as well (e.g., contractions of the gut). Behavior is how the mental processes of affect, cognition, and motivaiton manifest themselves and become tangible and concrete (Shweder, 1999). In lay terms, behavior is what we do.
Descriptions of affect tend to converge on the definition of affect as a higher order category subsuming valenced conditions such as moods, emotions, feelings, feeling-like states, and preferences (Mischel & Shoda, 1995; Ortony et al., 2005; Pytlik Zillig et al., 2002; Scherer, 1995). There is vigorous debate about how to distinguish these lower order constructs from each other, particularly how to differentiate emotions from moods (Carver & Scheier, 2009; Ellsworth, 1994; Ortony et al., 2005; Scherer, Schorr, & Johnstone, 2001). However, because affect subsumes each lower-order construct, distinguishing these constructs from each other is not within the scope of the purposes of this paper. In lay terms, affect is how we feel.
Cognition, or cognitive activity, is also thought of as a higher-order category and encompasses mental contents and processes (Gruszka, Matthews, & Szymura, 2010). Attention, memory, knowledge, problem-solving, beliefs, appraisals, interpretations, representations and expectations are all included in the domain of cognition (Cervone, 2004; Ellsworth, 1994; Ortony et al., 2005; Revelle, 1995; Mischel & Shoda, 1995). In lay terms, cognition is how we think.
The domain of desire refers to people’s (conscious or unconscious) goals, needs, wants and wishes (Winter et al., 1998). What binds these constructs in common is that they represent states that people would like to bring about or to prevent (Chulef, Read, & Walsh, 2001). Desire also encompasses the effort (trying hard and being involved in a task) that people put into trying to bring about those states (Humphreys & Revelle, 1984). Desire thus answers the “why” questions concerning behavior (McClelland, 1951) and can therefore be used to explain the tendency to behave in certain ways (Ortony et al., 2005; Mischel, 1973; Mischel & Shoda, 1995). In lay terms, desire is what we want. Desire is chosen over the term “motivation” due to desire’s more specific connotation of referring to what people want, as compared with motivation’s more general connotation of referring to the factors that energize, direct, and select behavior (Atkinson & Raynor, 1978; Heckhausen, 1991). Whereas the factors that guide behavior may include affect, cognition, desire, and even behavior itself, desire links more naturally to goals, wants, and wishes.
Historical definitions of personality traits have incorporated each of the ABCD domains to different extents. There is a broad consensus that global traits refer to patterns of behavior that transcend time and specific situations (Buss & Craik, 1983; Carr & Kingsbury, 1938; Cattell, 1946; Costa & McCrae, 1992a; Eysenck, 1952; Funder, 2001; Goldberg, 1974; Norman, 1963;). Definitions of traits typical also include affective and cognitive components, perhaps because many of the taxonomies of traits that were developed through factor analytic techniques had obvious affective and cognitive components (e.g., Cattell, 1957; Eysenck, 1959; Guilford & Zimmerman, 1949; Norman, 1963) Indeed, contemporary definitions of traits are notably uniform in asserting that traits are stable and consistent patterns of affect, behavior, and cognition that distinguish people from one another (Johnson, 1997; Mayer, 1998; McAdams & Pals, 2006; McCrae & Costa, 1997; Roberts & Jackson, 2008). In contrast to the consensus about the aforementioned ideas, there are a number of different viewpoints about the relation between traits and desires, such as goals, wants, wishes, needs and strivings; some see desire as a key component of traits (Atkinson & Birch, 1970; Borkenau, 1990; Cantor, 1990; Heckhausen, 1991; Read, Jones, & Miller, 1990), whereas others see them as separable constructs (McAdams, 1995; Pervin, 1994; Winter et al., 1998).
The concept of the personality trait is widely regarded as a (if not the) cornerstone of personality psychology (Funder, 1991; McAdams & Pals, 2006; Costa & McCrae, 1992a). However, the consensus of experts does not substitute for rigorous empirical investigation. The true measure of the status of any scientific concept is the degree of accuracy and precision with which the concept can be operationalized. On this note, we now turn to the question of how to operationalize the conceptual definition of personality traits.
Where should this process begin? We propose that it is reasonable to commence the investigation into the question “What is a Big Five trait?” by conducting a straightforward examination into the psychological contents of the Big Five traits included in the most widely used and comprehensive inventories. That is, is each Big-Five trait operationalized as reflecting individual differences in patterns of affect, behavior, and cognition (and possibly desire)? Saucier (2009) notes that science in general and personality science in particular are hindered when concepts are defined differently, and usually more broadly, than they are operationalized. Determining whether trait assessments match their conceptual definitions has the potential to validate current inventories or to reveal whether assessments should be modified to be in accordance with how they are defined conceptually.
It is not controversial to assert that constructs should be operationalized to measure their conceptual definitions. However, advocates of the factor analytic or principal components approaches of deriving traits (Ashton & Lee, 2005; Costa & McCrae, 1998; Goldberg, 1990) argue that assessments of traits need not be concerned with their conceptual contents because these procedures have identified the fundamental individual differences. This view has been criticized as naive at best and myopic at worst (Block, 1995). The use of such techniques without careful attention to conceptual detail may lead to the development of narrow scales of reiterative item content based on mixtures of personality content and response biases (Wilson, Gray, & Barrett, 1990). These methods can result in a number of discrepant findings even when applied to the same data set, as findings may differ as a function of the theory and/or method of rotation preferred by the investigator. These observations are not meant to demean factor analytic or principal components techniques as statistical methods, for they provide a useful technique for summarizing covariances or correlations between large numbers of items or scales. Indeed, these strategies certainly have the power to illuminate questions of personality. However, relying on factor analysis or principal components by themselves is insufficient, limiting, and can sometimes be misleading (Carver, 2001).
Examining the affective, behavioral, cognitive, and desire/motivational (ABCD) components of Big Five traits may not only serve to align assessments of traits with their conceptual definitions, it may also elucidate the nature of traits in terms of core or primary psychological constructs with universal appeal. The ABCD domains may be thought of as “primary parts” because, taken together, they have been hypothesized to “describe the totality of personality” (Mayer, 2001, p. 449). Whether or not the ABCDs do indeed represent the totality of personality will likely depend on one’s idiosyncratic meanings for the ABCDs and personality; a more reasonable statement is that the ABCDs are at least broadly applicable to a wide range of constructs in the personality domain (Bock, 2000; Shweder & Sullivan, 1993).
Three content analyses of personality inventories (Angleitner, John, & Löhr, 1986; Pytlik Zillig et al., 2002; Werner & Pervin, 1986) have shown that the individual items typically used to assess personality characteristics include ABCD content. Given these studies, and given the conceptual definition of a personality trait, the dearth of attention given to how the ABCDs are represented in Big Five trait assessments is surprising and needs remediation. As described below, there has been at least some consideration of how Big Five trait inventories align with the concepts that they presumably assess.
It is widely recognized that two of the Big Five, extraversion and neuroticism, are assessed with items including positive and negative affective content, respectively (Watson, 2000). There is also a growing realization that positive and negative affective content is distributed across the traits of agreeableness (e.g., negative emotion in response to interpersonal aggression), conscientiousness (energy directed toward tasks), and openness (interest in ideas) (Ashton & Lee, 2001; MacDonald, 1995). In the behavioral domain, it has been noted that assessments of extraversion include items measuring behavioral approach (Wilt & Revelle, 2009), and assessments of neuroticism typically include items measuring behavioral withdrawal (DeYoung et al., 2007). Locke and Latham (2006) noted that some items used to assess Big Five traits seem to fit within one of the A (“have a soft heart”), B (“start conversations”), or C (“have excellent ideas”) domains. Yet, even if items may refer to A, B, and/or C content, Big Five inventories are compiled unsystematically in regard to the ABCs (Jackson, Wood, Bogg, Walton, Harms, & Roberts, 2010). That is, the traits, facets, and items within the inventories are not delineated in ABC terms. There has been no recognition that motivational or D content is included in Big Five assessments.
Pytlik Zillig et al. (2002) took a first step toward determining how much ABC (but not D) content is included in Big Five trait inventories. In this study, participants rated individual items across a number of Big Five inventories according to their relative percentage of ABC content. It was determined that extraversion is assessed with items containing mostly A and B content with negligible C content; agreeableness items contained a mixture of A, B, and C content; conscientiousness items were dominated by B content with a modest amount of C content and little A content; emotional stability items contained mostly A content, with low amounts of B and C content; and openness items assess C content primarily, with modest amounts of A and B content. This study provides evidence for Jackson et al. (2010)’s claim that the most widely used Big-Five trait inventories (e.g. Costa & McCrae, 1992b; Goldberg, 1992) do not delineate traits according to ABC components. Rather, ABC content is emphasized differentially across traits.
Why do assessments of traits include different amounts of ABCD content? It is possible that the best indicators of traits truly do fall into one or two domains, but it may also be that item pools for assessing traits are biased (Pytlik Zillig et al., 2002). If the latter is true, then the unbalanced representations of the ABCs (and Ds) in trait inventories might pose major theoretical and practical problems for personality psychology. The theoretical problem is based on the idea that, in order to group constructs in any science together in the same class, those constructs must share a common set of defining characteristics. Current personality trait inventories group affective variables with behavioral and cognitive variables. Thus, it may not be justifiable to group current operationalizations of traits together in the same class. One practical problem with unbalanced representation of ABCDs is that it may limit the ability to understand how and why traits are related to criterion variables. A second potential problem is that it might create biased estimates of covariance amongst item sets with different relative saturations of ABCD content. For example, an extraversion measure saturated with behavioral content might be more strongly related to an agreeableness measure with a high degree of behavioral content than an agreeableness measure that emphasizes affective or cognitive content.
The potential problems discussed above cannot currently be evaluated because there have been no efforts to develop a trait taxonomy in which the ABCD content of each Big Five trait is explicitly specified. Thus, the first goal of this project was to develop a system of assessing personality traits that divides traits into their respective ABCD components. Two studies described in this manuscript aimed to directly extend the efforts of Pytlik Zillig et al. (2002) by (i) determining the ABC (and D) content of personality items used to assess the Big Five traits and (ii) use those results to create a relatively pure measures of ABCD components of each Big Five trait. The goal of these studies was to examine the hypothesis that Big-Five traits may be operationalized by distinct ABCD components.
In order to determine how the Big Five are operationalized with regard to ABCD components, it is important to obtain a comprehensive set of items that have been widely used to assess the Big Five. The most widely used inventory over the past two decades is the NEO-PI-R (Costa & McCrae, 1992b), having been administered in hundreds of studies across dozens of countries (McCrae, 2009). The NEO-PI-R also fulfills the criterion of being relatively comprehensive, as it covers 30 facets and contains 240 items. However, the NEO-PI-R is made less desirable by of its proprietary nature. Proprietary scales are less conducive to scientific progress than their open source counterparts, as proprietary scales are harder to obtain and less likely to quickly and flexibly adapt to empirical discoveries.
Thankfully, an open source scale designed to measure the traits of NEO-PI-R (10 items per facet, 300 total items) has been made available via the International Personality Item Pool (IPIP; Goldberg, Johnson, Eber, Hogan, Ashton, Cloninger, & Gough, 2006), a scientific collaboratory intended as an international effort to develop and continually refine a set of public domain personality items and scales. The IPIP version of the NEO-PI-R appears to be a good proxy for the NEO-PI-R, as the mean correlation between the 30 facet scales of the NEO-PI-R and the corresponding IPIP scales is .94 after correcting for attenuation (Goldberg, 1999). IPIP items are formatted as short, contextualized phrases (e.g., “feel comfortable around people”, “get chores done right away”). This format is conducive to ABCD ratings, particularly when compared with adjective markers, which may contain too little information to provide ABCD ratings, and paragraph descriptors, which may provide so much information as to become unwieldy for raters.
Because the NEO-PI-R technically assesses Five Factor Model traits (which are slightly different than the phenotypic Big Five), it would be ideal to also employ a scale that assesses the phenotypic Big Five that is similarly comprehensive to the NEO-PI-R. Again, the IPIP item pool has developed a scale with these qualities, the IPIP AB5C, which is a 485-item inventory assessing the 45 facets of the Abridged Big Five Circumplex (Hofstee et al., 1992). Using IPIP items also is desirable in any effort to construct personality assessments, as it is envisioned that the IPIP will serve to stimulate communication between researchers, provide a common and unifying language for personality assessment, and therefore move trait assessment toward the goal of becoming a cumulative science (Goldberg, 1999). The IPIP versions of the NEO-PI-R and AB5C were rated with respect to their ABCD content in Study 1. Items that had relatively pure A, B, C, and D content were retained and used in Study 2, which examined their psychometric structure.
Items were drawn from the International Personality Item Pool (IPIP; Goldberg, 1999) collaboratory, which includes scales designed to provide public domain measures parallel to scales found in commonly used personality inventories. Each of the inventories that were employed assesses the Big 5 traits. The IPIP items included were based on the following assessments: the revised NEO inventory (NEO-PI-R; Costa & McCrae, 1992) and the Abridged Big Five Circumplex (AB5C; Hofstee, de Raad, & Goldberg, 1992). These items from these inventories were chosen because they provide the most comprehensive assessment of items assessing personality traits (Grucza & Goldberg). The IPIP version of the NEO PI-R contains 300 items, and the IPIP AB5-C contains 485 items. There are 186 items that are common to both inventories, so the final item sample included 599 items.
Following previous item content analyses (Angleitner et al., 1986; Pytlik Zillig et al., 2002; Werner & Pervin, 1986), relatively few primary raters assessed item content. The first author, three additional psychology graduate students, and two undergraduate research assistants served as raters (2 men, 4 women). Raters other than the first author were compensated with $20 for completing the item ratings.
Items were presented to the participant raters in random order. Items were randomized to reduce the likelihood of item order resulting in a systematic bias in the item ratings. For example, if raters discerned a clear pattern in the order in which items appeared, that may raise the possibility that items perceived to assess the same or similar constructs would be rated more similarly in terms of their ABCD content.
Raters were provided with a spreadsheet including instructions for rating items and the 599 items. The instructions advised raters to “Please rate each of the following items with respect to the percentage of affective, behavioral, cognitive, and desire content included in each item. For each item appearing below, enter the percentage of A, B, C, and D content that the item comprises in the boxes corresponding to each component. The total percentage for each item should add up to 100%.” The ABCD dimensions were defined in a manner that paralleled definitions used in previous research (Angleitner et al., 1986; Pytlik Zillig et al., 2002; Werner & Pervin, 1986). Affects were defined as “internal and evaluative, valenced states, including patterns of feelings, emotions, and feeling-like states.” Behaviors were defined as “overt and directly observable actions, including both active (e.g., bike-riding) and passive (e.g., watching television) behaviors.” Cognitions were defined as “thoughts, beliefs, patterns, or modes of thinking.” Desires were defined as “goals, wants, strivings, and motivations.” The items appeared in the first column of the spreadsheet, and the second through fifth columns were labeled “affect”, “behavior”, “cognition”, and “desire.” The sixth column calculated the total percentage of ABCD content, allowing raters to check that their ratings added up to 100% for each item.
Prior to rating items, raters met with the first author individually in order to clarify any ambiguities present in the instructions. Raters were able to consult with the first author regarding conceptual issues (raters did not discuss specific items with the first author) during the time period during which they made their ratings, but they were discouraged from talking to each other about the ratings as doing so might have artificially inflated agreement among ratings. We believed that it was important not to limit the amount of consultation raters had with the first author in order to increase the likelihood that all raters had a good understanding of the task; this was especially important as undergraduates might need more clarification of the task due to not yet having the expertise in psychology of graduate student raters.
The technical accuracy of the ABCD ratings was very high. There were no missing ratings and all sets of component ratings summed to 100% for each item. For each ABCD domain, inter-rater reliabilities were calculated as intraclass correlations: ICC s indicate the consistency, or conformity, of measurements made by multiple observers measuring the same quantity (Shrout & Fleiss, 1979). ICC s are presented for a single rater (ICC31) and as the average of six fixed raters (ICC36). ICC31s were .56 (Affect), .45 (Behavior), .54 (Cognition), and .37 (Desire), indicating a moderate degree of reliability for ratings of any one rater. ICC36s were .88 (Affect), .83 (Behavior), .88 (Cognition), and .78 (Desire), indicating that mean of the six coders’ ratings of ABCDs items had high degrees of reliability across ABCD domains (Fleiss, 1981). Mean ABCD content was thus calculated across raters for each item. The line plots shown in Figure 1 (plotting ABCD content for each trait separately) indicate that the degree of ABCD content for each trait was similar across inventories, with the possible exception of A content for agreeableness, which appears to be higher in the NEO-PI-R than in the AB5C. The plots also suggest that the amount of ABCD content differs both within and across traits.
Figure 2 shows “violin plots” depicting the density of the distributions of ABCD content in items by trait for each inventory. The density of the plot at a particular point on the Y-axis (percentage) indicates the relative amount of items containing that percentage of ABCD content: for instance, using Conscientiousness as an example, nearly all items had between 0 and 20 percent affective content, whereas very few items had over 20 percent affective content. These plots highlight within-trait differences in each ABCD content. Some distinct patterns emerge from visual inspection of the plots. First, D content appears to be under-represented in items compared with ABC content. It appears that items assessing agreeableness have a relatively balanced representation of ABCs, but each other trait has one content that appears to be predominant: a large number of conscientiousness and extraversion items contained a high degree of behavioral content, whereas a large number of emotional stability items contained a high degree of affective content, and a large number of openness items contained a high degree of cognitive content.
Following Pytlik Zillig et al. (2002), to examine whether the amount of ABCD content differed across traits, four univariate ANOVAs were conducted (one for each ABCD domain) for each IPIP inventory (8 ANOVAs in all), treating the trait domain as a factorial independent variable and each ABCD component rating composite as a dependent variable. Each factorial univariate ANOVA revealed a main effect of trait, indicating that mean levels of ABCD content differed across traits.2
To examine whether the amount of ABCD content differed within each trait, we first organized the data such that, for each trait, percentage ratings were nested within items and ABCD contents were treated as repeated factors (i.e., each item had four percentage ratings, one for each ABCD content). This way of organizing the data allowed for multilevel modeling (MLM) analyses (Bliese, Chan, & Ployhart, 2007; Bryk & Raudenbush, 1992). In each MLM, the percentage value was treated as the outcome variable and was predicted from ABCD content. Each multilevel model revealed a main effect of the ABCD content variable which indicated that the amount of ABCD content differed within traits.
Pairwise follow-up analyses for both (i) across-trait, within ABCD domain differences and (ii) within-trait, cross-domain differences were conducted using Tukey’s HSD procedures (see Table 1 for significant pairwise differences). Each mean in this table is preceded by and/or followed by a superscript (or set of superscripts). For the superscript(s) preceding the means, common values indicate that cross-trait values (row-wise differences) are not significantly different within each inventory; for superscripts following the means, common values indicate that within-trait (column-wise differences) are not significantly different within each inventory.
The “purity” of an item with regard to ABCD content was defined in two different ways. The first way of conceptualizing the purity of an item did not take into account the trait domain which the item assessed, whereas the second way of conceptualizing purity did take the item’s trait domain into account.
The relative “purity” of an item with regard to ABCD content was first defined conceptually as containing a high amount of one ABCD content relative to each other content. For example, an item with 100 % A content and 0 % B, C, and D content would be considered a prototypical “pure” A item. The operational definition of “purity” was designed to match the conceptual definition. The first step to determine purity was to calculate the absolute value of the difference between each pair of ABCD contents for each item. The second step was to compute the means and standard deviations of these values across items for each ABCD content pair. These values are presented in Table 3.
The mean absolute difference values represent the expected difference between contents for a randomly chosen item. For example, the difference between A and B contents for a randomly chosen item would be expected to equal 32.32 (the standard deviation of this value, 20.71, represents the spread of the absolute difference values across items). For an item to be considered “pure” on one dimension with respect to another, the mean difference between contents for that item must exceed the mean absolute value of the difference between contents plus the standard deviation of the absolute difference between those respective contents. For example, for an item to be considered as being pure in A content with respect to B content, the A content for that item would have to be 53.03 greater than than the B content for the item (see Table 3). Items that were “pure” in one content with respect to each other content were selected as “pure” indicators of their dominant content (see the online supplementary material for ratings of ABCD content for each item).
A total of 133 items (22.2% of all 599 items) met criteria for purity in one domain. There were relatively equal numbers of pure A (43), B (36), and C (42) items, and a relative dearth of pure D (12) items. Within each respective ABCD content, pure items were mainly drawn from one or two different traits: most pure affect items assessed emotional stability; most pure behavior items assessed either extraversion or conscientiousness; most pure cognition items assessed openness; and most pure desire items assessed either emotional stability or conscientiousness. One of the main goals of this study was to determine the feasibility of developing an assessment of Big Five traits that contained relatively equal numbers of pure indicators of each ABCD content for each trait. (see Table 2) for a breakdown of pure ABCD items by trait. It is clear that this is not possible when using the first criteria for purity, as pure items within each ABCD domain were markers of only one or two traits.
The second conceptual definition of “purity” took into account each item’s trait domain. For each trait, items that contained the highest degree of each ABCD content were selected as relatively “pure” indicators of that content for each trait. The top 7 indicators of each ABCD content for each trait were selected (7 x 4 ABCD contents x 5 traits = 140 items) in order to arrive at a similar number of items selected by each definition of purity (definition 1: 133, definition 2: 140). See the online supplementary material for ratings of ABCD content for each item.
Sixty-eight items met criteria for both definitions of purity. These items were pure markers of their respective ABCD content and contained relatively high degrees of ABCD content with regard to their respective trait. For agreeableness, these items primarily fell in the domains of affect and cognition. For conscientiousness, these items mainly fell in the domains of behavior and desire. For emotional stability, these items fell in the domains of affect, behavior, and desire. For extraversion, these items fell in the domains of affect and behavior. For openness, these items mainly fell in the domain of cognition.
The ABCD content of traits and items was examined in a number of different ways: analyses (i) described the ABCD content of each trait (ii) compared ABCD content within-and between-traits, (iii) identified pure items according to their relative ABCD content (relative purity), and (iv) identified pure items for each trait (trait purity) according to their absolute amount of ABCD content. Some general conclusions about ABCD content for each trait are presented below.
Agreeableness contained a balance of ABC items and relatively few D items. Relatively few agreeableness items met criteria for relative purity, but those that did tended to assess affect and cognition. Conscientiousness contains a high amount of B content, relatively equal amounts of C and D content, and a relatively higher amount of D and low amount of A content compared to other traits. Conscientiousness items that met trait purity criteria for B and D domains also met relative purity criteria in those domains. Emotional stability is a predominantly affective trait containing a high number of pure A items and pure D items. Items that met trait purity criteria for B and D also tended to meet relative purity criteria. Extraversion is predominantly a behavioral-affective trait containing a large number of pure B items. Extraversion items with high amounts of affective and behavioral content (those that met the trait purity criteria) also met criteria for relative purity. Openness is a predominantly cognitive trait with a high number of pure C items. There was a drastic between-inventory difference in A content, with the NEO-PI-R containing a high amount of A content relative to the AB5C.
Notwithstanding the fact that Pytlik Zillig et al. (2002) did not assess D content of traits, it is interesting to compare the conclusions above to Pytlik Zillig et al. (2002)’s ABC content analysis of Big 5 traits. Similar to Pytlik Zillig et al. (2002), the present analyses showed that (i) agreeableness had balanced amounts of ABC content, (ii) conscientiousness was primarily a behavioral trait, (iii) emotional stability was predominantly affective, (iv) extraversion was primarily a behavioral-affective trait, and (v) openness was primarily cognitive in content. In the Pytlik Zillig et al. (2002) paper, traits were rated as having higher absolute amounts of their primary contents (traits tended to contain between 45% and 75% of their primary content) compared to the present study (traits tended to contain around 40% of their primary content). This difference may be attributable to differences in methodology: Pytlik Zillig et al. (2002) did not assess D content, but they did assess “bodily” and “other” content, and they also used proprietary scales rather than open-source items. Although Pytlik Zillig et al. (2002) did not attempt to identify “pure” items, it is possible that they may have been able to identify more items meeting the relative criteria for purity given their greater within-trait differences in ABC content.
By and large, this study echoes the main conclusion of Pytlik Zillig et al. (2002), that is, traits as assessed by existing personality inventories do not contain balanced amounts of ABC content. This study adds to this conclusion the observation that inventories assess very little D content. Another way in which this study goes beyond Pytlik Zillig et al. (2002) is that we are able to describe the kinds of ABCDs for each trait. For example, we are able to provide an answer as to what it means to say that a person is higher on the dimension of conscientiousness-affect than another person. Descriptions for each ABCD content according to trait are presented in Table 4. This table in some sense “translates” traits into terms that may be understandable semantically and psychologically. A person scoring highly on conscientiousness-affect is someone who feels relatively good when routines are being followed, and relatively poorly when they are not.
The potential problems associated with assessing traits as unbalanced representations of ABCD content, as elaborated in the introduction, may present serious conceptual and practical limitations to the current trait enterprise. The identification of pure items in Study 1 may allow identification of A, B, C, and D domains of Big 5 traits, possibly aligning assessments of traits more closely to trait theory and paving the way for investigations of how ABCD domains are related to important criterion variables. Study 2 is a first attempt to construct such an assessment.
The goals of Study 2 were to examine the psychometric structure of the pure ABCD items and generate an assessment of traits that explicitly identifies the ABCD content of each trait. To accomplish this goal, we limited our item pool to the 140 items meeting criteria for trait purity only. This method of item retention is equivalent to (i) selecting up to 7 items for each of the ABCDs for each domain from the pool of items meeting criteria for relative purity to the extent it was possible, and then (ii) supplementing this with items from the pool of items meeting criteria for trait purity when there were not 7 items meeting criteria for relative purity available for each respective trait-ABCD combination. Each trait scale thus contained 28 items and each ABCD scale within each trait contained 7 items. One concern about the 7-item scales was that this procedure might group items within the same scale that, despite assessing the same trait, do not share much conceptual similarity. For example, in the group of agreeableness-behavior items, the items “comment loudly about others” and “show my gratitude” may assess different aspects of agreeableness. This possible lack of homogenous conceptual content within certain scales would likely decrease estimates of internal consistency, and thus we note in our discussion of the results where this might have occurred.
We conducted a large-scale web study in order to examine the psychometric structure of the items meeting criteria for trait purity. Most web-based studies are limited to short questionnaires or basic cognitive tasks (Greenwald, Nosek, & Banaji, 2003); however, the technique that the sapa-project.org personality survey uses (Condon & Revelle, 2014; Revelle, Wilt, & Rosenthal, 2010) allows for collection of a large number of items and the ability to examine the covariance of the items. From a large set of items, a smaller subset of items are presented to any one subject. With random sampling of the items, all possible pairs of items are eventually presented, which allows for construction of correlation matrices and formal psychometric analyses. This technique may be thought of as a variant of planned missingness designs (e.g., Graham, 1955; Lord, 1955) and has been used in the private domain to develop educational (SAT and GRE) and occupational (nursing licensure exams) tests with great success, and it was applied to the ABCD items identified in the previously described content analysis.
Participants were 11,489 individuals (7,597 women) who completed the http://sapa-project.org personality survey3 from October, 2012 to January, 2013. The mean age of participants was 25.53 (Median = 21.0, SD = 10.2). Participant characteristics were remarkably similar to what has been reported in previous studies using this survey (Evans & Revelle, 2008). Education levels were: less than 12 years (3.1%), high school graduate (4.9%), some college but did not graduate (5.1%), currently attending college (54.5%), graduated from college (15.8%), and graduate or professional school (6.6%). Countries of origin were: USA (79.9%), Canada (5.1%), UK (2.0%), Australia (1.0%), India (0.8%), and other countries (11.2%). The survey may be accessed directly through the http://personality-project.org website or through other search engines. Daily visitors to this site see a small notice about a web-based personality test that offers personal feedback. From these visitors, as well as those who come from the results of on-line search engines, approximately 100 per day visit the site and complete the web-based survey.
In Study 1, a total of 205 items met at least one of the definitions for purity (relative purity, trait purity) with regard to ABCD content. Each participant was presented with 60 pure ABCD items (randomly sampled from the set of 205 items). Following completion of the survey, participants were given computer-generated feedback on their Big 5 scores based on the 60 items that they completed. It is important to clarify that the scores that were used for participant feedback were not used for further analyses in this study, as each individual received a unique combination of items to generate the scores used for feedback.
Using the psych (Revelle, 2014) package in R (R Development Core Team, 2014), we employed the SAPA technique (Condon & Revelle, 2014; Revelle et al., 2010) to generate a correlation matrix containing the 140 ABCD items that met criteria for trait purity only.4 With random sampling of the items, all possible pairs of items were eventually presented together. On average, the number of pairwise administrations on which pairwise correlations were based was 381 (min = 234, max = 637). The minimum number of pairwise administrations among items provided sufficiently high stability in the covariance matrix (Kenny, 2014). This allowed for construction of pairwise correlation matrices, which in turn allowed for formal psychometric analyses examining the structure of the items.
Given the sampling design, there were no participants with complete data. Thus, psychometric analyses were done based upon the pairwise complete correlation matrix of 140 items that met criteria for trait purity. From this correlation matrix, we were able to generate correlations within and between different composites of multiple items (these item composites will be referred to hereafter as scales for the sake of simplicity and brevity). The scales we constructed reflected each trait and each ABCD domain within each trait, for a total of 25 scales in all. We calculated correlations for items within each scale and the between-scale correlations from the composites of the raw item correlation matrices using the scoreOverlap function in the psych package (Revelle, 2014) in R; the code for this operation can be found by typing “scoreOverlap” in the R console with the psych package loaded.
Correlations between scales are shown graphically as a correlation plot (also called a heat map) in Figure 3. Correlation plots are particularly useful tool for revealing structure, as they are shaded to indicate the magnitude and direction of correlations between scales. Stronger positive correlations are shown as increasingly darker blue, and stronger negative correlations are shown as increasingly darker red. Correlations above the diagonal are corrected for attenuation based on alpha reliabilities, whereas correlations below are the raw (unattenuated) values. Correlations between trait domains and their constituent ABCDs were corrected for overlap due to their shared error variance (Bashaw & Anderson Jr., 1967; Cureton, 1966). This procedure (implemented in the scoreOverlap function) involves subtracting the overlapping item variance and then replacing this with the best estimate of common variance, the squared multiple correlation for that item.
Alpha reliabilities appear on the diagonal of the correlation plot. Although most reliabilities were acceptable, some scales (Ab, Ad, Cc, Sc, Oa, and Ob) had reliabilities between .47 and .54. We previously noted the possibility that a relatively low amount of homogenous conceptual content within facet scales would likely decrease estimates of internal consistency, and this appears to have been the case for these scales.
The correlation plot shows a number of results that illuminate the structure of the ABCD items and scales for items meeting criteria for trait purity. Starting with the Big Five traits (in the upper left corner), correlations between traits were positive and were of small magnitude, with the exception of the moderate correlation between extraversion and openness. These results are in line with previous research investigating associations among Big Five traits (Digman, 1997; DeYoung, 2006; Mount, Barrick, Scullen, & Rounds, 2005). As was to be expected, traits correlated highly with their respective ABCD domains, and ABCD domains within each trait were positively correlated with each other.
Additionally, we sought to determine the factorial structure of the 20 ABCD scales. In order to determine the number of factors to extract, we conducted a Very Simple Structure (VSS) analysis (Revelle & Rocklin, 1979) and also relied on Velicer’s (1976) minimum average partial correlation (MAP) test. Results from both analyses indicated that a 5-factor solution was optimal. Thus, we conducted a minimum residual factor analysis of the intercorrelations between the 20 ABCD scales, extracting 5 factors and using oblimin rotation. We set N equal to the average number of pairwise associations between items (381). The results revealed a structure that closely resembled the Big Five (see Table 5). These findings together with the examination of the correlation plot attest to the validity of composites of items as reflections of their respective Big Five traits.
In the introduction, we noted that scales including unbalanced ABCD content might create biased estimates of covariance amongst item sets with different relative saturations of ABCD content.5 A number of results from the correlation plot and factor analysis suggest that this is a likely scenario. Take for instance the unattenuated correlation of .42 between conscientiousness cognition (Cc in the plot) and emotional stability (Sc), and the unattenuated correlation of .48 between Cc and Oc. In the factor analysis results, Cc has a cross-loading on the S domain of .50, and the S domain is defined primarily by Sc; Cc has a cross loading of .30 on O, which is defined primarily by Oc. This pattern holds for the Oa-Aa correlation (Oa has a high loading on O) and the Sd-Cd correlation (Sd has a high loading on C). As such, when trait scales are created without attention to ABCD content, it would likely go unnoticed that their correlations might reflect covariance of ABCD content rather than trait domain content. Creating scales with balanced amounts of ABCD content shows that one potential source of overlap between traits is their ABCD content.
The high correlations between each ABCD component and their respective traits, as well as the factor analysis showing a Big-Five structure are encouraging, but they do not rule out the possibility that ABCD component scales are simply four different ways of measuring the same trait construct. That is, perhaps it is simply the Big-Five trait content within each ABCD scale that accounts for the coherence of ABCD scales measuring the same Big-Five trait. Is there anything to be gained by grouping items according to their ABCD content? The following analysis aims to answer this question by testing whether items within ABCD scales cohere more closely with one another than they do with their broad, trait measure. The composite scales measuring each Big-Five trait (28 items each) and each ABCD scale (7 items each) shown in the correlation plot were correlated with each item (correcting for item overlap) from their respective traits. The correction was done using the methods of Bashaw and Anderson (1967) and Cureton (1966), which were described previously.
The corrected correlations are shown in Tables 6, ,7,7, ,8,8, ,9,9, and and10.10. These tables also show the average of the absolute values of the correlations of items in each scale with their respective domain scale and each facet scale within the domain (correlations between all items and all scales can be found in the online supplemental material). These results clearly showed that (i) items tended to have their strongest correlations with their respective ABCD scale, that (ii) items typically were more strongly correlated with their composite trait scale (e.g., the correlation of items in the Aa scale with domain A) than they were with ABCD scales that did not share the item’s ABCD content (e.g., the correlations of items in the Aa scale with the Ab, Ac, and Ad scales), and that (iii) the average of the absolute values of the correlations for a particular set of items were highest with its respective facet scale (e.g., the average absolute value of the correlation of items assessing Aa was higher with the Aa scale than with the A, Ab, Ac, and Ad scales). These findings together attest to the ability of the ABCD scales to measure their respective ABCD content with good fidelity, and to the usefulness of the composite trait scales as measures of the broad Big-Five.
The trait enterprise appears to be thriving, yet that has not prevented some from leveling the criticism that little is known about the conceptual nature of individual traits. This criticism has been applied most often to traits included in the Big-Five (Deary, 2009; Lamiell, 2000; Mayer, 2001; Tellegen, 1991; Pytlik Zillig et al., 2002). This may seem surprising given the ubiquitous use of inventories such as the NEO-PI-R and AB5C, however, it might be that the perceived consensus about the structure of traits itself has contributed to stifling critical inquiries into their very nature (Deary, 2009).
The central thrust of the argument that individual traits are not conceptually sound is that, by operationally limiting personality traits as the trait-descriptive adjectives, sentences, paragraphs, etc. in natural language (Costa & McCrae, 1998), trait researchers have bypassed the conceptual problem of defining the range and scope of personality traits (Deary, 2009). Put another way, the use of natural language descriptors has allowed trait researchers to avoid specifying the contents of the traits in psychological terms (Mayer, 2001). To this point, Tellegen (1991) notes that the “psychological nature” (p. 30) of Big Five traits has yet to be specified. This dissatisfaction about the conceptual nature of the Big-Five/FFM has led to questions about “Just what are these Big Five?” (Lamiell, 2000, p. 2), and “What do we assess when we assess a Big Five trait?” (Pytlik Zillig et al., 2002, p. 846). These questions are eerily reminiscent of Carr and Kingsbury (1938)’s observation in the nascent years of traits theory: “psychology is at present groping somewhat blindly about because of the absence of any definite and accepted principles of orientation in reference to the concept” (p. 524). Over 70 years later, Roberts (2009, p. 140) noted that “a valid criticism of many modern personality trait theorists and researchers is that they have not provided a deeper analysis of the constituent elements that make up traits.”
What should be done in the face of empirical triumph but conceptual ambiguity? Deary (2009) poses two options. The first is to “extract defeat from the jaws of partial victory” (p. 104), meaning that personality researchers could abandon traits even though they have been shown to be reliable, stable, predict important outcomes, etc. The second is to “declare victory and withdraw prematurely” (p. 104), meaning that it is possible to ignore conceptual issues and celebrate the empirical achievements of traits. Given these options, it is not surprising that Deary settles on a compromise of continued efforts to refine the phenotypic structure of traits in light of discoveries about their conceptual nature. In the same vein, Block (1995) called for an iterative process of thorough conceptual analysis and focused empirical analyses aimed at refining and calibrating the Big Five trait constructs.
The current project was an effort to heed the advice of Deary (2009) and Block (1995). Study 1 aimed to elucidate the conceptual nature of traits by examining the ABCD content of the items included in the most commonly used inventories to assess the Big-Five (the IPIP versions of the NEO-PI-R and AB5C). In general, traits contained more ABC content compared to D content. With the exception of agreeableness, which contained relatively balanced amounts of ABC content, other traits clearly had a predominant content. Conscientiousness and extraversion were defined by behavioral content, emotional stability was defined by affective content, and openness was defined by cognitive content. Traits contained varying amounts of other contents in addition to their dominant content. In sum, the Big 5 traits seem to be very different from each other in basic dimensions of structure and substance.
What does this mean for trait theory? Taking a step back, consider Allport’s (1958) first and fundamental assumption regarding traits. “But let us not join the camp of skeptics who say an individual’s personality is “a mere construct tied together with a name” - that there is nothing outer and objectively structured to be assessed. No scientist, we think, could survive for long if he heeded this siren song of doubt, for it leads to shipwreck.” (p. 246). Stated bluntly, just as serious astronomers assume that stars are real, serious trait theorist assume that “traits are real” (Funder, 1991, p. 32), and treating traits as hypothetical constructs would quickly lead to the degradation of trait science. If it is assumed that (i) there is a very basic reality to Big 5 traits, and that (ii) the basic nature of traits is aligned with the conceptual definition of traits as reflecting ABCD dimensions, then the results from Study 1 suggest that operational definitions of traits in major inventories may miss the mark, and miss it in a variety of different ways according to the trait in question. The operationalized traits that have been studied for so long and have accumulated such a strong scientific reputation are, in some sense, not what we thought (or at least have been saying) that they were.
Is this problematic, or is it simply a matter of readjusting the definition of Big Five traits to more accurately reflect how they have been operationalized? That is, could the potential problem of differential amount of ABCD contents across and within traits be rectified by defining Big-Five traits individually by their constituent elements (i.e., Extraversion could be defined as a stable and consistent patterns of behavior and affect that distinguishes among people, whereas Openness could be defined as a stable pattern of cognition that distinguishes among people, etc.). This would seem to not only ignore Allport’s warnings regarding defining traits as constructs tied together with a name, but in doing so to tacitly accept a circular definition for traits as “what trait inventories measure.” It would be ignoring the issue of conceptual clarity with regard to traits.
Another goal of Study 1 was to identify an item pool with the potential to yield an assessment of Big 5 traits that matched the conceptual definitions of traits. Such a measure would need to be more explicitly and clearly aligned with the purported psychological components of traits, namely, the ABCDs. Thus, two different measures of “purity” were operationalized in order to identify a subset of items included in the IPIP NEO-PI-R and AB5C that were relatively “pure” measures of A, B, C, and D content.
The first definition of purity was based on an item containing relatively high amounts of one content in comparison with other with regard to the other contents (relative purity) and yielded 133 items that were spread relatively evenly across traits but unevenly with regard to content within traits. The uneven distribution of items measuring pure ABCD content within traits was due to the fact that most pure items for each trait assessed that trait’s dominant content (e.g, most pure Emotional Stability items were in the domain of affect, most pure Openness items were in the domain of cognition). Constructing an assessment of Big-Five traits using these items, although possible, would only exaggerate the unbalanced representation of ABCD content within traits.
The second definition of purity was that, for each trait, items that contained the highest degree of each ABCD content (the top 7 items) were selected as relatively “pure” indicators of that content for each trait (trait purity). This method yielded 140 items and forced an even distribution of pure ABCD items within traits, thus giving at least the potential to yield an assessment of the Big-Five with a balanced distribution of ABCD content within each trait. However, this method did not guarantee such an assessment, as the structure of the items may have been prohibitive to forming scales reflecting Big-Five traits (or ABCD facets of traits) from the selected items. Therefore, we found the correlation matrix of the 140 items that met criteria for trait purity and from this were able to find the correlation matrix of scales consisting of those items.
The structure of those scales was examined in a large-scale, online study. Correlational and factor analyses of the scales suggested a Big Five structure, and that the ABCD domain scales within each trait could be meaningfully differentiated from each other and their corresponding Big-Five trait. These findings justified creating ABCD scales.
So, where does the ABCD approach, and specifically, the ABCD scales assessment fit within the ever-growing repertoire of tools available for trait assessment? It is important that they avoid the problem described eloquently by Funder (2001): “Personality psychology has been long beset by a chaotic plethora of personality constructs that sometimes differ in label while measuring nearly the same thing” (p. 200). That is, it is important that the ABCD component scales are not redundant with other taxonomies.
We believe that the ABCD scales are one answer to a criticism of trait taxonomies that has been voiced repeatedly but rarely receives any empirical attention. Namely, there has been much confusion, if not controversy, concerning the key ingredients, components, constituencies of personality traits (Yang, Read, Denson, Xu, Zhang, & Pedersen, 2014). The ABCD scales cut through this confusion by clearly defining the dominant psychological contents within each scale. Although other scales may also lean toward one or two ABCD contents (Wilt & Revelle, 2009), this was an unintentional result of the scale-development process (Pytlik Zillig et al., 2002). For the first time, when assessing Big-Five traits, researchers can have confidence that they are assessing in a balanced and relatively independent way the ABCD contents of the traits. This realization might have important implications for construct validity, if the ideas presented by Loevinger (1957) are taken seriously. She went so far as to suggest that the construct validity of personality tests rests partly on showing that the content of measurement instruments corresponds to theoretical content. It could be argued that the ABCD scales are the very first assessment of the Big-Five to take this to heart with regard to making explicit the predominant type of psychological content measured in each scale.
This seems to be such a straightforward contribution that it is somewhat surprising that it could be thought of as a novel approach over a century after trait assessment began in earnest. Yet, it could also be argued that unbalanced operationalization of traits with regard to ABCD content actually reflects better the true nature of traits. That is, although the ABCD approach could be useful as an overarching conceptual framework for traits, perhaps individual traits are more about one or two ABCD contents. For example, perhaps extraversion is mostly about behavior and affect, and neuroticism is all about affect. If this were true, then our ABCD scales would be operationalizing trait content of each individual trait too broadly.
Questions regarding how to best conceptualize and operationalize traits are difficult to answer. These questions are not made any easier given that the lexical hypothesis – the idea that fundamental traits are encoded in the natural language – is agnostic about the whether the ABCDs should be equally present in the traits that emerge from its application, namely the Big Five (Goldberg, 1990). As such, perhaps our effort to create balanced ABCD scales was misguided. If balancing ABCD content across traits were not a goal to strive toward, this would have implications for trait theory and assessment. Theories of the causes and consequences of traits might then be wise to align their proposed explanations and outcomes of traits with the predominant ABCD contents of each trait. Assessments could be developed in order to focus on the predominant contents as well, and the ideal method of assessing different traits would likely differ according to content (e.g., observational assessment for behavioral content, self-report for affective content). Yet, at this point, it is unclear whether balanced or unbalanced assessment of ABCDs should be preferred.
We are optimistic that about at least two ways in which future research might help to resolve issues related to the degree to which ABCD content should be included in trait assessments. First, we believe that studies of the psychometric structure of items assessing Big Five traits might serve as an important role in this endeavor. In previous examinations of trait structure (e.g., Costa & McCrae, 1992b; DeYoung et al., 2007; Eysenck & Himmelweit, 1947; Hofstee et al., 1992; John et al., 1991), item pools to assess the Big Five were selected without regard to specific ABCD content of the items. Rather they were chosen as markers of the trait domains and facet domains (in the case of developing markers for lower-order dimensions). Yet, if an item pool were developed with the explicit aim of including ABCD markers of each trait, then it would be possible to conduct structural analyses (e.g., factor analyses, cluster analyses, etc.) of those items and see what dimensions emerged. The dimensions emerging from such an analysis would make the ABCD content of traits more clear. We believe that the items we identified in Study 1 are a good start toward developing such an item pool.
However, the dearth of items reflecting high amounts of D content, and the unbalanced number of ABCD items by trait identified by our first definition of purity are limitations of this item set. Thus, additional “pure” ABCD items would need to be generated prior to the analyses we advocated above. Such analyses could help to answer the question of whether it is advisable to assess traits as including a balance of ABCD content, or if some traits really include an imbalance of ABCDs.
Second, we believe that it is necessary for our ABCD scales (or any scales generated in the future that measure ABCD content explicitly) to exhibit incremental validity over traditional measures of the Big Five for predicting important criteria. That is, the utility of our assessment partially lies in its correlates, because those associations show how the assessment actually functions (Gough, 1965). It might be particularly useful to pit the ABCD scales against traditional Big Five measures for predicting psychological experience in daily life. A longstanding goal of personality is to characterize psychological experience in in naturally occurring environments, which may be understood as the coherent patterning of how people feel (A), act (B), think (C), and want (D) throughout their daily lives (Allport, 1937; Fleeson, 2001; Pervin, 1994). Although predicting ongoing functioning is extolled as a gold standard in personality research, this goal has too seldom been realized (Craik, 2000; Funder, 2001). The ABCD approach to trait assessment, because it is delineated into the domains that mirror dynamic personality processes, may prove more valuable in this regard than traditional personality assessments. For example, we might expect that trait items assessing a specific ABCD domain would be predictive of that domain in daily life above and beyond traditional domain measures (e.g., the A items assessing extraversion might predict positive affect in daily life above and beyond domain measures of extraversion).
There is good reason to think of ABCDs as primary parts (Mayer, 2001) in personality, but they by no means constitute an exhaustive list of the attributes and characteristics that make up traits. Previous item-content analyses of personality inventories have included physical characteristics and environmental factors (Angleitner et al., 1986; Werner & Pervin, 1986). Recently, Yang et al. (2014), inspired by Lewinian ideas on behavior (Lewin, Adams, & Zener, 1935), proposed that situations, behaviors, and explanations of the situation-behavior fit are the fundamental components of traits. Future research investigating such contents in addition to the ABCDs might lead to advances in ideas about the conceptual content of personality constructs.
The use of percentages to describe contents has been employed previously (Pytlik Zillig et al., 2002), but doing so may have artificially introduced negative correlations across the ABCD domains. That is, by using percentages, it is difficult for items to have high amounts of more than one ABCD content. For example, the items “yell at people” and “cry easily” were each rated as having high but relatively equal amounts of affective and behavioral content, and as such they were not selected as pure items in our study. Future research assessing the degree to which ABCD contents are present in items independently of each other domain could yield different results with regard to item purity and therefore different answers about the relative ABCD content of personality traits. It is important to investigate the conceptual content of traits in a variety of ways in order to see which findings stand the stringent test of replicability.
Our reliance on self-report in Study 2 comes with a number of well-known limitations (Paulhus & Vazire, 2007). People may be motivated to provide inaccurate reports in order to maintain consistency, for self-enhancement, or for impression management; even when people are motivated to present themselves accurately, memory biases and unconscious self-deception may limit the veracity of self-reports. The validity of self-reports may also suffer from response sets such as socially desirable responding, acquiescent responding, and extreme responding. Additionally, it has been argued that observer reports of personality - a person’s reputation - are the best predictors of performance (Hogan, 2007). However, there is an overwhelming amount of research linking subjective self-report data to more objective outcomes (Ozer & Benet-Martinez, 2006; Roberts et al., 2007), including predicting daily behavior just as well as observer and “expert” reports (Fleeson & Gallagher, 2009; Noftle & Fleeson, 2010; Vazire & Mehl, 2008). Moreover, for ABCD data, self-reports are thought to be particularly appropriate. Although the affective, cognitive, and desire components of traits may be inferred from observations of behavior, ACD components are held within individuals. Therefore, self-report is probably the most direct way to assess those components. This observation is not meant to demean peer-reports, direct observation, etc. (Baumeister, Vohs, & Funder, 2007) as methods for collecting data; those methods are merely beyond the scope of the present paper and should be employed in future efforts to validate the results presented herein.
The items meeting criteria for relative purity and trait purity were not ideal for the purposes of scale development. The items meeting criteria for relative purity were spread differentially across ABCD domains of traits, whereas the items meeting criteria for trait purity were differentially pure across ABCD domains according to trait (e.g., the affect items for emotional stability were highly pure in terms of affect, whereas the desire items for emotional stability were less pure in terms of desire content). Additionally, the reliabilities of scales created from items meeting criteria for trait purity were relatively low in some cases, perhaps because of a lack of homogenous conceptual content within facet scales. These results were was likely to be expected given that existing item inventories were not designed to measure pure ABCD contents, but that should not stand in the way of future research aimed at developing items with high purity in each trait domain across ABCD contents. Indeed, we believe that efforts to develop ABCD items could extend beyond assessing Big Five domains to their more narrow facets. These are probably the most logical “next steps” for research aimed at refining assessment of the ABCD components of traits.
The overarching reason for the research presented in this manuscript is synonymous with the unending conceptual task of personality psychology in general, to move ever-so-gradually toward a greater understanding of the whole person (Block, 2010; Funder, 2009). Specifically, the study of personality structure and its related functions is important for organizing and defining the parts of personality in relation to each other so as to move away from “all influences all” (Mayer, 2001, p. 459) position, which, although scientifically plausible, represents an immature theoretical state of affairs. This manuscript presented a system for personality organization, the ABCDs, that has also been proposed as the organizing system for individual psychological reality (Lewis-Fernandez & Kleinman, 1994). The ABCDs are, however, not meant to be a perfect or comprehensive model because all models at their best are only approximations of truth (Giere, 1999). “Every scientist in the back of his mind takes it for granted that even the best theory is likely to be an approximation to the true state of affairs.” (Meehl, 1990, p. 113). The ABCDs are rather meant to advance the organizing framework for personality because they may be useful in moving personality science toward a better understanding of various parts of a whole (i.e., the person), which is all that is asked of any scientific model (Giere, 1999; Shweder, 1999). The studies included in this manuscript contribute to this broad aim by facilitating a better understanding of what traits are and how they are operationalized.
We are grateful for funding support from the National Institute for Mental Health, National Research Service Award Grant # F31-MH093041.
1Emotional stability/neuroticism and openness/intellect will be used interchangeably in this paper. The names emotional stability and intellect are typically associated with the Big-Five tradition (Goldberg, 1990), whereas the names neuroticism and openness are more in line with the Five-Factor Model (FFM) of personality (McCrae & Costa, 2008).
2Following Pytlik Zillig et al. (2002) the analyses examining between and within trait differences in ABCD content were conducted on the percentage data. However, this may be problematic as percentage data are constrained to values between 0–100. Due to this constraint, percentage data may not be normally distributed and are not free to vary widely about the mean, violating assumptions of parametric statistical tests. Therefore, all percentages were transformed using the arcsin transformation, which moves very low or very high values towards the center, giving them more freedom to vary. Analyses conducted on the arcsin transformed percentages produced the same results as analyses on the non-transformed percentage data.
3At the time the test was taken (from October, 2012 to January, 2013) it was located at http://test-personality-project.org. It has since been moved to the sapa-project.org address.
4Thus, 65 items from the set of 205 were excluded because they only met criteria for relative purity; items meeting criteria for relative purity were less amenable to aggregation into scales assessing the Big Five or ABCD facets of the Big Five due to the imbalance of items across traits and the imbalance of ABCD content within traits and so were not included in the correlation matrix.
5We would like to thank reviewer Tom Booth for this idea and for suggesting that we discuss the following results in detail. We would also like to thank the two other reviewers of our paper, one Dustin Wood, and one anonymous reviewer, for their helpful comments.