PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
Pers Individ Dif. Author manuscript; available in PMC 2010 April 8.
Published in final edited form as:
Pers Individ Dif. 2008 November 1; 45(7): 649–654.
doi:  10.1016/j.paid.2008.07.009
PMCID: PMC2851096
NIHMSID: NIHMS189214

Reliability and Concurrent Validation of the IPIP Big-Five Factor Markers in China: Consistencies in Factor Structure between Internet-Obtained Heterosexual and Homosexual Samples

Abstract

Previous studies have suggested the cross-cultural generalizability of a 5-factor structure for personality traits. In this article, we analyzed the utility of 2 versions (100-item and 50-item) of the IPIP Big-Five factor markers in both heterosexual (N = 633) and homosexual (N = 437) samples in China. Factor analysis within versions showed that both versions of these IPIP measures showed clear 5-factor orthogonal structures that were nearly identical to the American structure in both subject samples. The reliabilities of the five factors were quite high except for the 50-item measure of Agreeableness. The part-whole correlations between the 100-item and 50-item factors were high, as were the factor congruence coefficients between the heterosexual and the homosexual samples. Both versions of the IPIP Big-Five factor markers were strongly correlated with the scales from the Big Five Inventory (BFI: John, Donahue & Kentle, 1991), thus providing some concurrent validation in a Chinese context.

Keywords: Personality structure, IPIP Big-Five factor markers, Sexual orientation, Long vs. short forms

1 Introduction

Goldberg (1999) has argued that “the science of personality assessment has progressed at a dismally slow pace since the first personality inventories were developed over 75 years ago.” Goldberg attributed the seeming lack of progress in part to the policies and practices of commercial inventory publishers, given that most broad-bandwidth personality inventories are proprietary instruments. As a consequence, there are many inconveniences to researchers, who require permission from the copyright holders and are charged for each questionnaire used. Goldberg therefore proposed an international collaboration that places a large set of personality items in the public domain, thus freeing researchers from the constraints imposed by copyrighted personality inventories. Hence the International Personality Item Pool (IPIP) was born. Over the years, the IPIP web site has provided an ever increasing set of measures, all in the public domain, available to scientists world wide. The IPIP has over 2500 items, and new sets of items are added each year. IPIP items have been constructed as public domain equivalents of the constructs in a variety of popular commercial personality inventories including the NEO-PI-R (Costa & McCrae, 1992), 16 Personality Factor Questionnaire (16PF: Conn & Rieke, 1994), California Psychological Inventory (CPI: Gough & Bradley, 1996), and the Hogan Personality Inventory (HPI: Hogan & Hogan, 1992). Currently, items from the IPIP have been translated from English into more than 25 other languages, and the rate of publications using IPIP scales has been increasing rapidly.

The IPIP Big-Five factor markers (Goldberg, 1992) consist of a 50 and 100-item version which are available on the IPIP web site. These markers have been translated into more than 10 languages, such as Arabic, Bulgarian, Chinese, Croatian, and so on (Goldberg et al., 2005). But there are only a few reports on the characteristics of the IPIP Big-Five factor markers in other languages and cultures. The IPIP have good internal consistency and relate strongly to major dimensions of personality assessed by the NEO-FFI and EPQ-R (Gow et al., 2005). Mlačić and Goldberg (2007) studied a Croatian version of both the 100-item and the 50-item versions of the IPIP Big-Five markers; both self-reports and peer ratings in large Croatian samples of research participants showed clear 5-factor orthogonal structures that were nearly identical to the American structure.

In the present study, we examined the consistency and concurrent validation of the IPIP Big-Five markers in China. To add generality to the findings from our study, we used two types of samples, which past research has shown to differ in Big Five factor traits (Lippa, 2005). Lippa (2005) synthesized the results of eight studies showing that gay men scored higher than heterosexual men in Agreeableness, Conscientiousness, Neuroticism, and Openness to experience. Heterosexual women scored higher than lesbians in Neuroticism, whereas lesbians scored higher than heterosexual women in Openness to experience. Analyzing a large international data set generated by a recent BBC Internet Survey, Lippa (2008) replicated these results across cultures in a large and diverse sample.

Consequently, we used separate samples of heterosexual and homosexual adults to examine the utility of the IPIP Big-Five factor markers in the Chinese context. We analyzed a Chinese version of both 100-item and the 50-item versions of the IPIP Big-Five factor markers, in both heterosexual and homosexual samples obtained from the Internet.

2 Method

2.1 Questionnaires

2.1.1 IPIP Big-Five factor markers (Goldberg, 2001)

The IPIP Big-Five factor markers consist of a 50-item and 100-item inventory which can be freely downloaded from the internet (Goldberg, 2001). The current study makes use of the 100-item version consisting of 20 items for each of the Big-Five personality factors: Extraversion (E), Agreeableness (A), Conscientiousness (C), Emotional Stability (ES), and Intellect (I). We administered the IPIP items with a 5-point, Likert-type scale ranging from 1 (very inaccurate) to 5 (very accurate) as in the original instrument (Goldberg, 1999). Because all of the items in the 50-item version (short form) of the IPIP instrument are included in the 100-item version, it is possible to analyze both versions from a single administration of the 100 items. The 100 IPIP Big-Five factor markers were translated into Mandarin Chinese (and then back-translated and checked for accuracy) by Xian Xu at the University of South Florida.

2.1.2 Big Five Inventory (BFI, John, Donahue, & Kentle, 1991)

The other instrument used in this study was the Big Five Inventory (BFI: John, Donahue, & Kentle, 1991). The BFI is a 44-item self-report inventory designed to assess the Big Five Factors of personality: Extraversion, Agreeableness, Conscientiousness, Neuroticism, and Openness to Experience. The BFI scales have shown substantial internal consistency, retest reliability, and clear factor structure, as well as considerable convergent and discriminant validity with longer Big Five measures (Benet-Martinez & John, 1998). Items on this scale were also scored on a Likert scale (5-point) from “disagree strongly” to “agree strongly.” Considering the length of the questionnaire and the participants’ patience, only a subsample of the heterosexual participants were administered the BFI.

2.2 Participants and procedures

Because it is difficult in China to recruit homosexual samples, we used the Internet. We recruited participants in a number of web sites including homosexual forums and homosexual chat rooms, and our research participants entered requests like “free personality test” in each of these web sites. Persons interested in completing the “personality test” sent us their e-mail addresses. Then the questionnaires were sent to the participants as e-mail attachments and returned to our e-mail address (again as e-mail attachments) after they had been completed. Finally, we sent their personal “personality test” feedback to all participants by e-mail. Based on the e-mail addresses, some duplicate questionnaires were excluded from our analyses. In total, we received 1,070 seemingly valid questionnaires between August and October of 2007. The heterosexual sample included 633 participants, and the homosexual sample included 437 participants. Within the heterosexual sample, 301 (105 men and 196 women) completed both the IPIP Big Five and the BFI.

Heterosexual sample

The heterosexual sample included 633 participants from 109 cities in China, 216 men (34%) and 417 women (66%). The mean age of the sample was 23 years and 3 months (sd = 3 years and 8 months), ranging from 13 years to 45 years. In response to a question about their occupations, 52% were currently students, and the other 48% were engaged in all kinds of occupations. In respect to education level, 14 (2%) reported junior high school level or less, 48 (8%) senior high school level, 483 (76%) college level, and 82 (13%) postgraduate level or more.

Homosexual sample

The homosexual sample included 437 participants from 111 cities in China, 220 gay and 217 lesbian. The mean age of the sample was 23 years and 5 months (sd = 4 years and 5 months), ranging from 13 years to 46 years. In response to a question about their occupations, 42% were currently students, and the other 58% were engaged in all kinds of occupations. In education level, 11 (2.5%) were junior high school or less, 64 (14.6%) senior high school level, 323 (74%) college level, and 39 (9%) postgraduate level or more.

3 Results

3.1 Factor structure of the 100 IPIP items

Heterosexual sample

To establish whether the expected 5-factor structure of the 100 IPIP items was present in the current Chinese data-set, five principal components were extracted and rotated using the Varimax procedure. The plot of the first 15 eigenvalues is presented in Figure 1. There was a clear discontinuity in the sizes of the eigenvalues between the fifth and the sixth factors in the scree plot. It suggested the extraction of 5 factors, accounting for 38.2% of the variance. The factor loadings are available from the first author. Of the 100 items, 93 (93%) loaded as expected based on the original findings in an American community sample (Goldberg et al., 2005).

Figure 1
Plot of the first 15 eigenvalues on 100 items in both heterosexual and homosexual samples

All Extraversion items loaded highest on the same factor, as did all of the Conscientiousness items. For the factor of Agreeableness, 15 items loaded most highly on the same factor. The item “Insult people” loaded highest on the Intellect factor. 4 items (“Am interested in people, Am not really interested in others, Am hard to get to know, Show my gratitude”) intended as a measure of Agreeableness loaded most highly on Extraversion. 19 of the Emotional Stability items loaded together, and one item (“Am relaxed most of the time”) loaded highest on the Conscientiousness factor. 19 of the Intellect items loaded together, while one item (“Try to avoid complex people”) loaded on Extraversion.

Homosexual sample

The first 15 Eigenvalues are also shown in Figure 1. The 5 factors explained over 35% of the total variance. (We also examined the six-factor solutions, but the sixth factors were tiny, including only two items in the 100-item version, explaining only about 3% of the variance; and three items in the 50-item version, explaining only about 4% of the variance.)

Of the 100 items, 94% loaded as expected. For Extraversion, Conscientiousness and Emotional Stability, all of the items had their highest loading on the appropriate factor. 15 of the Agreeableness items loaded together, the difference from heterosexual sample were: (a) The item “Show my gratitude” loaded on Agreeableness as expected; (b) 2 items “Am interested in people, Am hard to get to know” intended as a measure of Agreeableness, had no substantial loadings on any of the factors; (c) The item “often forget to put things back in their proper place” loaded on Intellect. For the factor of Intellect, 19 items loaded as expected, whilst one item “Try to avoid complex people” loaded on Extraversion, the same as in the heterosexual sample.

3.2 Factor structure of the 50 IPIP items

Heterosexual sample

Because all of the items in the short version of the IPIP instrument are included in the long version, it is possible to analyze the two forms from a single administration of the 100 items. We used the same analytical procedures to examine the factor structure of the 50-item version.

A plot of the first 15 eigenvalues is presented in Figure 2. Five factors were extracted from the 50 IPIP items in the heterosexual sample data set. The factor loadings from a varimax rotation of the 50 items are provided in Appendix 1. These 5 factors explained 42% of the total variance. Of the 50 items, 48 (96%) loaded as expected. All 10 of the intended items defined the factors of Extraversion, Conscientiousness, and Intellect, whereas 9 of the 10 intended items loaded most highly on the factors of Agreeableness and Emotional Stability. Only 2 items had their highest loadings on the “wrong” factor. The item “Insult people,” which was intended as a measure of Agreeableness, loaded most highly on the Emotional Stability factor just as it did in the analyses of the heterosexual sample and the homosexual sample with the 100 items. The item “I’m relaxed most of the time” which was intended as a measure of Emotional Stability, loaded slight highly on Conscientiousness, the same as in the analyses of the heterosexual sample with the 100 items.

Figure 2
Plot of the first 15 eigenvalues on 50 items in both heterosexual and homosexual samples

Homosexual sample

The first 15 Eigenvalues are also shown in Figure 2. We also extracted 5 factors to analyze the factor structure of the 50 IPIP items. The 5 factors explained 40% of the total variance. Of the 50 items, 46 (92%) loaded as expected. All Extraversion items loaded highest on the same factor, as did the Emotional Stability items. 8 of the Agreeableness items loaded together; the item “Insult people” loaded on Emotional Stability just as it did in the analyses of heterosexual sample of 50 items and both samples of the 100 items. The item “Make people feel at ease” loaded on Intellect, the same as in the analyses of the homosexual sample with the 100 items.9 of the 10 intended items loaded most highly on the factors of Conscientiousness and Intellect. The item “Shirk my duties” which was intended as a measure of Conscientiousness, loaded slightly higher on Emotional Stability. The item “Spend time reflecting on things” which was intended as a measure of Intellect, loaded most highly on Agreeableness.

3.3 Reliabilities of the long and the short scales in the two types of samples

We calculated the internal consistency reliabilities of long and short versions in the heterosexual and homosexual samples. The internal consistency reliability estimates are shown in Table 1. For the 20-item scales, all alpha coefficients were high, ranging from .87 to .93. For the short 10-item versions, the Agreeableness reliability was lower (.69 and .66, respectively), whereas all others were substantial (ranging from .76 to .87).

Table 1
Internal consistency reliability estimates (Coefficient Alpha) for the Chinese 20-Item and 10-Item IPIP scales in both heterosexuala and homosexualb samples

3.4 Correspondence between the factors from the 100 items and 50 items

To evaluate the similarity between the corresponding factors across different types of samples, we calculated the congruence coefficients between the factors derived in the heterosexual sample and the homosexual samples separately for both the 100 item and 50 item versions. The congruence coefficients are provided in Table 2. All of the corresponding congruence coefficients were high, ranging from .93 to .98, and averaging .96.

Table 2
Congruence coefficients between the corresponding factors in the heterosexual and homosexual samples

We also examined the correlations between the corresponding factors based on 100 and 50 items. These correlations, which are provided in Table 3, were all over .90. The average correlations were .94 in both samples. These results attest to the robustness of the 5-factor structures across the two types of samples, both for the long and the short scales of the IPIP instrument.

Table 3
Correlations between the 20-tem and 10 item scales

3.5 Correlations between IPIP factors and the BFI scales

Correlations between IPIP factors and the BFI scales are presented in Table 4, based on 301 heterosexual participants. There were clear one-to-one relations between all five corresponding factors in both short and long versions. The correlations between the corresponding scales from the BFI and the IPIP long form averaged .68. The correlations between the corresponding scales from the BFI and the IPIP short form averaged .65.

Table 4
Correlations between the BFI and the IPIP scales

3.6 Joint PCA (Principal Component Analysis) of the IPIP and BFI

The scale scores for the IPIP 100 items, 50 items, and the BFI were analyzed by PCA. The scree plot suggested the extraction of five factors accounting for 84% of the variance. The loadings were presented in Table 5. All components were as expected, and no substantial cross-loadings were observed.

Table 5
Factor loadings for the IPIP and BFI scale scores

3.7 Sex-orientation differences in IPIP scale scores

We analyzed the personality differences between the heterosexual and homosexual samples. The mean scores of each of the 5 factors were compared using independent-samples t-tests. The mean scores and standard deviations are shown in Table 6. There was a significant difference in Emotional Stability between gay men and heterosexual men (p < .05) both in the 100-item and 50-item versions; gay men had significantly lower Emotional Stability compared with heterosexual men. There also was a significant difference in Emotional Stability between lesbian women and heterosexual women (p < .05); lesbian women had significantly higher Emotional Stability compared with heterosexual women. There were no significant differences in the other four factors between the heterosexual and the homosexual samples.

Table 6
Means and standard deviations for the IPIP scales in each of the two samples

4 Discussion

This study attempted to validate the IPIP Big-Five markers in Asia. The results of the current study provided substantial support for the generalizability of the 5-factor IPIP structure in a Chinese context. Our results confirmed the factor structure proposed by Goldberg, both in heterosexual and homosexual samples, and for both the long and the short IPIP scales. Only minor deviations from the expected item loadings occurred. The reliabilities of the IPIP scales were high except for the short version of Agreeableness. The factor congruence coefficients between the heterosexual sample and the homosexual sample were high (averaging .96). The 100 items correlated highly with the 50 items. The relations between BFI and IPIP revealed unequivocally clear one-to-one relations between all five corresponding factors in both versions. The joint PCA of IPIP and BFI were also as expected. All these results suggested that the IPIP Big-Five factor markers have cross-cultural concurrent validation.

Although, the results of the current study supported the 5-factor IPIP structure in Chinese samples of both heterosexual and homosexual adults, they were not perfect. Specifically, the Agreeableness factor and the short Agreeableness scale might be improved. Although some items may need to be refined in specific cultures, these seem to be relatively rare, and they do not compromise the overall factor structure.

The internal consistency and concurrent validation of the IPIP Big-Five factor markers have now been validated in the U.S. (Goldberg et al., 2005), Scotland (Gow et al., 2005), New Zealand (Guenole & Chernyshenko, 2005), Croatia (Mlačić & Goldberg, 2007). Now, the utility of the IPIP Big-Five factor markers has been shown in the Chinese culture.

Acknowledgements

Funds for this research were provided by Grant NSKD06011 from Key National Discipline of Basic Psychology, Southwest University in China. Funds for the second author have been provided by Grant AG20048 from the National Institute on Aging, National Institutes of Health, U.S. Public Health Service. The authors would like to thank Xian Xu for translating and providing Mandarin Chinese version of IPIP Big-Five factor markers. We are also indebted to Boris Mlačić for his suggestions on congruence coefficient.

Appendix 1

Five-factor varimax-rotated loadings of the 50 IPIP Items in the heterosexuala and the homosexualb samples

E
A
C
ES
I
ItemsHetHomHetHomHetHomHetHomHetHom
1.63*.61*.03.05−.02−.06−.04.05.21.31
6.56*.52*.07.03.09.09.30.34.02.07
16.73*.69*.02−.03.06.04.11.09−.04.04
21.64*.65*.21.13.09.09.10.02.07.15
26.71*.68*−.03.00−.07−.07.08.07.24.23
31.68*.64*.13.06.06.06.07.01.15.19
36.63*.58*.08.04.05−.03.04−.02−.10.00
41.58*.48*.10−.02.04.01−.03.01.13.25
46.53*.58*.13.13.04.11.16.10−.04−.11
66.72*.70*−.05.12−.15−.14−.02.10.08.05
2−.14−.17.23.09.19.26.34*.30*−.03−.05
7.34.28.40*.32*−.10.02−.02−.10.15.12
12.25.28.56*.51*.09.14.19.15−.08−.04
17−.05−.11.71*.62*.00−.03−.09−.06.17.13
22.27.22.63*.64*.15.08.15.12−.18−.11
27−.05−.10.37*.48*−.11−.09−.27−.19.01.01
32.41.36.52*.52*.02−.01.10.05−.02.03
37.19.04.50*.57*.11.09.05.14.17.13
47−.02.03.44*.49*.09.05−.04.00.42.17
57.24.22.31*.22.02.10.27.21.23.34*
3.04.08.12.06.59*.57*.02−.01.18.19
8.01−.13−.09−.02.66*.65*.08.08−.19−.19
13.00.04.14.36.44*.44*−.09−.15.24.17
18.07.12−.06.04.48*.43*.37.37.02.15
23.13.14.00−.10.63*.54*.10.10.00.13
28−.04−.07−.07−.05.63*.67*.19.08−.12−.10
33−.06.01.08.26.64*.57*.01.00.11.07
38.16.19.06.03.42*.32.30.33*−.01.03
43.04−.06.08.02.67*.63*.11.03−.01.13
53−.03.05.07.09.54*.50*−.06−.07.18.20
4.13.03−.10−.10−.15−.13.68*.64*.05−.03
9.10.02.06.11−.35*−.34.34.35*.20.12
14.03.12−.23−.22−.18−.16.60*.60*−.07−.02
19.19.21−.06−.05.04−.06.62*.57*.12−.01
24.08.09.02.02.03.04.79*.73*.04.02
34.09.08.21−.01.11.03.69*.67*.06.06
44−.04−.09.09.04.18.15.70*.71*−.09−.09
54.08.01.01−.03.16.17.79*.76*−.05−.03
59−.05−.09.21.17.22.17.62*.57*.01−.05
64.18.15−.07−.02.10.01.76*.75*.05−.03
5.25.30−.12.12−.05−.03−.09.00.53*.56*
10−.05−.12.00.01.06.10.17.19.41*.58*
15.10.13.19.21−.03−.04−.10−.11.67*.67*
20−.08−.08.17.14.07.12.12.08.35*.44*
25.26.27−.03−.06.15.16.04−.02.60*.62*
30.11.17.08.21−.06−.01−.01−.04.69*.48*
35.05.16−.03.04.07.09.14−.02.59*.62*
45−.05.06.00−.10−.13.04−.24−.27.35*.45*
55−.06.00.31.35*.18.10−.18−.21.37*.25
65.29.35.04.04.07.09.03−.08.67*.61*

Note. Loadings over .30 are shown in bold.

*The highest factor loading for each variable is indicated with an asterisk.

IPIP = International Personality Item Pool; E = Extraversion; A = Agreeableness; C = Conscientiousness; ES = Emotional Instability; I = Intellect. The number of the items in the table identical with 100 items. Het=heterosexual sample; Hom=homosexual sample

aN=633.
bN=437.

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

Ethical statement

We hereby attest that our treatment of human subjects conforms to all of the requirements of the A.P.A. code of ethics.

References

  • Benet-Martinez V, John OP. Los Cinco Grandes across cultures and ethnic groups: Multitrait-multimethod analyses of the Big Five in Spanish and English. Journal of Personality and Social Psychology. 1998;75:729–750. [PubMed]
  • Conn SR, Rieke ML. The 16PF fifth edition technical manual. Institute for Personality and Ability Testing; Champaign, IL: 1994.
  • Costa PT, Jr., McCrae RR. Revised NEO Personality Inventory (NEO PI-R) and NEO Five-Factor Inventory (NEO-FFI) professional manual. Psychological Assessment Resources; Odessa, FL: 1992.
  • Goldberg LR. The development of markers for the Big-Five Factor Structure. Psychological Assessment. 1992;4:26–42.
  • Goldberg LR. What the hell took so long? Donald Fiske and the Big-Five factor structure. In: Shrout PE, Fiske ST, Fiske W, editors. Personality research, methods, and theory: A Festschrift honoring Donald. Erlbaum; Hillsdale, NJ: 1995. pp. 29–43.
  • Goldberg LR. A broad-bandwidth, public-domain, personality inventory measuring the lower-level facets of several Five-Factor models. In: Marveled I, Deary IJ, de Fruyt F, Ostendorf F, editors. Personality psychology in Europe. Vol. 7. Tilburg University Press; Tilburg: 1999. pp. 7–28.
  • Goldberg LR. International Personality Item Pool. 2001. Web address can be obtained from authors.
  • Goldberg LR, Johnson JA, Eber HW, Hogan R, Ashton MC, Cloninger CR, et al. The International Personality Item Pool and the future of public-domain personality measures. Journal of Research in Personality. 2005;40:84–96.
  • Gough HG, Bradley P. CPI manual. third ed. Consulting Psychologists Press; Palo Alto, CA: 1996.
  • Gow AJ, Whiteman MC, Pattie A, Deary IJ. Goldberg’s “IPIP” Big-Five factor markers: Internal consistency and concurrent validation in Scotland. Personality and Individual Differences. 2005;39:317–329.
  • Guenole N, Chernyshenko O. The suitability of Goldberg’s Big-Five IPIP personality markers in New Zealand: A dimensionality, bias, and criterion validity evaluation. New Zealand Journal of Psychology. 2005;34:86–96.
  • Hogan R, Hogan J. Hogan personality inventory manual. Hogan Assessment Systems; Tulsa, OK: 1992.
  • John O, Donahue E, Kentle R, Institute of Personality Assessment and Research . Technical Report. University of California, Berkeley; Berkeley, CA: 1991. The “Big Five” Inventory – Versions 4a and 54.
  • Lippa RA. Sexual orientation and personality. Annual Review of Sex Research. 2005;16:119–153. [PubMed]
  • Lippa RA. Sex differences and sexual orientation differences in personality: findings from the BBC internet survey. Archives of Sexual Behavior. 2008;37:173–187. [PubMed]
  • Mlačić B, Goldberg LR. An analysis of a cross-cultural personality inventory: The IPIP Big-Five factor markers in Croatia. Journal of Personality Assessment. 2007;88:168–177. [PubMed]