PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
Ethn Dis. Author manuscript; available in PMC 2010 June 16.
Published in final edited form as:
Ethn Dis. 2001 Spring–Summer; 11(2): 251–262.
PMCID: PMC2886583
NIHMSID: NIHMS150924

Development and Reliability of a Telephone-Administered Perceived Racism Scale (TPRS): A Tool for Epidemiological Use

Abstract

The conceptualization of perceived racism as a chronic stressor is relatively new to epidemiology. The Telephone-Administered Perceived Racism Scale (TPRS) captures the complexity of racism within five scales: Experience of Racism (by Blacks as a group and by the respondent), Emotional Responses, Behavioral Responses, Concern for Child(ren), and Past Experiences of Racism. The TPRS was developed for employed Black women. Exploratory factor analyses and tests of internal consistency were completed with 476 Black women, aged 36–53. Factor analyses on their responses to racism yielded five factors: passive emotions, active emotions, passive behaviors, internal active behaviors, and external active behaviors. Alpha reliability values ranged from 0.75 to 0.80 for the active and passive emotions subscales, from 0.59 to 0.69 for the passive behaviors subscaie, and greater than 0.76 for both active behaviors subscales. Alpha reliabilities were 0.82, 0.90, 0.88, and 0.82 for Past Experiences, Concern for Child(ren), Experience of Racism—Personal, and Experience of Racism—Group, respectively. Another 30 Black women were queried for test-retest reliability, with values ranging from 0.61 to 0.82. The TPRS was found to be reliable and should serve as a useful epidemiological tool in the examination of the effects of perceived racism on Black women’s health.

Keywords: Psychometrics, Racism, Reliability, Discrimination, Health Disparity

Introduction

The examination of perceived racism as a potential risk factor for adverse health outcomes is relatively new to epidemiology. The study of racism in health research stems from attempts to understand the reasons for the health disparities between Blacks and Whites. In the United States, being Black or non-White tends to be associated with certain poor health outcomes, with Blacks suffering disproportionately higher rates of infant mortality, preterm delivery, hypertension, stroke, diabetes, and obesity. This disparity persists for many of these conditions even after controlling for the traditional risk factors.1,2 One risk factor that may be associated with many of these chronic diseases is stress; however, previous research on psychosocial stress has included mainly White males, and traditional measures of stress have failed to recognize the stressful potential of racial discrimination.

Racism is “both a) a belief or attitude that some races are superior to others; and b) discrimination based on such a belief.”3 Racism is a unique chronic stressor that may either influence health directly or exacerbate the potential impact of other stressors on health. Because of the influence that experiences of racism could have on an individual’s physical and psychological well-being, the study of everyday occurrences of racism is important.4 These incidents could serve as constant reminders of the inequitable distribution of goods and services among Blacks in the United States. Any effect that racism may have on health is likely to be determined not only by the frequency of racial discrimination, but also by the coping responses one uses to deal with racism. These coping resources allow the body to manage specific demands assessed as taxing or exceeding the individual’s emotional and behavioral resources and to reduce the potential negative effects of the stressor on health.5

Racism has been studied for years in the social and psychology fields.6;7 However, only in recent years has racism been conceptualized as a chronic stressor and a potential risk factor for disease.4,813 The use of psychometrically sound instruments to measure the stress of racism in epidemiological research has been minimal.14 Of the six instruments reviewed by Utsey,15 none were designed for telephone use, a frequently utilized method of data collection in epidemiology. The telephone interview has the potential to achieve a high response rate, and it reduces the likelihood of missing data, since the interviewer will require a response before moving to the next question. Our focus was on the development of a tool to be used in studies of women’s health; therefore, our scale was tested only on women. Questions about workplace discrimination were included in the scale because our target population was working Black women. The term “Black” includes women who identified themselves as either “African American” or “Black.” The scale we developed was designed to measure perceptions of racism, including racial discrimination, as well as an individual’s responses to these experiences. The purpose of this paper is to describe the development of a telephone-administered perceived racism scale (TPRS) for epidemiological use, and to report the psychometric properties of the scale.

Methods

The TPRS is a 61-item multidimensional tool designed for epidemiological use among employed Black women. An outline of the TPRS is provided in Table 1. Time for completion of the interview is approximately 15 minutes. The TPRS was developed using some of the constructs and their items from the 51-item self-administered Perceived Racism Scale (PRS) by McNeilly and colleagues.3,16 Details about the PRS and its psychometric properties are published elsewhere.3,16 The TPRS includes five scales: Experience of Racism (by Blacks as a group and by individual respondents), Emotional Response, Behavioral Response, Concern for Child(ren), and Past Experiences of Racism. The emotional and behavioral responses were assessed for experiences of racism that occurred in two domains, on the job and in public. Through our psychometric testing of the emotional and behavioral response scales, five sub-scales were determined. These subscales characterize responses to racism in terms of active and passive coping responses: passive emotions, active emotions, passive behaviors, internal active behaviors, and external active behaviors.

Table 1
Components of the telephone administered perceived racism scale

To construct the TPRS, several steps were taken: a) initial review by a panel of experts; b) focus groups; c) pretesting; and d) psychometric testing. The National Institute of Environmental Health Sciences Institutional Review Board approved this research.

Panel review

Items and scales on the PRS3,16 were first reviewed by a panel of researchers comprised of middle-aged university faculty from diverse ethnic backgrounds. Questions from the PRS relating to students in academic settings were not included, since our emphasis was on adult women. We also excluded questions pertaining to one’s response to racist statements because the panel reviewers thought there would be little variability in responses. Each of the remaining questions was considered separately and when possible, those questions believed to measure the same construct were combined. After the initial reduction of the PRS, a panel of female, multi-ethnic researchers reviewed the content to be included in the TPRS and recommended the revision of some questions and the inclusion of new questions.

Focus groups on perceived racism

Focus groups evaluated the content and face validity of the TPRS and further developed the content of the scale.17 A convenience sample of participants volunteered from a local chapter of the Blacks in Government (BIG) organization. Two groups of 3–4 women were convened. Snacks were provided but no other incentives were offered. The ages of the women ranged from 34 to 52. The goals of the sessions were: 1) for members to share with the group personal experiences of racism in the workplace and in public; 2) to refine and modify questions; 3) to ensure the clarity of wording and meaning; 4) to provide insight into the development of new questions that further measure an individual’s perception of racism; and 5) for the focus group leader to observe the facial expressions and bodily reactions of group members in response to the questions.

The most important change based on input from the focus groups was the scaling of responses on the Experience of Racism subscale. The proposed response format was: “almost never,” “several times a year,” “several times a month,” “several times a week,” and “several times a day.” The structure of this response scale was problematic for many of the women because of the difficulty in quantifying the frequency of racial discrimination. One woman commented that. “The issues addressed are a way of life. They occur constantly and are an ingrained part of the system. “The women also did not want to be asked how they “felt” in response to an item. For example, some of the questions began, “Because I am Black. I feel … ” To address these concerns, the TPRS used two new response scales and reworded several questions. For example, responses for the Experience of Racism—Group subscale “Whites often assume that Blacks work in lower status jobs and, therefore, treat them as such” were changed to “strongly disagree,” “disagree,” “agree,” or “strongly agree” on the TPRS. The response format for the Experience of Racism—Personal subscale was changed to: “never,” “rarely,” “some of the time,” or “most of the time.” This response scale was used for questions like, “How often have Whites assumed because of your race that you work in a lower status job and therefore, treated you with less respect during your career?” Lastly, questions regarding concerns about the negative influences of racism on one’s child(ren) or grandchild(ren) and about the frequency and intensity of past experiences of racism were added and tested.

Pre-testing of the TPRS

The TPRS was pre-tested by telephone using another convenience sample of 10 Black women, aged 30 to 50 years, who were employed and living in the Durham. North Carolina community. At the end of the interview, each participant was asked to provide feedback on the clarity, content, and structure of the interview. No major changes resulted from the pre-testing.

Psychometric testing

Psychometric analyses included 1) test-retest reliability; 2) factor analysis to determine the subscales representing different types of coping responses; and 3) tests of internal consistency using item-total correlations and Cronbach alpha values. To determine the test-retest reliability of the TPRS, a convenience sample of 30 employed Black women was selected. No incentives were provided. They were members of a sorority, government employees, or employed women from the community (Durham, North Carolina). The age of the women ranged from 30 to 50 years (mean age = 37 years). The majority of these women held at least a college degree (N = 22). The women were called back approximately 2–6 weeks (mean = 3 weeks) after the first administration of the TPRS and asked a subset of the questions again (8–10 minute interview). The Experience of Racism—Personal subscale was not re-tested, and only 3 of the 8 questions in the Concern for Child(ren) scale were re-tested. Test-retest reliability estimates were determined using the intraclass correlation coefficient (ICC). A test-retest reliability estimate of greater than 0.7 was considered adequate.18

Factor analysis and assessment of internal consistency were completed on a sample of Black women, aged 36–53, who participated in a cross-sectional study on women’s health.19 The health study screened a randomly selected sample of women enrolled in a prepaid health care plan with an ethnically diverse membership in Washington, DC. The response rate among the screened was 83%. All participants were asked to complete a self-administered questionnaire, and some were asked to give a blood sample. Black participants who completed both aspects of the health study (N = 534) were invited to participate in the telephone interview on perceived racism. Data collection for the study on perceived racism occurred in 1998–99, after the health study data had been collected. The Black women eligible for the study on perceived racism were mailed a letter describing the purpose of the study, their rights as research participants, and the nature of the interview. Informed consent was obtained over the telephone before the start of the interview. All interviews were completed by a trained African-American interviewer. Of the 534 Black women, 8 were not interviewed because of death, speech impairment, or living in a shelter. From the remaining 527 women, 476 participated, 27 refused the interview, and 24 could not be contacted, yielding a response rate of 90%. Demographic information obtained from the participants’ completed questionnaires is presented in Table 2. Exploratory principal axis factor analysis (EFA) was conducted using oblique (Promax) rotation to determine the underlying dimensions or factor structures represented by the emotional and behavioral coping response items in the TPRS. The main goal of factor analysis was to analyze all coping responses for each domain (ie, on the job and in public) together and to determine types of emotional and behavioral coping responses. Items with a factor loading of 0.35 or higher were retained. Scree plots based on eigenvalues were examined to confirm the number of factors; factors with an eigenvalue greater than one were retained. The subscales representing the types of coping that emerged from factor analysis were then tested for reliability.

Table 2
Demographic characteristics of the 476 Black women included in psychometric analyses*

Internal consistency of the TPRS was assessed by item-total correlations and by Cronbach alpha. These two measures indicate how well each item relates independently to the rest of the items on a scale and how well all the items relate overall, respectively. Reliability is considered to be moderately high for items achieving a Pearson item-total correlation of greater than 0.30, and for subscales obtaining a Cronbach alpha score of greater than 0.70.20,21 Ah analyses were completed using SAS software. Items with a response of “don’t know,” “not applicable,” or “refused” were not included in analyses. The total score for a scale was the sum of the scores for each item except for the Experience of Racism—Group subscale and the Past Experiences scale. For the former, before adding the items, strongly disagree and disagree were combined because of insufficient numbers in the strongly disagree category for analyses. For the Past Experiences scale, the frequency score of the stress was multiplied by the intensity of the stress score for each age and the two products were added. Tests of internal consistency for the scales and subscales were completed separately for those participants who reported experiences of racism and those who responded ‘no’ to having experienced racism. For the Concern for Child(ren) scale, only those participants who reported having a child(ren) or grandchild(ren) were included in the analyses.

Results

Factor analyses of the items used to measure emotional and behavioral coping responses resulted in five subscales representing separate coping responses (eigenvalue of greater than 1) (Table 3). Factor 1 was responsible for 36% of the common variance, with 8 items loading on this factor. This factor represented active emotions. The emotions “anxious” and “sad” loaded on the active emotional response factor. Comments by the women suggest that these responses often meant “anxious to do something” or “sad for the person being racist.” Factor 2 was responsible for 21% of the common variance, with 6 items loading on this factor. These items represented passive behaviors. The third factor, “passive emotions,” was responsible for 13% of the common variance, with 4 items loading on this factor. Factor 4 (praying) was responsible for 10% of the common variance. This factor represented internal active behavior. The fifth factor, “working harder to prove Whites wrong,” accounted for 7% of the common variance and represented external active behavior. The behavioral response, “acting out angrily to others” did not load on any of the five factors (loading factor <0.35). It is possible that “acting out” is more of a male behavioral response than a female one.

Table 3
Rotated oblique factor pattern* of items on the emotional and behavioral response subscales

Tests of internal consistency were completed for all of the subscales (Table 4). Pearson item-total correlations for all items on each subscale indicated good reliability (0.32–0.79). Moderate to high Cronbach alpha values were determined for the scales: Experience of Racism–Group (α = 0.82), Experience of Racism–Personal (α = 0.88), Concern for Child(ren) (α = 0.90), and Past Experiences of Racism (α = 0.82). The subscales measuring emotional and behavioral responses to racism also had good internal consistency (alpha >0.75), except for “passive behaviors,” which had an alpha = 0.68.

Table 4
Cronbach alphas and item-total correlations for the telephone perceived racism scales and subscales

Results of test-retest reliability are shown in Table 5. Using the intraclass correlation coefficient, fair to adequate test-retest reliability was shown for all subscales (ICCs ranged from 0.61–0.82).

Table 5
Test-retest reliabilities* of selected items on the Telephone Perceived Racism Scale

Discussion

The TPRS provides an innovative approach for examining the epidemiology of racism as a chronic stressor. Within a multidimensional framework, the TPRS captures perceptions of and responses to racism within a 15-minute telephone interview. The TPRS adds to the field of epidemiology by providing not only a tool to examine perceived racism, but also by including questions to assess both the individual’s experiences of racism and the perceptions of experiences of racism faced by Blacks as a group. The rationale for including these two types of subscales is that an individual’s personal perceptions may be greatly influenced by his/her social environment, as well as by knowledge of how racism affects the group (Black race).

The TPRS broadens the conceptualization of racism by considering factors that could influence one’s personal experience of racism and responses to racist event(s). This instrument considers the influence of past experiences of racism by asking the participants about their childhood (before age 20) and experiences during their twenties. The TPRS is also the first instrument to acknowledge the stress of raising a child in a color-conscious society with the inclusion of questions about a mother’s concern for her Black son(s), daughter(s), or grandchildren. For example, a person who was the victim of racial discrimination while growing-up, or someone who has a child experiencing racism, may experience more stress than someone without these experiences.

Further, the TPRS extends the PRS to establish subscales for the emotional and behavioral responses to racism in terms of active and passive coping responses. Five emotional and behavioral response subscales were determined: passive emotions, active emotions, passive behaviors, internal active behavior, and external active behavior. Clustering of coping styles, as such, may be more advantageous in linking exposure to racism and coping with certain health outcomes.

The passive behaviors subscale (does not speak up, accept or keep to yourself, ignore or forget) represents a more passive style of coping where one chooses not to take action. This type of inhibitory response has been suggested as a risk factor for hypertension and other poor health outcomes.9,22 For example, among Black women who experienced unfair treatment, those who kept quiet and accepted it were four times as likely to have high blood pressure as those who talked about it or took action.23

Similarly, the external active behavior subscale, a form of “John Henryism,” is the active attempt to overcome adversity or prove oneself to others. This coping response has also been found to be maladaptive in terms of health.24,25 Praying, an internal active behavior, has been seen as a positive behavioral coping response for Black people because of the group’s strong affiliation with the church.4 Historically, the church has been viewed as a supportive social-network for Blacks, and may modify the negative consequences of racism.26

Items on the passive emotions subscale (eg, hopeless and powerless) are depressive symptoms that suggest increased mental distress and may be linked to negative health outcomes such as obesity.27,28 The items on the active emotions subscale (eg, angry, frustrated, anxious, sad) may be viewed as adaptive emotions because they are expressed, allowing the body to recover from stressful experiences.22 These emotions allow the body to respond immediately to the stressor when the expression of anger, frustration, or the externalization of feelings onto someone else are within moderation. In the TPRS, questions pertaining to emotional and behavioral responses to racism were asked in a hypothetical way of women who reported no personal experiences of racism. Thus, it provides data to address researchers’ speculations that women with high blood pressure who do not report experiences of racism may be internalizing the racial discrimination.9,23

With respect to psychometric properties, the alpha values for the TPRS ranged from 0.76 to 0.92 for all subscales, except for the passive coping behavior subscale (α = 0.68). The Cronbach alpha values for the Concern for the Children and Past Experiences of Racism scales were high (α = 0.90) to moderately high (α = 0.82), respectively. The test-retest reliabilities using the intraclass correlation coefficient were overall fair to adequate. Overall, the results from the tests of internal consistency and test-retest reliability found the TPRS to be psychometrically sound.

The TPRS was constructed specifically for use with employed Black women to study the effects of perceived racism on women’s health. The study sample was generally of middle income, though there was a wide range; a small proportion fell below poverty level, and nearly 20% had an annual household income of over $100,000. Further testing would be required to determine the psychometric properties of this instrument in a low socioeconomic sample. The youngest women in our sample were in their thirties. We expect that the questionnaire will be applicable to younger women as well, but further testing would be useful. To gain further insight into the role of perceived racism on health status, we plan to describe the prevalence of and responses to racism as captured by the TPRS using the sample of 476 Black women interviewed. We also plan to use the TPRS to examine the association between perceived racism and chronic diseases such as uterine fibroids and obesity. We encourage others to make use of this scale.

References

1. Lillie-Blanton M, et al. Racial differences in health: not just Black and White, but shades of gray. Annu Rev Public Health. 1996;17:411–448. [PubMed]
2. Williams DR. Racism and health: a research agenda. Ethn Dis. 1996;6:1–6. [PubMed]
3. McNeilly M, Anderson NA, Armstead CA, et al. The Perceived Racism Scale: a multidimensional assessment of the experience of White racism among African Americans. Ethn Dis. 1996;6:154–166. [PubMed]
4. Outlaw Freida Hopkins. Stress and coping: the influence of racism on the cognitive appraisal processing of African Americans. Issues Ment Health Nurs. 1993;14:399–409. [PubMed]
5. Lazarus Folkman. Stress, Appraisal and Coping. New York, NY: Springer; 1984.
6. Essed P. Understanding Everyday Racism; An Interdisciplinary Theory. London: Sage; 1992.
7. Feagin JR. Racial and Ethnic Relations. 3. Englewood Cliffs. NJ: Prentice Hall; 1989.
8. Williams DR. Race and health: basic questions, emerging directions. Ann Epidemiol. 1997;7:322–333. [PubMed]
9. Krieger N, Sidney S. Racial discrimination and blood pressure: The CARDIA Study of young Black and White adults. Am J Public Health. 1996;86(10):1370–1378. [PubMed]
10. McNeilly M, Robinson EL, Anderson NB, et al. Effects of racist provocation and social support on cardiovascular reactivity in African-American women. Int J Behav Med. 1995;2(4):321–338. [PubMed]
11. Neighbors HW, Jackson JS, Broman C, Thompson E. Racism and the mental health of African Americans: the role of self & system blame. Ethn Dis. 1996;6:167–175. [PubMed]
12. Armstead CA, Lawler KA, et al. Relationship of racial stressors to blood pressure responses and anger expression in Black college students. Health Psychol. 1989;8(5):541–556. [PubMed]
13. LaVeist TA. Why we should continue to study race but do a better job: an essay on race, racism and health. Ethn Dis. 1996;6:21–29. [PubMed]
14. Murrell NL. Stress, self-esteem, and racism: relationships with low birth weight and preterm delivery in African-American women. J Natl Black Nurs Assoc. 1996;8(1):45–53. [PubMed]
15. Utsey SD. Assessing the stressful effects of racism: a review of instrumentation. J Black Psychol. 1998;24(3):269–288.
16. McNeilly M, Anderson NB, Robinson E, Jones R, et al. Convergent, discriminant, and concurrent validity of the perceived racism scale: a multidimensional assessment of the experience of racism among African Americans. Handbook of Tests and Measurements for Black Populations. 2:350–375.
17. Green LW, Kreuler M. Health Promotion Planning: An Educational and Environmental Approach. Mountain View. Calif: Maylield Publishing Co; 1991.
18. Utsey SD, Ponterotto JG. Development and validation of the Index of Race-Related Stress (IRRS) J Couns Psychol. 1996;43(4):490–501.
19. Baird DD, Schectman JM, Dixon D, Sandler DP. African Americans at higher risk than Whites for uterine fibroids: ultrasound evidence (abstract) Am J Epidemiol. 1998;147(11):S90.
20. Streiner DL, Norman GR. Health Measurement Scales—A Practical Guide to the Development and Use. 2. New York, NY: Oxford; 1995.
21. Nunally J. Psychometric Theory. 2. New York, NY: McGraw-Hill; 1978.
22. Lok Chiu-Fen, Bishop GD. Emotion control, stress, and health. Psychol Health. 1999;14:813–827.
23. Krieger N. Racial and gender discrimination: risk factors for high blood pressure. Soc Sci Med. 1990;30(12):1273–1281. [PubMed]
24. James S, Hartnett S, Kalsbeek W. John Henryism and blood pressure differences among Black men. J Behav Med. 1983;6:259–278. [PubMed]
25. James S, Strogatz D, Wing S, Ramsey D. Socioeconomic status, John Henryism, and hypertension in Blacks and Whites. Am J Epidemiol. 1987;126:664–673. [PubMed]
26. King G, Williams DR. Race and health: a multidimensional approach to African-American health. In: Amick BC III, Levine S, Tarlov AR, Walsh DC, et al., editors. Society and Health. New York. NY: Oxford Univ Press; 1995.
27. Landrine H, Klonoff EA. The schedule of racial discrimination: a measure of racial discrimination and a study of its negative physical and mental health consequences. J Black Psychol. 1996;22(2):144–168.
28. Wing RR, Matthews KA, Kuller LH, et al. Waist to hip ratio in middle-aged women. Associations with behavioral and psychosocial factors and with changes in cardiovascular risk factors. Arterioscler Thromb. 1991;11(5):1250–1257. [PubMed]