|Home | About | Journals | Submit | Contact Us | Français|
The National Social Life, Health, and Aging Project (NSHAP) measures seven indicators of quality of life (QoL) and psychological health. The measures used for happiness, self-esteem, depression, and loneliness are well established in the literature. Conversely, measures of anxiety, stress, and self-reported emotional health were modified for their use in this unique project. The purpose of this paper is to provide (a) an overview of NSHAP's QoL assessment and (b) evidence for the adequacy of the modified measures.
First, we examined the psychometric properties of the modified measures. Second, the established QoL measures were used to examine the concurrent validity of the modified measures. Finally, gender- and age-group differences were examined for each modified measure.
The anxiety index exhibited good internal reliability and concurrent validity. Consistent with the literature, a single-factor structure best fit the data. Stress was satisfactory in terms of concurrent validity but with only fair internal consistency. Self-reported emotional health exhibited good concurrent validity and moderate external validity.
The modified indices used in NSHAP tended to exhibit good internal reliability and concurrent validity. These measures can confidently be used in the exploration of QoL and psychological health in later life and its many correlates.
THE National Social Life, Health, and Aging Project (NSHAP) adopts a broad conceptualization of health as consisting of physical and social well-being. The project is designed to examine the complex relationships between these domains. Undoubtedly, quality of life (QoL) and psychological health are both an outcome and a correlate of these relationships. Within the literature, QoL is defined and operationalized in a number of ways and can easily be confounded with health and social well-being (Gill & Feinstein, 1994; Victor, Scambler, Bond, & Bowling, 2000). Here, we will focus on the approach NSHAP used to assess QoL and the benefits and drawbacks of these choices.
Focusing on the relationships between health and social intimacy, NSHAP is interdisciplinary in its nature. The project involves a variety of investigators from different disciplines with different interests. With limited time for the in-home interview, the operationalization of QoL and psychological health needed to focus on the specific interests of the investigators while maintaining a brief, yet broad approach. Ultimately, they were operationalized through measures of self-reported emotional health, happiness, self-esteem, loneliness, depression, anxiety, and stress. Measures of happiness, self-esteem, depression, and loneliness are well established in the literature and, consequently, discussed only briefly in this manuscript. Alternatively, we will focus on QoL measures which were modified for their use in this project, including anxiety, stress, and self-reported emotional health. We aim to provide evidence for the adequacy of these measures.
A nationally representative probability sample of community-dwelling individuals aged 57–85 years was selected from households across the United States screened in 2004. African Americans, Latinos, men, and the oldest-old (75–84 years at the time of screening) were oversampled. In-home interviews were conducted in English and Spanish by professional interviewers between July 2005 and March 2006, yielding 3,005 respondents (1,455 men and 1,550 women). The weighted sample response rate was 75.5% (for extended info, see O’Muircheartaigh, Eckman & Smith in this special issue).
To decrease respondent burden, minimize in-home interview time, and maximize content, the interview used a modularized format. During the in-person interview, all respondents were administered indices of self-reported emotional health, happiness, and depression. For all respondents, the loneliness items were given in a self-administered paper-and-pencil questionnaire which the respondents completed after the interviewer had left and mailed back. Indices of self-esteem, anxiety, and stress were modularized. Respondents were randomly assigned to receive the questions in-person or in the leave-behind instrument. Approximately 33% of respondents received the questions in the self-administered leave-behind instrument, thus affording an opportunity to examine the impact of different modalities on respondent responses to these questions.
The statistical analysis for testing the psychometric properties of the QoL measures was preformed using Stata 9.0 and SAS. In order to calculate the internal consistency of the scale measures, we used the Cronbach's alpha coefficient. Concurrent validity was performed using Pearson correlations between the modified QoL measures and the established measures. Additional calculations, such as principal components analysis (PCA) for structural validity, were performed according to relevancy.
Seven QoL and psychological health indicators were included in NSHAP. First we will briefly introduce the indicators already established in the literature. Second, we will provide a detailed description of the modified measures.
Depressive symptomology was assessed using the Center for Epidemiological Studies–Depression (CES-D) scale (Radloff, 1997). In the United States, the CES-D is an effective and well-used measure of depression for older adults (Lawton, Brody, & Saperstein, 1989). NSHAP used the shorter 11-item Iowa form of the CES-D scale, utilizing the same response categories as in the original 20-item scale. Factor analysis reveals that the Iowa short form captures the same dimensions as the long-form CES-D scale while giving up little precision (Kohout, Berkman, Evans, & Cornoni-Huntley, 1993). Rather than functioning as a diagnostic tool to assess a depressive disorder, the CES-D indicates depressive symptomology in this older population. This is particularly the case with the shortened instrument, which has been used successfully in other studies of aging, including the Health and Retirement Study (HRS) at the University of Michigan. Subjects are asked to indicate on a 4-point scale how often in the past week they have felt in accordance with 11 different statements. Responses are summed for a total score, higher scores indicating more depressive symptomology. In the NSHAP sample, the standardized Cronbach's alpha was .80.
Respondents were asked a single item: “If you were to consider your life in general these days, how happy or unhappy would you say you are on the whole—extremely happy, very happy, pretty happy, unhappy sometimes, or unhappy usually?” The question was selected because of its equivalence to an item from the General Social Survey, which established concurrent validity (Lee & Roebuck Bulanda, 2005) and was used in a substantial number of different researches (Cohen, 2002; Kousha & Mohseni, 2000; Maselko & Kubzansky, 2006).
Time constraints did not allow for an in-depth assessment of self-esteem. However, Robins, Hendin, and Trzensniewski (2001) established strong convergent validity of a single item taken from the Rosenberg Self-Esteem Scale with the other self-esteem constructs. The item, “I have a high self-esteem,” also behaved similarly with a wide range of criterion measures. Respondents indicate how true the statement is for them: not very true, somewhat untrue, neither true nor untrue, somewhat true, or very true.
The three items indicating loneliness—“I lack of companionship,” “I feel left out,” and “I feel isolated”—as well as the response categories were adopted from the HRS 2002 loneliness module. These items were taken from the Revised UCLA Loneliness Scale (Russell, Peplau, & Cutrona, 1980). Responses to each question are summed with higher scores indicating greater loneliness. Internal reliability as found in the HRS module is 0.72 (Hughes, Waite, Hawkley, & Cacioppo, 2004). In our sample, however, the loneliness scale achieved a higher internal consistency with standardized alpha of 0.81.
The next QoL measures—anxiety, stress, and self-reported emotional health—were used in a modified way in NSHAP. Thus, we proceed with introducing their unique way of measurement followed by evidence of their adequacy.
A modified version of the seven-item anxiety subscale of the Hospital Anxiety and Depression Scale (HADS-A; Zigmond & Snaith, 1983) was used to assess feelings of anxiety. Originally designed as a screening instrument for the assessment of psychological distress in patients (Zigmond & Snaith, 1983), the HADS has been used successfully in population-based studies and has established reliability and validity (Mykletun, Stordal, & Dahl, 2001). Respondents report on feelings of anxious mood, thoughts, and restlessness over the past week on a 4-point Likert scale from 0 to 3, with higher values indicating higher anxiety levels; scores are summed, and a total score for the anxiety subscale ranges from 0 to 21. The interviewees are instructed to give their immediate reaction to each question.
There are two recent exhaustive reviews summarizing the psychometric properties of the original HADS, populations studied, testing characteristics, scale homogeneity, test–retest reliability, and anxiety-case identification properties (Bjelland, Dahl, Haug, & Neckelmann, 2002; Herrmann, 1997). The psychometric properties of the HADS anxiety index compare favorably with other well-known scales, such as the State-Trait Anxiety Inventory (STAI) and the Beck Anxiety Inventory (BAI) (Roemer, 2001). Its internal consistency in a large general population group reproduces that in patient populations and easily meets the Cronbach's alpha standard for internal consistency of .60 or greater for self-administered items (Caci et al., 2003; Mykletun et al., 2001).
In NSHAP, the same seven items of the anxiety subscale (HADS-A) were used (see Table 1). In the original HADS anxiety items, each question had a different set of response choices. To ease respondent burden and increase consistency throughout the questionnaire, the response categories were modified to match those of the CES-D. Hence, for each item, respondents were asked to indicate how often they felt this way: rarely or none of the time, some of the time, occasionally, or most of the time. Although these question response categories have not been tested in a population-based sample, their use greatly improved the efficiency of the questionnaire and decreased interview time.
As in the original anxiety subscale, the seven items were summed after reverse coding of the single “positive” item: “I can sit at ease and feel relaxed.” Thus, the modified scale range is 7–28, with higher values indicating higher levels of anxiety. Due to the change in response options, we proceeded with a detailed examination of the modified scale to determine the acceptability of the instrument as well as its reliability and validity.
The vast majority of the NSHAP respondents found the anxiety scale acceptable. The response rate for the seven-item scale was around 93%. This finding is similar to that found in the research field among patient and nonpatient populations (Herrmann, 1997). As described above, approximately a third of the respondents received the anxiety items in a leave-behind questionnaire. For the following description of item response, we omitted from the analyses respondents who did not return the questionnaire at all (17% of respondents receiving the items in the leave-behind instrument). In all, 96.9% of respondents answered all seven questions, and 97.9% answered at least 90% of the questions. Only 1.9% of these respondents answered less than 50% of the items.
Respondents who received the anxiety questions in the in-person interview responded to more of the anxiety questions than those respondents who received the anxiety questions in the leave-behind instrument. In all, 98.7% of respondents who received the questions in the in-person interview answered all the questions and 99.5% answered 90% or more. In contrast, 92.6% of respondents receiving the anxiety questions in the leave-behind instrument answered all the questions, and 93.8% answered at least 90% of the questions; 5.7% of those respondents using the leave-behind instrument answered less than 50% of the questions, compared with only 0.3% of in-person interview respondents. In addition, respondents using the leave-behind instrument reported statistically significant higher levels of anxiety than respondents using the in-person interview.
The standardized Cronbach's alpha coefficient indicating internal consistency for the HADS-A was 0.76 for the entire sample (The internal reliability across gender and age groups ranges from 0.73 to 0.78.). This value, albeit moderate, is in the range found in other studies. According to Bjelland et al. (2002), the reported HADS-A alpha in 15 studies varied from 0.68 to 0.93. Caci and colleagues argue that “such a result is not surprising in a non-clinical sample where the variances of the item scores are classically low” (Caci et al., 2003, p. 93). Thus, we feel the version of the HADS-A included in NSHAP to be within an acceptable internal consistency range.
Most studies examining the factor structure of the entire HADS scale achieved a two-factor solution (Bjelland et al., 2002), a factor for depression and a factor for anxiety. This two-factor structure was substantiated among patients (Moorey et al., 1991) as well as in a healthy large sample (Lisspers, Nygren, & Soderman, 1997). We used PCA to examine the factor structure of the HADS-A in NSHAP. In light of the factor structure found using the entire scale, we expected to achieve a one-factor solution, as we include only the anxiety subscale.
The PCA verified the one-factor solution. The seven items loaded heavily on one factor that accounted for 42% of the variance. This factor was the only factor with an eigenvalue higher than 1 (eigenvalue = 2.95). In order to test the stability of the factor structure achieved, we repeated the analysis among different age and gender subsamples (see Table 2). As found in Mykletun et al. (2001), when the two subscales of the HADS were tested separately, a single-factor solution was obtained for almost all subsamples. Among males, however, a two-factor structure was indicated, although all the seven items loaded heavily on the first factor, which explained more than 40% of the variance.
Within NSHAP, feelings of anxiety are conceptually incorporated into the larger concept of QoL and psychological health. As mentioned above, other indicators of QoL include depression (measured by the CES-D), stress (Perceived Stress Scale [PSS]-4), loneliness, self-esteem, emotional health and happiness. Concurrent validity for the HADS-A can be established by examining the correlation between these constructs. As seen in Table 3, the correlations are in the expected direction, with the strongest association found between depression and anxiety (.63, p < .001). Positive associations were found among anxiety, stress, and loneliness. Additionally, three negative correlations were found among the positive aspects of QoL: good emotional health, high self-esteem, and general happiness. As each of these correlations was in the expected direction, concurrent validity for the HADS-A was demonstrated.
As seen in Table 4, the mean HADS-A score was significantly higher among women compared with men. Additionally, the 65–74 age group was less anxious compared with the younger age group (57–64). However, we did not find a significant difference between the oldest age group and the youngest in this sample. The mean scores suggest a nonlinear relationship between anxiety and age.
In NSHAP, perceived stress was assessed using the four-item modification of Cohen's PSS (Cohen, Kamarck, & Mermelstein, 1983). The original PSS scale contains 14 items to assess the degree to which life situations are appraised as having been stressful during the past month. Respondents indicate how often they have felt or thought in the way indicated by the statement on a 5-point Likert-type scale. Responses range from never (0) to very often (4). The PSS score is obtained after reversing its seven positive items and summing all item scores (total score is 0–56), with higher scores indicating a higher perceived stress level (Cohen et al., 1983; Cohen, Sherrod, & Clark, 1986).
In the development of the 14-item scale, Cohen et al. (1983) also validated a shortened version. The four items (see Table 1) with the highest correlations to the 14-item scale were selected to comprise the PSS-4. The authors determined that the PSS-4 was a useful tool for assessing perceived stress in research where a shortened measure was required. For NSHAP, two changes were made in the PSS-4. These changes were primarily to increase consistency across the depression, anxiety, and stress scales. First, the respondents were asked about feelings during the last week instead of the last month (as in the original scale). Second, the response categories were changed to match those of the CES-D (see above). Due to these changes, we explored the acceptability, reliability, and validity of this modified measure.
The response rate for the modified PSS-4 scale was close to 92%. This finding suggests a good accessibility of the modified version used in NSHAP. As stated above, approximately one-third of the respondents received the stress items in the leave-behind instrument. When those who did not return the leave-behind instrument were omitted, 96.8% of remaining respondents answered all the stress questions; 98% answered 90% or above. Only 2% answered 50% or fewer of the questions.
Respondents who received the stress questions in the in-person interview responded to more questions than those respondents who received the stress questions in the leave-behind instrument. In all, 98.3% of respondents who received the questions in the in-person interview answered all the questions and 99.5% answered 90% or more. In contrast, 93% of respondents receiving the stress questions in the leave-behind instrument answered all the questions and 94.5% answered at least 90% of the questions; 5.5% of those respondents using the leave-behind instrument answered less than 50% of the questions, compared with only 0.5% of in-person interview respondents. In addition, respondents using the leave-behind instrument reported statistically significant higher levels of stress than respondents using the in-person interview.
The literature supports good internal reliability for the long version of the PSS (Cohen et al., 1983, 1986; Cohen & Lemay, 2007; Cohen, Tyrrell, & Smith, 1993; Hyman, Paliwal, & Sinha, 2007). The PSS-4 demonstrates lower reliability (0.72) (Cohen et al., 1983). Consistent with this literature, the reliability for the PSS-4 in our sample was 0.63. Testing the reliability of the PSS across gender and age showed the same tendency, with only one exception concerning the oldest age group (α = 0.55).
Stress as well as depression, anxiety, and loneliness reflect the negative aspect of QoL. Thus, these “negative” facets of QoL were assumed to be positively correlated. Indeed, The PSS-4 achieved strong positive correlations with anxiety, depression, and loneliness (.57, .59, .37, p < .001, respectively). We expected a negative correlation between perceived stress and the positive aspects of QoL, measured by happiness, high self-esteem, and good emotional health. As predicted, the three positive indicators have significant negative correlations with the PSS-4 scale (Table 3).
Similar to the results found for the anxiety scale, the mean score of the PSS-4 was significantly higher among women compared with men (see Table 4). The second age group (ages 65–74 years) had lower scores compared with their younger counterparts. No differences were found between the oldest and youngest age groups in the sample. As with anxiety, the mean scores of the PSS-4 suggest a nonlinear relationship between stress and age.
Self-reported physical health, a widely used single-item indicator of health, has been shown to be associated with a wide array of more specific health measures (Laumann, Paik, & Rosen, 1999; Ostbye et al., 2006), and some scholars argue that self-reported health is as useful an indicator of health and mortality as physician diagnoses (Ferraro & Farmer, 1999; Maddox & Douglas, 1973; Mossey & Shapiro, 1982). Additionally, self-reported measures are useful because they define health as well-being, not just the absence of disease (Ross & Wu, 1996).
With this in mind, we constructed a self-report emotional health measure to mirror the structure of the self-reported physical health indicator. NSHAP asked respondents to evaluate their emotional health status, using a modified version of a question asked in the 1992 HRS. The original wording of the emotional health question asked in HRS was “What about your emotional health—how good you feel or how stressed, anxious, or depressed you feel? Is it excellent, very good, good, fair, or poor?” whereas in NSHAP respondents were asked “What about your emotional or mental health? Is it excellent, very good, good, fair, or poor?” This question was asked of all respondents. The response categories were coded in NSHAP from 1 (poor) to 5 (excellent). Thus, higher scores reflect better perceived emotional health.
As self-reported emotional health contributes to positive QoL, we anticipated that this measure would positively correlate with the rest of the positive QoL indicators. As seen in Table 3, we found a strong positive correlation between emotional health and perceived happiness (.43, p < .001). We also found a significant correlation between self-esteem and self-reported emotional health. We also expected a negative correlation with anxiety, depression, stress, and loneliness. As expected, we found a strong negative correlation between good self-reported emotional health and anxiety (−.57, p < .001) and depression (−.49, p < .001).
As shown in Table 5, approximately 65% of respondents reported having very good or better emotional health. These results differ somewhat from those reported by the HRS in 1994. We suspect this variation is the result of both the time lapse between HRS and NSHAP data collection points as well as differences in question wording. While NSHAP asked respondents a general question about their emotional health status, HRS provided examples of possible emotional conditions. The inclusion of emotions that characterize reduced emotional health status may have encouraged respondents to limit the range of emotions considered as negative. The NSHAP question, without the specific clarifiers, should therefore produce a less biased self-reported measure.
We found significant variation between men's and women's self-reported emotional health. Specifically, women were more likely to report poorer emotional health; 12% of female respondents reported “fair” or “poor” health, as compared with only 8% of males. Similarly, only 22% of women reported “excellent” emotional health versus 30% of male respondents. Response distributions were found to be consistent across our two youngest age categories, with 66% of respondents in both categories reporting “excellent” or “very good” emotional health. However, fewer adults (57%) in the oldest age group reported such positive feelings (Table 5).
Seven indicators of QoL and psychological health were included in the NSHAP. Four of them—anxiety, depression, stress, and loneliness—reflect the more negative aspect of QoL, whereas the three others—emotional health, happiness, and self-esteem—indicate its positive side. Although NSHAP's framework encompasses established measures for happiness, self-esteem, loneliness, and depression, some modifications were made for the anxiety, stress, and emotional health measures. Thus, the central aim of this paper was to present the changes that were made in NSHAP and to provide empirical evidence for the adequacy of the modified measures.
The anxiety, stress, and emotional health indicators were modified for NSHAP to increase efficiency, maintain consistency in response categories across scales, and decrease respondent burden by decreasing the time it took to administer the questions. In this article, we established the usability of these modified indicators by demonstrating their basic psychometric properties. The modified HADS-A was found to be a reliable and valid measure in terms of internal consistency, factor structure, and concurrent validity. Internal reliability was established across the sample as well as in gender and age subgroups. The one-factor solution for the HADS-A found in NSHAP is supported by the literature. The modified anxiety measure achieved satisfactory concurrent validity as well, with significant correlations to the other indicators of QoL.
The modified PSS-4 achieved modest internal reliability, particularly with the oldest age group; caution should be used when applying the modified scale to the oldest-old. As with the modified HADS-A, the PSS-4 demonstrated strong concurrent validity. Finally, the single-item indicator of self-reported emotional health correlated significantly with the other measures of QoL in the expected directions.
Gender- and age-group comparisons were conducted for all three modified measures. Consistent with the literature, a systematic difference between men and women was found across all three indices. Women tend to be more anxious and are more likely to report poorer emotional health (Christensen et al., 1999; Fuentes & Cox, 2000; Livingston, Watkin, Milne, Manela, & Katona, 1997; Schoevers, Beekman, Deeg, Jonker, & van Tilburg, 2003). The findings regarding age differences, on the other hand, are somewhat more complex. Findings suggest a nonlinear relationship across age groups for anxiety and stress. Another explanation is that the relationship between age, stress, and anxiety is confounded with gender, as the oldest age group likely includes proportionally more women who tend to report higher distress levels. The complex relationships between age and QoL are certainly an area for further exploration.
Another important topic that could benefit further examination is the different modes of administration used in NSHAP. The findings substantiate differences between the in-person and the leave-behind questionnaires in reported anxiety and stress levels, with higher levels shown in the self-completion mode. A similar trend was found in another study exploring the SF-36 health status scale (Lyons et al., 1999) and emphasizes the need for further research.
Unfortunately, the inclusion of both the original indices and their modified forms in the NSHAP questionnaire was impossible due to time constraints. Consequently, we were unable to compare the modified indices with their original forms. As a result, a complete exploration of the validity of the modified indices is impossible with these data. Nonetheless, the modified QoL indicators used in NSHAP tended to exhibit good quality in terms of internal reliability and concurrent validity. We feel confident that these measures, as well as the more established indicators of happiness, self-esteem, depression, and loneliness, can be used in studies of QoL and psychological health among older adults.
NSHAP is supported by the National Institute on Aging, Office of Women's Health Research, Office of AIDS Research, and the Office of Behavioral and Social Science Research (5R01AG021487).
All authors had full access to the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. Shiovitz-Ezra, Leitsch, and Graber contributed to the conceptualization of the manuscript. Shiovitz-Ezra performed the primary statistical analyses and wrote the manuscript with the help of Leitsch. Graber and Karraker contributed to data analyses. All authors participated in editing the manuscript for intellectual content. The authors would like to thank Sanyu Mojola for her helpful feedback and comments on earlier drafts.