Developing Summary Scores of Health-Related Quality of Life for a Population-Based Survey

Public Health Rep. 2009 Jan-Feb; 124(1): 103–110.

PMCID: PMC2602935

Developing Summary Scores of Health-Related Quality of Life for a Population-Based Survey

Willi Horner-Johnson, PhD,^a Gloria Krahn, PhD, MPH,^a Elena Andresen, PhD,^b and Trevor Hall, PsyD^a, RRTC Expert Panel on Health Status Measurement^c

^aOregon Institute on Disability & Development, Oregon Health & Science University, Portland, OR

^bDepartment of Epidemiology and Biostatistics, College of Public Health and Health Professions, University of Florida Health Sciences Center, Gainesville, FL

^cRehabilitation Research and Training Center: Health and Wellness, Oregon Institute on Disability & Development, Oregon Health & Science University, Portland, OR

Address correspondence to: Willi Horner-Johnson, PhD, Oregon Institute on Disability & Development, Oregon Health & Science University, 707 SW Gaines Rd., Portland, OR 97239, Phone: 503-494-9273, Fax: 503-494-6868, ; Email: hornerjo/at/ohsu.edu

This article has been cited by other articles in PMC.

SYNOPSIS

Objective

Health-related quality of life (HRQOL) is an important indicator of public health. The Centers for Disease Control and Prevention's (CDC's) Behavioral Risk Factor Surveillance System (BRFSS) includes nine HRQOL items that can be used to monitor the health status of the nation. The objective of this study was to examine the numerical relationships among these HRQOL items to develop summary scores by combining items.

Methods

Using 2001 and 2002 BRFSS data from states that included all nine HRQOL questions, factor analyses were performed to determine whether the items would group together into multi-item scales.

Results

Two factors emerged, corresponding conceptually to a physical health construct and a mental health construct. The resulting scales demonstrated acceptable internal consistency and ability to distinguish between population subgroups known to differ on HRQOL.

Conclusions

This study provides support for condensing the BRFSS core and optional HRQOL questions into two scales. These scales provide more complete information about physical and mental HRQOL than is available from single items, while limiting the number of individual variables required for a given analysis. However, the four core HRQOL questions focus primarily on physical health. Thus, the five supplemental questions should be included when measuring mental health is of interest.

Within the context of public health, health-related quality of life (HRQOL) is recognized as an important health outcome, in addition to morbidity and mortality.¹^–³ HRQOL reflects an increased appreciation for not only how long one lives, but also how well one lives. HRQOL can encompass elements of physical health, mental health, social health, and role functioning.⁴^,⁵

To stimulate the development of surveillance mechanisms for tracking HRQOL at the state and national levels, the Centers for Disease Control and Prevention (CDC) convened a group of experts in the early 1990s to develop a definition, conceptual model, and measures of HRQOL.⁶ The working group's efforts resulted in a set of four surveillance questions for inclusion in the Behavioral Risk Factor Surveillance System (BRFSS). The BRFSS is an ongoing random-digit-dialed telephone survey of non-institutionalized civilian adults (≥18 years of age) designed to assess risk and protective behaviors associated with disease. It is implemented in each state and U.S. territory, and is a key source of data for informing public health policy and practice.⁷^–⁹ The four HRQOL questions measure: (1) self-perceived health (excellent, very good, good, fair, or poor), (2) number of days out of the past 30 that physical health was not good, (3) number of days out of the past 30 that mental health was not good, and (4) number of days out of the past 30 that usual activities were limited by poor physical or mental health. These four “core” questions were first included in the BRFSS in 1993⁶ and have been included in each subsequent year.

In 1995, CDC added five supplemental “healthy days” questions in an optional BRFSS module. These five questions assess number of days out of the past 30 that the respondent has experienced pain interfering with usual activities; felt sad, blue, or depressed; felt worried, tense, or anxious; not gotten enough rest or sleep; or felt very healthy and full of energy. Together with the four core questions, these items have demonstrated good test-retest reliability¹⁰ and have been used with populations that vary in age, gender, race¹¹^,¹² and nationality.¹³^–¹⁵ The healthy days questions have also been used to describe the health of groups with known health conditions including arthritis¹⁶ and diabetes,¹⁷ and to compare groups based on risk factors such as obesity.¹⁸ Use of the items individually, however, provides a series of information fragments about specific symptoms as opposed to a cohesive picture of health. Previous research indicates strong relationships among the core questions, suggesting that the items could be combined into summary scores.⁶ Empirically derived summary scores would reduce the potential for spurious findings by limiting the number of separate analyses needed in studying HRQOL. The richness of information provided by multiple items would be consolidated into a parsimonious, multidimensional assessment of population health.

Public health professionals currently combine two of the items (days physical health not good, and days mental health not good) into a summary measure called the Healthy Days Index.⁷ An alternative strategy for creating summary scores uses factor analysis. Factor analysis is a statistical method that examines numerical relationships among items to identify those factors (or latent variables) that describe the commonality among items. Factor analysis is a means of condensing information so that a small number of factors can be used to represent responses to a larger number of individual questions.¹⁹

Mielenz and colleagues²⁰ conducted a factor analysis of the nine BRFSS HRQOL questions as answered by people with arthritis in North Carolina. Their analysis found two factors, or possible subscales, among these nine HRQOL items for their sample. The purpose of the present study was to examine the same nine HRQOL questions in large, multistate, population-based samples to determine whether the same or different underlying factors could be identified in the general population that would lead to the development of one or more summary scores. Such scores could be used to summarize population health, compare among groups, or describe changes in HRQOL based on multi-item constructs rather than individual questions.

METHODS

Data source

This study used 2001 and 2002 BRFSS data from the states and territories that included all nine HRQOL questions in those years. Twelve states (Alaska, Arizona, Delaware, Georgia, Maryland, Minnesota, Nebraska, New Jersey, Ohio, Tennessee, Utah, and Virginia) and the District of Columbia used all nine HRQOL questions in 2001, yielding a sample size of 47,179. Nine states (California, Hawaii, Iowa, Kentucky, Minnesota, Missouri, New Jersey, Rhode Island, and Virginia) and Guam used all nine questions in 2002, resulting in a sample size of 45,413.

Analyses

Because a different mix of states used the HRQOL questions in each year, we conducted initial analyses with 2001 data and replicated them with 2002 data. These two separate analyses allowed us to see how similar the findings were with two different samples. Five data analysis steps were conducted separately with each year of data.

First, we conducted exploratory factor analyses with all nine HRQOL items. The factor extraction method used was principal axis factoring, which can accommodate non-normal data distributions. Factors were allowed to correlate with each other, based on our conceptual model that the component factors all relate to the broader construct of HRQOL. We followed standard procedures of retaining factors in the emerging model if the factor explained more variability in scores than a single item would (i.e., had an eigenvalue greater than or equal to one).¹⁹ Items were regarded as being part of a factor if they achieved a factor loading score of at least 0.40, a threshold conventionally used in health research.²¹ If an item loaded at 0.40 or higher on more than one factor, a difference in factor loadings of at least 0.25 was used to assign the item to the factor on which it loaded most highly. The factor extraction process identified how many summary scores were needed to adequately reflect the variability measured by these HRQOL questions. It also identified which items to assign to the different summary scores.

The second step was to determine the degree to which all items assigned to a factor were, in fact, measuring the same underlying construct. Cronbach's alpha was used to measure the internal consistency of the factors, with a higher score indicating greater reliability in measuring a common factor. We used the traditional cutoff value of 0.70 and higher as acceptable.²²

Third, we examined criterion validity of the scale scores by correlating them with the global general health question. This question is often used as a single-item representation of HRQOL.²³

Fourth, to determine whether the scale scores could distinguish among known groups, we compared the scores for groups that were expected to differ on HRQOL because of age or a chronic health condition such as diabetes, arthritis, or asthma. When comparing groups based on chronic health conditions, analyses were adjusted to account for age differences among groups.

Finally, we limited factor analysis procedures to only the original four HRQOL surveillance questions to determine whether a smaller number of HRQOL items would yield results comparable to those of the nine items. Procedures of steps 1 and 2 were repeated.

Factor analyses and Cronbach's alpha calculations were conducted using SPSS® software.²⁴ SAS® software²⁵ was used to calculate Spearman rank order correlations with the general health question. SUDAAN®²⁶ was used to conduct t-tests to determine whether factor scores were able to distinguish between groups.

RESULTS

Factor structure

For the nine HRQOL questions, a two-factor model emerged in both years of data (i.e., two factors had an eigenvalue ≥1). The unrotated two-factor solution accounted for 46% of the total explained variance of scores in 2001 and 48% of the total variance in 2002. In general, the factors that emerged corresponded conceptually to a physical health construct and a mental health construct. Table 1 presents the loading scores of items on each of the two factors in the 2001 and 2002 datasets, respectively.

Table 1

Rotated structure matrix for nine HRQOL questions, 2001–2002

The two factors were moderately correlated with each other (r=0.516 in 2001 and r=0.586 in 2002). Although some items loaded higher than 0.40 on both factors, in all cases but one, there was a difference of at least 0.25 between the physical health and mental health loadings of dual-loading items, allowing assignment to a single factor on which the item loaded highest. Table 1 indicates the factor to which each item was designated. Factor loadings and designation of items to factors were consistent across both datasets. The general health question loaded only on the physical health factor. “Days healthy and full of energy” loaded nearly equally on both the physical health and mental health factors in both datasets.

Scale scoring

Scale scores were based on the items exclusively designated to each factor. Because the item regarding “days very healthy and full of energy” loaded on both factors and could not be clearly assigned to a single scale, it was omitted from inclusion in either of the scales. Items were reverse-scored such that higher scores would represent better HRQOL. To combine the items on each factor into single-scale scores, raw responses to the items were converted to standard (z) scores. The mean z-scores for the items in each scale were calculated. Simple means were used rather than weighting the items by their factor loadings. The resulting means were then converted to T-scores to create distributions with a mean of 50 and standard deviation of 10.

Internal consistency

The physical health scale had a Cronbach's alpha of 0.777 in 2001. This number increased somewhat when the general health question was removed. Cronbach's alpha was lowered by removal of any of the other items; only removal of the item on “days physical health not good” reduced the alpha to <0.70. Cronbach's alpha for the mental health scale was 0.765. Removal of the question about sleep/rest raised the alpha. The alpha was lowered to <0.70 by removal of each of the other items (Table 2).

Table 2

Internal consistency of physical and mental health scales with individual items removed, 2001–2002

In the 2002 dataset, Cronbach's alpha for the physical health scale was 0.781 and was improved with removal of the general health question. Cronbach's alpha for the mental health scale was 0.784. Removing the item about sleep/rest again increased the alpha. Removal of each of the other items reduced the alpha to <0.70 (Table 2).

Criterion validity

In both 2001 and 2002, the general health question (reverse-scored) showed a positive and moderately high correlation with the physical health scale score (2001: r=0.86, p≤0.0001; 2002: r=0.85, p≤0.0001). Correlations with the mental health scale score were also positive, but weak (2001: r= 0.17, p≤0.0001; 2002: r=0.18, p≤0.0001).

Known-groups validity

In both years of data, people with diabetes, arthritis, or asthma had significantly lower age-adjusted physical health and mental health scores than people without these chronic conditions (Table 3). Compared with people aged 18 to 64 years, people ≥65 years of age had significantly lower physical health scale scores but significantly higher mental health scale scores (Table 3).

Table 3

Comparison of known groups on physical and mental health scale scores, 2001–2002^a

Four-item solution

When the four core items were factor analyzed, only one component had an eigenvalue ≥1 in either year. This component accounted for 41% of the total explained variance in 2001 and 43% in 2002. All four items loaded higher than 0.40 on the single factor (Table 4). For 2001, Cronbach's alpha for the four core items was 0.708. Removing the mental health question increased the alpha. Removal of any of the other items lowered the alpha to <0.70 (Table 4). For 2002, the alpha for the scale was 0.722. The alpha increased very slightly with the removal of the mental health question, and was lowered with removal of any of the other items.

Table 4

Factor loadings and internal consistency of four core HRQOL questions, 2001–2002

DISCUSSION

A consistent two-factor structure emerged from both years of data. The two factors supported the presence of two underlying constructs: physical health and mental health. Our findings did not identify any additional HRQOL constructs as measured by the nine HRQOL items of the BRFSS. The robustness of these factors was supported not only by the consistency across the two years of BRFSS data analyzed in this study, but by similar findings in another study with a very different sample. While the present study used two large, population-based samples of the general public from multiple states, Mielenz et al.²⁰ analyzed a relatively small clinical sample of people with arthritis in North Carolina. Despite the differences in sample types, both studies yielded a physical health factor and a mental health factor, with only minor differences in assignment of items to factors.

The general health item loaded only on the physical health factor. This finding suggests that using this single question may be useful for assessing physical health, but is less helpful for assessing HRQOL more broadly. This interpretation is consistent with other research on the meaning of self-rated general health, which has found that general health ratings are most heavily influenced by assessments of physical health.²⁷^–²⁹

The question about feeling very healthy and full of energy may have loaded on both factors because it measures two separate issues. Feeling very healthy is likely to be interpreted primarily in terms of physical health (as is the case with the general health item), whereas feeling full of energy may be more closely related to feelings of getting enough rest or sleep, which loaded only on the mental health factor in the general population samples used in this study.

In the Mielenz et al.²⁰ study, the rest/sleep item loaded on the physical health factor as well as the mental health factor. This may be due to the fact that the physical symptoms of arthritis can interfere with sleep in this population. The item on feeling full of energy loaded only on the physical factor for the arthritis sample.²⁰ Feeling full of energy for people with arthritis may be largely related to presence or absence of arthritis pain and stiffness. These differences between studies suggest that the factor structure of the nine HRQOL items should be further examined in various subgroups of the general population and in clinical populations. One factor structure may not perfectly fit all groups, although there may be many similarities.

The group differences found on the physical and mental health scale scores were consistent with what was expected based on previous research. People with chronic conditions such as diabetes, arthritis, and asthma have reported lower physical and mental HRQOL in previous studies using the BRFSS HRQOL items.¹⁶^,¹⁷^,³⁰ Older adults have consistently been found to report poorer physical health, but better mental health than younger adults.⁷ The combined scale scores continued to uncover similar distinctions. We found that adults with diabetes, arthritis, or asthma scored lower on both the physical and mental HRQOL scales. Adults aged 65 and older scored lower on the physical HRQOL scale than younger adults, but scored higher on the mental HRQOL scale. The scales thus demonstrate appropriate known-groups validity.

The four core BRFSS items provided only a single scale, primarily focused on physical health. The single item about days mental health was not good provides very limited information about the mental dimension of health. In the analysis of all nine items, the bulk of the mental health scale came from the five additional HRQOL items typically available as an optional BRFSS module. In situations when mental health is of interest, including these optional items is important to provide a more complete assessment of mental health.

Limitations

Limitations of this study should be noted. The factor analysis was exploratory in nature. However, the fact that the same factor structure was found in samples from two different years and different groups of states suggests that the factor structure is robust. Ability to establish construct validity of the scale scores resulting from our analyses was limited, given the secondary nature of the data. No criterion HRQOL measure with which to compare performance of the scales was administered to the samples. We could only correlate the scale scores with the individual HRQOL items used to make up the scales, which we acknowledge is a tautology. Further, while the BRFSS is a key source of population-based public health data, it is subject to bias stemming from the use of telephone interviews³¹ and the exclusion of military personnel and individuals living in institutions or other group quarters, such as college dormitories.⁸^,⁹

CONCLUSION

This research provides support for condensing eight of the nine HRQOL questions used in the BRFSS into two summary scores or scales for use with the general population. These scales provide a means of including more complete information about physical and mental HRQOL than is available from single items, while limiting the number of individual variables required for a given analysis. Because the four core BRFSS items appear primarily related to physical health, we recommend including the additional five items whenever a multi-item measurement of mental HRQOL is also needed. The physical and mental HRQOL scales derived from the expanded set of items have demonstrated preliminary evidence of criterion validity in correlations with the general health question and in their ability to distinguish among population subgroups in the BRFSS known to differ on HRQOL.

Further studies are needed to evaluate the ability of the scales to correctly differentiate among additional groups, and to examine test-retest reliability of the scales. In addition, future research should examine whether the same factor structure applies in various subgroups (e.g., women and men, people with and without disabilities, people of different ages). The scales described in this article are intended as a starting point for moving beyond individual questions to considering the BRFSS items as a cohesive measure of HRQOL.

Acknowledgments

The authors thank Susan Wingenfeld for assistance with references and formatting.

Footnotes

The members of the Rehabilitation Research and Training Center Expert Panel on Health Status Measurement are: Elena Andresen, PhD, University of Florida, Gainesville, Florida; Vincent Campbell, PhD, Centers for Disease Control and Prevention, Atlanta, Georgia; Brad Cardinal, PhD, Oregon State University, Corvallis, Oregon; Charles Drum, JD, PhD, Oregon Health & Science University, Portland, Oregon; Glenn Fujiura, PhD, University of Illinois at Chicago, Chicago, Illinois; Trevor Hall, PsyD, Oregon Health & Science University; Willi Horner-Johnson, PhD, Oregon Health & Science University; Eric Jamoom, MS, University of Florida; Gloria Krahn, PhD, Oregon Health & Science University; Margaret Nosek, PhD, Baylor College of Medicine, Houston, Texas; and Rie Suzuki, PhD, Oregon Health & Science University.

This work was supported by grant #H133B040034 from the National Institute on Disability and Rehabilitation Research to Oregon Health & Science University for the Rehabilitation Research and Training Center on Health and Wellness.

REFERENCES

1. Department of Health and Human Services (US) Healthy people 2000. Washington: DHHS, Office of the Surgeon General (US); 1992.

2. Department of Health and Human Services (US) Healthy people 2010. Washington: DHHS, Office of the Surgeon General (US); 2000.

3. Whiteneck G. Conceptual models of disability: past, present, and future. In: Field MJ, Jette AM, Martin L, editors. Workshop on disability in America: a new look. Washington: National Academies Press; 2006. pp. 50–66.

4. Patrick DL, Erickson P. Health status and health policy: quality of life in health care evaluation and resource allocation. New York: Oxford University Press; 1993.

5. Ware JE., Jr. Standards for validating health measures: definition and content. J Chronic Dis. 1987;40:473–80. [PubMed]

6. Hennessy CH, Moriarty DG, Zack MM, Scherr PA, Brackbill R. Measuring health-related quality of life for public health surveillance. Public Health Rep. 1994;109:665–72. [PMC free article] [PubMed]

7. Centers for Disease Control and Prevention (US) Measuring healthy days. Atlanta: CDC (US); 2000.

8. Gentry EM, Kalsbeek WD, Hogelin GC, Jones JT, Gaines KL, Forman MR, et al. The behavioral risk factor surveys: II. Design, methods, and estimates from combined state data. Am J Prev Med. 1985;1:9–14. [PubMed]

9. Remington PL, Smith MY, Williamson DF, Anda RF, Gentry EM, Hogelin GC. Design, characteristics, and usefulness of state-based behavioral risk factor surveillance: 1981–87. Public Health Rep. 1988;103:366–75. [PMC free article] [PubMed]

10. Andresen EM, Catlin TK, Wyrwich KW, Jackson-Thompson J. Retest reliability of surveillance questions on health-related quality of life. J Epidemiol Community Health. 2003;57:339–43. [PMC free article] [PubMed]

11. Moriarty DG, Kobau R, Zack MM, Zahran HS. Tracking healthy days—a window on the health of older adults. Prev Chronic Dis. 2005;2:A16. [PMC free article] [PubMed]

12. Zullig KJ, Valois RF, Huebner ES, Drane JW. Evaluating the performance of the Centers for Disease Control and Prevention core health-related quality of life scale with adolescents. Public Health Rep. 2004;119:577–84. [PMC free article] [PubMed]

13. Ounpuu S, Chambers LW, Chan D, Yusuf S. Validity of the U.S. Behavioral Risk Factor Surveillance System's health-related quality of life survey tool in a group of older Canadians. Chronic Dis Can. 2001;22:93–101. [PubMed]

14. Toet J, Raat H, van Ameijden EJ. Validation of the Dutch version of the CDC core healthy days measures in a community sample. Qual Life Res. 2006;15:179–84. [PubMed]

15. Moum TA. Health-related quality of life in Norway—a nationwide mixed-mode panel study of the CDC healthy days measures. Proceedings of the CDC Quality of Life Seminar; 1999 Jul 28; Atlanta.

16. Health-related quality of life among adults with arthritis—Behavioral Risk Factor Surveillance System, 11 states, 1996–1998. JAMA. 2000;283:2783–5. [PubMed]

17. Self-rated fair or poor health among adults with diabetes—United States, 1996–2005. MMWR Morb Mortal Wkly Rep. 2006;55(45):1124–7. [PubMed]

18. Ford ES, Moriarty DG, Zack MM, Mokdad AH, Chapman DP. Self-reported body mass index and health-related quality of life: findings from the Behavioral Risk Factor Surveillance System. Obes Res. 2001;9:21–31. [PubMed]

19. DeVellis RF. Scale development: theory and applications. Newbury Park (CA): Sage Publications; 1991.

20. Mielenz T, Jackson E, Currey S, DeVellis R, Callahan LF. Psychometric properties of the Centers for Disease Control and Prevention Health-Related Quality of Life (CDC HRQOL) items in adults with arthritis. Health Qual Life Outcomes. 2006;4:66. [PMC free article] [PubMed]

21. Lawlor DA, Ebrahim S, May M, Smith G Davey. (Mis)use of factor analysis in the study of insulin resistance syndrome. Am J Epidemiol. 2004;159:1013–8. [PubMed]

22. Nunnally JC. Psychometric theory. 2nd ed. New York: McGraw Hill; 1978.

23. Idler EL, Benyamini Y. Self-rated health and mortality: a review of twenty-seven community studies. J Health Soc Behav. 1997;38:21–37. [PubMed]

24. SPSS, Inc. SPSS: Version 14.0 for Windows. Chicago: SPSS, Inc.; 2006.

25. SAS Institute Inc. SAS: Version 9.1. Cary (NC): SAS Institute Inc.; 2003.

26. Research Triangle Institute. SUDAAN: Version 9.0.1. Research Triangle Park (NC): Research Triangle Institute; 2004.

27. Idler EL, Hudson SV, Leventhal H. The meanings of self-ratings of health: a qualitative and quantitative approach. Res Aging. 1999;21:458–76.

28. Krause NM, Jay GM. What do global self-rated health items measure? Med Care. 1994;32:930–42. [PubMed]

29. Smith KW, Avis NE, Assmann SF. Distinguishing between quality of life and health status in quality of life research: a meta-analysis. Qual Life Res. 1999;8:447–59. [PubMed]

30. Ford ES, Mannino DM, Homa DM, Gwynn C, Redd SC, Moriarty DG, et al. Self-reported asthma and health-related quality of life: findings from the Behavioral Risk Factor Surveillance System. Chest. 2003;123:119–27. [PubMed]

31. Mokdad AH, Stroup DF, Giles WH. Behavioral Risk Factor Surveillance Team. Public health surveillance for behavioral risk factors in a changing environment. Recommendations from the Behavioral Risk Factor Surveillance Team. MMWR Recomm Rep. 2003;52(RR-9):1–12. [PubMed]

Articles from Public Health Reports are provided here courtesy of
Association of Schools of Public Health