Search tips
Search criteria 


Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
COPD. Author manuscript; available in PMC 2009 December 1.
Published in final edited form as:
PMCID: PMC2667628

The COPD Severity Score: A Dynamic Prediction Tool for Health-Care Utilization

Theodore A Omachi, MD MBA,1 Edward H. Yelin, PhD,4,5 Patricia P. Katz, PhD,4 Paul D. Blanc, MD MSPH,1,6 and Mark D. Eisner, MD MPH1,3,6


A survey-based COPD severity scoring algorithm could prove useful for targeted disease management and risk-adjustment. For this purpose, we sought to prospectively validate a COPD severity score that had previously been cross-sectionally validated. Using a population-based sample of 267 adults with self-reported physician-diagnosed COPD, we examined the extent to which this COPD severity score predicts future respiratory hospitalizations, emergency department (ED) visits, and outpatient visits. Structured telephone interviews, conducted at baseline and on an annual basis in two subsequent years, determined COPD severity scores and health-care utilization. A basic predictive model for respiratory-specific health-care utilization was developed using sociodemographics, tobacco history, and medical comorbidity data in multivariate logistic regression analysis. The added predictive value of the COPD severity score over and above this basic model was then evaluated by testing the change in model concordance indices. Our analysis demonstrated that the COPD severity score did in fact increase the concordance-index of models predicting future respiratory hospitalizations (increase from 80% to 87%; p=0.03), ED visits (64% to 82%, p=0.003), and outpatient visits (59% to 77%, p<0.0001). In a separate analysis, both a greater baseline severity score and worsening of the severity score over time were prospectively associated with each outcome studied (p<0.05 for each). In conclusion, the COPD severity score adds predictive value in estimating future respiratory-specific health-care utilization and is longitudinally responsive to evolving changes in COPD status over time. This severity score may be used to adjust for disease severity or to identify high-risk populations.

Keywords: chronic bronchitis, pulmonary disease, chronic obstructive, outcome assessment (health care), pulmonary emphysema, severity-of-illness index


Chronic obstructive pulmonary disease (COPD) is estimated to cost over $37 billion annually in the United States, with hospitalizations representing 68% of this cost (1-3). Some of these hospitalizations may be avoidable (4-6), and for this reason, the Agency for Health Care Research and Quality and others have designated COPD as an ambulatory care sensitive condition (7). Consequently, methods to assess the risk of hospitalizations and other health-care utilization outcomes could prove useful for targeting ambulatory disease-management programs to patients most likely to benefit (8). Additionally, health-based risk adjustment is necessary for accurately determining capitated payment rates, identifying provider-level variation in the use of health care services, or risk-adjusting health care outcomes (9, 10). Furthermore, prior investigators have concluded that disease-specific scales rather than generic measures should be developed and tested to capture the most information for prediction (11).

Using cross-sectional data collected from a population-based sample of US adults with COPD, we previously developed and validated a COPD severity scoring algorithm.(12) This COPD score was shown to correlate with concurrent measures of pulmonary function (including forced expiratory volume in 1 second [FEV1] and peak expiratory flow rate) and with generic physical health and respiratory-specific health-related quality-of-life (HRQL) (12, 13). Subsequently, this COPD severity scoring algorithm has been used in another clinical cohort of COPD in the United States and translated and applied in a field study in Spain (14-16).

To establish this COPD severity’s scores validity as a tool for risk-adjustment and identification of high-risk patients, it must be shown to be predictive of health care outcomes. In this current analysis, we have therefore used longitudinal follow-up from the original cohort on which this COPD severity score was developed to examine the extent to which both the baseline score and change in the score are prospectively associated with health care utilization outcomes.



In a population-based sample of US adults with COPD, we examined the extent to which the previously developed and validated COPD severity score predicts disease-specific health-care utilization. Subjects were followed through four longitudinal waves of structured telephone interviews conducted on an annual basis. The first wave was used to develop and cross-sectionally validate the COPD severity score, as previously described (12). In this analysis, the remaining three waves of annual interviews were used to prospectively validate this COPD severity scoring algorithm as a predictor of disease-specific health care utilization. The study was approved by the University of California, San Francisco Committee on Human Research.

Subject Selection

Survey methods have previously been described in detail (12, 13, 17, 18). Subjects were selected from an initial sample of 2,113 US adults, aged 55 to 75 years, identified by random-digit dialing structured telephone interviews conducted in 2001. Initial telephone interviews were made during early evening hours and throughout the day on weekends; up to six further attempts to make initial contact were made at different times of day. Interviews were offered in both English and Spanish. Approximately one half of the overall sample (n = 1,001) was randomly identified among residents in the 48 contiguous states of the Unites States. In an attempt to increase the yield of identifying patients with COPD, the remainder of the sample (n=1,112) was drawn from geographic “hot spots”, based on Health Service Areas with the highest COPD mortality rates, derived from the National Institute of Occupational and Health’s Atlas of Respiratory Disease Mortality, United States: 1982-1993 (19). The telephone area codes corresponding best to the areas in the highest quartile of age-adjusted COPD mortality rates were selected in order to increase the recruitment of individuals with COPD.. The overall study participation rate was 53% among households meeting an initial brief eligibility screening.

During the initial recruitment interview, subjects were asked if they had ever received a physician’s diagnosis of any of several chronic respiratory conditions. As in prior analyses, subjects were classified as having COPD for the current analysis if they reported, in response to a specific query during the structured telephone interview, that a physician had diagnosed them with chronic bronchitis, emphysema, or specifically with COPD (17). We included subjects reporting a concomitant physician’s diagnosis of asthma because they clinically resemble persons with COPD alone (20). Using this criteria, the prevalence of COPD in households with subjects of appropriate age was 16.5% in the sample drawn from “hot spot” areas and 13.5% in sample drawn from randomly-identified areas of the United States (p=0.09).

From the 2,113 subjects interviewed in 2001, 383 met criteria for having COPD and were included in the original cross-sectional development of the COPD severity score. 267 subjects (70%) were re-interviewed in 2002 using these data to recalculate COPD severity scores. Thus we used 2002 data (subsequent to the original score development) as a “derivation cohort” for the development of our risk models (as described below). In subsequent longitudinal follow-up, 204 (76% of subjects interviewed in 2002) completed the 2003 interview, and 173 (85% of the 2003 wave; 65% of the derivation cohort) completed the 2004 interview.

There were no statistically significant differences among the various interview waves in terms of average values of age, gender, martial status, education, or COPD severity score for those who did and did not complete follow-up interviews (p>0.20 in all pair-wise comparisons). African-Americans, however, were less prevalent in follow-up interviews (5% prevalence in those who completed all waves of interviews versus 10% prevalence in those who completed all but the final interview; p=.05). Mortality accounted for at least 12 of 104 who were not re-interviewed; other deaths may have led to losses to follow-up not clearly identified as deaths.

To take losses to follow-up into account, probability-of-attrition weights were developed using all of the factors listed in Table 2 (21-23). This analysis developed a multivariate logistic regression model to determine the probability of follow-up and then weights subjects based on their likelihood of follow-up such that those who are less likely to reinterview are weighted more heavily. Probability-of-attrition weights have been shown to reduce bias from non-response in longitudinal cohort studies (21-23). However, the weighted analysis was not substantively different from the unweighted analysis, so only the unweighted analysis is reported here.

Table 2
Characteristics of 267 adults with COPD

COPD Severity Score

The COPD severity score is based on responses to survey items that comprise five domains of COPD severity: [1] degree of respiratory symptoms, [2] prior systemic corticosteroid use, [3] other COPD medication use, [4] previous hospitalization or intubation for respiratory disease, and [5] home oxygen use (Table 1) (12). The possible COPD severity score range is 0 to 35, with higher scores reflecting more severe disease. The scoring algorithm takes into account both current respiratory symptoms and the therapy necessary to achieve this symptom status. For example, a person with few respiratory symptoms who is receiving albuterol, ipratropium and salmeterol for COPD would have a higher COPD severity score than another person with comparable symptoms who was only taking albuterol and ipratropium. In developing the COPD severity score in prior published analyses, each survey item was assigned an a priori weight based on clinical aspects of the disease and its expected contribution to overall COPD severity. This weighting scheme was validated by showing that it correlated very closely with a score whose alternative weighting was derived from factor analysis with orthogonal rotation (12). Prior analyses have also demonstrated internal consistency reliability and concurrent validity (correlation with pulmonary function, physical HRQL, physical disability, and respiratory-specific HRQL) (12, 13, 16).

Table 1
COPD Severity Score Items (n = 267)

Survey data

Structured telephone interviews obtained survey data using computer assisted telephone interviewing (CATI) software (17). Each of the four annual structured interviews was administered through the same professional survey firm using trained interviewers (Field Research Corporation, San Francisco, California). Interviewers obtained sociodemographic data including whether respondents considered their racial background to be Non-Hispanic White, African- American / Black, Asian / Pacific Islander, Latino / Hispanic, or other. As previously described, we defined educational attainment as high school or less, some college, or completed college or greater (17). Tobacco usage was measured using questions developed for the National Health Interview Survey (24). Inputs for calculating the COPD severity score were assessed with each of the four annual structured interviews.

For outpatient utilization, we ascertained the number of respiratory-specific visits to a medical office, health clinic, or hospital outpatient center in the previous 12 months. Number of visits was then dichotomized for analysis using two or more outpatient visits as the cut-off value (defined a priori based on the median value). Respiratory-specific ED visits and hospitalizations over the prior 12 months were dichotomized as either any events or no events.

Statistical Analysis

Statistical analysis used Stata/SE software (version 9.2, College Station, TX). The predictive accuracy of logistic regression risk models was assessed by calculating a concordance index (c-index), which is the area under the receiver-operating characteristics (ROC) curve (25). A c-index of 1.0 reflect perfect prediction, with both test sensitivity and specificity being equal to 1. A c-index of 0.5 reflects discrimination no better than chance alone.

To test the extent to which the COPD severity score added predictive value to a risk model, we used a two step analytic process: [1] development of an initial logistic regression model (“Model 1”) to predict outcomes assessed in 2003 using clinical and sociodemographic factors assessed in our 2002 derivation cohort and then [2] addition of the COPD severity score to this initial prediction model to create “Model 2” which was then compared to “Model 1” to determine the incremental predictive value of the COPD severity score (i.e., the incremental c-index).

The c-index values for these risk models after addition of the COPD severity score were compared to the c-index of the models without the COPD severity score using a nonparametric approach described by Delong et al.(26). This method takes into account the correlated nature of the data (because the two different ROC curves are based on the same individuals), and the resulting test statistic uses a χ2 distribution to assess statistical likelihood.

Our goal in developing the initial prediction model was to maximize that model’s c-index. This provided the most stringent test of the additional predictive value of the COPD severity score; the higher the initial c-index, the more difficult it would be for an additional covariate (such as the COPD severity score) to result in a statistically significant increase in the c-index (25).

To develop the initial prediction model, we therefore determined the combination of covariates that maximized the c-index in a logistic regression analysis predicting the risk of respiratory-specific hospitalizations over a one-year period. To be consistent, we used this same combination of covariates in our initial prediction models for respiratory-specific ED visits and outpatient visits; we did not re-determine a different combination of covariates for each outcome. The covariates tested were age, gender, marital status, race, educational attainment, tobacco history, and self-reported medical comorbidities. We specifically tested comorbidities that are known to be associated with COPD: congestive heart failure, coronary artery disease (CAD), diabetes (through its effect on cardiovascular disease), and sleep apnea (27, 28).

Additionally, after completing this two-step analytic process, we performed an internal validation using another wave of follow-up data (2003 survey data as predictors of health-care utilization outcomes over the following year, as assessed in 2004) (29). The purpose of this internal validation was to determine whether the COPD severity score increased the c-index in more than one time period. For consistency, the initial prediction model (without COPD severity score) in this internal validation incorporated the same covariates as did the initial prediction model from the derivation cohort. For all logistic regression models, in both derivation cohort and internal validation, the Hosmer-Lemeshow test revealed adequate goodness-of-fit (p-values ranged from 0.13 to 0.96) (30).

We also evaluated the extent to which change in the COPD severity score was predictive of future health-care utilization. We used the t-test to determine whether the change in the score was statistically different than 0 (i.e., “no change”). We then constructed a model to predict health-care utilization that incorporated our sociodemographic, tobacco history and medical comorbidity covariates along with both [1] the absolute value of the COPD severity score and [2] the one-year patient-specific change in the score. We used a repeated measures analysis that incorporated both waves of follow-up data simultaneously. Specifically, factors from one wave were used as predictors of the subsequent wave outcomes (i.e. 2002 data was used to predict outcomes assessed in 2003, and 2003 data was used to predict outcomes assessed in 2004). We used generalized estimating equations with robust standard errors to take account of the fact that a given subject might contribute more than one observation to the analysis (31). All reported odds ratios for the COPD severity score were scaled by one-half of their standard deviation, which is generally considered to approximate a minimally clinically significant change (32).

Finally, we constructed risk nomograms in which we modeled the likelihood of health-care utilization over the possible range of COPD severity scores. In this analysis, the COPD severity score was modeled at the average value of the sociodemographic, tobacco history, and medical comorbidity covariates, thus controlling for these potential confounders (33).


Subject Characteristics

The mean age of the COPD subjects was 65 ± 6 years. The majority of subjects were non-Hispanic white (85%). Most subjects (82%) were former or current cigarette smokers. The baseline mean COPD severity score was 7.8 (SD=6.3). The mean baseline COPD severity score of subjects identified in geographic “hot spots” for COPD mortality was 7.7 (SD=6.2) while the mean baseline score for subjects recruited from the national random sample was 8.0 (SD=6.5) (p=0.70). Other subject characteristics are presented in Table 2.

Prediction Model

The covariates for the initial prediction model (without COPD severity score) were: age, race, education, tobacco history, and all of our tested self-reported medical comorbidities (Table 2). The c-index for predicting hospitalization demonstrated that the initial prediction model (without COPD severity score) had good discriminative ability: 80% in the derivation cohort and 79% when the model was tested for internal validation using the subsequent year data (model p<0.05 in both cases). The COPD severity score added substantive predictive value to the risk model for hospitalizations in both the derivation cohort and internal validation, raising the c-index to 87% and 91% respectively (p-values for the increase in c-index were 0.03 and 0.048 respectively).

For ED and outpatient visits, the initial prediction performed somewhat differently (Table 3). In all cases the c-index was less than that for hospitalization, and the initial prediction models were statistically significant only in the internal validation. However, the COPD severity score significantly improved the prediction models for ED and outpatient visits in both the derivation cohort and the internal validation.

Table 3
Logistic regression prediction models of health-care utilization for respiratory problems

Changes in COPD severity score over time

Figure 1 demonstrates the change in the COPD severity scores over a two-year time frame. There was a mean two-year increase in score of 0.65 (SE=0.29, p=0.03), consistent with an overall worsening in COPD severity over time. When annual change in the score for any given subject was modeled along with the absolute baseline value of the score, both the baseline value and the change were statistically significant predictors of future respiratory hospitalizations, ED visits and outpatient visits (Table 4).

Figure 1
Change in COPD Severity Score over time
Table 4
The prospective impact of baseline and annual change in COPD Severity Score

Risk Nomograms

The predicted probability of an individual patient having a respiratory-related hospitalization, ED visit, or two or more outpatient visits for all potential values of the COPD severity score, corrected for other covariates, is shown in Figure 2. For example, the average value of the COPD severity score at baseline for our cohort was 7.8, which corresponds to a 5% risk of hospitalization, an 11% risk of ED visit, and a 49% chance of two or more outpatient visits for respiratory conditions in the following year. A score two standard deviations above the mean (20.4) corresponds to a 32% risk of hospitalization, a 48% risk of ED visit, and a 90% likelihood of two or more outpatient visits over the ensuing twelve months.

Figure 2
COPD severity score as a predictor of utilization outcomes. COPD severity score is modeled at the average value of the model covariates: age, race, education, tobacco history, and medical comorbidities (heart failure, coronary artery disease, diabetes, ...


In a population-based cohort, the COPD severity score adds predictive value in estimating future respiratory-related hospitalizations, ED visits, and outpatient visits. Moreover, the COPD severity score changes over time, indicating that the score is not static, and these changes are themselves prospectively associated with health-care utilization, indicating that the score is responsive to change in a clinically relevant manner.

This COPD severity instrument may have several applications. It may be used as a tool to risk-adjust for disease severity within epidemiologic or outcomes research. The score may also be used to identify those at greater risk of hospitalizations or ED visits, possibly allowing resource-limited disease management programs to target patients at highest risk. Although the severity scoring instrument may be time-consuming to administer face-to-face in the context of a physician visit, the increasing availability of prior utilization data through electronic health records may assist in large-scale calculation of the score (34, 35). Alternatively, the score may potentially be derived from self-reported questionnaires. Importantly, the COPD severity score does not require measurement of pulmonary function, physical examination or other diagnostic testing, which may greatly facilitate its usage in population-based studies and in targeting high-resource utilizers. Finally, because the COPD severity score is responsive to change, it may be valuable, after further validation, as a measure of current clinical status or as an end-point of therapeutic interventions for COPD. We are currently planning further work on a different cohort of COPD subjects to establish the minimum clinically significant score change.

The risk nomogram assists with interpreting the COPD severity score’s association with utilization outcomes. Not surprisingly in a population-based cohort, hospitalizations and ED visits are a relatively uncommon event for subjects with average COPD severity. However, approximately 10% of the cohort had a COPD severity score greater than 15, which corresponds to a one-year hospitalization risk of about 17%. The threshold at which clinicians or disease management programs might target individuals would likely depend on resource constraints but ideally would target those above some threshold of risk (36). To place this in context, Kaiser Permanente’s extensive disease management program involved approximately 14% of their asthmatics over a 4-year time-frame (8). Thus it might be reasonable to choose some value of COPD severity score close to 15 to target a limited subset of the COPD population at significant risk of hospitalization.

Our approach to recruiting and following subjects with COPD resulted in both strengths and limitations. A major advantage of our population-based recruitment is the generalizability of findings to the population of COPD patients. This is particularly important if one is choosing a hospitalization-risk threshold for targeted intervention in a general population. One potential drawback of our approach is that we have relied on self-reported physician diagnosis, which may have resulted in some disease misclassification. However, our method for identifying subjects with COPD is a standard epidemiologic technique (37, 38). Also, as previously reported, the prevalence of COPD in the initial national sample (not including subjects surveyed in “hot-spots” for COPD) was 13.5%, which is similar to the 12.5% national COPD prevalence rate reported in the Third National Health and Nutrition Examination Survey (NHAINES III) (37, 38). Furthermore, the 21% prevalence rate in our cohort of self-reported CAD is close to the 19% prevalence rate of angina pectoris or prior myocardial infarction reported among older adults who attended the medical examination portion of NHANES III (39). These findings support the validity of our results.

Another limitation is our use of self-report to identify health-care utilization, which may have resulted in recall bias. The use of self-reported events is nonetheless supported by other studies, although there is a suggestion that older persons may underreport health-care utilization (40-42). Underreporting would likely cause underestimation of the effect of the COPD severity score on utilization. Loss to follow-up is an additional limitation. It is difficult to be sure of the effect of subject attrition. However, if subjects lost to follow-up had more severe COPD with higher health care utilization (e.g. at least 12 subjects did not reinterview because of death), then we would expect that subject attrition would tend to cause an underestimation of the incidence of hospitalization and possibly also the predictive ability of the COPD severity score. Regardless, subjects lost to follow-up did not appear statistically significantly different with respect to most sociodemographic factors and the COPD severity score. Furthermore, a probability-of-attrition weighted analysis did not change the results substantively, indicating that losses to follow-up did not affect our findings (data not shown). Nonetheless, the fact that African Americans were less likely to participate in the final interview may limit our ability to make strong conclusions about this racial-ethnic group.

Overall, the COPD severity score demonstrates prospective validity as a survey-based instrument for risk adjustment or the identification of high-risk cohorts. The severity score added predictive value to our estimate of future health-care utilization over and above a basic predictive model, and it did so in more than one time period. Moreover, the change in the severity score is itself associated with utilization outcomes, suggesting that the score is longitudinally responsive to changes in COPD status. Future work is needed to compare the score with other risk indices, determine a minimally clinically significant score change, evaluate the score’s predictive ability in other cohorts and for other important outcomes such as mortality, and to determine its ability to predict costs and thus its potential for use in risk-adjusting capitated payments. Nonetheless, this COPD severity score appears to have valuable potential as a disease-specific risk index for one of the leading causes of death and disability worldwide.


Financial support was provided by the National Institutes of Health grant R01 HL067438 from the National Heart, Lung, and Blood Institute. Dr. Omachi was supported by National Heart, Lung, and Blood Institute, grant number T32 HL007185. Dr. Eisner was supported by R01HL077618 National Heart, Lung, and Blood Institute, National Institutes of Health, with co-funding by the Social Security Administration


1. National Heart, Lung, and Blood Institute. Morbidity and Mortality: 2004 Chart Book on Cardiovascular, Lung, and Blood Diseases. National Heart, Lung, and Blood Institute; 2004.
2. Hurd S. The impact of COPD on lung health worldwide: epidemiology and incidence. Chest. 2000;117:1S–4S. [PubMed]
3. Strassels SA, Smith DH, Sullivan SD, Mahajan PS. The costs of treating COPD in the United States. Chest. 2001;119:344–52. [PubMed]
4. O’Malley AS, Pham HH, Schrag D, Wu B, Bach PB. Potentially avoidable hospitalizations for COPD and pneumonia: the role of physician and practice characteristics. Med Care. 2007;45:562–70. [PubMed]
5. Billings J, Zeitel L, Lukomnik J, Carey TS, Blank AE, Newman L. Impact of socioeconomic status on hospital use in New York City. Health Aff (Millwood) 1993;12:162–73. [PubMed]
6. Weissman JS, Gatsonis C, Epstein AM. Rates of avoidable hospitalization by insurance status in Massachusetts and Maryland. Jama. 1992;268:2388–94. [PubMed]
7. AHRQ Quality Indicators. Guide to prevention quality indicators: hospital admission for ambulatory care sensitive conditions. Rockville, MD: Agency for Health Care Research and Quality; 2001.
8. Fireman B, Bartlett J, Selby J. Can disease management reduce health care costs by improving quality? Health Aff (Millwood) 2004;23:63–75. [PubMed]
9. Kenneth P, O’Malley K, Byrne M, Souchek J, Petersen N, Ashton C. Physician-level variation in practice patterns in the VA healthcare system. Health Services Outcomes Res Methodol. 2002;3:95–106.
10. Petersen LA, Pietz K, Woodard LD, Byrne M. Comparison of the predictive validity of diagnosis-based risk adjusters for clinical outcomes. Med Care. 2005;43:61–7. [PubMed]
11. Hornbrook MC, Goodman MJ. Chronic disease, functional health status, and demographics: a multi-dimensional approach to risk adjustment. Health Serv Res. 1996;31:283–307. [PMC free article] [PubMed]
12. Eisner MD, Trupin L, Katz PP, Yelin EH, Earnest G, Balmes J, Blanc PD. Development and validation of a survey-based COPD severity score. Chest. 2005;127:1890–7. [PubMed]
13. Chen H, Eisner MD, Katz PP, Yelin EH, Blanc PD. Measuring disease-specific quality of life in obstructive airway disease: validation of a modified version of the airways questionnaire 20. Chest. 2006;129:1644–52. [PubMed]
14. Eisner MD, Katz P, Iribarren C, Yelin EH, Blanc PD. Measurement of COPD severity: further validation of a survey-based instrument. Eur Resp J. 2007;30:629S.
15. Miravitlles M, Llor C, Donado E, Ferrer M. Validation of the COPD Severity Score for Use in the Spanish Primary Care Setting, the NEREA Study. Am J Respir Crit Care Med. 2007;175:A850.
16. Miravitlles M, Llor C, Donado E, Perez I, Ferrer M. COPD severity score and physical activity in Spanish primary care patients: the NEREA study. Eur Respir J. 2007;30(Suppl 51):771S.
17. Trupin L, Earnest G, San Pedro M. The occupational burden of chronic obstructive pulmonary disease. Eur Respir J. 2003;22:462–9. [PubMed]
18. Blanc PD, Eisner MD, Trupin L, Yelin EH, Katz PP, Balmes JR. The association between occupational factors and adverse health outcomes in chronic obstructive pulmonary disease. Occup Environ Med. 2004;61:661–7. [PMC free article] [PubMed]
19. Kim J. Atlas of respiratory disease mortality, United States: 1982-1993. Cincinnati, OH: Department of Health and Human Services, National Institute for Occupational Safety and Health; 1998.
20. Eisner MD, Yelin EH, Trupin L, Blanc PD. The influence of chronic respiratory conditions on health status and work disability. Am J Public Health. 2002;92:1506–13. [PubMed]
21. Little RJA, Rubin DB. Statistical Analysis with Missing Data. Second Edition. Hoboken, NJ: John Wiley & Sons; 2002.
22. Rao RS, Sigurdson AJ, Doody MM, Graubard BI. An application of a weighting method to adjust for nonresponse in standardized incidence ratio analysis of cohort studies. Ann Epidemiol. 2005;15:129–36. [PubMed]
23. Schafer JL, Graham JW. Missing data: our view of the state of the art. Psychol Methods. 2002;7:147–77. [PubMed]
24. Cigarette smoking among adults--United States, 1997. MMWR Morb Mortal Wkly Rep. 1999 Nov 5;:993–6. [PubMed]
25. Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. 1982;143:29–36. [PubMed]
26. DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. 1988;44:837–45. [PubMed]
27. Sidney S, Sorel M, Quesenberry CP, Jr, DeLuise C, Lanes S, Eisner MD. COPD and incident cardiovascular disease hospitalizations and mortality: Kaiser Permanente Medical Care Program. Chest. 2005;128:2068–75. [PubMed]
28. Chaouat A, Weitzenblum E, Krieger J, Ifoundza T, Oswald M, Kessler R. Association of chronic obstructive pulmonary disease and sleep apnea syndrome. Am J Respir Crit Care Med. 1995;151:82–6. [PubMed]
29. Miller MK, Lee JH, Blanc PD, Pasta DJ, Gujrathi S, Barron H, Wenzel SE, Weiss ST. TENOR risk score predicts healthcare in adults with severe or difficult-to-treat asthma. Eur Respir J. 2006;28:1145–55. [PubMed]
30. Hosmer DW, Lemeshow S. Applied logistic regression. Hoboken, NJ: John Wiley & Sons, Inc; 2000.
31. Vittinghoff E, Glidden DV, Shiboski SC, McCulloch CE. Regression Methods in Biostatistics: Linear, Logistic, Survival, and Repeated Measures Models. New York: Springer Science & Business Media, Inc; 2005.
32. Norman GR, Sloan JA, Wyrwich KW. Interpretation of changes in health-related quality of life: the remarkable universality of half a standard deviation. Med Care. 2003;41:582–92. [PubMed]
33. Harrell FEJ. Regression modeling strategies. New York: Springer Science & Business Media, Inc; 2001.
34. Ford EW, Menachemi N, Phillips MT. Predicting the adoption of electronic health records by physicians: when will health care be paperless? J Am Med Inform Assoc. 2006;13:106–12. [PMC free article] [PubMed]
35. Mayo NE, Nadeau L, Levesque L, Miller S, Poissant L, Tamblyn R. Does the addition of functional status indicators to case-mix adjustment indices improve prediction of hospitalization, institutionalization, and death in the elderly? Med Care. 2005;43:1194–202. [PubMed]
36. Monninkhof E, van der Valk P, Schermer T, van der Palen J, van Herwaarden C, Zielhuis G. Economic evaluation of a comprehensive self-management programme in patients with moderate to severe chronic obstructive pulmonary disease. Chron Respir Dis. 2004;1:7–16. [PubMed]
37. Mannino DM, Homa DM, Akinbami LJ, Ford ES, Redd SC. Chronic obstructive pulmonary disease surveillance--United States, 1971-2000. MMWR Surveill Summ. 2002;51:1–16. [PubMed]
38. National Center for Health Statistics. Plan and Operation of the Third National Health and Nutrition Examination Survey, 1988-1994. Washington DC: US Dept of Health and Human Services; 1994.
39. Ford ES, Giles WH, Croft JB. Prevalence of nonfatal coronary heart disease among American adults. Am Heart J. 2000;139:371–7. [PubMed]
40. Petrou S, Murray L, Cooper P, Davidson LL. The accuracy of self-reported healthcare resource utilization in health economic studies. Int J Technol Assess Health Care. 2002;18:705–10. [PubMed]
41. Raina P, Torrance-Rynard V, Wong M, Woodward C. Agreement between self-reported and routinely collected health-care utilization data among seniors. Health Serv Res. 2002;37:751–74. [PMC free article] [PubMed]
42. Richards SH, Coast J, Peters TJ. Patient-reported use of health service resources compared with information from health providers. Health Soc Care Community. 2003;11:510–8. [PubMed]