|Home | About | Journals | Submit | Contact Us | Français|
A survey-based COPD severity scoring algorithm could prove useful for targeted disease management and risk-adjustment. For this purpose, we sought to prospectively validate a COPD severity score that had previously been cross-sectionally validated. Using a population-based sample of 267 adults with self-reported physician-diagnosed COPD, we examined the extent to which this COPD severity score predicts future respiratory hospitalizations, emergency department (ED) visits, and outpatient visits. Structured telephone interviews, conducted at baseline and on an annual basis in two subsequent years, determined COPD severity scores and health-care utilization. A basic predictive model for respiratory-specific health-care utilization was developed using sociodemographics, tobacco history, and medical comorbidity data in multivariate logistic regression analysis. The added predictive value of the COPD severity score over and above this basic model was then evaluated by testing the change in model concordance indices. Our analysis demonstrated that the COPD severity score did in fact increase the concordance-index of models predicting future respiratory hospitalizations (increase from 80% to 87%; p=0.03), ED visits (64% to 82%, p=0.003), and outpatient visits (59% to 77%, p<0.0001). In a separate analysis, both a greater baseline severity score and worsening of the severity score over time were prospectively associated with each outcome studied (p<0.05 for each). In conclusion, the COPD severity score adds predictive value in estimating future respiratory-specific health-care utilization and is longitudinally responsive to evolving changes in COPD status over time. This severity score may be used to adjust for disease severity or to identify high-risk populations.
Chronic obstructive pulmonary disease (COPD) is estimated to cost over $37 billion annually in the United States, with hospitalizations representing 68% of this cost (1-3). Some of these hospitalizations may be avoidable (4-6), and for this reason, the Agency for Health Care Research and Quality and others have designated COPD as an ambulatory care sensitive condition (7). Consequently, methods to assess the risk of hospitalizations and other health-care utilization outcomes could prove useful for targeting ambulatory disease-management programs to patients most likely to benefit (8). Additionally, health-based risk adjustment is necessary for accurately determining capitated payment rates, identifying provider-level variation in the use of health care services, or risk-adjusting health care outcomes (9, 10). Furthermore, prior investigators have concluded that disease-specific scales rather than generic measures should be developed and tested to capture the most information for prediction (11).
Using cross-sectional data collected from a population-based sample of US adults with COPD, we previously developed and validated a COPD severity scoring algorithm.(12) This COPD score was shown to correlate with concurrent measures of pulmonary function (including forced expiratory volume in 1 second [FEV1] and peak expiratory flow rate) and with generic physical health and respiratory-specific health-related quality-of-life (HRQL) (12, 13). Subsequently, this COPD severity scoring algorithm has been used in another clinical cohort of COPD in the United States and translated and applied in a field study in Spain (14-16).
To establish this COPD severity’s scores validity as a tool for risk-adjustment and identification of high-risk patients, it must be shown to be predictive of health care outcomes. In this current analysis, we have therefore used longitudinal follow-up from the original cohort on which this COPD severity score was developed to examine the extent to which both the baseline score and change in the score are prospectively associated with health care utilization outcomes.
In a population-based sample of US adults with COPD, we examined the extent to which the previously developed and validated COPD severity score predicts disease-specific health-care utilization. Subjects were followed through four longitudinal waves of structured telephone interviews conducted on an annual basis. The first wave was used to develop and cross-sectionally validate the COPD severity score, as previously described (12). In this analysis, the remaining three waves of annual interviews were used to prospectively validate this COPD severity scoring algorithm as a predictor of disease-specific health care utilization. The study was approved by the University of California, San Francisco Committee on Human Research.
Survey methods have previously been described in detail (12, 13, 17, 18). Subjects were selected from an initial sample of 2,113 US adults, aged 55 to 75 years, identified by random-digit dialing structured telephone interviews conducted in 2001. Initial telephone interviews were made during early evening hours and throughout the day on weekends; up to six further attempts to make initial contact were made at different times of day. Interviews were offered in both English and Spanish. Approximately one half of the overall sample (n = 1,001) was randomly identified among residents in the 48 contiguous states of the Unites States. In an attempt to increase the yield of identifying patients with COPD, the remainder of the sample (n=1,112) was drawn from geographic “hot spots”, based on Health Service Areas with the highest COPD mortality rates, derived from the National Institute of Occupational and Health’s Atlas of Respiratory Disease Mortality, United States: 1982-1993 (19). The telephone area codes corresponding best to the areas in the highest quartile of age-adjusted COPD mortality rates were selected in order to increase the recruitment of individuals with COPD.. The overall study participation rate was 53% among households meeting an initial brief eligibility screening.
During the initial recruitment interview, subjects were asked if they had ever received a physician’s diagnosis of any of several chronic respiratory conditions. As in prior analyses, subjects were classified as having COPD for the current analysis if they reported, in response to a specific query during the structured telephone interview, that a physician had diagnosed them with chronic bronchitis, emphysema, or specifically with COPD (17). We included subjects reporting a concomitant physician’s diagnosis of asthma because they clinically resemble persons with COPD alone (20). Using this criteria, the prevalence of COPD in households with subjects of appropriate age was 16.5% in the sample drawn from “hot spot” areas and 13.5% in sample drawn from randomly-identified areas of the United States (p=0.09).
From the 2,113 subjects interviewed in 2001, 383 met criteria for having COPD and were included in the original cross-sectional development of the COPD severity score. 267 subjects (70%) were re-interviewed in 2002 using these data to recalculate COPD severity scores. Thus we used 2002 data (subsequent to the original score development) as a “derivation cohort” for the development of our risk models (as described below). In subsequent longitudinal follow-up, 204 (76% of subjects interviewed in 2002) completed the 2003 interview, and 173 (85% of the 2003 wave; 65% of the derivation cohort) completed the 2004 interview.
There were no statistically significant differences among the various interview waves in terms of average values of age, gender, martial status, education, or COPD severity score for those who did and did not complete follow-up interviews (p>0.20 in all pair-wise comparisons). African-Americans, however, were less prevalent in follow-up interviews (5% prevalence in those who completed all waves of interviews versus 10% prevalence in those who completed all but the final interview; p=.05). Mortality accounted for at least 12 of 104 who were not re-interviewed; other deaths may have led to losses to follow-up not clearly identified as deaths.
To take losses to follow-up into account, probability-of-attrition weights were developed using all of the factors listed in Table 2 (21-23). This analysis developed a multivariate logistic regression model to determine the probability of follow-up and then weights subjects based on their likelihood of follow-up such that those who are less likely to reinterview are weighted more heavily. Probability-of-attrition weights have been shown to reduce bias from non-response in longitudinal cohort studies (21-23). However, the weighted analysis was not substantively different from the unweighted analysis, so only the unweighted analysis is reported here.
The COPD severity score is based on responses to survey items that comprise five domains of COPD severity:  degree of respiratory symptoms,  prior systemic corticosteroid use,  other COPD medication use,  previous hospitalization or intubation for respiratory disease, and  home oxygen use (Table 1) (12). The possible COPD severity score range is 0 to 35, with higher scores reflecting more severe disease. The scoring algorithm takes into account both current respiratory symptoms and the therapy necessary to achieve this symptom status. For example, a person with few respiratory symptoms who is receiving albuterol, ipratropium and salmeterol for COPD would have a higher COPD severity score than another person with comparable symptoms who was only taking albuterol and ipratropium. In developing the COPD severity score in prior published analyses, each survey item was assigned an a priori weight based on clinical aspects of the disease and its expected contribution to overall COPD severity. This weighting scheme was validated by showing that it correlated very closely with a score whose alternative weighting was derived from factor analysis with orthogonal rotation (12). Prior analyses have also demonstrated internal consistency reliability and concurrent validity (correlation with pulmonary function, physical HRQL, physical disability, and respiratory-specific HRQL) (12, 13, 16).
Structured telephone interviews obtained survey data using computer assisted telephone interviewing (CATI) software (17). Each of the four annual structured interviews was administered through the same professional survey firm using trained interviewers (Field Research Corporation, San Francisco, California). Interviewers obtained sociodemographic data including whether respondents considered their racial background to be Non-Hispanic White, African- American / Black, Asian / Pacific Islander, Latino / Hispanic, or other. As previously described, we defined educational attainment as high school or less, some college, or completed college or greater (17). Tobacco usage was measured using questions developed for the National Health Interview Survey (24). Inputs for calculating the COPD severity score were assessed with each of the four annual structured interviews.
For outpatient utilization, we ascertained the number of respiratory-specific visits to a medical office, health clinic, or hospital outpatient center in the previous 12 months. Number of visits was then dichotomized for analysis using two or more outpatient visits as the cut-off value (defined a priori based on the median value). Respiratory-specific ED visits and hospitalizations over the prior 12 months were dichotomized as either any events or no events.
Statistical analysis used Stata/SE software (version 9.2, College Station, TX). The predictive accuracy of logistic regression risk models was assessed by calculating a concordance index (c-index), which is the area under the receiver-operating characteristics (ROC) curve (25). A c-index of 1.0 reflect perfect prediction, with both test sensitivity and specificity being equal to 1. A c-index of 0.5 reflects discrimination no better than chance alone.
To test the extent to which the COPD severity score added predictive value to a risk model, we used a two step analytic process:  development of an initial logistic regression model (“Model 1”) to predict outcomes assessed in 2003 using clinical and sociodemographic factors assessed in our 2002 derivation cohort and then  addition of the COPD severity score to this initial prediction model to create “Model 2” which was then compared to “Model 1” to determine the incremental predictive value of the COPD severity score (i.e., the incremental c-index).
The c-index values for these risk models after addition of the COPD severity score were compared to the c-index of the models without the COPD severity score using a nonparametric approach described by Delong et al.(26). This method takes into account the correlated nature of the data (because the two different ROC curves are based on the same individuals), and the resulting test statistic uses a χ2 distribution to assess statistical likelihood.
Our goal in developing the initial prediction model was to maximize that model’s c-index. This provided the most stringent test of the additional predictive value of the COPD severity score; the higher the initial c-index, the more difficult it would be for an additional covariate (such as the COPD severity score) to result in a statistically significant increase in the c-index (25).
To develop the initial prediction model, we therefore determined the combination of covariates that maximized the c-index in a logistic regression analysis predicting the risk of respiratory-specific hospitalizations over a one-year period. To be consistent, we used this same combination of covariates in our initial prediction models for respiratory-specific ED visits and outpatient visits; we did not re-determine a different combination of covariates for each outcome. The covariates tested were age, gender, marital status, race, educational attainment, tobacco history, and self-reported medical comorbidities. We specifically tested comorbidities that are known to be associated with COPD: congestive heart failure, coronary artery disease (CAD), diabetes (through its effect on cardiovascular disease), and sleep apnea (27, 28).
Additionally, after completing this two-step analytic process, we performed an internal validation using another wave of follow-up data (2003 survey data as predictors of health-care utilization outcomes over the following year, as assessed in 2004) (29). The purpose of this internal validation was to determine whether the COPD severity score increased the c-index in more than one time period. For consistency, the initial prediction model (without COPD severity score) in this internal validation incorporated the same covariates as did the initial prediction model from the derivation cohort. For all logistic regression models, in both derivation cohort and internal validation, the Hosmer-Lemeshow test revealed adequate goodness-of-fit (p-values ranged from 0.13 to 0.96) (30).
We also evaluated the extent to which change in the COPD severity score was predictive of future health-care utilization. We used the t-test to determine whether the change in the score was statistically different than 0 (i.e., “no change”). We then constructed a model to predict health-care utilization that incorporated our sociodemographic, tobacco history and medical comorbidity covariates along with both  the absolute value of the COPD severity score and  the one-year patient-specific change in the score. We used a repeated measures analysis that incorporated both waves of follow-up data simultaneously. Specifically, factors from one wave were used as predictors of the subsequent wave outcomes (i.e. 2002 data was used to predict outcomes assessed in 2003, and 2003 data was used to predict outcomes assessed in 2004). We used generalized estimating equations with robust standard errors to take account of the fact that a given subject might contribute more than one observation to the analysis (31). All reported odds ratios for the COPD severity score were scaled by one-half of their standard deviation, which is generally considered to approximate a minimally clinically significant change (32).
Finally, we constructed risk nomograms in which we modeled the likelihood of health-care utilization over the possible range of COPD severity scores. In this analysis, the COPD severity score was modeled at the average value of the sociodemographic, tobacco history, and medical comorbidity covariates, thus controlling for these potential confounders (33).
The mean age of the COPD subjects was 65 ± 6 years. The majority of subjects were non-Hispanic white (85%). Most subjects (82%) were former or current cigarette smokers. The baseline mean COPD severity score was 7.8 (SD=6.3). The mean baseline COPD severity score of subjects identified in geographic “hot spots” for COPD mortality was 7.7 (SD=6.2) while the mean baseline score for subjects recruited from the national random sample was 8.0 (SD=6.5) (p=0.70). Other subject characteristics are presented in Table 2.
The covariates for the initial prediction model (without COPD severity score) were: age, race, education, tobacco history, and all of our tested self-reported medical comorbidities (Table 2). The c-index for predicting hospitalization demonstrated that the initial prediction model (without COPD severity score) had good discriminative ability: 80% in the derivation cohort and 79% when the model was tested for internal validation using the subsequent year data (model p<0.05 in both cases). The COPD severity score added substantive predictive value to the risk model for hospitalizations in both the derivation cohort and internal validation, raising the c-index to 87% and 91% respectively (p-values for the increase in c-index were 0.03 and 0.048 respectively).
For ED and outpatient visits, the initial prediction performed somewhat differently (Table 3). In all cases the c-index was less than that for hospitalization, and the initial prediction models were statistically significant only in the internal validation. However, the COPD severity score significantly improved the prediction models for ED and outpatient visits in both the derivation cohort and the internal validation.
Figure 1 demonstrates the change in the COPD severity scores over a two-year time frame. There was a mean two-year increase in score of 0.65 (SE=0.29, p=0.03), consistent with an overall worsening in COPD severity over time. When annual change in the score for any given subject was modeled along with the absolute baseline value of the score, both the baseline value and the change were statistically significant predictors of future respiratory hospitalizations, ED visits and outpatient visits (Table 4).
The predicted probability of an individual patient having a respiratory-related hospitalization, ED visit, or two or more outpatient visits for all potential values of the COPD severity score, corrected for other covariates, is shown in Figure 2. For example, the average value of the COPD severity score at baseline for our cohort was 7.8, which corresponds to a 5% risk of hospitalization, an 11% risk of ED visit, and a 49% chance of two or more outpatient visits for respiratory conditions in the following year. A score two standard deviations above the mean (20.4) corresponds to a 32% risk of hospitalization, a 48% risk of ED visit, and a 90% likelihood of two or more outpatient visits over the ensuing twelve months.
In a population-based cohort, the COPD severity score adds predictive value in estimating future respiratory-related hospitalizations, ED visits, and outpatient visits. Moreover, the COPD severity score changes over time, indicating that the score is not static, and these changes are themselves prospectively associated with health-care utilization, indicating that the score is responsive to change in a clinically relevant manner.
This COPD severity instrument may have several applications. It may be used as a tool to risk-adjust for disease severity within epidemiologic or outcomes research. The score may also be used to identify those at greater risk of hospitalizations or ED visits, possibly allowing resource-limited disease management programs to target patients at highest risk. Although the severity scoring instrument may be time-consuming to administer face-to-face in the context of a physician visit, the increasing availability of prior utilization data through electronic health records may assist in large-scale calculation of the score (34, 35). Alternatively, the score may potentially be derived from self-reported questionnaires. Importantly, the COPD severity score does not require measurement of pulmonary function, physical examination or other diagnostic testing, which may greatly facilitate its usage in population-based studies and in targeting high-resource utilizers. Finally, because the COPD severity score is responsive to change, it may be valuable, after further validation, as a measure of current clinical status or as an end-point of therapeutic interventions for COPD. We are currently planning further work on a different cohort of COPD subjects to establish the minimum clinically significant score change.
The risk nomogram assists with interpreting the COPD severity score’s association with utilization outcomes. Not surprisingly in a population-based cohort, hospitalizations and ED visits are a relatively uncommon event for subjects with average COPD severity. However, approximately 10% of the cohort had a COPD severity score greater than 15, which corresponds to a one-year hospitalization risk of about 17%. The threshold at which clinicians or disease management programs might target individuals would likely depend on resource constraints but ideally would target those above some threshold of risk (36). To place this in context, Kaiser Permanente’s extensive disease management program involved approximately 14% of their asthmatics over a 4-year time-frame (8). Thus it might be reasonable to choose some value of COPD severity score close to 15 to target a limited subset of the COPD population at significant risk of hospitalization.
Our approach to recruiting and following subjects with COPD resulted in both strengths and limitations. A major advantage of our population-based recruitment is the generalizability of findings to the population of COPD patients. This is particularly important if one is choosing a hospitalization-risk threshold for targeted intervention in a general population. One potential drawback of our approach is that we have relied on self-reported physician diagnosis, which may have resulted in some disease misclassification. However, our method for identifying subjects with COPD is a standard epidemiologic technique (37, 38). Also, as previously reported, the prevalence of COPD in the initial national sample (not including subjects surveyed in “hot-spots” for COPD) was 13.5%, which is similar to the 12.5% national COPD prevalence rate reported in the Third National Health and Nutrition Examination Survey (NHAINES III) (37, 38). Furthermore, the 21% prevalence rate in our cohort of self-reported CAD is close to the 19% prevalence rate of angina pectoris or prior myocardial infarction reported among older adults who attended the medical examination portion of NHANES III (39). These findings support the validity of our results.
Another limitation is our use of self-report to identify health-care utilization, which may have resulted in recall bias. The use of self-reported events is nonetheless supported by other studies, although there is a suggestion that older persons may underreport health-care utilization (40-42). Underreporting would likely cause underestimation of the effect of the COPD severity score on utilization. Loss to follow-up is an additional limitation. It is difficult to be sure of the effect of subject attrition. However, if subjects lost to follow-up had more severe COPD with higher health care utilization (e.g. at least 12 subjects did not reinterview because of death), then we would expect that subject attrition would tend to cause an underestimation of the incidence of hospitalization and possibly also the predictive ability of the COPD severity score. Regardless, subjects lost to follow-up did not appear statistically significantly different with respect to most sociodemographic factors and the COPD severity score. Furthermore, a probability-of-attrition weighted analysis did not change the results substantively, indicating that losses to follow-up did not affect our findings (data not shown). Nonetheless, the fact that African Americans were less likely to participate in the final interview may limit our ability to make strong conclusions about this racial-ethnic group.
Overall, the COPD severity score demonstrates prospective validity as a survey-based instrument for risk adjustment or the identification of high-risk cohorts. The severity score added predictive value to our estimate of future health-care utilization over and above a basic predictive model, and it did so in more than one time period. Moreover, the change in the severity score is itself associated with utilization outcomes, suggesting that the score is longitudinally responsive to changes in COPD status. Future work is needed to compare the score with other risk indices, determine a minimally clinically significant score change, evaluate the score’s predictive ability in other cohorts and for other important outcomes such as mortality, and to determine its ability to predict costs and thus its potential for use in risk-adjusting capitated payments. Nonetheless, this COPD severity score appears to have valuable potential as a disease-specific risk index for one of the leading causes of death and disability worldwide.
Financial support was provided by the National Institutes of Health grant R01 HL067438 from the National Heart, Lung, and Blood Institute. Dr. Omachi was supported by National Heart, Lung, and Blood Institute, grant number T32 HL007185. Dr. Eisner was supported by R01HL077618 National Heart, Lung, and Blood Institute, National Institutes of Health, with co-funding by the Social Security Administration