|Home | About | Journals | Submit | Contact Us | Français|
A simple method of identifying elders at high risk for Activity of Daily Living dependence (ADL) could facilitate essential research and implementation of cost-effective clinical care programs.
We used a nationally representative sample of 9,446 older adults free from ADL dependence in 2006 to develop simple models for predicting ADL dependence at 2008 follow-up and to compare the models to the most predictive published model. Candidate predictor variables were those of published models that could be obtained from interview or medical records data.
Variable selection was performed using logistic regression with backwards elimination in a two-thirds random sample (n=6,233) and validated in a one-third random sample (n=3,213). Model fit was determined using the c-statistic and evaluated vis-à-vis our replication of a published model.
At 2-year follow-up, 8.0% and 7.3% of initially independent persons were ADL dependent in the development and validation samples, respectively. The best fitting, simple model consisted of age and number of hospitalizations in past 2 years, plus diagnoses of diabetes, chronic lung disease, congestive heart failure, stroke, and arthritis. This model had a c-statistic of 0.74 in the validation sample. A model of just age and number of hospitalizations achieved a c-statistic of 0.71. These compared to a c-statistic of 0.79 for the published model. Sensitivity analyses demonstrated model robustness.
Models based on widely available data achieve very good validity for predicting ADL dependence. Future work will assess the validity of these models using medical records data.
One-fourth of the older adult population accounts for two-thirds of Medicare expenditures.(1) Persons with multiple chronic conditions (MCCs) who need basic activity of daily living (ADL) assistance are most likely to be high cost beneficiaries.(2) In 2007, the Agency for Healthcare Research and Quality (AHRQ) co-sponsored meetings that set research priorities for older adults with MCCs.(3) Underlying the research priorities were two related objectives—reducing Medicare expenditures and maintaining independence at home. Meeting either of the AHRQ objectives will require identifying individuals within the older adult population who are at elevated risk for dependency.
To be of greatest value, the risk identification model itself should be valid, low cost and have potential for wide application. Obtaining performance-based data on a population of older adults presents challenges and significant costs.(4) Models based on risk factors that can be obtained by interview or existing medical records data are likely to be more cost-effective. Existing theory and models indicate that demographic characteristics, prior health services use, and chronic disease diagnoses are all possible predictors of ADL dependency(1, 2) and these are variables that could be obtained by interview and often from medical records data.
For this report we replicated the most valid published model(5) which we refer to as the benchmark model, developed somewhat simpler models, and compared the predictive values of the benchmark and simple models. Analyses were based on a nationally representative sample of adults aged 65 or over who reported no ADL dependency in 2006. The outcome was ADL dependency status at the 2008 follow-up.
Data came from the Health and Retirement Study (HRS). The HRS is a panel study with a multistage area probability sample.(6) The HRS sample is nationally representative of community dwelling older adults. There were 9,966 self-respondents who were aged 65 years or over at the 2006 interview and independent in all ADLs (eating, dressing, bathing, transferring, toileting). For 458 of 574 respondents who died prior to the 2008 follow-up interview 2008 ADL dependency status was available from proxy “exit” interview data. We excluded 520 respondents who had missing data on one or more variables of interest leaving 9,446 for our analyses.
The study outcome is ADL dependency. Dependence is defined as receiving help from another person to complete the activity.(7) In the HRS, persons or their proxy were asked whether, because of a health or memory problem, they received assistance in ADLs. Consistent with recent literature,(5) we included five basic ADLs; bathing, dressing, eating, toileting, and transferring from a bed to a chair. We defined ADL dependency as a report of receiving personal help (use of assistive devices is not considered dependence) of any type in any one of the five ADLs.
Although the benchmark model included physical and cognitive impairments and ADL disabilities we elected not to include these as candidate variables for the simple model because they are rarely available in electronic medical records data or medical charts. Included as a candidate variable in the simple models but not in the benchmark model is prior hospitalization. In addition to often indicating an acute deterioration in health,(8) there is evidence that the hospitalization itself can accelerate ADL dependency and is thus an independent risk factor for ADL dependency.(9)
Age, gender, number of hospitalizations, total number of nights in hospital since last interview (2004), and chronic disease were candidate variables for the simple models. Chronic disease was based on a report that a doctor had told the respondent that he/she has the condition. The eight candidate conditions are listed at the bottom of Table 2. We included individual chronic disease rather than a count of diseases because a recent analysis of trajectories of ADL dependencies showed that mean number of chronic diseases does not differentiate trajectories.(10)
For replication of the benchmark model, we included the nine ADL dependency predictor variables reported in Covinsky et al., 2006. The nine variables are shown in Table 2. On all variables, the HRS 2006-2008 wording and response options are the same as those of the 1993-1995 Assets and Health Dynamics of the Oldest Old survey used by Covinsky et al., 2006.
Replication of the benchmark model and development of the simple models were completed on a random two-thirds (n=6,233) of the sample and validation of the models was carried out using the remaining one-third (n=3,213). To replicate the benchmark model, we used a logistic regression model. Development of the simple model was performed using logistic regression with backwards elimination. The model started with age, gender, number of hospitalizations, total number of nights in hospital, and the eight available chronic diseases as possible predictor variables and eliminated variables one at a time to improve fit. We also explored an even simpler model that did not include chronic disease diagnoses. Comparison of the models was accomplished using c-statistics.(11) As a general rule of thumb, a c-statistic of 0.7 to 0.79 is considered very good and 0.8 or greater is considered excellent.(12) To evaluate agreement (i.e., models agree that a subject is high or low risk) across benchmark and simple models in classifying individuals as high risk we used the Kappa method. (13)
We conducted several analyses to determine the robustness of the models. In these analyses, we assessed: 1) impact of attrition from death by estimating a model with a combined outcome of incident ADL or death using the development cohort, 2) alternative models using the development cohort that examined age alone, age and chronic diseases, and the benchmark model with hospitalizations included, and 3) model fit excluding persons 90 years or older.
The development and validation cohorts were similar on all variables (see Table 1). Table 2 shows results of ADL prediction models in the development cohort. At 2-year follow-up, 498 (8.0%) had developed ADL dependence. Replication of the benchmark model shows a c-statistic of 0.79, which is the same as that reported by Covinsky et al., 2006 using data from 1993-95. Results of the backwards elimination analyses for the simple model indicate that age, hospitalizations, and five chronic disease diagnoses give the best fitting model. The c-statistic in that model (using the model’s regression coefficients) is 0.76. Eliminating disease indicators from the simple model leaves just two variables; age and prior hospitalizations. The c-statistic drops to 0.73 in that model.
Two hundred thirty six (7.3%) in the 1/3rd validation sample developed ADL dependence. As shown in Table 3, the c-statistic for the benchmark model is the same as in the development samples (0.79). Compared to the development sample, the c-statistics for the simple models are slightly lower in the validation sample—0.74 and 0.71 for the models with and without chronic disease diagnoses, respectively. The parameter estimates for variables in the simple models are similar across the development and validation samples with the exception of age 90 years or over.
Based on rounded odds ratios from the development cohort (Table 2), we assigned points (0 for reference group, 1 for odds ratio of >1 and <1.5, 2 for odds ratio of ≥ 1.5 and < 2.5, etc) for model variables. Scoring is shown in Table 4. Compared to the models based on actual regression coefficients, this simple scoring strategy did not result in any change in the c-statistic of the benchmark model. There is a modest decline for the simple models. In Table 5 we show the distribution of incident ADL dependency in the validation cohort for each of the models using the variable scoring that is shown in Table 4. In both the benchmark model and the simple model including disease, 28.7% and 29%, respectively, of respondents with a score of 8 or more were ADL dependent at follow-up. Eight or more was identified by Covinsky et al. for the benchmark model as an optimal cut-point for clinical decision-making. With cut-off scores of 8 or more, the Kappa statistic showed 0.39 agreement between the benchmark and simple model with disease (not shown). Agreement between the benchmark and the simple model without disease was 0.27 while the Kappa between the two simple models was 0.68 (not shown).
The c-statistics for the combined death and incident ADL outcome models were 0.75 and 0.72 for simple models with and without chronic disease, respectively; very close to those reported in Table 2. Analyses of age alone showed a c-statistic of 0.70 for predicting ADL dependency. Adding chronic disease to age improved the model fit to 0.75. Including hospitalizations in the benchmark model increased the c-statistic for the benchmark model by 0.01 (0.79 up to 0.80). Finally, re-estimating the simple models excluding persons over 90 years of age decreased the c-statistics to 0.74 (with disease) and 0.70 (without disease).
We sought to identify simple ADL dependency prediction models. Although the predictive validity of the simple models is lower than that of the benchmark model, the results do indicate a very good fit for the simple models. A scoring system developed from the odds ratios of the simple models retained a very good fit to the data. About 30 percent of persons scoring eight or more on the simple model with disease or on the model without disease went on to develop ADL dependency; whereas 10%-14% of those scoring six or less did so. Twenty-nine percent of those scoring eight or more on the benchmark model experienced incident ADL. Although the percent identified as high risk at a cutoff of eight is similar in the benchmark and simple model with disease, the Kappa statistic showed only fair agreement(13) between these models. Future work might explore alternative cut points, particularly identification of optimal cut points for subgroups.
Replication of the benchmark model in the 2006-2008 data shows clinical utility that is very good and very similar to that found in its original development based on data from 1993-1995. The benchmark model has a c-statistic that is better than the best simple model (0.79 vs. 0.74, respectively) but it requires multiple measures of functional status that are not available in common medical records.
Although all of the models shown in this report were developed using survey data, having eliminated certain variables from the benchmark interview model, we have created simple models for which scores could be created from medical records data. Although standardization of electronic medical records (EMR) has not yet been achieved,(14) EMRs are expanding and will be increasingly used for health care decisions.(15) In fact, the Centers for Medicare and Medicaid have distributed $73 billion dollars in incentive payments to be made from 2011 to 2015 for organizations that meet EMR usage criteria.(16) EMR data are not generally based on self-report so replication using actual EMR data on clinical populations need to be done and may produce different results.
For an individual clinician, the value of these brief instruments may be in raising awareness of the relative importance of certain states and conditions in risk for dependency (compared to other potential risk factors). However, for health care systems, administrators, policy makers, and researchers, these brief models are likely efficient, valuable screening tools to identify a high risk subset of older adults within a population. For example, applying a score of four or more as a positive screen to the simple model that includes chronic disease would identify 36% of the population as elevated risk and capture 67% of the incident ADL cases (calculated from data shown in Table 5). Within the 36% elevated risk subgroup, additional evaluation via interview,(5) brief assessment,(17) or comprehensive geriatric assessment(18) might be used to tailor interventions to high risk individuals. Comprehensive geriatric assessments of those at elevated risk may be the ideal approach given evidence that collaborative care interventions can be cost saving in high risk elders.(19)
In discussion above, we have used the cutoff examples of both 4 and 8; someone aged 90 or over would have a minimum score of 15 or above in the two simple models. Thus, it may be most efficient to assume that persons 90 years of age or over are a high risk group. Other indicators in the scoring system are helpful in older adults less than 90 years of age and our sensitivity analyses provided supporting evidence for the validity of the model in the 65 to 90 year age group.
Although self-report hospitalizations have been shown to correlate well with actual hospitalization records,(20, 21) there are other limitations to the measure of hospitalizations available in the HRS. It is an all-cause indicator and the effect of hospitalization on risk for dependency may vary by type of hospitalization. Some hospitalizations are for procedures that can improve ADL capacity (e.g., joint replacements), while others are for acute deteriorations in health. Hospitalization rates also vary by region, urban versus rural communities, and community socioeconomic level.(22) Thus, the role of hospitalization in incident ADL may differ by region or community due to differing admission criteria.
The baseline data we have used to inform the predictor variables are based on self-report data. Validation of self-reported chronic disease diagnoses against medical records data has shown good to excellent agreement (23-25) but confirming this in the context of these models would be important. Also, we were only able to evaluate chronic disease diagnoses contained in the HRS and this did not include chronic renal failure. It is unclear whether this would have a significant effect on results. Gill et al, 2010 showed that dependency trajectories are highly heterogeneous for persons with organ failure.(10) ADL dependency at two year follow-up was based on a proxy report for 7.5% of the sample. Proxy respondents have been found to overestimate the amount of hours given to ADL assistance but overestimates of need for ADL assistance are less apparent.(26) The intraclass correlation between older adults and their proxies on reports of need for assistance in seven instrumental ADLs was found to be 0.85 in a study of hip fracture patients.(27)
For some of the reasons noted above, validation in clinical populations using clinical data is needed but these analyses suggest that a simple approach to identifying elders at elevated risk for ADL dependency may be possible, particularly as EMRs become widely available. Identifying such high risk elders is a necessary step in implementing models of care that have been shown to improve outcomes and reduce costs of care for vulnerable older adults.(18) Thus, in the context of further assessment and interventions, ADL risk assessment tools could contribute to the goals of reducing Medicare expenditures and maintaining independence at home.
The authors wish to thank Steven R. Counsell for comments on an earlier draft. The authors take sole responsibility for the content of this final manuscript and have no conflicts of interest to report. All authors had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. The project described was supported by Award Numbers R01 AG031222 and P30AG024967 from the National Institute on Aging to the IU Roybal Center. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institute on Aging or the National Institutes of Health.
Supported by National Institute on Aging grants P30 AG024967 and R01 AG031222.
This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.