|Home | About | Journals | Submit | Contact Us | Français|
Readmissions to hospital are common, costly and often preventable. An easy-to-use index to quantify the risk of readmission or death after discharge from hospital would help clinicians identify patients who might benefit from more intensive post-discharge care. We sought to derive and validate an index to predict the risk of death or unplanned readmission within 30 days after discharge from hospital to the community.
In a prospective cohort study, 48 patient-level and admission-level variables were collected for 4812 medical and surgical patients who were discharged to the community from 11 hospitals in Ontario. We used a split-sample design to derive and validate an index to predict the risk of death or nonelective readmission within 30 days after discharge. This index was externally validated using administrative data in a random selection of 1 000 000 Ontarians discharged from hospital between 2004 and 2008.
Of the 4812 participating patients, 385 (8.0%) died or were readmitted on an unplanned basis within 30 days after discharge. Variables independently associated with this outcome (from which we derived the nmemonic “LACE”) included length of stay (“L”); acuity of the admission (“A”); comorbidity of the patient (measured with the Charlson comorbidity index score) (“C”); and emergency department use (measured as the number of visits in the six months before admission) (“E”). Scores using the LACE index ranged from 0 (2.0% expected risk of death or urgent readmission within 30 days) to 19 (43.7% expected risk). The LACE index was discriminative (C statistic 0.684) and very accurate (Hosmer–Lemeshow goodness-of-fit statistic 14.1, p = 0.59) at predicting outcome risk.
The LACE index can be used to quantify risk of death or unplanned readmission within 30 days after discharge from hospital. This index can be used with both primary and administrative data. Further research is required to determine whether such quantification changes patient care or outcomes.
Readmission to hospital and death are adverse patient outcomes that are serious, common and costly.1,2 Several studies suggest that focused care after discharge can improve post-discharge outcomes.3–7 Being able to accurately predict the risk of poor outcomes after hospital discharge would allow health care workers to focus post-discharge interventions on patients who are at highest risk of poor post-discharge outcomes. Further, policy-makers have expressed interest in either penalizing hospitals with relatively high rates of readmission or rewarding hospitals with relatively low expected rates.8 To implement this approach, a validated method of standardizing readmission rates is needed.9
Two validated models for predicting risk of readmission after hospital discharge have been published.10,11 However, these models are impractical to clinicians. Both require area-level information (e.g., neighbourhood socio-economic status and community-specific rates of admission) that is not readily available. Getting this information requires access to detailed tables, thereby making the model impractical. Second, both models are so complex that risk estimates cannot be attained from them without the aid of special software. Although these models have been used by health-system planners in the United Kingdom, we are unaware of any clinicians who use them when preparing patients for hospital discharge.
Our primary objective was to derive and validate a clinically useful index to quantify the risk of early death or unplanned readmission among patients discharged from hospital to the community.
We performed a secondary analysis of a multicentre prospective cohort study conducted between October 2002 and July 2006. The study involved patients discharged to the community from the medical or surgical services of 11 hospitals (6 university-affiliated, 5 community) in five cities in Ontario after an elective or emergent hospital admission. To be eligible for inclusion, patients had to be adults, provide informed consent, have a telephone (to participate in follow-up telephone interviews), and be cognitively intact (to ensure validity of the consent process and accuracy of information given in the interviews). We recruited patients of medical and surgical services because such patients comprise most discharges from hospitals.
Given that the process of health care provided in nursing homes differs from processes of care for patients in the community (i.e., nursing home residents frequently receive care from an onsite physician), we restricted eligibility to patients who were not residents of nursing homes.
Before discharge from hospital, patients were interviewed by study personnel to identify their baseline functional status, living conditions and chronic medical conditions. Chronic medical conditions were confirmed by a review of the patient’s chart and hospital discharge summary, when available. The chart and discharge summary were also used to identify diagnoses made while in hospital and medications given at discharge. All medications given at discharge were compared with those documented on the admission note to determine which discharge medications had been started in hospital.
To determine whether patients had had an unplanned readmission to hospital or had died within 30 days of discharge, we contacted either patients or their principal contacts (identified by each patient at recruitment) one month after discharge. We combined unplanned readmissions with deaths to avoid bias caused by censoring deaths when hospital readmission alone is examined.12 We classified readmissions as unplanned if they had not been arranged or planned when the patient was originally discharged from hospital.
We chose a 30-day time frame for our primary outcome to increase the likelihood that poor outcomes would be related to the index admission or discharge process and would be more likely to be remediable. When analyzing this outcome by its components, we classified patients who were urgently readmitted within 30 days as having died if they subsequently died within 30 days of discharge from hospital.
Our study was approved by the Ottawa Hospital Research Ethics Board and by each participating site.
We randomly selected half of the participants for index derivation and used the other half for internal validation. We identified all patient-level and admission-level variables among the data set that we thought might influence outcomes. We used multivariable logistic regression to measure the independent association of these factors with early death or unplanned readmission to hospital. Patient-level variables and admission-level variables entered into the model are listed in Table 1. We also offered the presence or absence of most common diagnoses and procedures (Appendix 1, available at www.cmaj.ca/cgi/content/full/cmaj.091117/DC1).
We used fractional polynomial functions to determine the best linear or nonlinear form for continuous variables.15–17 Backward stepping with an α-error criterion of 0.01 was used to include only significant variables in the final multivariable logistic model. We used an inclusion criterion of 1% to ensure model stability. To account for clustering of patients within hospitals, we used generalized estimating equation methods. We found no changes in the parameter estimates for all model variables, and all variables remained significant at the 5% level.
We then used the methods described by Sullivan and colleagues18 to modify the final logistic model into a risk index. The number of points assigned to each significant covariate equaled its regression coefficient divided by the parameter estimate in the model with the smallest absolute value rounded to the nearest whole number. We then calculated each participant’s final score by summing up his or her points. The expected probability of early death or unplanned readmission associated with each score was the inverse of 1 + e–(intercept+b*total score), where b was the value of the coefficient in the regression model with the smallest absolute value.
Since the components of the final model were available from administrative data, the index was externally validated using three population-based administrative databases that capture data on all Ontarians. The Discharge Abstract Database records all hospital admissions. The National Ambulatory Care Reporting System records all emergency department visits, and the Registered Patient Database records all dates of death.
We used the Discharge Abstract Database to randomly select 1 000 000 (of 1 916 398) adult medical or surgical patients out of discharged to the community from Ontario hospitals between April 2004 and January 2008. This period was used to ensure that six months of preadmission data existed in the National Ambulatory Care Reporting System for all participants. The urgency and length of stay of each hospital admission were noted.
We calculated each patient’s Charlson comorbidity index score using the International Classification of Disease (ICD) codes cited by Quan and colleagues.19 We linked to the National Ambulatory Care Reporting System to measure the number of visits to an emergency department by each patient in the six months before admission. We determined patients’ status at 30 days post-discharge by linking to the Registered Patient Database for data related to deaths and to the Discharge Abstract Database for data related to unplanned urgent readmissions.
We used a C statistic with 95% confidence intervals (CIs)20 to measure the ability of the index to discriminate between patients who died or had an unplanned readmission within 30 days of discharge and those who did not. The C statistic expresses the proportion of times that the case in each case–noncase pair has a higher model-based predicted risk of the outcome. We measured the calibration of the score by comparing the observed and expected numbers of patients with the outcome for each score. We deemed the observed and expected death or urgent readmission rates to be similar if the 95% CI around the observed rate included the expected rate. We calculated 95% CIs for observed rate of death or urgent readmission rates using exact methods.21 We summarized overall calibration using a Hosmer–Lemeshow goodness-of-fit test.22
Between October 2002 and July 2006, we enrolled 5035 patients from 11 hospitals. We determined outcome status at 30 days for 4812 patients (95.6%). Of the remaining patients, 124 (2.5%) refused participation when contacted for follow-up, 83 (1.6%) were lost to follow-up, and 16 (0.3%) were removed from the study because they were admitted to a nursing home during the first month after discharge from hospital.
The study cohort is described in Table 1. Participants were middle-aged, and almost 95% were independent with regard to activities of daily living. Most participants were free of serious comorbidities, with more than 75% having a Charlson comorbidity index score of zero.13 Most admissions were emergent (58.1%), and almost half (44.9%) were to a medical service. The most common reasons for hospital admission included acute coronary syndromes, cancer diagnosis and complications, and heart failure (Appendix 1). Coronary artery bypass grafting and arthroplasty were the most common procedures. Patients in the derivation (n = 2393) and validation (n = 2419) cohorts were similar.
During the first 30 days after discharge, 385 (8.0%) patients died or were urgently readmitted (death 36 [9.4% of outcomes], unplanned readmission 349 [90.6% of outcomes]). Patients with one of the primary outcomes had more emergency department visits before admission and were more likely to be admitted emergently and for longer durations than patients who did not die (Table 1). Most other patient-related and admission-related variables appeared to have little influence on risk of early death or unplanned readmission.
Only four variables were independently associated with death or readmission within 30 days after discharge (Table 2). These variables were length of stay (“L;” odds ratio [OR] 1.47, 95% CI 1.25–1.73) acuity of the admission (“A;” odds ratio [OR] 1.84, 95% CI 1.29–2.63), patient comorbidity (as measured using the total Charlson comorbidity index score) (“C;” odds ratio [OR] 1.21, 95% CI 1.10–1.33), and emergency department use (measured as the number of visits in the previous six months) (“E;” odds ratio [OR] 1.56, 95% CI 1.27–1.92). Length of stay was modelled as a logarithm and the number of emergency department visits was modelled as a square root term. We found no significant interactions between these or other variables. The final logistic model was moderately discriminative (C statistic 0.700) and was well calibrated (Hosmer–Lemeshow goodness of fit statistic 6.99, 8 degrees of freedom, p = 0.54). None of the other variables listed in Table 1 met our criteria for inclusion in the model.
We modified this logistic model into an index to predict early death or unplanned readmission (Table 3). To facilitate recall of the components of the index, we titled the index using a simple mnemonic. The LACE index had a potential score ranging from 0 to 19. The total LACE score in the study population had a normal distribution that was slightly skewed to the right (Figure 1).
The LACE index had moderate discrimination for early death or readmission. The C statistic (95% CI) in the derivation was 0.7114 (0.6736–0.7491). In the validation, it was 0.6935 (0.6548–07321), and in the entire cohort, it was 0.7025 (0.6755–0.7295).
The expected probability of death or readmission within 30 days of discharge for each point ranged from 2.0% for a LACE score of 0 to 43.7% for a LACE score of 19 (Table 4). The expected probability of early death or unplanned readmission was within the 95% CIs of the observed rates for all LACE scores in both the derivation and validation cohorts (Table 4) as well as the entire cohort (Figure 1). The Hosmer–Lemeshow statistic in the derivation was 18.7 (p = 0.42). In the validation, it was 14.1 (p = 0.59), and in the entire cohort, it was 21.2 (p = 0.27) (Table 4).
The LACE score was strongly associated with each outcome individually. A 1-point increase in the LACE score increased the odds of unplanned readmission by 18% (odds ratio 1.18, 95% CI 1.14–1.21). The LACE index in the entire cohort was moderately discriminative for 30-day unplanned readmission (C statistic 0.679, 95% CI 0.650–0.708) and well calibrated (Hosmer–Lemeshow statistic 11.5, p = 0.18). A one-point increase in the LACE score increased the odds of early death by 29% (odds ratio 1.29, 95% CI 1.20–1.38). The LACE index was very discriminative for early death (C statistic 0.793, 95% CI 0.733–0.854) and well calibrated (Hosmer–Lemeshow statistic 4.7, p = 0.79).
The external validation group contained 1 000 000 randomly selected patients (mean age 59.1, standard deviation [SD] 18.4 years; 48.4% female). Patients had a mean length of stay of 5.1 days (SD 7.7), a mean Charlson comorbidity index score of 0.5 (SD 1.2), and a mean number of emergency department visits of 0.4 (SD 7.9), with 67.6% of the index admissions emergent. Patients had a mean LACE score of 6.0 (SD 3.1) and 7.8% of patients died (1.1%) or were urgently readmitted (7.3%) within 30 days of discharge. Discrimination of the LACE index was the same in this patient group (C statistic 0.684, 95% CI 0.679–0.691). The observed rate of early death or urgent readmission slightly exceeded the expected rates at most LACE scores (Figure 2). However, the median absolute difference between expected and observed rates was small, at 1.6% (range 0.04%–6.6%).
We have derived and validated an easy-to-use index that is moderately discriminative and very accurate for predicting the risk of early death or unplanned readmission after discharge from hospital to the community. Further research is required to determine whether such quantification changes patient care or outcomes.
We found its simplicity very notable. Although we derived the LACE index in a large cohort of patients using almost 50 factors — each of which could reasonably influence the risk of post-discharge outcomes — we found that four simple factors explained much of the variation in risk of early death or unplanned readmission after discharge from hospital. The LACE index therefore joins other indexes in which seemingly complex outcomes are predicted with a few simple factors.23
The LACE index has several strengths to support its use.24 The outcome predicted by the index is important, clinically relevant and reliably measured. Determination of this outcome for each patient was independent of the LACE score. Each component of the LACE index is readily and reliably determined. The methods we used to derive the LACE index were both valid and transparent. The discrimination of the LACE index was better than that of the widely used Framing-ham score in many populations,25–27 which suggests that the LACE index will be useful when applied at the individual patient level. The calibration of the LACE index was excellent, which suggests that it will also be useful when applied by policy-makers. Finally, the LACE index is easier to use than previous models, because the latter require variables —such as community admission rates28 or area-level socio-economic measures29 — that are usually unavailable to clinicians.
Three main limitations about the LACE index should be noted. First, the index cannot be used reliably in patient populations that were not involved in its derivation. Second, further work is required to identify additional factors that may increase the discrimination or accuracy of the index. Third, clinicians will find it difficult to commit to memory the point system and its expected risks. Therefore, use of the LACE index will usually require a computational aid. Until the LACE index is externally validated with primary data, we recommend that it be used for outcomes research and quality assurance rather than in decision-making for individual patients.
Notwithstanding its limitations, we believe that the LACE index can be used by researchers and administrators to predict the risk of early death or unplanned readmission of cognitively intact medical or surgical patients after discharge from hospital to the community. Further research is required to determine whether quantifying the risk of poor outcomes after discharge actually changes patient care or outcomes.
Funding: This study was funded by the Canadian Institutes of Health Research, the Physicians’ Services Incorporated Foundation and the Department of Medicine, University of Ottawa.
Previously published at www.cmaj.ca
Competing interests: None declared.
Contributors: All of the authors were involved in the conception and design of the study, the acquisition of data, the analysis or interpretation of data, the drafting of the manuscript and the critical revision of the manuscript for important intellectual content. All of them approved the final version submitted for publication. Dr. van Walraven had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
This article has been peer reviewed.