PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptNIH Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
J Pain Symptom Manage. Author manuscript; available in PMC Mar 1, 2013.
Published in final edited form as:
PMCID: PMC3289041
NIHMSID: NIHMS321419
Development of a Prognostic Model for Six-Month Mortality in Older Adults with Declining Health
Paul K.J. Han, MD, MA, MPH, Minjung Lee, PhD, Bryce B. Reeve, PhD, Angela B. Mariotto, PhD, Zhuoqiao Wang, MS, Ron D. Hays, PhD, K. Robin Yabroff, PhD, Marie Topor, MS, and Eric J. Feuer, PhD
Center for Outcomes Research and Evaluation (P.K.J.H.), Maine Medical Center, Portland, Maine; Division of Cancer Control and Population Sciences (M.L., B.B.R., A.B.M., K.R.Y., E.J.F.), National Cancer Institute, Bethesda, Maryland; Information Management Services, Inc. (Z.W., M.T.), Silver Spring, Maryland; and Division of General Internal Medicine (R.D.H.), University of California, Los Angeles, Los Angeles, California, USA
Adddress correspondence to: Paul K. J. Han, MD, MA, MPH, Center for Outcomes Research and Evaluation, Maine Medical Center, 39 Forest Avenue, Portland, ME 04105, USA, hanp/at/mmc.org
Context
Estimation of six-month prognosis is essential in hospice referral decisions, but accurate, evidence-based tools to assist in this task are lacking.
Objectives
To develop a new prognostic model, the Patient-Reported Outcome Mortality Prediction Tool (PROMPT), for six-month mortality in community-dwelling elderly patients.
Methods
We used data from the Medicare Health Outcomes Survey (MHOS) linked to vital status information. Respondents were 65 years old or older, with self-reported declining health over the past year (n=21,870), identified from four MHOS cohorts (1998–2000, 1999–2001, 2000–2002, and 2001–2003). A logistic regression model was derived to predict six-month mortality, using sociodemographic characteristics, comorbidities, and health-related quality of life (HRQOL), ascertained by measures of activities of daily living (ADLs) and the Medical Outcomes Study Short Form-36 Health Survey (SF-36®); k-fold cross-validation was used to evaluate model performance, which was compared to existing prognostic tools.
Results
The PROMPT incorporated 11 variables including four HRQOL domains: general health perceptions, ADLs, social functioning, and energy/fatigue. The model demonstrated good discrimination (c-statistic=0.75) and calibration. Overall diagnostic accuracy was superior to existing tools. At cutpoints of 10%–70%, estimated six-month mortality risk sensitivity and specificity ranged from 0.8%–83.4% and 51.1%–99.9%, respectively, and positive likelihood ratios at all mortality risk cut-points ≥40% exceeded 5.0. Corresponding positive and negative predictive values were 23.1%–64.1% and 85.3%–94.5%. Over 50% of patients with estimated six-month mortality risk ≥30% died within 12 months.
Conclusion
The PROMPT, a new prognostic model incorporating HRQOL, demonstrates promising performance and potential value for hospice referral decisions. More work is needed to evaluate the model.
Keywords: Prognosis, clinical prediction model, elderly, health-related quality of life
Prognostic estimates are important in numerous medical decisions, but perhaps nowhere do they play a more critical role than in the decision to initiate hospice care. This single decision formalizes the beginning of the end-of-life period and the transition from curative to palliative goals of care for many patients, and thus implicitly embodies some estimate of prognosis. In the U.S., the decision to initiate hospice care explicitly depends on prognostic estimates, since physicians must certify an expected survival of six months or less before patients can receive services under the Medicare Hospice Benefit.1, 2
It follows that difficulties in accurately and prospectively estimating six-month mortality may be an important determinant of the underutilization of hospice services—and the proportionate overutilization of aggressive curative interventions—known to characterize end-of-life care in the U.S.315 Physicians’ prognostic estimates are known to be generally inaccurate and optimistically biased.14, 16, 17 The extent and systematic nature of this inaccuracy and its correspondence with existing patterns of end-of-life care suggest that prognostic uncertainty—particularly with respect to six-month mortality—may be a key determinant of hospice underutilization.
Yet, few accurate and evidence-based prognostic tools exist to help clinicians estimate six-month mortality. Consensus guidelines were developed in 1996 by the National Hospice Organization (NHO) and adopted widely by clinicians and health policymakers.18 However, these guidelines were not evidence-based and have been shown to perform poorly in predicting six-month mortality.19, 20 Empirically-derived prognostic models also have shown limited accuracy. Perhaps the best-known, most rigorously-evaluated model, from the Study to Understand Prognoses and Preferences for Outcomes and Risks of Treatments (SUPPORT), used disease characteristics and physiologic variables to predict six-month mortality in critically-ill patients surviving hospitalization for any of nine serious illnesses.21 However, in a subsequent validation effort in patients with advanced lung, heart, and liver disease and a 25% six-month mortality, this model also demonstrated poor performance.19
More recent modeling efforts are promising but have been limited to specific diseases such as cancer and dementia,2226 or temporal endpoints other than six months. For example, models have been developed to predict short-term (less than six months) mortality27, 28 in terminally-ill patients already referred for hospice or palliative care, and longer-term (one year or more) mortality in hospitalized29 or community-dwelling elders.30, 31 These models are thus less useful for hospice referral decisions, although they demonstrate improved predictive performance, possibly because of their inclusion of patient-reported outcomes (PROs) such as self-reported functioning and well-being, or health-related quality of life (HRQOL). Accumulating evidence suggests that PROs assume greater prognostic power than other variables as the end of life approaches, presumably because the dying process represents a final common pathway characterized by a relatively small set of symptoms and functional impairments.14, 3247 PROs are also attractive as prognostic variables because they are feasibly ascertainable directly from patients.
To our knowledge, however, there have been no previous attempts to integrate HRQOL into prognostic models for six-month mortality in general patient populations. Our objective in the current study was to develop a broadly-applicable prognostic model incorporating HRQOL to predict six-month mortality, the Patient-Reported Outcome Mortality Prediction Tool (PROMPT), and to explore whether such a model could have sufficient accuracy to inform hospice referral decisions.
Data Source and Sample Population
This study used data from the Medicare Health Outcomes Survey (MHOS), an annual nationwide survey of Medicare managed care beneficiaries administered by the Centers for Medicare and Medicaid Services (CMS) since 1998 (www.hosonline.org).4850 The MHOS surveys a random sample of 1000 Medicare beneficiaries from each managed care plan under contract with CMS (between 250–320 participating plans yearly). Participants complete a baseline and a two-year follow-up survey if enrolled in the same plan. Institutionalized and disabled beneficiaries are included, but patients on Medicare solely because of end-stage renal disease are excluded. The MHOS utilizes a self-report questionnaire to collect data on patient sociodemographic characteristics, comorbidities, clinical symptoms, and HRQOL, as measured by activities of daily living (ADLs) and the Medical Outcomes Study Short Form-36 Health Survey (SF-36®, version 1).51 The MHOS is administered by mail, with telephone follow-up and administration to initial nonresponders.
We used data from four MHOS cohorts: 1998–2000, 1999–2001, 2000–2002, and 2001–2003. During this time, baseline and follow-up surveys were distributed to a total of 986,530 patients, of whom 912,703 were eligible for participation (more than 65 years old, managed care enrollees, not deceased). There were 634,892 total respondents (response rate 70%); we analyzed the last survey completed by any respondent (Fig. 1). We further limited the sample to patients for whom use of a prognostic tool for hospice decision making would be most clinically appropriate and useful, using the SF-36 health transition item to select patients reporting significantly declining health: “Compared to one year ago, how would you rate your health in general now?” This item is not scored in any SF-36 scale, and its use as an inclusion criterion has conceptual validity since physicians should be more apt to consider hospice referrals for patients with substantially declining health. We included only respondents who reported that their health was “much worse” (n=21,870); there were 3,295 deaths in this group, yielding a substantially higher observed six-month mortality than in the overall MHOS sample (15% vs. 2%).
Figure 1
Figure 1
Derivation of the study sample of adults ≥ age 65 with declining health, Medicare Health Outcomes Survey (MHOS)
Measures
Sociodemographic characteristics included in our analysis were self-reported age, sex, race/ethnicity, education, and current marital status.
Comorbidities included several self-reported diseases: hypertension, coronary artery disease, congestive heart failure, other heart conditions, stroke, chronic obstructive pulmonary disease, diabetes, and cancer. Arthritis, inflammatory bowel disease, and sciatica were ascertained but excluded from analyses because they are not leading causes of mortality in U.S. adults aged 65 years and older.52 Smoking status (current, former, never) also was ascertained.
HRQOL was measured in two ways. Functional status was assessed using six ADLs: bathing, dressing, eating, getting in or out of chairs, walking, and using the toilet. These items had three response options: “No, I do not have difficulty”/“Yes, I have difficulty”/“I am unable to do this activity.” We created a summary measure of the total number of ADLs for which respondents indicated “unable to do this activity;” scores ranged from 0–6, with higher scores indicating greater functional impairment. HRQOL also was ascertained using the SF-36, version 1,51 a widely-used, comprehensive, generic health status instrument comprising eight scales: physical functioning (10 items), role limitations because of physical health problems (four items), bodily pain (two items), general health perceptions (five items), energy/fatigue (four items), social functioning (two items), role limitations because of emotional problems (three items), and emotional well-being (five items). Response options range from two to six ordinal categories. SF-36 scale scores were normalized to the general U.S. population on a T-score metric (mean=50, standard deviation=10), with higher scores indicating better HRQOL.
The outcome variable was survival at six months since the last completed survey for each respondent. Vital status and date of death were obtained from the CMS Medicare Enrollment Database. Survey completion by proxy also was ascertained.
Model Development
The large number of predictor variables resulted in a substantial proportion of subjects with missing data (n=6,154, representing 28% of the sample). The proxy survey completion variable had a disproportionately high frequency of missing data (11%), and to avoid dropping cases, we created a dummy variable for non-response to this item. Missing data for all other individual variables were less than 8%, and handled through multiple imputation using the PROC MI and PROC MI-ANALYZE functions of SAS software, version 9.1.3 (SAS Institute Inc., Cary, NC). Missing data were imputed using a Markov chain Monte Carlo method with multiple chains, creating 10 imputed “complete” datasets.
Because of the large number of potential predictors, especially those related to HRQOL, we made several decisions to facilitate variable selection. We incorporated SF-36 scales rather than individual items to maximize measurement precision and because scale scores are normed to the U.S. general population. To further reduce variables in the model and because an a priori theoretical justification for variable selection is lacking, we applied a backward elimination strategy with an Akaike’s information criterion stopping rule53 to a model, including all predictor variables in each imputed dataset. This is equivalent to using a P-value of 0.157 for a variable with one degree of freedom.54 We then applied the majority method, including variables selected in five or more of the 10 imputed sets.55
Age, sex, race/ethnicity, education, proxy status, hypertension, congestive heart failure, stroke, chronic obstructive pulmonary disease, presence of any cancer, smoking status, ADL score, and SF-36 scores for bodily pain, general health perceptions, emotional well-being, social functioning, and energy/fatigue were selected into a final model. Of the continuous variables, age showed a nonlinear association with six-month mortality, and was, therefore, modeled as a restricted cubic spline with the 5%, 35%, 65% and 95% percentiles of age.
Some variables showed counterintuitive associations, with lower six-month mortality in both univariate and multivariate analyses: non-white race, low education, hypertension, stroke, greater bodily pain, and low emotional well-being. Some of these counterintuitive associations have been found in other studies,43, 5659 and may reflect confounding by unmeasured variables (e.g., health care access and quality), selection biases that could have altered the relative influence of competing mortality risks (e.g., restriction to a managed care sample, selection according to self-reported declining health), or the effects of illness adaptation on participants’ HRQOL ratings.60, 61 Counterintuitive associations potentially diminish the “sensibility” or face validity of risk prediction models for clinicians, and many modelers thus recommend excluding the variables involved.6264 Other modelers have additionally excluded race/ethnicity and education both because of confounding of their prognostic significance and to ethical concerns about the potential for models incorporating these variables to contribute to health disparities.31, 65 For these reasons and to maximize parsimony, we conducted sensitivity analyses both including and excluding counterintuitively-associated variables in multivariate regression models. Model fit, discrimination, and calibration were similar, and, therefore, we excluded these variables. Regression coefficients and standard errors for variables in the final model were computed by averaging across the 10 imputed datasets, following Rubin’s method.66
Statistical Analyses and Model Evaluation
Because the MHOS sample is composed of Medicare managed care beneficiaries whose access to care, health status, and thus mortality might differ from the general U.S. population,67 we generated life tables68, 69 comparing overall survival of the MHOS sample with that of the year 2000 general U.S. population to assess representativeness. We also calculated descriptive statistics on sociodemographic and health-related characteristics of the sample.
We used k-fold cross-validation (k=10) to validate the model. Each of the 10 imputed datasets were randomly partitioned into k subsamples, with each subsample used once as the validation set and the remaining k-1 set used as the training set. We assessed model discrimination by calculating the c-statistic, or area under the receiver operating characteristic curve (AUC), averaging the c-statistics across all imputed datasets and cross-validation samples.70 To further evaluate prognostic performance of the final model, we computed estimated six-month mortalities of individuals within each of the 10 imputed datasets, and obtained mortality estimates by averaging across them. We assessed calibration by dividing patients into seven overlapping groups according to their predicted six-month mortality risk (≥0.1, ≥0.2,…, ≥0.7) and comparing the average predicted six-mortality in each group with the actual proportion of patients who died in six months. To assess calibration graphically across estimated risk strata, we used a non-parametric method (PROC LOESS; SAS Institute Inc., Cary, NC) to produce a smoothed high-resolution calibration curve with histogram plot.71
We then calculated sensitivity, specificity, positive and negative predictive values, and positive and negative likelihood ratios at different estimated mortality risk thresholds to compare performance characteristics of the PROMPT to those of the NHO guidelines and SUPPORT model, reported by Fox et al.19 Finally, we generated Kaplan-Meier curves to compare extended survival of respondents across all risk strata.
The MHOS and general U.S. population survival curves were similar (Fig. 2), although the MHOS sample had slightly better survival than the U.S. population; the ages at median survival probability were 85 and 83 years, respectively, for the MHOS and U.S. populations. Table 1 shows the distribution of sociodemographic characteristics, comorbidities, and HRQOL.
Figure 2
Figure 2
Life tables of the Medicare Health Outcomes Survey study sample population (1998–2001 2-year cohorts) vs. 2000 US general population
Table 1
Table 1
Study Sample Characteristics, Medicare Health Outcomes Survey
Table 2 shows the 11 variables included in the PROMPT and their associations with six-month mortality in the entire study sample. HRQOL variables included ADLs, general health perceptions, social functioning, and energy/fatigue. The c-statistic obtained from tenfold cross-validation was 0.752, indicating good overall discrimination. The model was well-calibrated at lower estimated risk values; however, for estimated risk g than 50%, it overestimated mortality, likely reflecting the small number of events at higher risk strata (Fig. 3).
Table 2
Table 2
Final Multivariable Prognostic Model for Six-Month Mortality (PROMPT)
Figure 3
Figure 3
Calibration curve for actual vs. predicted 6-month mortality, Patient Reported Outcome Mortality Prediction Tool (PROMPT)
Table 3 shows the performance characteristics of the PROMPT compared with the NHO guidelines and the SUPPORT prognostic model. At estimated six-month mortality risk thresholds of 10%–70%, model sensitivity and specificity were 0.8%–83.4% and 51.1%–99.99%, respectively, and corresponding positive and negative predictive values were 23.1%–64.1% and 85.3%–94.5%. Positive likelihood ratios exceeded 5.0 at all risk thresholds of 40% or more, while negative likelihood ratios were near 1.0. At comparable estimated risk thresholds, diagnostic performance was superior to the NHO and SUPPORT models.
Table 3
Table 3
Performance Characteristics of the PROMPT for Six-Month Mortality Compared With NHO Guidelines and the SUPPORT Prognostic Model
Figure 4 shows Kaplan-Meier survival curves for respondents in different estimated risk strata. Observed six-month mortality corresponded well to estimated risk, supporting the model’s calibration, and by 12 months more than 50% of all patients in estimated risk strata of 30% or more had died.
Figure 4
Figure 4
Kaplan-Meier survival curves for Medicare Health Outcomes Survey respondents in different model-estimated 6-month mortality risk strata
In this study we developed a new prognostic model, the PROMPT, to predict six-month mortality in community-dwelling elderly patients with self-reported declining health. The model was developed using a large diverse sample, and utilized 11 total variables, including HRQOL, ascertained by patient self-report. The model demonstrated good calibration and discrimination overall. Importantly, diagnostic performance at various thresholds of estimated risk was superior to existing non-disease-specific models. Specificity was high, and at strata of estimated six-month mortality risk of 40% or more, the model yielded positive likelihood ratios of moderate to large magnitude (5.0 or more) with respect to clinical prediction—exceeding the performance of previous models and increasing the post-test odds of death to an extent generally considered useful in clinical decision making. The model’s positive predictive value in our study population was also high and commensurate to estimated risk; 53% of patients in the 50% risk stratum died by six months, and the proportions of observed deaths were correspondingly greater in higher risk strata. On extended observation, over half of all patients with estimated six-month mortality risk of 30% or more died by 12 months.
These promising performance characteristics are particularly noteworthy given the types of variables used in our model, and the nature of our study population. Unlike prior prognostic efforts, the PROMPT included no physiologic or laboratory data and relatively little clinical data regarding disease characteristics or health services utilization. The study population was clinically heterogeneous, ambulatory, community-dwelling, and relatively healthy, with a lower pretest mortality risk, compared with populations included in other prognostic modeling efforts and for whom hospice care is typically considered by clinicians. Yet, in spite of these significant constraints on prognostic power, the PROMPT still demonstrated superior performance compared to existing tools such as the SUPPORT model and NHO guidelines. This supports the model’s robustness and potential transportability to more narrowly-defined populations with higher pretest probabilities of mortality in which prognostic performance would likely be more optimal. These conclusions remain preliminary, however, since our model has yet to be externally validated and directly compared to other tools.
The primary limitation of the PROMPT is one shared by all existing prognostic tools for predicting short-term mortality: insufficient sensitivity to “rule out” death in a substantial proportion of patients. This is not surprising given that there are undoubtedly numerous causal factors and trajectories in the dying process,72,73 and no prognostic model has accounted for them all. For the PROMPT, furthermore, comorbidities were ascertained by self-report only, and some important ones, e.g., dementia, renal and liver disease, were not assessed. A substantial proportion of surveys (38%) also were completed by proxy, presumably because patients were too ill or impaired to do so themselves. Finally, our selection of patients on the basis of self-reported decline in health likely also led to the inclusion of patients with acute, self-limited conditions with little impact on mortality. All of these factors likely limit the PROMPT’s sensitivity and use as an exclusive means of determining hospice eligibility because this would result in the denial of hospice services for a majority of dying patients. This limitation has led other modelers to conclude that the goal of determining individuals’ risk of six-month mortality is unrealistic.19, 25
Yet, we believe our modeling effort offers important insights for future research, and that the PROMPT has significant potential utility for clinical care. Our study adds to mounting evidence of the prognostic power of HRQOL. The PROMPT’s superior overall performance compared with efforts incorporating disease and physiologic variables alone supports the hypothesis that as death approaches, HRQOL assumes greater prognostic significance.32, 33 The prominent role of similar HRQOL variables in other prognostic models in elderly patients with advanced illness24, 74 further bears this out, supporting the PROMPT’s validity and the value of integrating HRQOL in future modeling efforts.
Furthermore, despite its low sensitivity in ruling out imminent death, the PROMPT has significant potential to improve end-of-life care given the prevailing underutilization of hospice services, overutilization of life-prolonging interventions, and lack of more accurate, evidence-based and explicit prognostic methods. These circumstances alone raise the possibility that use of the model could increase hospice utilization and advance care planning. Yet, the PROMPT’s greatest potential value lies in its ability to confirm a poor six-month prognosis. Its very high specificity across a range of estimated mortality risks (97% or more for all estimated risk cutpoints of 40% or greater) makes the PROMPT an extremely valuable tool for “ruling in” imminent death, with very few false positives. From both an ethical and a clinical standpoint, this function has at least as much clinical importance as ruling out death. The potential harm of a false negative estimate of six-month mortality is overly aggressive care at the end of life. Although undesirable, this outcome is arguably more tolerable than the potential irreversible harm of a false positive estimate: mistakenly labeling patients as “dying” and forgoing potentially beneficial or curative interventions. This ethical concern may be an important reason for physicians’ clinical reluctance to render prognoses,22, 75 and patients’ reluctance to accept them.76 The PROMPT’s ability to identify imminently dying patients with very few false positives addresses this concern, providing physicians and patients with the necessary reassurance to make critical decisions about end-of-life care.
A final limitation of the PROMPT’s performance is its weaker calibration at higher estimated risk levels, at which it overestimated mortality. This is likely a consequence of the small number of total deaths in these subgroups, reflecting the low overall mortality rate of the study sample (15%). In sicker populations with a higher pretest likelihood of mortality, it is possible that the model’s calibration would be improved.
However, this remains to be seen, and further evaluation is needed before the PROMPT can be implemented clinically. Although the large size and geographic and clinical heterogeneity of the study population enhances the model’s generalizability, it needs to be validated prospectively in other populations with differing comorbidities and experiences with health care. The target population of any predictive model determines both its clinical appropriateness and performance characteristics, including sensitivity and specificity,77 and ours consisted of community-dwelling elders with self-reported declining health. However, the PROMPT might be more accurate and useful in alternative populations, for example, patients identified on the basis of comorbidities and health care utilization (as in the SUPPORT study19, 21 and more recent prognostic model efforts in nursing home patients with dementia24), or physicians’ own prognostic estimates,78, 79 recently operationalized through what has been termed the “surprise question:”8082 “Would I be surprised if this patient died in the next 12 months?” Applying our tool in such selected populations—with higher pre-test probabilities of six-month mortality—would likely improve prognostic performance. Future research also might fruitfully examine whether testing strategies combining multiple prognostic tools and factors could further enhance prognostic power. For example, sensitivity might be increased by employing the PROMPT in parallel with other approaches, such as physicians’ prognostic estimates or the CMS Local Coverage Determination guidelines83 used by hospice providers to determine hospice eligibility.
Finally, more work is needed not only to evaluate and improve the PROMPT, but to understand the optimal means and outcomes of applying prognostic models clinically. The landmark SUPPORT study demonstrated that simply providing physicians with prognostic information may not alter end-of-life decision making or patterns of care.84 Various factors including physician and patient attitudes14, 22 and the structures and processes of health care may limit effective utilization of prognostic models. The feasibility of implementing a PRO-based model such as ours, which utilizes 11 variables but 28 individual data elements, remains to be determined. These and other potential barriers need to be better understood, along with the appropriate methods for communicating prognostic information in a sensitive, comprehensible manner. The current effort provides a foundation for such work and for efforts to refine existing models and to determine optimal strategies for their implementation.
Acknowledgments
This study was supported by intramural research funds from the National Cancer Institute, National Institutes of Health (NIH). Ron Hays was supported in part by NIA grants (P30AG021684 and P30-AG028748) and an NCMHD grant (2P20MD000182).
We thank Robert Arnold, Rachel Ballard-Barbash, Steve Clauser, and anonymous reviewers for helpful comments on an earlier version of the manuscript.
Footnotes
Disclosures
The authors declare no conflicts of interest.
The findings and conclusions in this report are those of the authors and do not necessarily represent the views of the National Cancer Institute.
This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
1. Centers for Medicare and Medicaid Services (CMS) Pub 100-02 Medicare benefit policy manual. Baltimore, MD: CMS, U.S. Department of Health & Human Services; 2004.
2. Brody H, Lynn J. The physician’s responsibility under the new Medicare reimbursement for hospice care. N Engl J Med. 1984;310(14):920–922. [PubMed]
3. Christakis NA, Escarce JJ. Survival of Medicare patients after enrollment in hospice programs. N Engl J Med. 1996;335(3):172–178. [PubMed]
4. Emanuel EJ, Ash A, Yu W, et al. Managed care, hospice use, site of death, and medical expenditures in the last year of life. Arch Intern Med. 2002;162(15):1722–1728. [PubMed]
5. McCarthy EP, Burns RB, Ngo-Metzger Q, Davis RB, Phillips RS. Hospice use among Medicare managed care and fee-for-service patients dying with cancer. JAMA. 2003;289(17):2238–2245. [PubMed]
6. Connor SR, Elwert F, Spence C, Christakis NA. Geographic variation in hospice use in the United States in 2002. J Pain Symptom Manage. 2007;34(3):277–285. [PubMed]
7. Flory J, Yinong YX, Gurol I, et al. Place of death: U.S. trends since 1980. Health Aff (Millwood) 2004;23(3):194–200. [PubMed]
8. Angus DC, Barnato AE, Linde-Zwirble WT, et al. Use of intensive care at the end of life in the United States: an epidemiologic study. Crit Care Med. 2004;32(3):638–643. [PubMed]
9. Barnato AE, McClellan MB, Kagay CR, Garber AM. Trends in inpatient treatment intensity among Medicare beneficiaries at the end of life. Health Serv Res. 2004;39(2):363–375. [PMC free article] [PubMed]
10. Steinhauser KE, Christakis NA, Clipp EC, et al. Factors considered important at the end of life by patients, family, physicians, and other care providers. JAMA. 2000;284(19):2476–2482. [PubMed]
11. Steinhauser KE, Clipp EC, McNeilly M, et al. In search of a good death: observations of patients, families, and providers. Ann Intern Med. 2000;132(10):825–832. [PubMed]
12. Higginson IJ, Sen-Gupta GJ. Place of care in advanced cancer: a qualitative systematic literature review of patient preferences. J Palliat Med. 2000;3(3):287–300. [PubMed]
13. Friedman BT, Harwood MK, Shields M. Barriers and enablers to hospice referrals: an expert overview. J Palliat Med. 2002;5(1):73–84. [PubMed]
14. Lamont EB. A demographic and prognostic approach to defining the end of life. J Palliat Med. 2005;8(Suppl 1):S12–21. [PubMed]
15. Brickner L, Scannell K, Marquet S, Ackerson L. Barriers to hospice care and referrals: survey of physicians’ knowledge, attitudes, and perceptions in a health maintenance organization. J Palliat Med. 2004;7(3):411–418. [PubMed]
16. Christakis NA, Lamont EB. Extent and determinants of error in physicians’ prognoses in terminally ill patients: prospective cohort study. West J Med. 2000;172(5):310–313. [PMC free article] [PubMed]
17. Chow E, Harth T, Hruby G, et al. How accurate are physicians’ clinical predictions of survival and the available prognostic tools in estimating survival times in terminally ill cancer patients? A systematic review. Clin Oncol (R Coll Radiol) 2001;13(3):209–218. [PubMed]
18. Stuart B. The NHO medical guidelines for non-cancer disease and local medical review policy: hospice access for patients with diseases other than cancer. Hosp J. 1999;14(3–4):139–154. [PubMed]
19. Fox E, Landrum-McNiff K, Zhong Z, et al. Evaluation of prognostic criteria for determining hospice eligibility in patients with advanced lung, heart, or liver disease. SUPPORT Investigators. Study to Understand Prognoses and Preferences for Outcomes and Risks of Treatments. JAMA. 1999;282(17):1638–1645. [PubMed]
20. Schonwetter RS, Han B, Small BJ, et al. Predictors of six-month survival among patients with dementia: an evaluation of hospice Medicare guidelines. Am J Hosp Palliat Care. 2003;20(2):105–113. [PubMed]
21. Knaus WA, Harrell FE, Jr, Lynn J, et al. The SUPPORT prognostic model. Objective estimates of survival for seriously ill hospitalized adults. Study to understand prognoses and preferences for outcomes and risks of treatments. Ann Intern Med. 1995;122(3):191–203. [PubMed]
22. Glare PA, Sinclair CT. Palliative medicine review: prognostication. J Palliat Med. 2008;11(1):84–103. [PubMed]
23. Chow E, Fung K, Panzarella T, et al. A predictive model for survival in metastatic cancer patients attending an outpatient palliative radiotherapy clinic. Int J Radiat Oncol Biol Phys. 2002;53(5):1291–1302. [PubMed]
24. Mitchell SL, Kiely DK, Hamel MB, et al. Estimating prognosis for nursing home residents with advanced dementia. JAMA. 2004;291(22):2734–2740. [PubMed]
25. Mitchell SL, Miller SC, Teno JM, Davis RB, Shaffer ML. The advanced dementia prognostic tool: a risk score to estimate survival in nursing home residents with advanced dementia. J Pain Symptom Manage. 2010;40(5):639–651. [PMC free article] [PubMed]
26. Mitchell SL, Miller SC, Teno JM, et al. Prediction of 6-month survival of nursing home residents with advanced dementia using ADEPT vs. hospice eligibility guidelines. JAMA. 2010;304(17):1929–1935. [PMC free article] [PubMed]
27. Anderson F, Downing GM, Hill J, Casorso L, Lerch N. Palliative performance scale (PPS): a new tool. J Palliat Care. 1996;12(1):5–11. [PubMed]
28. Morita T, Tsunoda J, Inoue S, Chihara S. The Palliative Prognostic Index: a scoring system for survival prediction of terminally ill cancer patients. Support Care Cancer. 1999;7(3):128–133. [PubMed]
29. Teno JM, Harrell FE, Jr, Knaus W, et al. Prediction of survival for older hospitalized patients: the HELP survival model. Hospitalized Elderly Longitudinal Project. J Am Geriatr Soc. 2000;48(5 Suppl):S16–24. [PubMed]
30. Walter LC, Brand RJ, Counsell SR, et al. Development and validation of a prognostic index for 1-year mortality in older adults after hospitalization. JAMA. 2001;285(23):2987–2994. [PubMed]
31. Lee SJ, Lindquist K, Segal MR, Covinsky KE. Development and validation of a prognostic index for 4-year mortality in older adults. JAMA. 2006;295(7):801–808. [PubMed]
32. Mackillop WJ. Differences in prognostication between early and advanced cancer. In: Glare P, Christakis NA, editors. Prognosis in advanced cancer. New York: Oxford University Press; 2008. pp. 13–523.
33. Glare P, Christakis NA. Overview: advancing the clinical science of prognostication. In: Glare P, Christakis NA, editors. Prognosis in advanced cancer. New York: Oxford University Press; 2008. pp. 3–12.
34. Dorr DA, Jones SS, Burns L, et al. Use of health-related, quality-of-life metrics to predict mortality and hospitalizations in community-dwelling seniors. J Am Geriatr Soc. 2006;54(4):667–673. [PubMed]
35. Coates A, Porzsolt F, Osoba D. Quality of life in oncology practice: prognostic value of EORTC QLQ-C30 scores in patients with advanced malignancy. Eur J Cancer. 1997;33(7):1025–1030. [PubMed]
36. Vigano A, Donaldson N, Higginson IJ, et al. Quality of life and survival prediction in terminal cancer patients: a multicenter study. Cancer. 2004;101(5):1090–1098. [PubMed]
37. Rumsfeld JS, MaWhinney S, McCarthy M, Jr, et al. Health-related quality of life as a predictor of mortality following coronary artery bypass graft surgery. Participants of the Department of Veterans Affairs Cooperative Study Group on Processes, Structures, and Outcomes of Care in Cardiac Surgery. JAMA. 1999;281(14):1298–1303. [PubMed]
38. Chang VT, Thaler HT, Polyak TA, et al. Quality of life and survival: the role of multidimensional symptom assessment. Cancer. 1998;83(1):173–179. [PubMed]
39. Carey EC, Walter LC, Lindquist K, Covinsky KE. Development and validation of a functional morbidity index to predict mortality in community-dwelling elders. J Gen Intern Med. 2004;19(10):1027–1033. [PMC free article] [PubMed]
40. Meyer F, Fortin A, Gelinas M, et al. Health-related quality of life as a survival predictor for patients with localized head and neck cancer treated with radiation therapy. J Clin Oncol. 2009;27(18):2970–2976. [PubMed]
41. Dancey J, Zee B, Osoba D, et al. Quality of life scores: an independent prognostic variable in a general population of cancer patients receiving chemotherapy. The National Cancer Institute of Canada Clinical Trials Group. Qual Life Res. 1997;6(2):151–158. [PubMed]
42. Inouye SK, Peduzzi PN, Robison JT, et al. Importance of functional measures in predicting mortality among older hospitalized patients. JAMA. 1998;279(15):1187–1193. [PubMed]
43. Cunningham WE, Crystal S, Bozzette S, Hays RD. The association of health-related quality of life with survival among persons with HIV infection in the United States. J Gen Intern Med. 2005;20(1):21–27. [PMC free article] [PubMed]
44. Selim AJ, Berlowitz DR, Fincke G, et al. Risk-adjusted mortality rates as a potential outcome indicator for outpatient quality assessments. Med Care. 2002;40(3):237–245. [PubMed]
45. Covinsky KE, Justice AC, Rosenthal GE, Palmer RM, Landefeld CS. Measuring prognosis and case mix in hospitalized elders. The importance of functional status. J Gen Intern Med. 1997;12(4):203–208. [PMC free article] [PubMed]
46. Vigano A, Dorgan M, Buckingham J, Bruera E, Suarez-Almazor ME. Survival prediction in terminal cancer patients: a systematic review of the medical literature. Palliat Med. 2000;14(5):363–374. [PubMed]
47. Schonberg MA, Davis RB, McCarthy EP, Marcantonio ER. Index to predict 5-year mortality of community-dwelling adults aged 65 and older using data from the national health interview survey. J Gen Intern Med. 2009;24(10):1115–1122. [PMC free article] [PubMed]
48. Cooper JK, Kohlmann T, Michael JA, Haffer SC, Stevic M. Health outcomes. New quality measure for Medicare. Int J Qual Health Care. 2001;13(1):9–16. [PubMed]
49. Jones N, 3rd, Jones SL, Miller NA. The Medicare Health Outcomes Survey program: overview, context, and near-term prospects. Health Qual Life Outcomes. 2004;2:33. [PMC free article] [PubMed]
50. Haffer SC, Bowen SE. Measuring and improving health outcomes in Medicare: the Medicare HOS program. Health Care Financ Rev. 2004;25(4):1–3. [PubMed]
51. Ware JE, Jr, Sherbourne CD. The MOS 36-item short-form health survey (SF-36). I. Conceptual framework and item selection. Med Care. 1992;30(6):473–483. [PubMed]
52. Heron M, Hoyert D, Murphy S, et al. Deaths: final data for 2006. Natl Vital Stat Rep. 2009;57(14):1–134. [PubMed]
53. Atkinson AC. A note on the generalized information criterion for choice of a model. Biometrika. 1980;67:413–418.
54. Sauerbrei W. The use of resampling methods to simplify regression models in medical statistics. Appl Stat. 1999;48:313–329.
55. Vergouwe Y, Royston P, Moons KG, Altman DG. Development and validation of a prediction model with missing predictor data: a practical approach. J Clin Epidemiol. 2010;63(2):205–214. [PubMed]
56. Iezzoni LI, Foley SM, Daley J, et al. Comorbidities, complications, and coding bias. Does the number of diagnosis codes matter in predicting in-hospital mortality? JAMA. 1992;267(16):2197–2203. [PubMed]
57. Jencks SF, Williams DK, Kay TL. Assessing hospital-associated deaths from discharge data. The role of length of stay and comorbidities. JAMA. 1988;260(15):2240–2246. [PubMed]
58. Parkerson GR, Jr, Gutman RA. Health-related quality of life predictors of survival and hospital utilization. Health Care Financ Rev. 2000;21(3):171–184. [PubMed]
59. Kimmel PL, Peterson RA, Weihs KL, et al. Psychosocial factors, behavioral compliance and survival in urban hemodialysis patients. Kidney Int. 1998;54(1):245–254. [PubMed]
60. Riis J, Loewenstein G, Baron J, et al. Ignorance of hedonic adaptation to hemodialysis: a study using ecological momentary assessment. J Exp Psychol Gen. 2005;134(1):3–9. [PubMed]
61. Sprangers MA, Schwartz CE. Reflections on changeability versus stability of health-related quality of life: distinguishing between its environmental and genetic components. Health Qual Life Outcomes. 2008;6:89. [PMC free article] [PubMed]
62. Reilly BM, Evans AT. Translating clinical research into clinical practice: impact of using prediction rules to make decisions. Ann Intern Med. 2006;144(3):201–209. [PubMed]
63. Laupacis A, Sekar N, Stiell IG. Clinical prediction rules. A review and suggested modifications of methodological standards. JAMA. 1997;277(6):488–494. [PubMed]
64. Hadorn DC, Keeler EB, Rogers WH, Brook RH. Assessing the performance of mortality prediction models. Santa Monica, CA: RAND; 1993.
65. Schonberg MA, Davis RB, McCarthy EP, Marcantonio ER. Index to predict 5-year mortality of community-dwelling adults aged 65 and older using data from the National Health Interview Survey. J Gen Intern Med. 2009;24(10):1115–1122. [PMC free article] [PubMed]
66. Rubin DB. Multiple Imputation for non-response in surveys. New York: Wiley; 1987.
67. Ng JH, Kasper JD, Forrest CB, Bierman AS. Predictors of voluntary disenrollment from Medicare managed care. Med Care. 2007;45(6):513–520. [PubMed]
68. Cutler SJ, Ederer F. Maximum utilization of the life table method in analyzing survival. J Chronic Dis. 1958;8:699–712. [PubMed]
69. Dignam JJ, Huang L, Ries L, et al. Estimating breast cancer-specific and other-cause mortality in clinical trial and population-based cancer registry cohorts. Cancer. 2009;115(22):5272–5283. [PMC free article] [PubMed]
70. Subramanian J, Simon R. An evaluation of resampling methods for assessment of survival risk prediction in high-dimensional settings. Stat Med. 2011;30(6):642–653. [PubMed]
71. Harrell FE, Lee KL, Mark DB. Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med. 1996;15:361–387. [PubMed]
72. Dy S, Lynn J. Getting services right for those sick enough to die. BMJ. 2007;334(7592):511–513. [PMC free article] [PubMed]
73. Murray SA, Kendall M, Boyd K, Sheikh A. Illness trajectories and palliative care. BMJ. 2005;330(7498):1007–1011. [PMC free article] [PubMed]
74. Flacker JM, Kiely DK. Mortality-related factors and 1-year survival in nursing home residents. J Am Geriatr Soc. 2003;51(2):213–221. [PubMed]
75. Christakis NA. Death foretold: Prophecy and prognosis in medical care. Chicago: University of Chicago Press; 1999.
76. Fried TR, Drickamer M. Garnering support for advance care planning. JAMA. 2010;303(3):269–270. [PMC free article] [PubMed]
77. Moons KG, Harrell FE. Sensitivity and specificity should be de-emphasized in diagnostic accuracy studies. Acad Radiol. 2003;10(6):670–672. [PubMed]
78. Glare P, Virik K, Jones M, et al. A systematic review of physicians’ survival predictions in terminally ill cancer patients. BMJ. 2003;327(7408):195–198. [PMC free article] [PubMed]
79. Maltoni M, Caraceni A, Brunelli C, et al. Prognostic factors in advanced cancer patients: evidence-based clinical recommendations--a study by the Steering Committee of the European Association for Palliative Care. J Clin Oncol. 2005;23(25):6240–6248. [PubMed]
80. Moss AH, Ganjoo J, Sharma S, et al. Utility of the “surprise” question to identify dialysis patients with high mortality. Clin J Am Soc Nephrol. 2008;3(5):1379–1384. [PubMed]
81. Cohen LM, Ruthazer R, Moss AH, Germain MJ. Predicting six-month mortality for patients who are on maintenance hemodialysis. Clin J Am Soc Nephrol. 2010;5(1):72–79. [PubMed]
82. Pattison M, Romer AL. Improving care through the end of life: launching a primary care clinic-based program. J Palliat Med. 2001;4(2):249–254. [PubMed]
83. Centers for Medicare and Medicaid Services. [Accessed March 30, 2010.];LCD for hospice determining terminal status (L13653) 2009 Available from http://www.cms.hhs.gov/mcd/viewlcd.asp?lcd_id=13653&lcd_version=30&show=all.
84. The SUPPORT Principal Investigators. A controlled trial to improve care for seriously ill hospitalized patients. The Study to Understand Prognoses and Preferences for Outcomes and Risks of Treatments (SUPPORT) JAMA. 1995;274(20):1591–1598. [PubMed]