Search tips
Search criteria 


Logo of jpmMary Ann Liebert, Inc.Mary Ann Liebert, Inc.JournalsSearchAlerts
Journal of Palliative Medicine
J Palliat Med. 2012 December; 15(12): 1292–1296.
PMCID: PMC3509357

Selection Bias in Family Reports on End of Life with Dementia in Nursing Homes

Jenny T. van der Steen, Ph.D.,corresponding author1,* Luc Deliens, Ph.D.,2,3 Miel W. Ribbe, M.D., Ph.D.,1 and Bregje D. Onwuteaka-Philipsen, Ph.D.2



: Selective participation in retrospective studies of families recruited after the patient's death may threaten generalizability of reports on end-of-life experiences.


To assess possible selection bias in retrospective study of dementia at the end of life using family reports.


Two physician teams covering six nursing home facilities in the Netherlands reported on 117 of 119 consecutive decedents within two weeks after death unaware of after-death family participation in the study. They reported on characteristics; treatment and care; overall patient outcomes such as comfort, nursing care, and outcomes; and their own perspectives on the experience. We compared results between decedents with and without family participation.


The family response rate was 55%. There were no significant differences based on participation versus nonparticipation in demographics and other nursing home resident characteristics, treatment and care, or overall resident outcome. However, among participating families, physicians perceived higher-quality aspects of nursing care and outcome, better consensus between staff and family on treatment, and a more peaceful death. Participation was less likely with involvement of a new family member in the last month.


Families may be more likely to participate in research with more harmonious teamwork in end-of-life caregiving. Where family participation is an enrollment criterion, comparing demographics alone may not capture possible selection bias, especially in more subjective measures. Selection bias toward more positive experiences, which may include the physician's and probably also the family's experiences, should be considered if representativeness is aimed for. Future work should address selection bias in other palliative settings and countries, and with prospective recruitment.


Palliative care in nursing home residents is increasingly being studied.1,2 Many residents die with dementia. Most quantitative studies (about two-thirds) focusing on dementia have employed a retrospective design, whereas one-third included pre-death data in a prospective observational or intervention design.1

In retrospective—also called (mortality) follow-back—designs,3 data are collected on decedents, thus avoiding problems defining palliative care populations.4 This is a benefit for conditions with ill-defined prognoses such as dementia. Retrospective studies are appropriate and efficient where representativeness for a certain population is required and may provide valuable data for national policies on end-of-life decision making, for example.5,6

However, generalizability of findings is threatened if response is selective and response rates are low. Families frequently report on end-of-life experiences both as an outcome important on its own and as proxy respondents.1,7,8 Data are usually collected only if families participate. Reasons for this include for efficiency, a focus on families, integrating of multiple perspectives, or the need for consent regardless if charts are accessed. Concerns on selection bias arise, because typically one-fifth to half of families in observational studies on dementia in nursing homes do not consent to participate (e.g., 20%,9 29.2% or 29.3%,10 39%,11 40%,12 47%13). These families may differ from those who do participate. Frequently, however, comparison is made between participants' (respondents') and nonrespondents' demographics only, which may not fully capture possible differences in end-of-life experiences between these groups.

We retrospectively collected data on quality of care and dying in dementia, providing the physician's perspective both when families did and did not participate. We compare these physicians' reports and discuss possible selection bias.


The data were collected as part of the Dutch End of Life in Dementia (DEOLD) study describing quality of dying and end-of-life care and assessing associated factors.1,14 The protocol was approved by the Medical Ethics Committee of the VU University Medical Center in Amsterdam. Seventeen of 19 participating nursing home organizations each employing its own physician team of qualified elderly care physicians, averaging one full-time equivalent (fte) per hundred residents,15,16 collected data prospectively. To increase the number of reports on decedents while avoiding the complicated logistics involved in prospective studies, we proposed two teams (two organizations) covering six facilities (five nursing homes and one residential home) to collect data retrospectively after death only, which we report on in this contribution. The treating physicians completed written questionnaires within two weeks of consecutive deaths between November 1, 2007 (two facilities covered by one team) or February 1, 2009 (four facilities; the other team), and March 1, 2010.

Residents were eligible if they resided in a psychogeriatric ward or unit, had a physician's diagnosis of dementia, had been admitted for long-term care, and had a family representative able to understand and write Dutch or English. Six weeks after death, nursing home staff, with a letter, invited families of eligible residents to participate, which involved completing a questionnaire two months after death. Further correspondence was with the researchers only; staff was unaware of family participation. There were no reminders and no contacting by the researchers without consent. Residents were enrolled regardless of active family consent, if families did not object to coded data being provided to the researchers.

Outcome measures developed for retrospective assessment included the End-of-Life in Dementia (EOLD) scales,17 and we report on the Comfort Assessment in Dying (EOLD-CAD) scale, with better properties than the EOLD-Symptom Management.13 The Pain Assessment in Advanced Dementia (PAINAD) pain tool18 was applied retrospectively, as was the Quality of Life in Late-Stage Dementia (QUALID) scale.19 The Dutch versions of these measures had been tested prospectively in independent samples.20,21 As in previous work in the United States,22 advanced dementia was defined as a Global Deterioration Scale (GDS) score of 723 and a Cognitive Performance Scale (CPS) of 5 or 6.24 Four burdensome treatments in the last week of life were defined as hospitalization, emergency room visit, new or ongoing parenteral therapy, or tube feeding.22,25 Additionally, a variety of care and process-related items addressed issues relevant beyond biomedical aspects, such as on family-staff consensus and relationships, based on our earlier work in pneumonia26 or developed for the purpose of the study. We report on these measures specifically as possibly relevant to selection bias. Selected items could be completed by either the physician or the nurse who was most involved.

Selecting cases with complete physician reports, analyses included chi-square and t-tests to compare groups. Missing items in outcome measures were imputed with resident means if a maximum of one-third was missing.11 For factors that differed significantly by family participation status, we additionally examined a family's own rating of Satisfaction with Care (EOLD-SWC)17 scores by physician reports, selecting the families who responded. Analyses were performed with statistical software PASW 18.0.0 (IBM SPSS Statistics, IBM Corporation, New York).


Of 121 eligible cases, 119 were enrolled, with 117 having physician reports and 66 family reports (overall 55%; 54% for one home, and 57% for the other home). Most of the 53 families who did not participate did not respond (72%, 38 of 53). Eleven families (21%) refused, two refused but noted they might participate later, and two declined after judging the questionnaire as too difficult. Other reasons for refusal were being busy, being emotional, relating to closure, family ill health, and being disappointed with care.

Resident age and gender, length of stay, or dementia severity did not differ by family participation status (see Table 1). Further, treatment and resident outcome did not differ. However, of the nursing care and outcome, mouth care, and odor were perceived as significantly better when families participated. Participation was also more likely when no new family member became involved, and physicians then perceived consensus on treatment between staff and family as better and the resident's death as more peaceful. The difference in consensus could not be attributed to consensus among staff only (p=0.10) or among family only (p=0.22; not in Table 1). When physicians reported full consensus, satisfaction with care of participating families was significantly better (mean EOLD-SWC score 34.2 SD 4.2 versus 26.3 SD 7.9; p=0.001), as it was with a reportedly peaceful death (mean 32.7 SD 5.8 versus 26.7 SD 9.3; p=0.03; not in Table).

Table 1.
Physicians' Reports by Family Participation Status in After-Death Data Collection on Nursing Home Residents with Dementia


The DEOLD study provided a unique opportunity to examine methodological issues in studying end of life, with data available both when families participated and when they did not. We found that there were no significant differences according to after-death family participation between demographic and other nursing home resident characteristics such as cognition, and between treatment, care, and overall resident outcome. However, when families participated, physicians perceived aspects of nursing care and outcome and staff-family consensus on treatment as better, and death as more peaceful. Further, where additional family members became involved during the last phase, family participation was less likely.

Perhaps families are more willing to participate in research where there has been more harmonious teamwork among nurses and families. Such families may have been more involved overall, and family oversight may help notice, for example, odor or motivate nurses to provide better and more personally rewarding end-of-life care.27 Fortunately, physicians' reported treatment and main outcomes did not differ by family participation status.

Kahn and colleagues28 in a longitudinal study on quality of care in various settings found that patients who were least satisfied more frequently dropped out. In our study, less favorable family experiences likely explain nonparticipation, as reasons for refusal included being disappointed with care; and in responding families, the physicians' perceptions on consensus and a peaceful death related to considerably higher family satisfaction. Our findings on physicians' perspectives might therefore underestimate family response bias. More generally, such bias towards enrolling more favorable cases might partially explain why quantitative work often presents a more positive view than does qualitative work observing severe problems in end-of-life care.1 However, selection bias may differ in prospective studies, with families refusing participation because they perceive care as already optimal.14

Our work was limited by a modest sample of the retrospective part of DEOLD. Results were similar with adjustment for clustering of decedents with physicians, for which models could be fitted for most, but not all factors.

High response rates, higher than 60% as a rule of thumb,29 may remedy selection bias. However, with after-death recruitment, use of more personal recruitment strategies is complicated by limited contact between homes and bereaved family. Further, selection bias may also occur at a facility level. Participating homes may provide above-average quality of care as reported in publicly available databases,22 although quality was average in DEOLD.14 Of note, while 92% of Dutch dementia patients die in nursing or residential homes,30 almost half of the residents never reached the advanced stage to which many studies are limited.1


Aiming at representativeness, demographics alone may insufficiently reflect selection bias in enrolling families in cases where there has been more harmonious teamwork among and between staff and families. Using multiple proxy perspectives may help identify bias and interpret data properly. We encourage the addressing of methodological issues in the rapidly evolving research on end of life in nursing homes. Further work may address selection bias with prospective recruitment, where this affects associations, and whether bias differs between countries and settings.


This study was supported by a career award for JTS of the Netherlands Organisation for Scientific Research (NWO; Veni grant number 916.66.073), and by ZonMw the Netherlands Organisation for Health Research and Development (grant number 1151.0001), and the VU University Medical Center, EMGO Institute for Health and Care Research, Department of general practice & elderly care medicine, and Department of Public and Occupational Health, Amsterdam, the Netherlands.

Author Disclosure Statement

No competing financial interests exist.


1. van der Steen JT. Dying with dementia: What we know after more than a decade of research. J Alzheimers Dis. 2010;22(1):37–55. [PubMed]
2. Hall S. Kolliakou A. Petkova H. Froggatt K. Higginson IJ. Interventions for improving palliative care for older people living in nursing care homes. Cochrane Database Syst Rev. 2011;16(3):CD007132. [PubMed]
3. Sirken MG. Pifer JW. Brown ML. Survey procedures for supplementing mortality statistics. Am J Public Health Nations Health. 1960;50(11):1753–1764. [PubMed]
4. Borgsteede SD. Deliens L. Francke AL. Stalman WA. Willems DL. van Eijk JT. van der Wal G. Defining the patient population: One of the problems for palliative care research. Palliat Med. 2006;20(2):63–68. [PubMed]
5. Buiting HM. Rietjens JA. Onwuteaka-Philipsen BD. van der Maas PJ. van Delden JJ. van der Heide A. A comparison of physicians' end-of-life decision making for non-western migrants and Dutch natives in the Netherlands. Eur J Public Health. 2008;18(6):681–687. [PubMed]
6. Casarett DJ. Teno J. Higginson I. How should nations measure the quality of end-of-life care for older adults? Recommendations for an international minimum data set. J Am Geriatr Soc. 2006;54(11):1765–1771. [PubMed]
7. Teno JM. Measuring end-of-life care outcomes retrospectively. J Palliat Med. 2005;8(Suppl 1):S42–S49. [PubMed]
8. Addington-Hall J. McPherson C. After-death interviews with surrogates/bereaved family members: Some issues of validity. J Pain Symptom Manage. 2001;22(3):784–790. [PubMed]
9. Cohen LW. van der Steen JT. Reed D. Hodgkinson JC. van Soest-Poortvliet MC. Sloane PD. Zimmerman S. Family perceptions of end-of-life care for long-term care residents with dementia: Differences between the United States and Netherlands. J Am Geriatr Soc. 2012;60(2):316–322. [PubMed]
10. Teno JM. Gozalo PL. Lee IC. Kuo S. Spence C. Connor SR. Casarett DJ. Does hospice improve quality of care for persons dying from dementia? J Am Geriatr Soc. 2011;59(8):1531–1536. [PMC free article] [PubMed]
11. van der Steen JT. Gijsberts MJ. Knol DL. Deliens L. Muller MT. Ratings of symptoms and comfort in dementia patients at the end of life: Comparison of nurses and families. Palliat Med. 2009;23(4):317–324. [PubMed]
12. Arcand M. Monette J. Monette M. Sourial N. Fournier L. Gore B. Bergman H. Educating nursing home staff about the progression of dementia and the comfort care option: Impact on family satisfaction with end-of-life care. J Am Med Dir Assoc. 2009;10(1):50–55. [PubMed]
13. van Soest-Poortvliet MC. van der Steen JT. Zimmerman S. Cohen LW. Klapwijk MS. Bezemer M. Achterberg WP. Knol DL. Ribbe MW. de Vet HC. Psychometric properties of instruments to measure the quality of end-of-life care and dying for long-term care residents with dementia. Qual Life Res. 2012;21(4):671–684. [PMC free article] [PubMed]
14. van der Steen JT. Studying end of life in dementia: Deciding on the optimal research design. Palliat Med. 2012;26(4):448.
15. Conroy S. van der Cammen T. Schols J. Schols J. Van Balen R. Peteroff P. Luxton T. Medical services for older people in nursing homes: Comparing services in England and the Netherlands. J Nutr Health Aging. 2009;13(6):559–563. [PubMed]
16. Hoek JF. Penninx BW. Ligthart GJ. Ribbe MW. Health care for older persons, a country profile: The Netherlands. J Am Geriatr Soc. 2000;48(2):214–217. [PubMed]
17. Volicer L. Hurley AC. Blasi ZV. Scales for evaluation of end-of-life care in dementia. Alzheimer Dis Assoc Disord. 2001;15(4):194–200. [PubMed]
18. Warden V. Hurley AC. Volicer L. Development and psychometric evaluation of the Pain Assessment in Advanced Dementia (PAINAD) scale. J Am Med Dir Assoc. 2003;4(1):9–15. [PubMed]
19. Weiner MF. Martin-Cook K. Svetlik DA. Saine K. Foster B. Fontaine CS. The quality of life in late-stage dementia (QUALID) scale. J Am Med Dir Assoc. 2000;1(3):114–116. [PubMed]
20. Schalkwijk D. Verlare LR. Muller MT. Knol DL. van der Steen JT. Measuring quality of life in nursing home residents with severe dementia: Psychometric properties of the QUALID scale. Tijdschr Gerontol Geriatr. 2009;40(5):184–192. [PubMed]
21. Zwakhalen SM. van der Steen JT. Najim MD. Which score most likely represents pain on the observational PAINAD pain scale for patients with dementia? J Am Med Dir Assoc. 2012;13(4):384–389. [PubMed]
22. Mitchell SL. Kiely DK. Jones RN. Prigerson H. Volicer L. Teno JM. Advanced dementia research in the nursing home: The CASCADE study. Alzheimer Dis Assoc Disord. 2006;20(3):166–175. [PMC free article] [PubMed]
23. Reisberg B. Ferris SH. de Leon MJ. Crook T. The Global Deterioration Scale for assessment of primary degenerative dementia. Am J Psychiatry. 1982;139(9):1136–1139. [PubMed]
24. Morris JN. Fries BE. Mehr DR. Hawes C. Phillips C. Mor V. Lipsitz LA. MDS Cognitive Performance Scale. J Gerontol. 1994;49(4):M174–M182. [PubMed]
25. Mitchell SL. Teno JM. Kiely DK. Shaffer ML. Jones RN. Prigerson HG. Volicer L. Givens JL. Hamel MB. The clinical course of advanced dementia. N Engl J Med. 2009;361(16):1529–1538. [PMC free article] [PubMed]
26. van der Steen JT. Kruse RL. Szafara KL. Mehr DR. van der Wal G. Ribbe MW. D'Agostino RB. Benefits and pitfalls of pooling datasets from comparable observational studies: Combining US and Dutch nursing home studies. Palliat Med. 2008;22(6):750–759. [PubMed]
27. Bailey C. Murphy R. Porock D. Professional tears: Developing emotional intelligence around death and dying in emergency work. J Clin Nurs. 2011;20(23–24):3364–3372. [PubMed]
28. Kahn KL. Liu H. Adams JL. Chen WP. Tisnado DM. Carlisle DM. Hays RD. Mangione CM. Damberg CL. Methodological challenges associated with patient responses to follow-up longitudinal surveys regarding quality of care. Health Serv Res. 2003;38(6 Pt 1):1579–1598. [PMC free article] [PubMed]
29. Johnson TP. Wislar JS. Response rates and nonresponse errors in surveys. JAMA. 2012;307(17):1805–1806. [PubMed]
30. Houttekier D. Cohen J. Bilsen J. Addington-Hall J. Onwuteaka-Philipsen BD. Deliens L. Place of death of older persons with dementia. A study in five European countries. J Am Geriatr Soc. 2010;58(4):751–756. [PubMed]

Articles from Journal of Palliative Medicine are provided here courtesy of Mary Ann Liebert, Inc.