Search tips
Search criteria 


Logo of amjepidLink to Publisher's site
Am J Epidemiol. 2013 June 15; 177(12): 1452–1458.
Published online 2013 April 7. doi:  10.1093/aje/kws424
PMCID: PMC3676153

Responders Versus Nonresponders in a Dementia Study of the Oldest Old

The 90+ Study


Because of difficulties in finding, recruiting, and diagnosing dementia in the oldest old (ages ≥90 years), most incidence studies include few very elderly persons, and little is known about the characteristics of those who refuse participation. In a California longitudinal study of dementia and aging (The 90+ Study, 2003–2011), we compared nonresponders with responders with regard to information collected 20 years earlier and the impression of dementia as determined during telephone recruitment. Of 1,815 eligible subjects, 1,514 (83%) joined the study, 182 refused, and 119 could not be contacted. Responders did not differ from nonresponders by sex or previously collected medical history or lifestyle behaviors. Recruiters' impressions of dementia were similar in responders and nonresponders who refused (35% and 38%), and among responders, impressions of dementia showed high positive predictive value (95%) but low sensitivity (51%) for a diagnosis of dementia made during the study. Although epidemiologic studies among the very old have the potential for significant nonresponse bias due to a high proportion of frail, ill, and cognitively impaired persons, strategies can improve response rates to over 80%. Classifying nonresponders on cognitive ability at recruitment, though crude, will give some idea of the selective bias in dementia prevalence and incidence estimates introduced by nonresponse due to cognitive status.

Keywords: aged, bias, cohort studies, dementia, epidemiologic methods, refusal to participate

With longer life spans and high incidence rates of late-life dementia, dementia is becoming a major public health problem. Over the next few decades, the global prevalence of dementia will increase severalfold, to more than 81 million by the year 2040 (1).

Because of difficulties in finding, recruiting, and diagnosing dementia in the oldest old (persons aged ≥90 years), most studies do not have sufficient numbers of participants to estimate prevalence and incidence rates by age and sex above age 90 years (211). Additionally, nonresponse rates are often substantial, and little is known about the characteristics of persons who refuse participation in epidemiologic studies of dementia in the elderly. Concerns include decreasing response rates with age and the fact that the cognitive status of nonresponders is poorer than that of responders. Nonresponse can lead to bias in estimates of the prevalence and incidence of cognitive impairment in the oldest old and in patterns of change over time.

In light of these concerns, we characterized the nonresponders and compared them with participants in a large cohort study of the oldest old. We also attempted to determine whether the prevalence estimate of dementia from the cohort reflected the prevalence rate of the referent population.


The Leisure World Cohort Study was established in the early 1980s when 13,978 residents of a California retirement community (Leisure World Laguna Hills) completed a postal health survey. Residents were recruited in 4 waves: 1) persons who owned homes in Leisure World on June 1, 1981, and new residents who had moved into the community and were living there on 2) June 1, 1982, 3) June 1, 1983, and 4) October 1, 1985. The baseline survey asked for information on demographic characteristics (sex, marital status, height, weight), basic medical history (high blood pressure, heart attack, angina, stroke, diabetes, rheumatoid arthritis, glaucoma), and several personal habits (smoking, alcohol consumption, caffeine intake, vitamin supplement use, physical activity). Details on the study methods and on these variables have been presented in previous reports (1215).

Follow-up of the cohort is maintained through periodic resurveys (1983, 1985, 1992, 1998), review of local hospital discharge data (1981–2004), and determination of vital status by searches of governmental and commercial death indexes and ascertainment of death certificates.

The 90+ Study, a longitudinal study of aging and dementia in the oldest old, was established in 2003 (16, 17). Leisure World Cohort members aged 90 years or older were asked to join if they were still living on January 1, 2003; on January 1, 2008; and on January 1 thereafter. All eligible Leisure World Cohort members were mailed a brochure describing The 90+ Study, a letter inviting them to join, and a postage-paid form to return indicating a convenient time for us to contact them to schedule an appointment. About a month later, we telephoned those who had not returned the form to provide them with additional information, answer questions about the study, and encourage participation. If eligible individuals were unable or unwilling to be seen in person, we offered them the opportunity to be a telephone participant. If they were unwilling or unable to participate by telephone, we then contacted next of kin to obtain information so they could be included as an informant participant.

Participants in The 90+ Study were asked to undergo an in-person evaluation which included a neurological examination by a trained physician or nurse practitioner and a neuropsychological test battery that included the Mini-Mental State Examination (18). The structured neurological examination included mental status testing covering multiple domains and selected items regarding function. The neurological examiner used these to assign a diagnosis of dementia according to Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition, criteria for dementia (19). Some participants' poor health, frailty, or disability did not allow a full in-person evaluation, and others did not want a full in-person evaluation. Information about such participants was obtained by telephone or from informants. Participants who were evaluated by telephone completed the short version of the Cognitive Abilities Screening Instrument (20). For participants evaluated through informants, the Dementia Questionnaire (21) was completed over the telephone. The Dementia Questionnaire was also completed for participants who scored less than 25 on the Mini-Mental State Examination or less than 27 on the Cognitive Abilities Screening Instrument, and for all participants at the time of death. All participants (or their informants) completed a questionnaire that included demographic factors, medical history, and medication use. In addition, informants of all participants were asked about the participant's cognitive status (22) and functional abilities (23, 24), using a mailed questionnaire.

At a consensus conference, all available information about a subject was utilized to determine the presence of dementia. In addition to subject evaluations, we used laboratory tests, medical records, including computed tomography or magnetic resonance imaging scans of the head, other relevant studies, and Dementia Questionnaires.

When we telephoned potential subjects or their next of kin to provide them with information about the study, the recruiters (primarily B.D. and M.H.) made a determination from the conversation as to whether the subject was cognitively normal or cognitively impaired. If the participant had trouble tracking the conversation, made inappropriate responses, did not know where he/she lived, or did not recall a recent earlier conversation with us, or if the participant or an informant commented on his/her impaired memory or dementia, the subject was categorized as cognitively impaired or demented.

Chi-square tests were used for comparison of categorical variables, and t tests were used for testing differences between mean values of continuous variables. Statistical analyses were performed using SAS, version 9.2 (SAS Institute Inc., Cary, North Carolina).

The institutional review boards of the University of Southern California and the University of California, Irvine, approved the study.


Of the 1,815 Leisure World Cohort members who were eligible for The 90+ Study, 1,514 (83%) joined the study. Of 301 nonresponders, 182 (60%) refused and 119 (40%) could not be contacted by mail or phone (Figure 1).

Figure 1.
Recruitment of the Leisure World Cohort into The 90+ Study, California, 2003–2011.

Table 1 presents the characteristics of the participants in the 1980s, by response category. Responders did not differ significantly from nonresponders with regard to any of these variables except age. Nonresponders were slightly younger than responders (92.3 years vs. 92.7 years; P = 0.02). Persons we could not contact were more likely than refusals to be married (44% vs. 26%; P = 0.001) and were less likely to be diabetic (0% vs. 3%; P < 0.05).

Table 1.
Characteristics of the Leisure World Cohort in the 1980s, by Response to Recruitment for The 90+ Study, California, 2003–2011

Table 2 shows the previous follow-up rates and the proportions of participants with known dementia by follow-up survey, hospital record, or death certificate. Although responders did not differ from nonresponders with regard to completion of the first follow-up survey (in 1983), they were more likely to complete the 1985, 1992, and 1998 surveys (P < 0.0001). Compared with refusals, persons we were unable to contact for The 90+ Study had higher rates of dementia according to the 1992 and 1998 surveys and death certificates, though only the difference for death records was statistically significant.

Table 2.
Characteristics of the Leisure World Cohort Between 1981 and 2012, by Response to Recruitment for The 90+ Study, California, 2003–2011

Table 2 also shows that the recruiters' impressions of dementia did not differ between responders and nonresponders who refused. Thirty-five percent and 38%, respectively, were classified as cognitively impaired or demented at recruitment.

Table 3 compares the recruiters' impression of cognitive impairment with that determined by the neurological examiner from an in-person examination and by consensus conference using all available information. Cross-tabulation of the recruiters' impression with both the in-person examination and the consensus conference yielded kappa statistics (proportion of agreement above chance) showing low-to-fair agreement (κ = 0.25 and κ = 0.29, respectively). The recruiters had a high positive predictive value; persons whom the recruiters identified as demented tended to be so (95%–96%). However, the sensitivity was low (51%–53%); many impaired/demented persons were identified as cognitively normal by the recruiters.

Table 3.
Distribution of Participants by Impression of Cognitive Ability Upon Recruitment Versus Diagnosis of Cognitive Ability Upon Neurological Examination or Consensus Conference, The 90+ Study, California, 2003–2011


The 90+ Study had a high response rate (83%) in a very elderly group. Medical illness, frailty, fatigue, and sensory losses (particularly visual and hearing losses) are common in the oldest old and can affect the ability to recruit and assess these subjects. We tailored recruitment strategies to meet the needs and concerns of the persons of interest. Our research office is located in the center of the Leisure World community, near the homes of the majority of eligible subjects. Participants agreeing to be seen in our research office were offered free transportation. For frail persons and others who were unable (including those living outside the community) or unwilling to come into the research center, staff traveled to the subject's residence. Home visits greatly enhanced enrollment and probably reduced the rate of refusal due to health problems. We also chose procedures, with multimodality presentation, designed to minimize the impact of sensory losses. We used test stimuli with large bold print and provided hearing aids when testing participants. Frequent breaks and extending sessions over multiple visits were employed to minimize fatigue. We also used telephone procedures for participants who did not want an in-person visit and informant interviews for those who were unable to participate or recently deceased. Many informants also authorized us to visit their relatives for a neurological examination. Including these persons enabled us to gather information on those who were too physically frail or cognitively impaired to participate fully themselves. All of these approaches helped to increase our participation rate.

Although response rates tend to decrease with age or show a curvilinear association with age in large surveys (2529), participation rates tend to be higher when the surveys are designed specifically for the elderly (persons aged ≥65 years) (3034). The National Long-Term Care Survey (30) and the Longitudinal Study on Aging (31) had participation rates of over 90%. Other studies had response rates of 70%–85% (3234). Although some dementia studies of the oldest old have participation rates greater than 80% (3), others have lower rates (4, 7, 10).

Within the limited age eligibility of The 90+ Study, we noted only a weak relation between age and response status. Potential subjects—responders and nonresponders alike—were alive on their cohort recruitment date and were aged ≥90 years. Hence, all of them can be classified as “supersurvivors.” We expected that nonresponders, especially those we were unable to contact, would be older and in poorer health. In contrast, nonresponders were slightly younger on average than responders (92.3 years vs. 92.7 years), and those we could not contact were less likely to report diabetes on the original questionnaire than refusals (0% vs. 3%). We also found that persons who refused differed from those we were unable to contact with regard to marital status (26% vs. 44% were married in the 1980s). We have no explanation for these findings.

We found no evidence of a sex difference between response groups. In the community sample of the Saskatchewan Health Status Survey of the Elderly, response among the very elderly (ages ≥85 years) was disproportionally male (35), as it was in a sample of persons aged ≥70 years drawn from Bergen, Norway (29). In contrast, the World Health Organization MONICA Project (Monitoring of Trends and Determinants in Cardiovascular Disease) found that women were more likely to participate (36).

No difference among groups was observed for the other variables on which data were collected as part of the Leisure World Cohort Study in the 1980s. Because our population was relatively homogeneous with respect to race (white), socioeconomic class (moderately affluent), and education (highly educated), we could not evaluate patterns of response according to these variables. Not unexpectedly, continued active participation in the Leisure World Cohort Study by completion of follow-up surveys was higher in responders than in nonresponders.

Although the effects of nonresponse can often be evaluated by only a few demographic characteristics such as sex and age, several studies have found health differences between responders and nonresponders. Some studies of aging people have reported that, compared with responders, nonresponders have poorer self-rated health and mobility (33, 37) and higher rates of psychiatric illness, heart attack, stroke, and diabetes (38). In some studies, nonresponders more often used medical services and were more frequently admitted to hospitals (35), but not in others (29). In our study, no large differences in hospitalization (up to 2004) were found among responders (72%), nonresponders (68%), or refusals (70%) and the no-contact group (66%).

The presence of similar distributions of demographic variables in responders and nonresponders does not preclude the possibility of an unequal distribution of cognitive status (38). Akhtar (39), performing a medical examination on all but 2 persons who refused an interview (n = 50), found more dementia and anxiety and less depression in the nonparticipating group, although no difference was statistically significant. Similarly, nonresponse was associated with poorer cognitive performance in a study of persons aged ≥75 years living at home in a small city in Quebec, Canada (40); in the Australian Longitudinal Study of Aging, comprising persons aged ≥70 years (37, 41); and among ≥75-year-old heads of households in a rural community in Utah (42). In a Dutch population aged ≥65 years, reason for nonresponse was associated with a general practitioner's judgment about the subject's cognitive status, with persons who were ill or deceased being more likely to be demented than refusals or those who were not traceable (43). Although we could not medically rate cognitive ability in our group of nonresponders, our recruiters' impressions of dementia gave some reassurance that the proportions did not differ greatly between responders and refusals (35% and 38%, respectively). It does appear, however, that persons we were unable to contact were more likely to be demented than those who refused (e.g., 37% vs. 22% by death records; P = 0.03).

The proportion of persons identified with dementia through follow-up questionnaires, hospital records, and death records was undoubtedly an underestimate of the true prevalence. Although nonresponders to The 90+ Study were less likely to have completed previous follow-up surveys, among those with completed questionnaires the proportions identified with dementia were similar in responders and nonresponders, as were the proportions identified by means of hospital records and death records. The only statistical difference in dementia proportions was the difference on death records between refusals and persons with no contact (22% vs. 37%; P = 0.03). Because of missing follow-up questionnaires in nonresponders, the higher dementia proportion on death records in persons with no contact, and the slightly greater proportion believed to be demented by recruiters in refusals, the rate of prevalent dementia was probably greater in nonresponders than in responders. We previously reported an overall prevalence of dementia in persons aged ≥90 years of 41% (95% confidence interval: 38, 44), with dementia prevalence being higher in participants not seen in person than in those diagnosed through in-person procedures (16). If the dementia rate in nonresponders were twice that of responders, the overall prevalence rate would be 48%. If the prevalence were larger only in persons with no contact, then the overall prevalence rate would be 44%.

Epidemiologic studies among the very old are laden with difficulties and the potential for significant nonresponse bias. Although the effect of nonresponse is relevant to any health survey, it may be especially important in studies of the elderly with a high proportion of frail, ill, and cognitively impaired persons. If nonresponders differ from responders with regard to these factors, bias in disease estimates may occur. Strategies for improving the response rates of older adults are essential (44). These may include modification of interview length, an extended recruitment period, home visits, provision of transportation, and use of proxies. Many seniors will initially decline for legitimate, but acute and transient, reasons and may later become participants. Empathetic recruiters and interviewers with substantial training who understand seniors' concerns and can explain to seniors how the study needs their specific input are essential. Recruiters must emphasize the importance of each individual in the study, whether extremely healthy, extremely unhealthy, or somewhere in between. Knowing more about nonresponders will help us develop better recruitment strategies and improve participation rates among the oldest old in the future.


Author affiliation: Department of Neurology, School of Medicine, University of California, Irvine, Irvine, California (Annlia Paganini-Hill, Beverly Ducey, Marian Hawk).

This work was supported by the National Institutes of Health (grants R01CA32197 and RO1AG20155), the Earl Carroll Trust Fund, and Wyeth-Ayerst Laboratories.

Conflict of interest: none declared.


1. Ferri CP, Prince M, Brayne C, et al. Global prevalence of dementia: a Delphi consensus study. Lancet. 2005;366(9503):2112–2117. [PMC free article] [PubMed]
2. Hagnell O, Ojesjo L, Rorsman B. Incidence of dementia in the Lundby study. Neuroepidemiology. 1992;11(suppl 1):61–66. [PubMed]
3. Fichter MM, Schroppel H, Meller I. Incidence of dementia in a Munich community sample of the oldest old. Eur Arch Psychiatry Clin Neurosci. 1996;246(6):320–328. [PubMed]
4. Von Strauss E, Viitanen M, De Ronchi D, et al. Aging and the occurrence of dementia. Findings from a population-based cohort with a large sample of nonagenarians. Arch Neurol. 1999;56(5):587–592. [PubMed]
5. The Canadian Study of Health and Aging Working Group. The incidence of dementia in Canada. Neurology. 2000;55(1):66–73. [PubMed]
6. Ruitenberg A, Ott A, van Swieten JC, et al. Incidence of dementia: does gender make a difference? Neurobiol Aging. 2001;22(4):575–580. [PubMed]
7. Riedel-Heller SG, Busse A, Aurich C, et al. Prevalence of dementia according to DSM-III-R and ICD-10: results of the Leipzig Longitudinal Study of the Aged (LEILA75+) Part 1. Br J Psychiatry. 2001;179:250–254. [PubMed]
8. Miech RA, Breitner JCS, Zandi PP, et al. Incidence of AD may decline in the early 90's for men, later for women. The Cache County Study. Neurology. 2002;58(2):209–218. [PubMed]
9. Edland SD, Rocca WA, Petersen RC, et al. Dementia and Alzheimer disease incidence rates do not vary by sex in Rochester, Minn. Arch Neurol. 2002;59(10):1589–1593. [PubMed]
10. Börjesson-Hanson A, Edin E, Gislason T, et al. The prevalence of dementia in 95 year olds. Neurology. 2004;63(12):2436–2438. [PubMed]
11. Hall CB, Verghese J, Sliwinski M, et al. Dementia incidence may increase more slowly after age 90: results from the Bronx Aging Study. Neurology. 2005;65(6):882–886. [PubMed]
12. Paganini-Hill A, Ross RK, Henderson BE. Prevalence of chronic disease and health practices in a retirement community. J Chronic Dis. 1986;39(9):699–707. [PubMed]
13. Paganini-Hill A, Chao A, Ross RK, et al. Exercise and other factors in the prevention of hip fracture: the Leisure World Study. Epidemiology. 1991;2(1):16–25. [PubMed]
14. Paganini-Hill A, Kawas CH, Corrada M. Type of alcohol consumed, changes in intake over time and mortality: the Leisure World Cohort Study. Age Ageing. 2007;36(2):203–209. [PMC free article] [PubMed]
15. Paganini-Hill A, Kawas CH, Corrada M. Non-alcoholic beverage and caffeine consumption and mortality: the Leisure World Cohort Study. Prev Med. 2007;44(4):305–310. [PMC free article] [PubMed]
16. Corrada MM, Brookmeyer R, Berlau D, et al. Prevalence of dementia after age 90: results from the 90+ Study. Neurology. 2008;71(5):337–343. [PubMed]
17. Corrada MM, Brookmeyer R, Paganini-Hill A, et al. Dementia incidence continues to increase with age in the oldest old. The 90+ Study. Ann Neurol. 2010;67(1):114–121. [PMC free article] [PubMed]
18. Folstein MF, Folstein SE, McHugh PR. Mini-mental state: a practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res. 1975;12(3):189–198. [PubMed]
19. American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition. Washington, DC: American Psychiatric Association; 1994.
20. Teng EL, Hasegawa K, Homma A, et al. The Cognitive Abilities Screening Instrument (CASI): a practical test for cross-cultural epidemiological studies of dementia. Int Psychogeriatr. 1994;6(1):45–58. [PubMed]
21. Silverman JM, Keefe RS, Mohs RC, et al. A study of the reliability of the family history method in genetic studies of Alzheimer's disease. Alzheimer Dis Assoc Disord. 1989;3(4):218–223. [PubMed]
22. Clark CM, Ewbank DC. Performance of the dementia severity rating scale: a caregiver questionnaire for rating severity in Alzheimer disease. Alzheimer Dis Assoc Disord. 1996;10(1):31–39. [PubMed]
23. Pfeffer RI, Kurosaki TT, Harrah CH, et al. Measurement of functional activities in older adults in the community. J Gerontol. 1982;37(3):323–329. [PubMed]
24. Katz S, Ford AB, Moskowitz RW, et al. Studies of illness in the aged. The Index of ADL: a standardized measure of biological and psychosocial function. JAMA. 1963;185(12):914–919. [PubMed]
25. Herzog AF, Rogers WL. The use of survey method in research on older Americans. In: Wallace RB, Woolson RF, editors. The Epidemiologic Study of the Elderly. New York, NY: Oxford University Press;; 1991. pp. 60–90.
26. Nummela O, Sulander T, Helakorpi S, et al. Register-based data indicated nonparticipation bias in a health study among aging people. J Clin Epidemiol. 2011;64(12):1418–1425. [PubMed]
27. Jay GM, Liang J, Liu X, et al. Patterns of nonresponse in a national survey of elderly Japanese. J Gerontol. 1993;48(3):S143–S152. [PubMed]
28. Helliwell B, Aylesworth R, McDowell I, et al. Correlates of nonparticipation in the Canadian Study of Health and Aging. Int Psychogeriatr. 2001;13(suppl 1):49–56. [PubMed]
29. Hardie JA, Bakke PS, Mørkve O. Non-response bias in a postal questionnaire survey on respiratory health in the old and very old. Scand J Public Health. 2003;31(6):411–417. [PubMed]
30. Manton KG. A longitudinal study of functional change and mortality in the United States. J Gerontol. 1988;43(5):S153–S161. [PubMed]
31. Kovar MG, Fitti JE, Chyba MM. The Longitudinal Study of Aging: 1984–90. Hyattsville, MD: National Center for Health Statistics; 1992. (Vital and Health Statistics, series 1, no. 28). (DHHS publication no. PHS 92-1304) [PubMed]
32. Cornoni-Huntly JC, Foley DJ, White LR, et al. Epidemiology of disability in the oldest old: methodologic issues and preliminary findings. Milbank Mem Fund Q/Health Soc. 1985;63(2):350–376. [PubMed]
33. Hoeymans N, Feskens EJ, van den Bos GAM, et al. Non-response bias in a study of cardiovascular diseases, functional status and self-rated health among elderly men. Age Ageing. 1998;27(1):35–40. [PubMed]
34. Branch LG, Katz S, Kniepmann K, et al. A prospective study of functional status among community elders. Am J Public Health. 1984;74(3):266–268. [PubMed]
35. Rockwood K, Stolee P, Roberston D, et al. Response bias in a health status survey of elderly people. Age Ageing. 1989;18(3):177–182. [PubMed]
36. Wolf HK, Kuulasmaa K, Tolonen H, et al. Participation Rates, Quality of Sampling Frames and Sampling Fractions in the MONICA Surveys. Geneva, Switzerland: World Health Organization; 1999. WHO MONICA Project e-publications (ISSN 2242-1246), no. 5). (Accessed October 16, 2012).
37. Anstey KJ, Luszcz MA. Selective non-response to clinical assessment in the Longitudinal Study of Aging: implications for estimating population levels of cognitive function and dementia. Int J Geriatr Psychiatry. 2002;17(8):704–709. [PubMed]
38. Launer LJ, Wind AW, Derg DJH. Nonresponse pattern and bias in a community-based cross-sectional study of cognitive functioning among the elderly. Am J Epidemiol. 1994;139(8):803–812. [PubMed]
39. Akhtar AJ. Refusal to participate in a survey of the elderly. Gerontol Clin (Basel) 1972;14(4):205–211. [PubMed]
40. Hébert R, Bravo G, Korner-Bitensky N, et al. Refusal and information bias associated with postal questionnaires and face-to-face interviews in very elderly subjects. J Clin Epidemiol. 1996;49(3):373–381. [PubMed]
41. Jacomb P, Jorm AF, Korten AI, et al. Predictors of refusal to participate: a longitudinal health survey of the elderly in Australia. BMC Public Health. 2002;2:4. [PMC free article] [PubMed]
42. Norton MC, Breitner JC, Welsh KA, et al. Characteristics of nonresponders in a community survey of the elderly. J Am Geriatr Soc. 1994;42(12):1252–1256. [PubMed]
43. Boersma F, Eefsting JA, van den Brink W, et al. Characteristics of non-responders and the impact of non-response on prevalence estimates of dementia. Int J Epidemiol. 1997;26(5):1055–1062. [PubMed]
44. Williams B, Irvine L, McGinnis AR, et al. When “no” might not quite mean “no”; the importance of informed and meaningful non-consent: results from a survey of individuals refusing participation in a health-related research project. BMC Health Services Res. 2007;7:59. [PMC free article] [PubMed]

Articles from American Journal of Epidemiology are provided here courtesy of Oxford University Press