|Home | About | Journals | Submit | Contact Us | Français|
This study examined the operating characteristics of two-stage case finding to identify memory impairment and very mild dementia.
Primary care patients underwent two-stage testing and a subsequent diagnostic assessment to assess outcomes. Patients who screen positive for subjective cognitive decline on the Informant Questionnaire on Cognitive Decline in the Elderly undergo memory testing with the Free and Cued Selective Reminding Test with Immediate Recall. Outcomes were determined without access to these data. A split-half design with discovery and confirmatory samples was used.
One hundred seventeen of 563 (21%) patients had dementia and 68 (12%) had memory impairment but not dementia. Operating characteristics were similar in the discovery and confirmatory samples. In the pooled sample, combined, patients with memory impairment or dementia were identified with good sensitivity (72%) and high specificity (90%). Differences in ethnicity, educational level, or age (≤75, >75) did not affect classification accuracy.
Two-stage screening facilitates the efficient identification of older adults with memory impairment or dementia.
Policy recommendations for routine cognitive assessment in older adults are variable and evolving. The U.S. Preventive Services Task Force did not recommend cognitive screening or case finding in asymptomatic patients because the aggregate benefits have not been demonstrated to outweigh the aggregate costs and risks . The Centers for Medicare and Medicaid Services recommended cognitive assessment as part of the Annual Wellness Visit for older adults but did not specify the testing approach . In response, the Alzheimer's Association recommended assessment tools that include brief tests of memory and cognition as well as informant interviews . In the context of a screening or case-finding program, patients who screen positive for cognitive impairment are referred for a more detailed evaluation at a subsequent primary care visit or to a clinician with expertise in dementia.
As part of the broad public health effort to reduce the burden of cognitive disorders of late life, many groups have assessed potential screening and case-finding tools in primary care or population settings including in person mental status and brief memory tests , interviews , brief cognitive batteries , informant questionnaires , and two-stage assessment strategies , . Using our two-stage screening strategy, eligible subjects receive a brief, highly sensitive initial screen. Those who screen positive are followed up with a second-stage test to increase specificity . This strategy facilitates time-efficient screening at the time of a routine clinic visit . One strategy worked well at distinguishing patients with dementia from those without dementia in two demographically different primary care clinics located in the Bronx, NY , . The Informant Questionnaire on Cognitive Decline in the Elderly (IQCODE)  was administered in the first stage to identify patients who then undergo second-stage memory testing with the picture version of the Free and Cued Selective Reminding Test with Immediate Recall (pFCSRT + IR), which controls the learning conditions to identify memory impairment and dementia . The combined strategy provided high specificity (91%) and good sensitivity (77%) in identifying very mild dementia among black and white patients  and among younger and less-educated black and Latino patients .
Given the increased interest in identifying patients in predementia stages of Alzheimer's disease (AD) , , we sought to determine how well the strategy distinguished patients with no memory impairment (NMI) from patients with memory impairment but no dementia (MIND) and patients with dementia (DEM) at cross-section. Evidence is accumulating that in persons free of dementia, subjective cognitive decline (SCD) is predictive of future cognitive decline and incident AD dementia . In the first stage, informants complete the short form of the IQCODE . Persons who have SCD undergo second-stage memory testing with the pFCSRT + IR . We selected an IQCODE cut score based on previous studies to optimize sensitivity. The free recall cut score on the pFCSRT + IR to identify memory impairment was selected to balance sensitivity and specificity.
Herein, we combined the two Bronx-based primary care cohorts to create a heterogeneous patient sample with memory impairment or very early dementia that was demographically and educationally diverse and large enough to test the generalizability of the proposed screening strategy. We used a split-half design and derived empirical cut scores in a discovery sample and applied them to a confirmatory sample. Secondary goals were to determine classification accuracy in patients with low versus high levels of education, for Latino and non-Latino blacks as well as white patients, and for patients younger or older than 75 years.
The same screening and case-finding methods were used in two primary care settings in Bronx, NY. The IQCODE and the pFCSRT + IR were administered to all patients and comprised the screening assessment. The purpose of this analysis was to identify cutoff scores on the two instruments in tandem to optimize diagnosis of dementia in future two-stage screening programs. The diagnostic battery that consisted of a comprehensive neuropsychological evaluation and informant interviews described previously ,  were administered at a second visit. Experienced bilingual examiners approached eligible patients at their scheduled appointment, recruited interested patients, obtained written consent, and conducted the evaluation at the patient's convenience, before or after their physician visit. Testing was supervised by the same neuropsychologist (E.G.). Without knowledge of the pFCSRT + IR or IQCODE results, two raters (E.G. and A.E.) independently reviewed scores from the diagnostic battery and informant responses to determine the presence versus absence of memory impairment and dementia.
The study participants from two clinics associated with the Einstein College of Medicine, the Geriatrics Ambulatory Practice (GAP), an academic geriatrics practice, and from the Jacobi Adult Medicine Clinic (JAM). Eligible participants were aged 65 years or older, had adequate vision and hearing to complete the neuropsychological tests, and spoke English or Spanish. Each participant provided the name of a family member or friend who knew them for at least 5 years. GAP patients who scored below 19 on the Mini–Mental State Examination (MMSE)  were excluded as were JAM patients with a medical diagnosis of dementia at the baseline visit. Study participants gave informed consent using procedures approved by the institutional review boards at the Albert Einstein College of Medicine and Jacobi Medical Center.
A consensus diagnosis for each participant was established by the neuropsychologist (E.G.) and geriatrician (A.E.) using Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV) criteria for dementia  without input from the patient's primary care provider or knowledge of pFCSRT + IR performance or IQCODE responses. A report was generated for each patient containing informant's responses to the Clinical Dementia Rating (CDR) interview  augmented by their responses to the Alzheimer's Disease Cooperative Study Activities of Daily Living Scale  and the patient's tests scores from the diagnostic battery along with the 5th, 10th, and 50th percentile scores of the patients without dementia. Before the consensus conference, E.G. and A.E. reviewed the report, made an independent determination of the patient's diagnostic status as having NMI, MIND, or DEM. They also rated the patient's cognitive performance and activities of daily living using the CDR scale . Disagreement on DSM-IV criteria or CDR box scores for any patient was resolved at the consensus conference.
The IQCODE assesses 10-year change in memory and cognition as rated by a family member or friend . It is one of the most widely used informant interviews , . The short form, which includes 16 of the original 26 items and operates as well as the long form to distinguish between elderly with and without dementia , , is used here to identify SCD. A five-point scale indicates the degree of change in daily activities (e.g., remembering recent conversations and events, making decisions); a score of three indicates no change. A 5-year time frame was used which is long enough to observe functional decline but avoids the difficulty of finding informants who have 10 years of contact with the participant . Higher scores indicate greater cognitive decline.
Patients who fail the first stage undergo episodic memory testing with the pFCSRT + IR , impairment on which defines the core clinical phenotype for prodromal AD in the International Working Group criteria . It begins with a study phase in which participants search a card containing four pictures (e.g., grapes) for an item that goes with a unique category cue (e.g., fruit). After all four items are identified, the card is removed and immediate cued recall of the four items is tested (e.g., what was the fruit?). The study phase is continued for the next group of four items until all 16 items have been identified and retrieved in Immediate Recall. There are three test trials, each consisting of free recall followed by cued recall for items not retrieved by free recall. Items not retrieved by cued recall are re-presented as reminders. Each separate trial is followed by 20 seconds of interference. Total recall is the sum of free and cued recall. A Spanish version was constructed using standard back-translation methods . The dependent measure in this study was the sum of free recall over the three test trials for a maximum of 48 items.
Demographic and performance characteristics of the groups within and between clinics were compared using analyses of variance and t-tests for continuous variables and chi-square tests for categorical variables. Two equally sized stratified random samples (discovery and confirmatory) were constructed from the combined cohort balanced for number of patients in each patient group and number from each clinic. The operating characteristics of the IQCODE (sensitivity, specificity, positive predictive value [PPV], negative predictive value [NPV], and efficiency) in the discovery sample were estimated at clinically appropriate cut scores. The efficiency of the first stage was defined by the proportion of patients who undergo second-stage testing. The IQCODE cut score that maximized sensitivity for subjective cognitive decline while maintaining acceptable efficiency then determined which patients would be used to estimate the operating characteristics of free recall from the pFCSRT + IR at clinically appropriate cut scores. The IQCODE and pFCSRT + IR cut scores at the first and second stage, respectively, that maximized sensitivity while maintaining high specificity (≥90%) in the discovery sample were applied to the confirmatory sample. Classification accuracy between the discovery and confirmatory samples and between educational level (<8 years, 8–11 years, ≥12 years), race/ethnicity (Latino, white, black), and age (≤75, >75) were compared using the Pearson's chi-square test.
Table 1 shows the demographic characteristics and test scores of the GAP and JAM clinics separately and combined. The JAM cohort was largely black and Latino and was younger and less well educated than the GAP cohort, which was largely comprised of black and white patients. The combined sample was 76.4 years old at screening, 79% female, 27.2% were Hispanic and 41.4% non-Hispanic black. Of the 563 patients, 378 (67%) had NMI, 68 (12%) had MIND, and 117 (21%) had dementia. Despite the demographic disparity, the clinics did not differ in the proportion of patients in each group.
Of the patients who met DSM-IV criteria for dementia, the majority (57%) had very mild dementia (CDR = 0.5). Overall, patients with dementia were older (P = .0001) and had fewer years of education (P = .001) but did not differ from patients without dementia by gender (P = .08). SCD was more marked in the DEM than in MIND group, which was more marked than in the NMI group (3.8 vs. 3.3 vs. 3.1, P < .01). Free recall was lower in the DEM than in MIND group, which was lower than in the NMI group (18.1 vs. 23.6 vs. 30.8, P < .01). Similarly, total recall was lower in the DEM than in MIND group, which was lower than in the NMI group (40.0 vs. 45.5 vs. 47.5, P < .01).
We examined the cross-sectional operating characteristics of the IQCODE scores for distinguishing the NMI group from the MIND and dementia groups. Table 2 shows the sensitivity, specificity, PPV, NPV, and efficiency of the IQCODE at various cutoffs for the discovery sample (top) and the confirmatory sample (bottom). As efficiency improves, sensitivity decreases and specificity increases. The operating characteristics of the two samples were quite similar as shown by the overlap in confidence intervals. As predicted, the cut score of ≥3.2 had good sensitivity (80%) and specificity (82%) in distinguishing the NMI group from the MIND and DEM groups while maintaining acceptable efficiency (38% of the discovery sample screened positive compared to 39% in the confirmatory sample). The classification accuracy of first-stage testing did not differ between the discovery and confirmatory samples (82% vs. 81%, P < .65) at the cut score of ≥3.2.
Among patients who failed the IQCODE, we examined the discriminative validity of free recall for distinguishing NMI from MIND and DEM. Table 3 shows the sensitivity, specificity, PPV, and NPV of second-stage testing applied to patients who scored positive at the first stage (IQCODE ≥3.2) at clinically appropriate free recall cutoffs in the discovery sample (top) and confirmatory sample (bottom). The operating characteristics of free recall among patients who screened positive at stage 1 were very similar between samples as shown by the overlap in confidence intervals. The cut score of ≤30 maximized sensitivity (73%) while maintaining the high level of specificity needed in primary care screening (≥90%). The classification accuracy of second-stage testing did not differ between the discovery and confirmatory samples at the cut score of ≤30 (85% vs. 84%, P < .63). Thus, the samples were combined in the remaining analyses.
Table 4 shows the sensitivity and specificity values for two-stage case finding in the combined samples (stage 1: IQCODE ≥3.2; stage 2: FR ≤30), by ethnicity/race, and by education level. The overall sensitivity was 72%, specificity was 90%, and classification accuracy was 87.9%. Accuracy of correctly classifying patients was not affected by race (P < .18), education (P < 0.45), or age (P < .26).
Two-stage case finding was accomplished by identifying patients with SCD in the first stage using informant responses on the short IQCODE who then undergo memory testing with the pFCSRT + IR in the second stage to identify memory impairment and dementia. For purposes of the study, all participants received the pFCSRT + IR in the second stage and everyone underwent an independent diagnostic assessment. The strategy was applied in two Bronx primary-care clinics serving an older urban population of Latinos, non-Latino blacks, and whites. Thirty-three percent of the 563 patients met research criteria for memory impairment or dementia indicating a significant burden of unrecognized cognitive impairment despite excluding patients with a medical diagnosis of dementia in one cohort and those with a MMSE score of <19 in the other cohort.
The operating characteristics of the IQCODE to identify patients with SCD in the discovery sample overlapped those in the confirmatory sample. Thirty-eight percent of the patients screened positive in the first stage of the combined sample using a cut score of ≥3.2, originally chosen because of its high sensitivity compared to the recommended score of ≥3.4 (82% vs. 56%). The 10-point decrease in specificity (92%–82%) was offset by second-stage testing. Using a free recall cut score of ≤30 on the pFCSRT + IR, sensitivity was 72% and specificity was 90%. The operating characteristics of our case-finding strategy did not differ between the discovery and confirmatory samples. Differences in ethnicity, educational level, and age in the combined sample did not significantly affect the accuracy of correctly classifying patients with memory impairment or dementia and patients with no memory impairment.
According to a recent meta-analysis , the IQCODE is especially useful for ruling out those patients with no evidence of cognitive decline, making it ideal for identifying patients with SCD. Defining SCD through informant report of cognitive change departs from the conceptual framework proposed for research on SCD in preclinical AD that uses self-report . However, for identifying SCD in the more advanced predementia patients who are the target of our case-finding strategy, informant report may be a better predictor of objective performance than self-report . The IQCODE compared favorably with the MMSE in screening for prevalent dementia , distinguishing mild cognitive impairment (MCI) patients from healthy controls , and predicting future dementia . The IQCODE's focus on instrumental activities of daily living (IADLs) that require intact memory and cognition to complete successfully rather than routinely performed ADLs may contribute to its effectiveness and efficiency. Because low education is a risk factor for dementia onset , the IQCODE with its absence of educational bias  is ideal for minority cohorts. SCD was more severe in the DEM than in the MIND group, which was more severe than in the NMI group.
Second-stage testing was accomplished with the pFCSRT + IR, designed to overcome the confounding impairments of conventional word list learning tests that do not control the conditions of learning . As expected, the cutoff of 30, which maximized sensitivity and specificity, was higher than in previous studies. A score of 30 or less corresponds to the level of memory impairment in AD cases 7 years before clinical dementia was diagnosed . This cutoff is only appropriate for second-stage testing. Lower cutoffs should be used for case identification and prediction of future dementia when only the pFCSRT + IR is administered , . In the present study, a cut score of ≤25 had same sensitivity (71% vs. 72%) as the two-stage strategy but lower specificity (80% vs. 90%). pFCSRT + IR performance detects amnestic MCI and dementia, predicts future dementia and AD, and distinguishes AD dementias from non-AD dementias , , , , , , . Accumulating data demonstrating its association with the CSF AD signature , , structural and functional imaging , , , and autopsy-markers of AD  has prompted pFCSRT + IR's use in ongoing clinical trials , .
Although our screening and case-finding strategy complement studies demonstrating that patients with both impairment of IADLs and a diagnosis of MCI convert to future dementia faster and at higher rates than patients with MCI only , the study has several limitations. First, it was implemented in just two centers in a diverse urban area; its generalizability needs to be assessed. Second, the costs and benefits of case finding using this strategy have not been assessed. Third, memory screening may be suboptimal in detecting non-AD dementias. Fourth, nine percent (36/378) of patients who were classified clinically as having no memory impairment were false-positive cases using this strategy. These individuals should be followed more closely for the development of cognitive impairment. Ultimately, the optimal balance of sensitivity and specificity in a screening or case-finding program depends on the setting and consequences of screening positive or negative. In a setting where rescreening is available, the consequence of false negatives is reduced. If screen-positive patients are to undergo expensive or invasive diagnostic testing and resources are scarce, specificity may be more important than sensitivity.
A systematic review of the prevalence of missed and delayed dementia diagnosis of among primary care providers ranged from 9 to 41 percent . Although dementia screening programs have improved recognition, specificity has been modest resulting in unnecessary anxiety for patients and families , . In addition, most patients who screen positive do not return for the diagnostic visit, which usually includes further cognitive testing , , . Although composite scores comprised of several cognitive tests identify impairment with high sensitivity and specificity in research studies, their use may not be feasible in primary care settings with scarce financial and human resources .
We believe that routine cognitive assessment will become the standard of care as both screening strategies improve and as effective treatments emerge. An efficient, sensitive, and specific case-finding strategy could reduce the burden of unrecognized cognitive impairment and dementia in primary care settings and provide effective gate keeping for more invasive and expensive biomarker procedures . Ultimately, screening and case-finding procedures need to be integrated into clinical practice to assess the aggregate costs and benefits of screening . Our screening and case-identification strategy can be accomplished in a single patient visit, with the informant interview conducted before or at the clinic visit. Resources are focused on testing the memory of only patients with subjective cognitive decline, rendering it an efficient strategy when primary care screening of asymptomatic older adults is recommended. High specificity (≥90%) reduces unnecessary emotional and psychological stress. As new treatments and preventive approaches for AD emerge, they will be integrated in primary care settings where the majority of the older adults receive their care. Tools like this screening and case-finding strategy will be essential to implementing these treatments so that all older adults can benefit from early identification.
This work was supported by the National Institutes of Health (R21 AG036935) to PI: E.G.
The FCSRT + IR is copyrighted by the Albert Einstein College of Medicine and is made freely available for noncommercial purposes.
E.G. receives a small percentage of any royalties on the FCSRT + IR when it is used for commercial purposes.
R.B.L. receives research support from the NIH: PO1 AG003949 (program director), RO1AG025119 (investigator), RO1AG022374-06A2 (investigator), RO1AG034119 (investigator), RO1AG12101 (investigator); serves on the editorial boards of Neurology and as senior advisor to Headache; has reviewed for the NIA and NINDS; holds stock options in eNeura Therapeutics (a company without commercial products); serves as consultant, advisory board member, or has received honoraria from the following: Alder, Allergan, American Headache Society, Autonomic Technologies, Avanir, Boston Scientific, Bristol Myers Squibb, CoLucid, Dr. Reddy's, Electrocore, Eli Lilly, eNeura Therapeutics, Informa, Novartis, Teva, Vedanta.