PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of brainLink to Publisher's site
 
Brain. 2010 November; 133(11): 3290–3300.
Published online 2010 September 7. doi:  10.1093/brain/awq204
PMCID: PMC2965421

Relationship of dementia screening tests with biomarkers of Alzheimer’s disease

Abstract

Screening tests for Alzheimer’s disease lack sensitivity and specificity. We developed the AD8, a brief dementia screening interview validated against clinical and cognitive evaluations, as an improvement over current screening methods. Because insufficient follow-up has occurred to validate the AD8 against the neuropathologic findings of Alzheimer’s disease, we investigated whether AD8 scores correspond to impairment in episodic memory testing and changes in biomarkers of Alzheimer’s disease (cerebrospinal fluid and amyloid imaging with Pittsburgh compound B) characteristic of symptomatic Alzheimer’s disease. We also compared informant-based assessments with brief performance-based dementia screening measurements such as the Mini Mental State Exam. The sample (n = 257) had a mean age of 75.4 years with 15.1 years of education; 88.7% were Caucasian and 45.5% were male. The sample was divided into two groups based on their AD8 scores: those with a negative dementia screening test (AD8 score 0 or 1, n = 137) and those with a positive dementia screening test (AD8 score ≥2, n = 120). Individuals with positive AD8 scores had abnormal Pittsburgh compound B binding (P < 0.001) and cerebrospinal fluid biomarkers (P < 0.001) compared with individuals with negative AD8 scores. Individuals with positive AD8 tests and positive biomarkers scored in the impaired range on the Wechsler Logical Memory Story A (mean score 7.0 ± 4.5 for Pittsburgh compound B; mean score 7.6 ± 5.3 for cerebrospinal fluid amyloid beta protein 1–42). The AD8 area under the curve for Pittsburgh compound B was 0.737 (95% confidence interval: 0.64–0.83) and for cerebrospinal fluid amyloid beta protein 1–42 was 0.685 (95% confidence interval: 0.60–0.77) suggesting good discrimination. The AD8 had superior sensitivity in detecting early stages of dementia compared with the Mini Mental State Examination. The AD8 had a likelihood ratio of a positive test of 5.8 (95% confidence interval: 5.4–6.3) and likelihood ratio of a negative test of 0.04 (95% confidence interval: 0.03–0.06), increasing the pre-test probability of an individual having symptomatic Alzheimer’s disease. Individuals with AD8 scores of ≥2 had a biomarker phenotype consistent with Alzheimer’s disease and lower performance on episodic memory tests, supporting a diagnosis of Alzheimer’s disease. Informant-based assessments may be superior to performance-based screening measures such as the Mini Mental State Examination in corresponding to underlying Alzheimer’s disease pathology, particularly at the earliest stages of decline. The use of a brief test such as the AD8 may improve strategies for detecting dementia in community settings where biomarkers may not be readily available, and may enrich clinical trial recruitment by increasing the likelihood that participants have underlying biomarker abnormalities.

Keywords: AD8, Alzheimer’s disease, screening, biomarkers, preclinical, cognition

Introduction

Alzheimer’s disease, the most common cause of dementia, is often not diagnosed in its earliest symptomatic stages (Galvin et al., 2005a; Holsinger et al., 2007), including its prodromal syndromes such as mild cognitive impairment (Petersen et al., 2009). One possible explanation for the limited detection of Alzheimer’s disease may be the lack of brief screening tests that have been adequately validated to detect the earliest signs of impairment and that correspond to underlying Alzheimer’s disease pathology. The expansion of clinical, epidemiological and social-behavioural research is also hampered by the lack of valid screening instruments that can be applied in community settings. Much effort has been made to identify and verify diagnostic biomarkers for mild cognitive impairment and Alzheimer’s disease (Mintun et al., 2006; Fagan et al., 2007; Perrin et al., 2009) to improve antecedent detection and assist in the development of possible disease-modifying treatments, which may be most effective if initiated early in the disease process (Milne et al., 2008; Mattsson et al., 2009).

Biological markers of Alzheimer’s disease and mild cognitive impairment can serve as in vivo diagnostic indicators of underlying pathology, particularly when clinical symptoms are mild (Hampel et al., 2008; Perrin et al., 2009). In recent years, the search for fluid and imaging biomarkers has undergone a rapid evolution (Hampel et al., 2009) and combined analysis of different biochemical and neuroimaging studies may further increase diagnostic sensitivity and specificity (Fagan et al., 2006, 2007; Cedazo-Minguez and Winblad, 2010). Ongoing, large-scale, international, controlled, multi-centre trials will provide further validation of imaging and CSF biomarker candidates as outcome measures in early symptomatic Alzheimer’s disease for use in phase III clinical efficacy trials (Hampel and Broich, 2009; Perrin et al., 2009; Cedazo-Menguez and Winblad, 2010). Currently, the diagnosis of Alzheimer’s disease is based on exclusion of other forms of impairment, with definitive diagnosis requiring autopsy confirmation. Thus, there is a strong need to find easily measurable in vivo biomarkers of Alzheimer’s disease that could facilitate early and accurate diagnosis, as well as prognostic data to assist in monitoring therapeutic efficacy (Urbanelli et al., 2009).

Although the use of biomarkers is attractive, in community settings and developing countries PET or CSF studies may not be affordable, practical or acceptable to patients. In the busy office setting, a primary care provider may be more likely to use a brief screening test that can detect impairment if it has been validated against some form of ‘gold standard’. At this time, however, the United States Preventive Services Task Force (http://www.ahrq.gov/clinic/3rduspstf/dementia/dementrr.htm) concluded that the evidence is insufficient to recommend for or against routine screening for dementia in older adults. Many of the current brief screening measures, such as the Mini Mental State Examination (MMSE; Folstein et al., 1975), have modest sensitivity but only fair specificity in detecting dementia (Galvin et al., 2005). The accuracy of the MMSE depends on a person's age and educational level; using an arbitrary cut-point (typically < 23) may potentially lead to false positives among people with lower educational levels and false negatives among individuals with higher educational levels (Galvin et al., 2005).

Part of the problem may lie in the fact that many brief screening measures are performance based, so that an individual’s scores are compared to published norms. By the time some individuals reach published thresholds of impairment by MMSE, they may already be quite impaired as determined by other measures. The diagnosis of dementia requires the cognitive impairment to represent a change from premorbid status and to interfere with accustomed daily activities (McKhann et al., 1984; Diagnostic and Statistical Manual, 1994). Informant-based assessments offer the opportunity to collect measures of change and degree of interference but can typically be time consuming, making them impractical for the office setting, epidemiological fieldwork, or arenas outside of specialty centres. We developed a brief informant interview, the AD8 (Galvin et al., 2005a, 2006, 2007a, b), that reliably detects cognitive impairment, derived from the informant interview used to generate the Clinical Dementia Rating (CDR; Morris, 1993) and validated against both the CDR and neuropsychological testing as gold standards (Galvin et al., 2006). Furthermore, the AD8 has been translated into and validated in other languages, including Spanish (Ecuador, Espinosa et al., 2008), Portuguese (Brazil, Correia et al., 2009) and Korean (Ryu et al., 2009). Because the AD8 uses the individual as his or her own control, it can better serve as a measure of intra-individual change. Thus, early detection of dementia may be possible regardless of age, gender, education or language effects.

Here we test the hypothesis that a reliable and validated informant-based dementia screening test (the AD8) correlates with changes in biomarkers of Alzheimer’s disease, and if positive, screening with the AD8 clinically supports an Alzheimer’s disease clinical phenotype. If true, the use of the AD8 should improve detection of Alzheimer’s disease in the community in a brief, reliable and culturally sensitive fashion that overcomes one of the main objections to dementia screening put forth by the Preventive Services Task Force.

Material and methods

Participants

Research participants aged 50–91 years were enrolled in a longitudinal study of memory and ageing (Berg et al., 1998) at the Washington University Alzheimer Disease Research Centre. The longitudinal study focuses on characterizing the transition between healthy brain ageing and very mild dementia. All participants underwent identical detailed clinical and cognitive assessments, including all items from the Uniform Data Set (Morris et al., 2006). The Washington University Human Research Protection Office approved all procedures.

Clinical evaluation

Experienced, research-trained clinicians conducted semi-structured interviews with the participant and a knowledgeable collateral source (usually spouse or adult child) including a health history, medication and depression inventories, an aphasia battery and a neurological examination. Each participant was administered the MMSE (Folstein et al., 1975) and the Short Blessed Test (Katzman et al., 1983) as brief screening measures for dementia. Clinicians were blinded to all biomarker studies. The diagnosis of dementia was made independently of psychometric performance and was based on a history of gradual onset and progressive cognitive decline that interfered with the person’s ability to carry out accustomed activities (Diagnostic and Statistical Manual, 1994). The CDR was used to determine the presence or absence of dementia and, if present, to stage its severity (Morris, 1993). The CDR evaluates cognitive function in each of six categories (memory, orientation, judgement and problem solving, performance in community affairs, home and hobbies, and personal care) without reference to psychometric performance or results of previous evaluations. A CDR score of 0 indicates no dementia, and CDR scores of 0.5, 1, 2 and 3 correspond to very mild, mild, moderate and severe dementia, respectively. The CDR sum of boxes provides a quantitative expansion of the global CDR score, ranging from 0 (no impairment) to 18 (maximal impairment) (Berg et al., 1998). The CDR has high inter-rater reliability (Burke et al., 1988), is sensitive to clinical progression and is highly predictive (93%) of autopsy-confirmed Alzheimer’s disease (Berg et al., 1998; Galvin et al., 2005b).

A psychometric battery was administered to all participants by trained psychometricians 1–2 weeks after the clinical assessment. The battery assesses episodic and semantic memory, language and executive, attention and visuospatial tasks (Johnson et al., 2008; Weintraub et al., 2009) that are compromised in symptomatic Alzheimer’s disease. The battery includes: Associate Learning and Mental Control from the Wechsler Memory Scale (Wechsler and Stone, 1973); Logical Memory I (Story A only) and Digit Span forward and backward from the Wechsler Memory Scale–Revised (Wechsler, 1987); Information and Block Design from the Wechsler Adult Intelligence Scale (Wechsler, 1955); Digit Symbol and Similarities from the Wechsler Adult Intelligence Scale–Revised (Wechsler, 1981), the 30-item Boston Naming Test (Goodglass and Kaplan, 1983); letter fluency for S and P (Thurstone and Thurstone, 1949); category fluency for animals and vegetables (Weintraub et al., 2009); Trailmaking Test A and B (Armitage, 1946); Free and Cued Selective Reminding Test (Grober et al., 1988); and Form D of the Benton Visual Retention Test (Benton, 1963).

The clinical diagnostic criteria for probable Alzheimer’s disease were those according to the National Institute of Neurological and Communicative Disorders and Stroke and the Alzheimer’s Disease and Related Disorders Association (McKhann et al., 1984). Impaired individuals who had a CDR score of 0.5 but who did not meet dementia criteria were classified as ‘uncertain dementia’. These individuals have many characteristics of mild cognitive impairment, except their episodic memory testing is less impaired (Storandt et al., 2006). Many of these individuals go on to develop symptomatic Alzheimer’s disease (Morris et al., 2001). Demented individuals with a CDR score ≥2 were excluded, as these individuals have difficulty completing psychometric assessment and typically cannot undergo biomarker studies. As the aim of this study was to examine the relationship between dementia screening and biomarkers of Alzheimer’s disease, participants with other dementia diagnoses (Lewy body, frontotemporal, vascular) were excluded.

Administration of the AD8

The AD8 was developed at Washington University as a brief informant questionnaire to detect dementia (Galvin et al., 2005a). The AD8 consists of eight ‘Yes/No’ questions (repeats self, reduced interest in hobbies and activities, trouble with judgement, trouble operating appliances, forgets correct month/year, trouble with finances, forgets appointments, daily problems with memory/thinking). A score ≥2 suggests cognitive impairment. The AD8 takes less than 3 min to complete. In this study, the AD8 was embedded into the clinical assessment. The AD8 has been demonstrated to have intra- and inter-individual, intermodal and test-retest reliability, internal consistency and content, construct and criterion validity in research (Galvin et al., 2005a, 2007a, b) and clinical (Galvin et al., 2006) samples regardless of participant or informant age, gender, education, race or informant relationship to participant (Table 1). The area under the receiver operator characteristic curve discriminating individuals with a CDR score of 0 from those with a CDR score > 0 for the AD8 in English is 0.917, Spanish 0.869, Portuguese 0.861 and Korean 0.880, supporting our view that the AD8 is effective in detecting cognitive impairment regardless of language, education, culture or race. These characteristics of the AD8 make it an ideal dementia screening tool (Holsinger et al., 2007).

Table 1
Psychometric properties of the AD8

Pittsburgh compound B imaging

Amyloid imaging was performed on 151 individuals using PET with the Pittsburgh B compound (PiB) as previously described (Mintun et al., 2006). Imaging was conducted with a Siemens 961 HR ECAT scanner or a Siemens 962 HR+ ECAT scanner (CTI, Knoxville, Kentucky). After radiochemical synthesis of [11C] PiB (Mathis et al., 2003) ~12 mCi of the tracer was administered intravenously with simultaneous initiation of a 60 min dynamic scan. The PiB image analysis was achieved by registering each participant’s PET PiB image set to an MRI that was registered to a standard atlas (Talairach and Tournoux, 1988) target designed to minimize bias due to atrophy (Buckner et al., 2004). High-resolution regional time-activity curves were analysed for PiB-specific binding using the Logan graphical analysis, with the cerebellum as a reference tissue input function (Logan et al., 1996). The Logan analysis yielded a tracer distribution volume ratio, which was then converted to an estimate of the binding potential for each region of interest, such that binding potential = distribution volume ratio  1 (Mintun et al., 2006). The binding potential expresses regional PiB binding values in a manner directly proportional to the number of binding sites. The values of the binding potential from the prefrontal cortex, gyrus rectus, lateral temporal cortex and precuneus areas were averaged in each participant to calculate the mean cortical binding potential; these regions have high PiB uptake in participants with symptomatic Alzheimer’s disease (Mintun et al., 2006). Negative PIB scans are defined as a mean cortical binding potential < 0.18, while positive scans (that is, detection of amyloid binding) are defined as a mean cortical binding potential > 0.18 (Mintun et al., 2006).

Cerebrospinal fluid studies

Individuals (n = 149) underwent lumbar puncture for the collection of CSF using a standard procedure (Fagan et al., 2006, 2007). Briefly, 20–30 ml of CSF (free of any blood contamination) was collected in polypropylene tubes at 8:00 AM after overnight fasting. Samples were gently inverted to avoid gradient effects, briefly centrifuged at low speed to pellet any cellular elements, and aliquoted into polypropylene tubes (500 µl aliquots) before freezing at −84°C. Samples were continuously kept on ice and assays were performed on sample aliquots following a single thaw after initial freezing. The CSF samples were analysed for total tau, tau phosphorylated at threonine 181 and amyloid beta protein 1–42 (Aβ42) by means of a commercial enzyme-linked immunosorbent assay (Innotest; Innogenetics NV, Ghent, Belgium) as previously described (Fagan et al., 2006, 2007). Abnormal CSF Aβ42 levels are defined as measurements <500 pg/ml (Fagan et al., 2009).

Statistical analyses

All analyses were conducted using Statistical Package for the Social Sciences (SPSS v17.0, Chicago, IL). Descriptive statistics were used to characterize and compare groups. The groups were compared using t-tests and ANOVA for quantitative variables and chi square tests of independence for categorical variables. Spearman correlations were used to examine strengths of association. Receiver operator characteristic curves were used to assess the ability of the dementia ratings to discriminate between individuals with positive and negative biomarker studies reported as the area under the curve with 95% confidence intervals. Sensitivity, specificity and likelihood ratios were reported.

Results

Sample characteristics

A total of 257 participants were evaluated between 15 December 2005 and 2 January 2009, were administered the full clinical and cognitive evaluation (including the AD8) and consented to and completed PiB imaging (n = 151), CSF (n = 149) or both (n = 31) biomarker studies. The sample had a mean age of 75.4 ± 7.3 years with 15.1 ± 3.2 years of education. The sample was 88.7% Caucasian and 45.5% male, with a mean MMSE score of 27.2 ± 3.6. Informants comprised spouses (51%), children (29%) or other relatives and friends (20%). The formal diagnoses of the sample were: CDR score of 0, cognitively normal (n = 156); CDR score of 0.5, uncertain dementia (n = 23); CDR score of 0.5, Alzheimer’s disease (n = 53); CDR score of 1, Alzheimer’s disease (n = 25). The mean time between clinical assessment and biomarker study was 5.1 ± 9.7 months for PiB (range 0–30 months) and 0.8 ± 6.9 months for CSF studies (range 0-22 months). The sample was then divided into two groups based on AD8 scores: those who had a negative dementia screening test (AD8 score 0 or 1, n = 137) and those who had a positive dementia screening test (AD8 score ≥2, n = 120). Demographic information and scores on dementia rating scales are provided in Table 2.

Table 2
Sample characteristics

AD8 scores, Pittsburgh compound B and cerebrospinal fluid biomarkers

We compared biomarker profiles of Alzheimer’s disease between participants with positive AD8 tests (score ≥2) and negative AD8 tests (score <2). Significant differences are seen in PiB binding and CSF biomarkers between the groups (Table 2). Participants with positive AD8 scores exhibited the typical fluid biomarker phenotype of Alzheimer’s disease characterized by significantly lower mean levels of CSF Aβ42, greater CSF tau, tau phosphorylated at threonine 181, and the tau(s)/Aβ42 ratios (Fagan et al., 2009; Shaw et al., 2009; Jack et al., 2010). They also exhibited increased mean cortical binding potential for PiB imaging similar to those in studies of individuals with Alzheimer’s disease (Klunk et al., 2004; Mintun et al., 2006).

We next investigated whether the AD8 was sensitive to the inverse relationship between in vivo amyloid plaque burden as detected by PiB and levels of CSF Aβ42 (Fagan et al., 2006). Levels of CSF Aβ42 were plotted as a function of their mean cortical PiB binding potential (Fig. 1) in the 31 participants who underwent both studies. Participants were identified by the CDR stage (Fig. 1A), AD8 classification (Fig. 1B) or their clinical diagnosis (Fig. 1C). Participants with positive cortical PiB binding (mean cortical binding potential > 0.18) had the lowest levels of CSF Aβ42 (<500 pg/ml). The sensitivity and specificity of the AD8 for classifying cases according to biomarker profiles are similar to the CDR and clinical diagnoses made by independent clinicians blinded to biomarker data. Five types of discrepancies are noted (Table 3). Of the 11 subjects exhibiting positive PiB binding and low CSF Aβ42 values, 4 were diagnosed as cognitively normal and had a negative AD8 score, thus supporting previous reports that the presence of cortical amyloid and low CSF Aβ42 can be seen in the absence of cognitive impairment, representing a ‘preclinical’ pathology (Galvin et al., 2005b; Morris et al, 2009; Price et al., 2009). Among individuals with low CSF Aβ42 but normal PiB binding, one case was diagnosed as cognitively normal (CDR score of 0), but had a positive AD8 (score of 2), and one case had a negative AD8 test but had a CDR score of 0.5 and was rated as ‘uncertain dementia’. Of 16 cases with negative biomarkers for Alzheimer’s disease, one case had a positive AD8 (score of 2) but was rated cognitively normal (CDR score of 0). Another case had a positive AD8 (score of 5), a CDR score of 0.5 and dementia of the Alzheimer type with negative biomarkers for amyloid deposition, suggesting that this individual may have cognitive impairment not caused by Alzheimer’s disease.

Figure 1
Scatterplots showing CSF Aβ42 levels as a function of in vivo amyloid load as assessed by PiB binding. (A) Subjects are classified by CDR with circles signifying a CDR score of 0 (no dementia) and triangles signifying a CDR score of 0.5 (cognitive ...
Table 3
Types of discrepancies between biomarker profiles and clinical diagnoses

Because episodic memory deficits are considered a requirement for the diagnosis of Alzheimer’s disease and amnestic mild cognitive impairment, we examined the relationship between AD8 scores with immediate recall of Logical Memory Story A of the Wechsler Memory Scale (Wechsler and Stone, 1973) stratified by PiB status or by CSF Aβ42 levels (Fig. 2). Results were identical for all other components of the psychometric battery (Weintraub et al., 2009) except for Digit Span Forward and Backward, which were not significant (data not shown). Participants with negative AD8 tests scored higher (mean 12.8 ± 4.1) on Logical Memory Story A, regardless of biomarker status, than did participants with positive AD8 tests (mean 8.5 ± 5.3; P < 0.001). As illustrated in Fig. 2A, individuals with positive AD8 tests and positive PiB binding scored in the impaired range on Logical Memory Story A (mean 7.0 ± 4.5). An attenuated relationship was seen with CSF Aβ42 levels (Fig. 2B). Individuals with positive AD8 tests and low CSF Aβ42 levels had the lowest scores on Logical Memory Story A (mean 7.6 ± 5.3), suggesting Alzheimer’s disease as a probable aetiology.

Figure 2
Bar graph of scores on the Logical Memory Story A of the Wechsler Memory Scale (WMS) comparing AD8 status (positive versus negative) with PiB status (A) or CSF Aβ42 status (B). Scores for the AD8 negative groups were superior to AD8 positive groups. ...

Twenty-one participants had a positive AD8 but absent PiB binding or normal CSF Aβ42 levels. These individuals scored within the lower reference range on Logical Memory Story A, with mean values intermediate to those with negative AD8/negative biomarkers and AD8 positive/positive biomarker individuals (Logical Memory Story A scores for the AD8+/PiB– group were 12.2 ± 4.2; scores for the AD8+/ Aβ42– group were 9.8 ± 4.5). The majority of these individuals had a CDR score of 0.5 and were diagnosed with uncertain dementia or Alzeimer’s disease with an active mood disorder. Since episodic memory testing does not support an Alzheimer’s disease–phenotype and biomarkers of Alzheimer’s disease are negative, these mildly impaired individuals may have alternative causes of their cognitive impairment.

Discriminative ability of AD8 compared with other dementia ratings

We examined the strength of association between biomarkers of Alzheimer’s disease and four different dementia rating systems (Table 4): two informant-based assessments (CDR and AD8) and two performance-based assessments (MMSE and Short Blessed Test). The AD8 correlated strongly with all biomarkers of Alzheimer’s disease, as strongly as the CDR and with stronger correlations than MMSE and Short Blessed Test for CSF biomarkers. To examine the ability of the AD8 to detect changes in biomarkers for Alzheimer’s disease, we performed receiver operator characteristic analyses for PiB and CSF Aβ42 (Table 5), examining four methods of dementia rating: the AD8, CDR, MMSE and Short Blessed Test. The cut-off for a positive PiB test was a mean cortical binding potential >0.18 (Mintun et al., 2006) and CSF Aβ42 <500 pg/ml (Fagan et al., 2009). The AD8 area under the curve for PiB was 0.732 [95% confidence interval (CI): 0.64–0.82]; the area under the curve for CSF Aβ42 was 0.685 (95% CI: 0.60–0.77), suggesting good to very good discrimination. To assess whether these relationships hold true for the mildest cases of dementia, we repeated the analyses eliminating those cases with a CDR score of 1. The CDR and AD8 remained strongly correlated with all biomarkers (P-values range from 0.001–0.004), while the MMSE was no longer significant for CSF Aβ42 and tau phosphorylated at threonine 181. Similar findings occur when performing receiver operator characteristic curves with CDR and AD8 remaining significant, and MMSE losing significance for PiB and CSF Aβ42.

Table 4
Spearman correlations between dementia assessment measures and biomarkers of Alzheimer’s disease
Table 5
Receiver operator characteristic analyses for dementia rating measures

Finally we performed a sensitivity analysis for the AD8 and the MMSE (Table 6). The AD8 and MMSE scores are shown with corresponding clinical diagnosis of dementia, Logical Memory Story A scores, PiB binding potentials and CSF Aβ42 levels. Applying a cut-off of ≥2 for the AD8, 5 individuals with dementia would be missed and 25 false positives would occur for a sensitivity of 96.5% (95% CI: 0.92–0.99), a specificity of 83.4% (95% CI: 0.77–0.89), a positive predictive value of 84.7% (95% CI: 0.78–0.89) and a negative predictive value of 96.2% (95% CI: 0.91–0.98). In the same cohort, applying the published cut-off of ≤23 for the MMSE (Folstein, et al., 1975) 74 individuals with dementia would fail to be detected but there would only be one false positive case. This results in an unacceptable sensitivity of 14.9% (95% CI: 0.09–0.24) but a superior specificity of 99.4% (95% CI: 0.97–0.99).

Table 6
Sensitivity analyses of AD8 and MMSE scores to detect dementia

Another way to evaluate the utility of screening tests is with the likelihood ratio (McGee, 2002). The likelihood ratio of any screening test is the probability that a positive test is found in persons with disease divided by the probability of the same finding in persons without disease (McGee, 2002). Likelihood ratios range from 0 to infinity, with larger numbers providing more convincing evidence of disease; smaller numbers argue that disease is less likely. Ratios close to 1 lack diagnostic value. When the positive likelihood ratio is >5 or the negative likelihood ratio is <0.2, the pre-test probability of a patient having the disease tested can be used to estimate a post-test probability of the disease state existing. In the case of the AD8, the likelihood ratio of a positive test is 5.8 (95% CI: 5.4–6.3) and the likelihood ratio of a negative test is 0.04 (95% CI: 0.03–0.06). For the MMSE, the likelihood ratio of a positive test is 23.8 (95% CI: 1.4–397.9) and the likelihood ratio of a negative test is 0.86 (95% CI: 0.83–0.88).

Recent clinical trials in Alzheimer’s disease frequently use a more liberal cut-off of ≤26 for the MMSE as an inclusion criterion. Repeating the analyses using this cut-off, there would be 46 false negative and 15 false positive cases, providing a sensitivity of 72.6% (95% CI: 0.65–0.79), a specificity of 90.6% (95% CI: 0.85–0.94), a positive predictive value of 89.1% (95% CI: 0.82–0.93) and a negative predictive value of 75.8% (95% CI: 0.69–0.83). The likelihood ratio of a positive test decreases to 7.7 (95% CI: 6.7–8.8) and the likelihood ratio of a negative test improves slightly to 0.32 (95% CI: 0.29–0.32).

Discussion

We found a strong relationship between the AD8, a brief, informant-based, dementia screening tool, and biomarkers of Alzheimer’s disease. Individuals with AD8 scores of ≥2 had a biomarker phenotype (positive PiB binding, low CSF Aβ42, high tau, high tau phosphorylated at threonine 181) consistent with Alzheimer’s disease. A positive AD8 screening test corresponded to lower performance on tests of episodic memory, supporting a clinical phenotype of Alzheimer’s disease. A brief informant-based dementia screening assessment such as the AD8 performs as well as the ‘gold standard’ for informant-based assessments, the CDR, in its relationship to biomarkers of Alzheimer’s disease, and it correlates more strongly than the MMSE and Short Blessed Test to CSF biomarkers of Alzheimer’s disease. Lastly, informant-based assessments provide superior sensitivity to the MMSE in detecting dementia and changes in biomarkers of Alzheimer’s disease, particularly at the early symptomatic stages.

Alzheimer’s disease is under-recognized and under-diagnosed in the community (Boise et al., 1999, 2004; Meuser et al., 2004). This may be due, in part, to the lack of brief measures that can discriminate normal ageing from very mild dementia. A number of brief screening measures (i.e. the MMSE) are already in use, but most are based on patient performance and are therefore unable to detect or quantify change from previous levels of function. Some performance-based measures are also insensitive to subtle changes in high functioning individuals (i.e. ceiling effects) who may score well within the normal range throughout the early stages of dementia. These same measures also may prevent detection of dementia in individuals with poorer lifelong abilities. Furthermore, many cognitive tests are culturally insensitive and may underestimate the abilities of African Americans and other minority groups (Lorentz et al., 2002; Espino et al., 2004; Parker and Philp, 2004). The AD8 is without ceiling effects and is valid in assessing individuals regardless of age, gender, language, race or educational level (Galvin et al., 2006, 2007a, b).

Dementia screening has not been a routine medical practice, partly due to the lack of sensitive, specific and culturally sensitive means of detection. Published criteria for the diagnosis of Alzheimer’s disease (McKhann et al., 1984) require standard assessments of patients, and more recent recommendations also include querying a knowledgeable informant (Dubois et al., 2007). In a community study comparing individuals with mild cognitive impairment to determine who would progress to clinical dementia, only baseline functional impairment as measured by the CDR, without contribution from demographic, cognitive or neuroimaging variables, or mild cognitive impairment subtype, accounted for the differences in conversion rates across the two cohorts (Farias et al., 2009). The investigators concluded that the degree of functional impairment at baseline, rather than test performance, is the most important predictor of conversion to dementia. Not only is the CDR sensitive to early symptomatic Alzheimer’s disease, but unlike the AD8 it provides sufficient information to stage dementia severity and monitor dementia progression. Due to the length of time required to complete the interview, however, the CDR is unlikely to be suitable for general clinical practice. The AD8 has been validated against the CDR and neuropsychological testing to detect dementia (Galvin et al., 2005a, 2006, 2007a, b). The value of the AD8 is that it not only is brief but corresponds to more detailed evaluations, neuropsychological testing and biomarkers of Alzheimer’s disease. However, it is designed primarily as a screening tool to identify those individuals at risk for a more extensive work-up for staging and differential diagnosis such as the CDR and/or neuropsychological testing. It is important to note that both informant-based methods, the AD8 and the CDR, are at least as good as any objective measures. Thus if simple screening is the aim, because the AD8 is comparable to the CDR both in direct comparison and in its relationship to biomarkers, it could be recommended for this simple goal on the basis of utility and brevity.

In parallel, there has been much effort to improve the detection of the underlying pathological basis of Alzheimer’s disease by developing biomarkers of disease that may represent manifestations of disease during the earliest preclinical stages (Perrin et al., 2009). However, many biomarkers are invasive (lumbar puncture), expose individuals to radioactive tracers (PET), may be uncomfortable (MRI) and may not be readily available in all clinical settings.

Our finding that the AD8 is correlated with biomarkers of Alzheimer’s disease helps to address the gap in our understanding of how best to detect Alzheimer’s disease at the earliest possible stage in primary care settings. Expert centres have little difficulty in diagnosing Alzheimer’s disease, but in the community, diagnoses may be delayed because of an inability to effectively detect cognitive impairment in the setting of a busy office practice. The use of a brief test such as the AD8 may improve strategies for detecting dementia in the community and in developing countries where biomarkers may not be readily available, and may enhance and enrich clinical trial recruitment by increasing the likelihood that participants have underlying biomarker abnormalities that are increasingly becoming required for inclusion.

Our study has limitations. Our sample is not population based. The AD8 uses informant information; the absence of an observant informant may limit use in certain clinical situations. However, informant interviews can be successfully applied in populations with lower educational attainment (Galvin et al., 2007a; Espinosa et al., 2008; Malmstrom et al., 2009). Additionally, the AD8 can be used as a patient interview in the absence of an informant (Galvin et al., 2007a) and can be administered in person or by phone (Galvin et al., 2006). Dementia screening requires a consideration of the population at risk and the sensitivity and specificity of the instruments used (Holsinger et al., 2007). A large number of false positive individuals might expend limited health care funds. Conversely, a large number of false negative individuals would be denied treatment and might miss opportunities to participate in clinical research studies. A staged dementia screening approach would then make the most sense. The AD8 given to the informant would detect intra-individual change and interference with daily function and, if positive, a brief performance test—such as word-list or paragraph recall—would confirm the presence of episodic memory deficits (Galvin et al., 2007b). We have previously demonstrated that the MMSE did not add to the discriminative ability of the AD8 (Galvin et al., 2007b), and from the data presented here, the MMSE would fail to detect 85% of the individuals who were clinically rated as demented, performed poorly on tests of episodic memory and had biomarker changes supporting a diagnosis of Alzheimer’s disease. More detailed assessments, such as the CDR and neuropsychological testing, would then be used to stage dementia severity and assist in differential diagnosis. There are non-demented individuals in our sample who have biomarker abnormalities suggestive of preclinical Alzheimer’s disease that neither the AD8 nor the CDR can detect. At the present time, it is impossible to identify these individuals during a cross-sectional evaluation; however, clues from longitudinal analyses suggest that individuals with preclinical disease have absence of practice effects (Galvin, et al., 2005b) and greater rates of annual change on neuropsychological testing (Storandt, et al., 2009) with inflection points in performance occurring 1–3 years prior to clinical diagnosis (Johnson et al., 2009). Further work is needed to determine whether the AD8 is sensitive to longitudinal change.

Given the recent efforts to develop biomarkers for Alzheimer’s disease, it is important to determine how this information can be more readily translated for use in the busy primary practice setting. The American Medical Association (http://www.ama-assn.org/ama/pub/category/4789.html), American Geriatrics Society (http://www.americangeriatrics.org/products/positionpapers/stopscreening.shtml), American Academy of Neurology (http://www.aan.com/professionals/practice/pdfs/dementia_guideline.pdf) and the American Academy of Family Physicians (Santacruz and Swagerty, 2001) all recommend that clinicians remain diligent in the early identification of symptoms of Alzheimer’s disease in their patients. Patients are generally receptive to cognitive screening as part of their medical care (Galvin et al., 2008). The AD8, a brief, informant-based assessment, readily detects dementia and corresponds to underlying pathology of Alzheimer’s disease and assists in early detection of dementia.

Funding

National Institute on Ageing, National Institutes of Health grants P01 AG03991, P01 AG026276 and P50 AG05681. Drs Galvin and Morris hold the copyright for the AD8. Dr Mintun is a consultant for Avid Radiopharmaceuticals.

Acknowledgements

The authors thank the Clinical, Biomarker, Neuroimaging and Genetics Cores of the Washington University Alzheimer Disease Research Centre for the clinical, cognitive, ApoE genotyping and biomarker assessments and Dr Martha Storandt for her expert review. Dr Galvin currently is located at New York University Langone School of Medicine. A copy of the AD8 and scoring instructions may be viewed at http://alzheimer.wustl.edu/About_Us/PDFs/AD8form2005.pdf. Statistical analyses were performed by Dr Galvin.

Glossary

Abbreviations

42
 amyloid beta protein 1–42
CDR
 clinical dementia rating
MMSE
 Mini Mental State Examination
PiB
Pittsburgh compound B

References

  • American Academy of Neurology. http://www.aan.com/professionals/practice/pdfs/dementia_guideline.pdf (3 January 2010, date last accessed)
  • American Geriatrics Society. http://www.americangeriatrics.org/products/positionpapers/stopscreening.shtml (3 January 2010, date last accessed)
  • American Medical Association. Practical guide for the Primary Care Physician on the Diagnosis, Management and Treatment of Dementia. Program on Aging and Community Health. Chicago, IL, 2001. Available at: http://www.ama-assn.org/ama/pub/category/4789.html (3 January 2010, date last accessed)
  • Armitage SG. An analysis of certain psychological tests used in the evaluation of brain injury. Psych Mono. 1946;60:1–48.
  • Benton AL. The Revised Visual Retention Test: clinical and experimental applications. New York: Psychological Corporation; 1963.
  • Berg L, McKeel DW, Jr, Miller JP, Storandt M, Rubin EH, Morris JC, et al. Clinicopathologic studies in cognitively healthy aging and Alzheimer disease: relation of histologic markers to dementia severity, age, sex, and apolipoprotein E genotype. Arch Neurol. 1998;55:326–35. [PubMed]
  • Boise L, Camicioli R, Morgan DL, Rose JH, Congleton L. Diagnosing dementia: perspectives of primary care physicians. Gerontologist. 1999;39:457–65. [PubMed]
  • Boise L, Neal MB, Kaye J. Dementia assessment in primary care: results from a study in three managed care systems. J Gerontol. 2004;59:M621–6. [PubMed]
  • Buckner RL, Head D, Parker J, Fotenos AF, Marcus D, Morris JC, et al. A unified approach for morphometric and functional data analysis in young, old, and demented adults using automated atlas-based head size normalization: reliability and validation against manual measurement of total intracranial volume. Neuroimage. 2004;23:724–38. [PubMed]
  • Burke WJ, Miller JP, Rubin EH, Morris JC, Coben LA, Duchek J, et al. Reliability of the Washington University clinical dementia rating. Arch Neurol. 1988;45:31–2. [PubMed]
  • Cedazo-Minguez A, Winblad B. Biomarkers for Alzheimer's disease and other forms of dementia: clinical needs, limitations and future aspects. Exp Gerontol. 2010;45:5–14. [PubMed]
  • Correia CC, Almeida MSC, Junqueira F, Lima F, Bastos O, Petribu K, et al. AD8: cross-cultural validation in Portuguese. Dement Neuropsychologia. 2009;3(Suppl 1):7.
  • Diagnostic and Statistical Manual of Mental Disorders, 4th edn. Washington, DC: American Psychiatric Association; 1994.
  • Dubois B, Feldman HH, Jacova C, Dekosky ST, Barberger-Gateau P, Cummings J, et al. Research criteria for the diagnosis of Alzheimer's disease: revising the NINCDS-ADRDA criteria. Lancet Neurol. 2007;6:734–46. [PubMed]
  • Espino DV, Lichtenstein MJ, Palmer RF, Hazuda HP. Evaluation of the mini-mental state examination's internal consistency in a community-based sample of Mexican-American and European-American elders: Results from the San Antonio longitudinal study of aging. J Am Geriatr Soc. 2004;52:822–7. [PubMed]
  • Espinosa PS, Espinosa PH, Garzon YR, Mendiondo MS, Basantes AG, Abner EL, et al. Detecting dementia in Ecuador using the AD8 and CDR evaluations. Centro Internacional en Neurociencias. 2008 http://cien-ecuador.org/estudio_de_demencia_dementia_study (3 January 2010, date last accessed)
  • Fagan A, Mintun M, Mach R, Lee SY, Dence CS, Shah AR, et al. Inverse relation between in vivo amyloid imaging load and CSF Abeta42in humans. Ann Neurol. 2006;59:512–9. [PubMed]
  • Fagan AM, Mintun MA, Shah AR, Aldea P, Roe CM, Mach RH, et al. Cerebrospinal fluid tau and ptau(181) increase with cortical amyloid deposition in cognitively normal individuals: implications for future clinical trials of Alzheimer's disease. EMBO Mol Med. 2009;1:371–80. [PMC free article] [PubMed]
  • Fagan AM, Roe CM, Xiong C, Mintun MA, Morris JC, Holtzman DM. Cerebrospinal fluid tau/β-amyloid42 ratio as a prediction of cognitive decline in nondemented older adults. Arch Neurol. 2007;64:343–9. [PubMed]
  • Farias ST, Mungas D, Reed BR, Harvey D, DeCarli C. Progression of mild cognitive impairment to dementia in clinic- vs. community-based cohorts. Arch Neurol. 2009;66:1151–7. [PMC free article] [PubMed]
  • Folstein MF, Folstein SE, McHugh PR. Mini-mental State: A practical method for grading the cognitive state of patients for the clinicians. J Psychiatr Res. 1975;12:189–98. [PubMed]
  • Galvin JE, Fu Q, Nguyen JT, Glasheen C, Scharff DP. Psychosocial determinants of intention to screen for Alzheimer disease. Alzheimer Dement. 2008;4:353–60. [PMC free article] [PubMed]
  • Galvin JE, Powlishta KK, Wilkins K, McKeel DW, Jr, Xiong C, Grant E, et al. Predictors of preclinical Alzheimer disease and dementia: a clinicopathologic study. Arch Neurol. 2005b;62:758–65. [PubMed]
  • Galvin JE, Roe CM, Coats MA, Morris JC. Patients rating of cognitive ability: using the AD8, a brief informant interview as a self rating tool to detect dementia. Arch Neurol. 2007a;64:725–30. [PubMed]
  • Galvin JE, Roe CM, Morris JC. Evaluation of cognitive impairment in the older adult: combining brief informant and performance measures. Arch Neurol. 2007b;64:718–24. [PubMed]
  • Galvin JE, Roe CM, Powlishta KK, Coats MA, Muich SJ, Grant E, et al. The AD8: a brief informant interview to detect dementia. Neurology. 2005a;65:559–64. [PubMed]
  • Galvin JE, Roe CM, Xiong C, Morris JC. The validity and reliability of the AD8 informant interview for dementia. Neurology. 2006;67:1942–8. [PubMed]
  • Goodglass H, Kaplan E. The assessment of Aphasia and related disorders. 2. Philadelphia: Lea & Febiger; 1983.
  • Grober E, Buschke H, Crystal H, Bang S, Dresner R. Screening for dementia by memory testing. Neurology. 1988;3:900–3. [PubMed]
  • Hampel H, Broich K. Enrichment of MCI and early Alzheimer's disease treatment trials using neurochemical and imaging candidate biomarkers. J Nutr Health Aging. 2009;13:373–5. [PubMed]
  • Hampel H, Bürger K, Teipel SJ, Bokde AL, Zetterberg H, Blennow K. Core candidate neurochemical and imaging biomarkers of Alzheimer's disease. Alzheimers Dement. 2008;4:38–48. [PubMed]
  • Hampel H, Shen Y, Walsh DM, Aisen P, Shaw LM, Zetterberg H, et al. Biological markers of amyloid beta-related mechanisms in Alzheimer's disease. Exp Neurol. 2010;223:334–46. [PMC free article] [PubMed]
  • Holsinger T, Deveau J, Boustani M, Williams JW., Jr Does this patient have dementia? JAMA. 2007;297:2391–404. [PubMed]
  • Jack CR, Jr, Knopman DS, Jagust WJ, Shaw LM, Aisen PS, Weiner MW, et al. Hypothetical model of dynamic biomarkers of the Alzheimer's pathological cascade. Lancet Neurol. 2010;9:119–28. [PMC free article] [PubMed]
  • Johnson DK, Storandt M, Morris JC, Galvin JE. Longitudinal study of the transition from healthy aging to Alzheimer disease. Arch Neurol. 2009;66:1254–9. [PMC free article] [PubMed]
  • Johnson DK, Storandt M, Morris JC, Langford ZD, Galvin JE. Cognitive profiles in dementia: Alzheimer disease versus nondemented aging. Neurology. 2008;71:1783–9. [PMC free article] [PubMed]
  • Katzman R, Brown T, Fuld P, Peck A, Schechter R, Schimmel H. Validation of a short orientation-memory-concentration test of cognitive impairment. Am J Psychiatry. 1983;140:734–9. [PubMed]
  • Klunk WE, Engler H, Nordberg A, Wang Y, Blomqvist G, Holt DP, et al. Imaging brain amyloid in Alzheimer's disease with Pittsburgh Compound-B. Ann Neurol. 2004;55:306–19. [PubMed]
  • Logan J, Fowler JS, Volkow ND, Wang GJ, Ding YS, Alexoff DL. Distribution volume ratios without blood sampling from graphical analysis of PET data. J Cereb Blood Flow Metab. 1996;16:834–80. [PubMed]
  • Lorentz W, Scanlan J, Borson S. Brief screening tests for dementia. Can J Psychiatry. 2002;47:723–33. [PubMed]
  • Malmstrom TK, Miller DK, Coats MA, Jackson P, Miller JP, Morris JC. Informant-based dementia screening in a population-based sample of African Americans. Alzheimer Dis Assoc Disord. 2009;23:117–23. [PMC free article] [PubMed]
  • Mathis CA, Wang Y, Holt DP, Huang GF, Debnath ML, Klunk WE. Synthesis and evaluation of 11C-labeled 6-substituted 2-arylbenzothiazoles as amyloid imaging agents. J Med Chem. 2003;46:2740–54. [PubMed]
  • Mattsson N, Zetterberg H, Hansson O, Andreasen N, Parnetti L, Jonsson M, et al. CSF biomarkers and incipient Alzheimer disease in patients with mild cognitive impairment. JAMA. 2009;302:385–93. [PubMed]
  • McGee S. Simplifying likelihood ratios. J Gen Intern Med. 2002;17:646–9. [PubMed]
  • McKhann G, Drachman D, Folstein M, Katzman R, Price D, Stadlan EM. Clinical diagnosis of Alzheimer's disease: report of the NINCDS-ADRDA Work Group under the auspices of Department of Health and Human Services Task Force on Alzheimer's Disease. Neurology. 1984;34:939–44. [PubMed]
  • Meuser TM, Boise L, Morris JC. Clinician beliefs and practices in dementia care: Implications for health educators. Educational Gerontol. 2004;30:491–516.
  • Milne A, Culverwell A, Guss R, Tuppen J, Whelton R. Screening for dementia in primary care: a review of the use, efficacy and quality of measures. Int Psychogeriatr. 2008;20:911–26. [PubMed]
  • Mintun MA, LaRossa GN, Sheline YI, Dence CS, Lee SY, Mach RH, et al. [11C] PIB in a nondemented population: potential antecedent marker of Alzheimer disease. Neurology. 2006;67:446–52. [PubMed]
  • Morris JC. The clinical dementia rating (CDR): current version and scoring rules. Neurology. 1993;43:2412–4. [PubMed]
  • Morris JC, Roe CM, Grant EA, Head D, Storandt M, Goate AM, et al. Pittsburgh compound B imaging and prediction of progression from cognitive normality to symptomatic Alzheimer disease. Arch Neurol. 2009;66:1469–75. [PMC free article] [PubMed]
  • Morris JC, Storandt M, Miller JP, McKeel DW, Price JL, Rubin EH, et al. Mild cognitive impairment represents early-stage Alzheimer disease. Arch Neurol. 2001;58:397–405. [PubMed]
  • Morris JC, Weintraub S, Chui HC, Cummings J, Decarli C, Ferris S, et al. The Uniform Data Set (UDS: clinical and cognitive variables and descriptive data from Alzheimer Disease Centers. Alzheimer Dis Assoc Disord. 2006;20:210–6. [PubMed]
  • Parker C, Philp I. Screening for cognitive impairment among older people in black and minority ethnic groups. Age Ageing. 2004;33:447–52. [PubMed]
  • Perrin RJ, Fagan AM, Holtzman DM. Multimodal techniques for diagnosis and prognosis of Alzheimer's disease. Nature. 2009;461:916–22. [PMC free article] [PubMed]
  • Petersen RC, Roberts RO, Knopman DS, Boeve BF, Geda YE, Ivnik RJ, et al. Mild cognitive impairment: ten years later. Arch Neurol. 2009;66:1447–55. [PMC free article] [PubMed]
  • Price JL, McKeel DW, Jr, Buckles VD, Roe CM, Xiong C, Grundman M, et al. Neuropathology of nondemented aging: presumptive evidence for preclinical Alzheimer disease. Neurobiol Aging. 2009;30:1026–36. [PMC free article] [PubMed]
  • Ryu HJ, Kim HJ, Han SH. Validity and reliability of the Korean version of the AD8 informant interview (K-AD8) in dementia. Alzheimer Dis Assoc Disord. 2009;23:371–6. [PubMed]
  • Santacruz KS, Swagerty D. Early diagnosis of dementia. Am Fam Physician. 2001;63:703–13. [PubMed]
  • Shaw LM, Vanderstichele H, Knapik-Czajka M, Clark CM, Aisen PS, Petersen RC, et al. Alzheimer's Disease Neuroimaging Initiative. Cerebrospinal fluid biomarker signature in Alzheimer's disease neuroimaging initiative subjects. Ann Neurol. 2009;65:403–13. [PMC free article] [PubMed]
  • Storandt M, Grant EA, Miller JP, Morris JC. Longitudinal course and neuropathologic outcomes in original vs. revised MCI and in pre-MCI. Neurology. 2006;67:467–73. [PubMed]
  • Storandt M, Mintun MA, Head D, Morris JC. Cognitive decline and brain volume loss as signatures of cerebral amyloid-beta peptide deposition identified with Pittsburgh compound B: cognitive decline associated with Abeta deposition. Arch Neurol. 2009;66:1476–81. [PMC free article] [PubMed]
  • Talairach J, Tournoux P. Co-planar Stereotaxic Atlas of the Human Brain: 3-D Proportional System: An Approach to Cerebral Imaging. Stuttgart, Germany. Thieme Medical Publishers. 1988
  • Thurstone LL, Thurstone LG. Examiner Manual for the SRA Primary Mental Abilities Test. Chicago: Science Research Associates; 1949.
  • Unites States Preventive Services Task Force. http://www.ahrq.gov/clinic/3rduspstf/dementia/dementrr.htm (3 January 2010, date last accessed)
  • Urbanelli L, Magini A, Ciccarone V, Trivelli F, Polidoro M, Tancini B, Emiliani C. New perspectives for the diagnosis of Alzheimer's disease. Recent Pat CNS Drug Discov. 2009;4:160–81. [PubMed]
  • Wechsler D. Manual: Wechsler Adult Intelligence Scale. New York: Psychological Corporation; 1955.
  • Wechsler D. Manual: Wechsler Adult Intelligence Scale–Revised. New York: Psychological Corporation; 1981.
  • Wechsler D. Manual: Wechsler Memory Scale–Revised. San Antonio, TX: Psychological Corporation; 1987.
  • Wechsler D, Stone CP. Manual: Wechsler Memory Scale. New York: Psychological Corporation; 1973.
  • Weintraub S, Salmon D, Mercaldo N, Ferris S, Graff-Radford NR, Chui H, et al. The Alzheimer's Disease Centers' Uniform Data Set (UDS): the neuropsychologic test battery. Alzheimer Dis Assoc Disord. 2009;23:91–101. [PMC free article] [PubMed]

Articles from Brain are provided here courtesy of Oxford University Press