PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
Brain Imaging Behav. Author manuscript; available in PMC 2013 December 1.
Published in final edited form as:
PMCID: PMC3532526
NIHMSID: NIHMS423424

Dysexecutive and amnesic AD subtypes defined by single indicator and modern psychometric approaches: relationships with SNPs in ADNI

Shubhabrata Mukherjee, PhD,1 Emily Trittschuh, PhD,2 Laura E. Gibbons, PhD,1 R. Scott Mackin, PhD,3 Andrew Saykin, PsyD,4 and Paul K. Crane, MD, MPH1,*, for the Alzheimer’s Disease Neuroimaging Initiative

Abstract

Background

Previous investigators have suggested the existence of distinct cognitive phenotypes of Alzheimer’s disease (AD): a dysexecutive subgroup with executive functioning worse than memory and an amnesic subgroup with memory worse than executive functioning.

Methods

We evaluated data from the AD Neuroimaging Initiative. We assigned people with AD to dysexecutive and amnesic subgroups using single indicators, and analogously using the ADNI-Mem and ADNI-EF composite scores developed using modern psychometric approaches. We evaluated associations between subgroup membership, APOE genotype, and SNPs associated with AD, and brain vascular disease defined as white matter hyperintensities (WMH) and MRI-identified infarcts. We hypothesized that APOE ε4 and alleles associated with higher risk for AD would predict amnesic subgroup membership; alleles associated with higher WMH or infarct burden would predict dysexecutive subgroup membership.

Results

Classification agreement between the two approaches was only fair (kappa = 0.23). There was no relationship between APOE alleles and the dysexecutive or amnesic phenotypes defined by either categorization approach. There were 58 AD-related and 25 WMH- or infarct-related SNPs for which odds ratios were > 1.5 or < 0.67 for dysexecutive vs. amnesic subgroup defined by either categorization approach. Higher proportions of SNPs had odds ratios in the hypothesized direction for the subgroups defined by the modern psychometric approach for AD-related (58% vs. 38%, p-value < 0.001) and brain vascular disease-related SNPs (48 vs. 32%, p-value = 0.01).

Conclusions

Genetic variation may underlie differential performance in memory and executive functioning among people with AD. Modern psychometric composite scores produced group assignments with more SNP associations in the hypothesized direction.

Keywords: Memory, executive functioning, Alzheimer’s disease, phenotype, genetic analyses, psychometrics

Background

Clinical diagnosis of Alzheimer’s disease (AD) is based on a decline in memory and at least one other cognitive function which has resulted in loss of independence in daily living (American Psychiatric Association. Task Force on DSM-IV 1994; McKhann et al. 1984). While memory problems are requisite for a diagnosis of AD, there is still considerable variation in memory performance among people with AD. Likewise, there is considerable variation in executive functioning among people with AD.

Clinicians working with people with AD have long noted that while some of these individuals present with a primary amnestic syndrome, many others have a clinical syndrome marked by deficits in memory (necessary for the diagnosis of AD) accompanied by marked deficits in executive functioning (Storey et al. 2002). For example, Johnson and colleagues (Johnson et al. 1999) identified 3 individuals with significant levels of impairment of executive functioning from a series of 63 clinically documented and pathologically confirmed AD cases. Among people alive in their clinic, they identified 14% of those with clinically diagnosed AD who presented in the mild stages of dementia who showed a similar pattern of executive predominance. Likewise, Binetti et al. (Binetti et al. 1996) reported that 7 of 25 mildly demented patients with AD who were otherwise cognitively indistinguishable from patients with typical AD had severe impairments on tests of executive functioning. Frontal hypometabolism has been identified in a subset of individuals with AD (Haxby et al. 1988). Clinico-pathological correlation studies have demonstrated that patients with confirmed AD pathology can present initially with either the classic amnestic syndrome or a neocortical presentation in which executive-attentional function is notably compromised (Cappa et al. 2001; Galton et al. 2000; Nelson et al. 2012; Nelson et al. 2009).

Dickerson and Wolk (Dickerson and Wolk 2011) noted that an elegant way to identify people living with AD with prominent executive deficits was to compare performance on memory and executive functioning. They labeled people with memory deficits much more profound than executive functioning deficits as having an “amnesic” subtype of AD, and people with executive functioning deficits much more profound than memory deficits as having a “dysexecutive” subtype of AD. They used this framework with data from ADNI, focusing on people with a Clinical Dementia Rating (CDR) Scale of 0.5, consistent with very early AD or mild cognitive impairment (MCI). They found differences between dysexecutive and amnesic groups in terms of neuropsychological performance, neuroimaging, and genetics; the APOE e4 allele was more common in the amnesic group.

Dickerson and Wolk used single indicators of memory and executive functioning to categorize people as amnesic, dysexecutive, or neither. They defined memory performance by the word recognition task from the ADAS-Cog (Mohs et al. 1997), and defined executive functioning performance as the difference between Trails B and Trails A times (Reitan 1958). Single indicator approaches are common in neurobehavioral research, though several investigators have recommended composite scores using modern psychometric methods (reviewed in (Crane et al. 2008).

Elsewhere in this special issue, investigators describe the development and assessment of composite scores for memory (ADNI-Mem, (Crane et al. 2012)) and executive functioning (ADNI-EF, (Gibbons et al. 2012)). ADNI-EF was shown to perform as well as or better than the difference between Trails B and Trails A in a variety of comparisons (Gibbons et al. 2012). The memory paper (Crane et al. 2012) did not evaluate the specific scoring of the ADAS-Cog recognition task used by Dickerson and Wolk.

The first task of the present paper then was to compare and contrast dysexecutive and amnesic subgroups defined by the single indicator approach to similar subgroups defined by using modern psychometric composite scores for memory and executive functioning.

Dickerson and Wolk evaluated the prevalence of APOE genotypes in the dysexecutive and amnesic subgroups, finding a higher prevalence of APOE e4 among those in the amnesic subgroup. We extended this hypothesis by considering the prevalence of single nucleotide polymorphisms (SNPs) associated with risk for AD and SNPs associated with risk for vascular brain disease, here defined specifically as white matter hyperintensities (WMH, (Fornage et al. 2011) and MRI-defined infarcts (Debette et al. 2010). The ADNI cohort was highly selected to have minimal vascular burden (Hachinski scores <=3 at study entry), so we did not anticipate seeing important differences between the amnesic and dysexecutive subgroups in terms of overt signs of vascular brain disease. Nevertheless, genetic factors associated with vascular brain disease may also be associated with subclinical levels of vascular disease that could manifest in people with early AD as prominent deficits in executive functioning. Our general hypothesis is that the amnesic group may have less vascular brain burden and reflect “purer” AD such that genetic associations with AD-related SNPs may be stronger, while the dysexecutive group may have more vascular brain burden and may have genetic profiles consistent with vascular phenotypes.

The second task of the present paper was thus to evaluate SNPs associated with AD and with vascular brain disease. Our hypothesis was that risk alleles for AD would be more commonly found in the amnesic subgroup, while risk alleles for vascular brain disease would be more commonly found in the dysexecutive subgroup.

Methods

Alzheimer’s Disease Neuroimaging Initiative

Data used in the preparation of this article were obtained from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) database (adni.loni.ucla.edu). The ADNI was launched in 2003 by the National Institute on Aging (NIA), the National Institute of Biomedical Imaging and Bioengineering (NIBIB), the Food and Drug Administration (FDA), private pharmaceutical companies and non-profit organizations, as a $60 million, 5-year public-private partnership. The primary goal of ADNI has been to test whether serial magnetic resonance imaging (MRI), positron emission tomography (PET), other biological markers, and clinical and neuropsychological assessment can be combined to measure the progression of mild cognitive impairment (MCI) and early Alzheimer’s disease (AD). Determination of sensitive and specific markers of very early AD progression is intended to aid researchers and clinicians to develop new treatments and monitor their effectiveness, as well as lessen the time and cost of clinical trials.

The Principal Investigator of this initiative is Michael W. Weiner, MD, VA Medical Center and University of California—San Francisco. ADNI is the result of efforts of many co-investigators from a broad range of academic institutions and private corporations, and subjects have been recruited from over 50 sites across the U.S. and Canada. The initial goal of ADNI was to recruit 800 adults, ages 55 to 90, to participate in the research, approximately 200 cognitively normal older individuals to be followed for 3 years, 400 people with MCI to be followed for 3 years and 200 people with early AD to be followed for 2 years. Longitudinal imaging data, including structural 1.5 Tesla MRI scans, were collected on the full sample. Neuropsychological and clinical assessments were collected at baseline, and at follow-up visits of six-to-twelve month intervals. Other available data used in the present analysis included APOE ε4 genotype and genome-wide SNP data obtained as part of a GWAS in the full ADNI sample. Further information about ADNI can be found in (Weiner et al. 2010) and at http://www.adni-info.org. The study was conducted after Institutional Review Board approval at each site. Written informed consent was obtained from all study participants, or their authorized representatives.

ADNI sought to enroll participants with minimal vascular disease burden. It used a modified Hachinski score (Rosen et al. 1980) threshold of 4 points or fewer. In this scale, abrupt onset of cognitive difficulties receives 2 points, stepwise deterioration 1 point, somatic complaints 1 point, emotional incontinence 1 point, history or presence of hypertension 1 point, history of stroke 2 points, focal neurological symptoms 2 points, and focal neurological signs 2 points.

1.5 T MRI Neuroimaging

All participants received 1.5 Tesla structural magnetic resonance imaging (MRI). The neuroimaging methods utilized by ADNI have been described in detail previously (Jack et al., 2008) utilizing calibration techniques to maintain consistent protocols across scanners and sites. Raw dicom data of T1-weighted MP-RAGE scans acquired from 1.5 Tesla scanners at baseline visits from all participants were obtained via the ADNI database (http://www.loni.ucla.edu/ADNI/). Only MRI assessments with an overall quality control of “Pass” were included in these analyses. Images were processed using Freesurfer software (http://surfer.nmr.mgh.harvard.edu), an atlas-based approach that has been validated for use in subjects with a great deal of morphologic variability (Desikan et al. 2006). White matter hyperintensities (WMH) were detected on coregistered T1-, T2-, and PD-weighted images using an automated method described previously (Schwarz et al. 2009; Carmichael et al. 2010). WMH were detected in MDT space at each voxel based on corresponding PD, T1, and T2 intensities there, the prior probability of WMH there, and the conditional probability of WMH there based on the presence of WMH at neighboring voxels. The resulting map of WMH voxels across the brain is summarized by an estimate of total WMH volume. WMH volumes estimated with this method agreed strongly with WMH volumes estimated from fluid-attenuated inversion recovery (FLAIR) MRI in a large, diverse elderly sample (Schwarz et al. 2009). In our analyses we used total hippocampal volume, total WMH volume, and mean left and right thickness data for the following cortical regions: entorhinal, fusiform, pars triangularis, caudal middle frontal, superior frontal, medial orbitofrontal, rostral middle frontal, and lateral orbitofrontal.

Neuropsychological tests

ADNI administers an extensive neuropsychological battery to participants at each study visit, including several measures of memory and executive function.

Memory tasks administered include immediate and delayed recall of Logical Memory from the Wechsler Memory Scale-Revised (Wechsler 1987), a word list learning task from the ADAS-Cog and its delayed recall and recognition (Mohs et al. 1997), the Rey Auditory Verbal Learning Test (Rey 1964), and the recall task from the Mini-Mental State Examination (Folstein et al. 1975). Executive functioning tasks include parts A and B of the Trailmaking task (Reitan 1958), a clock drawing task (Goodglass and Kaplan 1983), animal and vegetable category fluency (Morris et al. 1989), digit span backwards from the Wechsler Memory Scale-Revised (Wschsler 1987), and the digit-symbol substitution task from the Wechsler Adult Intelligence Test-Revised (Wechsler 1981).

Single indicator approach to memory and executive functioning

For memory, Dickerson and Wolk focused on the recognition task from the ADAS-Cog (Mohs et al. 1997), scored using techniques derived from signal detection theory. For executive functioning, they used the difference between times for Trails B (which includes both letters and numbers) and Trails A (which includes only numbers) to isolate the executive component of Trails B (i.e. set shifting between numbers and letters) from the motor sequencing component (i.e. time to move the pencil across the page in numerical order).

Modern psychometric approach to memory and executive functioning

For details regarding the development of ADNI-Mem and ADNI-EF, please refer to the companion papers in this volume (Crane et al. 2012). Briefly, we used modern psychometric theory methods applied to item-level data from the ADNI neuropsychological battery to develop composite scores separately for memory and executive functioning. ADNI-Mem and ADNI-EF scores are available along with other ADNI on request from ADNI.

Comparison of CDR 0.5 and CDR 1.0

Dickerson and Wolk limited their analyses to those with MCI or AD with a CDR of 0.5. We compared the scatter plot of memory vs. executive functioning scores with people with AD with CDR of 0.5 (light symbols) and CDR of 1.0 (dark symbols) (Figure 1). We did not see a clear pattern of better scores for people with CDR of 0.5 compared to people with CDR 1.0, so we combined all people with AD in subsequent analyses. (ADNI was restricted to participants with CDR 1.0 or less at baseline.)

Figure 1
Scatter plot of memory and executive functioning among people with AD in ADNI. Lighter symbols correspond to people with CDR = 0.5 and darker symbols correspond to people with CDR = 1.0. People categorized as amnesic are shown in red, people categorized ...

Comparison of modern psychometric composites to the single indicator scores

We plotted memory scores and executive functioning scores derived using a single indicator approach against the psychometric composites for memory (ADNI-Mem) and executive functioning (ADNI-EF), and determined the correlation between these scores.

We compared the single indicator and the modern psychometric composite scores in terms of the rate of change over time among people with AD, and their strength of association with neuroimaging parameters selected a priori as likely to be associated with memory (i.e. hippocampal volume, thickness of the entorhinal cortex, and thickness of the fusiform gyrus) and with executive functioning (i.e. the natural log of the volume of white matter hyperintensities, and thickness of caudal portion of the middle frontal cortex, superior frontal cortex, rostral middle frontal cortex, lateral orbitofrontal cortex, medial orbitofrontal cortex, and pars triangularis) among people with AD. We estimated rates of change in mixed models with random intercepts and slopes and an unstructured covariance matrix, controlling for age, education, sex and the presence of any APOE ε4 alleles. We converted visit month to years for use as the measure of time. We compared the resulting z-statistics (coefficient/standard error) for year. We used the coefficient for year and the adjusted residual standard deviation from each model to determine sample sizes needed to detect a 25% reduction in the rate of decline in 12 months, with 80% power and α = 0.05, two-sided. We determined association with baseline MRI measures using regression models controlling for age, education, sex, presence of any APOE ε4 alleles, and intracranial volume, comparing z-statistics for the MRI predictor. White matter hyperintensities were transformed to the log scale.

Categorization of ADNI participants as amnesic, dysexecutive, or neither

Dickerson and Wolk used data from ADNI participants with normal cognition to generate a mean and standard deviation (SD) for their memory score and the difference between Trails B and Trails A. They used these values to calculate z-scores for memory and executive functioning for all ADNI participants. They defined the dysexecutive subtype as having an executive performance z-score ≥ 2 SDs worse than memory performance, and the amnesic subtype as having a memory performance z-score ≥ 2 SDs worse than executive functioning performance. For all graphs and analyses in this paper, we have reversed scores as needed such that higher values always indicate better performance.

To categorize participants as dysexecutive or amnesic using the composite psychometric approach, we determined the number of people identified as dysexecutive or amnesic by the single indicator approach, and set thresholds for the difference between ADNI-Mem and ADNI-EF to identify the same number of people.

We set z equal to the difference of ADNI-EF and ADNI-Mem, and chose thresholds of < 0.6 for dysexecutive and > 2.395 for amnesic. We chose this data-driven approach to ensure that differences in findings between the single indicator approach and the modern psychometric composite approach would not be due to different sample sizes. We determined agreement beyond chance between these assignments using the kappa coefficient (Landis and Koch 1977).

Acquisition of genotype data

Methods for acquisition and processing of genotype data for the ADNI sample have been previously described (Saykin et al. 2010). The Human610-Quad BeadChip (Illumina, Inc., San Diego, CA) was used to analyze samples with all sources of DNA according to the manufacturer’s protocol (Infinium HD Assay; Super Protocol Guide; rev. A, May 2008). SNP genotypes were generated from bead intensity data using Illumina BeadStudio 3.2 software. The two APOE SNPs (rs429358, rs7412) that define the ε2, ε3 and ε4 alleles were not available on the Illumina Human610-Quad BeadChip. These SNPs were genotyped separately and are available in the ADNI database (Potkin et al. 2009).

Selection of AD-and vascular-related SNPs and genetic analyses

We identified AD-related SNPs using data made available by the AD Genetics Consortium (Naj et al. 2011). There are 10 loci identified for AD (Naj et al. 2011), each of which has several SNPs with suggestive p-values. We selected the most impressive of these—ensuring we had representation from all 10 loci—from the supplemental material available from (Naj et al. 2011). We identified SNPs associated with brain vascular disease defined as WMH (Fornage et al. 2011) and MRI-identified lacunes (Debette et al. 2010) using the same approach. In all, we analyzed SNP data from 132 markers (Supplementary Table 1) plus APOE.

We hypothesized that APOE ε4 and alleles associated with increased risk of AD would be associated with higher risk of membership in the amnesic subgroup; alleles associated with protection from AD would be associated with higher risk of membership in the dysexecutive subgroup. Similarly, we hypothesized that alleles associated with greater burden of WMH or MRI-identified infarcts would be associated with higher risk of membership in the dysexecutive subgroup, while alleles associated with less burden of WMH or infarcts would be associated with higher risk of membership in the amnesic subgroup. A priori, we picked values of greater than 1.5 (or less than 0.67) as notable odds ratios. We determined the number of SNPs for which either the single indicator approach or the modern psychometric composite approach produced odds ratios more extreme than these values, and considered the proportion of those SNPs for which the direction of association was in the hypothesized direction.

Results

Demographic and clinical characteristics of the sample

There were 183 people with AD with complete data for memory and executive functioning. Of these, 155 (85%) were white, with complete genetic data and were included in the genetic analyses.

Demographic and clinical data for the cohort analyzed are shown in the top section of Table 1, stratified by their categorization as amnesic vs. dysexecutive using the psychometric composite approach. People categorized as dysexecutive were somewhat younger on average than people categorized as amnesic, but this difference was not statistically significant. Years of education and dementia severity were similar; dementia severity was measured by the proportion with CDR 0.5 vs. CDR 1 and by the CDR sum of boxes. On average WMH burden was greater among the dysexecutive subgroup than the amnesic subgroup. As expected with the low Hachinski score exclusion criterion, the number of individuals at baseline with MRI-identified infarcts was very small.

Table 1
Comparison of people with AD categorized on the basis of ADNI-Mem and ADNI-EF into amnesic, neither, or dysexecutive

Comparison of single indicator and modern psychometric composite memory and executive functioning scores

In Figure 2 we show scatter plots of memory (2a) and executive function (2b) as measured by the single indicator approach (Y axis) and as measured by the psychometric composite approach (X axis). The correlation between the two memory scores was 0.46, and the correlation between the two executive functioning scores was 0.58. These moderate-sized correlations suggest that there may be important differences between these two scoring approaches. In Figure 2a, the distribution of red dots along the Y axis indicates that people categorized as amnesic by the single indicator approach had a broad range of ADNI-Mem scores. The distribution of green dots along the Y axis similarly indicates that people categorized as dysexecutive by the single indicator approach had a broad range of ADNI-Mem scores. There is broad overlap of ADNI-Mem scores for people categorized by the single indicator approach as amnesic and people categorized as dysexecutive. In Figure 2b, the somewhat higher correlation between the two executive functioning scores is marked by a stronger diagonal relationship between the two scores. Many people categorized as amnesic subtype by the single indicator approach have higher ADNI-EF scores than people categorized as dysexecutive, but there is still broad overlap in the distribution of ADNI-EF scores for those categorized as amnesic and those categorized as dysexecutive by the single indicator approach.

Figure 2Figure 2
Figure 2a. Scatter plot of ADNI-Mem (X axis) and memory scores derived from single indicator approach (Y axis). Color coding indicates assignment to dysexecutive (green), amnesic (red), and neither (gray) category based on single indicator approach.

Table 2 shows z-statistics for associations between memory and executive functioning scores and several volumes and thicknesses derived from baseline MRIs. Values greater than 1.96 (or less than −1.96) are statistically different from 0 at the p-value < 0.05 level, and are shown in bold in the table. In all comparisons, ADNI-Mem and ADNI-EF measures were more strongly associated with all the imaging parameters we considered than the scores derived by the single indicator approach.

Table 2
Z-statistics for the association of memory and executive functioning scores with a priori selected MRI measures, from regression models controlling for age, education, gender, any APOE ε4 alleles, and intracranial volume. Bolded scores indicate ...

The relationships in these analyses are all cross-sectional. We were also interested in whether candidate scores for memory and executive functioning would be responsive to changes over time. We were not able to calculate the single indicator memory score used by Dickerson and Wolk at time points after baseline because three different word lists were used for the ADAS-Cog (Crane et al. 2012). We were able to determine responsiveness to change for ADNI-EF and the single indicator measure for executive functioning. For ADNI-EF, the effect size was −12.34, corresponding to a sample size of 374 needed to detect a 25% reduction in the rate of decline over two years. The effect size for the single indicator measure of executive functioning was −5.90, corresponding to a sample size of 2183 (over six times as many) needed to detect a 25% reduction over two years.

In summary, the two ways of measuring memory were moderately correlated, as were the two ways of measuring executive functioning. ADNI-Mem and ADNI-EF had stronger associations with nearly all of the a priori selected imaging parameters than their single indicator counterparts. ADNI-EF also had a higher standardized coefficient for change over time, suggesting increased responsiveness.

Categorizations into amnesic and dysexecutive groups

Figure 1 above showed the scatter plot of single indicator scores for memory and executive functioning. In Figure 2 we show scatter plots of ADNI-Mem and ADNI-EF. Figure 3 shows results when we categorized people using the psychometric composite analogously to the single indicator approach by categorizing people at the extreme top left of this plot as “dysexecutive” and people at the extreme bottom right as “amnesic”.

Figure 3
Scatter plot of ADNI-Mem and ADNI-EF. Red dots represent people categorized as amnesic and green dots represent people categorized as dysexecutive using the psychometric composite approach.

The middle section of Table 1 shows the cross-tabulation of assignments to amnesic, dysexecutive, or neither using the psychometric composite and the single indicator approaches. Only 57% of the people with AD were assigned to the same category (i.e., amnesic, dysexecutive, or neither) across the two categorization schemes. Much of this agreement could be attributed to chance alone, as the kappa statistic was only 0.23, categorized as a “fair” level of agreement beyond chance.

Genetic analyses

As shown in Table 1, a higher proportion of people classified as amnesic had one or more APOE ε4 alleles (71%) than did people classified as dysexecutive (50%) when they were categorized using psychometric composite approach. When people were categorized using the single indicator approach, 64% of those categorized as amnesic had one or more APOE ε4 allele, and 50% of those categorized as dysexecutive had one or more APOE ε4 allele; these proportions were not statistically different from each other.

Other than APOE, we identified 81 AD-related SNPs (Naj et al. 2011), 31 WMH-related SNPs (Fornage et al. 2011), and 18 MRI-identified infarct-related SNPs (Debette et al. 2010). We determined the strength of association with the amnesic vs. dysexecutive categories using the single indicator and the psychometric composite approaches. Complete results for all SNPs are shown in the Supplementary Table.

Of the 81 AD-related SNPs, 58 had an odds ratio greater than 1.5 or less than 0.67 for membership in the dysexecutive vs. amnesic subgroup as defined by either the single indicator or the psychometric composite approach (or by both approaches). We show a plot for these 58 SNPs in Figure 4. Results in the hypothesized direction—when alleles associated with greater AD risk were also associated with greater risk of membership in the amnesic subgroup—are shown with positive values. Results in the opposite of the hypothesized direction—when alleles associated with greater AD risk were associated with greater risk of membership in the dysexecutive subgroup—are shown with negative values. The single indicator approach produced odds ratios in the hypothesized direction for 22 of these SNPs (38%), while the psychometric composite approach produced odds ratios in the hypothesized direction for 34 of these SNPs (59%, Fisher’s exact p-value for the comparison between the two approaches < 0.001).

Figure 4
Odds ratios for AD-related SNPs for amnesic vs. dysexecutive subgroups of people with Alzheimer’s disease using the single indicator (light blue and dark blue circles) and the psychometric composite (gray and black squares) categorizations*

The single SNP with the most extreme odds ratio was rs4420638 in the APOC gene on chromosome 19. The minor allele at that SNP was associated with increased risk of AD (Naj et al. 2011), so we hypothesized that it would be associated with the amnesic subgroup. For the single indicator approach, presence of one or more minor alleles of this SNP were associated with a 5-fold increased risk of the dysexecutive subgroup—strongly in the opposite of the hypothesized direction—while for the psychometric composite categorization, presence of one or more minor alleles of this SNP were associated with a 1.3-fold increased risk of membership in the amnesic subgroup—weakly in the hypothesized direction.

Of the 49 WMH- and MRI-identified infarct-related SNPs, 25 had an odds ratio greater than 1.5 or less than 0.67 for membership in the dysexecutive vs. amnesic subgroup as defined by either the single indicator or the psychometric composite approach (or by both approaches). We show a plot for these 25 SNPs in Figure 5. We hypothesized that alleles associated with increased brain vascular disease defined as WMH or MRI-detected infarcts would be associated with an increased risk for membership in the dysexecutive subgroup. SNPs with odds ratios in the hypothesized direction are shown with positive values in Figure 5. The single indicator approach produced odds ratios in the hypothesized direction for 8 of these SNPs (32%), while the psychometric composite approach produced odds ratios in the hypothesized direction for 12 of these SNPs (48%, Fisher’s exact p-value for the comparison of the two approaches = 0.01).

Figure 5
Odds ratios for WMH-related (to left) and MRI-defined infarct-related (to right) SNPs for amnesic vs. dysexecutive subgroups of people with Alzheimer’s disease using the single indicator (light and dark blue circles) and the psychometric composite ...

Adjusting these analyses by controlling for WMH and MRI-defined infarcts made little influence on results (Appendix Table C).

Discussion

This paper extends the investigation of the dysexecutive grouping of ADNI participants initiated by Dickerson and Wolk (Dickerson and Wolk 2011). We compared single indicators of memory and executive functioning to modern psychometric composite scores for memory and executive functioning. In nearly all comparisons, the modern psychometric composite scores appear superior to the single indicator scores. We then used both scoring approaches to assign people to the amnesic subgroup, the dysexecutive subgroup, or neither. Agreement between the categorizations made by the two approaches was only fair, with a kappa value of 0.23. We then evaluated the strength of association between AD-related and brain vascular disease-related SNPs and subgroups defined by the single indicator and the modern psychometric composite approaches. We found significantly higher numbers of cases in which AD risk alleles were associated with the amnesic subgroup using the modern psychometric composite approach than using the single indicator approach. Similarly, we found significantly higher numbers of cases in which brain vascular disease risk alleles were associated with the dysexecutive subgroup using the modern psychometric composite approach than using the single indicator approach.

There are several possibilities to explain why the modern psychometric composite approach found more genetic associations in the hypothesized direction than the single indicator approach. First, our methodology for identifying the amnesic and dysexecutive predominant phenotypes was based on modern psychometric methods. As outlined in the ADNI-Mem and ADNI-EF papers elsewhere in this special issue (Crane et al. 2012; Gibbons et al. 2012), composite scores include broader representation of the underlying construct than any particular subtest. The single indicators of measures of memory and executive functioning had only moderate correlations with ADNI-Mem and ADNI-EF, and had poorer performance in the various validity assessments. We suggest that ADNI-Mem and ADNI-EF may simply be better measures of memory and of executive functioning. As noted elsewhere in this special issue, several factors likely contribute to the superior performance of psychometrically sophisticated composite scores compared with the single indicator approach, including the use of multiple indicators that should reduce noise, and improved measurement properties across the range of abilities measured (Crane et al. 2012; Gibbons et al. 2012). One of the ways they are better is that they resulted in categories of dysexecutive and amnesic individuals that are statistically associated with genetic indicators in hypothesized directions.

Secondly, there may be limitations to the specific single indicators chosen. The ADAS-Cog Word Recognition score, especially when used in isolation, may not be optimal for a number of reasons. Retrieval scores appear to be more sensitive than recognition scores for early impairments. Indeed, Benge and colleagues found that the recognition score was the poorest of five candidates from the ADAS-Cog for early impairments (Benge et al. 2009). The use of a difference score between Trails B and A has some support in the research literature as being a good marker for set-shifting (an aspect of executive function) after controlling for visuomotor speed alone (Arbuthnott and Frank 2000; Sanchez-Cubillo et al. 2009), but a global measure may better represent the overall concept of executive functioning.

Another possibility is that our findings may be due to chance. Similar investigations should be performed in one or more additional data sets to determine if our findings are replicable.

Our results support the underlying hypothesis that there may be different genetic architecture underlying differences in memory and executive functioning abilities among people with AD (Dickerson and Wolk 2011). These findings suggest that there may be substantial genetic heterogeneity among people with clinical AD. This is consistent with clinicopathological correlation studies which have shown relative heterogeneity of cognitive profiles, such that amnestic and dysexecutive features may be on continua, and can be associated with the same AD pathological diagnosis (Galton et al. 2000; Nelson et al. 2012; Nelson et al. 2009).

ADNI tried to minimize the influence of vascular pathology, and used the Hachinski ischemic index to exclude participants if they had significant vascular disease burden at study baseline. Our analyses are based on the initial baseline visit, so incident vascular events cannot explain our findings. As part of our sensitivity analyses we specifically included covariates for imaging-identified WMH, infarctions, and Hachinski score, so these vascular factors are not driving our findings. In short: we do not think that vascular pathology—at least as captured by the Hachinski score, infarctions, and WMH visible on MRI—are driving our findings. Of course, we cannot rule out the possible role of sub-clinical vascular pathology. Indeed, this may be precisely what is reflected by the effects of WMH and infarction-related SNPs.

In the current study, we found striking suggestions that the dysexecutive subgroup appears to be more common among people with vascular brain disease risk alleles—even controlling for vascular brain disease—while the amnesic subgroup appears to be more common among people with AD risk alleles. This finding has important implications. These implications include the possibility that other currently unrecognized genetic loci may be associated with these phenotypes, which may in turn lead to drug development that may improve the lives of people currently categorized as having “Alzheimer’s disease”. Also, if different genetic architecture underlies observed differences in memory and executive functioning among people with AD, and these genetic factors identify individuals with different susceptibility to medications designed to modify the underlying biology, then recruitment into trials should take these subgroups into account to ensure at least that effects are consistent across subgroups. As our knowledge and understanding of these phenomena grow, it may be appropriate to limit enrollment to the specific subgroup that may benefit from the intervention.

Some limitations should be considered. Our sample sizes were small, which leads to some instability in odds ratios. Nevertheless, our sample sizes were sufficient to detect signals with the psychometric composite approach that were not detectable with the single indicator approach. While our SNPs were highly selected as those having the strongest relationships to AD, infarcts, or WMH, they were selected for these analyses a priori and were not selected on the basis of their strength of relationship with amnesic or dysexecutive subgroups. If the finding that the genetic architecture of subgroups of AD cases is distinct is confirmed in other similar analyses of AD-related and brain vascular disease related SNPs, then this should be followed-up with very large samples to see whether there may be additional loci associated with these phenotypes. Our sample size was certainly too small for a genome-wide search; one would need to pool resources across multiple studies to have sufficient power for genome-wide significance levels. As of today, scoring algorithms for modern psychometric composite scores are not available at the point of care. Ongoing initiatives may address this in the future. At present, however, this approach of necessity is limited to the research setting.

Our results provide support for two conclusions. First, there may be improved power to address scientific conclusions when using psychometrically sophisticated approaches to measuring memory and executive functioning. Second, these results provide additional support for the hypothesis that genetic architecture may be to some extent responsible for differences between memory and executive functioning observed among people with AD. This hypothesis should be tested in other study settings. If confirmed, these findings have important implications for research on AD, both in terms of the possibility of searching for new genetic loci that may serve as the basis for future drug discovery to improve the lives of people with AD, and in terms of clinical study design to ensure that people enrolled in trials are at equal risk to benefit from interventions.

figure nihms423424f6afigure nihms423424f6b
Figure A. Scatter plot of ADNI-Mem and ADNI-EF (as in Figure 3). Red dots represent people categorized as amnesic and green dots people categorized as dysexecutive as derived from single indicator approach.

Supplementary Material

11682_2012_9207_MOESM1_ESM

Acknowledgments

Data collection and sharing for this project was funded by the Alzheimer’s Disease Neuroimaging Initiative (ADNI) (National Institutes of Health Grant U01 AG024904). ADNI is funded by the National Institute on Aging, the National Institute of Biomedical Imaging and Bioengineering, and through generous contributions from the following: Abbott; Alzheimer’s Association; Alzheimer’s Drug Discovery Foundation; Amorfix Life Sciences Ltd.; AstraZeneca; Bayer HealthCare; BioClinica, Inc.; Biogen Idec Inc.; Bristol-Myers Squibb Company; Eisai Inc.; Elan Pharmaceuticals Inc.; Eli Lilly and Company; F. Hoffmann-La Roche Ltd and its affiliated company Genentech, Inc.; GE Healthcare; Innogenetics, N.V.; IXICO Ltd.; Janssen Alzheimer Immunotherapy Research & Development, LLC.; Johnson & Johnson Pharmaceutical Research & Development LLC.; Medpace, Inc.; Merck & Co., Inc.; Meso Scale Diagnostics, LLC.; Novartis Pharmaceuticals Corporation; Pfizer Inc.; Servier; Synarc Inc.; and Takeda Pharmaceutical Company. The Canadian Institutes of Health Research is providing funds to support ADNI clinical sites in Canada. Private sector contributions are facilitated by the Foundation for the National Institutes of Health (www.fnih.org). The grantee organization is the Northern California Institute for Research and Education, and the study is coordinated by the Alzheimer’s Disease Cooperative Study at the University of California, San Diego. ADNI data are disseminated by the Laboratory for Neuro Imaging at the University of California, Los Angeles. This research was also supported by NIH grants P30 AG010129 and K01 AG030514.

APPENDIX

Table A

Baseline neuropsychological test scores for the two AD subtypes (dysexecutive and amnesic) as derived by the two approaches (psychometric composite and single indicator).

Dysexecutive (n=45)Amnesic (n=27)
Trails B277.58 (49.13)115.85 (56.28)
Trails A97.64 (36.54)38.85 (11.76)
Category Fluency (Animals)10.44 (4.07)14.59 (4.52)
Category Fluency (Vegetables)6.91 (2.94)9.30 (3.18)
Digit Span Backward4.16 (1.55)5.73 (2.09)
Digit Span Forward7.47 (1.74)8 (1.86)
Digit Symbol Total Correct15.64 (8.62)42.85 (8.97)
Logical Memory - Immediate Recall4.64 (3.31)3.85 (2.52)
Logical Memory - Delayed Recall2.18 (2.40).44 (1.54)
ADAS-Cog Word Recall6.03 (1.46)6.02 (1.13)
ADAS-Cog Delayed Word Recall7.96 (1.89)9.30 (3.18)
ADAS-Cog Word Recognition5.91 (2.73)6.89 (2.76)
ADAS-Cog Total18.25 (6.09)17.84 (4.29)
REY AVLT Total Trials 1–524.62 (7.85)24.42 (5.22)
Rey AVLT List B Total2.6 (1.39)2.77 (1.18)
Rey AVLT delayed recall2.48 (2.49)1.38 (1.30)

Table B

Correlation between the psychometric composites/single indicators for memory and executive functioning with neuroimaging ROIs

MemoryADNI-MemSI-Mem
 Hippocampal volume0.270.22
 Entorhinal thickness0.240.30
 Parahippocampal thickness0.170.18
Executive functioningADNI-EFTrails difference
 White matter hyperintensity volume (natural log)−0.16−0.02
 Caudal middle frontal thickness0.250.07
 Superior frontal thickness0.160.02
 Medial orbitofrontal thickness0.180.05
 Rostral middle frontal thickness0.18−0.02
 Pars triangularis thickness0.160.04
 Lateral orbitofrontal0.14−0.02

Table C

Genetic association results adjusting for WMH and infarcts.

Top 10 SNPs for GWAS adjusting for PCs, WMH & infarctsTop 10 SNPs for GWAS adjusting for PCs & WMHTop 10 SNPs for GWAS adjusting for PCs & infarcts

GWAS case-control analysis results for subtypes derived using the psychometric composite approach

CHRSNPP-valueCHRSNPP-valueCHRSNPP-value
19rs81069220.0094rs117314360.00919rs29274380.012
19rs29274380.01319rs29274380.01519rs81069220.013
4rs117314360.01519rs81069220.01814rs23183080.014
10rs49484820.01610rs25889620.0184rs117314360.016
10rs25889660.01610rs25889640.01810rs25889620.024
10rs13735220.01610rs49484820.01810rs25889640.024
10rs25889640.01610rs25889660.01810rs49484820.024
10rs25889620.01610rs13735220.01810rs25889660.024
14rs23183080.01714rs23183080.01910rs13735220.024
10rs25889690.02510rs25889690.02319rs116731390.027

GWAS case-control analysis results for subtypes derived using the single indicator approach

CHRSNPP-valueCHRSNPP-valueCHRSNPP-value

19rs29274380.01719rs29274380.01719rs29274380.012
11rs49389330.03319rs81002390.03211rs49389330.045
19rs81002390.03311rs49389330.03319rs81002390.046
19rs29651010.03919rs29651010.03910rs25889640.047
10rs25889640.04612rs118335790.04519rs29651010.048
11rs118247340.04610rs25889640.04619rs37522460.050
11rs118247730.04611rs49393380.04611rs118247730.056
11rs107922630.04611rs107922630.04611rs49393380.056
11rs49393380.04611rs118247730.04611rs118247340.056
12rs118335790.05111rs118247340.04611rs107922630.056

Footnotes

*Data used in preparation of this article were obtained from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) database (adni.loni.ucla.edu). As such, the investigators within the ADNI contributed to the design and implementation of ADNI and/or provided data but did not participate in analysis or writing of this report. A complete listing of ADNI investigators can be found at: http://adni.loni.ucla.edu/wp-content/uploads/how_to_apply/ADNI_Acknowledgement_List.pdf

References

  • American Psychiatric Association. Task Force on DSM-IV. Diagnostic and statistical manual of mental disorders : DSM-IV. 4. Washington, DC: American Psychiatric Association; 1994.
  • Arbuthnott K, Frank J. Trail making test, part B as a measure of executive control: validation using a set-switching paradigm. Journal of clinical and experimental neuropsychology. 2000;22(4):518–528. doi: 10.1076/1380-3395(200008)22:4;1-0;FT518. [PubMed] [Cross Ref]
  • Benge JF, Balsis S, Geraci L, Massman PJ, Doody RS. How well do the ADAS-cog and its subscales measure cognitive dysfunction in Alzheimer’s disease? Dementia and geriatric cognitive disorders. 2009;28(1):63–69. doi: 10.1159/000230709. [PubMed] [Cross Ref]
  • Binetti G, Magni E, Padovani A, Cappa SF, Bianchetti A, Trabucchi M. Executive dysfunction in early Alzheimer’s disease. Journal of neurology, neurosurgery, and psychiatry. 1996;60(1):91–93. [PMC free article] [PubMed]
  • Cappa A, Calcagni ML, Villa G, Giordano A, Marra C, De Rossi G, et al. Brain perfusion abnormalities in Alzheimer’s disease: comparison between patients with focal temporal lobe dysfunction and patients with diffuse cognitive impairment. [Comparative Study] Journal of neurology, neurosurgery, and psychiatry. 2001;70(1):22–27. [PMC free article] [PubMed]
  • Carmichael O, Schwarz C, Drucker D, Fletcher E, Harvey D, Beckett L, et al. Longitudinal changes in white matter disease and cognition in the first year of the Alzheimer disease neuroimaging initiative. Archives of neurology. 2010;67(11):1370–1378. doi: 10.1001/archneurol.2010.284. [PMC free article] [PubMed] [Cross Ref]
  • Crane PK, Carle A, Gibbons LE, Insel P, Mackin RS, Gross A, et al. Development and assessment of a composite score for memory in the Alzheimer’s Disease Neuroimaging Initiative (ADNI) Brain imaging and behavior. 2012 doi: 10.1007/s11682-012-9186-z. [PMC free article] [PubMed] [Cross Ref]
  • Crane PK, Narasimhalu K, Gibbons LE, Pedraza O, Mehta KM, Tang Y, et al. Composite scores for executive function items: demographic heterogeneity and relationships with quantitative magnetic resonance imaging. [Research Support, N.I.H., Extramural Research Support, Non-U.S. Gov’t] Journal of the International Neuropsychological Society : JINS. 2008;14(5):746–759. doi: 10.1017/S1355617708081162. [PMC free article] [PubMed] [Cross Ref]
  • Debette S, Bis JC, Fornage M, Schmidt H, Ikram MA, Sigurdsson S, et al. Genome-wide association studies of MRI-defined brain infarcts: meta-analysis from the CHARGE Consortium. Stroke. 2010;41(2):210–217. doi: 10.1161/STROKEAHA.109.569194. STROKEAHA.109.569194 [pii] [PMC free article] [PubMed] [Cross Ref]
  • Desikan RS, Segonne F, Fischl B, Quinn BT, Dickerson BC, Blacker D, et al. An automated labeling system for subdividing the human cerebral cortex on MRI scans into gyral based regions of interest. [Research Support, N.I.H., Extramural Research Support, Non-U.S. Gov’t] NeuroImage. 2006;31(3):968–980. doi: 10.1016/j.neuroimage.2006.01.021. [PubMed] [Cross Ref]
  • Dickerson BC, Wolk DA. Dysexecutive versus amnesic phenotypes of very mild Alzheimer’s disease are associated with distinct clinical, genetic and cortical thinning characteristics. J Neurol Neurosurg Psychiatry. 2011;82(1):45–51. [PMC free article] [PubMed]
  • Folstein MF, Folstein SE, McHugh PR. “Mini-mental state”. A practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res. 1975;12(3):189–198. [PubMed]
  • Fornage M, Debette S, Bis JC, Schmidt H, Ikram MA, Dufouil C, et al. Genome-wide association studies of cerebral white matter lesion burden: the CHARGE consortium. Ann Neurol. 2011;69(6):928–939. doi: 10.1002/ana.22403. [PMC free article] [PubMed] [Cross Ref]
  • Galton CJ, Patterson K, Xuereb JH, Hodges JR. Atypical and typical presentations of Alzheimer’s disease: a clinical, neuropsychological, neuroimaging and pathological study of 13 cases. [Research Support, Non-U.S. Gov’t] Brain : a journal of neurology. 2000;123(Pt 3):484–498. [PubMed]
  • Gibbons LE, Carle AC, Mackin RS, Harvey D, Mukherjee S, Insel P, et al. A composite score for executive functioning, validated in Alzheimer’s Disease Neuroimaging Initiative (ADNI) participants with baseline mild cognitive impairment. Brain imaging and behavior. 2012 doi: 10.1007/s11682-012-9176-1. [PMC free article] [PubMed] [Cross Ref]
  • Goodglass H, Kaplan D. The assessment of aphasia and related disorders. 2. Philadelphia: Lea & Febiger; 1983.
  • Haxby JV, Grady CL, Koss E, Horwitz B, Schapiro M, Friedland RP, et al. Heterogeneous anterior-posterior metabolic patterns in dementia of the Alzheimer type. Neurology. 1988;38(12):1853–1863. [PubMed]
  • Johnson JK, Head E, Kim R, Starr A, Cotman CW. Clinical and pathological evidence for a frontal variant of Alzheimer disease. [Research Support, U.S. Gov’t, P.H.S.] Archives of neurology. 1999;56(10):1233–1239. [PubMed]
  • Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33(1):159–174. [PubMed]
  • McKhann G, Drachman D, Folstein M, Katzman R, Price D, Stadlan EM. Clinical diagnosis of Alzheimer’s disease: report of the NINCDS-ADRDA Work Group under the auspices of Department of Health and Human Services Task Force on Alzheimer’s Disease. Neurology. 1984;34(7):939–944. [PubMed]
  • Mohs RC, Knopman D, Petersen RC, Ferris SH, Ernesto C, Grundman M, et al. Development of cognitive instruments for use in clinical trials of antidementia drugs: additions to the Alzheimer’s Disease Assessment Scale that broaden its scope. The Alzheimer’s Disease Cooperative Study. Alzheimer Dis Assoc Disord. 1997;11(Suppl 2):S13–21. [PubMed]
  • Morris JC, Heyman A, Mohs RC, Hughes JP, van Belle G, Fillenbaum G, et al. The Consortium to Establish a Registry for Alzheimer’s Disease (CERAD). Part I. Clinical and neuropsychological assessment of Alzheimer’s disease. Neurology. 1989;39(9):1159–1165. [PubMed]
  • Naj AC, Jun G, Beecham GW, Wang LS, Vardarajan BN, Buros J, et al. Common variants at MS4A4/MS4A6E, CD2AP, CD33 and EPHA1 are associated with late-onset Alzheimer’s disease. Nat Genet 2011 [PMC free article] [PubMed]
  • Nelson PT, Alafuzoff I, Bigio EH, Bouras C, Braak H, Cairns NJ, et al. Correlation of Alzheimer disease neuropathologic changes with cognitive status: a review of the literature. [Research Support, N.I.H., Extramural Research Support, Non-U.S. Gov’t Review] Journal of neuropathology and experimental neurology. 2012;71(5):362–381. doi: 10.1097/NEN.0b013e31825018f7. [PMC free article] [PubMed] [Cross Ref]
  • Nelson PT, Braak H, Markesbery WR. Neuropathology and cognitive impairment in Alzheimer disease: a complex but coherent relationship. [Research Support, N.I.H., Extramural Research Support, Non-U.S. Gov’t Review] Journal of neuropathology and experimental neurology. 2009;68(1):1–14. doi: 10.1097/NEN.0b013e3181919a48. [PMC free article] [PubMed] [Cross Ref]
  • Potkin SG, Guffanti G, Lakatos A, Turner JA, Kruggel F, Fallon JH, et al. Hippocampal atrophy as a quantitative trait in a genome-wide association study identifying novel susceptibility genes for Alzheimer’s disease. PLoS ONE. 2009;4(8):e6501. doi: 10.1371/journal.pone.0006501. [PMC free article] [PubMed] [Cross Ref]
  • Reitan RM. Validity of the Trail Making Test as an indicator of organic brain damage. Percept Mot Skills. 1958;8:271–276.
  • Rey A. L’examen clinique en psychologie. Paris: Presses Universitaires de France; 1964.
  • Rosen WG, Terry RD, Fuld PA, Katzman R, Peck A. Pathological verification of ischemic score in differentiation of dementias. [Research Support, U.S. Gov’t, P.H.S.] Annals of neurology. 1980;7(5):486–488. doi: 10.1002/ana.410070516. [PubMed] [Cross Ref]
  • Sanchez-Cubillo I, Perianez JA, Adrover-Roig D, Rodriguez-Sanchez JM, Rios-Lago M, Tirapu J, et al. Construct validity of the Trail Making Test: role of task-switching, working memory, inhibition/interference control, and visuomotor abilities. Journal of the International Neuropsychological Society : JINS. 2009;15(3):438–450. doi: 10.1017/S1355617709090626. [PubMed] [Cross Ref]
  • Saykin AJ, Shen L, Foroud TM, Potkin SG, Swaminathan S, Kim S, et al. Alzheimer’s Disease Neuroimaging Initiative biomarkers as quantitative phenotypes: Genetics core aims, progress, and plans. Alzheimers Dement. 2010;6(3):265–273. doi: 10.1016/j.jalz.2010.03.013. S1552-5260(10)00082-8 [pii] [PMC free article] [PubMed] [Cross Ref]
  • Schwarz C, Fletcher E, DeCarli C, Carmichael O. Fully-automated white matter hyperintensity detection with anatomical prior knowledge and without FLAIR. Information processing in medical imaging. 2009;21:239–251. [PMC free article] [PubMed]
  • Storey E, Slavin MJ, Kinsella GJ. Patterns of cognitive impairment in Alzheimer’s disease: assessment and differential diagnosis. [Review] Frontiers in bioscience : a journal and virtual library. 2002;7:e155–184. [PubMed]
  • Weiner MW, Aisen PS, Jack CR, Jr, Jagust WJ, Trojanowski JQ, Shaw L, et al. The Alzheimer’s disease neuroimaging initiative: progress report and future plans. [Historical Article Research Support, Non-U.S. Gov’t Review] Alzheimer’s & dementia : the journal of the Alzheimer’s Association. 2010;6(3):202–211. e207. doi: 10.1016/j.jalz.2010.03.007. [PMC free article] [PubMed] [Cross Ref]
  • Wechsler D. Wechsler Adult Intelligence Scale-Revised. NY: Psychological Corporation; 1981.
  • Wechsler D. WMS-R: Wechsler Memory Scale - Revised manual. NY: Psychological Corporation/HBJ; 1987.