|Home | About | Journals | Submit | Contact Us | Français|
Practice effects on cognitive tests have been shown to further characterize patients with amnestic Mild Cognitive Impairment (aMCI), and may provide predictive information about cognitive change across time. We tested the hypothesis that a loss of practice effects would portend a worse prognosis in aMCI.
Longitudinal, observational design following participants across one year.
Three groups of older adults: 1. cognitively intact (n=57), 2. aMCI with large practice effects across one week (MCI+PE, n=25), and 3. aMCI with minimal practice effects across one week (MCI−PE, n=26).
After controlling for age and baseline cognitive differences, the MCI−PE group performed significantly worse than the other groups after one year on measures of immediate memory, delayed memory, language, and overall cognition.
Although these results need to be replicated in larger samples, the loss of short-term practice effects portends a worse prognosis in patients with aMCI.
Practice effects are improvements in cognitive test performance after repeated exposure to the same test materials. Such improvements have traditionally been viewed as sources of error (1). Although practice effects are robust in cognitively intact older adults (1, 2) and largely absent in patients with dementia (3, 4), evidence is equivocal about practice effects in amnestic Mild Cognitive Impairment (aMCI). Several studies have reported an absence of practice effects in many individuals with aMCI across short and long retest intervals (5–8). Conversely, other individuals with aMCI demonstrate practice effects on cognitive and motor tests even across brief retest periods (9–11).
Although most of the aMCI studies above were retrospective, one was designed to examine the question prospectively (11). We have previously examined a group of older adults with aMCI who were administered a battery of cognitive tests at baseline and one week later to quantify practice effects. Across one week, the aMCI group improved significantly more than a group of cognitively intact elders on two delayed recall measures (29% vs. 11% improvement on repeat testing, respectively). The two groups demonstrated similar levels of improvements on the other cognitive tests. Upon closer examination within the aMCI, nearly half of these mildly memory impaired individuals improved to the point that they could no longer be classified as aMCI but instead appeared “normal.” The remainder still met aMCI criteria after one week. However, further follow-up of these groups is needed.
The current study sought to examine the prognostic value of practice effects in older adults with aMCI by following the three groups from our prior work (11) for one year. Based on our earlier study, we hypothesized that aMCI patients who displayed minimal practice effects across one week would have significantly worse cognitive outcomes after one year than those aMCI patients who displayed the large, expected practice effects. We will be examining the prognostic value of practice effects above and beyond baseline cognitive functioning, as minimal practice effects were associated with poorer cognition in our prior study. Clinically, practice effects determined after one week, might allow healthcare providers to predict cognitive trajectories (e.g., worsening or improvement) of aMCI patients and better allocate limited resources for intervention (e.g., offering treatment to those that are declining faster). In research, practice effects might be used as a screening measure in clinical trials to enrich samples with cases of aMCI who would not unexpectedly revert to normal across one year. They might also be used to define a subgroup of aMCI patients more likely to benefit from cognitive rehabilitation strategies.
Participants and Procedures. All procedures were approved by an Institutional Review Board at the University of Iowa. The participants include the one hundred eight older adults from Duff et al. (11) who completed their one year follow-up visits. These individuals were recruited from independent living facilities and community senior centers following educational talks on cognitive changes associated with aging. All participants had no history of major neurological (e.g., traumatic brain injury, stroke, dementia) or psychiatric illness (e.g., Schizophrenia, Bipolar Disorder) or current depression (either self-report or 30-item Geriatric Depression Scale [GDS] score of >12). Participants were also excluded if they reported taking any prescribed cognitive enhancer (e.g., donepezil, galantamine), medications with known cognitive effects (e.g., anti-epileptics), or medications that suggest the presence of a major psychiatric illness (e.g., anti-psychotics). All participants completed a brief telephone screening (12), to assist in identifying aMCI. After screening positive on the telephone, participants provided written informed consent and completed a baseline assessment, which included: clinical interview, GDS, Wide Range Achievement Test – 3 (WRAT-3) Reading subtest, temporal and spatial orientation items of the Modified Mini Mental Status Examination (3MS), Repeatable Battery for the Assessment of Neuropsychological Status (RBANS), Brief Visuospatial Memory Test – Revised (BVMT-R), Hopkins Verbal Learning Test – Revised (HVLT-R), Controlled Oral Word Association Test (COWAT), animal fluency, Trail Making Test (TMT) Parts A and B, and Symbol Digit Modalities Test (SDMT). Using results from the baseline assessment, individuals were classified as cognitively intact or aMCI using existing criteria (13). To be classified as aMCI, subjective memory complaints had to be reported by the participant or a collateral source. Objective memory deficits (i.e., age-corrected scores at or below the 7th percentile on at least two of the three delayed recall measures [RBANS Delayed Memory Index, HVLT-R, BVMT-R] relative to a premorbid intellectual estimate [WRAT-3 Reading]) also had to be present. The 7th percentile is 1.5 standard deviations below the mean, which is a typical demarcation point for cognitive deficits in MCI (13). Cognition was, otherwise, generally intact (i.e., non-memory age-corrected scores above the 7th percentile) and no functional impairments (e.g., denied needing assistance with managing money, taking medications, driving) could be reported. To be classified as “cognitively intact,” objective memory and non-memory performances were at least above the 7th percentile. All data were reviewed by two neuropsychologists (KD, LJB) to confirm the classification of individuals as aMCI (n = 51) or cognitively intact (n = 57). As is consistent with the shift to examine subtypes of MCI, we further classified these individuals based on deficits in non-memory domains (executive functioning via TMT-B, language via animal fluency, attention via SDMT). Of the 51 MCI individuals, 37 were classified as single domain memory impaired MCI, 6 were classified as multidomain memory and executive functioning impaired, 2 were classified as multidomain memory and language impaired, 4 were classified as memory, executive functioning, and attention impaired, and 1 was classified as memory, executive functioning, and language impaired. No one was classified with dementia (i.e., memory and other cognitive deficits and functional impairments). Only baseline cognitive performances were used in these classifications.
Approximately one week after the baseline visit, all participants completed a repeat cognitive assessment, which included all the baseline measures, except WRAT-3 Reading, 3MS, and RBANS. Practice effects was the primary focus of the study, so alternate forms of the tests were not used in order to maximize practice effects. Using data from only this one-week visit, all participants were reclassified using procedures similar to those used during the original classification (e.g., same two neuropsychologists reviewed all one-week data and reclassified participants as aMCI or intact based on the presence/absence of objective memory deficits on delayed recall measures from the one-week assessment). It should be reiterated that the RBANS was not re-administered at one-week, so only the HVLT-R and BVMT-R were used for memory measures on this reclassification. Results of this one-week reclassification yielded 26 aMCI participants and 82 cognitively intact participants. Over the course of one week, no participant significantly declined in non-memory performances or functional abilities to be classified as demented. All reclassifications were made independent of and “blinded” to the original classifications.
Using the results of the original classification and the one-week reclassification, all participants were assigned to one of three groups: 1) MCI−PE (MCI with minimal practice effects; i.e., originally classified and reclassified after one week as aMCI; n=26), 2) MCI+PE (MCI with large practice effects; i.e., originally classified as aMCI but reclassified after one week as intact, n = 25), and 3) cognitively intact (i.e., originally classified and reclassified after one week as intact, n = 57). Demographic and baseline and one-week assessment scores are in Table 1.
After one year, all participants were re-evaluated with the RBANS. The RBANS was chosen to examine cognitive outcomes because it was not re-administered after one week and it less contaminated by practice effects over one year (14). This measure also taps multiple areas of cognition (e.g., memory, attention, language, perception and construction).
Baseline age, education, gender, WRAT-3 standard scores (estimate of premorbid IQ), and GDS raw scores (depression) were compared among the three groups with ANOVA and chi-square analyses. The groups were significantly different on age, so age was used as a covariate in all the remaining analyses. To support the classification of these subjects as either aMCI or intact on baseline data, two MANCOVAs compared the groups on baseline memory (BVMT-R Total and Delayed Recall, HVLT-R Total and Delayed Recall) and non-memory (COWAT, Animals, TMT-A, TMT-B, SDMT) cognitive scores. To support the reclassification of these subjects as either aMCI or intact on one-week data, two additional MANCOVAs compared the groups on one-week memory and non-memory cognitive scores. These four preliminary MANCOVAs were used to help control for Type I errors (rather than two sets of nine individual univariate tests).
The three groups were also compared on baseline RBANS Index scores. As expected, there were group differences on some of these cognitive measures (Immediate Memory, Attention, Delayed Memory, Total Scale), so they were also used as covariates in the primary analyses. Including the baseline RBANS scores as covariates, we are able to examine the prognostic value of practice effects, above and beyond those baseline cognitive differences. The primary analyses involved a series of six ANCOVAs, where the dependent variable was the one year follow-up score on the RBANS Index of interest, the independent variable was the group (MCI−PE, MCI+PE, Intact), and the covariates were age and baseline RBANS Index score of interest (where appropriate). The alpha level was set at 0.05 for these primary analyses.
The three groups differed on age (F[2,105]=7.5, p<0.01), but they were comparable for education, gender, estimated premorbid intellect, and depression. All participants were Caucasian. After controlling for age, the two aMCI groups (−PE and +PE) performed significantly below the Intact participants on the baseline memory measures (multivariate F[8,204]=11.56, p<0.001), which is consistent with existing criteria for aMCI (13). There were differences between the three groups on the five baseline non-memory measures (multivariate F[10,194]=2.71, p=0.004), with the MCI−PE group performing below the other two groups on two of the measures (TMT-B and SDMT). However, compared to normative data (15, 16), both TMT-B and SDMT for the MCI−PE group remained within normal limits (e.g., 30th – 45th percentiles). The results of these two MANCOVAs support the classification of participants as aMCI or intact based on the baseline assessment.
Using data from the one-week assessment, the MCI−PE group scored significantly below the other two groups on the one-week memory measures (multivariate F[8,204]=11.48, p<0.001). Group differences were observable on non-memory measures (multivariate F[10,198]=2.45, p=0.009). For two of the non-memory measures (COWAT and TMT-A), all three groups were comparable. For the other three non-memory measures (Animals, TMT-B, and SDMT), the MCI−PE group performed significantly worse than the other two groups. The results of these two MANCOVAs on the one-week test-retest data seem to support the separation of the three groups.
After controlling for age and baseline RBANS Total Scale score, the three groups differed on the one-year follow-up RBANS Total Scale score (multivariate F[2,103]=4.42, p=0.014, partial eta2=0.08), with the MCI−PE group declining more than either the MCI+PE or Intact subjects. This same pattern was also observed on the Delayed Memory Index (multivariate F[2,103]=8.21, p<0.001, partial eta2=0.14), Immediate Memory Index (multivariate F[2,103]=4.84, p=0.010, partial eta2=0.09), and Language Index (multivariate F[2,103]=3.02, p=0.05, partial eta2=0.06) of the RBANS. No group differences in decline across one year were observed on the other two RBANS Indexes (Visuospatial/Constructional Index, Attention Index). Table 2 presents the RBANS baseline and one-year scores for all three groups. Figure 1 presents baseline and one year scores on the Delayed Memory Index of the RBANS in the three groups (using estimated marginal means that correct for age and baseline cognitive differences, from the above ANCOVA).
Given the results of the primary analyses, secondary analyses compared the two MCI subgroups (MCI+PE and MCI−PE) on cognitive outcomes after controlling for age and baseline cognitive scores. Similar to the findings of the primary analyses, the two aMCI subgroups significantly differed on their one year scores on the RBANS Total Scale score (multivariate F[1,47]=8.94, p=0.004, partial eta2=0.16), Delayed Memory Index (multivariate F[1,47]=7.96, p=0.007, partial eta2=0.15), Immediate Memory Index (multivariate F[1,47]=7.10, p=0.011, partial eta2=0.13), and Language Index (multivariate F[1,47]=4.10, p=0.049, partial eta2=0.08) of the RBANS. No group differences in decline across one year were observed on the other two RBANS Indexes (Visuospatial/Constructional Index, Attention Index). In each significant analysis, the MCI+PE group performed significantly better than the MCI−PE group on one year follow-up RBANS scores.
In an earlier study (11), we observed that some patients with aMCI demonstrated significant improvements on cognitive test scores after one week (i.e., practice effects), whereas other patients with aMCI did not. The current study followed those individuals over one year to examine the cognitive trajectories of these two aMCI groups. Consistent with our hypotheses, the aMCI individuals who showed large one-week practice effects remained relatively stable across one year, whereas those who minimally improved after repeated exposure to test materials declined across time. It is important to note that we used baseline cognitive scores as covariates in these analyses, so practice effects appear to have prognostic value above and beyond any baseline cognitive differences. To our knowledge, this is the first prospective study to report on the prognostic value of practice effects in predicting future cognitive outcomes in patients with aMCI.
On the RBANS Delayed Memory Index, individuals with aMCI who showed little benefit from practice effects across one week declined on this measure after one year (-10 standard score points based on estimated marginal means from ANCOVA). Conversely, individuals with aMCI who significantly benefited from practice across one week remained stable across one year (<1 standard score point). Individuals judged to be cognitively intact at baseline and one week tended to slightly improve across one year on this measure of delayed recall (+5 standard score points). These changes across time on the Delayed Memory Index for each group are depicted in Figure 1, with scores adjusted for age and baseline differences between the groups. Similar, albeit smaller, changes were observed on the RBANS Indexes for Immediate Memory and Language, as well as the Total Scale score. When the ANCOVAs were repeated on only the two aMCI subgroups (i.e., without the cognitively intact subjects), the results were very similar. Since this study focused on the amnestic subtype of MCI, the declines across time on the Immediate and Delayed Memory Indexes of the RBANS were expected. Similarly, the Total Scale score of the RBANS is weighted towards memory (i.e., 2/5 of the Indexes assess learning and memory), so declines on this global measure are also not surprising. Finally, other studies have observed that non-memory domains are affected in aMCI, including confrontational naming and semantic fluency (17–19), which are the two subtests that make up the RBANS Language Index.
Although the current findings are consistent with prior work (9, 10), it conflicts with other studies where practice effects have been largely absent in patients with MCI and early dementia (5–8). Some of the discrepancies between our findings and other studies in the literature are related to methodological differences. For example, two prior studies (7, 8) used very long retest intervals (e.g., 1 – 3 years) and practice effects are likely to be attenuated across such periods. Additionally, other studies used alternate test forms to minimize practice effects, whereas our procedures tried to maximize practice effects by purposely using the same form of each test. One key difference between our study and those in the literature is that we looked for different subtypes of aMCI cases – those that showed minimal or large benefits from practice.
One final possibility for the discrepancy between our findings and those from the existing literature is that some of our impaired subjects might not have had aMCI. The cognitive data in Table 1 and the classification analyses of the baseline cognitive scores in the Results might suggest that our MCI+PE group might reflect an “accidental” MCI group (20) or a pre-MCI group (21). Similarly, one could argue that our MCI−PE group might more adequately be described as multi-domain MCI, as they differed from their peers on non-memory measures. Indeed, whereas the MCI+PE group was predominantly (i.e., 96%) single domain amnestic MCI, the MCI−PE group contained more individuals with deficits in memory and other cognitive domains (e.g., 50% single domain memory impaired, 23% multidomain memory and executive functioning impaired, 15% multidomain memory, executive functioning, and attention impaired). Alternatively, our two MCI subgroups might reflect a continuum of disease phase or severity of brain dysfunction (e.g., MCI+PE being a very early case of MCI and MCI−PE being a moderate case of MCI), but that they will both eventually end up with the same outcome. Both our MCI subgroups were clearly “not normal” and “not demented,” which reflects the broader view of MCI. Both groups were also clearly identified as meeting criteria for aMCI by two neuropsychologists. Some of these semantic differences are expected as the concept of MCI continues to evolve (22). The present findings might support changes within the defining criteria of MCI. For example, more recent MCI criteria (23) recommend “evidence of decline over time” as a method of improving diagnostic accuracy. This new criterion for MCI might also be met by a loss of practice effects over a short period of time.
If our finding of two aMCI subtypes identified based on short-term practice effects, each with different long-term outcomes, then this would have important clinical and research implications. Assessment of short-term practice effects might allow healthcare providers to predict long-term cognitive outcomes more accurately (e.g., decline, stability, or improvement), with important ramifications for long-term patient and family planning (e.g., residential placement, durable power of attorney). Additionally, clinicians might make treatment decisions based on the likelihood of a patient getting better or worse across time. For example, a provider might offer treatment sooner to those that are declining faster. From a research standpoint, practice effects might be used as a screening measure in clinical trials. Past clinical trials in MCI have struggled to identify individuals who are most appropriate to participate in studies. For example, short-term practice effects could be used to enrich samples with cases of aMCI that won’t unexpectedly revert to normal across one year (24). Additionally, cholinesterase inhibitors and other cognitive enhancing medications might work optimally in the more severely impaired patients who demonstrate little capacity to learn after a week (i.e., MCI−PE). However, it should be noted that although we divided our aMCI subjects into two subgroups (MCI+PE and MCI−PE), practice effects scores, like other cognitive test scores, might be better viewed on a continuum. Prior work (9) has demonstrated that practice effects as a continuous variable also has prognostic value.
Several limitations of the current study and future directions should be noted. First, the sample sizes, especially for the two aMCI subtypes, were small and replication with larger samples is needed. Replication is also recommended because we did not control for multiple comparisons within our primary analyses, and some spurious findings may have occurred. Second, the follow-up period of one year should be extended. Third, the current method of determining change across time was decline on cognitive test scores, and future studies might consider conversion to dementia as the “gold standard” outcome measure. Fourth, the participants were a high functioning group of Caucasian retirees, with an average estimated premorbid IQ of 108 and 15+ years of education, and the generalizability of these findings to other samples (e.g., lower education, non-Caucasian) is unclear. Our sample was also 80% female, and the generalizability to a predominantly male sample is uncertain. Current participants did not undergo extensive medical work-ups (e.g., physical exam, neuroimaging) to confirm their MCI status, and information beyond cognitive test scores needs to be considered in participant classification. Lastly, there were some inconsistencies in our classification of subjects at baseline and their reclassification following one week (e.g., the RBANS Delayed Memory Index was not re-administered at one-week), and future studies could correct these inconsistencies.
In conclusion, practice effects, frequently considered to be a source of error variance in repeat cognitive testing, might hold valuable information for clinicians and researchers about prognosis in aMCI. Consistent with existing research, there appears to be two subtypes of these patients: those that remain cognitively stable across time and those that decline. The current results suggest that practice effects across one week can discriminate between these two subtypes. If replicated, then practice effects could serve as a simple, convenient, and non-invasive marker for monitoring an individual patient’s cognitive status, and they could have valuable implications for clinical practice and research. Future studies might also examine the meaning of practice effects in these cognitively impaired individuals to see if they also provide information about cognitive (e.g., role of priming, interaction with other cognitive domains) and social (e.g., test or generalized anxiety) factors, as well as underlying biological mechanisms.
The project described was supported research grants from the National Institutes on Aging: K23 AG028417-01A2 (KD) and P50AG005 (CGL, Hopkins Alzheimer’s Disease Research Center). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institute on Aging or the National Institutes of Health.