In this study, a series of regression models were performed on a sample of nondemented, community-dwelling older adults to predict 1-year follow-up performances on a battery of commonly used neuropsychological measures. Consistent with the existing literature (Duff et al., 2004
;Duff, Schoenberg, 2008
; Hermann et al., 1996
; McSweeny et al., 1993
; Sawrie et al., 1996
; Temkin et al., 1999
), the best predictor of follow-up performance (i.e., 1-year scores) was initial performance (i.e., baseline scores) on that same measure. Across cognitive measures, baseline scores shared between 25% and 58% of the variance with 1-year scores (see
the top row of R2
values for each cognitive score in Table ). Although these findings do not capture the entirety of the 1-year scores, they are similar to those reported by others using patient and control samples.
To our knowledge, this is the first study to examine the possible influence of short-term practice effects on SRBs. Although practice effects have routinely been viewed as error variance in retesting paradigms, recent research suggests that these improvements in test scores might have diagnostic and prognostic utility in neuropsychologically impaired older samples. Diagnostically, several researchers have observed that individuals with MCI tend to benefit less from practice than healthy peers (Cooper, Lacritz, Weiner, Rosenberg, & Cullum, 2004
; Darby, Maruff, Collie, & McStephen, 2002
; Duff, Beglinger, et al., 2008
; Yan & Dick, 2006
; Zehnder, Blasi, Berres, Spiegel, & Monsch, 2007
). Prognostically, an absence of practice effects has been linked to eventual decline in MCI (Duff et al., 2007
; Howieson et al., 2008
). In the present study, 1-week practice effects on all nine of the cognitive variables examined significantly improved predictions of test scores at 1 year. The short-term practice effects used in the current study might allow clinicians and researchers to identify these “at-risk” individuals even sooner than reported previously (e.g., Howieson et al. examined practice effects across a 1-year interval). Practice effects also might have implications for interpreting the results of longitudinal studies (Salthouse & Tucker-Drob, 2008
). Admittedly, the relative contribution of practice effects above and beyond baseline scores was small in these analyses (e.g., 3%–22% of shared variance, see
the second row of R2
values for each cognitive score in Table ). The practice effects retest interval in the current study was 1 week, and future studies might investigate if shorter or longer retest intervals can lead to practice effects with greater contributions to prediction accuracy. For example, Attix and colleagues (2009
) used change across 1 year to better determine cognitive trajectories across longer periods of time.
As can be seen in the last three columns of Table , the magnitude of practice effects varies by test and retest intervals. Although others have commented on this fact (McCaffrey et al., 2000
; Salthouse & Tucker-Drob, 2008
), the largest differences occurred on measures of learning and memory. For example, the effect sizes for the BVMT-R and HVLT-R between baseline and 1 week averaged 1.15, whereas the effect sizes across this same interval for non-memory tests averaged 0.25. Similarly, the longer the retest interval, the smaller the practice effect (e.g., average effect size: Baseline and 1 week = 0.65, 1 week and 1 year = −0.45, baseline and 1 year = 0.13). This decreasing magnitude across time is probably also related to the number of assessments, as practice effects tend to decrease by the third assessment point for some tests (Beglinger et al., 2005
The combined effects of baseline scores and practice effects in predicting future cognition deserve some additional comment. It should not be too surprising that baseline test scores predict future test scores, as the majority of cognitive abilities do not normally change a lot over the course of 1 or 2 years. For example, the average correlation between baseline and 1-year scores in the current study was 0.64, and the majority of 1-year scores are within a couple of points of their respective baseline scores (Table ). In this way, the baseline score provides a fair amount of information about an expected follow-up score. Practice effects, however, appear to provide some indication of expected change from that baseline level. Short-term improvements in test scores might suggest the presence of additional cognitive reserve or plasticity. An absence of practice effects or short-term declines (i.e., negative practice effects) might suggest neuropsychological dysfunction. Although these hypotheses require further study, practice effects seem to be another clinically relevant variable.
Prior SRB studies have observed that demographic variables (e.g., age, education, and gender) provide a small but statistically significant role in predicting the follow-up cognitive scores. For example, in their report on SRBs for the 12 subtests of the Repeatable Battery for the Assessment of Neuropsychological Status, Duff and colleagues (2005)
found that age contributed to eight SRBs, education contributed to five, gender contributed to two, and race contributed to one. The results of the present study were quite different. As can be seen in Table , demographic variables only contributed to three of the nine models (i.e., gender contributed to TMTB, age contributed to HVLT-R Delayed Recall, and COWAT). It is possible that some restriction in the range of demographic variables in the current study lead to their exclusion from SRB models. For example, since all participants in the current study were Caucasian, race would not contribute to any of the models. However, other demographic variables in the current sample seemed to have sufficient variation (e.g., age: 65–96 years, education: 8–24 years, and 20.4% men). It is also possible that the variance captured by demographic variables in prior studies is now accounted for in the practice effects score.
Another aspect of the current study that warrants comment is the composition of the sample used to develop the SRBs. Prior studies have tended to use relatively homogeneous samples to generate SRBs. For example, in their original study on SRBs, McSweeny and colleagues (1993)
used only seizure patients to develop change formulas. Conversely, Temkin and colleagues (1999)
used only neurological healthy individuals to predict follow-up cognitive scores. The current study used both healthy elders and those classified with amnestic MCI. In some ways, these two subsamples do reflect a single group: Non-demented, community-dwelling elders. However, almost by definition, one group suffers from at least “mild” memory problems, whereas the other group does not. It was our intent to combine both the subsamples to increase the variability of cognitive scores, which increases the potential of developing SRBs that would be applicable across a broad segment of the older adult population. In a related vein, Heaton and colleagues (2001)
have observed that SRBs and other change formula developed on healthy samples might be less applicable in clinical samples. In their work, the authors developed change formulas on healthy adults, but then examined their validity in patients with schizophrenia (who were presumed to be relatively stable). Fewer than expected numbers of these patients with schizophrenia were identified as “not changing” across time. Heaton and colleagues suggested that samples used to develop SRBs might include individuals who are neurological stable, but not necessarily cognitively normal, so that a wide range of baseline and follow-up scores are represented. Unwittingly, we might have achieved this directive, as our combined sample of amnestic MCI and healthy elders contained a broader range of cognitive functioning that might yield more accurate prediction formulas in clinical samples. Nonetheless, we also included MCI status as another variable in the regression models, which allows us to see if memory impairment might differentially affect retesting performances. In the current analyses, MCI status only contributed to one of the regression models, BVMT-R Total Recall. In this model, the negative β weight seems to indicate that being identified as amnestic MCI lowers the expected follow-up score on this measure of visual learning. The general lack of further cognitive decline in our MCI sample across 1 year (as indicated by the two MANCOVAs on 1-year scores) may have also “restricted the range” of this variable.
Despite the potential benefits of using SRBs (e.g., increased precision in the assessment of change), it should be noted that we are not advocating for a strictly psychometric assessment of any patient (i.e., based solely on test data). Complementing historical information, behavioral observations, and laboratory results are also vital pieces of clinical information. We are, however, attempting to provide the necessary psychometric information to assist in the clinical decision-making process. For those interested in utilizing this information, a copy of the computer program used to calculate the predicted scores, difference scores, and test of the differences' significance can be obtained from the first author. It should also be noted that some findings suggest that SRBs are no better than other indexes of reliable change in patient samples (Heaton et al., 2001
Some limitations with the present study are acknowledged. As with most regression-based prediction formulas (Tabachnick & Fidell, 1996
), less accurate estimates are possible for individuals whose cognitive functioning falls at the extremes (e.g., <2nd percentile or >98th percentile) at baseline. In these cases, the prediction equations are more susceptible to regression to the mean and other fluctuations. However, the present study utilized a sample with both intact and impaired participants, which might lessen the chance of these statistical fluctuations. Caution should be exercised when using these formulas outside the demographic and situational parameters of the sample (e.g., <64 or >96 years old; relatively brief or extended retest intervals; non-Caucasians). Since all subjects in this study were evaluated at baseline, 1 week, and 1 year, it is unknown how accurate these SRBs would be for a patient who did not have the 1-week assessment. Finally, the stability of the regression equations needs to be validated in an independent sample, as the reliability of practice effects, especially in impaired samples, is not known.
In conclusion, the present SRB algorithms have the potential to provide more accurate assessments of cognitive change in older adults by considering the influence of initial performance, practice effects, and other demographic factors. These equations were developed on measures widely used in neuropsychological practice. These formulas also, for the first time, specifically utilize the short-term practice effects, which appear to lead to more accurate predictions of follow-up cognitive scores. Although validation of the effectiveness of these formulas in clinical samples is needed, they have the potential to contribute to the clinical decision-making process.