PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptNIH Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
J Clin Exp Neuropsychol. Author manuscript; available in PMC Jun 1, 2010.
Published in final edited form as:
PMCID: PMC2879263
NIHMSID: NIHMS197451
The Item-Specific Deficit Approach to evaluating verbal memory dysfunction: Rationale, psychometrics, and application
Matthew J. Wright,1 Ellen Woo,1,2 Maureen Schmitter-Edgecombe,3 Charles H. Hinkin,1,4 Eric N. Miller,1,5 and Amanda L. Gooding4
1Semel Institute for Neuroscience and Human Behavior, University of California, Los Angeles, CA, USA
2Alzheimer’s Disease Research Center, University of California, Los Angeles, CA, USA
3Washington State University, Pullman, WA, USA
4Veterans Administration Greater Los Angeles Healthcare System, Los Angeles, CA, USA
5Department of Psychology, University of California, Los Angeles, CA, USA
Address correspondence to Matthew J. Wright, Harbor-UCLA Medical Center, 1000 W. Carson St., D-2 Annex, #5, Box 495, Torrance, California 90509, USA (mwright/at/labiomed.org)
In the current study, we introduce the Item-Specific Deficit Approach (ISDA), a novel method for characterizing memory process deficits in list-learning data. To meet this objective, we applied the ISDA to California Verbal Learning Test (CVLT) data collected from a sample of 132 participants (53 healthy participants and 79 neurologically compromised participants). Overall, the ISDA indices measuring encoding, consolidation, and retrieval deficits demonstrated advantages over some traditional indices and indicated acceptable reliability and validity. Currently, the ISDA is intended for experimental use, although further research may support its utility for characterizing memory impairments in clinical assessments.
Keywords: Memory, Psychometrics, Brain injury, Traumatic brain injury, Human immunodeficiency virus
Verbal memory is often the focus of neuropsychological studies, in which descriptive indices are frequently extracted from list-learning data. These are used to identify memory process disruptions. Such indices are easily derived from list-learning tests and can be utilized when experimental manipulations may not be feasible. Unfortunately, the verbal memory profiles derived using these indices often lead to mixed results across studies and/or disagree with results from studies using experimental manipulations and measures of list-learning characteristics (e.g., human immunodeficiency virus, HIV: White et al., 1997 vs. Scott et al., 2006; traumatic brain injury, TBI: Vanderploeg, Crowell, & Curtiss, 2001 vs. DeLuca, Schultheis, Madigan, Christodoulou, & Averill, 2000; Parkinson’s disease: Beatty et al., 2003 vs. Faglioni, Saetti, & Botti, 2000). The objective of the current study is to introduce and evaluate a new set of memory process indices that can be applied to list-learning data and account for weaknesses in previously used measures.
Psychologists have often employed a three-stage model when discussing episodic memory (Ellis & Hunt, 1983; but see Howe, 1988). Here, memory is conceptualized as three distinct process units. The first is encoding, where information is taken in and transformed into a format that can be stored in the brain. The second stage is consolidation, where the transformed information is stored in the brain for later use. The third is retrieval, or extraction of the stored information for use. While both cognitive and neuropsychological studies have shown that the encoding, consolidation, and retrieval processes overlap, there is evidence that some distinctions do exist among them (e.g., Bernard, Desgranges, Platel, Baron, & Eustache, 2001; Greicius et al., 2003; Nyberg, Cabeza, & Tulving, 1996; Tulving & Markowitsch, 1997; Tulving & Osler, 1968; Tulving & Thompson, 1973). That said, the constructs of encoding, consolidation, and retrieval remain useful for discussing temporally graded memory processes that are, to some degree, subserved by different neuroanatomical networks (see Bernard et al., 2001; Fuster, 2000; Greicius et al., 2003; Nielsen-Bohlman & Knight, 1994; Nyberg et al., 1996; Nyberg, Forkstam, Petersson, Cabeza, & Ingvar, 2002). Therefore, this terminology is used in the following discussion.
Taken together, functional imaging and lesion studies have implicated the left prefrontal cortex (namely, the left inferior frontal gyrus) and the left medial temporal regions (primarily the parahippocampal gyrus) in the encoding of verbal memories (Alessio et al., 2004; Ariza et al., 2006; Bor, Cumming, Scott, & Owen, 2004; Buckner, Kelly, & Petersen, 1999; Dolan & Fletcher, 1999; Fernandez et al., 1998; Gimranov & Mal’tseva, 2005; Habib, Nyberg, & Tulving, 2003; Karlsgodt, Shirinyan, van Erp, Cohen, & Cannon, 2005; Strange, Otten, Josephs, Rugg, & Dolan, 2002; Vingerhoets, Miatton, Vonck, Seurinck, & Boon, 2006). Verbal memory encoding ability is often operationally defined in terms of the learning slope: the increase in recall across consecutive learning trials (Delis, Kramer, Kaplan, & Ober, 2000). Performance on Trial 1 is typically contrasted to performance on subsequent learning trials, with higher scores on later trials reflecting greater encoding. Unfortunately, attention deficits, which are common in many neurological and psychiatric conditions (Cohen, Malloy, Jenkins, & Paul, 2006), are likely to reduce list-learning performances, especially on Trial 1 (Wiegner & Donders, 1999), and an increased slope would indicate initial attention difficulties that are reduced with repetitive learning rather than encoding abilities. The encoding skills of participants with inattention might be better characterized by contrasting scores on Trial 2 to subsequent learning trials, but such indices would likely lead to an inaccurate representation of encoding in participants with intact attentional abilities (e.g., healthy controls), and therefore this alteration to the learning slope would be of limited utility for group comparisons. Another approach to assessing encoding ability is to sum the number of items recalled across learning trials (Delis et al., 2000). Although this approach is less reliant on initial learning, it can still be negatively impacted by poor Trial 1 performance, making it susceptible to the confounding effects of inattention.
A rather large body of work indicates that medial temporal lobe structures—namely, the hippocampus—play a major role in the consolidation of memories, although with the passage of time memory storage appears to come under the control of the neocortex (Squire, 1980, 1994; Squire & Zola, 1998; Tulving & Markowitsch, 1998). Data from a number of studies indicate that verbal memory consolidation tends to be controlled by the left hemisphere (Alessio et al., 2004; Ariza et al., 2006; Gimranov & Mal’tseva, 2005; Vingerhoets et al., 2006). At the behavioral level, consolidation deficits are often indexed by forgetting/retention calculations (Lezak, Howieson, Loring, Hannay, & Fischer, 2004). These measures typically compare recall from the last learning trial to that from a delayed-recall trial (Delis et al., 2000). Here, a significant decrease in delayed recall is considered to indicate rapid forgetting or a consolidation deficit. This method assumes that lower delayed recall reflects poor consolidation rather than retrieval difficulties. Unfortunately, this assumption makes forgetting calculations a somewhat nonspecific indicator of memory storage. It should be noted that this index is often considered in conjunction with discrepancies between recall and recognition scores (suggested to reflect retrieval deficits when recognition is superior to recall), although this index suffers from psychometric issues that are discussed below.
An alternative approach is the assessment of proactive interference (PI). PI refers to the adverse effect of older learning on new learning (Postman, 1971). This is a normal phenomenon and is suggested to reflect ongoing consolidation of the previously presented information. In a list-learning paradigm, PI can be introduced by presenting a second to-be-learned list following the learning trials of the first list (e.g., List A Learning Trials 1, 2, 3, …, k, List B Learning Trial 1; Delis et al., 2000). This procedure is common to many standard list-learning tests—for example, Rey Auditory–Verbal Learning Test (RAVLT), California Verbal Learning Test (CVLT). Here, PI is assessed by comparing the recall performance from Trial 1 of the first list (List A) and performance on the second list (List B). A decrement in recall of the second list (List A recall > List B recall) is suggested to reflect PI and hence ongoing consolidation of the first list. Given this framework, low PI can be used as a marker of consolidation difficulties. However, as noted above, initial recall performances during list learning (List A Trial 1, List B Trial 1) are likely subject to the confounding effects of inattention. Therefore, PI is probably an invalid index of consolidation for most neuropsychological investigations.
While encoding of verbal memories appears to be supported by the left prefrontal cortex and hippocampus, research suggests that the retrieval of verbal memories is primarily under the control of the right prefrontal cortex and, in some cases, the hippocampus (Dolan & Fletcher, 1999; Greicius et al., 2003; Habib et al., 2003; Karlsgodt et al., 2005; Nyberg, Cabeza, & Tulving, 1996). Retrieval ability has been routinely assessed via recognition and free-recall discrepancies. This method assumes that greater recognition in comparison to recall is suggestive of partial retrieval deficits (Delis et al., 2000), in addition to possible disruptions in encoding or consolidation (Lezak et al., 2004; Squire, 1980). Furthermore, converging lines of evidence suggest that recall–recognition discrepancies may be an imprecise indicator of retrieval ability. For example, Wilde, Boake, and Sherer (1995) showed that recognition–recall discrepancies did not agree with other hypothesized indices of retrieval problems (e.g., increased recall via semantic cueing, inconsistent recall across learning trials) calculated from the CVLT in a sample of TBI participants. This research met with some criticism due to the choice of retrieval indices used and the fact that an external verbal memory criterion was not utilized to validate the retrieval indices in the study (Delis et al., 2000). Nevertheless, the use of recognition–free-recall discrepancies has resulted in mixed findings in other studies as well (see Riefer & Batchelder, 1991). It has been suggested that results based on such discrepancies can be misleading, as they reflect multiple processes. Indeed, a growing body of literature suggests that recall is a product of recollection, whereas recognition performances are comprised of both recollection and familiarity, which each have differing neural substrates (see Aggleton & Brown, 2006).
Several memory process indices have been proposed and utilized widely in clinical practice and research (e.g., Beatty et al., 2003; Delis et al., 2000; Delis, Peavy, Heaton, & Butters, 1995; Peavy, Jacobs, Salmon, & Butters, 1994; Vanderploeg et al., 2001; Weschler, 1997; White et al., 1997). Although these indices reflect aspects of encoding, consolidation, and retrieval, their precision could be improved upon. One possible means of improvement would be to consider list-learning performances at the item level while accounting for patterns of performance on learning and delayed-recall trials. This way the confounding effects of inattention, as well as adequate index separation between different constructs (e.g., consolidation and retrieval), might be achieved. The Item-Specific Deficit Approach (ISDA) was developed within this rational framework. The ISDA encoding deficit index is a reflection of low acquisition across learning trials. It is derived by counting the number of items on a list-learning test that are not recalled on more than half of the learning trials (for example, on a task with five learning trials, it would be the number of items that were recalled on two or fewer learning trials; see Appendix for ISDA worksheet). The greater this value (maximum value = total number of list items), the greater the encoding difficulties are presumed to be. This encoding deficit index is not particularly reliant on any given learning trial, so it is less likely to be affected by inattention than are previously used encoding/acquisition indices.
APPENDIX
APPENDIX
ISDA Worksheet
The ISDA consolidation deficit index is calculated by summing the individual items that were recalled during list learning but not recalled on subsequent delayed-recall trials (multiple delayed recall trials are necessary to calculate this index). For comparative purposes, this value can be divided by the sum of the individual items recalled during list learning (i.e., the total items recalled at least once during learning; maximum value = total number of list items) to control for differing acquisition levels between groups. Since the loss of previously learned items is accounted for across delayed-recall trials, the ISDA consolidation deficit index is more likely to reflect storage difficulties than retrieval problems.
The ISDA retrieval deficit index is calculated by summing the individual items that were recalled during list learning but inconsistently recalled across delayed-recall trials (multiple delayed-recall trials are necessary to calculate this index). Similar to the consolidation deficit index, this value can be divided by the sum of the individual items recalled during list learning to control for acquisition level. Additionally, since the consistency of individual list items is tracked across delayed-recall trials, the ISDA retrieval deficit index is more likely to reflect retrieval problems than consolidation difficulties.
In the current study we applied the ISDA to CVLT data obtained from healthy and neurologically compromised (HIV+ and TBI) participants. We chose to include HIV+ and TBI participants in our neurologically compromised group, as both populations routinely evidence verbal memory impairments (Heaton et al., 1995; Levin & Goldstein, 1986). We assessed the ISDA’s relationship with traditional memory process indices, its reliability, and its ability to classify neurologically compromised participants, as well as its capacity to predict poor verbal memory performances on an external criterion.
Participants
Data from 132 participants (53 healthy participants and 79 neurologically compromised participants—37 HIV+ participants and 42 TBI participants) were analyzed (see Table 1 for sample characteristics). Table 1 provides detailed information regarding the study sample. All participants consented to voluntary participation in institutional review board (IRB) approved studies conducted in California (Los Angeles area; all HIV+ participants and 12 healthy participants), Tennessee (Memphis area; 18 TBI and 10 healthy participants), and Washington (Spokane/Pullman area; 24 TBI and 31 healthy participants). All participants were free of current psychiatric illness and substance abuse. All neurologically compromised participants were free of any other central nervous system disorder or disease (e.g., cerebrovascular accident, neurosyphilis, epilepsy) other than their index condition. All TBI participants had estimates of posttraumatic amnesia and/or loss of consciousness that indicated severe brain injury (Williamson, Scott, & Adams, 1996). Additionally, all TBI survivors suffered from acceleration–deceleration injuries and were at least one year post injury (range = 1–27 years). For the HIV+ participants, HIV infection was confirmed with serologic testing for HIV antibody (enzyme-linked immunosorbent assay, ELISA, followed by Western blot), and of the 34 HIV+ participants with recent CD4 counts, 26% met the Center for Disease Control’s criteria for an AIDS diagnosis. As can be seen in Table 1, our sample was diverse in terms of ethnic background, although these ethnic differences were not equally distributed across each subgroup. Previous work has shown that ethnic minority group status is associated with lower CVLT performances (Norman, Evans, Miller, & Heaton, 2000), an effect that is likely driven by inequalities in educational experiences (Manly, Byrd, Touradji, & Stern, 2004; Manly, Jacobs, Touradji, Small, & Stern, 2002). Given this, we analyzed our data for ethnic group differences on CVLT indices displayed in Table 1. We found no differences between Caucasians (n = 80) and non-Caucasians (n = 52), Fs ≤ 0.05, ps ≥ .83. Similarly, we found no differences between Caucasians and African Americans (n = 41), Fs ≤ 1.19, ps ≥ .28.
TABLE 1
TABLE 1
Sample characteristics
Procedure
All participants were administered the CVLT, a list-learning test that was used to derive conventional memory process indices, as well as the ISDA indices. A subset of participants were administered the Digit Span subtest from the Wechsler Adult Intelligence Scale–Revised (WAIS–R; Weschler, 1981; 50 neurologically compromised and 12 healthy) and the Logical Memory subtest from the Wechsler Memory Scale–Third Edition (WMS-III; Weschler, 1997; 20 neurologically compromised and 31 healthy) in order to assess attentional and memory factors independent of the list-learning task. All tests were administered and scored in accordance with standard instructions. The CVLT and the WMS-III were scored and normed via the software packages provided by the test manufacturers.
CVLT
The CVLT (Delis, Kramer, Kaplan, & Ober, 1987) is a standardized verbal list-learning test composed of 16 words that can be grouped into four different semantic categories. The list is presented orally to participants over five learning trials. Subsequently, a distractor list is presented, and participants are asked to recall the distractor items. Following the distractor trial, participants are administered a short-delay free-recall test, a short-delay cued-recall test, a long-delay (20-minute) free-recall test, a long-delay cued-recall test, and recognition trial.
The ISDA was applied to CVLT data to derive deficit-focused indices of encoding, consolidation, and retrieval. In order to accomplish this, CVLT performances were coded at the item level, indicating whether or not each word was recalled on each trial. The ISDA encoding deficit index was calculated by counting the items on the CVLT (out of a possible maximum of 16) that were recalled two or fewer times across the CVLT’s five learning trials. The ISDA consolidation deficit index was calculated by adding the individual items that were recalled during list learning, but not recalled on any subsequent recall trial.1 The ISDA retrieval index was derived by summing the individual items that were recalled during list learning, but inconsistently recalled across delayed-recall trials.
In addition to the ISDA indices, other proposed indicators of encoding ability—that is, learning slope and total recall across learning trials (total learning), consolidation (recall differences between the last learning trial and short-delay free recall (forgetting), and retrieval (differences between recognition hits and long-delay free recall (recognition–recall discrepancy)—generated by the CVLT scoring program were retained.
WMS-III Logical Memory
The Logical Memory subtest of the WMS-III is a verbal memory test for two orally presented stories (Wechsler, 1997). The first story is presented once followed by an immediate recall trial. The second story is read to examinees twice, with a recall trial following each presentation. Following this, there is a 25–35-minute retention interval. After the retention interval, free recall of both stories is obtained. Finally, examinees are administered a recognition test for the elements of each story. Immediate- and delayed-recall performances on the Logical Memory subtest were normed via the WMS-III scoring program, converted to T-scores, and then transformed to deficit scores per the method developed by Heaton and colleagues (see Carey et al., 2004), where T-scores below 40 are considered to reflect deficit performances.
WAIS–R Digit Span
Finally, Digit Span from the WAIS–R (Wechsler, 1981) is a measure of auditory attention. This test is composed of trials where examinees are read strings of digits and are asked to repeat them back aloud. The length of the digit strings is incrementally increased over successive trials. On the first set of trials, an examinee is asked to repeat the digits back in the same order in which they were presented. On the second set of trials, the examinee is required to repeat back the digits in reverse order. Performances on both sets of trials are summed for an overall indicator of auditory attention.
Analysis plan
Differences in continuous variables (demographics and CVLT performances) were determined via one-way analyses of variance (ANOVAs). Categorical differences were assessed via chi-square analyses. Pearson product–moment correlations were carried out on the Digit Span, ISDA encoding index, learning slope, and total learning to determine any possible associations. Additional correlational analyses were conducted on the ISDA and traditional indices to determine how well they represented different aspects of memory processing. The internal consistency/reliability of the ISDA indices were evaluated with coefficient alpha or the KR-20 statistic, depending on the nature of the index (continuous vs. dichotomous; Pedhazur & Schmelkin, 1991). Both the traditional memory process indices and the ISDA indices were submitted to linear discriminant function analyses to determine how well they discriminated between neurologically compromised and healthy participants. Finally, a second set of discriminant function analyses were conducted to assess how well the ISDA and traditional memory process indices could predict deficit performances on Logical Memory. A p < .05 was set for all analyses.
As can be seen in Table 1, the neurologically compromised participants evidenced verbal memory deficits in comparison to the healthy participants as expected. Correlational analyses were used to determine the degree that the proposed encoding indices were related to each other and whether they might be confounded by inattention. In the participants that completed WAIS–R Digit Span (n = 62), the ISDA encoding index was highly related to CVLT total learning (r = −.96, p < .001) and moderately associated with learning slope (r = .33, p < .05). However, only CVLT total learning was significantly associated with WAIS–R Digit Span (r = .29, p < .05); neither learning slope nor the ISDA encoding deficit index (r = −.21) was correlated with Digit Span (r = −.14). To determine whether these indices behaved differentially as a function of inattention, as hypothesized, we reran the analysis in a subgroup of participants (n = 53)—45 neurologically compromised and 8 healthy; 72% male; 20 Caucasians, 28 African Americans, 2 Mexican Americans, 1 Asian American, 1 Native American, 1 “other;” mean age = 39.79 years (SD = 7.21); mean years of education = 13.68 (SD = 1.82)—that exhibited inattention on WAIS–R Digit Span (T < 40; see Carey et al., 2004). As can be seen in Table 2, the pattern of associations is very different for participants with attention deficits. The ISDA encoding deficit index did not correlate with either of the traditional indices (rs ≤ −.17) or Digit Span (r = −.10). However, CVLT total learning was correlated with Digit Span (r = .33) and learning slope (r =.30); the relationship between learning slope and Digit Span approached significance (r = −.25, p = .07).With regard to the other memory process index correlations (see Table 3), numerous significant intercorrelations were found. However, it should be noted that forgetting and recognition–recall discrepancies were moderately associated (r = −.45), while the ISDA consolidation deficit index was unrelated to the ISDA retrieval deficit index (r = .03). Also, the ISDA retrieval deficit index was not associated with the recognition–recall discrepancies index (r = .03). Additionally, although total learning and the ISDA encoding index were highly correlated in the sample as a whole, these indices were shown to be functionally distinct in a group of inattentive participants (see Table 2).
TABLE 2
TABLE 2
Correlations between Digit Span and indices of encoding in inattentive participants
TABLE 3
TABLE 3
Correlations between memory process indices
The ISDA encoding index provides dichotomous outcomes for each list item (recalled vs. not recalled), so its internal consistency was evaluated with the KR-20 statistic. In contrast, the acquisition-adjusted consolidation and retrieval deficit indices, which are adjusted for the number of items gained during learning, are not composed of dichotomous values, so they were evaluated with coefficient alpha. While high internal consistency coefficients (> .80) are routinely expected of indices used to make important decisions about level of impairment (Anastasi, 1988; Urbina, 2004), indices that are used to characterize these impairments are not judged by the same standard since they are descriptive (not diagnostic) in nature (Kline, 2005). The internal consistency estimates of such “process” indices are often decreased by two factors: a small number of items and heterogeneity in item responses due to individual differences in test-taking approach (Cronbach, 1984; Kehoe, 1995; Pedhazur & Schmelkin, 1991). Heterogeneity in item responses is indeed an issue in memory test performances, as common individual differences in list-learning approaches (levels of self-initiated rehearsal, intensity of study) differentially impact immediate- and delayed-recall patterns (e.g., Geiselman, Woodward, & Beatty, 1982). It has been suggested that moderate internal consistency estimates (.50–.70) are acceptable for experimental measures and/or descriptive indices comprised of a relative small number of items (Kehoe, 1995; Kline, 2005).
As can be seen in Table 4, ISDA indices of encoding, consolidation, and retrieval deficits showed acceptable internal consistency (.58–.77). In order to obtain internal consistency estimates that were less influenced by individual differences, we analyzed data from a subset of 67 participants that utilized a consistent list-learning strategy (indicated by a z-score of ≤1 on the CVLT’s semantic or serial clustering indices). This led to higher internal consistency estimates (.64–.84; see Table 4). Finally, to determine whether the number of items for each of the indices was impacting their internal consistency, we artificially doubled the responses for each index and calculated internal consistency coefficients for the whole sample as well as the subset that indicated a consistent learning strategy. This analysis showed that the number of items did influence our internal consistency estimates as the coefficients increased dramatically (.80–.89 and .82–.92, respectively; see Table 4).
TABLE 4
TABLE 4
Internal consistency coefficients for the ISDA as applied to the CVLT
Linear discriminant function analysis was employed to determine how well the ISDA indices could predict neurological status. The ISDA indices significantly discriminated between neurologically compromised and healthy participants: Wilks’s λ= 0.33, χ3(3) = 143.45, p < .001; canonical r(cr) =.82, η2 = .67. The encoding deficit index was the main contributor to this discriminant function (cr = .71), followed by the retrieval deficit index (cr = .54) and the consolidation deficit index (cr = .45). Overall, the ISDA indices correctly classified 93.2% of the participants as either neurologically compromised or intact, with 92.4% sensitivity and 94.3% specificity.
A follow-up linear discriminant function analysis was conducted to determine how well traditional memory process indices (i.e., learning slope,2 forgetting, recognition-recall discrepancy) could predict neurological compromise. The traditional indices significantly discriminated between neurologically compromised and healthy participants: Wilks’s λ = 0.60, χ2 (3) = 64.31, p < .001; cr =.63, η2 = .40. The traditional consolidation index (forgetting) was the main contributor to this discriminant function (cr = .83), followed by the encoding index (learning slope; cr =.48) and the retrieval index (recognition-recall discrepancy; cr = −.13). Overall, the traditional memory process indices correctly classified 82.2% of the participants as either neurologically compromised or intact, with 90.6% sensitivity and 78.9% specificity.
Discriminant function analysis was conducted on a subsample of participants (n = 51)—20 neurologically compromised and 31 healthy; 82% male; 44 Caucasians, 1 African American, 6 “other;” mean age = 34.45 years (SD = 9.39); mean years of education =14.29 (SD = 2.18)—that had completed the Logical Memory subtest of the WMS-III, a measure of verbal learning and memory. Impaired performance on Logical Memory was determined by converting norm-derived scaled scores for total story units recalled in Logical Memory I and Logical Memory II to T-scores and then transforming these to deficit scores (see Carey et al., 2004). Performances resulting in a deficit score of 1 or greater (i.e., T < 40) on either Logical Memory I or Logical Memory II were considered to reflect impaired verbal memory. The ISDA indices significantly predicted impaired verbal memory on Logical Memory: Wilks’s λ = 0.65, χ2(3) = 26.41, p < .001; cr = .59, η2 = .35. This function was primarily driven by the consolidation deficit index (cr =.89), while the encoding (cr =.14) and retrieval deficit (cr =.20) indices played a minor role. It should be noted that the chief role of the consolidation deficit index in this function is not particularly surprising, given that meaningful prose provides a good deal of encoding and retrieval support (e.g., Jefferies, Lambon Ralph, & Baddeley, 2004). This discriminant function correctly classified 82.4% of the Logical Memory performances, with 74.0% sensitivity and 89.3% specificity.
An additional discriminant function analysis was carried out to determine how well traditional memory process indices (i.e., learning slope, forgetting, recognition-recall discrepancy) predicted performance deficits on the Logical Memory subtest. The traditional indices also predicted Logical Memory deficits: Wilks’s λ = 0.79, χ2(3) = 11.25, p = .01, cr =.46, η2 = .21. This function was primarily driven by forgetting (cr =.67), followed by recognition-recall discrepancies (cr = −.53) and learning slope (cr = −.11). This discriminant function correctly classified 72.5% of the Logical Memory performances, with 93.0% sensitivity and 48.0% specificity.
Given the satisfactory psychometric findings regarding the ISDA as applied to the CVLT, we further analyzed the healthy participants’ (n = 53) data in order to derive useful normative references for future research. It should be noted that we used acquisition-corrected indices of consolidation and retrieval deficits for these analyses. Correlational analyses among age, education, and the ISDA indices indicated a positive association between age and the encoding deficit index (r = .48, p < .001). Neither the consolidation deficit nor the retrieval deficit indices were related to age (r =.15, r = .07, respectively). None of the ISDA indices was associated with years of education (rs = −.09 to −.11).
To further explore the relationship between age and the encoding deficit index, we stratified the healthy sample into three age groups (18–28 years, n = 12; 29–39 years, n = 19; 40–50 years, n = 22) and conducted an Age Group × Encoding Deficit Index ANOVA. The ANOVA revealed a significant main effect of age, F(2, 50) = 8.40, p = 0.001, ηp2 = .25. Tukey post hoc tests indicated that 40–50-year-olds had greater encoding deficits than the other two younger age groups. The 18–29- and the 29–39-year-olds did not differ with respect to the ISDA encoding deficit index. Based on these results, we present preliminary normative data for the ISDA indices stratified by two age groups (18–39 and 40–50 years) in Table 5.
TABLE 5
TABLE 5
Age corrected experimental normative data for the ISDA as applied to the CVLT
Traditional verbal memory process indices are routinely generated from list-learning data (total learning, learning slope, forgetting/retention, recognition–recall discrepancy) for clinical use and research purposes. These measures have yielded mixed results, have disagreed with other methods, and are suggested to be relatively nonspecific estimates of encoding, consolidation, and retrieval. Improved methods for discerning specific deficits in encoding, consolidation, and retrieval from list-learning data are of clinical and experimental importance. Indeed, the cognitive rehabilitation task force assembled by the European Federation of Neurological Societies indicated that better clinical and pathological characterizations are needed in intervention studies (Cappa et al., 2003). Better memory process indicators would not only improve diagnostic accuracy via fractionation of amnestic samples into subgroups, but they could also suggest optimal targets for interventions. Further, such psychometric improvements would aid investigators in more thoroughly evaluating the utility of behavioral and pharmacological interventions for memory impairment such that modest, mixed, or partially beneficial effects might be better explained (e.g., Evans et al., 2000; Rund & Borg, 1999; Stip, Sepehry, & Chouinard, 2007; Whitehead et al., 2004).
In the current study, we evaluated a novel item analytic approach (the ISDA) to assess memory process breakdowns in a mixed sample of healthy and neurologically compromised participants. We found that the ISDA demonstrated several strengths when applied to CVLT data. First, the ISDA encoding deficit index showed no association with impaired auditory attention (Digit Span), while a more commonly used index of encoding/acquisition (i.e., total words recalled from Trials 1–5 on the CVLT) did. This suggests that the ISDA encoding deficit index is less susceptible to the confounding effects of inattention. Second, the ISDA consolidation deficit index was shown to be unrelated to the ISDA retrieval deficit index. This contrasts with traditional indices of consolidation (forgetting/retention) and retrieval (recognition–recall discrepancies) that are moderately associated. Our data suggest that the ISDA produces functionally distinct indices of consolidation and retrieval, while traditional indices overlap. The overlap between traditional consolidation and retrieval indices is problematic for clinical and research use, as consolidation can contaminate retrieval measures and vice versa.
Since the ISDA indices of encoding, consolidation, and retrieval deficits are composed of itemlevel responses, we were able to evaluate the internal consistency of each index. Overall, the ISDA indices indicated moderate internal consistency (.58–.77). Such coefficients are considered to reflect adequate reliability for descriptive (not diagnostic) indices/scales comprised of a small number of items (Kehoe, 1995; Kline, 2005; Pedhazur & Schmelkin, 1991). That said, these estimates were likely reduced by differences in strategy use, as such differences are associated with disparate patterns of immediate and delayed recall (e.g., Geiselman et al., 1982), and such individual disparities often translate into greater item variances (Pedhazur & Schmelkin, 1991). Indeed, the ISDA indices reliability estimates increased (.64–.84) when individual differences in learning strategy use were partially controlled for. Beyond differences in strategy use, the internal consistency estimates of the ISDA indices were likely lowered by the relatively low number of items comprising each index and our sample size. Future work employing test–retest reliability estimates might better ascertain the reliability of the ISDA indices.
With regard to validity, discriminant function analyses revealed that the ISDA indices had a slight advantage over traditional indices in terms of classifying participants as neurologically compromised or intact: ISDA, 93.2 % classified correctly (92.4% sensitivity, 94.3% specificity), versus traditional indices, 82.2% classified correctly (90.6% sensitivity, 78.9% specificity). Additional discriminant function analyses conducted with a subset of participants (n = 51) showed that the ISDA outperformed traditional indices in predicting low memory performances on the Logical Memory subtest of the WMS-III. Specifically, the ISDA correctly predicted 82.4% of poor performances on Logical Memory, with 74.0% sensitivity and 89.3% specificity, while the traditional indices correctly predicted 72.5% of deficit performances on Logical Memory, with 93.0% sensitivity and 48.0% specificity. It should be noted that the ISDA’s discriminant function for Logical Memory deficits was primarily driven by the ISDA consolidation deficit index. Although this was not predicted, it was not particularly surprising, since meaningful prose, such as that provided in the Logical Memory stories, provides a good deal of encoding and retrieval support (e.g., Erber, Galt, & Botwinick, 1985; Jefferies et al., 2004; Spilich & Voss, 1982). This reduces any variance that could be accounted for by the ISDA encoding and retrieval deficit indices. In future research, another list-learning measure (e.g., RAVLT) should be used to provide an external criterion for the ISDA as applied to the CVLT.
In summary, while traditional memory process indices have yielded mixed results, have disagreed with experimental manipulations, and have been suggested to suffer from methodological difficulties; the ISDA was shown to be a reliable and valid method when applied to the CVLT. The current study suggests that the ISDA provides a decided advantage over traditional indices for experiments identifying the specific deficits that are responsible for memory impairments. However, additional work is necessary to determine the clinical utility of the ISDA. Future work should focus on alternative approaches to assessing the reliability (e.g., test–retest) and validity (e.g., experimental manipulations) of the ISDA across populations and using other list-learning measures. Applications of the ISDA to the CVLT-II and RAVLT, for example, have yet to be investigated. Additionally, studies are needed to determine how well the ISDA can discriminate memory deficit profiles and patterns of brain pathology. Nevertheless, the ISDA is a new analytic approach that has strengths over previously used methods to dissociate memory deficits. Furthermore, the ISDA may prove to be a useful clinical tool if future research continues to indicate that it has good psychometric properties and can meaningfully distinguish memory-disordered groups.
Acknowledgments
Special thanks go to Anna Curren and for her assistance. This work was supported by a Marchionne Memorial Doctoral Fellowship (Washington State University) as well as National Institutes of Health funding (RO3 HD35838, R01 NS47690, R01 MH585522, and T32 MH019535). Profitable use of the item-specific deficit approach (ISDA) is under the protection of U.S. Patent No. 60/862,837.
Footnotes
Full terms and conditions of use: http://www.informaworld.com/terms-and-conditions-of-access.pdf
This article may be used for research, teaching and private study purposes. Any substantial or systematic reproduction, re-distribution, re-selling, loan or sub-licensing, systematic supply or distribution in any form to anyone is expressly forbidden.
The publisher does not give any warranty express or implied or make any representation that the contents will be complete or accurate or up to date. The accuracy of any instructions, formulae and drug doses should be independently verified with primary sources. The publisher shall not be liable for any loss, actions, claims, proceedings, demand or costs or damages whatsoever or howsoever caused arising directly or indirectly in connection with or arising out of the use of this material.
1For the purposes of group comparison, the ISDA indices of consolidation and retrieval deficits should be divided by the sum of the individual items recalled during list learning to control for possible differences in acquisition.
2Learning slope was used as the traditional index of encoding in discriminant function analyses as it was found to be relatively less confounded by auditory attention deficits, in contrast to total learning.
  • Abou-Hamden A, Blumbergs PC, Scott G, Manavis J, Wainwright H, Jones N, et al. Axonal injury in falls. Journal of Neurotrauma. 1997;14:699–713. [PubMed]
  • Aggleton JP, Brown MW. Interleaving brain systems for episodic and recognition memory. Trends in Cognitive Sciences. 2006;10:455–463. [PubMed]
  • Alessio A, Damasceno BP, Camargo CHP, Kobayashi E, Guerreiro CAM, Cendes F. Differences in memory performance and other clinical characteristics in patients with mesial temporal lobe epilepsy with and without hippocampal atrophy. Epilepsy & Behavior. 2004;5:22–27. [PubMed]
  • Anastasi A. Psychological Testing. 6th ed. New York: Macmillan Publishing Company; 1988. Reliability; pp. 109–137.
  • Ariza M, Pueyo R, Junqué C, Mataró M, Poca MA, Mena MP, et al. Differences in visual vs. verbal memory impairments as a result of focal temporal lobe damage in patients with traumatic brain injury. Brain Injury. 2006;20:1053–1059. [PubMed]
  • Beatty WW, Ryder KA, Gontkovsky ST, Scott JG, McSwan KL, Bharucha KJ. Analyzing the subcortical dementia syndrome of Parkinson’s disease using RBANS. Archives of Clinical Neuropsychology. 2003;18:509–520. [PubMed]
  • Bernard F, Desgranges B, Platel H, Baron J, Eustache F. Contributions of frontal and medial temporal regions to verbal episodic memory: A PET study. Neuroreport. 2001;12:1737–1741. [PubMed]
  • Bor D, Cumming N, Scott CEL, Owen AM. Prefrontal cortical involvement in verbal encoding strategies. European Journal of Neuroscience. 2004;19:3365–3370. [PubMed]
  • Buckner RL, Kelley WM, Petersen SE. Frontal cortex contributes to human memory formation. Nature Neuroscience. 1999;2:311–314. [PubMed]
  • Cappa SF, Benke T, Clarke S, Rossi B, Stemmer B, van Heugten CM. EFNS guidelines on cognitive rehabilitation: Report of an EFNS task force. European Journal of Neurology. 2003;10:11–23. [PubMed]
  • Carey CL, Woods SP, Gonzalez R, Conover E, Marcotte TD, Grant I, et al. Predictive validity of global deficit scores in detecting neuropsychological impairment in HIV infection. Journal of Clinical and Experimental Neuropsychology. 2004;26:307–319. [PubMed]
  • Cohen RA, Malloy PF, Jenkins MA, Paul RH. Disorders of attention. In: Snyder PJ, Nussbaum PD, Robins DL, editors. Clinical neuropsychology: A pocket handbook for assessment. 2nd ed. Washington, DC: American Psychological Association; 2006. pp. 572–606.
  • Cronbach LJ. Essentials of psychological testing. 4th ed. New York: Harper & Row; 1984. How judge tests; pp. 158–184.
  • Delis DC, Kramer JH, Kaplan E, Ober E. California Verbal Learning Test. San Antonio, TX: Psychological Corporation; 1987.
  • Delis DC, Kramer JH, Kaplan E, Ober E. California Verbal Learning Test–II. San Antonio, TX: Psychological Corporation; 2000.
  • Delis DC, Peavy G, Heaton R, Butters N. Do patients with HIV-associated minor cognitive/motor disorder exhibit a “subcortical” memory profile? Evidence using the California Verbal Learning Test. Assessment. 1995;2:151–165.
  • DeLuca J, Schultheis MT, Madigan NK, Christodoulou C, Averill A. Acquisition vs. retrieval deficits in traumatic brain injury: Implications for memory rehabilitation. Archives of Physical Medicine and Rehabilitation. 2000;81:1327–1333. [PubMed]
  • Dolan RJ, Fletcher PF. Encoding and retrieval in human medial temporal lobes: An empirical investigation using functional magnetic resonance imaging (fMRI) Hippocampus. 1999;9:25–34. [PubMed]
  • Ellis HC, Hunt RR. Fundamentals of human memory and cognition. 3rd ed. Dubque, IA: Wm. C. Brown Company Publishers; 1983. pp. 84–130.
  • Erber JT, Galt D, Botwinick J. Age differences in the effects of contextual framework and word-familiarity on episodic memory. Experimental Aging Research. 1985;11:101–103. [PubMed]
  • Evans JJ, Wilson BA, Schuri U, Andrade J, Baddeley A, Bruna O, et al. A comparison of “errorless” and “trial-and-error” learning methods for teaching individuals with acquired memory deficits. Neuropsychological Rehabilitation. 2000;10:67–101.
  • Faglioni P, Saetti MC, Botti C. Verbal learning strategies in Parkinson’s disease. Neuropsychology. 2000;14:456–470. [PubMed]
  • Fernandez G, Weyerts H, Schrader-Bolsche M, Tendolkar I, Smid HGOM, Tempelmann C, et al. Successful verbal encoding into episodic memory engages the posterior hippocampus: A parametrically analyzed functional magnetic resonance imaging study. The Journal of Neuroscience. 1998;18:1841–1847. [PubMed]
  • Fuster JM. Cortical dynamics of memory. International Journal of Psychophysiology. 2000;35:155–164. [PubMed]
  • Geiselman RE, Woodward JA, Beatty J. Individual differences in verbal memory performance: A test of alternative information-processing models. Journal of Experimental Psychology: General. 1982;111:109–134.
  • Gimranov RF, Mal’tseva EA. Effect of transcranial magnetic stimulation on short- and long-term memory in healthy subjects and patients with Parkinson’s disease. Human Physiology. 2005;31:398–401. [PubMed]
  • Greicius MD, Krasnow B, Boyett-Anderson JM, Stephan E, Schatzberg AF, Reiss AL, et al. Regional analysis of hippocampal activation during memory encoding and retrieval: fMRI study. Hippocampus. 2003;13:164–174. [PubMed]
  • Habib R, Nyberg L, Tulving E. Hemispheric asymmetries of memory: The HERA model revisited. Trends in Cognitive Sciences. 2003;7:241–245. [PubMed]
  • Heaton RK, Grant I, Butters N, White DA, Kirson D, Atkinson JH, et al. The HNRC 500-neuropsychology of HIV infection at different disease stages. HIV Neurobehavioral Research Center. Journal of the International Neuropsychological Society. 1995;1:231–251. [PubMed]
  • Howe ML. Measuring memory development in adulthood: A model-based approach to disentangling storage-retrieval contributions. In: Howe ML, Brainerd CJ, editors. Cognitive development in adulthood. New York: Springer-Verlag; 1988. pp. 39–64.
  • Jefferies E, Lambon Ralph MAL, Baddeley AD. Automatic and controlled processing in sentence recall: The role of long-term working memory. Journal of Memory and Language. 2004;51:623–643.
  • Karlsgodt KH, Shirinyan D, van Erp TD, Cohen MS, Cannon TD. Hippocampal activations during encoding and retrieval in a verbal working memory paradigm. NeuroImage. 2005;25:1224–1231. [PubMed]
  • Kehoe J. Basic item analysis for multiple-choice tests. Practical Assessment, Research & Evaluation, 10. 1995. Retrieved January 1, 2008, from http://pareonline.net/getvn.asp?v=4&n=10.
  • Kline TJB. Psychological testing: A practical approach to design and evaluation. Thousand Oaks, CA: Sage Publications; 2005. Reliability of test scores and test items; pp. 167–183.
  • Levin HS, Goldstein FC. Organization of verbal memory after severe closed-head injury. Journal of Clinical and Experimental Neuropsychology. 1986;8:643–656. [PubMed]
  • Lezak MD, Howieson DB, Loring DW, Hannay HJ, Fischer JS. Neuropsychological assessment. 4th ed. New York: Oxford University Press; 2004. Memory I: Tests; pp. 414–478.
  • Manly JJ, Byrd DA, Touradji P, Stern Y. Acculturation, reading level, and neuropsychological test performance among African American elders. Applied Neuropsychology. 2004;11:37–46. [PubMed]
  • Manly JJ, Jacobs DM, Touradji P, Small SA, Stern Y. Reading level attenuates differences in neuropsychological test performance between African American and white elders. Journal of the International Neuropsychological Society. 2002;8:341–348. [PubMed]
  • Nielsen-Bohlman L, Knight RT. Electro-physiological dissociation of rapid memory mechanisms in humans. Neuroreport. 1994;5:1517–1521. [PubMed]
  • Norman MA, Evans JD, Miller SW, Heaton RK. Demographically corrected norms for the California Verbal Learning Test. Journal of Clinical and Experimental Neuropsychology. 2000;22:80–94. [PubMed]
  • Nyberg L, Cabeza R, Tulving E. PET studies of encoding and retrieval: The HERA model. Psychonomic Bulletin & Review. 1996;3:135–148. [PubMed]
  • Nyberg L, Forkstam C, Petersson KM, Cabeza R, Ingvar M. Brain imaging of human memory systems: Between-systems similarities and within-system differences. Cognitive Brain Research. 2002;13:281–292. [PubMed]
  • Peavy G, Jacobs D, Salmon DP, Butters N. Verbal memory performance of patients with human immunodeficiency virus infection: Evidence of subcortical dysfunction. Journal of Clinical and Experimental Neuropsychology. 1994;16:508–523. [PubMed]
  • Pedhazur EJ, Schmelkin LP. Measurement, design, and analysis: An integrated approach. Hillsdale, NJ: Lawrence Erlbaum Associates, Inc.; 1991. Reliability; pp. 81–117.
  • Postman L. Kling JW, Riggs LA. Experimental psychology. New York: Holt, Rinehart and Winston; 1971. Transfer, interference and forgetting.
  • Riefer DM, Batchelder WH. Age differences in storage and retrieval: A multinomial modeling analysis. Bulletin of the Psychonomic Society. 1991;29:415–418.
  • Rund BR, Borg NE. Cognitive deficits and cognitive training in schizophrenic patients: A review. Acta Psychiatrica Scandinavica. 1999;100:85–95. [PubMed]
  • Scott JC, Woods SP, Patterson KA, Morgan EE, Heaton RK, Grant I, et al. Recency effects in HIV-associated dementia are characterized by deficient encoding. Neuropsychologia. 2006;44:1336–1343. [PubMed]
  • Spilich GJ, Voss JF. Contextual effects upon text memory for young, aged-normal and aged memory-impaired individuals. Experimental Aging Research. 1982;8:45–49. [PubMed]
  • Squire LR. Specifying the deficit in human amnesia: Storage, retrieval, and semantics. Neuropsychologia. 1980;18:368–372. [PubMed]
  • Squire LR. Memory and forgetting: Long-term and gradual changes in memory storage. International Review of Neurobiology. 1994;37:243–269. [PubMed]
  • Squire LR, Zola SM. Episodic memory, semantic memory, and amnesia. Hippocampus. 1998;8:205–211. [PubMed]
  • Stip E, Sepehry AA, Chouinard S. Add-on therapy with acetylcholinesterase inhibitors for memory dysfunction in schizophrenia: A systematic quantitative review, Part 2. Clinical Neuropharmacology. 2007;30:218–229. [PubMed]
  • Strange BA, Otten LJ, Josephs O, Rugg MD, Dolan RJ. Dissociable human perirhinal, hippocampal, and parahippocampal roles during verbal encoding. Journal of Neuroscience. 2002;15:523–528. [PubMed]
  • Tulving E, Markowitsch HJ. Memory beyond the hippocampus. Current Opinion in Neurobiology. 1997;7:209–216. [PubMed]
  • Tulving E, Markowitsch HJ. Episodic and declarative memory: Role of the hippocampus. Hippocampus. 1998;8:198–204. [PubMed]
  • Tulving E, Osler S. Effectiveness of retrieval cues in memory for words. Journal of Experimental Psychology. 1968;77:593–601. [PubMed]
  • Tulving E, Thompson DM. Encoding specificity and retrieval processes in episodic memory. Psychological Review. 1973;80:352–373.
  • Urbina S. Essentials of psychological testing. Hoboken, NJ: John Wiley & Sons; 2004. Essentials of reliability; pp. 117–150.
  • Vanderploeg RD, Crowell TA, Curtiss G. Verbal learning deficits in traumatic brain injury: Encoding, consolidation, and retrieval. Journal of Clinical and Experimental Neuropsychology. 2001;23:185–195. [PubMed]
  • Vingerhoets G, Miatton M, Vonck K, Seurinck R, Boon P. Memory performance during the intracarotid amobarbital procedure and neuropsychological assessment in medial temporal lobe epilepsy: The limits of material specificity. Epilepsy & Behavior. 2006;8:422–428. [PubMed]
  • Wechsler D. Wechsler Adult Intelligence Scale–Revised. New York: Psychological Corporation; 1981.
  • Wechsler D. Wechsler Memory Scale. 3rd ed. San Antonio, TX: The Psychological Corporation; 1997.
  • White DA, Taylor MJ, Butters N, Mack C, Salmon DP, Peavy G, et al. Memory for verbal information in individuals with HIV-associated dementia complex. Journal of Clinical and Experimental Neuropsychology. 1997;19:357–366. [PubMed]
  • Whitehead A, Perdomo C, Pratt RD, Birks J, Wilcock GK, Evans JG. Donepezil for the symptomatic treatment of patients with mild to moderate Alzheimer’s disease: A meta-analysis of individual patient data from randomised controlled trials. International Journal of Geriatric Psychiatry. 2004;19:624–633. [PubMed]
  • Wiegner S, Donders J. Performance on the California Verbal Learning Test after traumatic brain injury. Journal of Clinical and Experimental Neuropsychology. 1999;21:159–170. [PubMed]
  • Wilde MC, Boake C, Sherer M. Do recognition-free recall discrepancies detect retrieval deficits in closed-head injury? An exploratory analysis with the California Verbal Learning Test. Journal of Clinical and Experimental Neuropsychology. 1995;17:849–855. [PubMed]
  • Williamson DJG, Scott JG, Adams RL. Traumatic brain injury. In: Adams RL, Parsons OA, Culbertson JL, Nixon SJ, editors. Neuropsychology for clinical practice: Etiology, assessment, and treatment of common neurological disorders. Washington, DC: American Psychological Association; 1996. pp. 9–64.