|Home | About | Journals | Submit | Contact Us | Français|
Repeated study improves memory, but the underlying neural mechanisms of this improvement are not well understood. Using functional magnetic resonance imaging and representational similarity analysis of brain activity, we found that, compared with forgotten items, subsequently remembered faces and words showed greater similarity in neural activation across multiple study in many brain regions, including (but not limited to) the regions whose mean activities were correlated with subsequent memory. This result addresses a longstanding debate in the study of memory by showing that successful episodic memory encoding occurs when the same neural representations are more precisely reactivated across study episodes, rather than when patterns of activation are more variable across time.
Repeated study of the same materials can significantly strengthen memory representations and make them more resistant to forgetting (1), but not all repetitions are equal. One fundamental issue is how multiple study episodes add up to improve later memory. A widely accepted theory, often referred to as the encoding variability hypothesis (2–5), proposes that each study episode is encoded differently as a result of contextual drift with time, and that greater encoding variability leads to better memory. Alternatively, it has been claimed that each subsequent study episode serves as a retrieval cue to reactivate and strengthen the memory representation of the information stored during earlier study episodes (6). Evidence for this reactivation view comes from the finding in an AB–AC paradigm in which the presence of A during AC study reinstated AB and therefore also improved memory for B (7), an effect related to activity in the posterior medial temporal lobe (8). However, previous work has not yet established a link between the nature of neural representations during encoding and later memory.
In this study, we used functional MRI and representational similarity analysis (9) to examine how the similarity in patterns of neural activity across multiple study presentations is related to subsequent memory. The encoding variability hypothesis predicts that better subsequent memory should be associated with greater dissimilarity between activity patterns across study presentations. To the contrary, we found across three studies that better subsequent recognition and recall is associated with greater similarity between neural activity patterns across repetitions, consistent with the hypothesis that practice improves memory by retrieving and strengthening a consistent representation.
In the first experiment, 24 subjects were scanned while memorizing 120 novel faces (Fig. 1A) (10). Each face was presented four times, with an inter-repetition-interval (ITI) ranging from 1 (i.e., consecutive) to 20 faces. One hour after the scan, subjects were given a recognition memory test, during which a total of 240 faces (half learned, half new) were randomly mixed together. For each stimulus, the subjects had to decide whether or not it had been presented before by responding on a 6-point confidence scale, from 1 (definitely new) to 6 (definitely old). Out of the 120 old faces, subjects on average recognized with high confidence (e.g., a 5 or 6 rating) 51.7±18.6 items and forgot (a 1 or 2 rating) 37.3 ± 16.9 items (table S1). Using a subsequent memory paradigm (11, 12), we compared encoding-related brain activity for subsequently recognized faces with that for subsequently forgotten faces across four repetitions. Consistent with previous literature (13–15), this comparison identified stronger activation for subsequently remembered faces than for subsequently forgotten faces in the left (Montreal Neurological Institute, MNI: −46, −60, −8, Z = 3.88) and right (MNI, 44, −60, −10, Z = 3.58) fusiform gyrus, extending into the inferior temporal gyrus and lateral occipital cortex (fig. S1).
We then tested the core hypothesis that pattern similarity in this region is associated with subsequent memory. To do this, we re-estimated the model with unsmoothed data.
We then extracted the signal for each individual voxel within anatomically defined regions of interest (ROIs), and used representational similarity analysis (9) to examine the degree of similarity in the fMRI activation patterns between repetitions (averaged over stimuli), using the Pearson correlation coefficient as the similarity metric. Because pattern similarity can be affected by the number of trials in each condition (fig. S2), our analyses were based on a model that matched the number of trials included in the regressors for remembered and forgotten items as well as their repetition lags.
We focused our analyses on 20 independent anatomically defined regions in the dorsal and ventral visual stream, frontoparietal cortex and middle and medial temporal cortex (table S3), all of which have been previously shown to be important for visual object perception and memory encoding. Nine of these regions showed significantly higher degrees of pattern similarity for subsequently remembered faces than subsequently forgotten faces (P < .05, 2 regions remained significant by Bonferroni correction), whereas no regions showed the opposite effect (Fig. 2–3, fig. S3, table S4). Only four regions (LIFG, RIFG, RFUS, and RPHG) showed stronger overall activation for subsequently remembered faces than subsequently forgotten faces (table S4). To ensure that the signficantly higher pattern similarity was not caused by differences in mean activity, we reran this analysis in the bilateral ventral visual stream (LOC, FUS and ITG) after removing voxels showing significant subsequent memory effects in mean activity under a liberal threshold (P < .05, uncorrected). Even after removing these mean-responsive voxels, there was a significantly higher degree of pattern similarity for remembered versus forgotten faces (F(1,23) = 6.16, P = .02) (Fig. 3, table S4).
The results from our first experiment suggest that the degree of pattern similarity between successive study episodes is associated with subsequent memory performance in a recognition test. Since free recall is more sensitive to contextual associations than recognition, the encoding variability hypothesis thus predicts that subsequently recalled items might be associated with more divergent contexts than items that are not subsequently recalled (16–18). To provide a further and more direct test of the encoding variability hypothesis, we conducted a second experiment to examine whether greater pattern similarity is also associated with better free recall performance. Subjects (n = 22) were asked to perform a semantic (concrete vs. abstract) judgment task on familiar words during the scan (10). Each word was repeated three times, with an inter-repetition interval ranging from 1 to 18 trials. After the study session, participants were asked to return six hours later to perform two memory tests. In the first test, subjects were asked to recall the words they had studied in the scanner. They were then asked to perform a recognition test similar to that described in Experiment 1. Out of the 180 items, 44.7 ± 21.7 items were categorized as Recalled (items correctly recalled on the free recall test), 81.2 ± 25.5 items as Recognized (items recognized with high confidence [score of 5 or 6] but not recalled), and 54.1± 27.4 items as Forgotten (items that were neither Recalled nor Recognized) (table S1).
Table S2 and fig. S4 depict the differences in behavioral performance and mean activation during the semantic task among recalled, recognized, and forgotten items. Based on the findings from Experiment 1, we constructed a new model using equal numbers of trials from the Recalled, Recognized, and Forgotten conditions and calculated the pattern similarity in the same 20 regions. We found that out of the 20 regions, 15 regions showed a higher level of pattern similarity across repetitions (again averaging over items) for subsequently recalled items than for subsequently recognized or forgotten items (one region remained significant by Bonferroni correction); no region showed the opposite effect (fig. S5, table S5). Only four regions showed stronger mean activation for subsequent recalled words than for subsequently forgotten words (table S5). Again, after removing voxels showing subsequent memory effects for mean activation (P < .05, uncorrected), the remaining voxels in the left dorsal visual stream (dLOC and IPL) also showed greater pattern similarity across repetitions for subsequently recalled than subsequently recognized or forgotten words (F(1,21) = 4.64, P = .015) (fig. S5, table S5).
In both Experiments 1 and 2, the use of rapid event-related designs did not allow for reliable estimates of item-specific BOLD activation patterns, and pattern similarity was calculated from the aggregated BOLD activation pattern across stimuli in each condition. There are thus two possible explanations that are consistent with these results. First, it is possible that the results reflect overlap in item-level encoding processes. Alternatively, it is possible that they reflect process-level overlap, such that better memory occurs when the same general processes (e.g., perceptual, attentional, or semantic processes) are engaged across repetitions. In order to more directly test the hypothesis of item-specific pattern overlap, we performed a third experiment using a slow event-related fMRI design (12sec for each trial), which enabled us to extract BOLD signal patterns associated with each single trial. In this experiment, 22 young adults were asked to perform a semantic (living vs. nonliving) judgment task on sixty familiar words during the scan (10). To prevent further encoding of each item during the inter-trial intervals, subjects performed a highly engaging self-paced visual orientation judgment task for 8 s after each semantic judgment task (lasting for 3s), and the next trial started after a 1 s delay (fig. S6). Each item was repeated three times, with an inter-repetition interval ranging from 4 to 9 trials. Thirty minutes after the study session, participants were asked to freely recall the words they had studied in the scanner. Out of the 60 items, subjects on averaged recalled 13.5 ± 4.7 items (table S1).
Subjects’ response time on the semantic judgment task decreased across repetitions (F(2,42) = 42.96, P < .001); accuracy was high (mean = 97.5% ± 2%) and did not change across repetitions (F(2,42) = 1.68, P = .19). Accuracy (F(1,21) = 1.79, P =.19) and response times (F(1,21) = 0.15, P =.70) did not differ between subsequently recalled items and forgotten items (table S2). There were no significant interactions between repetition and subsequent memory for either accuracy (F(2,42) = 0.14, P =.087) or response time (F(2,42) = 1.69, P =.20). Functional imaging data revealed that, similar to Experiment 2, there were significantly stronger activations for subsequently recalled items than for subsequently forgotten items in the left middle/inferior frontal gyrus (LMFG/LIFG) (MNI: −50 14 34, Z = 4.17), and the left dorsal lateral occipital lobe (LdLOC) and adjacent inferior parietal lobule (LIPL) (MNI: −40 −66 46, Z = 3.48) (fig. S7).
We then examined whether the degree of pattern similarity in the 20 anatomically defined regions was associated with subsequent memory performance. We constructed a beta-series model (19) with one regressor for each trial and estimated the model using ridge regression (20). Consistent with the first two experiments, 7 of the 20 regions showed a significantly higher level of pattern similarity across repetitions for subsequently recalled items than for subsequently forgotten items (P < .05; one region remained significant by Bonferroni correction), whereas no region showed the opposite effect (Fig. 4, fig. S8, table S6). Taking the LdLOC as an example, we found that the level of pattern similarity across repetitions showed a significant subsequent memory effect (F(1,21) = 18.69, P = .0003) (Fig. 4D). Again, after removing voxels showing subsequent memory effects in terms of activation levels (P < .05, uncorrected), the remaining voxels in the left dorsal visual stream (dLOC and IPL) also showed stronger pattern similarity across repetitions for subsequently recalled than subsequently recognized or forgotten words (F(1,21) = 7.97, P =.01) (Fig. 4F, table S6).
Finally, if the degree of pattern similarity truly reflects item-specific reinstatement of activation patterns, we should expect a higher level of pattern similarity within items (i.e., cross-repetitions) than between items. To test this prediction, we calculated the averaged across-item pattern similarity of all possible pairings (except the within-item, cross-repetition pairings), separately for recalled items and forgotten items. The results showed that within-item correlation was higher than that for cross-item correlation, especially for recalled items (fig. S8, table S7). Again taking the LdLOC as an example, we found that the degree of pattern similarity across repetitions in this region for recalled items was significantly higher than cross-item pattern similarity (t(21) = 3.13, P =.005), whereas the difference was not significant for forgotten items (P =.41) (Fig. 4D), suggesting that repeatedly studying the same material is not sufficient to introduce activation reinstatement at the item-specific level, and failure of pattern reinstatement is associated with forgetting.
We took a number of measures to ensure that our results were not due to the effects of repetition lag or repetition priming. In addition to matching the number of trials, we also carefully matched the repetition lags between remembered and forgotten items (Experiment 1: remembered vs. forgotten: 6.11 vs. 6.05; t = 1.02, P = .31; Experiment 2: recalled vs. recognized vs. forgotten: 6.03 vs. 6.02 vs. 5.63, F (2,42)= 1.41, P = .26). There was also no difference in the repetition lag between recalled and forgotten trials in Exp 3 (6.4 vs. 6.3, t = 1.22, P =.23). In addition, we did not find a significant interaction between repetition priming and subsequent memory in most of the brain regions in any of the experiments (Tables S4–6). Third, in Experiment 3, the reaction times and accuracy in the semantic judgment task and the orientation judgment task that followed subsequently remembered and forgotten items were not different (table S2). Finally, for Exps 1 and 2, the degree of pattern similarity is not an artifact of design matrix orthogonality (fig. S9 and S10).
Taken together, our results suggest that episodic memory encoding is enhanced by reactivating the initial neural representation in each subsequent study episode. Using different study materials and different memory tests, our data suggest that pattern reinstatement can account for subsequent memory effects for both verbal and nonverbal materials and in both recall and recognition tests. While the within- versus across-items analysis in Experiment 3 demonstrates a significant effect of pattern overlap at the level of individual items, the results of Experiments 1 and 2 are also consistent with an effect of process-level overlap; we propose that both of these are likely to be important in determining the effectiveness of repeated study. We suggest that repeated study episodes lead to more effective encoding when the same neural representation is reinstated, which is incompatible with the encoding variability hypothesis. Consistent with our results, it has recently been shown that more reproducible neural patterns are associated with more conscious cognitive processing (21), suggesting that consistency of pattern engagement may be a more general marker of effective cognitive processing (perhaps due to its effects on memory encoding).
Previous studies have shown that memory retrieval is associated with reactivation of some of the same sensory regions that were activated during perception of those items (22–26). This category-specific or sequence-specific activation reinstatement precedes memory (27) and is associated with performance in free recall (28). The present study extends these findings and shows that during subsequent learning where no explicit retrieval was required, item-specific pattern reinstatement occurs, resulting in a stronger episodic encoding event that supports subsequent memory. Our results are also consistent with evidence showing that memory consolidation, whether during sleep or awake periods following learning, involves replay of neural activation patterns during learning (29, 30). Given the important role of memory retrieval on memory retention (31), these results suggest that reactivation of the same neural pattern during initial learning, either during repeated practice, memory consolidation and/or memory retrieval, can enhance memory.
Although most previous studies on the subsequent memory effect focused on one-shot learning, real learning in daily life often involves repeated practice. Our study suggests that, for repeated study events, pattern reinstatement is as sensitive as, if not more sensitive than, overall activation (11, 12, 32) as a predictor of subsequent memory. Our approach can readily be used to examine the neural mechanisms underlying other manipulations that affect memory encoding during repeated practice, such as the spacing effect and the variance effect (7), which would help to clarify the effects of encoding variability. However, fMRI data are a relatively coarse aggregate measure of the responses of large populations of neurons, and thus may not necessarily capture all of the aspects of encoding variability that might be at play. Future studies need to further examine this issue by applying similar approaches using complementary neuroimaging techniques, such as electroencephalography and single-unit recording.
Supporting Online Material
Materials and Methods
Figs. S1 to S10
Tables S1 to S7