PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
Psychol Sci. Author manuscript; available in PMC 2016 July 1.
Published in final edited form as:
PMCID: PMC4643667
NIHMSID: NIHMS668452

Predicting and improving recognition memory using multiple electrophysiological signals in real time

Abstract

Although we are capable of storing a virtually infinite amount of information in memory, our ability to encode new information is far from perfect. The quality of encoding varies from moment to moment and renders some memories more accessible than others. Here we show that we can forecast the likelihood that a given item will be later recognized by monitoring two dissociable fluctuations of the electroencephalogram (EEG) during encoding. Next we show that we can identify individual items that are poorly encoded using our electrophysiological measures in real time, and successfully improve the efficacy of learning by having subjects restudy these items. Thus, our memory forecasts using multiple electrophysiological signals demonstrate the feasibility and the effectiveness of using real-time monitoring of the moment-to-moment fluctuations of the quality of memory encoding to improve learning.

Keywords: visual memory, human electrophysiology, event-related potential, memory encoding, alpha oscillations

Humans are capable of encoding and storing a virtually infinite amount of visual information in long-term memory (Brady, Konkle, Alvarez, & Oliva, 2008; Voss, 2009). Yet our ability to remember this information fluctuates significantly across individuals (Friedman & Trott, 2000; Golby et al., 2005), and from moment to moment within an individual (Fernandez et al., 1999; Paller & Wagner, 2002; Wagner et al., 1998). Is there a way for us to reliably forecast whether we will remember a particular piece of information by monitoring electrophysiological brain signals during a single, brief encoding event? If so, can we take advantage of these measurements to improve the efficacy of learning by identifying items that require additional study?

Cognitive neuroscientists have found several encoding-related neural signals that differentiate remembered items from later forgotten items (Friedman & Johnson, 2000; Paller & Wagner, 2002). Specifically, recordings of the electroencephalogram (EEG) and averaged event-related potentials (ERPs) have provided two excellent candidates. First, a larger sustained positivity is observed at frontal electrodes during encoding for later remembered items than for those later missed (Friedman & Trott, 2000; Paller, Kutas, & Mayes, 1987; Paller, McCarthy, & Wood, 1988). Second, alpha-band activity is more suppressed during encoding for later remembered items than for those later missed (Hanslmayr, Spitzer, & Bauml, 2009; Klimesch et al., 1996).

Even though these two neural measures of the quality of memory encoding are well established, it is unclear whether they can be utilized in real time to predict whether a stimulus will be later remembered. Indeed, we typically follow the convention of averaging hundreds of trials worth of data to derive reliable ERPs because the single-trial EEG has a lower signal-to-noise ratio (Luck, 2005; Woodman, 2010). However, it is unknown if electrophysiological memory effects are of sufficient magnitude to predict subsequent memory after encoding a single visual stimulus. If we can establish that these electrophysiological signals can reliably forecast later recognition after a single stimulus presentation, then it may be possible to use them to monitor the moment-to-moment fluctuations in our memory encoding ability.

Currently, we do not know how the frontal positivity and the occipital alpha power are related to each other. These two signals have only been studied independently. Each separate line of work suggests that the electrophysiological signal indexes the depth of encoding of the to-be-remembered stimuli (Hanslmayr et al., 2009; Hanslmayr & Staudigl, 2014; Otten, Henson, & Rugg, 2001). So, perhaps they measure the same mechanism. However, because no study has examined these two signals simultaneously, it is unclear whether they index the same or different aspects of memory encoding. If the signals index different mechanisms necessary for successful encoding, then these two signals should account for unique variance and we should be able to improve our predictive power by combining the two independent brain signals.

Experiment 1

In Experiment 1, we determined if the two electrophysiological measures index the same or separable mechanisms operating at encoding. Experiment 1 also served the broader goal to establish the feasibility of using these measures to forecast the later recognition of a particular stimulus, the question addressed directly in Experiment 2.

Method

Participants

According to the sample size estimation based on a preliminary dataset, we aimed to collect data from 20 participants across 500 trials. After consenting to procedures approved by the Institutional Review Board of Vanderbilt University, 23 individuals (10 males and 13 females, 18–32 years of age) participated in exchange for $30. All volunteers self reported that they were neurologically normal, had normal or corrected-to-normal visual acuity, and no color blindness. The data from three participants were excluded from analyses because they did not complete the session.

Stimuli and Procedures

The stimuli and the task are illustrated in Figure 1. The stimuli were adapted from a published set of photographs (Brady et al., 2008). During the encoding task, subjects were sequentially presented with 500 pictures of real-world objects with short breaks every 50 pictures. They were instructed to study each item while holding central fixation so that they could later perform a recognition-memory test. Subjects initiated each trial with a button press on a gamepad. After a 1250ms pre-encoding period, a picture was presented for 250ms, followed by a 1000ms encoding period during which the computer screen remained blank. Then, a central fixation dot was presented to indicate the beginning of the next trial. After the encoding task, we measured subjects’ eyes open and closed resting-state EEG activity for 15 minutes. Then, we tested subjects’ memory for the pictures using a recognition task.

Figure 1
The schematics of the encoding task (1A) and the recognition test (1B) in Experiment 1.

The recognition memory test started with the onset of the central fixation dot. Subjects initiated each test trial with a button press on the gamepad. After initiating each trial, they were instructed to maintain central fixation without blinking until each trial was over. Following a 1250ms blank period after the trial was initiated, a picture of a real-world object was presented at the center of the screen, with new and old pictures randomly interleaved. After the picture had been presented for 1250ms, a blue and a red dot appeared, with one dot on each side of the picture. At this point, subjects indicated whether they remembered seeing this picture during the study phase. The position of the red dot indicated the side of the buttons on the gamepad to hit if they remembered seeing the picture, and the blue dot indicated the buttons to hit if they did not. For instance, if they remembered seeing the picture with 100% confidence, then they were instructed to hit the outmost button on the gamepad indicated by the spatial position of the red dot (e.g., the left side of the gamepad in the example trial shown in Fig. 1). If they were less confident about having seen the picture, they hit the other two buttons on the red side to indicate the varying degree of confidence (the middle button for 80% confidence, and the inner button for 60% confidence). If they thought that they had not seen the picture during the encoding task with 100% certainty, they were instructed to press the outmost button on the side of the gamepad indicated by the position of the blue dot, and so forth. The sides of the red and blue dots are randomized from trial to trial.1 After the response, the trial was over and the subjects were provided with a self-paced interval to rest their eyes and blink. Subjects were tested on 500 studied pictures and 250 new pictures.

Data acquisition and analysis

EEG acquisition and pre-processing

The EEG was recorded using a right-mastoid reference, re-referenced offline to the average of the left and right mastoids. We used the 10–20 electrode sites (Fz, Cz, Pz, F3, F4, C3, C4, P3, P4, PO3, PO4, O1, O2, T3, T4, T5 and T6) and a pair of custom sites, OL (halfway between O1 and OL) and OR (halfway between O2 and OR). Eye movements were monitored using electrodes placed 1cm lateral to the external canthi for horizontal movement and an electrode placed beneath the right eye for blinks and vertical eye movements. The signals were amplified with a gain of 20,000, a bandpass of 0.01–100hz, and digitized at 250hz. Trials accompanied by horizontal eye movements (> 30uV mean threshold across observers) or eye blinks (> 75uV mean threshold across observers) were rejected before further analyses.

ERP pre-processing

To measure the ERPs preceding memory encoding, we time-locked to the button-press response that initiated a trial and examined the waveforms recorded during a time window from −1250ms to 0ms relative to the onset of the picture. These epoched EEG segments were baseline corrected to the mean EEG amplitude measured −400–0ms prior to the beginning of the measurement epoch of interest.

To examine EEG activity during memory encoding, we time-locked to the onset of memory stimuli and examined the EEG recording during a time window from 0ms to 1250ms following the onset of each memory stimulus. These epoched EEG segments were baseline corrected to the mean EEG amplitude −400–0ms relative to the stimulus onset. For presentation purposes, we needed to concisely summarize the relationship between our electrophysiological measures and behavior. As a result, the epoched pre-encoding and encoding signals were binned and averaged based on the recognition performance in the memory test. More precisely, the EEG activity recorded as the subjects viewed the items that were later recognized with 100% confidence were binned as high confidence hit trials (High Confidence), and those recorded as the subjects viewed the items that were later recognized at lower confidence levels (80% and below) were binned as low confidence hit trials (Low Confidence). The EEG segments recorded as the subjects viewed the items that were later missed were binned as miss trials (Miss). As described below, these binned averages also allowed us to confirm that our findings replicate previous reports of the traditional mean amplitudes across these types of trials.

Time-frequency pre-processing

To examine the oscillatory responses, we measured frequency content during the same pre-encoding and encoding epochs described above on a trial-by-trial basis. The spectral decomposition with a fixed window size of 400ms and a window overlap of 380ms was performed in MATLAB (spectrogram.m) for each single-trial EEG epoch to obtain the time-frequency representation of the signal. Then, the resultant time-frequency representation for each epoch was sorted into the appropriate High Confidence, Low Confidence, or Miss bin.

Results

Behavioral results

For studied objects, participants recognized 63% of the stimuli with 100% confidence (High Confidence) and 14% of the stimuli at or below 80% confidence (Low Confidence). Participants failed to recognize the remaining 23% of the stimuli (Miss). They successfully rejected 76% of new objects that they had not studied during the encoding phase. Table 1 reports the proportions of trials used to derive the receiver operating characteristic (ROC) curves in this experiment. The mean area under the ROC curve (AUC) was 0.82. These results demonstrate that, on average, subjects performed the memory task accurately.

Table 1
Behavioral results of Experiment 1.

Traditional ERP and EEG analysis

We found that frontal waveforms exhibited a sustained positivity of larger amplitude for High Confidence items than for Low Confidence and Miss items (Figure 2A, see also Figure S2 and Supplemental Materials) using traditional ERP analyses. We quantified the sustained frontal positivity as the mean amplitude in the time window 200ms to 1000ms after the onset of each studied item at the mid-frontal channel (i.e., channel Fz) where the effect was maximal. An ANOVA confirmed that this subsequent memory effect was highly significant (F(2,38) = 15.34, p < .001, ηp2 = .45), driven by the High Confidence items being more positive than both Low Confidence items (t(19) = 3.30, p < .01, 95% confidence interval, CI, of difference = .45 ~ 1.99 uV, Bayes Factor = 15.5) and Miss items (t(19) = 5.75, p < .001, 95% CI of difference = 1.30 ~ 2.78uV, Bayes Factor = 2349.0). These observations are supported by other work that has examined such differences using conventional mean ERP analyses (Friedman & Johnson, 2000).

Figure 2
The results of Experiment 1. Figure 2A and 2B depict the ERP response at Fz and alpha power response at O2 during encoding task, respectively. The gray areas represent the time windows over which the ERP amplitude and alpha power were averaged to quantify ...

We worried that the mean amplitude differences might be driven by the more jittered onset times across participants due to smaller number of trials for the Low Confidence (14% of trials) and Miss items (23% of trials) than High Confidence items (67% of trials). If it were the case, the amplitude of the frontal positivity measured with the fewest trials (i.e., Low Confidence items) should be the lowest due to the largest variability of onset times. However, the fact that the mean amplitude for Low Confidence items was significantly higher than Miss items (t(19) = 2.11, p < .05, 95% CI of difference = .01 ~ 1.63uV, Bayes Factor = 1.6) rules out this simple explanation.

Next, we examined the oscillatory activity during the encoding period. As shown in Figure 2B (see also Figure S4 and Supplemental Materials), the EEG during the encoding period showed a clear suppression of occipital alpha power following the onset of the to-be-remembered items (Hanslmayr & Staudigl, 2014). Occipital alpha power was quantified as the mean power between 8 and 12 Hz in the time window 400–1250ms after the onset of the study items at a right occipital channel (i.e., channel O2, but was similar across occipital channels, see Supplemental Materials). An ANOVA confirmed that the occipital alpha power varied as a function of subjects’ later recognition (F(2,28)= 4.88, p = .01, ηp2= .20). High Confidence items exhibited lower occipital alpha power than Low Confidence (t(19) = 2.12, p < .05, 95% CI of difference = .01 ~ 1.55uV2, Bayes Factor = 1.6) or Miss items (t(19) = 2.80, p = .01, 95% CI of difference = .24~1.69uV2, Bayes Factor = 5.7). The only other oscillation that was related to subjects’ later recognition was a low frequency frontal effect underlying the aforementioned frontal positivity (see Figure S5 and the Supplemental Materials).

No pre-encoding ERPs or oscillations were predictive of successful memory encoding in our paradigm (see the analyses in Supplemental Materials and Figures S1 & S3). This demonstrates that the memory effects were not simply due to tonic changes in brain activity that were present prior to the presentation of the memoranda. Instead, these signals reflect the ability of the brain to encode accurate representations of the items immediately following their presentation.

Forecasting later recognition of an object

How would one forecast the later recognition of an item based on the electrophysiological signals of memory encoding? Our approach in Experiment 1 was to compute measures of successful memory encoding given the magnitude of the frontal positivity and the strength of occipital alpha power suppression for each trial. We calculated the area under the ROC curve (AUC) and the proportion of High Confidence responses to provide a diversity of the measures of successful memory encoding (also see the Supplemental Materials where we show the same pattern using the da metric of performance). We first sorted the stimuli based on the magnitude of each memory-encoding signal. Then, we computed the memory metrics in each pentile bin (i.e., each bin contained 20% of the trials). These measures estimated the strength of encoded memory given the magnitude of the electrophysiological signals.

When we sorted trials by the amplitude of the frontal positivity, there was a monotonic increase in the strength of encoded memory as a function of its magnitude (Figure 3A). We observed a significant increase in AUC, from 0.79 to 0.84 (F(4,76)=9.63, p < .001, ηp2 = .34, Figure 3B) from the first pentile to the fifth pentile, and the likelihood of a High Confidence response showed a similar increase, from 58% to 68% (F(4,76)=14.15, p < .001, ηp2 = .43, Figure 3C). When we sorted trials by the magnitude of the occipital alpha power, there was a highly significant monotonic decline in the memory strength as a function of the alpha power (Figure 3D). We observed a significant decrease in AUC, from 0.84 to 0.79 from the first pentile to the fifth pentile (F(4,76)=8.97, p < .001, ηp2 = .32, Figure 3E), and the likelihood of a High Confidence response showed a similar decrease, from 68% to 58% (F(4,76)=6.38, p < .001 ηp2 = .26, Figure 3F). These results demonstrate the reliability of both the frontal positivity and the occipital alpha power as predictors of subsequent recognition memory when measured on each trial.

Figure 3
The ROC curves and recognition performance as a function of EEG signals. The top row shows the ROC curve (panel A), AUC (panel B), and High confidence likelihood (panel C) as a function of the magnitude of the frontal positivity defined by pentiles. The ...

To test for independence between the frontal positivity and the occipital alpha power, we examined the correlation between the two signals across trials within each subject. Although the correlation coefficient was reliably different from zero (mean coefficient = −0.06, t(19) = −4.33, p < .001, 95% CI = −0.08 ~ −0.03, Bayes Factor = 132.1), the relationship accounted for less than 0.3% of the variance (see Figure S6 for the scatterplots).2 This negligible correlation between the two electrophysiological signals suggests that these signals index dissociable aspects of memory encoding.

If these signals index different encoding mechanisms, then combining these measures on each trial should result in an increase in our ability to forecast later memory performance. To test this, we sorted each trial into a two-dimensional array using the frontal positivity and the occipital alpha power as two orthogonal axes. As Figure 4 shows, for the 20% of trials with the largest frontal positivity and the lowest occipital alpha power the AUC and the likelihood of a High Confidence response was 0.77 and 75%, respectively. In contrast, for the 20% of trials with smallest frontal positivity and the highest occipital alpha power the AUC and the likelihood of High Confidence response was 0.86 and 55%, respectively. Thus, our ability to predict later memory improved substantially when we combined the two electrophysiological signals.

Figure 4
The heatmap depicting the combined predictive power of the frontal positivity and the occipital alpha power. The color of each pixel represents the AUC (panel A) and the proportion of High Confidence responses (panel B) with a certain magnitude of the ...

Discussion

In Experiment 1, we showed that the frontal positivity and the occipital alpha power indexed dissociable mechanisms of memory encoding that could predict whether a given stimulus would be remembered. Next, we asked the following two questions. First, what encoding mechanisms do our electrophysiological measures of memory encoding reflect? One hypothesis is that they index the difficulty of encoding determined by the physical properties of a stimulus (e.g., a bright object might be easier to remember than a dim object). Alternatively, they might reflect the variance in the quality of endogenous memory encoding processes. Second, can we select individual items that were poorly studied using our neural measures, target such items for remedial study, and improve the efficacy of the learning period?

Experiment 2

In Experiment 2, subjects studied 800 pictures while we recorded their EEG. Immediately following the initial study phase, we categorized the items based on the amplitudes of the two neural signals as either poorly studied or well-studied items. Subjects then restudied half of the poorly studied and well-studied items. If the frontal positivity and the occipital alpha power are stimulus-driven measures, then the restudy EEG signals should continue to respect the poorly studied and well-studied categories to the same degree. However, if the two signals reflect the endogenous variance of memory encoding, then the amplitudes of restudy EEG signals should track later recognition memory performance, instead of the categories defined during the initial study phase. Additionally, if our EEG-based memory forecasting is useful in identifying objects that are poorly studied, and, thus, needing additional studying, then we should expect that the benefit of restudying is greater for poorly studied items than that for well-studied items.

Method

Participants

A new group of 20 participants (12 males and 8 females, 18–32 years old) volunteered, following the same procedures and compensation used in Experiment 1.

Stimuli and procedures

The initial study phase was identical to that of Experiment 1, except subjects studied 800 pictures instead of 500. Approximately 5 minutes after the initial study phase, the subjects completed a restudy phase where they restudied half of poorly studied and half of the well-studied items, as defined by the EEG signals recorded during the initial study phase. We defined the well-studied items as those that elicited the largest 40% of all frontal positivities and the lowest 40% of occipital alpha power measurements. The poorly studied items were defined as those that elicited the smallest 40% of all frontal positivities and the highest 40% of occipital alpha power measurements. The pictures were presented in the same format as in the initial study phase. On average, subjects restudied 58 poorly studied pictures and 56 well-studied pictures during restudy. After the restudy phase, subjects’ eyes open and closed resting-state EEG was recorded for 15 minutes. Then, they performed the memory test. The recognition-memory test was identical to that of Experiment 1 except that subjects were tested on 5 categories of pictures; poorly studied baseline pictures (58 pictures on average), well-studied baseline pictures (56 pictures on average), poorly studied restudied pictures (58 pictures on average), well-studied restudied pictures (56 pictures on average), and 160 new pictures.

Results

Participants recognized 80%, 79%, 53%, and 44% of pictures with 100% confidence for the well-studied restudy items, poorly studied restudy items, well-studied baseline items, and poorly studied baseline items, respectively. Of the remaining, 9%, 10%, 17% and 21% of studied items were recognized with moderate (<=80%) confidence for the well-studied restudy items, poorly studied restudy items, well-studied baseline items, and poorly studied baseline items, respectively. For new items, subjects successfully rejected 73% of items. Table 2 reports the proportion of trials used to derive the ROC curves. The AUC values were 0.76, 0.73, 0.88, and 0.88 for well-studied baseline items, poorly studied baseline items, well-studied restudied items, and poorly studied restudied items, respectively. These results demonstrate that participants learned the pictures reasonably well and benefitted from restudy.

Table 2
Behavioral results of Experiment 2.

Figure 5 shows the amplitude of the frontal positivity and the occipital alpha power during the initial study phase (Figure 5A & B) and during the restudy phase (Figure 5C & D) elicited by poorly studied and well-studied items. Figure 5C shows that the difference in the sustained frontal positivity was significant (t(19) = 2.59, p < .05, 95% CI of difference = .20 ~ 2.00 uV, Bayes Factor = 3.8), but much reduced during the restudy phase. Figure 5D shows that the difference in the occipital alpha power was much reduced and not significant in the restudy phase (t(19) = 1.57, p > .1, Bayes Factor in favor of the null = 1.44). These findings are inconsistent with what we should have observed if the neural signals were due to the physical characteristics of the stimuli, and consistent with the signals tracking the endogenous state of the subject during encoding.

Table 2 and Figure 6 show the performance from the final recognition test. First, we replicated the results from Experiment 1. That is, we found that for baseline (i.e., not restudied) items, the memory strength was significantly weaker for the items that elicited a low frontal positivity and high occipital alpha power (i.e., poorly studied items) than those that elicited a high frontal positivity and low occipital alpha power (i.e., well-studied items) (t(19) = 2.63, p <.02, 95% CI of difference = .01 ~ .06, Bayes factor = 4.1 for AUC, t(19) = 4.22, p <.0001, 95% CI of difference = 4 ~ 13%, Bayes Factor = 105.0 for the likelihood of High Confidence responses). More critically, recognition performance was essentially identical across the two types of restudied items (t(19) = 0.24, p >8, Bayes Factor in favor of the null = 4.5 for AUC, t(19) = 0.23, p >.8, Bayes Factor in favor of the null = 4.5 for the likelihood of High Confidence responses), leading to a significant interaction between item category (poorly studied versus well studied) and study condition (baseline versus restudy) (F(1,19) = 8.8, p < .01, ηp2 = .32 for AUC, F(1,19) = 13.48, p < .01, ηp2 = .42 for the likelihood of High Confidence responses). In fact, the restudy effect in terms of the likelihood of High Confidence responses was 1.3 times larger for poorly studied items than well-studied items (27% versus 35%, in percent change, respectively).

Figure 6
The behavioral restudy effect in Experiment 2

Next we addressed the possibility that the lack of the difference between recognition accuracy for well-studied and poorly studied items following restudy was simply due to a ceiling effect that eliminated the true difference that would otherwise be observed. In other words, maybe the restudy benefit was larger for poorly studied items than for the well-studied items because every restudied stimulus was relearned maximally. If so, there should be no variability left in recognition performance to be explained by the electrophysiological signatures measured during the restudy phase. To address this, we classified the restudied items as poorly restudied and well restudied based on the signals recorded during the restudy phase. Again, we found that well-restudied items had a significantly higher memory strength than poorly restudied items (0.92 versus 0.89 for AUC, t(19) = 2.3, p < .05, 95% CI of difference = 0.02 ~ 0.52, Bayes Factor = 2.2; .85 versus .78 for the likelihood of High Confidence responses, t(19) = 2.9, p = .01, 95% CI of difference = 2 ~ 12%, Bayes Factor = 6.9). This indicated that not all the restudied items were encoded to ceiling. Instead, the variability in the encoding quality for restudied items was still distinguishable using the frontal positivity and the occipital alpha power. Therefore, the significant interaction between study condition and item category does not appear to be due to a ceiling effect for restudied items obscuring a potential difference.

Discussion

In Experiment 2, we discriminated between exogenous and endogenous explanations of the variability in our electrophysiological indices of memory encoding. Our results indicate that both the frontal positivity and the occipital alpha power heavily reflect endogenous variability in memory encoding processes. There appears to be only a hint of exogenous contribution to the difficulty of encoding on these electrophysiological signals evidenced by a small, but preserved difference in the frontal positivity for poorly studied and well-studied items during the restudy phase. Furthermore, by having subjects restudy the items that were classified as poorly studied by our electrophysiological signals, we were able to dramatically enhance the efficacy of learning. Thus, these results provide theoretical insight as to the nature of the frontal positivity and the occipital alpha signals of memory encoding, and provide a clear demonstration of the practicality of our EEG-based learning intervention.

General Discussion

Our ability to encode new information fluctuates from moment to moment. It would be extremely valuable if we could identify in real time when we are not encoding information into memory to the best of our ability. Numerous studies have successfully identified neural signals sensitive to success in later recognition memory tests. However, no study so far had examined the usefulness of such signals in forecasting the later recognition of each studied stimulus, and using this forecast to improve learning as we study.

In Experiment 1, we simultaneously measured two electrophysiological signals that differentiated later recognized items from later missed items, the sustained frontal positivity and the occipital alpha power. We found that these signals revealed a reliable and dissociable ability to predict subsequent memory, and combining them improved the predictive power. These findings support the hypothesis that underlying the frontal positivity and the occipital alpha power are dissociable cognitive subprocesses that conjunctively determine the efficacy of memory encoding.

In Experiment 2, we used the two brain signals to identify items that needed restudying during the learning episode, allowing us to intervene and improve our subjects’ recognition memory. Here we hypothesized that restudying items that were initially poorly studied (i.e., forecasted to be recognized at a low rate) would lead to a greater enhancement of overall recognition memory than restudying initially well-studied items (i.e., forecasted to be recognized at a high rate). We indeed found that restudying the poorly studied items led to a benefit of restudying that was 30% larger than the benefit for restudying initially well-studied items. This restudy effect, along with the much reduced difference in the brain signals in the restudy phase between poorly studied and well-studied items, suggests that the encoding quality read out by the two brain signals are due to internal fluctuations in the ability of subjects to store information in memory, rather than low-level variability of the stimuli themselves.

Our findings have broad theoretical and practical implications. Our evidence that the modulations of both the frontal positivity and the occipital alpha power reflect endogenous variability in memory encoding is in line with previous studies suggesting that both signals are sensitive to the depth of processes brought to bear on to-be-remembered information (Hanslmayr et al., 2009; Hanslmayr & Staudigl, 2014; Otten et al., 2001). Interestingly, our correlational analysis revealed that the two measures account for dissociable variance in memory performance. What might each neural correlate represent? Fernandez and colleagues (Fernandez et al., 1999) showed that the characteristics of the frontal positivity closely resembled the local-field potentials recorded at the hippocampus, but not at the rhinal cortex. This observation suggests that the frontal positivity reflects the hippocampus-dependent encoding processes such as formation of source memories for later recollection (see Diana, Yonelinas, & Ranganath, 2007 for review). As for the occipital alpha power suppression, one might hypothesize that it reflects a higher level of arousal. However, if that were the case, one would expect alpha suppression to be evident even before the stimuli appeared. The fact that the alpha power suppression was stimulus-locked, not preceding the stimulus, suggests that this effect was specific to the memory encoding itself. One potential explanation offered by Klimech (2012) is that the sustained alpha band suppression indicates successful access to information already stored in long-term memory, thus indicating better associative learning. To better characterize the functional differences of the two neural signals, it is critical to experimentally dissociate these two signals in the future studies.

Importantly, we are not claiming that the two electrophysiological measures we used are the only signals that predict successful memory encoding. Previous studies that utilized different experimental procedures have shown that other electrophysiological signals differentiated later-recognized items from later-forgotten items (Addante, Watrous, Yonelinas, Ekstrom, & Ranganath, 2011; Dube, Payne, Sekuler, & Rotello, 2013; Karis, Fabiani, & Donchin, 1982; Osipova et al., 2006; Otten, Quayle, Akram, Ditewig, & Rugg, 2006; Otten, Quayle, & Puvaneswaran, 2010). Therefore, it will also be important for future studies to systematically examine what determines the usefulness of each signal in predicting successful memory encoding. This will be critical for using these signals in the real world to improve learning, as we discuss next.

From a practical perspective, our findings demonstrate the feasibility of monitoring the moment-to-moment fluctuations of encoding in real time using noninvasive electrophysiology. The relative ease and cost effectiveness of acquiring EEG data compared to other neural signals (e.g., BOLD responses in fMRI) means that the present measurements and procedure could quickly translate into real-world applications. The fact that our analysis required only two recording electrodes to successfully forecast subsequent memory performance is an additional advantage. The results from Experiment 2 demonstrate one way to utilize this electrophysiology-based forecasting to efficiently improve an individual’s subsequent memory by measuring activity in real time as people learn new information.

Similar approaches of monitoring the quality of encoding have been attempted by assessing learners’ subjective judgments about the quality of learning (i.e., judgments of learning, or JOL, (Metcalfe, 2009). Although some studies showed that JOL can be a reliable measure of successful learning in certain situations (Nelson & Dunlosky, 1991; Underwood, 1966), other studies showed that the reliability of such meta-memory judgments varied wildly depending on the specific task or the subject population (Daniels, Toth, & Hertzog, 2009; Kornell & Bjork, 2007; Maki, 1998; Serra & Metcalfe, 2009; Townsend & Heit, 2011). Using neural signals as the predictors of encoding quality could potentially bypass such problems. The methods developed here could be particularly advantageous for individuals who exhibit conditions that impair learning (e.g., dyslexia or attention-deficit hyperactivity disorder).

Supplementary Material

sup file

Acknowledgments

This work was supported by grants from the National Institutes of Health (R01-EY019882, and P30-EY08126) and National Science Foundation (BCS-0957072). We thank Stephan Lindsay, Chad Dube, and anonymous reviewers for helping us to improve the paper.

Footnotes

1This was done to remove the potential confound of lateralized response-related potentials (e.g., the lateralized-readiness potential) from the recognition effect (i.e., “old/new” effect).

2To achieve a normal distribution for occipital alpha power, the alpha power was log transformed before examining the correlation with the frontal positivity. Of note, the correlational analysis using the raw alpha power revealed the same result.

Competing Financial Interests

None declared

References

  • Addante RJ, Watrous AJ, Yonelinas AP, Ekstrom AD, Ranganath C. Prestimulus theta activity predicts correct source memory retrieval. Proc Natl Acad Sci U S A. 2011;108(26):10702–10707. [PubMed]
  • Brady TF, Konkle T, Alvarez GA, Oliva A. Visual long-term memory has a massive storage capacity for object details. Proceedings of the National Academy of Sciences of the United States of America. 2008;105(38):14325–14329. [PubMed]
  • Daniels KA, Toth JP, Hertzog C. Aging and recollection in the accuracy of judgments of learning. Psychol Aging. 2009;24(2):494–500. [PMC free article] [PubMed]
  • Diana RA, Yonelinas AP, Ranganath C. Imaging recollection and familiarity in the medial temporal lobe: a three-component model. Trends Cogn Sci. 2007;11(9):379–386. [PubMed]
  • Dube C, Payne L, Sekuler R, Rotello CM. Paying attention to attention in recognition memory: insights from models and electrophysiology. Psychol Sci. 2013;24(12):2398–2408. [PMC free article] [PubMed]
  • Fernandez G, Effern A, Grunwald T, Pezer N, Lehnertz K, Dumpelmann M, et al. Real-time tracking of memory formation in the human rhinal cortex and hippocampus. Science. 1999;285(5433):1582–1585. [PubMed]
  • Friedman D, Johnson R., Jr Event-related potential (ERP) studies of memory encoding and retrieval: a selective review. Microsc Res Tech. 2000;51(1):6–28. [PubMed]
  • Friedman D, Trott C. An event-related potential study of encoding in young and older adults. Neuropsychologia. 2000;38(5):542–557. [PubMed]
  • Golby A, Silverberg G, Race E, Gabrieli S, O’Shea J, Knierim K, et al. Memory encoding in Alzheimer’s disease: an fMRI study of explicit and implicit memory. Brain. 2005;128(Pt 4):773–787. [PubMed]
  • Hanslmayr S, Spitzer B, Bauml KH. Brain oscillations dissociate between semantic and nonsemantic encoding of episodic memories. Cereb Cortex. 2009;19(7):1631–1640. [PubMed]
  • Hanslmayr S, Staudigl T. How brain oscillations form memories--a processing based perspective on oscillatory subsequent memory effects. Neuroimage. 2014;85(Pt 2):648–655. [PubMed]
  • Karis D, Fabiani M, Donchin E. “P300” and memory: Individual differences in the von Restorff effect. Cognitive Psychology. 1982;16(2):177–216.
  • Klimesch W. alpha-band oscillations, attention, and controlled access to stored information. Trends Cogn Sci. 2012;16(12):606–617. [PMC free article] [PubMed]
  • Klimesch W, Schimke H, Doppelmayr M, Ripper B, Schwaiger J, Pfurtscheller G. Event-related desynchronization (ERD) and the Dm effect: does alpha desynchronization during encoding predict later recall performance? Int J Psychophysiol. 1996;24(1–2):47–60. [PubMed]
  • Kornell N, Bjork RA. The promise and perils of self-regulated study. Psychon Bull Rev. 2007;14(2):219–224. [PubMed]
  • Maki RH. Predicting performance on text: delayed versus immediate predictions and tests. Mem Cognit. 1998;26(5):959–964. [PubMed]
  • Metcalfe J. Metacognitive Judgments and Control of Study. Curr Dir Psychol Sci. 2009;18(3):159–163. [PMC free article] [PubMed]
  • Nelson TO, Dunlosky J. When Peoples Judgments of Learning (Jols) Are Extremely Accurate at Predicting Subsequent Recall - the Delayed-Jol Effect. Psychological Science. 1991;2(4):267–270.
  • Osipova D, Takashima A, Oostenveld R, Fernandez G, Maris E, Jensen O. Theta and gamma oscillations predict encoding and retrieval of declarative memory. J Neurosci. 2006;26(28):7523–7531. [PubMed]
  • Otten LJ, Henson RN, Rugg MD. Depth of processing effects on neural correlates of memory encoding: relationship between findings from across- and within-task comparisons. Brain. 2001;124(Pt 2):399–412. [PubMed]
  • Otten LJ, Quayle AH, Akram S, Ditewig TA, Rugg MD. Brain activity before an event predicts later recollection. Nat Neurosci. 2006;9(4):489–491. [PubMed]
  • Otten LJ, Quayle AH, Puvaneswaran B. Prestimulus subsequent memory effects for auditory and visual events. J Cogn Neurosci. 2010;22(6):1212–1223. [PMC free article] [PubMed]
  • Paller KA, Kutas M, Mayes AR. Neural correlates of encoding in an incidental learning paradigm. Electroencephalogr Clin Neurophysiol. 1987;67(4):360–371. [PubMed]
  • Paller KA, McCarthy G, Wood CC. ERPs predictive of subsequent recall and recognition performance. Biol Psychol. 1988;26(1–3):269–276. [PubMed]
  • Paller KA, Wagner AD. Observing the transformation of experience into memory. Trends Cogn Sci. 2002;6(2):93–102. [PubMed]
  • Serra MJ, Metcalfe J. Effective Implementation of Metacognition. In: Hacker DJ, Dunlosky J, Graesser AC, editors. Handbook of Metacognition in Education. New York, NY: Taylor & Francis; 2009.
  • Townsend CL, Heit E. Judgments of learning and improvement. Mem Cognit. 2011;39(2):204–216. [PMC free article] [PubMed]
  • Underwood BJ. Individual and group predictions of item difficulty for free learning. J Exp Psychol. 1966;71(5):673–679. [PubMed]
  • Voss JL. Long-term associative memory capacity in man. Psychonomic Bulletin & Review. 2009;16(6):1076–1081. [PubMed]
  • Wagner AD, Schacter DL, Rotte M, Koutstaal W, Maril A, Dale AM, et al. Building memories: remembering and forgetting of verbal experiences as predicted by brain activity. Science. 1998;281(5380):1188–1191. [PubMed]