|Home | About | Journals | Submit | Contact Us | Français|
The hippocampus is required for the encoding, consolidation, and retrieval of event memories. While the neural mechanisms that underlie these processes are only partially understood, a series of recent papers point to awake memory replay as a potential contributor to both consolidation and retrieval. Replay is the sequential reactivation of hippocampal place cells that represent previously experienced behavioral trajectories and occurs frequently in the awake state, particularly during periods of relative immobility. Awake replay may reflect trajectories through either the current environment or previously visited environments that are spatially remote. The repetition of learned sequences on a compressed time scale is well suited to promote memory consolidation in distributed circuits beyond the hippocampus, suggesting that consolidation occurs in both the awake and sleeping animal. Moreover, sensory information can influence the content of awake replay, suggesting a role for awake replay in memory retrieval.
Our memories are central to our sense of self. We use past experiences to guide current decisions, an ability that requires both memory storage and retrieval. The highly plastic hippocampal circuit is believed to be the initial site of encoding episodic memories. Evidence suggests that there is a subsequent process during which the hippocampus interacts with the rest of the brain to engrain stable, long lasting representations in hippocampal-neocortical circuits1–4. While there is still debate about how long this consolidation process lasts and whether memories ever become truly independent of the hippocampus5, 6, it is clear that the hippocampus plays an essential role in the initial encoding and subsequent stabilization of long term memories7–9. This stabilization is thought to depend on the reactivation of previously encoded associations, engraining those associations into the less-plastic neocortex10, 11. Mechanisms underlying this process would need to have several properties. First, putative patterns of hippocampal activity supporting consolidation should repeatedly reactivate mnemonic representations in the absence of behavioral repetition. Further, as consolidation refers to the progressive stabilization of a memory trace over time, processes that support consolidation would need to persist for some time period following the experience. Finally, consolidation processes would need to promote plasticity in distributed neocortical circuits, allowing memories that initially depend on the hippocampus to become encoded in more distributed networks over time.
The reactivation of stored hippocampal representations during sharp wave ripples (SWRs) is a physiological pattern of activity that exhibits all of these properties, suggesting a role for reactivation in consolidation11–15. Reactivation of cell pairs following an experience was first studied during sleep13, 16 which reinforced the idea that sleep is a privileged state for memory consolidation processes14, 17. Kudrimoti and colleagues18 first showed that SWR reactivation occurs in the awake animal. Recently it has become clear that hippocampal representations are replayed frequently and with high fidelity in the awake state, suggesting that replay during both waking and sleep support memory consolidation.
After initial learning, we are able to retrieve stored associations about past experiences to guide ongoing behavior. Lesions of the hippocampus lead to severe deficits in the retrieval of recently stored associations, suggesting that the hippocampus plays a fundamental role in memory retrieval for at least a period of time after the experience3, 19, 20. The patterns of hippocampal activity that support retrieval likely exhibit several properties. First, retrieval events must occur during waking behavior and display the “mental time-travel” associated with episodic memory recall21. In other words, retrieval should reactivate the internal representation of an experience in the absence of behavioral repetition. Second, we can recollect a memory on a faster timescale than the original experience, suggesting that retrieval of stored representations can be compressed in time. Third, current sensory input should be able to influence which stored representations are retrieved, as in the case of cued memory recall. Finally, in order to retrieve a memory stored in distributed hippocampal-neocortical circuits, a retrieved hippocampal trace would need to either initiate or be part of a broader neocortical retrieval event that could be used to make memory guided decisions11, 22, 23.
It is unknown what patterns of hippocampal activity support memory retrieval. Theta coherence between the hippocampus and pre-frontal cortex has been observed at locations where animals must execute a memory guided decision24, 25. As place field activity is prominent during theta, theta coherence seems well suited to allow information from the hippocampus about the animal’s current position or current trajectory26–28 to trigger place-related associations in downstream regions such as the prefrontal cortex. Larger place fields in the ventral hippocampus may also provide information about the broader spatial context29. Nonetheless, during theta, only cells whose place fields overlap with the animal’s current location will be activated. Distinct environments are generally represented by distinct sets of place cells30, suggesting that place cell activity in one environment is unlikely to be able to retrieve representations of a previously experienced environment. In contrast, awake replay is consistent with the patterns of activity necessary to support a hippocampal retrieval mechanism, suggesting that hippocampal replay during the awake state is a common neural mechanism for both consolidation and retrieval31, 32.
Sharp-wave ripples originate in the hippocampus and are triggered by synchronized activation of CA3 pyramidal cells, leading to characteristic negative potentials (sharp waves) in the CA1 stratum radiatum12, 33. When recording in area CA3, this synchronized activation of CA3 pyramidal cells can be measured as a 100–150 Hz oscillation34. The population burst in the CA3 region recruits CA1 pyramidal cells as well as basket and chandelier cells, leading to a transient (~100 ms) “ripple” oscillation (150–250Hz) in the CA1 pyramidal cell layer35. The short latency bursts of CA3 and CA1 neurons during SWRs appears well suited to induce synaptic plasticity12, 36. SWRs are prominent during sleep as well as in the awake state during immobility, consummatory behavior, grooming12, 37, and can also be seen during running38, 39. Activity during SWRs propagates from CA3 to CA1, one of the major output areas of the hippocampus, and then out to neocortex40–42.
The extensive excitatory recurrent connections among CA3 pyramidal cells43 have led some to suggest that CA3 acts as an auto-associative pattern completion network44, 45. Thus the activation of a small subset of CA3 neurons could initiate a cascade of excitation across previously modified synapses, leading to reverberating activity that eventually settles into an attractor state corresponding to a previously stored memory. Reinstatement of stored representations in CA3 could then reinstate the corresponding representations in CA1 through feed forward excitation. Hippocampal replay during SWRs is thought to reflect this type of auto-associative pattern completion in CA3 in conjunction with feed forward recruitment of pyramidal cells and interneurons in CA1. This model is consistent with the observation that the fidelity of reactivation (measured across cell pairs) is higher in CA3 than CA131.
Foster and Wilson46 first demonstrated that entire behavioral sequences are replayed during awake SWRs. They found that when animals stopped and consumed reward following traversal of a linear track, sequences of place cells representing trajectories on the track were replayed during individual SWRs. These awake replay events originated at the animal’s current position and reactivated place cells in the reverse order as had been experienced behaviorally (see Box for discussion of methodological issues). Reverse replay was seen immediately after the very first traversal; demonstrating that the hippocampus can replay sequences that are experienced only once and suggesting that replay contributes to one-trial learning. Behaviorally relevant sequences were replayed repeatedly, such that sequences were replayed more often than the trajectories they represented had been experienced.
Further examination of SWR activity revealed that awake replay could occur in either the same (forward) or the opposite (reverse) direction as observed during behavioral traversal47 (Fig. 1). Diba and Buzsaki found that the direction of replay was related to the animal’s behavior. Reverse replay occurred preferentially at the end of runs when the animal reached the reward location, potentially linking behavioral trajectories to their outcomes46. Forward replay occurred preferentially at the beginning of runs, perhaps providing information relevant for evaluating future trajectories47.
These studies focused on relatively small environments where replay occurring during the time frame of a single SWR (~100ms) could represent an entire behavioral trajectory. It was therefore unclear whether SWR replay could recapitulate the extended sequences an animal might experience in a more naturalistic setting. Davidson and colleagues48 examined hippocampal activity while animals ran along a 10 meter track and found that both forward and reverse replay events spanned long distances and could extend across multiple SWRs. These results indicate that the timescale of a single SWR event does not limit the extent of memory reactivation.
In all of these studies, animals traversed locations in both directions of motion, so forward replay in one direction was similar to reverse replay in the other (see Box). The presence of both forward and reverse replay was verified in a recent study in which the animal’s direction of travel along the track was restricted, resulting in overwhelming experience in one direction49. Furthermore, activity consistent with forward and reverse replay has also been observed in an open-field maze50, where animals do not traverse paths in a stereotyped fashion. These results suggest that replay reflect trajectories through a “cognitive map” that represents the relationships between locations or episodes51 rather than simply recapitulating recent experiences. This hypothesis is consistent with the observation that replay events can occasionally represent “short-cut” sequences made up of joined forward and backward sequences that were never experienced together49.
While the presence of awake replay has been conclusively established, there are a number of outstanding questions. First, it is unclear how sequences experienced in only one direction can be stored in such a way as to allow for both forward and reverse replay. It is also not clear whether forward and reverse events serve similar or different roles. Forward replay during both behavior and subsequent sleep seems well suited for consolidation of representations related to experienced trajectories and forward replay during behavior may enable the retrieval of future paths to aid memory guided decision making31, 47. Reverse replay during behavior seems well suited for linking recently experienced sequences to their outcome, which may promote task learning46, 47, 52. Intriguingly, reverse replay does not seem to be present during sleep53. This raises the possibility that the content of hippocampal replay is actively controlled, and that different directions of replay serve different functions.
Sleep replay reinstates neural representations of previous experiences in the absence of associated sensory inputs13. In contrast, early observations of awake replay suggested that sensory input could explain the sequential reactivation during replay. One model posited that progressive depolarization during SWRs unmasks sub-threshold place field, resulting in sequential reactivation46, 47, 50. While this model is appealing, recent observations of remote replay during waking behavior demonstrate that this sub-threshold model cannot fully explain the occurrence of awake replay. Jackson and colleagues54 first suggested that activity during awake SWRs could reactivate past, rather than current, experiences by measuring the similarity of pair-wise neural co-firing during awake SWRs and during awake exploration. For some recording sessions, the same cell assemblies were active during awake SWRs as had been activated during exploration in a previously experienced environment54.
Karlsson and Frank31 demonstrated that entire sequences of neurons representing trajectories through a previously explored environment were replayed during subsequent experience in distinct environments (Fig. 2). The reactivation of a distant memory during ongoing behavior raises the intriguing possibility that awake remote replay, in conjunction with local place-related input, could contribute to the formation of associations between past and current events. Remote replay was detected in over 40% of the SWRs for which sufficient numbers of neurons with place fields in the remote environment were active and remote replay continued throughout a two-hour recording session. Replay of the local environment was similarly detected in over 40% of the SWRs with sufficient levels of place field activity. Most replay events corresponded to trajectories through either the local or the remote environment, but not both31 (Fig. 2). Replay of the local environment could represent a sequence of locations distant from the animals’ current position, a result which was been replicated in subsequent studies31, 48, 49. Surprisingly, Karlsson and Frank found that awake replay is a higher fidelity recapitulation of past experiences than replay seen during quiescent, sleep-like states31, suggesting that awake replay is particularly important for memory consolidation.
The observation that a large proportion of replay events represent coherent trajectories through either the local or a remote environment raises the possibility that a complete sampling of all experienced environments would reveal that most SWR activity is consistent with experiences in some environment. Alternatively, some SWRs may represent patterns of activity generated as a result of anatomic and synaptic connectivity that does not reflect past experience. Future theoretical work on the structure of all SWRs and experiments that record place cell activity and replay across all of the environments an animal experiences on a daily basis will help determine the extent to which SWR activity reflects experience.
We form strong long lasting memories of salient experiences, suggesting that these experiences enhance memory consolidation processes. Novel and rewarding experiences, which are likely to be particularly salient, are marked by enhanced awake replay.
Memory encoding and consolidation are particularly important when new information needs to be learned, as in a novel environment. Awake replay is more prevalent in a novel environment than in a familiar one46, 47. In addition to being more prevalent, reactivation is more temporally precise in a novel environment and as locations become more familiar, the precision of awake reactivation decreases39. This enhanced precision may be important for consolidation processes as precise co-firing of neurons is thought to drive synaptic plasticity55, 56. In addition, novelty increases the reactivation of neurons associated with new experiences during subsequent sleep57.
While receipt of reward is not necessary for awake replay to occur50, 54, SWRs occur frequently when animals are at rewarded locations. To investigate the relationship between reward and reactivation, Singer and Frank52 developed a sequence-switching task where animals relied on changing reward contingencies to guide their behavior. They found that there were more SWRs and that CA3 pyramidal cells were more active during SWRs at the end of rewarded trajectories as compared to unrewarded trajectories. The activity during these SWRs was consistent with ordered replay of paths to and from rewarded locations. Although the sequence-switching task occurred in a familiar environment, reward related SWR activity was enhanced when animals first learned a novel reward contingency, suggesting that activity during awake SWRs contributes to learning the associations between actions and their outcomes.
It is well established that salient events alter hippocampal activity in part through the activation of neuromodulatory inputs such as the dopaminergic58, noradrenergic59 and cholinergic systems60, and that these neuromodulatory systems modify hippocampal synaptic plasticity (for reviews see:61–63). For example, both novelty and reward are associated with activation of midbrain dopaminergic neurons58. Late phase long-term potentiation (LTP) in the hippocampus is enhanced by dopamine agonists64 and impaired by blockade of dopamine receptors65. Furthermore, novelty leads to a dopamine-dependent reduction in the threshold for LTP induction at the CA3-CA1 synapse66. These findings suggest that during exploration of novel environments and following receipt of reward, the hippocampus exists in a privileged state for the induction and maintenance of plasticity. As spiking during SWRs has temporal structure similar to that used to induce LTP12, replay events occurring during these highly plastic periods may contribute to consolidation by supporting the stabilization of representations within the hippocampal network. Furthermore, effects of neuromodulatory systems such as this may contribute to the preferential encoding of salient memories outside the hippocampus as well.
The discovery of awake remote replay demonstrates that awake replay can reinstate stored representations that are not dependent on local place field activity. However, current sensory input can bias the initiation of awake replay events. Awake replay often begins at the animal’s current location and moves away from the animal in either the forward or reverse direction31, 46–48. At the onset of an awake SWR, neurons with place field centers near the animal’s current location have a higher probability of firing and show a shorter latency to fire38, 50. The effect of location on firing probability decreases throughout the replay event50. Cells active in the current location may influence the initiation of awake replay by acting as “initiator cells”36 that trigger the reactivation of previously stored sequences. This is similar to a cued memory retrieval process where current sensory input triggers retrieval of relevant episodic sequences. The retrieval of relevant sequences may help evaluate future choices and enable memory-guided decision making.
Local spatial input also affects remote replay. Karlsson and Frank31 demonstrated that when animals are awake, cells that initiate remote replay events have high local spatial firing rates. This appears to be a result of the overlap in the set of cells that are active in any two distinct environments. In general, hippocampal place cells form unique maps of each experienced environment and only about 40–50% of CA1 place cells are active in any given environment67, 68. As a result, if an animal has experienced two distinct environments, a subset of neurons that are active in one environment will also be part of the representation of the other. Local spatial input to cells which represent locations in both environments can therefore participate in replay of sequences from either environment31. These cells can act as “bridges” or “nodes” across multiple representations and may enable the retrieval of related past experience in order to guide behavior in the current environment (Fig. 3)11.
Awake replay has the potential to meet all of the criteria for a mechanism that supports memory consolidation. Awake replay repeatedly reactivates patterns associated with past experience on a timescale consistent with the induction of synaptic plasticity in the absence of behavioral repetition. Consistent with this possibility, Dupret and colleagues recently demonstrated that the strength of awake reactivation is predictive of subsequent memory69. This study examined reactivation during and after experiences where animals learned novel associations between locations and rewards. They found that the intensity of reactivation occurring during brief pauses in exploration was related to subsequent performance, while the intensity of reactivation occurring during extended immobility was not. In addition, performance on this memory task was correlated with reactivation during a rest period immediately after the experience.
Causal evidence for a role of awake replay in memory consolidation is still lacking, but two recent studies have used real-time disruption of SWRs during sleep to demonstrate the link between hippocampal SWRs during sleep and learning70, 71. These studies disrupted SWRs during a sleep session immediately following the training sessions and found that task acquisition was impaired. A similar study of the role of awake SWRs during spatial and task-related learning would clarify the link between awake SWRs and memory consolidation. Furthermore, appropriate controls must be done to demonstrate that disruption of SWRs does not cause synaptic plasticity or disrupt hippocampal representations.
During sleep, activity during SWRs propagates from the hippocampus to prefrontal cortex42 and entorhinal cortex40, 72, and replay of mnemonic activity during sleep has been observed in multiple brain areas73–77. Similar studies of awake replay will be required to determine how awake replay events affect regions outside the hippocampus. Ideally these studies would also evaluate the effect of disrupting awake replay on representations in these distributed circuits. It will also be necessary to extend the investigation of replay beyond spatial sequences using multi-modal behavioral tasks, since the hippocampus is known to be involved in transfer of rules and schemas related to behavior78. Replay is most prevalent immediately following an experience and decays with time, however replay persists at above chance levels even 18–24 hours after an experience18, 31. Future studies measuring and disrupting replay over longer periods of time in behavioral tasks for which the time course of consolidation has been worked out will be needed to address questions about whether replay persists and is necessary for the entire time course of consolidation. Finally, it is important to remember that consolidation is a complex process that involves neural activity, protein synthesis, and gene expression79. Understanding the associated patterns of neural activity will only address part of the consolidation process.
Awake replay also appears well suited to support memory retrieval processes. Awake replay reactivates memory traces from past experience on a compressed timescale. Local input can bias the content of awake replay, suggesting that awake replay can act as a cue-triggered retrieval process. Awake replay is also very similar to the “vicarious trial and error” events reported by Johnson and Redish80. They showed that replay of possible future trajectories is seen in a multiple-T maze tasks where animals must make memory guided decisions80, 81. While these vicarious trial-and-error events were reported to occur outside of SWRs, this study used a very restrictive criterion to define SWR activity and examined SWRs in CA3, for which the criterion for identifying SWRs is less well established34 (see Box). Thus it remains possible that these vicarious trial and error events are similar to awake replay events occurring during SWRs. In support of this idea, Karlsson and Frank found that awake replay during SWRs are seen frequently at the “choice point” in a similar task where animals must make a memory guided decision between two trajectories31. Further, these awake replay events frequently began well before the choice point and extended out to the left or the right arms of the track, as would be expected if the animal were playing out possible upcoming choices, consistent with the activity observed in the vicarious trial and error events.
The link between awake replay and memory retrieval remains correlative and a number of important questions remain to be answered. It will be essential to specifically disrupt awake SWRs at times when a memory-guided decision must be made. Further, future studies will need to investigate the relative contributions of awake replay and place-related firing during theta to memory retrieval processes. We suggest that awake replay and place field activity during theta support different types of retrieval, with awake replay supporting the retrieval of spatially or temporally remote experiences and theta supporting the retrieval of associations related to the animal’s current location.
We also need to understand how forward and reverse replay affect neocortical regions, in both animal models and humans. We are able to trace events backwards in time, but the conscious experience of memory retrieval occurs forward in time. We speculate that forward and reverse replay are triggered at different times to support different types of memory processes. Finally, it is unknown whether the relatively short bursts of activity seen during single SWRs can trigger long lasting reverberatory processes48 in distributed circuits that correspond to the perceived timescale of memory retrieval processes.
The hippocampus contributes to the rapid encoding, subsequent consolidation, and ongoing retrieval of stored memories. Both consolidation and retrieval are thought to depend on the reactivation of previously stored patterns of neural activity. However, since consolidation has been primarily associated with sleep and retrieval with waking behavior, the idea that a common process supports both functions is not widely accepted. Awake replay is robust and prevalent in brief periods of stillness during behavior and is enhanced during both novel and rewarding behaviors. The presence of remote awake replay suggests that animals do not have to sleep to consolidate memories of previous experiences and that the consolidation process begins as soon as new memories are encoded in the hippocampus. Moreover, awake replay may also play a major role in memory retrieval. Local inputs bias the content of both local and remote replay, suggesting that sensory input to the hippocampus serve as a cue for targeted memory retrieval. One particularly interesting possibility is that replay events that begin at the animal’s current location provide information about possible upcoming trajectories to target brain structures such as the prefrontal cortex and nucleus accumbens. These downstream structures could then assess the value of different trajectories and make an informed choice about future actions. Similarly, awake replay of remote experience could reinstate previously learned associations in distributed hippocampal-neocortical circuits that are relevant for ongoing behavior. The possibility of replay as a common neural mechanism underlying the hippocampal contribution to memory retrieval and consolidation is parsimonious, but by no means established. We expect that efforts to address these issues will lead to a new and deeper understanding of the hippocampal contribution to memory processes.
Identifying replay requires both detection of sharp wave ripple (SWR) events and quantification of reactivated sequences. SWRs are detected in CA1 by identifying times when power in the ripple-band (150–250Hz) exceeds some threshold above the mean. Thresholds as high as seven standard deviations46, 50, 80 and as low as three standard deviations above the mean have been used for SWR detection31, 39, 47, 48, 52. There is no obvious bimodality in the SWR amplitude distribution, so there is no “correct” threshold39. While using a lower threshold may lead to incorrect classification of some events as SWRs, high thresholds will lead to the exclusion of lower amplitude SWRs. Depending on the question being asked, it is important to determine which type of error leads to more conservative estimates. SWRs can also be identified as periods with transient elevations in the multiunit activity, as these times are generally associated with increased ripple band power48. In addition, to distinguish awake replay events from sequential firing occurring during movement-related phase precession, analyses are often limited to times when the animal is immobile and the theta to delta ratio is low.
Replay is marked by the ordered reactivation of hippocampal neurons reflecting previous behavioral experiences. Two methods have been employed to detect awake replay when large ensembles of neurons have been recorded: template matching and decoding. Template matching quantifies the correlation between the sequence of firing observed during candidate events and the order of place cells as determined by the location of the place fields observed during locomotion46, 47, 49. A variant of template matching that compares spike times during exploration and awake SWRs has been used to quantify awake replay in an open field50. Template matching can reliably detect large replay events where most cells with place fields are reactivated. However, because the correlation used to identify significant replay events is between one spike fired during each SWR and the relative location of place field centers, template matching is less effective when only a subset of neurons participate in the SWR event. Furthermore, because place fields are not always well defined by their center location, there can be ambiguity in defining the template sequence. Finally, in a novel environment, place cells tend to be active in both directions of motion along linear tracks82, making it difficult to construct unambiguous directional templates.
Bayesian decoding is a more powerful method of detecting sequential reactivation, partially because this approach utilizes all of the spiking during SWRs and the entire place field structure. Decoding translates the ensemble spiking during a period of time into an estimate of a series of positions31, 48, 80. An event is considered to be a significant replay event if this series of decoded positions is more consistent with an ordered trajectory through the environment than a large set of shuffled decoded trajectories. To avoid temporal smoothing across neighboring estimates, and thus the false detection of replayed trajectories, it is important that the decoding algorithm be memoryless31, 48.
Ensemble measures such as Bayesian decoding require that large populations of neurons be recorded simultaneously. When only a small population is sampled, pair wise measures can identify activity that is consistent with awake replay. For clarity, we refer to ensemble measures as replay and pair-wise measures as reactivation although the two methods likely detect the same network events. Pair-wise measures quantify the likelihood of two cells firing together during an awake SWR as a function of their co-activity during behavior 38, 39, 52, 54. Enhanced co-activation during awake SWRs for pairs of cells that are co-active during behavior is consistent with awake replay. Furthermore, pair-wise comparisons of the distance between place fields and the time between spikes within SWRs, can reveal activity consistent with ordered replay of behaviorally-relevant sequences31, 52.