|Home | About | Journals | Submit | Contact Us | Français|
According to the Hebb rule, the change in the strength of a synapse depends only on the local interaction of presynaptic and postsynaptic events. Studies at many types of synapses indicate that the early phase of long-term potentiation (LTP) has Hebbian properties. However, it is now clear that the Hebb rule does not account for late LTP; this requires an additional signal that is non-local. For novel information and motivational events such as rewards, this signal at hippocampal CA1 synapses is mediated by the neuromodulator, dopamine. In this Review, we discuss recent experimental findings that support the view that this “neoHebbian” framework can account for memory behavior in a variety of learning situations.
The idea that association underlies memory was already emphasized by the Greeks . The discovery of neurons and synapses raised the question of whether these structures are modifiable by experience and could thereby mediate association. A potential answer to this question was proposed by Donald Hebb . According to the Hebb rule, associations are encoded if synaptic plasticity obeys a simple rule: the synapse between cell A and cell B will be strengthened if two conditions are met: 1) the synapse from cell A onto cell B is active; and 2) cell B responds to this and other inputs by strong depolarization that triggers action potentials. Thus, if cell A represented object A and cell B represented object B, the co-occurrence of the two objects would, by the Hebb rule, strengthen the synaptic linkage between these cells. This link would subsequently be evident when only object A was presented because it would lead to the firing of cell B, thus bringing object B to mind by association.
Experimental support for the Hebb rule came through the study of long-term potentiation (LTP), an activity-dependent change in the strength of synapses. LTP has been found in many brain regions but has been most extensively studied in the CA1 region of the hippocampus. Early experiments showed that LTP in CA1 is governed by the Hebb rule: the induction of LTP requires both presynaptic input and strong postsynaptic depolarization (the role of Na+ spikes remains unclear) . Furthermore, preventing strong depolarization by injecting negative current prevents the induction of LTP . Implicit in Hebb’s rule is that LTP is specific to the synapses at which the rule is met (e.g., inactive synapses should be unaffected despite the strong postsynaptic depolarization). The ability to induce LTP at single visualizable synaptic connections by 2-photon uncaging of glutamate has directly confirmed the synapse specificity of LTP .
Other work has revealed some of the molecular mechanisms that underlie Hebbian plasiticity, allowing tests of the role of this plasticity in memory. Remarkably, the Hebbian computation in CA1 is done by a single type of molecule, a type of glutamate-activated channel termed the NMDA receptor (NMDAR). The opening of the NMDAR is Hebbian: the channel opens only if there is both presynaptic glutamate release and strong postsynaptic depolarization. When these channels open, the resulting influx of Ca2+ activates the enzyme, Ca2+/Calmodulin-Dependent Protein Kinase II (CaMKII), which then triggers the local biochemical changes that strengthen the synapse. Experiments show that genetic or pharmacological interference with NMDARs or CaMKII strongly interferes with memory formation . There is thus little doubt that a Hebbian form of LTP is important for memory formation.
But is the simple form of association envisioned by Hebb the whole story? From our daily experiences, we know that items may co-occur but be only briefly registered in conscious memory if we don’t attach importance to them. This suggests that there are additional factors that determine whether information is stored persistently. Importantly, studies of LTP (mostly in CA1) show that there is an additional factor: the persistence of LTP depends not just on the two factors of the Hebbian condition (glutamate release and postsynaptic depolarization), but also on a third, the action of the neurotransmitter dopamine  (see Box 1). Importantly, the dopamine release depends on systems-level processes that include motivation and stimulus novelty  (Box 2, Box 2 Figure 1). We term this “neoHebbian” to indicate that in addition to the 2-factor Hebbian process, stable synaptic modification requires a third signal dependent on a systems level computation.
Dopamine enhances protein synthesis within dendrites of hippocampal neurons . Other results point to a role of brain-derived neurotrophic factor (BDNF) in triggering protein synthesis . Experimental results suggest that D1 receptor- and BDNF-mediated pathways interact in the activation of extracellular-signal regulated kinase (ERK) 1/2 [mitogen activated kinase (MAPK)] activity . Active ERK1/2 can then induce nuclear transcription [via cyclic AMP response element-binding protein (CREB)], regulate translation [eg. by activating the eukaryotic translation initiation factor 4E (eIF4E)], and stimulate ribosomal function. Hippocampal late LTP is blocked when ERK1/2 activation is blocked by a dominant-negative mutation of MEK1, the kinase that directly phosphorylates ERK1/2 . The pathway for D1-dependent control of ERK1/2 has been most thoroughly studied in the striatum. Activation of cAMP, protein kinase A (PKA) and dopamine releasing protein (DARP), lead to the stimulation of protein phosphatase 1 (PP1). ERK1/2 activation occurs because PP1 dephosphorylates (thereby inactivating) the striatal enriched protein tyrosine phosphatase (STEP) that dephosphorylates active ERK1/2 .
In primates, novel stimuli can trigger burst firing of dopamine cells which habituates as stimuli become familiar . Microdialysis [90, 91] and fast scan cyclic voltammetry  show that various types of novelty produce dopamine release in the rat striatum. These include social novelty , novel tastes , and novel environments . Placing a rat in a novel environment leads to dopamine release in the hippocampus , and there is converging evidence from fMRI studies that the human SN/VTA is also activated by novelty (see main text and Figure I).
In reinforcement learning, dopamine responses conform with a signed prediction error; they increase for better than expected (reward) outcomes and decrease for worse than expected (aversive) outcomes . In rare cases, clinical requirements provide an opportunity to obtain direct recordings from the SN (e.g. in PD patients). SN neurons display higher spike rates to unexpected gains and lower rates to unexpected losses with a timing (ie. 150–375 ms) that is largely consistent with non-human primate studies. In the basal ganglia, dopamine release can reinforce action choices (‘go’ responses) within the so-called direct pathway to actively obtain rewards, whereas dips in dopamine firing can reinforce behavioral inhibition (‘no go’ responses) within the indirect pathway to passively avoid aversive outcomes  (note that an elevation of dopamine enhances LTP in the striatum, whereas a drop enhances LTD [94–96]). This functional architecture provides a plausible mechanism for instrumental learning of active responses through positive reinforcement and passive avoidance responses through punishment.
It is now well-established that dopamine neurons can also fire in response to unexpected aversive outcomes and cues predicting punishments . In monkeys, a substantial fraction of dopamine neurons in the dorsolateral SN/VTA are responsive to conditioned cues predicting aversive events; in contrast, those in the more ventromedial portions of the SN/VTA are inhibited by aversive stimuli  (but see ). Similar findings have been demonstrated in anesthetized rats, where some VTA dopamine neurons are excited by noxious stimuli , whereas others are inhibited . Recent studies in freely behaving mice have shown that VTA dopaminergic neurons can exhibit responses to different valences. Thus, although 89% of VTA dopaminergic neurons respond with activation to a conditioned reward, they also respond to fearful events, with the majority of the neurons showing suppression. Interestingly, 25% respond with excitation to both positive and fearful events . It is important to note that the fearful stimuli are maintained over the time course of the fear-inducing event, in contrast to the transient nature of a brief noxious stimulus as mentioned above. Indeed, sustained aversive stimuli or stressors (eg. repetitive footshocks or physical restraint), increase the number of dopamine neurons firing over longer periods of time . This response is consistent with the time course of the increase in dopamine release produced by sustained aversive events measured with microdialysis [103, 104].
The human lateral SN/VTA has been similarly shown to be activated during anticipation (i.e. before execution of motor responses) to acquire a reward or to avoid a punishment . This suggests that SN/VTA responses to aversive events are observed primarily under conditions that require active, rather than passive, avoidance. This view is largely consistent with the suggestion that dopamine may promote a learning process that enhances the motivational salience, the ‘incentive salience’, of reward cues  and with linking dopamine to ‘wanting’ rather than ‘liking’ in the field of addiction .
In this review, we will first summarize the evidence for the dopaminergic involvement in LTP and learning. We will then briefly discuss the different types of stimuli that can lead to dopamine release. We will then turn to evaluate a simple idea: that all stimuli leading to dopamine release should enhance memory. This has now been examined in both animal models and in humans and the results have interesting implications for how learning occurs. Finally, we will discuss the tag and capture model , that makes further predictions about how stimuli interact to affect memory. Readers interested in additional perspectives may find other recent reviews useful (eg. [10–12]).
Early experiments revealed that dopamine antagonists applied during the LTP induced by multiple tetani (generally 100Hz for 1s) had an effect that was strongly time dependent . Whereas the first phase of LTP was not strongly affected, the potentiation measured an hour after induction was strongly attenuated. LTP at these later times (termed late LTP) is dependent on protein synthesis, whereas the LTP at early times is not . It is therefore thought that dopamine antagonists prevent late LTP because they prevent the required protein synthesis (see Box 1).
Much additional research has strengthened the evidence that dopamine is required for late LTP in CA1 and identified the key role of the D1 class of dopamine receptors. Late LTP is blocked by D1 antagonists  (Fig. 1A) and by genetic deletion of D1 receptors (Fig. 1C) [15, 16]. The experiments described above were done in the slice preparation; similar effects of D1 antagonist have been observed when experiments are done in vivo [16, 17] (Fig. 1B). Furthermore, depletion of dopamine selectively blocks late LTP , a block that can be reversed by a D1 agonist (Fig. 1D). Together these results make a strong case for the involvement of dopamine in late LTP. Other experiments have directly measured event-dependent release of dopamine in the hippocampus ; less is known about the location of dopaminergic axons and the release/uptake mechanisms (Box 5).
An important question to address further is whether dopamine is involved in the stabilization of human memory. Available dopaminergic drugs have substantial D2 receptor affinity, and selective D1/D5 compounds are not readily available for use in healthy humans (but see ). Another approach is to enhance dopamine levels with L-DOPA. This has been investigated mainly with respect to working memory [124, 125], but there is one report that L-DOPA improved long-term memory .
Protein synthesis-dependent forms of consolidation are not restricted to the hippocampus . It would therefore be important to test whether dopamine plays a role in consolidation of non-episodic memory such as stimulus-familiarity.
We have focused in this review on how dopamine affects consolidation. However, dopamine may be released in anticipation of information under various conditions (Box 2), and this could affect memory encoding. There are many ways this could occur: 1) Pathways can be turned off or on by dopamine ; 2) Network oscillations can be regulated [129, 130]; 3) Inhibition can be attenuated by a postsynaptic D3-dependent process ; 4) Encoding could be inhibited due to hyperpolarization by dopamine D2 receptors leading to an encoding of only strong stimuli (ie. increasing the signal to noise ratio ); 5) Explorative behavior could be energized (particularly in response to novelty) , , ; 6) Working memory capacity and attentional control  could be affected.
Concern about encoding effects (see above) would be obviated if the “strong” stimulus that induces dopamine release enhances the memory for a ‘weak’ (e.g. ‘boring’ or pre-familiarized) stimulus given before the strong stimulus.
LCholinergic  and noradrenergic  projections to MTL can also modulate LTP and long-term memory. Activation of the brainstem locus coeruleus, theorigin of noradrenergic projections to the CNS, can enhance protein synthesis in the dentate gyrus and hippocampal stimulation can lead to increased hippocampal release of noradrenaline . There is also crosstalk: the primary cholinergic drive of the hippocampus is strongly affected by dopamine , as is the locus coeruleus . It is possible that noradrenaline is more important for consolidation of long-term memory for arousing and negative emotional events  than dopamine. Finally, serotonin may be necessary to consolidate memories about those types of aversive episodes that were associated with action inhibition (see  for a discussion).
Whereas the Hebbian NMDAR-dependent processes in the slice occur only during the initial induction of LTP, events in vivo also involve NMDAR-dependent processes triggered by hippocampal replay processes . In such replay, cells that were co-active during experience are reactivated during quiet wakefulness or sleep (reviewed in ). Such processes underlie consolidation events that both fix information at synapses and allow transfer of information from one region to another. Dopamine antagonists given only during such consolidation disrupt memory measured many hours later [144, 145].
It will be important to learn more about the time course of dopamine release, the duration of synaptic tags and the lifetime of plasticity related proteins, all of which will determine the properties of the penumbra. This may help to resolve the following puzzle. Reward-related SN/VTA activation improves memory for the rewarded stimulus, but not for the non-rewarded stimuli given in close temporal proximity , implying either a very short or very stimulus-specific penumbra. This is at odds with the observation that novelty-related activation of the SN/VTA has a long (ca. 30 minutes) penumbra that affects memory for unrelated information (e.g. exposure to novel scenes can affect memory for words) . One possible resolution is that the duration or stimulus-specificity of the penumbra depends on the type of motivational event that triggers dopamine release.
Older methods for labeling dopamine axons can give ambiguous results. Modern genetically-based labeling methods are needed to determine the location and density of these axons.
Two types of experiments demonstrate that dopamine can strengthen the synaptic potentiation produced by learning itself. In vivo recordings show that CA3/CA1 synapses are strengthened over time during trace eyeblink conditioning but that this strengthening is greatly reduced either in D1 receptor knockout mice or after reducing D1 receptors by virally delivered siRNA . Another line of experiments examined the learning that stabilizes place fields in CA1 after the animal enters a new environment. It was found that D1 antagonists prevent this stabilization .
Late-phase LTP can be induced in the slice preparation without dopamine application; a result that seemed to indicate that dopamine was not required. However, the block of LTP by dopamine antagonists suggests that dopamine is indeed required. This apparent contradiction was resolved by the demonstration that the extracellular stimulation procedure used to induce LTP releases endogenous dopamine from axon terminals in the hippocampus . Another quandary was posed by early work showing that dopamine agonists alone could produce potentiation . However, it was subsequently shown that this potentiation depended on the action of test pulses given during agonist application and that activation of NMDARs by the test pulses was required . Thus, it is now clear that dopamine by itself does not produce potentiation; rather, it acts as a permissive signal for the late-phase LTP induced by a NMDAR-dependent process.
Results such as those shown in Fig. 1 (top traces) suggest that early LTP is completely unaffected by dopamine antagonist. Subsequent work, however, showed that, with less strong induction protocols, there is a partial inhibition of early LTP . Thus, both early and late LTP are dopamine-dependent, but late LTP has a stronger dependence. Based on these results, one would predict that dopamine antagonists would produce a weak reduction of memory at short times after learning but would produce a strong reduction at later times. In the next section we discuss experiments that tested this prediction.
Plasticity in the hippocampus subserves the particular form of memory called episodic memory . This form is rapidly acquired and is referenced to a specific time and place. The investigation of the role of dopamine in episodic memory has been facilitated by the development of a rodent model of episodic-like memory: after learning several reward-location associations in an environment, rats rapidly (i.e., in only one trial) update their memory if one of the reward-location associations is changed . This one-trial memory for novel associations is critically dependent on the hippocampus and on glutamatergic neurotransmission during encoding [25, 26]. Intrahippocampal infusion of the D1/D5 dopaminergic antagonist SCH23390 just before encoding causes impaired memory for the new reward-location association measured 24 hours later, but has no effect on early memory measured at 30 minutes  (Fig. 2). A similar observation was made for one-trial encoding of the location of the escape platform in the Morris water maze . These data demonstrate that D1/D5 receptor-mediated effects do not strongly affect encoding or early memory, but do affect memory at later times. Dopamine has no effect on the previously formed memories and is not required at the time of retrieval . Further insight into the role of dopamine receptors in memory will come from the use of hippocampal-specific knockout of dopamine receptors (e.g. Sarinana and Tonegawa, Soc for Neurosci abstract 608.11, 2010).
Given the importance of dopamine release for LTP and memory, it becomes crucial to understand the properties of dopaminergic neurons and the kind of stimuli that lead to dopamine release. Midbrain dopamine neurons consist of three largely contiguous cell groups: the retrorubral field (cell group A8 in the rat nomenclature), the substantia nigra pars compacta (SNc, A9), and the ventral tegmental area (VTA, A10) [28, 29]. In primates, the SNc is further divided into a dorsal and ventral tier . Although the SN and VTA are known as dopaminergic regions, some cells in these regions are GABAergic or glutamatergic, and others release both glutamate and dopamine [31, 32]. Dopamine cells can be identified by their electrophysiological waveform signature and firing characteristics [33–35].
Dopamine cells can be silent, fire single spikes somewhat irregularly (tonic mode) or fire brief bursts of spikes. Studies performed both in vivo and in vitro show that the tonic mode is driven by intrinsic pacemaker conductances [34, 36]. This firing is modulated by GABAergic inputs from the ventral pallidum; a structure in which cells have high spontaneous activity . As a result of this inhibition, up to 50% of dopamine neurons are in the silent mode at baseline. Activation of the ventral subiculum (an output structure of the hippocampus) excites the nucleus accumbens (Nac), which, in turn, inhibits the ventral pallidum. This reduces the inhibition of dopamine cells and thereby increases the number of tonically firing cells [38, 39]. Tonically firing neurons (but not silent neurons) can be driven into a bursting mode by stimuli such as those associated with reward (see below). These bursts last about 200–500 msec, have interspike intervals <100 msec, and generally contain about 3–5 spikes. Burst firing in dopaminergic neurons is potently driven by the brainstem pedunculopontine tegmentum (PPTg), which provides a glutamatergic input that acts via NMDARs ([39, 40] see Box 3). Recent work demonstrated the functional importance of these bursts: selective genetic inactivation of NMDARs in dopaminergic cells blocked burst production and prevented conditioned behavioral responses that were dependent on dopamine ; conversely, simulating burst firing using channelrhodopsin is sufficient to induce behavioral conditioning . These results indicate that burst firing controls some downstream functions, but others may depend on how many cells are in the tonic mode (see Boxes 3 and 4).
The novelty processing hierarchy of the MTL (Figure I) is part of the circuitry that can regulate dopamine responses to novelty. The perirhinal cortex , for instance, shows much stronger responses to novel items than familiar items and may form connects to the PPTg (Figure I), a structure that regulates the SN/VTA. Strong evidence for hippocampal involvement in novelty-dependent dopamine release comes from experiments showing that this release is blocked by inactivating the subiculum, an output structure of the hippocampus that projects, by a polysynaptic pathway, to the SN/VTA (Figure I). In humans, hippocampal event-related potentials signaling unexpected events precede those in the Nac , which is compatible with the possibility that information flows from the hippocampus to the Nac and from there to the SN/VTA (Figure I). Hippocampal novelty signals incorporate context information by signaling mismatches for learned objects-spatial location associations , temporal sequences  and during integrative encoding .
The PPTg regulates burst firing of dopamine cells rather than their tonic resting activity []. It is known to be driven by prefrontal afferents, and responds to single sensory events from different modalities with earlier burst firing than dopamine neurons []. It is unclear to what extent PPTg responses are modified by contextual factors and/or conditioning [] or simply relay ‘accurately timed and attended sensory information’ []. The PPTg drive  depends on a permissive ‘gating’ input from the laterodorsal tegmentum (LDT) [[107, 116, 117]]. The LDT, in turn, receives substantial input from the medial prefrontal cortex (PFC) [], allowing the PFC to indirectly affect mesolimbic dopaminergic neuron activity[].
Another brain region which is known to participate in the modulation of dopamine neurons is the lateral habenula. The lateral habenula provides SN/VTA dopamine neurons with a negative reward-prediction signal through the fasciculus retroflexus fiber bundles []. The rostral medial tegmental nucleus (RMTg), a GABAergic afferent to midbrain dopaminergic neurons , may be the interface between the lateral habenula and the SN/VTA.
The tonic and phasic dopamine signals have consequences for both memory and behavior. In a predictable, familiar (and safe) environment, the ventral subiculum would be in a low-activity state, causing the number of tonically firing dopaminergic neurons firing to be minimal. A phasic stimulus would produce burst firing only in the tonically firing dopamine neurons, leading to a moderate dopamine output. As a result, the attentional state of the animal would be little affected. However, if the animal is in an unpredictable, novel (and presumably less safe) context (e.g. hunting) a novelty signal from ventral subiculum would raise the fraction of dopamine cells that are tonically firing. Under these conditions, a behaviorally relevant stimulus would cause a stronger phasic dopamine signal  (referred to as ‘saliency capture’ in [54, 61]).
A crucial control point in this process is the ventral striatum, which receives input from both the subiculum and PFC. Electrophysiological and behavioral studies suggest that these two inputs compete for regulation of the ventral striatum and are regulated in an opposite manner by the dopamine system. Tonic dopamine, which works through the higher-affinity D2 receptors, attenuates medial PFC (mPFC) inputs. In contrast, higher-concentration phasic dopamine signals activate D1 receptors and thereby enhance subicular input . In a condition in which the animal’s behavior leads to a better than expected outcome and phasic release (i.e. positive prediction error), the subicular input would be potentiated, causing the animal to maintain focus on the current task, thus allowing rewarded behaviors to be maintained. However, in the absence of an expected reward, or if a worse than expected outcome occurs (i.e., a negative prediction error), the consequent depression of dopaminergic neuron activity would attenuate dopamine-mediated potentiation of the ventral subiculum while at the same time removing inhibition of the mPFC input. Consequently the mPFC would drive the animal to switch response strategies .
The types of stimuli that increase burst firing of dopaminergic neurons have been extensively studied in monkeys. Classical work showed the importance of reward as a stimulus for the bursting of these cells , while subsequent studies have demonstrated that a novel stimulus (in the absence of reward) can also produce burst firing . More recent work has demonstrated conditions under which dopamine cells can be driven by aversive stimuli (reviewed in ; see also Box 2).
An important recent advance has been the ability to use functional magnetic resonance imaging (fMRI) to determine whether similar classes of stimuli activate the dopamine system in humans [46–49] (reviewed in ). Currently, most fMRI studies do not have sufficient resolution to distinguish subregions of the SN/VTA (this will be achievable with ultrahighfield fMRI, which has greater than 1mm resolution). A further difficulty is that fMRI cannot distinguish tonic or bursting modes. Despite these difficulties, there is converging evidence for a close relationship between the fMRI response in the SN/VTA and dopamine release (reviewed in ). Such studies (e.g. [48, 51]) have demonstrated that the human SN/VTA is activated by rewards, a finding confirmed by a recent study using electrophysiological recordings . Other work using fMRI has demonstrated that activation can also occur due to aversive events  as well as novel stimuli . Thus, novelty, reward stimuli and aversive stimuli are all able to activate the dopamine system in both monkeys and humans.
It is to an organism’s advantage to recollect circumstances which led to rewards. Indeed, according to the neoHebbian framework, activating the SN/VTA by a reward should enhance hippocampal-dependent episodic memory of information present at the time of reward. Furthermore, this improvement should be small with short delays when early LTP is present, but large at much longer delays when late LTP is present. A cognitive paradigm used to test these predictions in humans involved activating the SN/VTA at the time of encoding by giving reward. It was found that the prospect of receiving reward (e.g., monetary gain) improved long-term episodic memory for novel stimuli in incidental [47, 55–57] and intentional  encoding paradigms. This reward-related memory enhancement was associated with a co-activation of SN/VTA, striatum, and hippocampus, as detected by fMRI [46, 47]. Memory enhancement after long retention intervals (e.g. 24 hours) has been consistently found [56, 57]. Moreover, the enhancement was greater at late timepoints than at early intervals (ie. 3 weeks vs. 20 minutes) .
It is noteworthy that earlier cognitive psychology experiments on the effects of reward on memory used shorter retention intervals than the aforementioned studies (e.g. [58, 59]). Such studies did not show consistent memory enhancement and led to the idea that motivation was unimportant for learning. From the perspective of the neoHebbian framework, this negative result is understandable; i.e., the SN/VTA activity-related enhancement of memory would be more apparent if testing is conducted after long retention intervals. However, more studies are needed to clarify whether reward-related activation of the SN/VTA is associated with memory enhancement only after long retention intervals, compatible with the effects of intra-hippocampal injections of specific dopamine antagonists (e.g., ) (see Boxes 4 and 5).
In changing environments, it is vital to form long-lasting memories for novel events and, by definition, this requires encoding and consolidation after a single exposure. The role of hippocampal dopamine for consolidating novel associations after only one trial in rats  raises the question as to whether the human dopaminergic midbrain is activated by trial-unique novelty. Indeed, fMRI studies (for examples see Box 2 Figure 1) confirm SN/VTA activation by associative novelty  and stimulus novelty  but less by other salient events such as rareness or negative emotional content . Interestingly, the personality trait of novelty seeking is correlated with the magnitude of the SN/VTA novelty fMRI response even when novel stimuli predict the absence of reward , suggesting that this effect of novelty is hard to change by learning. In further support of such hard-wiring, positron emission tomography (PET) studies in humans show that novelty-seeking traits are inversely associated with D2-like receptor availability in the SN/VTA region  and are positively correlated with white matter connectivity between the hippocampus and the ventral striatum . The human SN/VTA is also activated by cues that predict novel images, or by unexpected novel images that followed familiarity-predictive cues (ie. an ‘unexpected novelty’ response) .
Some progress has been made in understanding the circuitry involved in novelty-dependent dopamine release (see Box 3). There is now converging evidence that different forms of novelty signals are computed along the medial temporal lobe (MTL) processing hierarchy from the parahippocampal and perirhinal cortices to the entorhinal cortex and from there to the hippocampus (see Box 3). In the perirhinal cortex, for instance, stimulus-related incoming information can be compared to stored information and a stimulus-related novelty signal is computed . The hippocampus, on the other hand, integrates information about stimuli and their spatial location into a contextual representation and can compute various types of novel changes in these relationships.
It may seem evident that a general purpose system like episodic memory should also enable long -term memory consolidation for events associated with aversive outcomes. Surprisingly, there is relatively little work comparing the consolidation of rewarding and aversive events (e.g. losses) in humans (although there is extensive work on the consolidation of emotionally negative stimuli). However, the observation that there are dopaminergic responses to aversive events (see Box 2) suggests that dopaminergic consolidation mechanisms similar to those seen with novelty and reward may apply. Consistent with this, learning of responses to aversive stimuli in rodents (ie. during fear conditioning) is inhibited by dopaminergic antagonists [66–68]. Moreover, dopamine is required for association between stimuli and aversive events in the amygdala .
As noted before, the finding that late LTP is more strongly dependent on dopamine than early LTP has been influential because it inspired investigation into the effect of dopamine on early vs. late learning. This suggests that other physiological findings regarding the role of dopamine in LTP might also be relevant to behavior. There are indeed several additional results that fall into this category and these have been incorporated into the Synaptic Tagging and Capture model developed by Frey and Morris .
According to this model, weak stimulation induces only early LTP. In contrast, stronger stimulation produces the dopamine-dependent protein synthesis that allows late LTP. The proteins required for late LTP are termed plasticity-related proteins (PRPs). Given the synapse specificity of LTP, these proteins are presumed to interact only with synapses that have undergone Hebbian LTP. Frey and Morris propose that this specificity occurs because LTP immediately produces a synapse-specific “tag” and that PRPs can only produce late LTP at tagged synapses. However, the synapses that produce the PRPs can be different from those at which the tag is set. It appears that both the tag and the PRPs have considerable lifetimes. This follows from the finding that stimulation of protein synthesis and the induction of Hebbian plasticity do not need to be coincident; indeed, the trigger for protein synthesis can occur either before or after Hebbian plasticity (with intervening periods of many minutes). Taken together, these findings suggest that there is a “penumbra” surrounding events that cause dopamine release; the memory for events that occur before or after the dopamine release would depend not only on their own properties, but also on whether they fell within the penumbra of a dopamine-releasing stimulus.
A number of animal behavioral studies have used novelty exploration as a ‘strong’ event that is likely to induce dopamine release [70–72]. In agreement with the synaptic tagging and capture model, exploration of a novel environment can make a memory for a ‘weak’ event more persistent [70–72]. For instance, a single exposure to the location of a small reward is rapidly forgotten in rodents within a few hours, but persists for at least 24 hours if it is preceded or followed (within 30 minutes) by novelty exploration . Also consistent with the synaptic tagging and capture hypothesis, intrahippocampal application of the D1/D5 receptor antagonist SCH23390 during novelty exploration diminishes memory retention .
Similar to the enhancement of long-term memory by reward in humans [47, 56, 57, 73], 24hr memory is enhanced in rats for rewarded locations (even after only a single exposure); moreover, this long-term memory can be blocked by a D1/D5 receptor antagonist at the time of encoding . That this drug-induced memory impairment is not due to an effect of dopamine on encoding is nicely demonstrated by the rescue of long-term memory if there is a preceding exposure to a novel environment .
Studies of LTP have shown that strong stimuli can enhance the encoding of weak stimuli even if the strong stimulus is given many minutes before or after the weak one. We term this temporal smearing the “penumbra” of the strong stimulus. Experiments have been conducted in humans that specifically looked for this penumbra. Individuals were exposed to novel photographs of natural scenes (‘strong events’, like those one would expect to see in the magazine National Geographic), for 5 minutes before being presented with the ‘weak’ encoding events, single words that were pre-familiarized on the day before and hence were not by themselves novel. The result was a higher proportion of recollected (as opposed to those recognized on the basis of familiarity) words when compared to individuals who were exposed to familiar pictures prior to encoding. This effect was stronger after 24 hours as compared to 30 minutes after encoding . Of note, pre-exposure to familiar, but emotionally very negative, photographs did not cause a comparable enhancement. It is therefore unlikely that affective arousal can account for the memory enhancement after novelty exposure. Hence, the behavioral penumbra seen after exposure to novelty in humans is compatible with the synaptic tag and capture model.
Although the concept of a penumbra is promising, further work to investigate its experimental basis is required. To understand factors that determine the duration of the penumbra, measurements need to be made on the duration of the dopamine signal in the hippocampus. The dopamine transporter (DAT), which rapidly clears dopamine from the synaptic cleft, is much less abundant in the hippocampus than in the striatum , so the duration of dopamine elevations may be relatively long. Recent elegant work is providing information about the spatial and temporal properties of protein-synthesis processes during LTP , which may also influence the time course of the penumbra. Behavioral experiments suggest that there are differences in the penumbra duration for novel as compared to rewarding events  (see Box 5). Perhaps a mechanistic understanding of the underlying events will be able to explain this difference. Additional experiments at the behavioral level are needed to examine the role of attention or arousal in the penumbra effect: if strong stimulation enhances the memory of a subsequent weak stimulus, perhaps this is due to stimulation of attention or arousal by the strong stimulus or to effects at encoding (see Box 5).
The results reviewed here indicate that the neoHebbian framework has explanatory power in the understanding of memory. A key insight to emerge from physiological experiments is that LTP at short times is relatively insensitive to dopamine, whereas LTP with longer delays is sensitive. Moreover, other properties of LTP (weak/strong; penumbra) appear also to have predictive power in explaining properties of memory.
The idea that long-term memory depends on more than the Hebbian condition makes sense, given the likelihood that too much information written into memory might lead to overwriting (and thus degradation) of pre-existing memories . According to the neoHebbian model, the default condition is that information is stored automatically in the hippocampus by early LTP based on a local Hebbian process. Generally, this newly stored information will fade away as early LTP declines. Only if a systems-wide computation determines that there is a high level of novelty or motivational salience will dopamine be released, allowing the biochemical processes of late LTP to incorporate the information into long-term memory.
The conditions that lead to activation of the human dopamine system, as judged by fMRI signals, are similar to the conditions leading to activation in non-primate mammals. It therefore seems likely that these principles may apply across all mammals. Nevertheless, experiments that further test the neoHebbian ideas in humans are warranted. An important prediction is that long-term memory would be affected positively (but possibly with an inverted U-shape pattern) by drugs that increase dopamine availability, such as levodopa (L-DOPA), which might enhance the burst-evoked release of dopamine. There has been remarkably little work on the effect of dopaminergic manipulations on human long-term memory. The long-term safety of available dopaminergic drugs is an important problem that needs to be overcome. Alternatively, progress may be made by taking advantage of genetic diversity in the human population with respect to genes that strongly affect dopaminergic function, which has already proven valuable for studying the human reward system (eg. ).
A better understanding of the human dopaminergic system is relevant to several human neurological or psychiatric disorders and to aging. Parkinson’s disease (PD) usually begins with loss of nigrostriatal dopamine cells in the ventral tier SNc, leaving the dorsal tier projection neurons to the hippocampus, ventral striatum, and cortex intact . It is thus understandable that severe episodic memory problems are not a symptom of early PD . However, at late stages, when all types of dopaminergic projection neurons are affected, memory problems do become severe. In schizophrenia, the enhanced activation of the hippocampal system may increase dopamine release and skew behavioral choice towards repetitive behavior (see Box 3). In aging, there is now converging evidence from human imaging studies for a structural degeneration of the VTA/SN (for reviews, see [54, 81]). Consistent findings have also been recently reported in aged rats . The possible negative effect of such a decline on the ability of information to enter into long-term memory has not yet been adequately considered.
Understanding the role of dopamine in memory may lead to methods that can be used to improve learning and teaching methods. First, based on the penumbra hypothesis, exposure to novel or salient information could be used to improve the long-term persistence of other information given in temporal proximity. Second, strategies based on the ability of external reward or reward anticipation to release dopamine might prove to be useful in increasing the retention of learned material. Finally, just as with external performance positive feedback, memory persistence might benefit from internally generated reward signals. For instance, the ability to recollect newly acquired information may be intrinsically rewarding. In fact, the study of human learning has revealed an interesting puzzle; long-term retention is not helped by simple re-exposure to recently learned material but is greatly helped by retesting even when subjects already know the answer . One interesting possibility is that retesting provides an opportunity to generate intrinsic reward signals, thereby enhancing long-term persistence of newly learned material.
This research was supported by National Institutes of Health grants (R01 NS027337 and R01 DA027807 to JL, MH57440 and NS15408 to A.A.G), the Deutsche Forschungsgemeinschaft (SFB 776, TP A7 and KFO 163 to E.D.), the German Centre for Neurodegenerative Disorders (DZNE), and a National Science Foundation (NSF) grant (SBE-0935288) to J.L.
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.