|Home | About | Journals | Submit | Contact Us | Français|
Since the discovery of place cells – single pyramidal neurons that encode spatial location – it has been hypothesized that the hippocampus may act as a cognitive map of known environments. This putative function has been extensively modeled using auto-associative networks, which utilize rate-coded synaptic plasticity rules in order to generate strong bi-directional connections between concurrently active place cells that encode for neighboring place fields. However, empirical studies using hippocampal cultures have demonstrated that the magnitude and direction of changes in synaptic strength can also be dictated by the relative timing of pre- and post-synaptic firing according to a spike-timing dependent plasticity (STDP) rule. Furthermore, electrophysiology studies have identified persistent “theta-coded” temporal correlations in place cell activity in vivo, characterized by phase precession of firing as the corresponding place field is traversed. It is not yet clear if STDP and theta-coded neural dynamics are compatible with cognitive map theory and previous rate-coded models of spatial learning in the hippocampus. Here, we demonstrate that an STDP rule based on empirical data obtained from the hippocampus can mediate rate-coded Hebbian learning when pre- and post-synaptic activity is stochastic and has no persistent sequence bias. We subsequently demonstrate that a spiking recurrent neural network that utilizes this STDP rule, alongside theta-coded neural activity, allows the rapid development of a cognitive map during directed or random exploration of an environment of overlapping place fields. Hence, we establish that STDP and phase precession are compatible with rate-coded models of cognitive map development.
The hippocampus and surrounding medial temporal lobe are implicated in declarative memory function in humans and other mammals (Morris, 2007). It has been demonstrated that the firing rate of place cells within the hippocampus encodes for spatial location in the corresponding place field within an environment (O'Keefe and Dostrovsky, 1971; O'Keefe, 2007). It has therefore been suggested that the hippocampus may act as a cognitive map of known locations (O'Keefe and Nadel, 1978; Muller et al., 1991). Behavioral experiments supporting this hypothesis demonstrate that the hippocampus is required for efficient navigation and spatial learning in novel environments (Morris et al., 1982; McNaughton et al., 2006; Morris, 2007).
At the cellular level, processes of learning and memory are believed to be mediated by synaptic plasticity (Hebb, 1949; Martin et al., 2000; Neves et al., 2008). Hippocampal mnemonic function has most frequently been modeled using auto-associative networks – recurrent neural architectures that implement a Hebbian learning rule (Marr, 1971; Hopfield, 1982; Burgess, 2007; Rolls, 2008). The biological correlate of these models is generally taken as the CA3 region of the hippocampus, which exhibits extensive recurrent connectivity and wherein synaptic plasticity can be easily and reliably induced (Rolls and Kesner, 2006; Rolls, 2007).
Initially, empirical studies of activity dependent changes in synaptic strength utilized tetanic stimulation protocols to induce long-term potentiation (LTP) with high frequency stimulation and long-term depression (LTD) with low frequency stimulation (Bliss and Lomo, 1973; Bienenstock et al., 1982; Dudek and Bear, 1992; Malenka and Bear, 2004). Consequently, in models of spatial learning, rate-coded synaptic plasticity rules have been implemented to produce connection weights between place cells that correlate with the relative distance between the corresponding place fields (Muller et al., 1991; Burgess et al., 1994; Burgess, 2007). This approach is supported by evidence for the potentiation of synaptic connections between place cells during exploration of a novel environment (Isaac et al., 2009).
Subsequent research using cultures of hippocampal neurons has demonstrated that bi-directional changes in synaptic strength can also be induced by temporal correlations in low frequency stimulation according to a spike-timing dependent plasticity (STDP) rule (Bi and Poo, 1998; Debanne et al., 1998; Wang et al., 2005). Concurrently, persistent temporal correlations in place cell activity, characterized by the phase precession phenomenon, have been observed during spatial exploration (O'Keefe and Recce, 1993; O'Keefe, 2007; Huxter et al., 2008). Furthermore, the selective elimination of causal temporal correlations in place cell activity prevents the potentiation of interconnecting synapses (Isaac et al., 2009). These findings suggest that rate-based models may not be adequate to replicate the dynamics of hippocampal spatial learning.
Although STDP, combined with the temporally coded patterns of neural activity observed in hippocampus, is ideally suited for route or sequence learning, it has been suggested that the inherent asymmetry of this plasticity rule may not allow the development of strong bi-directional connections between place cells that are required by cognitive map theory (Skaggs et al., 1996; Song and Abbott, 2001; Wagatsuma and Yamaguchi, 2007; but see Mongillo et al., 2005; Samura and Hattori, 2005). Computational modeling has also demonstrated that STDP can provide a putative homeostatic function by reducing synaptic currents in response to an increase in input firing rate, and thus provide global stability for local Hebbian learning rules (Song et al., 2000; Song and Abbott, 2001). However, these emergent dynamics directly contradict empirical observations and computational models of rate-coded synaptic plasticity (Bush et al., 2010a).
Here, we present a spiking recurrent neural network model with theta-coded neural dynamics and an STDP rule based on empirical data obtained from the hippocampus in vitro. We demonstrate that this form of STDP is compatible with rate-coded Hebbian learning, producing strong bi-directional connections between neurons that fire stochastically at an elevated rate with no persistent sequence bias. We subsequently demonstrate that this network model can produce a cognitive map of arbitrary one- and two-dimensional environments during shuttle runs and random exploration respectively, such that the magnitude of bi-directional connections between place cells correlates with the relative distance between corresponding place fields. The asymmetry of the plasticity rule dictates that the strength of synaptic connections between place cells are also biased by the frequency and direction in which the corresponding place fields are traversed during exploration. This indicates that STDP can mediate both rate- and temporally coded learning, corresponding to cognitive map formation during open field exploration and sequence learning during route navigation, respectively. However, our results also demonstrate that this form of STDP does not generate any inherent synaptic competition, and some additional mechanism is therefore required to guarantee the long-term stability of network operation.
The neural network consists of NP simulated place cells that are fully recurrently connected except for self-connections. Simulated place cells operate according to the Izhikevich (2004) spiking model, which dynamically calculates the membrane potential (v) and a membrane recovery variable (u) based on the values of four dimensionless constants (a–d) and a dimensionless current input (I) according to Eq. 1. The values used to replicate firing of a standard excitatory neuron are (a=0.02, b=0.2, c=−65, d=6). Under these conditions, simulated neurons fire single spikes at low levels of stimulation, but produce complex spike bursts that are representative of hippocampal pyramidal cells (i.e., several action potentials at a spontaneous rate of ~150Hz) at higher levels of stimulation (Ranck, 1973; Izhikevich, 2004; O'Keefe, 2007).
Each simulated neuron has a randomly chosen axonal delay in the range (1ms:5ms), this being realistic of the CA3 region (Miles, 1990). Previous research has indicated that the scale of axonal delays can have a significant effect on the quantitative, but not qualitative, nature of synaptic dynamics produced by STDP (Bush et al., 2010b). However, it is important to note that the delay between the firing of an action potential and its arrival at post-synaptic targets is unlikely to be uniform for a single neuron in vivo.
Mathematically, with s= tpost− tpre being the time difference between pre- and post-synaptic spiking, the change in the weight of a synapse (Δw) according to a standard implementation of additive STDP can be calculated using Eq. 2. The parameters A+ and A− correspond to the maximum possible change in the synaptic weight per spike pair, while τ+ and τ− denote the time constants of exponential decay for potentiation and depression increments respectively. In accordance with experimental observations, coincident pre- and post-synaptic firing elicits maximal depression (Debanne et al., 1998). A nearest-neighbor spike pairing scheme – which dictates that values of P±→ A± upon afferent or efferent firing – is employed. In all simulations, hard limits are placed on the achievable strength of synapses, such that synaptic weights are maintained continuously in the range (0:wmax).
Empirical data obtained from the hippocampus suggests that A+> A− and τ+< τ− (Bi and Poo, 1998; Debanne et al., 1998; Wang et al., 2005). Accordingly, we employ values of (A+=0.015; A−= −0.012; τ+=20ms; τ− =50ms). Experimental observations in cultures of hippocampal neurons also suggest a non-linear integration of potentiation and depression processes at short temporal offsets (Wang et al., 2005; Pfister and Gerstner, 2006). We therefore utilize an additional potentiation term to replicate this data. Whenever a synaptic connection is depressed, the absolute magnitude of weight decrease is recorded as a parameter P++ which subsequently decays with a time constant of τ++=20ms. The value of P++ is then used to supplement any subsequent potentiation of the same synaptic connection (after time s++ =tΔw+ −tΔw−). While this method differs from that employed in the computational model of Pfister and Gerstner (2006), it corresponds to the parameters and τx utilized in that study.
It is important to note that the STDP rule utilized here, like the majority of previous computational models, is based primarily on data obtained from cultures of hippocampal pyramidal neurons (Bi and Poo, 1998; Debanne et al., 1998; Wang et al., 2005). More recent empirical studies utilizing acute slices have delineated a more complex relationship between temporal correlations in neural activity and subsequent changes in synaptic strength: primarily, that multiple post-synaptic spikes are required to induce LTP using low frequency temporal correlations (Pike et al., 1999; Meredith et al., 2003; Wittenberg and Wang, 2006; Buchanan and Mellor, 2007; but see Kwag and Paulsen, 2009; Buchanan and Mellor, 2010). In order to retain computational efficiency, we do not aim to explicitly model these findings, but appraise the properties of our STDP rule in light of that data at all relevant junctures.
Equation 2: The spike-timing dependent plasticity rule.
Our initial aim is to characterize the emergent synaptic dynamics produced by the STDP rule described above when neural activity corresponds to a variety of standard stimulation protocols used experimentally to induce LTP and LTD (Bi and Poo, 1998; Wang et al., 2005; Wittenberg and Wang, 2006). Firstly, we examine changes in the strength of bi-directional connections between Np=2 simulated neurons when pairs and triplets of pre- and post-synaptic spikes are delivered at low frequency (1Hz) for a period of 60s. These include single pre- and post-synaptic spikes with a temporal offset of 10ms; a single action potential and a complex burst of two to three spikes with ~5ms inter-spike intervals (ISI) offset by 10ms; and a single action potential with two spikes of differing temporal correlations at various temporal offsets (Figure (Figure11).
Secondly, we examine emergent synaptic dynamics in a small, fully recurrently connected network (excluding self-connections) of Np=100 simulated neurons when neural activity corresponds to previous rate-coded auto-associative network models (Marr, 1971; Hopfield, 1982; Burgess, 2007; Rolls, 2008). A subset of simulated place cells (10%) fire stochastically at an elevated “foreground” rate (rfore≈ 20Hz) while the remainder fire stochastically at a low “background” rate (rback ≈0.1Hz, these values being realistic of CA3) (Frerking et al., 2005). This activity is generated using external excitatory input drawn from a random, uniform distribution in the range (0:Imax), with Imax=12 for foreground neurons and Imax=4.5 for background neurons. An examination of the resultant neural dynamics confirms that this form of input produces a Gaussian distribution of ISI (data not shown). In each of these simulations, all synaptic connections in the network are initialized with a weight of wij=0.3 unless specified otherwise.
In further simulations, theta-coded neural dynamics are dictated by a phenomenological phase precession model (O'Keefe and Recce, 1993; O'Keefe, 2007; Huxter et al., 2008). A variable θ, which oscillates sinsuoidally in the range (0:1) at a rate of 8Hz, is used to represent theoretical theta frequency oscillations in the local field potential (LFP) that dominate the hippocampal EEG during stereotyped learning behavior (Buzsaki, 2006; O'Keefe, 2007). Inhibitory input to each simulated neuron at each millisecond time step is drawn randomly from a Gaussian distribution with mean Iinh =−15(1 −θ) and standard deviation σinh =2. Neural noise at a rate of ~0.1Hz (this being realistic of CA3) is generated in the network by the constant application of excitatory current drawn from a uniform random distribution in the range (0:Inoise) where Inoise=0.8 in all simulations (Frerking et al., 2005). The interplay between afferent inhibitory and excitatory currents means that the majority of firing due to neural noise tends to occur around the peak of the LFP, as defined by the value of θ.
External stimulation of place cells is subsequently provided within different, discrete theta phase windows depending on hypothetical location within (or outside of) the corresponding place field (Figure (Figure2).2). Circular place fields of 80cm diameter are divided into eight equally sized sections of 10cm, and theta oscillations in the LFP (as defined by the value of θ) are similarly divided into “phase windows” of π/4 between π/8 and 15π/8. External input, randomly sampled from a normal distribution with mean Iext=0 and standard deviation σext=30, is subsequently applied to simulated place cells for the duration of a theta phase window if the hypothetical position lies within the corresponding place field section. This generates a mean in-field firing rate of ~15Hz, with active place cells tending to fire stochastic bursts at the peak of the LFP (as defined by the value of θ) and single spikes on the ascending and descending slope, in accordance with empirical data (O'Keefe and Recce, 1993; Huxter et al., 2008). It is important to note that the phenomenological phase precession mechanism utilized here, wherein the phase of place cell firing is dictated by the timing of external excitatory input, contrasts with previous computational models which suggest that phase precession may itself be the result of cell assemblies produced by STDP (Jensen and Lisman, 1996; de Almeida et al., 2007). However, there is considerable debate regarding the mechanisms by which theta-coded neural dynamics are produced in vivo, and modeling studies suggest that phase precession in CA3 may largely be inherited from upstream circuits (Baker and Olds, 2007; Maurer and McNaughton, 2007).
Using this phenomenological phase precession model, we first examine changes in the strength of bi-directional connections between Np=2 simulated neurons when neural activity corresponds to that observed in hippocampal place cells encoding for overlapping place fields. One simulated place cell consistently fires a complex spike burst (two to three action potentials with ~5ms ISI) at the peak of theta (as defined by the value of θ) while the second neuron fires stochastically within a different theta phase window for each set of 10s simulations performed.
Having established the synaptic dynamics produced by our STDP rule with various sample spike trains, we then examine a fully recurrently connected network of Np=490 simulated neurons with theta-coded neural dynamics that correspond to various spatial exploration strategies within a hypothetical square arena. The arena consists of a grid of 49 evenly distributed place fields, each encoded for by 10 place cells, that are offset by 10cm and traversed by a hypothetical agent at a constant speed of 10cm s−1. In the first set of “shuttle run” simulations, a randomly selected one-dimensional route of 10 place fields in length is traversed 10 times in alternating directions. Next, in “random exploration” simulations, initial position within the arena is randomly selected, the subsequent direction of movement is chosen randomly at the beginning of each second of simulated time, and thus the agent traverses the arena in an undirected manner for a total period of 490s. In both cases, these parameter values dictate that there are generally seven different simultaneously active ensembles of 10 place cells, each encoding for a different place field and therefore firing within a different theta phase window. After each 1s of simulated time, the mean phase of firing in each place cell ensemble recedes to an earlier theta phase window as the hypothetical agent advances to the next place field segment. In each of these simulations, all synaptic connections in the network are initialized with a weight of wij=0.01 unless specified otherwise.
Experimental data suggests that identical stimulation protocols can induce a differing degree and magnitude of synaptic plasticity within the hippocampus depending on their timing relative to ongoing theta oscillations – potentiation generally being incurred at the peak of the LFP, while depression (or depotentiation) is incurred at the trough (Hyman et al., 2003). This is most likely a result of the transient changes in membrane depolarization that proceed over the course of a theta cycle. Here, we perform simulations both with and without plasticity modulation. When dynamic plasticity modulation is implemented, all synaptic weight changes dictated by the STDP rule are scaled by the instantaneous value of θ according to Eq. 3.
Fifty independent simulations are performed for each set of experiments described above, and the Mann–Whitney U-test is used to assess the significance of differences in resultant synaptic weight distributions.
Equation 3: Details of the dynamic synaptic plasticity modulation mechanism.
In order to appraise the accuracy with which our phenomenological plasticity rule can replicate empirical data obtained from the hippocampus, we first examine the emergent synaptic dynamics generated when neural activity corresponds to several different stimulation protocols commonly used to induce LTP and LTD. Initially, we employ two different spike pairing protocols in which either single pre- and post-synaptic spikes or a single spike and complex burst are paired at short temporal offsets (Figures (Figures1A,B).1A,B). This generates asymmetric connections that correspond to the “causal” input correlation, with synaptic weights being driven toward either the upper or lower bounds (Figure (Figure3A).3A). These results concur with empirical data obtained using similar stimulation protocols in cultures of hippocampal pyramidal neurons (Bi and Poo, 1998; Debanne et al., 1998; Wang et al., 2005). However, studies using acute hippocampal slices suggest that multiple post-synaptic spikes are required to generate potentiation (Pike et al., 1999; Meredith et al., 2003; Wittenberg and Wang, 2006; Buchanan and Mellor, 2007). Our STDP rule does not replicate this data – although it is important to note that potentiation is more rapid, and thus a greater mean synaptic weight is achieved over the 60s stimulation period, when the post-synaptic neuron fires multiple spikes.
Secondly, we employ triplets of pre- and post-synaptic spikes at various temporal offsets (Figure (Figure1C).1C). Under these conditions, the non-linear integration of depression and potentiation processes dictated by the P++ term generate more complex emergent synaptic dynamics that depend explicitly on the order of spike pairings. When the temporal offset of each spike pairing is equal, then the synaptic connection that experiences a potentiating (“causal”) spike pairing after a depressing spike pairing is significantly potentiated (Figure (Figure3B).3B). Conversely, spike pairings in the opposite order (potentiation then depression) generate a much less significant increase in synaptic weight. In the majority of cases – particularly for post–pre–post spike triplets, the magnitude of synaptic changes produced in these simulations approximates that observed in hippocampal cultures when identical stimulation protocols are employed (Wang et al., 2005; Pfister and Gerstner, 2006).
However, asymmetry in the profile of the STDP window employed in this study (i.e., A+ >A− and τ+ <τ−) dictates that the potentiation and subsequent depression increments produced by pre–post–pre triplets are not equal and do not directly negate. Hence, mild LTP is generated by pre–post–pre triplets with equal temporal offsets (i.e., t1 =t2) and mild LTD is induced when a potentiating spike pairing with a longer temporal offset precedes a depressing increment with a shorter temporal offset (i.e., w21 when t1 >t2). This is in contrast to empirical data, where no significant weight change is observed in either case (Wang et al., 2005; Pfister and Gerstner, 2006). Furthermore, the magnitude of potentiation incurred when a depressing spike pairing with a shorter offset precedes a potentiating increment with a longer temporal offset is much less significant than that observed experimentally. This is surprising, considering the fact that the profile of the STDP window employed in this study is also based directly on empirical data from hippocampal cultures (Bi and Poo, 1998).
Thirdly, we demonstrate that this form of STDP can mediate rate-coded Hebbian learning, in accordance with empirical data regarding observations of LTP induced by tetanic stimulation protocols (Bliss and Lomo, 1973; Malenka and Bear, 2004). When stochastic neural activity at high (“foreground”) and low (“background”) firing rates is generated in distinct groups of neurons within a small, fully recurrently connected network, those synaptic connections with elevated pre- and post-synaptic firing rate are rapidly, selectively and significantly potentiated (Figures (Figures3C,D).3C,D). Synaptic connections with solely elevated post-synaptic firing rate undergo modest but continual potentiation, while those with solely elevated pre-synaptic firing rate are significantly depressed. This indicates that the STDP rule reduces the weight of connections that have a negative (i.e., “non-causal”) rate correlation. Finally, the strength of synapses with background pre- and post-synaptic firing rate does not change significantly. This is in contrast to the robust LTD usually observed following prolonged low frequency stimulation, and most likely due to the small number of temporally proximate spike pairings that occur at these connections (Dudek and Bear, 1992; Buchanan and Mellor, 2010).
However, over very long periods of simulated time (~20,000s or more) the mean weight of all pre-synaptic connections of neurons firing at an elevated rate saturate at a similarly high value (~0.8wmax, data not shown). This indicates that the STDP rule utilized here fails to generate competition between synaptic inputs to a single neuron. This absence of hetero-synaptic competition, and subsequent issues with global network stability, is a common feature of previous rate-coded Hebbian plasticity rules, and require the inclusion of some additional mechanism to ensure stable long-term operation (Desai, 2004).
Finally, we examine the synaptic dynamics generated when neural activity approximates the theta-coded temporal activity correlations observed in place cells encoding for overlapping place fields within the hippocampus (O'Keefe and Recce, 1993; O'Keefe, 2007; Huxter et al., 2008). We consider two bi-directionally connected place cells that fire stochastically at differing mean phase of a putative theta cycle (Figure (Figure4A).4A). Our results demonstrate that asymmetric connections of varying strength are generated when a persistent temporal correlation in pre- and post-synaptic activity exists, while strong bi-directional connections are generated when pre- and post-synaptic activity is stochastic but approximately synchronous with no persistent sequence bias (Figures (Figures4B,C).4B,C). The presence of strong bi-directional connections in these simulations is again indicative of a lack of synaptic competition generated by this form of STDP. When dynamic plasticity modulation is implemented, the mean weight of bi-directional connections between neurons firing synchronous, stochastic spike bursts at the peak of theta is significantly greater. Furthermore, the mean weight of asymmetric connections generated when the pre-synaptic neuron fires a complex burst and the post-synaptic neuron fires fewer spikes (i.e., when there is a negative rate correlation) are significantly lower. It is also important to note that previous research has delineated a significant correlation between the magnitude of bi-directional connections between simultaneously active simulated neurons and the scale of recurrent axonal delays between those neurons (Bush et al., 2010b).
In summary, these results demonstrate that the phenomenological plasticity rule utilized here can approximate the primary characteristics of synaptic plasticity data obtained in vitro using either rate or temporally coded stimulation protocols. The STDP model can generate both asymmetric and bi-directional connectivity patterns, according to long-term correlations in neural activity, and thus mediate both hetero-associative and auto-associative learning in recurrently connected networks (Bush et al., 2010b). However, these emergent properties come at the cost of the inherent synaptic competition observed in previous computational models of STDP (Song et al., 2000; Song and Abbott, 2001; Bush et al., 2010a). It is important to note that the qualitative nature of these findings do not depend explicitly upon the use of a triplet-based STDP rule (i.e., one which incorporate a P++ term), but can also be achieved by a pair-based STDP rule if certain constraints are placed on the profile of the STDP window and spike pair interactions, although the latter cannot replicate empirical data obtained with triplets of pre- and post-synaptic action potentials delivered at low frequencies (Bush et al., 2010a,b).
Having established that the emergent synaptic dynamics produced by this form of STDP concur broadly with empirical data, we next implement the plasticity rule within an abstract spiking recurrent neural network model of hippocampal spatial learning. In these simulations, neural dynamics correspond hypothetically to those observed in hippocampal pyramidal neurons during spatial exploration, with theta-coded activity patterns created in simulated place cells by the phenomenological phase precession model (O'Keefe and Recce, 1993; O'Keefe, 2007; Huxter et al., 2008). We consider different navigation strategies in a square arena of equidistant place fields, each of which is encoded for by an ensemble of place cells.
Initially, neural activity corresponds hypothetically to repeated shuttle runs along a one-dimensional route of overlapping place fields. Subsequently, interconnected place cells encoding for neighboring place fields exhibit temporal correlations in firing activity with repeatedly alternating direction but approximately equal ISIs. It has been suggested that the asymmetric nature of the STDP learning window dictates that purely asymmetric connections will develop under these conditions (Skaggs et al., 1996; Wagatsuma and Yamaguchi, 2007; Song and Abbott, 2001; but see Mongillo et al., 2005; Samura and Hattori, 2005). For the purposes of cognitive map theory, however, it is essential that strong bi-directional connections which reflect the relative location of corresponding place fields in the environment and the long-term history of navigation through this one-dimensional environment are generated (Muller et al., 1991; Burgess et al., 1994; Burgess, 2007).
Our results demonstrate that, following 10 shuttle runs along the route, the mean weight of hetero-associative connections between place cells encoding for each place field and those up to three steps in either direction on the learned route are selectively and significantly potentiated (Figures (Figures5A,B).5A,B). Strong bi-directional connections also develop between place cells that encode for the same location. When no dynamic plasticity modulation is implemented, these auto-associative connections are weaker than hetero-associative connections between place cells encoding for adjacent place fields. These results correspond closely with the synaptic dynamics observed at connections between hippocampal neurons with overlapping place fields in vivo (Figure 5G in Isaac et al., 2009).
An examination of the synaptic dynamics generated in these simulations demonstrates that, although the mean weight of hetero-associative connections remains significantly higher than that of other synaptic weights in the network, those against the direction of motion are significantly depressed during a single traversal of the route (Figures (Figures5C,D).5C,D). The relative magnitude of hetero-associative connections corresponding to opposing directions of traversal therefore fluctuate continually during these shuttle runs. Hence, the resultant “cognitive map” of the learned route is biased in the direction that the environment was most recently explored. This implies that, if a sequence of place fields is repeatedly traversed in a single direction, then hetero-associative connections between the corresponding place cells will become purely asymmetric, as synapses against the direction of motion undergo sustained depression (Bush et al., 2010b).
Next, we examine the emergent synaptic dynamics produced during random exploration of a two-dimensional arena of place fields, each of which is encoded by an ensemble of place cells that exhibit theta-coded neural dynamics. Our results demonstrate that, following approximately 8min of random exploration, a putative cognitive map of the environment is encoded in recurrent connections such that the relative distance between place fields for which pre- and post-synaptic place cells encode correlates with the magnitude of interconnecting synaptic weights (Figures (Figures6A,B).6A,B). Strong, bi-directional (auto-associative) connections again develop between place cells that encode for the same location, although these are weaker than hetero-associative connections between place cells that encode for neighboring locations in the environment when no plasticity modulation is implemented. Hetero-associative connections are also more rapidly potentiated (Figure (Figure6C),6C), although this depends on the corresponding place fields being traversed in the necessary order, while auto-associative connections are strengthened whenever a place field is entered. An examination of sample “cognitive maps” for single place fields illustrates that uniform, omnidirectional connections between place cells are not created in all locations (Figure (Figure6D).6D). Hence, while the overall/mean connectivity pattern of the network is characterized by strong bi-directional connections between place cells encoding for proximate place fields in the environment, local connectivity patterns tend to reflect the specific trajectories taken through that place field during random exploration. It seems likely that this effect would disappear over longer periods, as all possible trajectories through each place field are explored.
It is important to note that “background” connections in the network – i.e., synapses between place cells encoding for place fields that do not overlap – undergo modest but sustained potentiation throughout these simulations (Figure (Figure6C).6C). This is again indicative of the lack of synaptic competition generated by this form of STDP. Over prolonged periods of operation, the increase in background synaptic weights eventually destabilizes network operation by generating runaway recurrent excitation. The length of time for which the network maintains stable operation is inversely correlated with the background firing rate, as higher background firing rates generate a more rapid potentiation of background connections due to the increased frequency of spike pairings (data not shown). However, the inclusion of any additional mechanism that generates hetero-synaptic competition between all synaptic inputs of a single neuron can ameliorate this issue (Desai, 2004). For example, a small, global, uniform weight decay term eliminates the potentiation of background connections without any significant effect on the generation of auto- and hetero-associative connections between place cells encoding for overlapping place fields, and thereby stabilizes long-term network operation (data not shown).
Finally, we examine the emergent synaptic dynamics generated when neural activity corresponds to the repeated traversal of a one-dimensional route through the environment of place fields in a single direction, following the random exploration simulations described above. While the overall shape of the synaptic weight matrix changes very little (Figures (Figures6A6A and and7A),7A), the strength of connections between place cells encoding for those place fields that lie on the learned route are significantly altered. For example, Figure Figure7B7B illustrates how the cognitive map formed for a single place field (that illustrated in Figure Figure6D)6D) changes when a route of seven place fields that horizontally transects that place field is subsequently traversed 10 times. Strong asymmetric connections are formed between place cells encoding for place fields that are successively activated, while those connections against the direction of motion are fully depressed.
These novel hetero-associative connections are embedded within the previously learned map, which reflects routes taken through that place field during the initial random exploration, without any significant effect on previously learned associations that do not form part of the newly learned route. Hence, the hetero-associative connections formed by place cells encoding for each place field represent the relative distance between place fields and the relative frequency with which those place fields have been sequentially traversed. These results also indicate that STDP and theta-coded neural dynamics allow rapid sequence learning, via the formation of asymmetrically connected place cell assemblies, in accordance with previous models of route learning in recurrent networks (Skaggs et al., 1996; Wagatsuma and Yamaguchi, 2007; Bush et al., 2010b).
We have presented a phenomenological STDP rule that approximates empirical synaptic plasticity data obtained using both rate and temporal stimulation protocols, and can thus generate both bi-directional and asymmetric connections in response to persistently synchronous or sequential correlations in pre- and post-synaptic activity respectively. We have subsequently demonstrated that a spiking recurrent neural network model which utilizes this STDP rule alongside theta-coded neural dynamics can mediate the development of a cognitive map during shuttle runs along a one-dimensional track or open field exploration within a two-dimensional environment, such that the geometric distance between place fields is encoded by the magnitude of bi-directional synaptic connections between place cells. This demonstrates that temporally asymmetric synaptic plasticity and phase precession are compatible with previous computational models of rate-coded cognitive map formation. Furthermore, it allows the integration of those rate-coded auto-associative network models with temporally coded, hetero-associative, spiking neural network models of route and sequence learning, and thus provides them with a firmer basis in modern neurobiology.
However, the STDP rule described here also exhibits several theoretical and functional weaknesses. Firstly, it fails to replicate recent experimental data obtained from acute hippocampal slices that suggest a requirement for multiple post-synaptic spikes to induce LTP with low frequency temporally correlated stimulation (Pike et al., 1999; Meredith et al., 2003; Wittenberg and Wang, 2006; Buchanan and Mellor, 2007, 2010), although it is important to note that empirical findings often disagree and other studies have demonstrated robust potentiation using single spike pairs in similar preparations (Kwag and Paulsen, 2009). These differences can be attributed to the multiple factors (such as membrane depolarization, the state of network inhibition, developmental stage and neuromodulatory tone) that influence post-synaptic calcium influx, which is generally recognized as the critical trigger for synaptic plasticity (Buchanan and Mellor, 2010). Furthermore, the vast majority of empirical plasticity data is obtained at the CA3-CA1 synapse, while significant differences may exist at recurrent CA3 connections. Place cells are known to receive rhythmic inhibitory input during spatial exploration, and be significantly depolarized when the corresponding place field is traversed (which concurs with the experimental procedure employed in Kwag and Paulsen, 2009), while cholinergic tone is vastly increased during stereotype learning behavior (Harvey et al., 2009; Isaac et al., 2009). Each of these factors could contribute to the induction of LTP with less intensive stimulation protocols. A wider range of experimental data might be approximated by incorporating a more explicit consideration of intra-cellular calcium dynamics, NMDA receptor kinetics and/or the backpropagation of action potentials in the dendritic tree.
Secondly, it has been empirically demonstrated that reversing the temporal order of action potentials in pairs of place cells that exhibit asymmetric spike timing cross-correlations does not prevent the induction of LTP at the interconnecting synapse, casting doubt on the existence of STDP in vivo (Isaac et al., 2009). However, it is important to note that place cells in that study exhibited only weak sequence bias in pre- and post-synaptic spike timing, while each firing at an elevated rate – circumstances that would likely generate significant bi-directional potentiation using the STDP rule presented here (Figures (Figures3B,C).3B,C). Furthermore, in the same study, the selective elimination of all spike pairings with a positive temporal correlation of <100ms prevented the induction of LTP, which suggests some requirement for temporal asymmetry to generate potentiation.
Finally, our STDP rule fails to induce any competition between the inputs of a single neuron. Some additional mechanism – such as synaptic scaling, weight decay or metaplasticity – is therefore required to ensure the long-term stability of network operation, particularly at higher background firing rates (Desai, 2004). For example, the inclusion of a slow, global weight decay term ameliorates the potentiation of background connections and thus maintains stable long-term network operation without affecting the generation of auto- and hetero-associative connections between place cells encoding for overlapping place fields (data not shown). It is important to note that this propensity for potentiation is a feature of all STDP rules that can mediate rate-coded Hebbian learning (Bush et al., 2010a,b). This might be considered a significant functional weakness and failure to replicate bi-directional changes in synaptic strength. However, it is important to note that no significant depression of synaptic connections between hippocampal place cells is observed during spatial exploration (Isaac et al., 2009). In fact, there is a convincing similarity between the data presented here and that recorded at synaptic connections between place cells whose place field have varying degrees of overlap in vivo (i.e., Figure Figure5A5A in this manuscript and Figure 5G in Isaac et al., 2009). It has also been demonstrated that synaptic connections throughout the cortex undergo net potentiation during waking, and are subsequently depressed during sleep (Vyazovskiy et al., 2008). It may be that the rapid formation of novel associations in the hippocampus is achieved at the cost of a slower but continual positive drift in background synaptic weights.
Although the neural network presented here is not intended to address the function of CA3 directly, it can also be appraised in terms of more general auto- and hetero-associative models of spatial learning. One significant omission, which therefore represents a critical direction for future research, is an examination of recall dynamics mediated by recurrent excitation, whereby external stimulation following learning generates pattern completion in encoded static or sequential activity patterns. Although we have previously demonstrated this ability following the learning of single, isolated auto- and hetero-associative activity patterns (Bush et al., 2010b), the overlap of encoded associations required by cognitive mapping generates the undirected and unrestricted spread of recurrent excitation throughout the entire network (Molter et al., 2007). Conversely, to be of any functional use, recall activity within network models of spatial learning needs to be biased by goal-directed excitatory or inhibitory input (Burgess, 2007). It seems feasible that inhibitory plasticity during spatial learning could contribute to disambiguating overlapping encoded sequences during subsequent reactivation via putative winner-take-all dynamics. This highlights a need for the inclusion of feedback inhibition in future extensions of this model, which might also dynamically regulate the firing rate of background neurons, reducing the potentiation of redundant synaptic connections and thus contributing to the long-term stability of network operation.
Finally, it is unclear to what extent rate-coded models of cognitive map formation can explain the involvement of the hippocampus in spatial learning and navigation. There is no evidence for fixed point attractor dynamics in CA3, and it remains to be seen if models of cognitive mapping can be reconciled with the transient, temporally coded cell assembly dynamics that characterize hippocampal activity during exploration (Harris, 2005). Furthermore, it is generally accepted that the role of the hippocampus in declarative memory is time-limited, and recent experimental evidence demonstrates that the neocortex is involved in spatial learning and memory consolidation from a very early stage (Leon et al., 2010). In accordance with standard “two stage” models of declarative memory processing, it seems likely that a global cognitive map would ultimately be stored in the neocortex, while recent routes or environments – “episodes” encoded in a single trial – are stored for a limited period in the hippocampus for rapid integration with this “semantic” spatial knowledge (McClelland et al., 1995). The exact contribution of the hippocampus and neocortex to spatial memory and goal-directed navigation remains to be elucidated.
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
The authors would like to thank Jesper Sjostrom, Bart Baddeley, the anonymous referees and members of the CCNR for useful discussions during the preparation of this manuscript. This work was supported by Wellcome VIP funding and the EPSRC grant EP/H024638/1.