|Home | About | Journals | Submit | Contact Us | Français|
The definition of episodic memory includes the concept of mental time travel: the ability to re-experience a previously experienced trajectory through continuous dimensions of space and time, and to recall specific events or stimuli along this trajectory. Lesions of the hippocampus and entorhinal cortex impair human episodic memory function and impair rat performance in tasks that could be solved by retrieval of trajectories. Recent physiological data suggests a novel model for encoding and retrieval of trajectories, and for associating specific stimuli with specific positions along the trajectory. During encoding in the model, external input drives the activity of head direction cells. Entorhinal grid cells integrate the head direction input to update an internal representation of location, and drive hippocampal place cells. Trajectories are encoded by Hebbian modification of excitatory synaptic connections between hippocampal place cells and head direction cells driven by external action. Associations are also formed between hippocampal cells and sensory stimuli. During retrieval, a sensory input cue activates hippocampal cells that drive head direction activity via previously modified synapses. Persistent spiking of head direction cells maintains the direction and speed of the action, updating the activity of entorhinal grid cells that thereby further update place cell activity. Additional cells, termed arc length cells, provide coding of trajectory segments based on the one-dimensional arc length from the context of prior actions or states, overcoming ambiguity where the overlap of trajectory segments causes multiple head directions to be associated with one place. These mechanisms allow retrieval of complex, self-crossing trajectories as continuous curves through space and time.
Episodic memory includes the capacity to internally re-experience the sequence of events that occurred at particular places and times, in what has been termed “mental time travel” (Eichenbaum & Cohen, 2001; Tulving, 2001; 2002). Episodic memory includes the capacity to mentally retrace trajectories through previously visited locations, including re-experiencing specific stimuli encountered on this trajectory, and the relative timing of events. For example, you can probably remember the route you followed when you left your home this morning, with a memory of the locations you visited and the time you spent in individual locations. You can use this memory to remember where you parked the car, who you saw on your trip, or where you left your car keys. This aspect of episodic memory requires some means by which neurons can code continuous trajectories through space with time intervals representing the original episode. This also requires some means for encoding the location and time of specific events or stimuli encountered along this trajectory.
Physiological data shows that hippocampal activity during REM sleep can replay the relative time intervals of spiking activity evoked by different spatial locations during waking (Louie & Wilson, 2001), indicating the capacity to replay spatiotemporal trajectories with the same time scale as actual behavior. Other experiments also show that spiking activity in the hippocampal formation can maintain information about the relative timing of events (Hoehler & Thompson, 1980; Berger et al., 1983; Deadwyler & Hampson, 2006).
Lesion data suggests that encoding and retrieval of previously experienced episodic trajectories involves the entorhinal cortex and hippocampus. In humans, lesions of these structures cause profound impairments of episodic memory, tested both qualitatively and with quantitative measures in verbal memory tasks (Scoville & Milner, 1957; Corkin, 1984; Graf et al., 1984; Rempel-Clower et al., 1996; Eichenbaum & Cohen, 2003). Impairments in formation of object-location associations occur with right hippocampal or parahippocampal lesions (Milner et al., 1997; Bohbot et al., 1998; Bohbot et al., 2000; Stepankova et al., 2004). In rats, hippocampal manipulations impair performance in tasks that can be solved using episodic retrieval of specific recent trajectories, including the 8-arm radial maze (Bunce et al., 2004), delayed spatial alternation (Ennaceur et al., 1996), the Morris water maze with new platform location on each day (Buresova et al., 1986; Steele & Morris, 1999) and a task testing a sequence of spatial locations (Lee et al., 2005). Spatial memory is also impaired by lesions of the entorhinal cortex (Steffenach et al., 2005) and postsubiculum (Taube et al., 1992). Learning of spatial trajectories may be a special case of a general capacity for learning sequences within the hippocampus (Eichenbaum et al., 1999), including the sequential order of sensory stimuli (Kesner & Novak, 1982; Agster et al., 2002; Fortin et al., 2002; Kesner et al., 2002).
Many previous models of hippocampal function focus on its role in spatial navigation to goals (Touretzky & Redish, 1996; Burgess et al., 1997; Foster et al., 2000; Trullier & Meyer, 2000), but not on episodic retrieval of specific trajectories. Most previous hippocampal models that focus on encoding and retrieval of sequences (McNaughton & Morris, 1987; Minai & Levy, 1993; Treves & Rolls, 1994; Jensen & Lisman, 1996a; b; Levy, 1996; Tsodyks et al., 1996; Wallenstein & Hasselmo, 1997; Redish & Touretzky, 1998; Hasselmo & Eichenbaum, 2005; Zilli & Hasselmo, 2008c) focus on encoding associations between discrete sequential states (items or locations). However, recent data on grid cell firing in the entorhinal cortex (Hafting et al., 2005; Sargolini et al., 2006; Barry et al., 2007; Moser & Moser, 2008) suggests a different approach (Hasselmo, 2008b) in which each individual state (place) is associated with an action (the velocity coded by speed-modulated head direction cells).
This model of the episodic encoding and retrieval of trajectories can use either of two main classes of grid cell models. One class of models generates grid cells based on interference patterns (Burgess et al., 2007; Burgess, 2008). This model could use mechanisms of membrane potential oscillations shown in entorhinal neurons (Alonso & Llinas, 1989; Giocomo et al., 2007; Hasselmo et al., 2007; Giocomo & Hasselmo, 2008b; a), or could use mechanisms of stable persistent spiking (Egorov et al., 2002; Fransén et al., 2006; Tahvildari et al., 2007; Hasselmo, 2008a). The other class of models uses attractor dynamics to generate grid cell activity (Fuhs & Touretzky, 2006; McNaughton et al., 2006). The first type of model is used here, but either or both types of models could be used, because both models update grid cell position with a velocity signal from head direction cells. As shown here, a circuit mechanism using grid cells provides a substrate for encoding and retrieval of trajectories defined on continuous dimensions of space and time.
The model presented here will consider encoding and retrieval of a trajectory of movement through the environment by an agent over time. The agent could be a human being or other mammal. The circuit model of encoding and retrieval is summarized in Figure 1. The physiological data used to justify the model has primarily been obtained from the rat, including data on entorhinal grid cells (Hafting et al., 2005; Fyhn et al., 2007; Moser & Moser, 2008), head direction cells in structures such as postsubiculum (Taube et al., 1990a; Sharp, 1996; Taube, 1998; Taube & Bassett, 2003), hippocampal place cells (O’Keefe & Burgess, 1996; 2005), and context-dependent firing in hippocampus (Eichenbaum et al., 1999; Ferbinteanu & Shapiro, 2003; Lee et al., 2006; Smith & Mizumori, 2006; Griffin et al., 2007). Analogous neural responses have been demonstrated in non-human primates (Rolls et al., 1997; Robertson et al., 1999) and humans (Ekstrom et al., 2003). The following sections describe how the model uses representations of head direction activity, entorhinal grid cell activity, and hippocampal cell activity to perform episodic encoding and retrieval of trajectories.
The model can encode the trajectory through the environment by encoding associations between location and velocity at each location. The two dimensional location of an animal at one time can be represented with the row vector = [x y]. The trajectory through different locations over time can be represented by a sequence of vectors (t). This representation can be obtained by integration of the velocity row vector (t) = [dx/dt dy/dt]. Integrating the velocity vector over time yields the current location vector (t) at time t relative to the starting location vector 0, as follows:
The use of a velocity signal was motivated by data showing that hippocampal spiking activity during REM sleep replays not only the sequence of spatial locations coded by cells, but maintains temporal intervals (Louie & Wilson, 2001), in contrast to the compressed sequences observed during sharp wave/ripples (Skaggs & McNaughton, 1996). Coding of these temporal intervals requires the speed component of velocity in addition to direction.
The encoding of associations between location and velocity requires a representation of velocity. Velocity includes instantaneous direction and speed. These components of velocity can be provided by neural activity coding head direction and translational speed.
Extensive experimental data describes head direction cells in the postsubiculum (dorsal presubiculum) and the anterior thalamus (Taube et al., 1990b; Taube & Burton, 1995; Sharp, 1996; Taube et al., 1996; Sharp et al., 2001; Taube & Bassett, 2003) and in deep layers of the entorhinal cortex (Sargolini et al., 2006). These neurons show selective firing when the rat is facing in a narrow range of directions. In the model, head direction activity is combined with speed modulation. This can be obtained by combining head direction activity with input from cells that are modulated by translational speed, as shown in the postsubiculum (Taube et al., 1990a; Sharp, 1996) and other associated neural structures (O’Keefe et al., 1998; Sharp & Turner-Williams, 2005; Sharp et al., 2006).
In this model, the firing of head direction cells combined with speed modulation is represented with the vector (t). The model computes the speed-modulated head direction response from the velocity vector of movement in space. These two vectors differ in the model in that the dimensions of the head direction vector are at intervals of 120 degrees, rather than using the 90 degree interval between the x and y dimensions of cartesian coordinates. The head direction vector is obtained by multiplying the velocity vector by the head direction transformation matrix:
In this matrix, ι represents the selectivity of individual head direction cells with index i. This transformation essentially maps x,y coordinates into coordinates relative to dimensions skewed to the angles of head direction preference. This yields the speed- modulated head direction vector: (t)= (t)H. As the velocity vector changes over time (t), this changes the speed modulated head direction vector over time (t).
Because head direction can be obtained from velocity by the head direction transformation matrix (t) = (t)H, this means that integration of the speed modulated head direction over time corresponds to position transformed by the same matrix H: . In the model, the integration of speed modulated head direction is performed by entorhinal grid cells, and provides a mechanism for path integration (O’Keefe & Burgess, 2005; Burgess et al., 2007; Hasselmo et al., 2007; Hasselmo & Brandon, 2008).
In the figures, for the purpose of demonstrating how the entorhinal grid cells represent location, the integral of speed modulated head direction can be transformed back into Cartesian spatial coordinates. This is done by multiplying the integral of head direction activity by the inverse of the head direction transformation matrix: . The location coded by grid cell firing phase can be obtained from just two components of the integrated head direction, allowing use of a two component inverse matrix (Strang, 1988) as follows:
How does the model perform integration of the velocity signal? In the model, the integration necessary to keep track of spatial location is performed by shifts in the relative phase of oscillations in a simulation of entorhinal grid cells. An example of the shift in relative phase of neural oscillations is shown in Figure 2. Oscillations can be described by the instantaneous phase at each time point ϕ = 2πft, which starts with an initial phase at time zero. The relative phase used in this model is the difference between the instantaneous phase of one oscillation and the instantaneous phase of a baseline oscillation at that same time point. Figure 2 shows two oscillations that start out with relative phase of zero. A velocity input then shifts the frequency of one oscillation, causing an additional change in its instantaneous phase that results in a gradual change in relative phase compared to a baseline oscillation. Dashed lines illustrate the phase of the peak of the shifted oscillation relative to the time of the peak of the baseline oscillation with fixed frequency. This mechanism can be used to generate grid cell firing patterns as shown in Figure 2A.
The integration of velocity in the model arises due to input from head direction cells depolarizing entorhinal neurons. As shown in Figure 2, depolarizing input in the model causes a shift in the frequency of membrane potential oscillations in the model, thereby shifting their relative phase. This change in relative phase of oscillations could underlie the pattern of entorhinal grid cell firing, as initially shown in the model developed by Burgess and colleagues (Burgess et al., 2005; O’Keefe & Burgess, 2005; Burgess et al., 2007). Experimental data from entorhinal cortex suggests that intracellular membrane potential oscillations could provide this mechanism of the model (Giocomo et al., 2007; Hasselmo et al., 2007; Giocomo & Hasselmo, 2008b). The model is based on data showing that entorhinal stellate cells exhibit membrane potential oscillations around theta frequency (Alonso & Llinas, 1989; Alonso & Klink, 1993; Dickson et al., 2000; Erchova et al., 2004; Giocomo et al., 2007; Engel et al., 2008; Giocomo & Hasselmo, 2008b), and these membrane potential oscillations appear with different frequencies at different membrane potentials (Giocomo & Hasselmo, 2008a). There are excitatory projections to entorhinal cortex stellate cells from the postsubiculum (Caballero-Bleda & Witter, 1993) and deep layers of entorhinal cortex (Sargolini et al., 2006), both of which contain head direction cells. Thus, active head direction cells can directly depolarize entorhinal stellate cells.
If we consider a pure velocity input, then the instantaneous phase of shifted oscillations s would depend upon the baseline frequency f and the shift in frequency due to velocity, as follows: . The relative difference in instantaneous phase between the shifted oscillation (with index s) and an oscillation with the baseline frequency (index b) will be: . As shown in Figure 2, the shift in relative phase of the oscillations depends upon the magnitude and duration of the velocity input.
The Burgess model (Burgess et al., 2007) uses the speed modulated head direction (t) to regulate different oscillation frequencies as summarized by the following equation:
In most simulations presented here, this grid cell model equation was replaced with an equation that computes the relative phase difference more rapidly, and generates denser grid cell firing and more effective place cell formation. Rather than computing the full sum of multiple oscillations, this equation directly computes the relative phase difference caused by head direction input as follows:
Where g(t) represents the firing of a single grid cell over time. Πi represents the product of the different dendritic oscillations receiving input from different components of the head direction vector with index i, and Θ represents a Heaviside step function output (the model has output 1 for any value above a threshold). The intrinsi oscillations change in frequency in proportion to a constant B. The oscillations each have an initial phase ϕ(0)ik. Figure 3 shows simulations from the model of different grid cells with differences in size and spacing of fields (due to different values of Bj) and different initial phases ‘i’ and ‘k’. These differences in spacing and phase of grid cells are essential to the coding of spatial location and generation of place cell responses in the model. The grid cell spacings shown here prove effective for coding location. Even though the grid cells have repeating fields, the combination of input from different grid cells can result in place cells that are selective over large distances. This mechanism can function on a larger scale as well, as some cells receiving output from these grid cells will respond on the basis of the least common integer multiple of the spacing of a single grid cell (Gorchetchnikov & Grossberg, 2007), so that spacings of 30, 40 and 50 cm can drive selective coding at a scale of 450 cm. The scale of spatial specificity can be further expanded by including grid cells with larger spacing intervals up to 10 meters as shown in more ventral regions of entorhinal cortex (Brun et al., 2008).
The difference in size and spacing of grid cell firing fields depends upon differences in the effect of depolarization on intrinsic frequency that have been shown along the dorsal to ventral axis of medial entorhinal cortex (Giocomo et al., 2007; Hasselmo et al., 2007; Giocomo & Hasselmo, 2008b). The original Burgess model proposes that grid cells arise from interference, but is open to different mechanisms for the oscillations undergoing interference. The equation could represent the interaction of membrane potential oscillations within a single neuron (Giocomo et al., 2007; Hasselmo et al., 2007), or it could represent the interaction of phasic synaptic input from neurons firing rhythmically due to persistent spiking (Hasselmo, 2008a) or due to oscillatory network dynamics. The progressive scaling of neural responses proves essential to this model, and could prove to be a more general phenomenon that appears in other areas such as medial prefrontal cortex and piriform cortex.
The stability of grid cell and place cell firing over multiple sessions requires sensory input. The velocity input alone only maintains the stability of firing response during the time the velocity is present. To main stability of firing across different sessions, sensory cues are necessary to set the phase of firing to the same starting point. In the simulations presented here, the sensory cues representing specific items are associated with specific sections of the trajectory, and then these sensory cues can cue retrieval of the trajectory. This influence of sensory cues is summarized in the section on Associations with Sensory Stimuli.
The phase of oscillations changes linearly with location, as shown in Figure 2G, which plots the phase of a single modelled oscillation as the simulated rat moves along a complex trajectory through the environment. The plot of phase shows a single plane of activity that is related linearly to the two-dimensional location of the animal. Thus, the internal representation of location relative to starting position can be extracted directly from the phase of two oscillations bymultiplying the relative phase of the oscillations by the inverse of the head direction transformation matrix (H) and dividing by the beat frequency (2πfB) as follows:
This process is used to create figures showing how the activity of the model represents spatial location over time, allowing plotting of the trajectory retrieved by internal activity over time. This also indicates that for return to a familiar environment or for off-line retrieval of a trajectory, associative retrieval based on sensory cues or internal spread can cue the correct initial phase of the dendritic oscillations ϕ⃑(0), which is the initial location of the rat multiplied by the head direction transformation matrix scaled by the phase of the beat frequency 2πfB. Thus, the initial phase: ϕ⃑(0) = 2πfB0H.
The grid cell firing fields are stable in a familiar environment, indicating that the initial phase of oscillations can be set by familiar stimuli in the environment. This setting of phase could be provided by the angle or distance of stimuli viewed in the environment.
The phase of grid cell oscillations effectively represents location, but how can this be accessed by other neural structures? The read-out of spatial location can be provided by hippocampal place cells. Thus, the next stage of the model involves grid cell effects on place cells. Examples of place cells that can be generated by the model are shown in Figure 3.
The influence of grid cells on place cells has been modelled in different ways (Fuhs & Touretzky, 2006; McNaughton et al., 2006; Rolls et al., 2006; Solstad et al., 2006; Gorchetchnikov & Grossberg, 2007), but research has not converged on a single model. In general, the place cell representation p(t) arising from the grid cell representation g(t) in entorhinal cortex would depend upon the synaptic connectivity WGP from grid cells to place cells.
In the model, place cells are created by selection of random subsets of three grid cells inputs and computation of the overlap in firing for these three grid cells. The place cell is assumed to fire anywhere these three grid cells fire simultaneously. This generates a spatial firing pattern for each place cell that is evaluated by taking the standard deviation of spiking location in the x and y dimension. If the standard deviation of spiking location is smaller than a previously set parameter in both dimensions, then this place cell is selected for inclusion. The input synapses from the randomly selected subset of three grid cells to this place cell are then strengthened according to the outer product equation:
Where (t)T is a column vector of presynaptic grid cell activity including the three currently selected grid cells, and (t) is a row vector of place cell activity with only the currently selected place cell active. This ensures that the same place cells are reliably activated in the same location dependent on the pattern of grid cell spiking induced by the phase of grid cells. Examples of place cell firing generated by the model are shown in Figure 3.
During both encoding and retrieval of an episode, the place cell row vector (t) results from the pattern of activity in the grid cell vector (t) dependent upon multiplication by the previously modified synaptic connectivity WGP between these regions as follows:
How does the model encode a trajectory? This depends upon sequentially encoding associations between place cells and velocity, and later sequentially retrieving these associations. The third stage of the model plays the primary role in encoding of a new trajectory.
This encoding is performed by modification of synaptic connections between active hippocampal place cells and active head direction cells. This synaptic modification allows formation of associations between individual states (spatial locations) and the actions (movements) performed in those states.
During initial encoding of a trajectory, the activity of all cells is determined by the actual velocity of the animal during movement. The sensory input about actual speed and head direction drives the head direction cell activity vector h(t). The head direction input updates the grid cells according to the grid cell model in equation 3, and the grid cells activate individual place cells according to equation 7.
Initial encoding of a trajectory involves strengthening a set of synapses WPH associating each location (coded by place cells p(t)) and the associated movement action performed at that location (coded by the speed modulated head direction cell vector h(t)). Thus, a trajectory is stored by formation of associations via synaptic modification of the synaptic connectivity matrix WPH between the population of place cells (t) and the population of head direction cells (t), as follows:
This synaptic modification during encoding of the trajectory completes the circuit shown in Figure 1A & 1B that forms the basis for later retrieval of the trajectory.
The synaptic connections between place cells and head direction cells could correspond to direct projections from region CA1 of the hippocampus to the postsubiculum (van Groen & Wyss, 1990), or to polysynaptic effects of projections from the hippocampus to subiculum (Swanson et al., 1978; Amaral & Witter, 1989), combined with anatomical projections from the subiculum to the postsubiculum and medial entorhinal cortex (Naber & Witter, 1998).
The cue for retrieval can take a number of forms. The cue can be the actual current location of the rat, coded by place cells updated by grid cells. The cue could also be environmental stimuli that have been associated with a particular pattern of place cells, that can reactivate the place cell pattern. Sensory stimuli are used to cue retrieval in the examples shown later, using the mathematical representation of associations between sensory stimuli representing items or events and places, as described below in equations 15 and 16. Finally, a cue that does not correspond to the actual current stimulus or location in the environment could be activated by internal mechanisms.
Whatever the source, the place cell cue can initiate retrieval of a trajectory by activating the associated action. Thus, activity h(t) can be retrieved from the current place cell activity as follows:
This forms one component of a functional loop for retrieval in the model. This loop retrieves trajectories via three stages: 1.) place cells activate associated head direction activity via matrix WPH (equation 9), 2.) head direction cells update the phase of entorhinal grid cells (equation 3 or 4), and 3.) grid cells g(t) update place cells via matrix WGP (equation 7). The new place cell activity then activates the associated head direction pattern, and the loop continues driving retrieval of the trajectory. This loop is summarized in Figure 1B.
Retrieval activity in the model can be summarized with the following equation:
The previous place cell activity spreads across the matrix WHP to drive activity in the head direction cells. The grid cell function g(t) from equation 3 integrates this activity to update grid cell phase. Then the grid cell activity is transformed through a matrix WPG representing the synaptic drive of grid cells on hippocampal place cells. Thus, the pattern of activity in these two matrices encodes the episodic memory for the trajectory in the model.
The effect of head direction on grid cell activity during retrieval requires that the head direction vector (coding a velocity vector corresponding to rat action) needs to maintain activity for the period Δt until a new place cell representation causes retrieval of the next head direction response. This is justified by graded levels of persistent firing that have been shown in intracellular recording from neurons in deep layers of entorhinal cortex slice preparations (Egorov et al., 2002; Fransén et al., 2006) and intracellular recording in the postsubiculum (Yoshida & Hasselmo, 2009). This suggests that head direction cells in the deep layers of entorhinal cortex (Sargolini et al., 2006) and postsubiculum (Taube et al., 1990a; Sharp, 1996) may be involved in encoding and retrieval of trajectories.
Note that retrieval has different dynamics from encoding, as shown in Figure 1A and 1B. During encoding, the sensory input of velocity determines activity of head direction cells, which then drive grid cells and place cells. During retrieval, sensory input does not influence the system, and head direction activity is determined internally. Awareness of this difference in dynamical state allows differentiation of retrieved events from current sensory input. The difference in dynamics could be determined by modulatory or attentional influences on the postsubiculum or deep entorhinal cortex determining the relative influence of different synaptic inputs. Encoding and retrieval of single associations or items have been modelled as occurring on different phases of each theta cycle (Hasselmo et al., 2002), but the sequential retrieval of a full trajectory requires updating of phase over multiple cycles, and might not be feasible to interleave with encoding on each cycle. This slower transition between encoding and retrieval could involve modulatory regulation of afferent versus feedback transmission by muscarinic acetylcholine receptors (Hasselmo, 2006).
Trajectory retrieval can be greatly enhanced by combining the mechanism based on place cells p(t) with cells that code time intervals alone, or by cells that code arc length of the trajectory (Hasselmo, 2007). Cells with fixed differences in frequency will cause interference at a fixed time interval. In contrast, arc length cells will activate at a specific arc length (travel distance) along the trajectory from a given reference location. Both types of cells can be obtained in simulations by assuming that a specific item or location activates persistent spiking neurons in the entorhinal cortex. Persistent spiking neurons with fixed frequency will interfere with a baseline frequency at a specific time interval from their onset. Neurons involved in arc length could fire at a rhythmic rate that is modulated by speed but not by head direction. The modulation by speed drives neurons at frequencies that differ from the baseline frequency of other persistent spiking cells. The neurons receiving convergent input from these persistent spiking cells will fire dependent upon when the shift in frequency causes constructive interference with the baseline cells.
In mathematical terms, the mechanism of determining the activity a of arc length cells uses an interaction of two oscillations as follows:
This resembles the model of grid cells, except that the interaction involves only two oscillations and the modulation of frequency depends on speed s(t) only rather than speed-modulated head direction. Note that each arc length cell with index ‘i’ will fire at an arc length dependent upon its initial phase ϕi(0). The threshold and frequency of arc length cells was set so that they would fire once during the trajectories studied here.
Simulations of arc length cells (Hasselmo, 2007) are able to replicate a number of features of the physiological data from the hippocampal formation, including the phenomenon of context-dependent “splitter” cells that fire selectively for right or left turn trials in continuous spatial alternation (Wood et al., 2000; Lee et al., 2006), the forward shifting of these splitter cells toward goal locations (Lee et al., 2006), and the context-dependent firing of neurons in a delayed non-match to position task (Griffin et al., 2007) or plus maze tasks (Ferbinteanu & Shapiro, 2003; Smith & Mizumori, 2006). A similar function could be obtained from cells with frequencies fixed in time that can code pure temporal intervals (Hasselmo et al., 2007).
A circuit composed of arc length cells, speed modulated cells and cells with oscillation frequencies modulated by speed can model sequential activation of arc length cells (Figure 1C). Analogous to the connection between place cells and head direction cells (WPH), a connectivity matrix WAS links the column vector a(t)T representing arc length cell activity with the associated speed s(t) at each position along the trajectory.
This process does not depend upon retrieval activating an accurate vector of head direction activity at each step. Instead, it only requires that retrieval activate a representation of speed. Thus, retrieval drives phase changes in one dimension along the trajectory instead of two dimensions. The activity a of arc length cells drives the activity of speed cells as follows:
Note that the arc length cell could be associated with the mean speed on a segment of the trajectory rather than the instantaneous speed, as speed can vary considerably along a segment, and the total arc length A computed by the phase shift is a constant for each segment that results from proportional changes in mean speed and transition time, such that lower mean speed means longer transition time: A=tS.
Once an arc length cell has activated the appropriate mean speed during retrieval, the neurons representing the mean speed should modulate the oscillation frequency of another set of neurons, potentially the persistent firing cells in layer III of entorhinal cortex (Tahvildari et al., 2007). The change in frequency of these cells can update the phase of the interference giving rise to arc length cells. In simulations, the change in instantaneous phase of interference ϕa giving rise to arc length cell firing depends upon the speed modulation s(t) over each time interval Δt:
In addition to the associations with speed, arc length cells are concatenated with the place cell activity and represented together by the vector of place cell activity p(t) so they form associations with speed modulated head direction cells in the matrix WPH created in equation 8. This forms associations between arc length cells and the action at the corresponding position on the trajectory. These cells and their phases are also associated with stimuli representing items or events as described in the next section.
During encoding, arc length cells i are created at regular intervals of arc length when the magnitude of interference in equation 11 crosses a threshold. In these simulations, the threshold and frequency shift of arc length cells were set so they would be active at a single contiguous segment on a trajectory. At the start of retrieval, the arc length phases are reset to the appropriate values and equation 11 is computed based on retrieved speed rather than external speed. This causes sequential activation of arc length cells to drive speed-modulated head direction and thereby drive the correct update of grid cell phase and place cell firing over time.
The simulations included formation of associations between the place cell vector p(t) (which included the arc length vector) and a vector o(t) representing the stimulus features of individual objects or events experienced at specific positions on a trajectory. Individual objects were assigned locations near the trajectory, and the object vector was active when the agent was within a specific distance of the object location. During encoding, Hebbian modification of synapses WPO formed associations between the stimulus feature vector o(t) (potentially arriving from the lateral entorhinal cortex) and the concatenated place cell vector p(t) (that included the arc length vector) as follows:
The retrieval of items shown in some figures resulted from the place cell vector p(t) (that included arc length cells) activating the stimulus feature vector o(t) according to:
Because the place cell vector p(t) includes arc length cell activity, the matrix WPO also allows arc length cells activated at a specific position along a trajectory to cue the retrieval of stimulus features associated with objects or events.
In some examples, retrieval was cued by a specific stimulus feature vector. This was done by using bidirectional Hebbian modification so that the inverse of WPO would code associations between the object stimulus feature vector and the place cell vector (including arc length cells). In addition, a matrix coded associations between individual arc length activity and the vector of oscillation phases in the entorhinal grid cells. This allowed an individual stimulus feature vector to activate the associated pattern of place cells and arc length cells, and the arc length cell representation to activate the associated grid cell oscillation phase. This allowed cueing of retrieval from a specific stimulus or position along the trajectory.
The model presented here is effective for performing episodic encoding and retrieval of trajectories and items or events along the trajectory. Performance of the model was tested on a variety of trajectories. Some tests were performed with trajectories from experimental data in the Moser laboratory (Hafting et al., 2005), posted by Hafting and Fyhn on the web site at http://www.cbm.ntnu.no/. In this data, a rat moves along a winding, self-crossing trajectory as it forages for randomly scattered food pellets in a circular open field with diameter of 2 meters. In most simulations, simple trajectories were created by combinations of linear segments of movement in the environment.
At the start of retrieval, the simulation is cued with activation of the initial grid cell phase vector ϕ(0) associated with the start of the trajectory, or with a sensory stimulus that activates the grid cell and arc length cell phases at a particular point. This retrieval cue could be triggered by excitatory input from prefrontal cortex. Examples of trajectory retrieval are shown in the Figures.
As shown in the figures, the model described here performs encoding and retrieval of complex spatial trajectories. Figure 4 illustrates the basic components of the trajectory retrieval, which includes head direction cell activity (4A), grid cell phase (4B), and place cell activity (4C). In the figure, the actual trajectory run by the rat is shown in gray. This trajectory is from experimental data obtained in the Moser laboratory (Hafting et al., 2005). The rat forages along a meandering trajectory through a circular open field environment. In Figure 4A, examples of head direction vectors activated by place cell activity at particular locations are shown as straight line segments. These head direction vectors guide the direction of shifts in grid cell phase shown by the black line in Figure 4B. The black line shows retrieval of the trajectory over the full period of retrieval, based on applying the inverse head direction transformation matrix to two components of the phase vector of an individual grid cell (corresponding to the internal representation of location during retrieval). In Figure 4C, the individual place cells activated by the phase change are shown as open circles. Each of these place cells activates an associated head direction vector shown in Figure 4A.
In simulations, during retrieval, the network can retrieve the full sequence from a cue consisting of just the initial grid cell phase representation (Figure 4B) of starting place along the trajectory (in simulations, the phase representing first location was presented using: ϕ(0) = Hx0). This grid cell phase drives place cell activity (Figure 4C) that then activates the head direction vector h (Figure 4A) associated with that pattern of place cell activity. The head direction vector drives the update of the phase vector of the grid cells for a period of time. This occurs through the influence of the head direction activity on grid cell oscillation frequency. Eventually, the change in grid cell phase activates a new place cell representation (Figure 4C). Activation of new place cells activates the associated head direction neuron activity vector (Figure 4A), and this new head direction vector drives further update of the grid cell phase representing location. Thus, a recursive process allows place cell activity (state) to drive retrieval of the next action (speed modulated head direction) to update grid cell phase to activate the next place pattern (state) in a repetitive manner through the length of the trajectory.
As shown in Figure 5, the network retrieves complex spatial trajectories with accuracy dependent upon the resolution of place cell representation. In the left column of Figure 5A, the black lines represents the change in grid cell phase that codes specific locations. Grid cell phase is continuous, so it does not limit resolution, but accuracy of retrieval is limited by the number of place cells. With 400 place cells, the network can retrieve the encoded trajectory reasonably well. For lower numbers of place cells, the network shows progressively poorer performance, and the internal representation of place deviates more from the actual trajectory. The right column of Figure 5A shows the error in performance over time for trajectory retrieval in each example shown on the left. Error is computed as the difference (in centimeters) between each time point of the encoded trajectory and the time point of the retrieved trajectory (note that this requires similar timing, as retrieval of the same trajectory at a different speed would cause a buildup of error). Note that with only one place cell, the trajectory rapidly deviates from the encoded trajectory, but with larger numbers the error increases more slowly. Figure 5B shows the mean error averaged across the full trajectory for different numbers of place cells (1, 16, 64, 144, 256, 400). Mean error is computed as the mean difference between each time point of the encoded trajectory and each time point of the retrieved trajectory. Mean error decreases with increasing numbers of place cells, due to the more accurate mapping between individual locations and the corresponding difference in speed-modulated head direction at each location.
Figure 6 shows the encoding and retrieval of a simple trajectory with individual objects (or events) at individual points near the trajectory. In this simulation, individual items are located near specific points along the trajectory, as shown by the location of the gray symbols. The trajectory first passes the gray plus sign, then the gray asterisk, then the gray circle. During encoding, the pattern of place cell activity is associated with patterns representing individual items. During retrieval, the trajectory progressively activates place cell activity that activates vectors representing individual items. The locations where individual item representations are activated are shown with black symbols. Note that during retrieval, the network effectively retrieves first the black plus sign (at the first turn), then the black asterisk (at the second turn), and finally the black circle.
As shown for a different trajectory in Figure 6C, items or events can be retrieved based on the temporal intervals at which they were encountered. In this different trajectory example, five items are encountered during movement in one dimension. Because retrieval depends upon velocity, two items (circle and triangle) encountered at a slow velocity (1 cm/sec) are encoded and retrieved at a long temporal interval (12 sec, x axis) despite being close in space (12 cm, y axis). In contrast, two items (triangle and asterisk) encountered at high velocity (20 cm/sec) are encoded and retrieved at a short temporal interval (3–4 sec, x axis) despite being far apart in space (60 cm, y axis). As shown in Figures 6D and 6E, the model encodes the different velocities at different positions (going from 10 cm/sec to 1 cm/sec to 20 cm/sec to 2 cm/sec to 5 cm/sec), and effectively replicates these different velocities during retrieval. The Figure 6C simulation used 100 arc length cells. With 400 place cells, the temporal interval accuracy was similar for widely spaced items, but was poorer for the circle and triangle interval due to the resolution limit caused by place field size.
Figure 7 shows the performance for different numbers of place cells and arc length cells in this simulation. Again, increasing the number of place cells increases the performance. In addition, increasing the number of arc length cells results in better performance.
Figure 8 shows a problem faced by the encoding of the trajectory as a chain of associations between locations and actions. When there is ambiguity due to overlap in the location (state) representations, the network can miss segments of the trajectory or become trapped in loops. In Figure 8A, the network shows retrieval which misses a large loop in the upper right portion of the trajectory. This occurs because at the point of trajectory overlap, place cells are associated with two different actions (go North and go West) on the trajectory. In this example, the place cells activate the action (go West) associated with the later portion of the trajectory and causes retrieval to skip ahead to the last segment of the trajectory. Performance plotted in Figure 8B shows that this problem cannot be overcome by increasing the number of place cells. The mean error for place cells stays at a high level (poor performance) even for 400 place cells.
Figure 8C shows a mechanism for overcoming the problem of trajectory overlap. This mechanism uses the arc length cells that provide a context-dependent signal for different portions of the trajectory, allowing differential representation of the same location depending on the preceding spatial locations. As described above, the arc length cells respond on the basis of the arc length along the trajectory rather than actual spatial position, thereby disambiguating overlapping locations occurring at different arc lengths along the trajectory. Each time a new arc length cell is activated, the arc cell activates the correct head direction cells and drives the grid cell phase to the correct spatial location. Figure 8C shows how retrieval of self-crossing trajectories is enhanced by inclusion of arc length cells. At the overlapping segment, the arc length cells activated for the early part of the trajectory are distinct from those coding the later segment, so they guide the grid cell phase through the overlapping segment and allow activation of the grid cell phase and place cells representing the loop in the upper right corner of the trajectory. Performance plotted in Figure 8D shows that mean error during retrieval with arc length cells reaches very small values when the number of arc cells reaches 50, indicating that retrieval very closely matches the originally encoded trajectory.
Figure 9 illustrates how the model presented here can be used to encode and retrieve common types of episodic memories, such as the location that one parks one’s car. The trajectories in Figure 9 illustrate the example of parking a car in the same parking lot on two different days. On the first day, the trajectory represents the car driving through the entrance (plus sign) and parking in a location to the left (circle). The trajectory continues with the driver walking into the building entrance (top) and later returning to the car on the left and driving back out the parking lot entrance from the left. On the second day, the trajectory represents the car driving through the same entrance but with a different cue indicating the second day (square symbol). This could indicate a different event or stimulus on that day. The trajectory then goes to a parking location on the right (asterisk). The trajectory continues with the driver walking into the building entrance (top).
Retrieval of the episodic memory is tested on the second day, as the driver comes back out of the building. In Figure 9A, the cue for memory retrieval is a specific event or stimulus (square symbol) that took place during the drive to the entrance on the second day (cue today). This activates reset of arc length cells that guide correct retrieval of the trajectory to the right side parking location (asterisks). Alternately, if necessary, the arc length cells can mediate retrieval of the parking location on the previous day (cue yesterday), as shown in Figure 9B. This uses a cue for memory retrieval in the form of a stimulus or event (plus sign) that took place during the drive to the entrance on the first day (yesterday). This activates reset of arc length cells that then guide retrieval of the trajectory to the left side parking location (circle).
Can the system work without arc length cells? Yes, but only for selection of the more recent trajectory. As shown in Figure 9C, a network without arc length cells, but with 400 place cells can be cued with an event from the current dat at the entrance of the parking lot, and can retrieve the recent episode (square symbol, cue today) of parking on the right (asterisk). However, a retrieval cue using an event at the entrance on the first day (plus sign, cue yesterday) does not retrieve the left side parking location (circle). Instead, the trajectory retrieval follows the recent trajectory to the right side (asterisk), as shown in Figure 9D. This occurs because the recent modification of synapses between place cells at the entrance and speed modulated head direction cells coding the right turn reduces the strength of previously modified synapses associated with the previous left turn. Thus, the entrance location is associated with the recent trajectory and cannot retrieve the older trajectory. Pure place to action associations represent temporal recency by differences in strength. In contrast, the arc length cells in this model allow distinct encoding and retrieval of the location that a car was parked on a previous day, as long as a distinct cue can be accessed to appropriately cue the reset of arc length cells.
The sensory cueing of trajectory retrieval is demonstrated further in Figure 10A. This Figure shows examples of a trajectory passing multiple items. During retrieval, presentation of individual items as cues (such as the ‘x’, the ‘o’ and the diamond) can activate the associated phase and arc length cell activity, initiating retrieval of the remainder of the trajectory and subsequent items. As shown in Figure 10B, the arc length cell representation allows successful retrieval of one trajectory even if it has strong overlap with a different trajectory. The retrieval of pathways with different lengths of overlap has been addressed in Figure 10C, which shows that arc length coding is effective for retrieving trajectories that overlap for extended segments of 10 cm, 20 cm or 40 cm, whereas place cell coding breaks down for all types of overlap.
The model presented here uses grid cells, place cells, head direction cells and a set of context-dependent cells termed arc length cells to encode and retrieve neural activity associated with specific spatiotemporal trajectories through the environment. During encoding, external input drives head direction activity that drives entorhinal grid cell and hippocampal cell activity. Synaptic modification strengthens connections between hippocampal place cells and head direction cells. During retrieval, the phase of neural activity in grid cells represents the retrieved spatial state of the animal, which can be read out through the influence of grid cells on the activity of hippocampal place cells. The spread across modified synapses from place cells to head direction cells represents retrieval of the action associated with each state. In addition, the spread of activity from hippocampal place cells to neurons representing objects or events allows retrieval of the specific location and timing of each item or event. The temporally structured replay of memory shown here is an important property of episodic memory (Eichenbaum & Cohen, 2001; Tulving, 2002). Reminiscence reactivates the sequential events and locations in an episode with information about time intervals and relative spatial locations.
The model draws on recent physiological data to provide important functional components of the circuit. These mechanisms use phase coding that could arise from two different intrinsic cellular mechanisms in entorhinal cortex. The phase coding could depend upon shifting the baseline frequency of persistent spiking activity (Hasselmo, 2008a) that has been shown as an intrinsic mechanism of pyramidal cells in the entorhinal cortex (Egorov et al., 2002; Fransén et al., 2006; Tahvildari et al., 2007; Yoshida et al., 2008). The phase coding could also depend upon the phase of intracellular membrane potential oscillations that have been shown as an intrinsic mechanism in stellate cells in entorhinal cortex (Alonso & Llinas, 1989; Giocomo et al., 2007; Giocomo & Hasselmo, 2008b). In both cases, the voltage-dependent shift in frequency shifts the phase of grid cells, allowing continuous retrieval of a trajectory until the next closest spatial state is activated in the form of new place cell activity. As shown in the figures, complex trajectories can be encoded and retrieved accurately, depending on the resolution of the place cell representation.
This framework can be enhanced by the mechanism of arc length cells. Arc length cells respond on the basis of arc length of the trajectory from the most recent reset location. This allows disambiguation of retrieval at locations where trajectories overlap and more than one head direction (action) has been associated with a particular spatial state (place cell activity). In this case, the arc length activity drives the correct head direction activity or object representation dependent upon the arc length of that place from the location where arc length oscillations were most recently reset.
As shown in the figures, the model can retrieve multiple different events or items at particular positions along a trajectory, and individual events or items can cue retrieval from a particular position on the trajectory. The coding of associations with stimulus feature vectors representing items or events could depend upon convergence of two different pathways. The medial entorhinal cortex could provide a temporal and spatial context that can be associated with specific objects that arrive via input from the lateral entorhinal cortex and postrhinal cortex (Hargreaves et al., 2005; Eichenbaum & Lipton, 2008).
This model uses interference between different oscillations or rhythmic spiking activity with different phases to code the location or timing of individual movements on a trajectory and to encode associations to specific sensory stimuli. This use of oscillations has some similarities to the use of multiple oscillators for encoding the order of words in serial recall tasks (Brown et al., 2000). In addition, this use of oscillations resembles the use of oscillations for encoding temporal intervals in models of the timing of behavioural responses (Miall, 1989; Matell & Meck, 2004).
This model provides explicit physiological mechanisms for elements of episodic memory, including the perception of mental time travel during the retrieval of a prior episodic memory. The model presented here has the capacity to retrieve multiple continuous dimensions of an experience, including an explicit representation of the time duration during and between individual events in an episode, as well as the sense of spatial location, heading direction and movement speed. Though this has not been quantified extensively, subjective experience of episodic memories includes a sense of heading direction and movement speed. For example, the memory of a recent social event could include turning toward individual people, and moving through rooms with a particular direction and speed.
There are limits to the capacity of such a system to encode and retrieve multiple overlapping trajectories in the same environment. As shown in the figures, self-crossing trajectories cannot be accurately encoded by place cells, even when the number of place cells is large. Arc length cells can code self-crossing trajectories, but the number of arc length cells required will increase with the length of the trajectory. This suggests that trajectories are more effectively encoded in novel or distinctive environments, whereas in a highly familiar environment memory is usually best for the most recent trajectory or the earliest trajectories.
The use of oscillatory interference to encode episodes illustrates the strength of this mechanism for encoding a wide range of phenomena in continuous time and space. The same techniques could potentially be used for encoding of general perceptual features, such as computation and prediction of stimulus change (i.e. the trajectory of a baseball, or the frequency transitions in speech signals), or the proprioceptive tracking of limb movements. The baseline frequency used here is in the theta range, but similar mechanisms of oscillatory interference could function in other bands, such as the gamma frequency range.
As shown here, the cellular mechanism of persistent spiking (Egorov et al., 2002; Fransén et al., 2006; Tahvildari et al., 2007; Yoshida et al., 2008; Yoshida & Hasselmo, 2009) could contribute to the encoding and retrieval of episodes. Persistent spiking at stable frequencies (Egorov et al., 2002; Fransén et al., 2006) could hold states such as locations or items (Hasselmo, 2008a), allowing updating of these states by action inputs. Persistent spiking could also hold actions for updating of states (Hasselmo, 2008a). For example, persistent spiking in postsubiculum (Yoshida & Hasselmo, 2009) could hold head direction activity for updating of internal states. Functional imaging indicates a role for persistent activity in entorhinal and perirhinal cortex in the encoding of items for subsequent memory (Schon et al., 2004; Schon et al., 2005; Hasselmo & Stern, 2006).
The framework described here has advantages over models that encode trajectories or sequences as state to state associations. For example, several models replicate theta phase precession by strengthening excitatory synapses between place cells activated in a trajectory (McNaughton & Morris, 1987; Minai & Levy, 1993; Treves & Rolls, 1994; Jensen & Lisman, 1996a; b; Levy, 1996; Tsodyks et al., 1996; Wallenstein & Hasselmo, 1997; Hasselmo & Eichenbaum, 2005). These models can retrieve sequences, but they suffer from difficulties in timing the retrieval. The time scale of synaptic transmission is much faster than the time scale of initial activation of the neurons, resulting in very rapid sequence retrieval relative to behavioural time scales. In contrast, in this model, the state represented by place cells activates an action represented by persistent spiking in head direction cells (Yoshida & Hasselmo, 2009). This allows a transition in continuous phase space between discrete places, with variable timing of intervals between discrete locations. Thus, this network could provide temporally structured replay of neural activity.
The temporally structured replay shown in this model provides a potential mechanism for the neurophysiological evidence of temporally structured replay during rapid eye movement (REM) sleep (Louie & Wilson, 2001). In addition, this model generates the prediction that head direction cells during REM sleep should also show temporally structured replay of spiking activity observed previously during waking. This prediction has been tested in neural spiking in the postsubiculum during REM sleep. The model also predicts that replay of overlapping or self-crossing trajectories should show selectivity for the traversed pathway. Modulation of the magnitude of the synaptic strength WHP or the frequency of oscillations could scale the intervals between sequential activation of discrete locations, allowing replay at fast rates as seen during slow wave sleep (Skaggs & McNaughton, 1996) or during waking (Foster & Wilson, 2006; Johnson & Redish, 2006; Diba & Buzsaki, 2007).
The simulations presented here provide a mechanism for retrieval of trajectories in continuous space that could be incorporated in models of memory guided behaviour. Previous work on memory guided behaviour in a spatial alternation task (Hasselmo & Eichenbaum, 2005) used episodic retrieval of sequences of discrete states (locations), due to associations between cells representing individual places. That model used retrieval at each location, but was extended with a model using selection of memory actions such as “encode” and “retrieve” to perform memory functions only when necessary for task performance (Zilli & Hasselmo, 2008c). The selection of memory actions could guide the selection of activity to initiate retrieval as used in the present paper. This framework could allow interaction of different types of memory for solution of tasks (Zilli & Hasselmo, 2008b; a). For example, the active maintenance of a phase representation can be seen as working memory. This active maintenance of phase could cue episodic retrieval (working memory cueing episodic memory), or could hold the output from prior episodic retrieval. The mechanisms presented here could also be used to integrate phase during the trajectory from a given place to a given goal. The integrated phase could be associated with specific place cells to code the head direction to the goal location from individual places (Hasselmo and Brandon, 2008).
This process is not necessarily limited to memory encoding and retrieval. If the actions are induced by prefrontal input rather than hippocampal feedback, then these mechanisms could allow mental time travel during planning of imaginary or future trajectories through a known environment. The process could also prove useful for building up a longer term semantic representation of a trajectory, such as a the route from home to work followed by a commuter. The mechanisms of encoding for a single trajectory shown here could build up over multiple experiences of the trajectory to provide progressively higher resolution memory for the trajectory.
In this framework, lesions of the hippocampus would impair the capacity for sequential retrieval of episodic memories. By preventing retrieval of previous trajectories, this could cause impairments in the 8-arm radial maze (Bunce et al., 2004), delayed spatial alternation task (Ennaceur et al., 1996) and the Morris water maze using a new platform location on each day (Buresova et al., 1986; Steele & Morris, 1999). Loss of entorhinal grid cells or postsubiculum head direction cells would prevent continuous update of spatial location during retrieval, thereby accounting for impairments of spatial behaviour after lesions of the entorhinal cortex (Steffenach et al., 2005) and postsubiculum (Taube et al., 1992).
Research supported by Silvio O. Conte Center grant NIMH MH71702, NIMH R01 60013, NIMH MH61492, NSF SLC SBE 0354378, and the Office of Naval Research.