Search tips
Search criteria 


Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
Neuron. Author manuscript; available in PMC 2010 July 30.
Published in final edited form as:
PMCID: PMC2726251

Mind the gap: Binding experiences across space and time in the human hippocampus


A fundamental goal in memory research is to understand what class of learning problem the hippocampus is uniquely designed to solve. While much controversy surrounds the particular types of memories the hippocampus is thought to support, one hypothesized function possibly linking divergent frameworks is the capacity to bind mnemonic representations across spatial and temporal gaps in our experience. In our current functional magnetic resonance imaging (fMRI) study, we systematically controlled the extent to which a target and an event detail have to be integrated across spatiotemporal discontiguities during associative memory formation. Although the encoding task, the type of association and subsequent memory performance were held constant, engagement of the hippocampus during successful associative binding was directly modulated by increases in spatial and temporal discontiguities across episodic elements. These results suggest that a core mnemonic function of the hippocampus is to bridge representational gaps in our experience.


Episodic memories allow us to relive experiences that typically contain multiple disparate elements and unfold over extended time windows (Tulving, 1985). In order to be accessible for future retrieval, these elements have to be associatively linked into a durable memory trace. The leading neural mechanism proposed to underlie associative memory formation is synaptic long-term potentiation (LTP). Specifically, if two neurons co-activate within ~100 ms, future synaptic transmission is potentiated (Levy and Steward, 1983). This time window, however, poses a conundrum; how are representations that are discontiguous in space and time, and thus experienced across temporal gaps larger than 100 ms, made amenable to the temporal demands of LTP in order to get bound into episodic memory?

To date, a few computational models have incorporated the capacity to bridge spatiotemporal discontiguities as a core function of the hippocampal memory system (Lisman, 1999; Wallenstein et al., 1998). These models are inspired by examination of the kinds of deficits seen in animals following hippocampal damage. Namely, lesion studies in rats have shown that spatial navigation requires an intact hippocampus when performance relies on cues that are spatially distributed, but not when the same cues are clustered and overlap (Eichenbaum et al., 1990; O'Keefe and Conway, 1980). Moreover, in classical conditioning paradigms, successful acquisition of conditioned stimulus (CS) – unconditioned stimulus (US) associations has been shown to rely on the hippocampus in trace conditioning, where CS and US are separated by a temporal gap, but not in delay conditioning, where CS and US overlap in time (Clark and Squire, 1998; Solomon et al., 1986).

Direct empirical evidence for a role of the human hippocampus in bridging spatiotemporal discontiguities would not only elucidate how the disparate elements of our experiences are integrated, but would also offer key insights into the much debated functional contribution of the hippocampus to episodic memory formation (Squire et al., 2004). Competing extant models of hippocampal function posit that it supports spatial memory (Bird and Burgess, 2008), relational memory (Cohen and Eichenbaum, 1993), conjunctive learning (O'Reilly and Rudy, 2001) or recollection- rather than familiarity-based recognition (Eichenbaum et al., 2007). Although these models differ in their details, upon closer examination they appear to share the idea that the hippocampus plays a role in forming mnemonic links between elements that are initially experienced across representational gaps, such that those separate representations can later be accessed together. Thus, one principal role of the hippocampus in human memory formation might be directly related to the need to overcome representational gaps across episodic elements.

Many neuroimaging studies in humans that find hippocampal engagement during the successful binding of event details happened to present those details discontiguous in space (Jackson and Schacter, 2004; Kirwan and Stark, 2004; Staresina and Davachi, 2006, 2008) or time (Qin et al., 2007). However, none of these studies systematically varied the demand to integrate the same event detail across increasing gaps in space and time. It therefore remains unclear whether human hippocampal engagement is directly modulated by the need to integrate episodic elements across representational gaps or whether the binding of any elements, irrespective of whether they are presented overlapping or discontiguous, will engage hippocampal mechanisms.

In this functional magnetic resonance imaging (fMRI) study, we directly tested whether the successful binding of representations across different levels of spatiotemporal discontiguity will lead to functional activation changes in the human hippocampus. On each trial, a target object and an associated event detail (color) were presented in one of three ways: On combined presentations, the object was shown in a specific color, constituting an overlapping target-detail association with minimal demands on integration. On spatially discontiguous presentations, the color was presented spatially separated from the object, and on spatiotemporally discontiguous presentations, the object and the associated color were additionally separated in time (Figure 1A). For all trials, participants were instructed to perform the same encoding task: to decide if the integrated representation (the object in the respective color) was plausible in the real world. Thus, while the types of episodic elements (object and color) and decision processes (plausibility judgments) were held constant, the only difference between experimental conditions was the format of the event representation and the corresponding need to overcome representational discontiguities in order to bind the target object with the specific color. A subsequent surprise memory test was used to determine which encoding trials resulted in successful object encoding and, critically, in successful mnemonic integration of the object and the associated color (Figure 1B). If hippocampal encoding operations are critical for bridging representational gaps in experience, it is expected that the engagement of the hippocampus in successful binding will systematically increase across the three presentation conditions, despite the fact that the remembered detail per se is invariant.

Figure 1
Experimental design. (A) Example trials for each presentation condition, illustrating systematically increasing representational gaps between object and color. (B) Surprise memory test. For objects classified as “old”, memory for the associated ...


Behavioral Results

As shown in Table 1, during encoding participants distributed their plausibility ratings for the object/color combinations evenly across the three response options (low, medium and high plausibility). Assessed via a repeated-measures ANOVA, there was no difference in the proportion of Plausibility Ratings (low, medium, high) across participants (F(2,34) = 2.15, p > .12). More critically, given that we previously found a direct relationship between plausibility ratings and subsequent memory performance (Staresina et al., 2009), it was important to establish that plausibility ratings did not differ across presentation conditions. Indeed, as reflected by the absence of an interaction between Plausibility Ratings and Presentation Condition (combined, spatially discontiguous, spatiotemporally discontiguous) (F(4,68) = .23, p > .91), the distribution of plausibility ratings did not differ across the three presentation conditions.

Table 1
Plausibility ratings during encoding

Response times (Table 2), separated for trials leading to successful versus unsuccessful subsequent color memory, showed no main effect of Color Binding (successful, unsuccessful) (F(1,17) = .30, p > .58), nor a Color Binding × Presentation Condition interaction (F(2,34) = .76, p > .91). These results indicate that participants did not spend differentially more time on successful compared to unsuccessful object/color binding during discontiguous trials. Note that there was a significant main effect of Presentation Condition on response times (F(2,34) = 744.36, p < .001), due to the fact that participants were on average ~200 msec faster to indicate their plausibility rating following object onset during spatiotemporally discontiguous compared to spatially discontiguous and combined trials, respectively. Although this may not be surprising given that participants had 1 second to process the color before object onset during spatiotemporally discontiguous trials, this result emphasizes the difference in the trial structure during spatiotemporally discontiguous trials. For this reason, we avoid any direct comparisons across presentation conditions and it deserves emphasis that our main hypotheses focus on comparing the difference between successful versus unsuccessful color binding across presentation conditions and thus control for differences across presentation conditions per se.

Table 2
Response times during successful and unsuccessful color binding trials

Turning to subsequent memory performance, behavioral data show that the three presentation conditions did not differ with respect to subsequent object recognition or color memory (Table 3). Applying repeated-measures ANOVAs, we first observed no effect of the factor Presentation Condition on subsequent object recognition (F(2,34) = 2.27, p > .11). Second, no effect of Presentation Condition was seen on subsequent overall color memory, taken as the proportion of all valid encoding trials within each presentation condition (F(2,34) = .70, p > .49), or as the proportion of correctly recognized trials (‘hits’) only (F(2,34) = 2.29, p > .11). For objects correctly recognized, color memory was well above chance (25%) in all three presentation conditions (all ts(17) > 14.02, p < .001). Finally, no effect of Presentation Condition was seen on subsequent high confidence color memory, taken as the proportion of all valid encoding trials within each presentation condition (F(2,34) = 1.78, p > .17) or as the proportion of correct color memory trials only (F(2,34) = 1.51, p > .23). These data suggest (i) that the success of both object encoding and color binding was unaffected by the presentation condition, and (ii) that the resulting color memory traces were qualitatively similar across the presentation conditions as assessed via confidence ratings.

Table 3
Subsequent memory performance

FMRI Results

The critical factors in this study were Presentation Condition, i.e., the representational format in which a target object and an associated color were presented (combined, spatially discontiguous or spatiotemporally discontiguous), and Color Binding, i.e., the success or failure of incorporating these two elements into an episodic memory trace (successful or unsuccessful). As outlined in the Introduction, we hypothesized that increasing representational gaps between a target and an associated event detail would result in increasing engagement of hippocampal encoding operations in order to bind these elements into an episodic memory trace. In other words, we predicted to see an interaction of Presentation Condition and Color Binding in hippocampal activation during encoding, such that stronger color binding effects would be seen for discontiguous object/color presentations.

Omnibus F-test

As a first step, to approach our fMRI data in an entirely unbiased manner, we identified medial temporal lobe (MTL) regions that were sensitive to any effect of our experimental factors. This was done via a voxelwise omnibus F-test in the context of a repeated-measures ANOVA, entering participant-specific beta weights for each of the six conditions (Presentation Condition (3) × Color Binding (2)) as dependent measures. For voxels whose F statistic surpassed the threshold of p < .001 (uncorrected, with a minimum of 5 contiguous voxels), follow-up analyses were then conducted to examine the underlying effects. As shown in Figure 2A, the only MTL cluster that emerged from the omnibus F-test was located in the right anterior hippocampus (peak xyz = 27, -9, -21). Follow-up analysis on participant-specific beta weights averaged across the hippocampal cluster revealed a significant main effect of Color Binding (F(1,17) = 9.89, p < .01), and most critically, a significant Color Binding × Presentation Condition interaction (F(2,34) = 3.87, p < .05). As illustrated in Figure 2C, this interaction was due to the color binding effect (i.e., the difference in encoding activation between successful minus unsuccessful color binding trials) showing a stepwise increase in magnitude from combined to spatially discontiguous to spatiotemporally discontiguous trials. Further analysis of the color binding effect across presentation conditions revealed that the effect did not differ from zero (assessed via a two-tailed, one-sample t-test) for combined trials (t(17) = .12, p > .89), but was significantly greater than zero for spatially discontiguous trials (t(17) = 2.49, p < .05) and - to a greater extent - significantly greater than zero for spatiotemporally discontiguous trials (t(17) = 3.98, p < .005). This stepwise increase was formally confirmed by a significant linear term underlying the Color Binding × Presentation Condition interaction (F(1,17) = 10.96, p < .005).

Figure 2
Object/color binding across representational gaps in the hippocampus. (A) Medial temporal lobe clusters emerging form an unbiased whole-brain omnibus F-test, overlaid on the mean anatomical image across subjects. (B) Differential color binding effects ...

Briefly summarized, the results of our omnibus F-test revealed a cluster in the right anterior hippocampus that showed no effect of color binding for combined object/color presentations, but an increasingly significant color binding effect as the representational discontiguity between object and color increased. However, a critical question is whether this pattern is indeed driven by the increasing demand to overcome representational discontiguities in the service of episodic binding, or whether it can be accounted for by enhanced overall difficulty/working memory demands during discontiguous trials. To address the possibility that the differential hippocampal binding effects reflect differential levels of overall effort during discontiguous trials, we first isolated, via a parametric analysis (Buchel et al., 1998), regions in which trial-by-trial changes in BOLD signal covary with response times for the plausibility ratings. To be explicit, this analysis identified regions whose engagement was directly modulated by response time, in the sense that trials for which the plausibility rating is given relatively late induce enhanced activation in those regions, compared to trials for which the plausibility rating is given relatively early. Given the different timing parameters for the spatiotemporally discontiguous presentation trials (see behavioral results above), this was first done separately for each presentation condition. The resulting statistical parametric maps were then collapsed across presentation conditions, yielding a statistical parametric map that highlights regions that are sensitive to trial-by-trial variations in response latency across all conditions in our experimental paradigm. As shown in Supplemental Figure 1, this analysis revealed a network of brain regions, which included, among others, frontal and parietal regions consistently related to working memory processes (Corbetta and Shulman, 2002; D'Esposito et al., 1998; D'Esposito et al., 1995; Ravizza et al., 2004; Smith and Jonides, 1999; Wager and Smith, 2003). This result indicates that we had sufficient power in our paradigm to detect regions directly modulated by overall time on task/difficulty/working memory demands as captured by response latencies. This parametric modulation map was then used as a mask to exclude regions from the above omnibus F-test that show modulations by overall effort in our paradigm. Critically, the right hippocampal cluster survived this exclusive mask, even when using a very liberal threshold of p < .1 (uncorrected, no minimum cluster size required). This result strongly suggests that the differential hippocampal binding effects we observed as a function of representational discontiguity do not reflect mere increases in overall difficulty/working memory demands across presentation conditions.

Lastly, we wanted to assess whether other brain regions showed a pattern of encoding activation similar to the one we found in the hippocampus. Thus, we extended the omnibus F-test to the whole brain, masking out regions that (i) showed a modulation by response latency (using the parametric mask described above) and (ii) would exhibit an inverse color binding effect, i.e., enhanced activation for unsuccessful relative to successful color binding (see Experimental Procedures). The only cluster that resulted from this analysis (in addition to the right hippocampal cluster) was located in the left occipital cortex (peak xyz = -9, -93, -6). However, the follow-up analysis revealed only a main effect of Presentation Condition (F(2,34) = 9.69, p < .001), but no effect of Color Binding (F(1,17) = .46, p > .49), nor a Presentation Condition × Color Binding interaction (F(2,34) = 2.00, p > .14). Thus, the right hippocampus was the only brain region whose activation was not sensitive to variations in response time and in which color binding effects systematically increased with increasing representational discontiguity.

Targeted contrasts

A critical aspect of the above results is that the hippocampus did not show a color binding effect for combined trials (i.e., there was no difference in encoding activation between successful and unsuccessful color binding trials), but an increasingly significant color binding effect for spatially and spatiotemporally discontiguous trials, respectively. However, the omnibus F-test that revealed the right hippocampal cluster is a fairly conservative statistical assessment, and, theoretically, other hippocampal clusters might in fact show a color binding effect for combined trials in a more targeted analysis. Thus, we next assessed possible color binding effects in the MTL separately for each presentation condition via directed contrasts (see Experimental Procedures). Interestingly, no MTL cluster emerged from this analysis for combined trials, even after relaxing the statistical threshold to p < .05 (uncorrected), despite the fact that significant clusters emerged, among others, in prefrontal and ventral temporal regions (Supplemental Figure 2A). Conversely, hippocampal clusters that overlapped with the hippocampal region reported above emerged in the directed contrast for spatiotemporally discontiguous trials at p < .001 (Supplemental Figure 2C), and for spatially discontiguous trials at p < .008 (Supplemental Figure 2B).

Finally, to formally confirm, via a stringent one-step procedure, the reliability of hippocampal color binding effects for discontiguous but not for already integrated (combined) object/color presentations, we employed a conjunction analysis (Nichols et al., 2005). That is, we applied a conjunction of color binding effects for both spatially and spatiotemporally discontiguous trials (both directed contrasts thresholded at p < .0316 to result in a conjoint probability of p < .001) and excluded regions that would show a color binding effect for combined trials at the liberal threshold of p < .1. This analysis again revealed a right hippocampal cluster overlapping with the one reported above (peak xyz = 27, -12, -21), together with a slightly smaller left hippocampal cluster (peak xyz = -30, -15, -12). Intriguingly, when applying the same parametric response latency mask used above to this conjunction analysis, the bilateral hippocampal clusters were the only regions to show a conjoint color binding effect for discontiguous- but not combined trials across the whole brain (Supplemental Figure 3). Inclusion of the left hippocampal cluster into the same type of ANOVA conducted above still produced the same pattern of results, revealing a main effect of Color Binding (F(1,17) = 21.56, p < .001) as well as a Presentation Condition × Color Binding interaction (F(2,34) = 4.04, p < .05), the latter again showing a significant linear term (F(2,34) = 5.64, p < .05).

In sum, results from our complementary targeted contrast approach bolstered our results by showing increasingly robust hippocampal color binding effects for spatially and spatiotemporally discontiguous trials, respectively, but no hippocampal color binding effect for combined trials even at markedly relaxed statistical thresholds.

Specificity to associative binding

One remaining question is whether the hippocampus shows differential encoding effects across presentation conditions for successful memory formation in general, or, as we hypothesized, whether this pattern is specific to the actual associative binding of the discontiguous episodic elements. To address this question, we examined whether the factor Presentation Condition affected successful non-associative object encoding, i.e., successful encoding of the object irrespective of the associative binding of the color, in a similar way. In order to assess potentially differential object encoding effects, we separately modeled, for each presentation condition, trials for which the object was later misclassified as new (subsequent misses) and trials for which the object was later correctly classified as old (subsequent hits, collapsed across successful and unsuccessful color binding). Note that two participants had to be excluded from this analysis for providing an insufficient number of miss trials for each presentation condition. The corresponding beta weights were then extracted from the hippocampal cluster emerging from the omnibus F-test reported above and subjected to a repeated-measures ANOVA, including the factors Object Encoding (successful, unsuccessful) and Presentation Condition (combined, spatially discontiguous and spatiotemporally discontiguous). Critically, we observed no main effect of Object Encoding (F(1,15) = .70, p > .41), nor an Object Encoding × Presentation Condition interaction (F(2,30) = .59, p > .55). Thus, the pattern of differential hippocampal encoding effects as a function of representational discontiguity was specific to the successful binding of the associated color. It should also be noted that the same pattern of results was obtained when separating subsequent hits into object only trials (subsequent hits, unsuccessful color binding) and object and color trials (subsequent hits, successful color binding) trials. Comparing subsequent misses to either of these trials resulted in no main effect of Object Encoding (both Fs(1,15) < 2.30, p > .14), nor an Object Encoding × Presentation Condition interaction (both Fs(2,30) < 1.89, p > .16).

Finally, to ensure that the lack of a (differential) object encoding effect in the hippocampus was not the result of poor overall power to detect such an effect in our design, we assessed whether other MTL regions showed object encoding effects. To this end, we applied a directed contrast to reveal regions that show enhanced activation for successful object encoding (subsequent hits, collapsed across successful and unsuccessful color binding) compared to unsuccessful object encoding (subsequent misses), irrespective of presentation condition. The only MTL clusters emerging from this analysis were located in MTL cortex, including a cluster in perirhinal cortex (PrC) both in the left (peak xyz = -27, -6, -39) and the right (peak xyz = 30, -3, -42) hemisphere (Figure 3A), as well as a slightly more posterior cluster in the left hemisphere extending towards entorhinal cortex (peak xyz = -21, -9, -27). We limit our subsequent analysis to the two clusters located within PrC, but it should be mentioned that the pattern of results remains unchanged when also including the more posterior left cluster. Critically, despite (i) showing a strong main effect of Object Encoding (F(1,15) = 16.65, p < .001) and (ii) the effect sizes for successful object encoding being significantly greater than zero for all three presentation conditions (all ts > 2.63, p < .05, collapsed across hemispheres, Figure 3B), there was no Object Encoding × Presentation Condition or Object Encoding × Presentation Condition × Hemisphere interaction in PrC (both Fs(2,30) < .31, p > .73). Since this study is targeted at the role of the hippocampus in memory formation, we defer a more exhaustive presentation and discussion of the pattern in PrC to the Supplemental Material (Supplemental Figure 4). Importantly, the results from the non-associative object encoding analysis suggest that the pattern of differential hippocampal encoding effects as a function of representational discontiguity was indeed specific to associative object/color binding and was not driven by a globally enhanced involvement of the hippocampus in episodic encoding during discontiguous trials.

Figure 3
Object encoding in the perirhinal cortex (PrC). (A) Medial temporal lobe clusters revealing global object encoding effects across presentation conditions. (B) The effect of successful object encoding (arbitrary units; B successful – B unsuccessful) ...


Our current data suggest that a core mnemonic function of the hippocampus is the capacity to bridge representational gaps between elements of our experiences. We presented an object and an associated color in three different ways (Figure 1A), systematically controlling the level of spatiotemporal discontiguity between these elements (combined, spatially discontiguous, spatiotemporally discontiguous). As the target object and the associated color were presented across increasing gaps in space and time (from combined to spatially discontiguous to spatiotemporally discontiguous), hippocampal engagement for successfully binding these elements into an associative memory trace likewise increased (Figure 2B). Importantly, this differential binding effect was observed in the same episodic memory paradigm, where the encoding task (plausibility judgments), the type of association (object/color binding) and subsequent memory (high confidence color memory) were held constant.

An important potential caveat is that above and beyond the level of representational discontiguity (increasing gaps across space and time), our presentation conditions may differ along other dimensions such as mere task difficulty. Differential hippocampal engagement for successful color binding might thus reflect increasing levels of attention or working memory efforts for discontiguous trials. However, our data argue against this possibility in several ways. First, the critical measure in this study was the magnitude of the color binding effect across presentation conditions, i.e., the difference in encoding activation during successful compared to unsuccessful color binding. In other words, this measure is derived from the color binding effect within each presentation condition and thus controls for global effects such as different levels of difficulty or different timing parameters across presentation conditions. Second, behavioral data showed not only that subsequent memory performance (correct object and color memory) did not differ across presentation conditions (Table 3), but there was no difference in the duration participants spent on successful compared to unsuccessful color binding trials across presentation conditions (Table 2). Third, we restricted our fMRI analysis, via an exclusive masking procedure, to regions that were insensitive to trial-by-trial variations in response latencies for the plausibility judgments. Although this analysis identified a network of fronto-parietal regions (Supplemental Figure 1) previously shown to track working memory load and attention demands (Corbetta and Shulman, 2002; D'Esposito et al., 1998; D'Esposito et al., 1995; Ravizza et al., 2004; Smith and Jonides, 1999; Wager and Smith, 2003), the hippocampus did not emerge in this analysis even at a strongly reduced statistical threshold. This is consistent with previous work showing that hippocampal engagement tracks success rather than effort during both episodic encoding (Reber et al., 2002) and retrieval (Dobbins et al., 2003). Finally, we found that differential encoding effects in the hippocampus across presentation conditions were specific to the binding of the target object and the associated color and were not seen for (non-associative) object encoding. This again suggests that the pattern we observed in the hippocampus was not due to global features of discontiguous trials such as potentially different working memory loads. In order to ensure that the lack of (differential) object encoding effects in the hippocampus was not due to reduced power for detecting such effects in our paradigm, we tested whether other MTL regions may show object encoding effects. Indeed, robust effects of successful object encoding - unaffected by presentation condition - were observed in the perirhinal cortex (Figure 3), consistent with a role of this region in object- or item memory (Aggleton and Brown, 2005; Brown and Aggleton, 2001; Davachi, 2006; Eichenbaum et al., 2007; Meunier et al., 1993; Murray and Bussey, 1999) (see Supplemental Material for more exhaustive discussion of PrC effects in our study). In sum, we suggest that increasing engagement of the hippocampus during successful color binding across presentation conditions was driven by the increasing demand to integrate the target object and the associated color across representational gaps.

How does the proposed role of the hippocampus in mnemonically overcoming representational discontiguities relate to prevalent models of hippocampal function? For example, the relational memory framework put forth by Cohen and Eichenbaum (Cohen and Eichenbaum, 1993) highlights specific characteristics of memory traces mediated by the hippocampus, particularly their flexibility, i.e., access to a memory trace through multiple cues or by virtue of inference (Eichenbaum, 2004; Eichenbaum et al., 1992). Similarly, recent neuroimaging studies in humans showing that hippocampal engagement specifically predicts later memory for contextual/source information (Davachi et al., 2003; Kensinger and Schacter, 2006; Ranganath et al., 2004) have lead to the notion that the hippocampal binding mechanisms support recollection- rather than familiarity-based recognition (Diana et al., 2007; Eichenbaum et al., 2007), a long-held distinction regarding different phenomenological qualities of episodic memory traces (Jacoby, 1991; Mandler, 1980). Our current data, however, rather than emphasizing characteristics of the mnemonic output mediated by the hippocampus (e.g., as being flexible or as mediating recollection), show how characteristics of the informational input and the ensuing demands on integration directly modulate hippocampal engagement during successful memory formation. In particular, holding the quality of the output, i.e., the resulting memory trace (as assessed via performance scores and confidence ratings), constant, we show that engagement of the hippocampus in associative encoding is contingent on the need to overcome representational gaps across the episodic elements. When speaking of characteristics of the encoding input, we want to emphasize that we do not suggest that hippocampal engagement is restricted to a specific stimulus domain, e.g. spatial versus non-spatial stimuli (Bird and Burgess, 2008). Not only have we provided recent evidence for domain-generality of hippocampal encoding operations (Staresina and Davachi, 2008), but all three presentation conditions in our current paradigm consisted of the same stimulus types, namely an object and an associated color. Instead, we refer to the characteristics of the encoding input with respect to representational discontiguity, i.e., the format of the to-be-encoded event details and the ensuing extent to which these details have to be integrated across gaps in our experience. We would argue that this idea is consistent with the frameworks described above, but emphasizes the characteristics of the encoding input rather than those of the mnemonic product.

Focusing on the need to integrate disparate elements of an experience during memory formation, the current findings on the function of the hippocampus contrast with those from other recent animal work and neuroimaging in humans that have tended to focus on integration across already formed memories, potentially mediated by retrieval. Specifically, the hippocampus has been previously implicated in sequence disambiguation of already learned sequences (Agster et al., 2002; Fortin et al., 2002; Kumaran and Maguire, 2006) and in making inferences across memories with overlapping elements (Heckers et al., 2004; Preston et al., 2004; Shohamy and Wagner, 2008). However, these experiments highlight the capability of hippocampal processing in pattern separation in order to keep similar sequences distinct and in the ability to treat overlapping elements as potential links across memories. Thus, the present work is complementary to these findings in that our results focus on how episodic elements are experienced during the initial encounter, and not on the flexibility of memory per se once those memories are already formed.

Finally, if hippocampal involvement during successful memory formation is modulated by the need to integrate information across representational gaps, a strong prediction of these data is that the role of the hippocampus in associative memory formation might be diminished when the criterial information is experienced in an integrated fashion. Indeed, not only did we not observe hippocampal color binding effects in the combined presentation condition in our current study (Figure 2B, Supplemental Figure 2), but a survey of other recent fMRI studies in humans suggests that when episodic elements do not have to be integrated across representational gaps, subsequent memory for those elements may not elicit enhanced hippocampal engagement during encoding (Cansino et al., 2002; Eldridge et al., 2005; Tendolkar et al., 2007). Of course, caution is warranted in interpreting null effects and given that the involvement of a region or the putative lack thereof is always a consequence of statistical thresholding in fMRI data, our results do not allow for strong conclusions on whether or not the hippocampus is ultimately needed for binding episodic elements that are experienced in an integrated fashion. Instead, we can only assert that the hippocampus is differentially more engaged during successful associative encoding of representationally discontiguous elements. However, stronger inference on the necessity of the hippocampus for binding integrated versus discontiguous elements have been derived from lesion studies in animal models and from neuropsychological data in humans. For example, as previously mentioned, lesion studies in rats have shown that the hippocampus is needed for spatial navigation when performance can be guided by a spatial distribution of contextual cues, but not - analogous to the combined object/color presentation in our current paradigm - by the same cues when they are clustered and overlap (Eichenbaum et al., 1990; O'Keefe and Conway, 1980). Moreover, the critical role of the hippocampus in trace conditioning has recently been shown to be diminished when the conditioned stimulus (CS) is re-presented at the onset of the unconditioned stimulus (US) and their temporal contiguity is thus restored (Bangasser et al., 2006). Finally, neuropsychological studies in humans show that patients with hippocampal damage are, relative to controls, dramatically impaired in associative word-word learning when the two words are unrelated or separated by a sentence frame, but not when the same two words are shown as a compound representation (Giovanello et al., 2006; Quamme et al., 2007). Together, these data from animal and human studies suggest that while the hippocampus is needed to mnemonically overcome discontiguities in space or time, its role may be diminished when the same information is presented in an integrated fashion.

In sum, using a novel experimental paradigm that controls for potentially confounding effects of stimulus domain, task demands and subsequent memory, we here show that the contribution of the human hippocampus to associative memory formation is systematically modulated by the level of spatiotemporal discontiguity across episodic elements.

Experimental Procedures


Eighteen (10 female) right-handed native English speakers with normal or corrected-to-normal vision participated in the experiment (mean age: 24 years, range: 18-34). Informed consent was obtained in a manner approved by the institutional review board at New York University and participants were paid for their participation.


The stimuli consisted of 450 grayscale object images, 300 of which served as study items and 150 of which served as lures during a recognition memory test. The 300 study items were divided into three sets of 100 items per presentation condition (see below) that were evenly assigned to the colors blue, green, red and brown (25 trials per color). The stimulus material was counterbalanced so that across participants, every object was shown with every color during each presentation condition and was used both as a study item and as a lure for the subsequent recognition memory test.


For each scanned encoding trial, participants were presented with an object and an associated color. Participants were instructed to imagine the given object in the associated color in real life/nature and to rate the plausibility of that particular object/color combination, with the response options being “plausible high”, “plausible medium” and “plausible low”. Reponses were given via a magnet-compatible button box placed under the participant's left hand. Importantly, participants were instructed to press a separate button in case they could not perform the task on a given trial, be it because they did not recognize the object or could not imagine the object in the given color. Those trials, as well as trials for which no response was given by the end of the trial, were excluded from all further analyses.

The critical manipulation in this experiment was the way in which the target object and the associated color were visually presented. On combined presentations, the object was presented in the given color and was surrounded by a transparent frame. On spatially discontiguous presentations, the object was presented in grayscale and was surrounded by a colored frame. Finally, on spatiotemporally discontiguous presentations, the trial started with a 500 msec presentation of the color frame only, followed by a 500 msec delay interval (blank screen), followed again by a 3000 msec presentation of a grayscale object surrounded by a transparent frame.

Following the encoding session, participants were given an unscanned and self-paced surprise recognition memory test, consisting of all 300 previously presented objects as well 150 novel objects (lures). First, participants were instructed to indicate whether the object was old (presented during the encoding session) or new (not presented during the encoding session). For objects endorsed as old, participants were then asked to indicate the color with which the object was associated during encoding and to rate their confidence in their color response (high, low, guess). This testing protocol was instrumental to back-sort the scanned encoding trials not only based on successful and unsuccessful object encoding, but also and more importantly based on failure and success of incorporating the color into an episodic memory trace.

MRI Scanning and Data Analysis

The scanned encoding portion of the experiment was divided into two runs. 400 volumes were acquired in each run (TR = 2000 msec, TE = 30 msec, 35 slices oriented perpendicular to the hippocampal axis, 3 × 3 × 3 mm voxel size, 0.6 mm inter-slice gap). Encoding trials were intermixed with an active, sensorimotor baseline task (‘arrows-task’ (Stark and Squire, 2001)). Arrows that randomly pointed to the left or to the right for 1 second were repeatedly presented for the length of a baseline trial (2–12 seconds), and subjects had to press the left middle finger key if the arrow pointed to the left and the left index finger key if it pointed to the right. The sequence of encoding trials of each presentation condition (combined, spatially discontiguous and spatiotemporally discontiguous presentations) and of variable duration baseline trials was pseudorandom and optimized for rapid event-related fMRI (Dale, 1999).

Statistical analysis of fMRI data was performed using the general linear model (GLM) implemented in SPM2 (Wellcome Department of Cognitive Neurology, London). For each presentation condition, encoding trials were first separated into successful color binding trials (objects subsequently recognized and eliciting high confidence correct color memory responses) and unsuccessful color binding trials (objects subsequently recognized and eliciting guess or low confidence incorrect color memory responses). Successful and unsuccessful object/color encoding trials were modeled as boxcar events spanning the entire trial period. It should be noted that we also modeled, in a separate analysis, the spatiotemporally discontiguous trials as 3 sec events spanning only the trial period after object onset. However, as the critical statistical comparisons first derive the magnitude of successful encoding effects (successful minus unsuccessful) within each presentation condition, differences in the length of the modeled trial period should not affect the subsequent comparisons of effects across presentation conditions. Indeed, all results showed exactly the same pattern irrespective of modeling spatiotemporally discontiguous trials as 3- or 4 sec long events, and we chose to report only the data from modeling all presentation conditions with equal (4 sec) duration.

For each of the six conditions of interest (successful and unsuccessful color binding for each of the three presentation conditions), the corresponding boxcar events were then convolved with a canonical hemodynamic response function along with its first-order temporal derivative and entered as regressors into a fixed-effects GLM, together with nuisance regressors modeling session means and scanner drift. Parameter estimates (beta weights) for each condition of interest were derived for each participant and carried forward to a second-level group analysis. Here, individual participants' beta weights for the six conditions of interest were entered into a repeated-measures ANOVA, corrected for non-sphericity and for correlated repeated measures. The omnibus F-test as well as the subsequent directed contrasts described above were conducted within this whole-brain ANOVA model.

For the directed color binding contrasts, a weight of +1 was assigned to successful color binding trials, and a weight of -1 to unsuccessful color binding trials. This was done separately for each presentation condition, while successful and unsuccessful color binding trials from the remaining two presentation conditions were excluded from the contrast analysis by assigning weights of 0. To reveal regions involved during (non-associative) object encoding irrespective of presentation condition, a weight of +1 was assigned to successful object encoding trials, and a weight of -1 to unsuccessful object encoding trials across all presentation conditions. To mask out regions from the omnibus F-test that would show an inverse memory effect, a directed contrast was applied that assigned a weight of +1 to unsuccessful color binding trials in each presentation condition, and a corresponding weight of -1 to successful color binding trials. The resulting statistical map was again liberally thresholded at p < .1 (uncorrected, no minimum cluster size required). For the parametric response latency analysis described above, the parametric effect across presentation conditions was first derived for each participant, and the resulting maps were carried forward to the second level via a one-sample t-test comparing the group effect size to zero.

Unless otherwise noted, statistical significance for all mapwise analyses within the SPM approach was assessed at five contiguous voxels exceeding an uncorrected threshold of p < .001. Voxel coordinates are reported in Montreal Neurological Institute (MNI) space. Although all statistical analyses were conducted on subject-specific beta weights, we additionally present the deconvolved BOLD time courses for each condition to provide a complementary illustration of the data. Time course data were extracted using the MarsBaR toolbox (Brett et al., 2002).

Supplementary Material



This work was funded by NIMH MH074692 to L.D. We thank James C. Gray for help with data acquisition and analysis.


Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.


  • Aggleton JP, Brown MW. Contrasting hippocampal and perirhinal cortex function using immediate early gene imaging. Q J Exp Psychol B. 2005;58:218–233. [PubMed]
  • Agster KL, Fortin NJ, Eichenbaum H. The hippocampus and disambiguation of overlapping sequences. J Neurosci. 2002;22:5760–5768. [PMC free article] [PubMed]
  • Bangasser DA, Waxler DE, Santollo J, Shors TJ. Trace conditioning and the hippocampus: the importance of contiguity. J Neurosci. 2006;26:8702–8706. [PMC free article] [PubMed]
  • Bird CM, Burgess N. The hippocampus and memory: insights from spatial processing. Nat Rev Neurosci. 2008;9:182–194. [PubMed]
  • Brett M, Anton JL, Valabregue R, Poline JB. Region of interest analysis using an SPM toolbox. Neuroimage. 2002;16:1140–1141.
  • Brown MW, Aggleton JP. Recognition memory: what are the roles of the perirhinal cortex and hippocampus? Nat Rev Neurosci. 2001;2:51–61. [PubMed]
  • Buchel C, Holmes AP, Rees G, Friston KJ. Characterizing stimulus-response functions using nonlinear regressors in parametric fMRI experiments. Neuroimage. 1998;8:140–148. [PubMed]
  • Cansino S, Maquet P, Dolan RJ, Rugg MD. Brain activity underlying encoding and retrieval of source memory. Cereb Cortex. 2002;12:1048–1056. [PubMed]
  • Clark RE, Squire LR. Classical conditioning and brain systems: the role of awareness. Science. 1998;280:77–81. [PubMed]
  • Cohen NJ, Eichenbaum HE. Memory, Amnesia, and the Hippocampal System. Cambridge, MA: MIT Press; 1993.
  • Corbetta M, Shulman GL. Control of goal-directed and stimulus-driven attention in the brain. Nat Rev Neurosci. 2002;3:201–215. [PubMed]
  • D'Esposito M, Aguirre GK, Zarahn E, Ballard D, Shin RK, Lease J. Functional MRI studies of spatial and nonspatial working memory. Brain Res Cogn Brain Res. 1998;7:1–13. [PubMed]
  • D'Esposito M, Detre JA, Alsop DC, Shin RK, Atlas S, Grossman M. The neural basis of the central executive system of working memory. Nature. 1995;378:279–281. [PubMed]
  • Dale AM. Optimal experimental design for event-related fMRI. Human Brain Mapping. 1999;8:109–114. [PubMed]
  • Davachi L. Item, context and relational episodic encoding in humans. Curr Opin Neurobiol. 2006;16:693–700. [PubMed]
  • Davachi L, Mitchell JP, Wagner AD. Multiple routes to memory: distinct medial temporal lobe processes build item and source memories. Proc Natl Acad Sci U S A. 2003;100:2157–2162. [PubMed]
  • Diana RA, Yonelinas AP, Ranganath C. Imaging recollection and familiarity in the medial temporal lobe: a three-component model. Trends Cogn Sci. 2007;11:379–386. [PubMed]
  • Dobbins IG, Rice HJ, Wagner AD, Schacter DL. Memory orientation and success: separable neurocognitive components underlying episodic recognition. Neuropsychologia. 2003;41:318–333. [PubMed]
  • Eichenbaum H. Hippocampus: cognitive processes and neural representations that underlie declarative memory. Neuron. 2004;44:109–120. [PubMed]
  • Eichenbaum H, Otto T, Cohen NJ. The hippocampus--what does it do? Behav Neural Biol. 1992;57:2–36. [PubMed]
  • Eichenbaum H, Stewart C, Morris RG. Hippocampal representation in place learning. Journal of Neuroscience. 1990;10:3531–3542. [PubMed]
  • Eichenbaum H, Yonelinas AP, Ranganath C. The medial temporal lobe and recognition memory. Annu Rev Neurosci. 2007;30:123–152. [PMC free article] [PubMed]
  • Eldridge LL, Engel SA, Zeineh MM, Bookheimer SY, Knowlton BJ. A dissociation of encoding and retrieval processes in the human hippocampus. J Neurosci. 2005;25:3280–3286. [PubMed]
  • Fortin NJ, Agster KL, Eichenbaum HB. Critical role of the hippocampus in memory for sequences of events. Nat Neurosci. 2002;5:458–462. [PMC free article] [PubMed]
  • Giovanello KS, Keane MM, Verfaellie M. The contribution of familiarity to associative memory in amnesia. Neuropsychologia. 2006;44:1859–1865. [PMC free article] [PubMed]
  • Heckers S, Zalesak M, Weiss AP, Ditman T, Titone D. Hippocampal activation during transitive inference in humans. Hippocampus. 2004;14:153–162. [PubMed]
  • Jackson O, 3rd, Schacter DL. Encoding activity in anterior medial temporal lobe supports subsequent associative recognition. Neuroimage. 2004;21:456–462. [PubMed]
  • Jacoby LL. A Process Dissociation Framework: Separating Automatic from Intentional Uses of Memory. Journal of Memory and Language. 1991;30:513–541.
  • Kensinger EA, Schacter DL. Amygdala activity is associated with the successful encoding of item, but not source, information for positive and negative stimuli. J Neurosci. 2006;26:2564–2570. [PubMed]
  • Kirwan CB, Stark CE. Medial temporal lobe activation during encoding and retrieval of novel face-name pairs. Hippocampus. 2004;14:919–930. [PMC free article] [PubMed]
  • Kumaran D, Maguire EA. The dynamics of hippocampal activation during encoding of overlapping sequences. Neuron. 2006;49:617–629. [PubMed]
  • Levy WB, Steward O. Temporal contiguity requirements for long-term associative potentiation/depression in the hippocampus. Neuroscience. 1983;8:791–797. [PubMed]
  • Lisman JE. Relating hippocampal circuitry to function: recall of memory sequences by reciprocal dentate-CA3 interactions. Neuron. 1999;22:233–242. [PubMed]
  • Mandler G. Recognizing: The judgment of previous occurrence. Psychological Review. 1980;87:252–271.
  • Meunier M, Bachevalier J, Mishkin M, Murray EA. Effects on visual recognition of combined and separate ablations of the entorhinal and perirhinal cortex in rhesus monkeys. J Neurosci. 1993;13:5418–5432. [PubMed]
  • Murray EA, Bussey TJ. Perceptual-mnemonic functions of the perirhinal cortex. Trends Cogn Sci. 1999;3:142–151. [PubMed]
  • Nichols T, Brett M, Andersson J, Wager T, Poline JB. Valid conjunction inference with the minimum statistic. Neuroimage. 2005;25:653–660. [PubMed]
  • O'Reilly RC, Rudy JW. Conjunctive representations in learning and memory: principles of cortical and hippocampal function. Psychol Rev. 2001;108:311–345. [PubMed]
  • O'Keefe J, Conway DH. On the trail of the hippocampal engram. Physiological Psychology. 1980;8:229–238.
  • Preston AR, Shrager Y, Dudukovic NM, Gabrieli JD. Hippocampal contribution to the novel use of relational information in declarative memory. Hippocampus. 2004;14:148–152. [PubMed]
  • Qin S, Piekema C, Petersson KM, Han B, Luo J, Fernandez G. Probing the transformation of discontinuous associations into episodic memory: an event-related fMRI study. Neuroimage. 2007;38:212–222. [PubMed]
  • Quamme JR, Yonelinas AP, Norman KA. Effect of unitization on associative recognition in amnesia. Hippocampus. 2007;17:192–200. [PubMed]
  • Ranganath C, Yonelinas AP, Cohen MX, Dy CJ, Tom SM, D'Esposito M. Dissociable correlates of recollection and familiarity within the medial temporal lobes. Neuropsychologia. 2004;42:2–13. [PubMed]
  • Ravizza SM, Delgado MR, Chein JM, Becker JT, Fiez JA. Functional dissociations within the inferior parietal cortex in verbal working memory. Neuroimage. 2004;22:562–573. [PubMed]
  • Reber PJ, Siwiec RM, Gitelman DR, Parrish TB, Mesulam MM, Paller KA. Neural correlates of successful encoding identified using functional magnetic resonance imaging. J Neurosci. 2002;22:9541–9548. [PubMed]
  • Shohamy D, Wagner AD. Integrating memories in the human brain: hippocampal-midbrain encoding of overlapping events. Neuron. 2008;60:378–389. [PMC free article] [PubMed]
  • Smith EE, Jonides J. Storage and executive processes in the frontal lobes. Science. 1999;283:1657–1661. [PubMed]
  • Solomon PR, Vander Schaaf ER, Thompson RF, Weisz DJ. Hippocampus and trace conditioning of the rabbit's classically conditioned nictitating membrane response. Behav Neurosci. 1986;100:729–744. [PubMed]
  • Squire LR, Stark CE, Clark RE. The medial temporal lobe. Annu Rev Neurosci. 2004;27:279–306. [PubMed]
  • Staresina BP, Davachi L. Differential encoding mechanisms for subsequent associative recognition and free recall. J Neurosci. 2006;26:9162–9172. [PubMed]
  • Staresina BP, Davachi L. Selective and shared contributions of the hippocampus and perirhinal cortex to episodic item and associative encoding. J Cogn Neurosci. 2008;20:1478–1489. [PMC free article] [PubMed]
  • Staresina BP, Gray JC, Davachi L. Event congruency enhances episodic memory encoding through semantic elaboration and relational binding. Cereb Cortex. 2009;19:1198–1207. [PMC free article] [PubMed]
  • Stark CE, Squire LR. When zero is not zero: the problem of ambiguous baseline conditions in fMRI. Proc Natl Acad Sci U S A. 2001;98:12760–12766. [PubMed]
  • Tendolkar I, Arnold J, Petersson KM, Weis S, Anke BD, van Eijndhoven P, Buitelaar J, Fernandez G. Probing the neural correlates of associative memory formation: a parametrically analyzed event-related functional MRI study. Brain Res. 2007;1142:159–168. [PubMed]
  • Tulving E. Memory and consciousness. Canadian Psychology. 1985;26:1–12.
  • Wager TD, Smith EE. Neuroimaging studies of working memory: a meta-analysis. Cogn Affect Behav Neurosci. 2003;3:255–274. [PubMed]
  • Wallenstein GV, Eichenbaum H, Hasselmo ME. The hippocampus as an associator of discontiguous events. Trends Neurosci. 1998;21:317–323. [PubMed]