|Home | About | Journals | Submit | Contact Us | Français|
Spatial attention is known to gate entry into visual short-term memory, and some evidence suggests that spatial signals may also play a role in binding features or protecting object representations during memory maintenance. To examine the persistence of spatial signals during object short-term memory, the activity of neurons in the Frontal Eye Field (FEF) of macaque monkeys was recorded during an object-based delayed match-to-sample (DMS) task. In this task monkeys were trained to remember an object image over a brief delay, irrespective of the locations of the sample or target presentation. FEF neurons exhibited visual, delay, and target period activity, including selectivity for sample location and target location. Delay period activity represented the sample location throughout the delay, despite the irrelevance of spatial information for successful task completion. Furthermore, neurons continued to encode sample position in a variant of the task in which the matching stimulus never appeared in their response field, confirming that FEF maintains sample location independent of subsequent behavioral relevance. FEF neurons also exhibited target-position-dependent anticipatory activity immediately prior to target onset, suggesting that monkeys predicted target position within blocks. These results show that FEF neurons maintain spatial information during short-term memory, even when that information is irrelevant for task performance.
It is known that spatial information—in the form of attentional cues—can enhance object and feature information during perception of visual stimuli (Carrasco, Penpeci-Talgar, & Eckstein, 2000; Posner, 1980; Vogel, Woodman, & Luck, 2005). Likewise, spatial cueing can gate the entry of objects or features into short-term memory (Averbach & Coriell, 1961; Schmidt et al. 2002; Sperling, 1960). The role of spatial information during object memory maintenance, however, is less clear. Some studies suggest that a persistent spatial signal may contribute to object memory maintenance (Fougnie & Marois, 2009; Treisman & Zhang, 2006; Wood, 2011). For example, several labs have now demonstrated the ability of spatial cues provided during memory maintenance, well after stimulus offset, to improve both accuracy and reaction time on short-term object memory tasks (Griffin & Nobre, 2003; Matsukura, Luck, & Vecera, 2007; Theeuwes, Kramer, & Irwin, 2011). If, as these studies suggest, spatial information helps maintain object representations during short-term memory, then spatial information itself should be maintained even during a purely object-based task. Here we test this hypothesis neurophysiologically and show that maintenance of spatial information indeed occurs during object-based short-term memory, and therefore may contribute to performance.
The Frontal Eye Field (FEF) has long been known to exhibit persistent delay period activity during memory guided saccade tasks (Bruce & Goldberg, 1985). More recently it has been shown that this sustained spatial selectivity is present in the FEF even in the absence of eye movements (Armstrong, Chang, & Moore, 2009; Lawrence & Snyder, 2009). The FEF has also been implicated in both mediating the behavioral benefits of attention (Monosov & Thompson, 2009; Moore & Fallah, 2001; Schafer & Moore, 2011) and modulating activity in visual cortex (Ekstrom et al., 2009; Moore, 2006; Noudoost & Moore, 2011). In addition, the FEF is reciprocally connected with posterior visual areas (Webster, Bachevalier, & Ungerleider, 1994) and nearby prefrontal regions (Stanton, Bruce, & Goldberg, 1993) where object and feature-selective delay activity has been reported during short-term memory (Miller, Erickson, & Desimone, 1996; Miyashita & Chang, 1988; Zaksas & Pasternak, 2006). Unlike ventrolateral prefrontal cortex, however, the FEF exhibits little or no object or feature selectivity (Bichot, Schall, & Thompson, 1996; Peng et al., 2008). These functional and anatomical properties make the FEF a candidate for maintaining spatial signals that interact with feature information during short-term memory.
To test whether spatial information is maintained during object-based short-term memory, we recorded from the FEF during an object-based delayed match-to-sample (DMS) task, in which monkeys remember object identity irrespective of changes in location. Despite the irrelevance of sample location for task performance, FEF neurons encoded sample position throughout the delay period. Furthermore, neurons continued to encode sample position in a variant of the task in which the matching stimulus never appeared in their response field. FEF neurons also exhibited target-position-dependent anticipatory activity immediately prior to target onset, suggesting that the monkeys can predict target position within blocks. The persistence of this spatial information in the FEF during an object memory task is consistent with its possible use in the maintenance of object memory.
Two male rhesus monkeys (Macaca mulatta, 11 and 12 kg) were used in these experiments. All experimental procedures were in accordance with the National Institutes of Health Guide for the Care and Use of Laboratory Animals, the Society for Neuroscience Guidelines and Policies, and Stanford University Animal Care and Use Committee. General surgical procedures have been described previously (Armstrong, Fitzgerald, & Moore, 2006). Each animal was surgically implanted with a titanium head post and a scleral search coil. Surgery was conducted using aseptic techniques under general anesthesia (isoflurane), and analgesics were provided during postsurgical recovery. Structural magnetic resonance imaging was performed to locate the arcuate sulcus in each monkey for the placement of a recording chamber in a subsequent surgery. A craniotomy was performed on each animal, allowing access to the FEF on the anterior bank of the arcuate sulcus.
Single-neuron recordings in awake monkeys were made through a surgically implanted cylindrical titanium chamber (20 mm diameter) overlaying the arcuate sulcus. Electrodes were lowered into the cortex using a hydraulic microdrive (Narishige International USA, INC., East Meadow, NY.). Activity was recorded extracellularly with varnish-coated tungsten microelectrodes (FHC Inc., Bowdoinham, ME) of 0.2–1.0 MΩ impedance (measured at 1 kHz). Extracellular waveforms were digitized and classified as single neurons using both template-matching and window-discrimination techniques either online or offline (FHC Inc., Bowdoinham, ME; Plexon, Dallas, TX). During each experiment, a recording site in the FEF was first localized by the ability to evoke fixed-vector, saccadic eye movements with stimulation at currents of <50 μA (Bruce et al., 1985). Electrical microstimulation consisted of a 100 ms train of biphasic current pulses (0.25 ms, 200 Hz) delivered with a Grass stimulator (S88) and two Grass stimulation isolation units (PSIU-6) (Grass Instruments, West Warwick, RI). Current amplitude was measured via the voltage drop across a 1 kΩ resistor in series with the return lead of the current source. During each experimental session, we mapped the saccade vector elicited via microstimulation at the cortical site under study with a separate behavioral paradigm (Moore and Fallah, 2001). In this paradigm, the monkey was required to fixate on a visual stimulus (0.48° diameter circle) for 500 ms, after which time a 100-ms stimulation train was delivered on half the trials. Evoked saccades had vectors with amplitudes ranging from 5-13° eccentricity and angles of -90 to 65° theta (left FEF, monkey H), and 135 to 220° theta (right FEF, monkey S). Landing points of microstimulation-evoked saccades were considered as the center of the response field (RF) of FEF site under study (FEF RF). After mapping the saccade vector, we recorded the responses of any neuron that could be isolated by advancing the electrode within 0–250 μm of the stimulation site (average distance from stimulation site was <150 μm) while monkeys performed the DMS task. FEF neurons with visual activity generally responded to stimuli positioned at the location to which saccades could be evoked with microstimulation (Bruce and Goldberg, 1985). In some experiments, we also measured the visual responses of studied FEF neurons in a memory-guided saccade task to assess the extent of the visual RF. These measurements confirmed that the responses to visual stimuli were stronger at the estimated RF compared to other locations displaced 90° theta. These observations are consistent with previous measurements of the extent of visual RFs in the FEF (e.g. Schall et al. 1995; Schafer and Moore, unpublished observations)
Throughout the experimental session, monkeys were seated in a primate chair and eye position was monitored with a scleral search coil with a spatial resolution of <0.1° (Armstrong, Fitzgerald, & Moore, 2006) and was digitized at 100-200 Hz. Monkeys were trained to fixate within a 1.5-3° diameter error window surrounding a central spot (0.4° diameter). DMS task is depicted in Fig 1A. At 250-750 ms after fixation, a colored photo image (5° diameter) was presented for 300ms (sample period). A delay period of 1014ms followed the sample offset (delay period), after which two images—one match, one nonmatch— appeared (target period), and the monkey was rewarded for making saccades directly to the match. Monkeys were required to maintain fixation throughout the sample presentation and delay; breaks in fixation before the trial was completed immediately terminated the trial, and these trials were not included in the data analysis. Three images were used in each experimental session, and all three images appeared with equal frequency as the sample/match and the nonmatch. The location of the match was randomized with respect to sample location.
The target array could appear in one of two configurations, with the match and nonmatch appearing either in the two potential sample locations (aligned targets) or in positions rotated 90° with respect to the sample positions (orthogonal targets). In the orthogonal block, once the sample disappeared from the screen its location was irrelevant for the remainder of the trial: neither match nor nonmatch ever appeared at the sample location, and saccades to that location were not rewarded. To allow maximum familiarity with the block structure, only two blocks were run in each experimental session: target positions were held constant for a block of 200-400 trials, then switched for a second block of similar duration. The order of the aligned and orthogonal blocks was randomized for Monkey H, whereas the orthogonal block was always first for Monkey S. All sample location, sample/match identity, and nonmatch target identity conditions were pseudorandomly interleaved and were controlled by the CORTEX system for data acquisition and behavioral control. During each experiment, the two sample positions were selected so that one stimulus was positioned inside the RF of the FEF site, based on the endpoints of saccades evoked with microstimulation (5-13° eccentricity). Both monkeys were initially trained exclusively on the orthogonal targets version of the task, and only learned the aligned targets version after reaching criterion (70%) performance with the orthogonal targets. All visual stimuli were displayed on a liquid crystal display monitor (52 cm vertical × 87 cm horizontal) positioned 57 cm in front of the monkey, with a refresh rate of 60 Hz. It should be emphasized that stimulus images were not selected in such a way as to either optimize or conclusively prove the existence of object selectivity in the FEF. Any neuron might have show greater object selectivity to other pairs of images, or less selectivity if controlling for color, shape, or luminance. Shape selectivity in the FEF has been more rigorously demonstrated by Sereno and colleagues, and our observations are consistent with their findings (Peng et al., 2008).
All data analysis was performed in Matlab (MathWorks). Only completed trials were included in the analysis. A criterion level of p< 0.05 was used in all statistical analysis; p values not specified below 10–7. All p-values are based on the Wilcoxon sign-rank test (for paired comparisons) or the Mann-Whitney U-test (for unpaired comparisons), unless otherwise specified; all reported average values are the median of the distribution unless otherwise specified. Task responsiveness was determined based on significant effects (p< 0.05) in a time × condition two-way ANOVA on the firing rate of each recorded neuron. Normalized firing rate (FR) histograms were calculated according to the formula FR(t) = (Rate(t) –Baseline)/(MaxRate – Baseline), where Baseline is the average rate during the 200ms of the fixation period prior to sample onset, and MaxRate is the maximum across all timepoints and conditions.
Object and spatial selectivity were quantified using Receiver Operating Characteristic (ROC) analysis (Green & Swets, 1966). ROC analysis was carried out on the distributions of neuronal firing rates measured during the execution of the delayed saccade task. The areas under ROC curves (AROC) were used as an index of stimulus discrimination and were calculated as in previous studies (Armstrong & Moore, 2007; Britten et al 1992). Specifically, we computed the average firing rate in a moving 100 ms window, during various epochs within the trial. We then computed the probability that the firing rate in each stimulus condition exceeded a criterion. The criterion was incremented from 0 to the maximum firing rate, and the probability of exceeding each criterion was computed. Thus, a single point on the ROC curve is produced for each increment in the criterion, and the entire ROC curve is generated from all of the criteria. The area under the ROC curve is a normalized measure of the separation between the two firing rate distributions obtained with the preferred and non-preferred RF stimuli, and provides a measure of how well the neuronal response discriminates the two stimuli. We also quantified sample position selectivity during the delay period using a location selectivity index (SI), computed as SI = (FRIN – FROUT)/(FRIN + FROUT), where FRIN and FROUT correspond to the firing rate of the neuron on trials in which the sample was presented inside and outside of the RF of the FEF site under study, respectively.
We report the activity of 147 neuronal recordings (78 single and 69 multi-unit) during the aligned target block of the DMS task. A subset of 113 of these neurons was further studied during the orthogonal target block. 18 recordings showed suppression with visual stimulation within the neuronal receptive field (RF) and were excluded from the analysis.
The response of a representative example neuron during the object DMS task (Fig 1A) is shown in Fig 1B, for samples presented either inside of (Sample In) or opposite (Sample Out) its RF, with the match appearing either inside of (Match In) or opposite (Match Out) the RF. This example neuron illustrates several properties observed in the population response: a visual response to a sample in the RF, sustained delay activity representing the previous sample location, and match location selective activity following target array onset. The visual response to a sample image appearing in the RF was significant whether comparing firing rates from 50-350ms after the sample onset to rates during fixation, or to the same time period on Sample Out trials (both p< 10–7). This sample location selectivity persisted throughout the delay period, with Sample In activity remaining elevated for the period from 250ms after sample offset until 100ms before target onset (Sample In firing rate vs. baseline or vs. Sample Out, both p< 10–7). Following target onset, activity reflected the location of the matching target (Match In vs. Match Out, p< 10–7). Some neurons with no visual response to the sample also exhibited spatially selective delay activity. An example of such a neuron is shown in the bottom panel of Fig 1B. The response of this neuron did not change from baseline when the sample appeared in the RF (Sample In visual response vs. baseline, p =0.730). However, after sample offset, the neuron's response increased and remained elevated in a manner dependent on prior sample location (Sample In delay response vs. baseline, p< 10–7; Sample In vs. Sample Out, p< 10–7).
The normalized response of a population of 129 FEF neuronal recordings is shown in Fig 2A. As expected, FEF neurons showed selectivity for the location of the sample during its presentation. More importantly, the sample location selectivity persisted throughout the delay period. The distributions of population AROC areas during different task epochs, reflecting the ability of neurons to discriminate between different sample locations and sample objects, are shown in Fig 2B-D. Both object and location information were greatest during the sample period ([ROCsample –ROCdelay]; location = 0.153, p< 10–7; object = 0.049, p =0.0220), but significant selectivity for these properties persisted during the delay period (location ROCdelay=0.580, p<10–7; object ROCdelay=0.506, p =0.031). Although there was object selectivity during the delay period, both the magnitude of that selectivity and the proportion of neurons with significant selectivity were less than those for location selectivity during the same period (99 location-selective neurons vs. 29 object selective neurons, Fisher's exact test p< 10–7; object ROCdelay vs location ROCdelay p< 10–7). Delay period location selectivity was not significantly different for cells with and without a visual response (p = 0.736). Following target array onset, FEF activity reflected both the matching target location (location ROCtarget=0.544, p =7.46×10-4) and target identity (object ROCtarget=0.544, p =1.35×10-4).
The relative positions of sample and target stimuli during the orthogonal block, in which the target positions were rotated 90° with respect to the FEF RF, are shown in Fig 3A. The activity of an example neuron during the aligned and orthogonal blocks is shown in Fig 3B. As expected given the change in target position, activity during the target period was greatly reduced in the orthogonal block (p<10–7). The critical question was whether the delay period activity would be affected by the change in target position: it was not. The delay period activity of this example neuron did not significantly change between blocks (p = 0.699). The population response during aligned and orthogonal target blocks is shown in Fig 4A (n=95). As expected, responses to the targets were significantly reduced in the orthogonal block, both for Match In/Ipsi trials (Fig 4B; aligned block = 0.553, orthogonal block = 0.171, p<10–7) and for Match Out/Contra trials (aligned block 0.462, orthogonal block = 0.235, p<10–7). However, delay period selectivity across the population, measured with a location selectivity index (SI), was not significantly different between the aligned and orthogonal target blocks (Fig 4C), either for the population as a whole (p = 0.812), or considering only neurons with significant delay selectivity (n=64, p = 0.961). Restricting the delay period analysis to neurons with a significant change in target period activity between blocks also did not yield a difference in delay period selectivity (n=80, Sample In, p = 0.465; SI, p = 0.775). We considered that if delay period activity reflects anticipation of the target array, then the change in Sample In activity during the delay period should correlate with the change in target period response for each recording. However, a comparison between blocks revealed that the fractional change in Sample In firing rate was significantly greater during the target period than during the delay period (Fig 4D; log(aligned/orthogonal), target period = 0.390, delay period = -5.10×10–3, target vs. delay, p< 10–7). The fractional change in activity between aligned and orthogonal blocks was still significantly larger during the target period than during the delay period when limiting the analysis to neurons with significant delay period selectivity (log(aligned/orthogonal), delay period = -0.0119, target period, 0.370; target vs. delay, p< 10–7). Thus, delay period activity reflected sample location independent of the upcoming target array position.
Delay period selectivity was statistically identical between aligned and orthogonal blocks. However, within a brief window prior to target onset FEF activity reflected the location of the upcoming target array. This ‘pre-target’ period began 100ms prior to target onset and continued until the onset of the earliest visual response (50ms after target onset). As shown in Fig 5A, during the pre-target period there was increased activity in the aligned block as compared to the orthogonal block. This target position dependent difference in activity was observed regardless of prior sample location (aligned vs. orthogonal block, Sample In p = 1.61×10–5; Sample Out p = 0.0013; average across sample positions shown in 5B). We examined the magnitude of this pre-target activity over the time-course of the aligned and orthogonal blocks to see whether it was affected by familiarity with the target locations. An ANOVA comparing pre-target activity across trials within a block showed a significant effect of neuron and aligned vs. orthogonal block (p< 10–7), but not of trial (p = 0.885). Furthermore, a significant effect of target position on pre-target period firing rates was detected within the first ten trials of each block (p = 0.0407), suggesting that monkeys quickly transitioned between the target position expectations of the two blocks. Delay period selectivity was likewise present in these early trials within a block, and statistically indistinguishable from that seen in the remainder of the block (delay SI for first 20 trials vs remainder, orthogonal block p = 0.911, aligned block p = 0.588).
The delay period selectivity observed in both the aligned and orthogonal target blocks varied with performance. Overall, there was higher delay period selectivity on error trials than on correct trials (Fig 6A,B). Using recordings with at least five incorrect trials of each type, delay SIs were found to be significantly larger on error trials, both for the aligned block (Fig 6A; n=109, p =1.57×10-3; n=76 with paired orthogonal recordings, p = 2.91×10-3) and the orthogonal block (Fig 6B; n=83, p = 5.23×10-3). The magnitude of the difference in SIs between correct and error trials did not significantly differ for the aligned vs. orthogonal block, either for the population as a whole (n=76 neurons with sufficient incorrect trials in both blocks; change in SI, correct – incorrect, aligned block = -0.0173, orthogonal block = -0.0188, p = 0.864), or for neurons with significant delay selectivity (n=51, p = 0.708). No such reduction in SI was observed during the visual responses (orthogonal block, p = 0.482; aligned block, p = 0.125). In the aligned block, target period activity often reflected target location (as shown in Fig 2B). We examined the target location selectivity of the FEF for correct vs. error trials, where target selectivity on error trials indicates higher activity when the matching target appears in the RF (as opposed to indicating the direction of the saccade). We found that target selectivity across the population was the same for error trials (Fig 6C; n=103 neurons recorded during experiments with sufficient incorrect trials to perform analysis, p =0.265; n=85 neurons with significant target location selectivity, p =0.731). Pre-target anticipatory activity was likewise unaltered on error trials ([aligned – orthogonal] for correct trials vs. incorrect trials, Sample In p = 0.804, Sample Out p = 0.959). Despite the irrelevance of sample location to task performance, sample location information was maintained throughout the delay period, and the magnitude of that spatially selective delay activity was correlated with performance in both the aligned and orthogonal versions of the task.
FEF neurons encoded sample position throughout the delay period of an object-based short-term memory task, despite the irrelevance of sample location for task performance. Furthermore, neurons continued to encode sample position in a variant of the task in which the matching stimulus never appeared in their RF, confirming that FEF maintains sample location independent of subsequent behavioral relevance. FEF neurons also exhibited target-position-dependent activity prior to target onset, suggesting that the monkeys anticipated target position within blocks. The persistence of this spatial information in the FEF during an object memory task is consistent with an involvement of spatial information in the maintenance of object memory.
The sample-location selectivity exhibited by FEF neurons during the aligned block is inconsistent with a pure “saccade probability” or “upcoming sensory discrimination” account of FEF activity (Basso & Wurtz, 1998; Zhou & Thompson, 2009). Since the probability of a potential target appearing in the RF (100%) and the probability of making a subsequent eye movement to that location (50%) were identical for Sample In and Sample Out trials, this persistent spatial activity represents at minimum an interaction between prior history and upcoming expectations. Furthermore, the fact that this selectivity was unaltered during the orthogonal block, when the location represented by the maintained activity was never a potential target location, confirms that the delay selectivity was independent of that location's relevance to subsequent behavioral responses.
The presence of a target-position dependent difference in activity during the pre-target period of the aligned vs. orthogonal blocks suggests that monkeys anticipated the target positions in the two blocks. This result also confirms that the absence of differential activity between blocks during the delay period did not result from a lack of statistical power. The anticipatory pre-target activity may reflect the allocation of attention to an upcoming sensory discrimination, or motor preparation toward the location of a future saccadic target, or both. An increase in activity later in the delay, in anticipation of target onset, has been previously observed in MT during a motion memory task (Bisley et al., 2004), where it likewise occurs across the population irrespective of a neuron's prior contribution to the representation of the sample stimulus. The early emergence of anticipatory activity during the pre-target period, significant within the first 10 trials of each block, suggests that monkeys were quickly able to anticipate the different target positions within blocks. However, this change in anticipated target position does not affect spatially selective delay activity.
Although neurons in some areas of PFC have been reported to represent only behaviorally relevant stimulus properties (Everling et al., 2006; Rainer, Asaad, & Miller, 1998), we found that location information, in the form of spatially selective delay period activity, was maintained despite its irrelevance for correctly selecting the Match in the target array. The maintenance of spatial information during an object-based task, a task in which correspondence between sample and match location need not be remembered for correct performance, raises the question of whether that information actually contributes to performance. Irrespective of whether the maintenance of spatial information occurs ‘by default’ or reflects a strategy by the monkey, one can ask whether that maintenance relates to performance. We found that despite the irrelevance of sample location to task performance, the magnitude of spatially selective delay activity was nonetheless correlated with performance in both the aligned and orthogonal versions of the task. Spatial selectivity was elevated for error trials compared to correct trials. This result might indicate that any maintenance of sample location information is detrimental to task performance. Or, it may be indicative of a more complex relationship between the maintenance of spatial and object signals during short-term memory. For example, there may be an optimal level of spatial information maintenance during object memory such that at suboptimal levels, spatial and object maintenance are positively correlated and at supraoptimal levels, they are negatively correlated. At the very least, the fact that the maintenance of spatial information correlates at all with memory performance seems to indicate that such maintenance is not independent of object memory.
The ability of spatial retro-cues to protect remembered objects from degradation during memory maintenance suggests that a spatial signal can modulate object representations within short-term memory (Griffin & Nobre, 2003; Makovski et al., 2008; Matsukura et al., 2007; Theeuwes et al., 2011), and some behavioral evidence suggests that short-term sensory memory is stored in a spatially specific manner (Zaksas, Bisley, & Pasternak, 2001). In this case, rather than location and object information being stored completely independently, maintenance of a spatial ‘tag’—whether voluntary or automatic—may contribute to maintaining an associated object representation within memory. A second way in which spatial signals may influence object representations in memory is proposed by the “feature binding theory” of object vision. The feature binding theory posits a special role for spatial attention in creating associations among different features of an object during visual perception, ‘binding’ separate features such as color, shape, size, etc into a unified object representation (Treisman & Gelade, 1980; Treisman & Schmidt, 1982). It has also been suggested that spatial attention is required to maintain these bindings during short-term memory (Wheeler & Treisman, 2002; Treisman & Zhang, 2006). A role for spatial attention in maintaining feature bindings in short-term memory is supported by the finding that attentive tracking during memory maintenance selectively interferes with memory for feature bindings (Fougnie & Marois, 2009). The ‘retro-cuing’ effect suggests that a spatial signal during object memory could contribute to performance, while the feature binding theory predicts that such a spatial signal is necessary to maintain object bindings. Given that the FEF has been widely implicated in the control of spatial attention (Moore, 2006), one might speculate that the persistent signaling of spatial information by FEF neurons during object-based short-term memory suggests a role for spatial attention in maintaining object memory. Nonetheless, future experiments will need to test the causal role of the observed persistent spatial signal in object memory and the relationship of that signal to attention.
We thank Doug S. Aldrich for technical assistance. This work was supported by NIH EY014924, NSF IOB-0546891, The McKnight Foundation, and Stanford Bio-X Fellowship to K.C.