|Home | About | Journals | Submit | Contact Us | Français|
It is widely reported that the activity of single neurons in visual cortex is correlated with the perceptual decision of the subject. The strength of this correlation has implications for the neuronal populations generating the percepts. Here we asked whether microsaccades, which are small, involuntary eye movements, contribute to the correlation between neural activity and behavior. We analyzed data from three different visual detection experiments, with neural recordings from the middle temporal (MT), lateral intraparietal (LIP), and ventral intraparietal (VIP) areas. All three experiments used random dot motion stimuli, with the animals required to detect a transient or sustained change in the speed or strength of motion. We found that microsaccades suppressed neural activity and inhibited detection of the motion stimulus, contributing to the correlation between neural activity and detection behavior. Microsaccades accounted for as much as 19% of the correlation for area MT, 21% for area LIP, and 17% for VIP. While microsaccades only explain part of the correlation between neural activity and behavior, their effect has implications when considering the neuronal populations underlying perceptual decisions.
Microsaccades are small, involuntary eye movements thought to counteract drift of the eyes (Cornsweet, 1956) and visual fading (Martinez-Conde et al., 2006), as well as to improve discrimination of high spatial frequencies (Bridgeman and Palca, 1980; Rucci et al., 2007). Microsaccades may also increase visual detection thresholds (Ditchburn, 1955; Beeler, 1967; but see Krauskopf, 1966; Sperling, 1990). Microsaccades have also been shown to affect neural activity throughout the visual system (Bair and O'Keefe, 1998; Leopold and Logothetis, 1998; Martinez-Conde et al., 2000; Snodderly et al., 2001; Martinez-Conde et al., 2002). However, the behavioral and neuronal effects of microsaccades have not been examined in the same experiment. If microsaccades affect both perception and neuronal activity, it is possible that microsaccades could contribute to apparent correlations between the two.
Numerous studies have shown that the activity of single neurons in visual cortex can be correlated with behavioral performance on a trial-by-trial basis (for reviews, see Parker and Newsome, 1998; Romo and Salinas, 2001). The strength of the correlation between neural activity and behavior has been termed choice probability (CP) for discrimination tasks (Britten et al., 1996) or detect probability (DP) for detection tasks (Cook and Maunsell, 2002b). These two measures have been used to shed light on the identity of the cortical areas involved in a perceptual decision (Britten et al., 1996; Cook and Maunsell, 2002b; Grunewald et al., 2002; Williams et al., 2003; de Lafuente and Romo, 2005; Liu and Newsome, 2005; de Lafuente and Romo, 2006; Gu et al., 2007), the degree of correlation between neurons (Shadlen et al., 1996; Bair et al., 2001; Dodd et al., 2001; Schoppik et al., 2008), how the activity of different neurons are weighted to form the decision (Britten et al., 1996; Purushothaman and Bradley, 2005; Uka et al., 2005; Gu et al., 2007), and how different phases of neural activity are linked to detection (Masse and Cook, 2008). These studies have found detect-probability and choice-probability values range between ~0.5 and 0.7, depending on cortical area and experimental paradigm.
According to this interpretation, trial-by-trial variation in a neuron's response to a repeated stimulus is assumed to be noise. The observation that this variability is correlated with the animal's choice suggests that the brain is “listening to” that neuron to make the choice. However, one must be careful to rule out other sources of correlation between neural activity and behavior. For example, trial-by-trial variability in the stimulus could cause correlation between activity and behavior (Cook and Maunsell, 2002b). Because microsaccades can likewise affect both behavior and neuronal responses, they are another potential source of correlation.
We examined the simultaneous effects of microsaccades on behavior and neuronal responses in three different visual detection paradigms, covering the middle temporal (MT), lateral intraparietal (LIP), and ventral intraparietal (VIP) areas, and involving seven macaque monkeys. We found that microsaccades inhibited the monkeys' ability to detect visual stimuli in our three experimental paradigms. We also found that microsaccades, on average, suppressed neural activity across the cortical areas examined. On average, the contribution of microsaccades to detect probability ranged from 7 to 19% for area MT (in two experiments, excluding the trials with microsaccades reduced detect probability from 0.560 to 0.556 and from 0.558 to 0.547), 21% for area LIP (detect probability was reduced from 0.584 to 0.566) and 0 to 17% for area VIP depending on the time points analyzed (detect probability was reduced from 0.558 to 0.548).
To investigate the effect of microsaccades on neural activity and visual perception, we analyzed data from three different motion-detection experiments, all using a random dot stimulus. In the first experiment, two monkeys were trained to respond to a transient increase in speed while neural activity was recorded from areas MT and LIP. In the second experiment, two monkeys were trained to detect a coherent motion step while neural activity was recorded from MT and VIP. In the third experiment, three monkeys were trained to detect a brief pulse of coherent motion in two slightly different paradigms. Neural activity was recorded from area MT in two of the monkeys. All four paradigms (the third experiment had two variations) are outline in Figure 1.
Two monkeys (Macaca mulatta) were trained to perform a spatially cued speed-pulse detection task. At the beginning of a trial the stimulus consisted of a central fixation spot and two annuli, one red and one green, in opposite hemifields at equal eccentricity (Fig. 1A). The monkey had to maintain gaze within a fixation window throughout the trial (2° × 2° square, centered on a fixation spot). After the monkey fixated, there was a 500 ms delay before two fields of coherently moving random dots appeared within the annuli. The monkey's task was to detect a transient increase in the speed (53 ms, four video frames) of either dot patch, and to respond by releasing a touch bar within a requisite time window (200–600 ms). The color of the fixation point (red or green) cued the monkey as to which patch (surrounded by red or green annulus) was more likely to contain the motion pulse (85% valid cues, 15% invalid cues). On 40% of trials, the fixation point color cue switched at an unpredictable time during the trial to indicate that the likely motion pulse location had switched. Each trial had at most one cue switch. After an initial fixed delay of 400 ms, additional delays until motion pulses and cue switches, as well as between cue switches and motion pulses, were selected randomly from an exponential distribution (mean = 1 s). This main purpose of the study was to examine effects of switching spatial attention, butthose results are not relevant to the questions examined here.
For the analysis of the correlation between microsaccades and behavior (see Fig. 3A) we only used trials with valid cues. For the analysis of the neural response to coherent motion (see Fig. 4A, B) or the correlation between neural activity and behavior (see Fig. 5A, B) we only used trials in which the speed pulse was validly cued to occur inside the neuron's receptive field.
Stimuli were presented on a computer monitor positioned 57 cm in front of the animal (40° × 30°, 75 Hz refresh, 1152 × 870 resolution). Background luminance was near black (0.001 cd/m2). The fixation point was a 0.4 degree diameter red or green circle (luminance in cd/m2: monkey M, red: 2.7, green: 3.0; monkey B, red: 2.4, green: 5.2). Dot-patch stimuli consisted of 100% coherently moving, unlimited-lifetime, random dots. Dots were squares with 0.1 degree sides, at a density of 7 dots/degree2 and moving at 12 degrees/s. Dot luminance was 0.01 cd/m2. Annuli surrounding the moving dot patches were 0.5 degrees thick and separated from the perimeter of the dot patches by 0.5 degrees.
Where possible, dot patches were placed in the center of the receptive field of the recorded neuron. The dot-patch motion in the receptive field was set in the neuron's preferred direction as determined by a direction-mapping task that we ran before the main task for each neuron. The other dot patch was always placed at the equivalent position reflected across the fixation point and had the opposite direction of motion. The size of the dot patches was scaled with eccentricity (ranging from 4.5 to 9.4 degrees in diameter). The magnitude of the speed pulse was chosen to maintain valid correct performance in the target range (65–75% correct) and varied from session to session (range for monkey B: 1.6× to 2.5×, range for monkey M: 1.35× to 1.7×).
The recording chamber was placed at stereotactic coordinates P3 L10, which allowed a dorsal approach to areas MT and LIP. The chamber was outfitted with a guide-tube/grid system (Crist Instrument). Magnetic resonance imaging was used to confirm sulcal anatomy and chamber placement. Single unit recordings were conducted using tungsten microelectrodes (Frederick Haer; 75 μm diameter, 5 MΩ impedance). Single unit action potentials were isolated using a dual window discriminator (Bak Electronics) and recorded at 1 ms resolution. Horizontal and vertical eye positions were monitored using a scleral search coil (Riverbend Instruments) and recorded at 200 Hz.
MT and LIP cells were identified by reference to sulcal anatomy and characteristic physiology. MT cells were characterized by highly direction-selective receptive fields with diameters approximately equal to eccentricity (Maunsell and Van Essen, 1983, 1987). LIP cells were characterized by robust, spatially tuned responses in a memory delayed saccade task (Colby et al., 1996). Additionally, cells were considered within the target area if they were encountered between cells with characteristic properties. All such stably isolated units were recorded. In all, we recorded from 118 LIP neurons and from 67 MT neurons.
The data set analyzed for this study comes from several previous studies (Cook and Maunsell, 2002a,b, 2004; Masse and Cook, 2008). Two monkeys were trained to perform a spatially cued motion detection task (Fig. 1B). The monkey initiated the trial by depressing a lever and fixating on a central point. The monkeys were trained to release a lever when coherent motion began in one of two random dot patches diametrically opposite of the fixation point. After the cue was presented, 0% coherent motion began in the two patches followed by coherent motion occurring in one of the two patches at a random time (flat hazard function) 500–8000 ms afterward. The location of the coherent motion was cued to the monkey at the start of the trial with static dots and this cue was valid on 80% of the trials. The strength of coherent motion was varied between three levels (low, medium and high) with the monkey correctly detecting the coherent motion 50%, 90%, and 99% of the time for the three levels. The analysis of the behavioral effect of microsaccades (see Fig. 4B) only used trials with low-level coherent motion. Analysis of the effect of microsaccades on neural activity during 0% coherent motion used only trials where the monkey was cued to attend inside the neuron's receptive field (see Fig. 5C,D, left panel). Analysis involving the neural response to coherent motion (see Fig. 5C,D, right panel) or the partial correlation analysis (see Fig. 6C,D) only involved trials using low-level coherent motion that was validly cued to occur inside the neuron's receptive field.
In all experimental sessions, the direction and the speed of the coherent motion were matched to the preferred direction and speed of the neuron under study. The coherent motion lasted 750 ms and the monkey had to release the lever from 200 to 750 ms after the onset of coherent motion to obtain a reward. Trials where the monkey failed to release the lever or released the lever too late were deemed missed trials. Only correct and missed trials were included in the analysis. Trials where the monkey released the lever too early or was unable to maintain fixation were discarded from the analysis.
The animal sat 62 cm from a computer monitor (±17° × ±13° of visual angle; 1600 × 1200 pixels; 75 Hz refresh). The stimuli consisted of two patches of white dots (each dot 0.25° diameter; 78 cd/m2) on a dark gray background (12 cd/m2) with a dot density of 2.1 dots/degree2. Each patch of dots was updated on every other video frame (approximately every 27 ms) using the following procedure. The dots in each patch were evenly divided into two groups. On each update, one group was replaced with new, randomly positioned dots, whereas dots in the other group were displaced by a fixed distance. The dots in this latter group determined the motion coherence. For 0% coherence, all the dots in this group moved a fixed distance in a random direction. For coherent motion greater than zero, a proportion of the dots moved with a fixed distance in the same direction. This proportion determined the strength of the coherent motion. On the next update (27 ms later), the groups were switched. This arrangement insured that all the dots had a lifetime of four video frames (i.e., two stimulus updates) before they were replaced.
Recordings were made from well isolated single neurons in area MT and VIP in both animals, using standard extracellular recording techniques (Gibson and Maunsell, 1997). After a neuron was isolated, the receptive field (RF) was mapped using a manually controlled bar while the animal fixated on a central spot. The preferred speed was also judged using a bar moved by hand. The animals were trained to perform the task at slow or moderate motion speeds, so neurons with a preferred speed between 4 and 12°/s were usually selected. Once the RF location, size, preferred direction, and speed were determined, the motion detection task was then run, and the neuron was recorded from for as long as possible. The number of completed trials per coherence level for the motion detection task ranged from 15 to 175 (median, 35). The monkey's performance varied with patch location, size, and motion speed, which were determined by the response properties of the neuron under study. Consequently, different neurons were tested with different coherence levels. The animal's eye position was measured every 5 ms using a scleral search coil (Robinson, 1963; Judge et al., 1980) and the times of action potentials were recorded to the nearest millisecond. In total, 93 MT and 104 VIP neurons were recorded in two monkeys.
Data from two different experimental paradigms, both involving the detection of transient motion, were combined for the purpose of this analysis. In the first (Fig. 1C), two monkeys were trained to detect a 50 ms coherent motion pulse that occurred in one or both random dot patches. Monkeys fixated on a central point and depressed a lever to initiate the trial, after which motion in two random dot patches started moving with 0% coherence. Dots could either move in the neuron's preferred or null direction, always by the same distance. Coherent motion would then appear in one or both random dot patches at a random time (flat hazard function), anytime from 500 to 10,000 ms after the start trial. The coherent motion lasted 50 ms and was of a consistent strength for each random dot patch throughout the experimental session. The ratio of trials with motion in two patches to motion in one patch was varied between two to one and four to one. The monkey was rewarded by releasing the lever from 200 to 800 ms after the start of coherent motion.
In the second experiment, a single monkey was trained to detect a 33 ms coherent motion pulse in a single random patch. As above, coherent motion would begin at a random time from 500 to 10,000 ms after the start of the trial, and the monkey had to release the lever from 150 to 650 ms after the motion pulse to receive a reward. This experiment was used for a microstimulation study, and some trials also contained a 33 ms probe of subthreshold microstimulation to area MT or a 33 ms probe of subthreshold motion coherence at various times relative to the main motion signal. For the purpose of this study, all trials were considered the same for this analysis.
Stimuli were presented on a computer monitor positioned 57 cm in front of the animal (120 Hz refresh, 1600 × 1200 resolution). Random dot patches consisted of white dots moving in either the preferred or null direction of the neuron, by always the same distance. In the first motion-pulse experiment, dots were assigned a probability of moving in the preferred direction for each frame, whereas in the second motion-pulse experiment the total number of dots moving in the preferred direction was controlled for each frame. For example, during 0% coherent motion, each dot has a 50% probability moving the preferred direction whereas in the second experiment, exactly 50% of the dots each frame would move in the preferred direction. For both experiments, the direction that a dot moved in one frame was not related to its direction for any other frame. If a dot moved outside the random dot patch, it would be randomly replotted anywhere in the opposite hemifield of the random dot patch. Dots were circles with 0.15-degree radius, at a density of 10 dots/degree2.
Recordings were made from well isolated single neurons in area MT animals in the first motion-pulse experiment and from multi-unit activity in area MT in the second motion-pulse experiment. The RF was mapped using a manually controlled bar while the animal fixated on a central spot. The preferred speed and direction by recording activity in response to 100% coherent motion in various speeds and directions while the monkey fixated. Once the RF location, size, preferred direction, and speed were determined, the motion detection task was then run, and the neuron(s) was recorded from for as long as possible. The monkey's performance varied with patch location, size, and motion speed, which were determined by the response properties of the neuron under study. Consequently, different neurons were tested with different coherence levels. The animal's eye position was measured every 5 ms using a video-tracking system (ASL 6000, Applied Science Laboratories or Eyelink 1000, SR Research).
We detected microsaccades in the eye-position records using an adaptation of a previously described technique (Martinez-Conde et al., 2000). Since eye positions were captured by different methods for each experiment, the parameters used for microsaccade detection were adjusted slightly to each one. In all experiments, horizontal and vertical eye positions, sampled at 200 Hz, were used to calculate an instantaneous eye velocity. The velocity vectors were smoothed by a 25 ms boxcar for the speed pulse and the motion-step experiments. The velocity vectors for the 33 ms motion-pulse experiment was only smoothed by a 15 ms boxcar since eye positions were filtered with a 100 Hz low-pass Butterworth filter during the course of the experiment.
The velocity vectors for the 50 ms motion-pulse experiment were not smoothed since eye positions were already filtered with a 20 Hz low-pass Butterworth filter during the course of the experiment. However, this filtering complicated our estimates of peak speed and amplitude by rendering the eye speed highly positively auto-correlated at short time lags. Thus, we linearly interpolated the eye position to increase the sampling rate from 200 Hz to 1 kHz. Interpolation was only used to measure peak speed and amplitude; microsaccade detection used non-interpolated eye position data. More sophisticated interpolation methods using the MATLAB resample function or by fitting the eye position with cubic splines produced similar results.
Instances where the eye speed crossed 8 degrees/s for the motion-step and 33 ms motion-pulse experiment and 10 degrees/s for the speed pulse and 50 ms motion-pulse experiment were considered possible microsaccades. The reason for the difference in threshold was that the eye position signal for the motion-step and 50 ms motion-pulse experiments was less noisy than the others, and thus microsaccades could be detected with greater confidence.
To be considered a microsaccade, the eye movement had to last at least 10 and not more than 300 ms, could not have started within 20 ms of a previous microsaccade, was at least 0.05° in length, and the eye direction could change no more than 30 degrees every 5 ms for the duration of the microsaccade. Only events that satisfied all the criteria above were deemed microsaccades, and the starting time of the microsaccade was used in all further analysis. Accuracy of the saccade algorithm was further confirmed by visual inspection of raw eye movement traces for a subset of the data.
We wanted to examine the effects of microsaccades on behavioral choice and neuronal firing in visual and parietal cortex and to determine the extent to which microsaccades contributed to the correlation between the two. Our analysis was based on three different visual detection tasks illustrated in Figure 1. All three used random-dot motion as a stimulus; one involved detecting a change in speed and two involved detecting a change in motion coherence. Data were collected from three different laboratories, using a total of seven macaque monkeys as subjects.
In the first experiment (speed-pulse experiment) (Fig. 1A) (see Materials and Methods), the monkey had to release a lever in response to a transient (53 ms) increase in speed in one of two, 100% coherently moving random dot patches. In the second experiment (motion-step experiment) (Fig. 1B) the monkey had to release a lever in response to the onset of coherent motion in one of two, initially 0% coherent, moving random dot patches. The third experiment (motion-pulse experiment) (Fig. 1C,D) was a combination of two, slightly different motion-pulse detection tasks. In the first, the monkey had to release a lever in response to a transient (50 ms) pulse of coherent motion in one of two, initially 0% coherent, moving random dot patches. In the second, the monkey detected a 33 ms pulse of coherent motion in a single random dot patch.
The correlation between neural activity and behavior has frequently been measured by using motion-based stimuli and neural recordings from direction-selective neurons (Britten et al., 1996; Bair et al., 2001; Dodd et al., 2001; Cook and Maunsell, 2002b; Grunewald et al., 2002; Krug et al., 2004; Uka and DeAngelis, 2004; Liu and Newsome, 2005; Purushothaman and Bradley, 2005). By combining data from three experiments with different motion-based stimuli along with direction-selective neural activity from three cortical areas, we wished to determine the effects of microsaccades that may apply broadly to experiments of this nature.
We detected microsaccades in the eye-position records using an adaptation of a previously described technique (Martinez-Conde et al., 2000) (see Materials and Methods). To confirm the accuracy of our microsaccade-detection algorithm, we compared the properties of the microsaccades for our experiments to those of past studies on microsaccades in non-human primates (Table 1) (Martinez-Conde et al., 2004). Because past studies captured the eye position and analyzed the resulting data in different ways, there is substantial variability in the reported parameters. Nonetheless, the microsaccade parameters from our experiments were consistent with those previously reported.
Previous studies have described a linear relationship between saccadic peak speed and amplitude that is common to all saccades, including microsaccades (Zuber and Stark, 1965; Bahill et al., 1975). Microsaccades that obey this relationship are said to fall along the “main sequence.” Figure 2 is a two-dimensional histogram showing the distribution of microsaccade peak speed and amplitude for all of the microsaccades in each of the three experiments. The number of microsaccades falling in each bin is shown by the shade of gray; a bin with a frequency of zero is white and the bin with the highest frequency for that experiment is black. Microsaccades were detected by smoothing the eye velocity as described in Materials and Methods, but peak speed was calculated from unsmoothed eye velocity. Additionally, we linearly interpolated the eye position data from the 50 ms motion-pulse experiment before calculating peak speed and amplitude (see Materials and Methods). All four distributions showed a strong linear relationship between peak speed and amplitude (Spearman's rank correlation coefficients of 0.74, 0.73, 0.91, and 0.77 for the speed-pulse, motion-step, 50 and 33 ms motion-pulse experiments, respectively).
Although it was reassuring to find the main sequence and microsaccade parameters fall within previously observed ranges (Table 1), this does not exclude the possibility that our algorithm either missed microsaccades, misclassified non-microsaccade events as microsaccades or both, especially for the 50 ms motion-pulse experiment. The 20 Hz low-pass filter applied to this experiment makes it likely that brief, low-amplitude microsaccades may have been missed in this data set. We address the impact of such errors in the Discussion.
Figure 3 provides three example trials from the speed-pulse task that illustrate the hypothesis that microsaccades can affect behavior and neural activity and can contribute to the correlation between the two. Figure 3A shows data fromalong trial with a single MT neuron's response to sustained motion in the preferred direction. The individual spike times are denoted by black dots and the instantaneous spike rate by the gray line. During this trial there were three microsaccades (asterisks) evident in the eye-speed trace (black line). For this neuron, each microsaccade was followed by a brief pause in the spike train.
Figure 3B shows data from a trial in which the monkey correctly detected the speed pulse, which occurred at the vertical dashed black line. The spike rate increased after the speed pulse, possibly contributing to the monkey's correct detection. Figure 3C shows data from another trial from the same experimental session. In this trial, the monkey failed to detect the speed pulse. Notably, the monkey happened to make a microsaccade at nearly the same time as the onset of the speed pulse. Not only did the animal fail to detect the speed pulse, but the neural activity was suppressed after the microsaccade, as in Figure 3A. If microsaccades inhibit a subject's ability to detect visual stimuli and at the same time suppress neural activity, then it is possible that microsaccades may contribute to the detect probability. The goal of our study was to examine this hypothesis in detail.
We first determined what effect microsaccades had on the monkeys' ability to detect the different types of visual stimuli used in the three tasks. Previous reports have differed on whether microsaccades modulate visual thresholds, but to our knowledge, the effect of microsaccades on detecting motion or speed signals has not been examined.
We measured the effect of microsaccades on the monkeys' detection performance in two different ways. First (Fig. 4A–C, left panel), the microsaccade rate is shown relative to onset of the test stimulus (the speed pulse, motion step or motion pulse) for trials in which the monkey correctly detected the stimulus (blue) and trials in which the monkey failed to detect the stimulus (red). For the experiments using a pulse stimulus (Fig. 4A,C, left panel), the microsaccade rate was higher on failed than correct trials for times immediately around the onset of the test stimulus. Similarly, for the motion-step experiment (Fig. 4B, left panel) the microsaccade rate was greater on failed than correct trials for times after the onset of coherent motion.
Second, we calculated the monkeys' detection performance as a function of the time of microsaccades relative to the test-stimulus onset (Fig. 4A–C, right panels). In this case, time 0 refers to trials in which a microsaccade occurred simultaneously with the onset of the test stimulus, −200 ms refers to trials in which a microsaccade occurred 200 ms before the onset of the test stimulus, and so on. For the two experiments using pulse stimuli (Fig. 4A,C, right panels), the monkeys' exhibited dramatically reduced performance when a microsaccade occurred near the time of the test-stimulus onset. In contrast, microsaccades occurring >200 ms before the test-stimulus onset had no effect on performance. For the motion-step experiment (Fig. 4B, right panel), the monkeys' performance was reduced when a microsaccade occurred over a period of hundreds of microseconds after stimulus onset.
These results demonstrate that microsaccades can dramatically reduce performance on a range of motion-based visual detection tasks, especially tasks involving transient motion stimuli. Importantly, there was no abrupt change in the overall rate of microsaccades around the time of test-stimulus onset (thin black curve, Fig. 4A–C, left panels). Rather, performance was reduced on that subset of trials in which a microsaccade happened to occur, by chance, near the time of the test-stimulus onset.
The effect of microsaccades on neural activity in the visual system is variable and not completely understood. Studies have generally found that microsaccades increase neural activity in early stages of the visual system, including the LGN (Martinez-Conde et al., 2002) and V1 (Martinez-Conde et al., 2000; Snodderly et al., 2001), and decrease neural activity in later stages, including V2 and V4 (Leopold and Logothetis, 1998). However, clear exceptions exist (Leopold and Logothetis, 1998) and care must be taken when considering new paradigms. For example, the effect of microsaccades on neural activity may depend on the whether a stationary or moving stimulus was presented. In area MT, microsaccades have been reported to increase or decrease activity depending on the baseline firing rate and on whether the eye movement produced motion in the neuron's preferred or null direction (Bair and O'Keefe, 1998).
We examined the relationship between microsaccades and neural activity for the three different tasks and cortical areas. The left column of Figure 5 shows the average neural response aligned on the time of a microsaccade. The neural response for all cells was normalized so that average activity before the microsaccade was equal to one. Because the test stimulus usually changed the firing rate of the neurons, only microsaccades that occurred at least 300 ms before the onset of the test stimulus were included in the analyses shown in the left panels in Figure 5.
Microsaccades, on average, caused a large decrease in neural activity in the speed-pulse experiment for areas MT and LIP (Fig. 5A,B). In these experiments, the subjects were always viewing coherent motion in the neuron's preferred direction. One possible reason why microsaccades produced large suppression in neural activity was that any eye movement away from the null direction would produce apparent motion in a non-optimal direction. In the other two tasks, microsaccades before the test-stimulus onset had either a smaller effect (MT in motion-pulse task) (Fig. 5C) or no clear effect (VIP in motion-pulse task and MT in motion-step task; Fig. 5D,E). Unlike in the speed-pulse task, in these tasks the monkeys viewed 0% coherent motion that did not optimally drive the recorded neurons and therefore apparent motion induced by microsaccades had less net effect on the response. However, it is possible that microsaccades affect neural activity by other mechanisms. For example, the suppression in neural activity in LIP neurons (Fig. 5A) appeared to begin before the microsaccade, which is not consistent with a response to a microsaccade-induced motion signal.
Although microsaccades had little or no effect on neural activity for the tasks where the monkey viewed 0% coherent motion before the test-stimulus onset (Fig. 5C–E, left column), they may have had a stronger effect on the response to the test stimulus, which is the key parameter in determining detect probability (see below). To examine the effect of microsaccades on the response to the test stimulus we used a different approach. In the right column of Figure 5, we show the average neural responses aligned to the onset of the test stimulus. Trials were averaged separately depending on whether there was a microsaccade (gray curve) or no microsaccade (black curve) “near” the time of the test-stimulus onset. To obtain an accurate measurement of the neural response, we only included neurons that had at least 10 trials with microsaccades near test-stimulus onset and 10 without. This condition was satisfied by 67 LIP and 47 MT neurons from the speed-pulse experiment, 28 MT and 29 VIP neurons from the motion-step experiment, and 49 MT neurons from the motion-pulse experiment.
For the two experiments with transient test stimuli (the speed-pulse and motion-pulse experiments), the two trial types were separated based on whether a microsaccade occurred from 200 ms before to 100 ms after the test-stimulus onset. For the motion-step experiment the window was from 100 before to 200 ms after test stimulus onset. These windows were selected based on the results in Figure 4 and represent the time windows for each task in which microsaccades affected detection of the test stimulus. For all three experiments and cortical areas shown in Figure 5, we found that, on average, the neurons increased their firing rate in response to the test stimulus. Moreover, the neural responses to the test stimuli were greater when no microsaccade occurred near the time of the test-stimulus onset.
To quantify the difference between the neural responses to the stimulus with and without microsaccades, we used a receiver operating characteristic (ROC) analysis (Green and Swets, 1966). For the two experiments with transient test stimuli, we measured the spike count from 50 to 200 ms after test-stimulus onset. For the motion-step experiment, we extended this window from 50 to 300 ms after test-stimulus onset. The area under the ROC curve provides a nonparametric index of the separation between the spike-count distributions for trials with microsaccades versus trials without microsaccades. Values >0.5 indicate that the spike count is on average greater when no microsaccades occurred near stimulus onset. For all experiments and brain areas, the average AROC (area under the ROC curve) values were >0.5, indicating that microsaccades were associated with a reduced neural response to the test stimulus (speed-pulse experiment: LIP, AROC = 0.666, p < 0.001; MT, AROC = 0.602, p < 0.001; motion-step experiment: MT, AROC = 0.546, p = 0.056; VIP, AROC = 0.568, p = 0.023; motion-pulse experiment: MT, AROC = 0.524, p = 0.048, all statistical tests were two-sided t tests unless otherwise specified). These results are consistent with the interpretation that at least some of the suppression in activity is the result of a less-preferred apparent motion signal caused by microsaccades (Bair and O'Keefe, 1998), but we cannot exclude the possibility that microsaccades affected neural activity through other mechanisms.
So far we have shown that, on average, microsaccades decrease both detection performance and neural responses across three motion-based detection tasks. Thus, microsaccades may have contributed to the correlation between neural activity and behavior. To more closely examine this, we calculated the partial correlation between behavioral performance, the presence of microsaccades and the neural response. Partial correlation provides a measure of the correlation between each pair of variables after removing the effect of the third.
Each of our detection tasks had two measures of behavior: detection performance (correct or failed) and reaction time. For each measure of behavior, we computed the partial correlations between neural activity, microsaccade occurrence and behavior (Fig. 6). We calculated the neural activity as the number of spikes that occurred in the 100 ms before each time point while the number of microsaccades was calculated using a window from 100 to 200 ms before each time point. The partial correlations were computed at 10 ms intervals relative to stimulus onset.
In the left column, the partial correlation between microsaccades and behavior (green curve) shows that the presence of microsaccades around the test-stimulus onset was negatively correlated with detection of the test stimulus, indicating that microsaccades were associated with reduced detection performance. In the right column, the presence of microsaccades around the test-stimulus onset was usually positively correlated with reaction times (but not in the speed pulse experiments) indicating that microsaccades were associated with a delayed behavioral response. These results were consistent with those illustrated in Figure 4.
The partial correlation between microsaccades and neural activity is shown by the red curves in Figure 6. For the speed-pulse task, the partial correlation between neural activity and microsaccades was consistently below zero, confirming that microsaccades were associated with a strong decrease in activity independent of time relative to the test-stimulus onset. For the other two experiments, this partial correlation was only marginally negative before stimulus onset. However, the value decreased after the onset of the test stimulus, when coherent motion was presented in the preferred direction for the neurons. This further suggests that the suppression of neural activity by microsaccades was stronger when the monkey was viewing a stimulus moving in the neuron's preferred direction. These results were consistent with those illustrated in Figure 5.
Last, we show the partial correlation between neural activity and behavior after removing the effects of microsaccades (blue curve). In all experiments, there remained a robust positive correlation between neural activity and detection performance (left column) and a negative correlation between neural activity and reaction time (right column). Thus after removing the effects of microsaccades, greater neural activity is still associated with a correct behavioral response and shorter reaction times on a trial-by-trial basis. Although microsaccades affected both neural activity and detection, their effect cannot entirely account for the choice-related correlation between the two. These results predict that microsaccades would contribute to some, but not all, of the choice-related correlation between neural activity and behavior such as detect probability. We confirmed this in the following analysis.
Detect probability, which is similar to the AROC metric used in Figure 5 above, has been widely used to quantify the relationship between neural activity and behavioral outcome. Whereas the partial correlation analysis in Figure 6 measures the strength of the linear relationship between spike rate and detection performance, detect probability is not a correlation coefficient but rather a nonparametric measure of this relationship that corresponds to the probability that a randomly chosen spike rate from a correct trial is greater than a randomly chosen spike rate from a missed trial. We chose to compute detect probability to facilitate comparison with previous studies describing correlations between neural responses and behavior. However, using other metrics to express the link between neural activity and behavioral performance, such as d′ or differences in median spike rate, produced very similar results.
To determine the effect of microsaccades on detect probability, we calculated the detect probability twice—with all trials included and with only those trials that did not have a microsaccade near the time of the test-stimulus onset (Fig. 7). We used the same windows to count spikes and microsaccades as those used in the ROC analysis of the effect of microsaccades on neural activity (see above). These windows are indicated by the black and gray horizontal bars in Figure 6. Since the probability of a microsaccade in these windows was small on any given trial, we required a sufficient number of correct and failed trials to measure the effect of microsaccades on detect probability. Thus we only included neurons with at least 15 correct and 15 failed trials regardless of whether microsaccades occurred.
To express the change in detect probability as a percentage, we first subtracted 0.5 from both detect probabilities to “zero” the values. We were mindful that detect probability is limited to values between 0 and 1, and is consequently highly nonlinear for values near 1. However, over the range of detect probabilities in our data (0.55–0.65), the detect probability (minus 0.5) is proportional to d′ or the difference in median spike rate and the percentage-change values reported here are equivalent for any of these metrics.
In all three cortical areas and detection tasks, we found that removing trials containing microsaccades reduced the detect probability (Fig. 7). For the speed-pulse experiment, eliminating trials with microsaccades reduced the DP by 19% for MT and by 21% for LIP (Fig. 7A, B) (MT, all trials, mean DP = 0.558, trials without microsaccades, mean DP = 0.547, p = 0.002; LIP, all trials, mean DP = 0.584, trials without microsaccades, mean DP = 0.566, p < 0.001). In the motion-step experiment, the decrease in detect probability was 19% in MT (Fig. 7C) (all trials, mean DP = 0.588, trials without microsaccades, mean DP = 0.571, p = 0.016). However, using the windows defined above, we did not find any change in the detect probability for area VIP (Fig. 7D) (all trials, mean DP = 0.632, trials without microsaccades, mean DP = 0.633, p = 0.89).
Given that microsaccades suppressed neural activity in VIP during coherent motion, we were surprised that eliminating microsaccade-containing trials did not alter the detect probability. VIP is known to exhibit strong responses correlated with perceptual choice, which may be resulting from feedback (Cook and Maunsell, 2002b). One possibility is that this feedback could have overwhelmed the effect of microsaccades on neural activity. We thus repeated the detect probability calculation for VIP using a spike window of 50–150 ms after motion onset and a microsaccade window from 200 ms before 50 ms after motion onset. We reasoned that truncating the spike window at 150 ms would reduce the potential contribution of feedback on the detect probability. Using these new windows, eliminating trials with microsaccades reduced the detect probability in VIP by 17% (Fig., 7E) (all trials, mean DP = 0.558, trials without microsaccades, mean DP = 0.548, p = 0.035).
Last, in the motion-pulse experiment eliminating trials with microsaccades reduced the detect probability by a relatively smaller yet still significant 7% in area MT (Fig. 7F) (all trials, mean DP = 0.560, trials without microsaccades, mean DP = 0.556, p = 0.002). Although detect probability decreased for all experiments when microsaccades were removed, the reduced values were still all significantly >0.5 (p < 0.001 for all experiments and areas).
One possibility is that the observed decrease in detect probability is solely the result of reducing the number of trials in the analysis and not a specific effect of removing trials with microsaccades. In theory, reducing the number of trials should not bias the outcome one way or another, although it will reduce the reliability of the measurement. To confirm this assumption, we performed a resampling analysis. For each neuron, we calculated the detect probability after eliminating a set of random trials equal to the number eliminated in the original microsaccade elimination analysis. We repeated this calculation 10,000 times for each neuron and took the mean value. As expected, eliminating the trials did not bias our population estimate of detect probability for any of the experiments (range of mean change in DP, −2 × 10−5 to 4 × 10−4).
We have shown that the paired effect of microsaccades on neural activity and behavior can contribute up to 20% of the observed correlation between the two. This suggests that microsaccades are a significant source of behaviorally relevant neuronal covariance in our tasks. It does not necessarily follow, however, that microsaccades are a significant source of neuronal response variance in general. In fact, previous studies addressing this question have produced conflicting results (Gur et al., 1997; Bair and O'Keefe, 1998). To quantify the contribution of microsaccades to spike-rate variance we calculated the variance-to-mean ratio (Fano factor) for correct trials with and without microsaccades. The time windows for detecting microsaccades and measuring the neural response were the same as those described above. For the speed-pulse task, elimination of trials with saccades produced a small but significant reduction in Fano factor for both MT (Fano factor ± SE, all trials, 1.27 ± 0.07, trials without microsaccades, 1.26 ± 0.06, paired t test p = 0.015) and LIP (all trials, 1.213 ± 0.055, trials without microsaccades, 1.207 ± 0.055, p = 0.002). For the motion-step task there was a small, but not significant change for MT (all trials, 1.30 ± 0.09, trials without microsaccades, 1.23 ± 0.08, p = 0.19) and for VIP (all trials, 1.49 ± 0.09, trials without microsaccades, 1.47 ± 0.09, p = 0.41). The result was similar for the motion-pulse task in MT (all trials, 1.73 ± 0.15, trials without microsaccades, 1.72 ± 0.15, p = 0.25).
We examined the effect of microsaccades on neural activity and behavioral performance in three motion-based detection tasks and estimated the contribution of microsaccades to the correlation between neural firing and behavior. We found that microsaccades were associated with significantly reduced detection performance for transient changes in speed or coherency. Microsaccades were also associated with slower response times. Additionally, in all three experiments, we found that microsaccades suppressed neural activity during stimulus presentation. All together, the dual effect of microsaccades on neurophysiology and behavior contributed 7–19% of the detect probability in area MT, 21% in area LIP, and up to 17% in area VIP, depending on the time windows used for the analysis. The observation that neural activity and behavior are correlated has profoundly shaped our view of visual system function. Our observation that microsaccades can account for up to one-fifth of this correlation between neural activity and behavior is surprising and may have implications for how these data are used to constrain models of the neural signals underlying sensory perception.
Low amplitude, brief events such as microsaccades are inherently difficult to detect and characterize. Although we believe our detection algorithm was robust, it is possible that some microsaccades went undetected or that non-microsaccade eye-movement signals were misclassified as microsaccades. Critically, neither type of error would be expected to produce our findings. For example, when calculating the detect probability we separated trials based on whether they contained a microsaccade near stimulus onset (Fig. 6). If the algorithm missed a microsaccade, and accidentally included the trial in the no-microsaccade group, the detect probability of that group should increase, reducing our estimate of microsaccades' contribution to the detect probability. Alternatively, if the algorithm improperly classified an event as a microsaccade (akin to randomly eliminating a trial from the no-microsaccade group), this would not bias our estimate of the microsaccade-free detect probability in either direction, although it would make our estimate of the underlying value less reliable. Therefore, with respect to errors in microsaccade detection, our estimate of the contribution of microsaccades to detect probability could be considered a lower bound.
However, we do not believe that we greatly underestimated the contribution of microsaccades to the detect probability. Our velocity thresholds for detecting microsaccades (between 8 and 10 degrees/s) were chosen to be consistent with past studies (Bair and O'Keefe, 1998; Leopold and Logothetis, 1998), and the rate of microsaccades in our three experiments (0.3–1.3 Hz) was in the range of previous observations (Table 1). Additionally, shifting these thresholds more than a couple of degrees/s only reduced the contribution that microsaccades made to the detect probability (data not shown). Therefore, we do not believe that we have grossly underestimated the contribution of microsaccades to the correlation between neural activity and behavior.
The effect of microsaccades on detect probability varied among the different experiments and cortical areas: we found the strongest effects for MT and LIP in the speed-pulse experiment as well as MT for the motion-step experiment, differing effects on VIP depending on which time windows were considered and a small effect for area MT in the motion-pulse experiment (Fig. 6). However, these apparent differences should be treated cautiously. Undoubtedly, a main source of variability is that the data were collected from different animals in different laboratories using slightly different methodologies. While this experimental diversity generally underscores the robustness of our findings, it likely accounted for some of the variability in the findings. For example, the smaller effect on detect probability that we found for the motion-pulse experiment was most likely because eye positions were smoothed with a 20 Hz low pass filter that made it difficult to detect small, rapid eye movements. Nonetheless, despite experimental differences, it is remarkable that microsaccades contributed a relatively consistent fraction of the detect probability in all three cortical areas.
For VIP during the motion-step experiment, microsaccades contributed almost nothing to the detect probability when counting spikes from 50 to 300 ms after the onset of coherent motion, but contributed ~17% when spikes were only counted from 50 to 150 ms. One possible explanation is that decision-related “feedback,” which is prominent in VIP (Cook and Maunsell, 2002b), might have overwhelmed any effect microsaccades had on the detect probability by providing a much stronger source of behaviorally correlated neural activity at later time points. By limiting the window used to count spikes to immediately after test-stimulus onset, we likely reduced any potential role for feedback.
Subjectively, we do not perceive the world to move during microsaccadic eye movements. This suggests that there are compensatory mechanisms to counter the motion signals that microsaccades generate (Murakami and Cavanagh, 1998, 2001). This compensatory mechanism might interfere with a subject's ability to detect a motion stimulus near the time of the microsaccade, leading to the observed reduction in performance. However, studies of the impact of microsaccades on visual detection tasks have produced contradictory results, with some reporting increased detection thresholds (Ditchburn, 1955; Beeler, 1967) and others reporting no change (Krauskopf, 1966; Sperling, 1990). Unlike our experiments, these studies examined detection thresholds for flashed stimuli. It is possible that motion detection is specifically suppressed during microsaccades, which would be especially evident in the threshold motion-detection tasks used in our study. It would be interesting to examine the effects of microsaccades on neural responses and behavior in other visual tasks that do not employ moving stimuli.
We observed that microsaccades reduced detection performance and, on average, suppressed neural firing during our tasks. The two effects were correlated on a trial-by-trial basis, so that microsaccades contributed positively to the magnitude of the correlation between activity and behavior. Minimally, our findings serve as a caution that microsaccades should be taken into consideration when measuring correlations between neuronal activity and behavior. However, our specific results may not extend to all such experiments. For example, in our design we used test stimuli that were chosen to match the recorded neuron's preferred direction and we thus expected the test stimulus to trigger an increase in the neuron's response. Consider a hypothetical experiment in which motion in the preferred direction is followed by a test stimulus in a less-preferred direction. In such an experiment the neuron's expected response would be a decrease in firing. If, as we observed, microsaccades suppressed neural firing and decreased behavioral detection, we would instead expect microsaccades to decrease the magnitude of the correlation between activity and behavior.
There have been many studies with motion-based stimuli that measured choice probability in two-alternative forced-choice paradigms (Britten et al., 1996; Dodd et al., 2001; Grunewald et al., 2002; Williams et al., 2003; Huk and Shadlen, 2005; Liu and Newsome, 2005; Purushothaman and Bradley, 2005). Could microsaccades have made a similar contribution to these results? Few previous studies have considered this issue. Dodd et al. demonstrated that the direction of microsaccades did not correlate with the animal's choice, but did not assess whether they impact the animal's behavior in other ways (by degrading accuracy, for example). As our values of detect probability were in the range of those previously reported for choice probability, even if microsaccades made a similar contribution it is unlikely that they would have significantly altered the main conclusions of these studies. In theory, microsaccades could have had a more significant impact in other studies that reported smaller choice-probability values for V2 (Nienborg and Cumming, 2006) and V1 (Palmer et al., 2007). However, these experiments did not involve motion stimuli and one of these studies (Nienborg and Cumming, 2006) examined the possible effect of microsaccades (although in different a manner than ours) and found no contribution.
Thus it is not clear what impact microsaccades should be expected to have on choice probability. Although choice probability and detect probability are analogous quantities, there are critical distinctions in the associated paradigms. Consider a two-alternative forced-choice task in which the monkey must report whether low-coherence motion was in one of two opposed directions. In the case where there is no relation between microsaccades and the choice of the subject, microsaccades, regardless of direction, would not bias the subject toward one choice or the other. However, microsaccades would tend to decrease the measured choice probability by introducing spike-rate variance that was not correlated with behavioral choice. Alternatively, consider the case where the frequency or direction of microsaccades influences the subject's choice. This wouldbe the case if microsaccades produced a direction-specific neural response and a motion percept that biased the subject's choice depending on microsaccade direction. In this case, microsaccades would tend to increase the measured choice probability (assuming the neural and perceptual effects were aligned). One could easily imagine other scenarios where the effect of microsaccadeson choice probability would be task and stimulus dependent.
The observation that single-trial responses from individual neurons are correlated with behavioral choice almost certainly requires that trial-by-trial variance in neural responses are correlated across many neurons (Shadlen et al., 1996), which has been demonstrated through paired recordings (Zohary et al., 1994). This correlation may arise from many possible sources including intrinsic properties of the neural networks, variance in the stimulus, feedback activity related to choice, fluctuations in attention or vigilance or, as we have demonstrated here, eye movements. The nature of the causal link between this correlated neural response variance and behavior is the critical question common to all choice- or detect-probability results. For example, in our experiments it is possible that microsaccades cause parallel, unrelated effects on detection behavior and neuronal firing. It is also possible that behavioral effects of microsaccades are mediated in part or solely through the firing of neurons such as those recorded in this study. In this case, microsaccades might be considered a bona fide source of noise that influences behavior by modulating the firing of those neurons. While our experiment did not address this issue directly, our partial correlation analysis suggested that at least some of the effect of microsaccades on behavior was independent of the firing of single neurons. These questions point to the importance of better understanding the sources of variation in sensory encoding that lead to variation in behavior. Our findings suggest that microsaccades may be one such source of variation.
This work was supported by grants from the Canadian Institute of Health Research and the Natural Sciences and Engineering Research Council of Canada (E.P.C.). N.Y.M. was supported by Le Fonds québécois de la recherche sur la nature et les technologies. J.A.A. and T.M.H. were supported by two grants from the National Eye Institute, EY-12106 to J.A.A. and Vision Core Grant EY12196, and by a grant from the Medical Scientist Training Program to T.M.H. (T32 GMO7753-26).