|Home | About | Journals | Submit | Contact Us | Français|
Flexible links between sensory stimuli and behavioral responses underlie many cognitive processes. One process that contributes to flexible decision-making is categorization. Some categories are innate or overlearned, but in many cases category boundaries represent flexible decision criteria that can shift “on the fly” to adapt to changes in the environment. The ability to shift category boundaries allows decision-making to adapt to changing circumstances. Here we show that monkeys are able to switch rapidly between two category boundaries when classifying the speed of a moving dot pattern, and that neurons in monkey frontal eye field (FEF) change their activity when the boundary changes. The responses of a subpopulation of FEF neurons that were sensitive to both stimulus and boundary speed were used to classify the stimuli as accurately as the monkeys’ performance.
The FEF is a region of prefrontal cortex from which eye movements can be evoked by electrical stimulation with low currents (1, 2). The FEF has been shown to play a role in target selection for voluntary eye movements and spatial attention (3, 4, 5). Recently, FEF neurons have been found to have robust shape selectivity (6), as well as selectivity for direction and speed of motion (7). They can also exhibit selectivity for features such as color when they are linked to specific motor responses (8, 9). However, it is not clear whether frontal eye field has a role in functions that are thought to be specific to the domain of object vision, such as categorizing a stimulus independently of a specific saccadic eye movement.
Categorical decision-making is an important element of cognitive flexibility. Moveable category boundaries allow for flexible mapping between stimuli and responses. To investigate the role of FEF in categorical decision-making, we developed a speed categorization task in which monkeys were presented with a random dot motion stimulus and indicated whether the stimulus was moving “slow” or “fast.” The task was designed so that the speed categories were independent of direction of the eye movement response. The category boundary was determined arbitrarily and the monkeys learned it by trial and error. After monkeys had learned one boundary speed, the boundary was shifted to a new speed and the monkeys learned the new boundary. Once learned, monkeys were able to perform the task with two different boundary speeds, one of which was selected randomly on each trial.
We recorded from 96 FEF neurons in two monkeys to determine if their firing activity was affected by changes in the category boundary. Activity during stimulus presentation was significantly modulated by stimulus speed in roughly one-third of the neurons. However, over 40% of FEF neurons had a significant change in activity when the category boundary changed. There was a systematic relationship between stimulus and category preference; cells that responded better to fast stimuli also had higher firing rates on trials with a slower boundary speed. The converse pattern was found for cells that responded better to slower stimuli.
These results provide evidence that FEF activity is influenced by stimulus category and suggest a novel mechanism for categorical decision-making. Categorical decision-making is thought to involve the accumulation of sensory evidence toward a threshold (10). Little is known about how this process adapts to different contexts. The current results support the idea that modulation of response gain in neuronal subpopulations with different stimulus selectivities may be a mechanism for implementing context-dependent shifts in decision criteria.
To obtain evidence of flexible decision-making in monkeys, we asked whether a shift in the category boundary led to a change in the way monkeys classified the stimuli. Two monkeys performed the speed categorization task shown in Fig. 1 while we recorded neurons in their frontal eye fields. Behavioral data from all recording sessions (n = 96) are shown in Fig. 2. The average number of trials per session was 1254. The data in Fig. 2A,B show the proportion of trials for which the subject categorized the stimulus as “fast.” The small points show the percentage of “fast” choices for each session, while the large symbols are averages over all sessions. Both subjects were more likely to categorize speeds 6–16 deg s−1 as “fast” on trials with the slower boundary (yellow data points) as compared to the faster boundary (blue data points). The subjects’ behavior thus showed a dependence on boundary speed as well as stimulus speed, providing evidence for a shift in the internal reference used to categorize the stimulus. Note that for each boundary speed, the stimulus probabilities were adjusted so that on any given trial, the stimulus speed was equally likely to be drawn from the “slow” category as from the “fast” category (see Methods).
An estimate of the internal reference can be obtained by determining the “point of subjective equivalence” (PSE), i.e. the speed for which the monkey classified the stimulus as “fast” on 50% of the trials. This was done by fitting a smooth function to the percentage of “fast” choices and then finding the speed for which that function yielded a value of 50%. Fig. 2A,B show functions fit to the averages for all sessions. The data for each individual session were also fit, and the distributions of those PSEs are shown in Fig. 2C. The distributions were well-separated; the average difference between the PSEs was 3.9 deg s−1 (paired t-test, p < 10−10). Despite the apparent overlap of the distributions, there was only one session for which the PSE for slow-boundary (yellow) trials was greater than the PSE for fast boundary (blue) trials. Nevertheless, it is clear that the means of the distributions (dotted black lines) do not coincide with the actual category boundaries (yellow and blue dashed lines in Fig. 2C). Hence, the data show that the monkeys adjusted their internal reference speed when the actual boundary shifted, but fell short of ideal performance.
To determine if the frequency of boundary shifts affected performance, we compared overall percent correct for sessions in which the boundary changed once per ~100 trials (“blocked”, n=46 sessions), to performance during sessions in which the boundary was selected randomly on each trial (“random”, n=50 sessions). The average percent correct across all trials for blocked sessions was 72.7% vs. 73.4% for random sessions (t-test, p = 0.65). We also classified individual trials as “switch” trials if the boundary was different on the previous trial. Performance averaged over all trials was 73.1% correct, while performance on switch trials was 72.3% correct. These results show that the frequency of changing the category boundary did not affect the monkeys’ accuracy, and suggest that monkeys were able to shift their internal reference speed on a trial-by-trial basis.
To determine whether FEF neurons carry signals related to flexible decision-making, we analyzed the activity of 96 FEF neurons. The timecourse of neural activity around the decision period is shown for one neuron Fig. 3. The neuron had a robust response to the stimulus (Fig. 3A, dashed vertical line indicates stimulus onset). This response was strongly modulated by boundary speed (yellow lines = slow boundary, blue lines = fast boundary). Fig. 3B shows the distribution of spike counts during the first 200 msec of the stimulus presentation, sorted by boundary speed and stimulus speed. ROC analysis showed that, based on the spike count during the first 200 msec of stimulus presentation, an ideal observer would have been able to correctly guess the boundary speed on 74% of the trials (Fig. 3C).
Decision-period neural activity was quantified by computing the average firing rate during the interval between stimulus onset and the behavioral response. Data for two recording sessions are shown in Fig. 4. For one session (monkey C), the behavioral responses are shown on the left (Fig. 4A) and the neuronal responses are shown on the right (Fig. 4B). It is important to note that each data point in Fig. 4B represents an average across target positions (and hence saccade direction). In Fig. 4B, the firing rate was modulated by stimulus speed; the neuron fired more for faster speeds. The firing rate also depended on the position of the category boundary; across all speeds, the neuron was activated more strongly on trials with the slower boundary. Fig. 4C,D shows data for a second session (monkey F). The behavioral data (Fig. 4C) are similar to those in Fig. 4A. However, the neuron shows a complementary pattern of activity. It responded best to slower speeds, and activity was higher across all stimulus speeds for trials with the faster boundary speed.
The statistical reliability of the effects of stimulus speed and boundary position on firing rate for the neuron in Fig. 4B was tested with a two-way ANOVA and both effects were significant (p < 0.05). This neuron might be signaling not stimulus speed per se, but whether the monkey judged the stimulus as “fast” or “slow” relative to the category boundary (reference speed). We analyzed the activity of all 96 FEF neurons by performing a 4-way ANOVA on each cell with boundary speed, stimulus speed, outcome (correct/incorrect), and choice (fast/slow) as the explanatory variables (thus, there were two task-related EVs and two behavioral EVs). We found that many FEF neurons were significantly (p < 0.05) modulated by one or more variables: 45 neurons (46%) by boundary speed (p < 0.05), 36 (37%) by stimulus speed; 25 (26%) by outcome and 12 (13%) by choice (none of the 6 first-order interactions was significant, p<0.05, for more than 10% of the cells). Hence, more cells were significantly modulated by boundary and stimulus speed than by behavioral variables (outcome and choice).
The results suggested a systematic relationship between the effects of stimulus and boundary speed. For example, the neuron whose responses are shown in Fig. 4B had higher firing rates for faster moving stimuli and therefore could be considered a “fast preferring” neuron. Other cells (e.g. Fig. 4D) preferred slower stimuli. We noticed that the “fast preferring” cell also tended to respond more on trials with the slow boundary speed, and the “slow preferring” cell tended to respond more on trials with the faster boundary. To examine this effect at the population level, we used a linear regression model with boundary speed and stimulus speed as explanatory variables. The regression model attempted to fit the average firing rate on each trial (FRi) using the following equation:
Where Sb is boundary speed, Ss is stimulus speed, Nt is trial number, and k0, kb, ks, and kt are constants. The trial number regressor (Nt) was included to account for slow drifts in neuronal responsiveness that might be confounded with effects of boundary speed. This model provided a significant fit (standard regression of predicted vs. actual firing rate, p < 0.01) for 87/96 (91%) neurons. Note that the model estimates a single value of each constant (k0, kb, ks, and kt) for each neuron.
The 4-parameter model of Eqn 1 was compared to two 6-parameter models that included either 1) trial outcome (correct/incorrect) and category choice (fast/slow) or 2) target position and saccade direction. Adding these covariates improved the fit of the model marginally. However, it did not substantially affect the parameter estimates for boundary and stimulus speed. Details are provided in the Supporting Online Material.
The data of Fig 4B,D suggest that there might be a systematic relationship between the changes in activity due to stimulus speed, and those due to boundary speed; cells that prefer faster speeds (positive values of ks) appeared to fire more strongly on trials with the slower boundary speed (negative values of kb), and vice-versa. This effect should manifest in an inverse relationship between kb (the coefficient for boundary speed), and ks (the coefficient for stimulus speed). This trend was confirmed by the parameter estimates obtained by fitting the regression model to each cell. The correlation between kb and ks across the population was negative (r = −0.41, p < 0.0001, n = 96, Fig. 5E).
The estimates of ks obtained from the regression model were used to classify cells as “fast-preferring” (ks > 0.01) or “slow-preferring” (ks < −0.01) based on their stimulus preference (this excluded 42 cells for which the speed dependence was very close to zero). A total of 54/96 (55%) neurons were classified as either “slow-preferring” (n = 24), or “fast-preferring (n = 30). For this subset of neurons, we removed differences in average activity across cells by “de-meaning”, i.e. subtracting the mean firing rate during the decision period across all trials from the firing rate on each individual trial. We then averaged the de-meaned firing rates within each class of cell, sorting by stimulus and boundary speed. Even though the cells were classified as “fast” or “slow” based only on stimulus preference, the “fast” cells as a group had higher firing rates for slow boundary, and the “slow” cells for the fast boundary (Fig. 5A,B). This pattern of activity makes sense: on trials with the slow boundary speed, the monkey was more likely to categorize most stimulus speeds as “fast” and therefore cells that prefer faster speeds were more active. Similarly, on trials with the faster boundary speed, the monkey was likely to categorize most of the stimulus speeds as “slow”, hence slow preferring neurons were more active.
To demonstrate how the neural modulation shown in Fig. 5A,B can implement shifting decision criteria, we devised a simple algorithm for reading out the responses of the neurons to predict the animal’s behavioral response. The algorithm is based on the idea that fluctuations in a neurons’ firing rate represent a vote for one category or the other. If a neuron prefers faster stimuli, then its activity is interpreted as a vote to categorize a stimulus as “fast” on trials where its firing rate is significantly greater than its mean firing rate. However, if its firing rate is below average, then it votes to categorize the stimulus as “slow.” A complementary rule is applied to the activity of neurons that prefer slower moving stimuli. To implement this, we constructed, for each cell, the distribution of firing rates across all trials (including all stimulus and boundary speeds) and calculate the mean and s.d. of that distribution. Then, on any given trial, we compared the firing rate on that trial (FRi) to the mean rate (FR), scaling by the standard deviation of FR [i.e. (FRi − FR)/s.d.]. If that number was greater than a threshold, δ (which was the same for all neurons), then it represented a vote for the cell’s preferred speed (fast or slow). If the number was smaller than −δ, then it represented a vote for the opposite category.
To illustrate, the distributions of de-meaned firing rates for the population of slow-preferring neurons are shown in Fig. 5C along with the thresholds for voting “fast” or “slow.” Not every neuron voted on every trial. If a neuron’s firing rate on a given trial was within ±δ of its mean, it did not cast a vote on that trial. In addition to this “threshold” model, we also simulated a “proportional” model in which the probability of each response category was linearly proportional to the normalized activity of the cell.
We applied both algorithms to the subset of neurons whose average responses are shown in Fig. 5A,B. For the threshold model, the predicted proportion of “fast” choices for each boundary speed is shown in Fig. 5D as solid lines, while the actual behavioral choices for the same set of trials is shown as open circles. The correlation between the predicted and actual choice probabilities was strong and significant (r2 = 0.95, p<0.0001, n = 16). For the proportional model, the predictions are shown as dashed lines (r2 = 0.94, p < 0.0001, n = 16). Hence a subpopulation comprising 55% of FEF neurons and selected only on the basis of their stimulus selectivity, was sufficiently strongly modulated by category boundary position to quantitatively account for subjects’ behavioral choices.
It is important to note that the category selectivity of the neuron was independent of the eye movements that the monkey made to indicate his choices. The monkey signaled his choice by making a saccadic eye movement to one of two response targets. The targets were different colors (green or red) and the monkey learned that “fast” = green and “slow” = red. However, neither target was in the receptive/movement field of the neuron. Furthermore, the positions of the targets were randomized so that a given target color was not always associated with the same movement. A similar strategy was used previously (11) to distinguish perceptual decisions from motor responses in the superior colliculus.
A central issue in the neurobiology of decision-making is how sensory representations are transformed into categorical or “decision-based” representations. A categorical decision process is one that maps a continuous sensory input onto a finite number of responses in a many-to-one manner (i.e. multiple stimuli are associated with a single behavioral response; different classes of stimuli map onto different behavioral responses). Categorical decisions are closely linked with object recognition, abstract concept formation (12), and the development of language (13, 14). Through a combination of psychophysical (15) and neurophysiological approaches (16, 17), we are now beginning to understand the neural basis of categorical decision-making.
Visual categorization is often associated with the ventral visual processing stream, which includes visual areas of the inferior temporal lobe, and is involved in object recognition (18). This view is supported by several physiological studies (19, 20, 21) as well as the observation of category-specific agnosias following temporal lobe lesions (22). However, there is increasing evidence that the dorsal visual pathway might play a role in object recognition (23) and visual categorization (17).
The dorsal and ventral visual pathways both send anatomical projections to the frontal eye field (24). Frontal eye field (FEF) neurons have recently been found to have robust shape selectivity (6), as well as selectivity for direction and speed of motion (7). They can also exhibit selectivity for features such as color when they are linked to specific motor responses (8, 9). The current experiments suggest that frontal eye field may play a role in functions that are thought to be specific to the domain of object vision, such as categorizing a stimulus independently of a specific saccadic eye movement.
Categorization is closely linked to feature-selective attention. Recent work suggests that attention is drawn to informative features during categorization tasks (25). It is possible that our results reflect an enhanced representation of stimulus features that are relevant to the categorical decision. Moreover, it may be possible to not only attend to a particular feature (i.e. speed), but to limit attention to a subset of values within that stimulus dimension, i.e. to attend only to faster or slower speeds, and thereby to enhance the activity of cells whose selectivity is most relevant to the decision at hand.
Experiments were performed on 2 adult male rhesus monkeys (Macaca mulatta) weighing between 6 and 8 kg. All methods were approved by the Institutional Animal Care and Use Committee at Columbia University and the New York State Psychiatric Institute. Monkeys were prepared for experiments by surgical implantation of a post used for head restraint and a recording chamber to give access to the cortex. Eye position was recorded using a monocular scleral search coil. All surgical procedures were performed using aseptic technique and general (isoflurane 1–3%) anesthesia. Monkeys were trained to sit in a primate chair for the duration of the experiment with their heads restrained and perform the behavioral tasks. Correct performance of the task was reinforced by liquid reward.
Visual stimuli were generated and controlled by a Cambridge Research Systems VSG2/3F video frame buffer. The output from the video board was displayed on a calibrated 37 in. color monitor (Mitsubishi) with a 60 Hz non-interlaced refresh rate. The monitor stood at a viewing distance of 24 in. so that the display area subtended roughly 40 deg. horizontally by 30 deg. vertically. The spatial resolution of the display was 1280 pixels by 1024 lines. Visual stimuli used during the task consisted of the following: a 0.5 deg square white fixation target; a 1.0 deg circular yellow cue or square blue cue; an 8 deg diameter round patch of random moving dots; a 1.0 deg square red or green saccade target. All stimuli were presented on a uniform black background. The frame buffer was programmed to send out digital pulses (frame sync) for timing purposes at the beginning of each video frame in which a target was turned on or off. These pulses were recorded by the computer using a hardware timer, and stored together with the neuronal and eye movement data.
Eye position was monitored using a monocular scleral search coil system (CNC Engineering). The eye position signals were then digitally sampled by computer at 1 kHz per channel and digitized with 12-bit resolution, and stored on a disk for offline analysis. Velocity was computed from eye position information using a differentiating filter algorithm. Eye position and velocity were used to estimate saccade parameters. Saccade onsets and offsets were computed using an acceleration criterion.
Recording chambers (20 mm diameter) were implanted on the skull overlying the arcuate sulcus, positioned at stereotaxic coordinates 25A, 15L. At the start of each recording session, a hydraulic microdrive (Kopf) was mounted on the recording chamber. Recordings were made using platinum-iridium electrodes with impedances of 0.1 – 1 Mohm. Signals from the microelectrode were amplified, filtered and monitored on an oscilloscope and audio monitor. A time-amplitude window discriminator converted extracellular action potentials into digital pulses (TTL), which were sampled by the computer with 0.01 ms time resolution. Units were isolated on the basis of waveform. When a unit was isolated, stimulus parameters such as position and size of the moving dot pattern were adjusted to optimize its response. Neuronal spike trains were collected and stored along with eye position data.
Electrical microstimulation was used to map the region of cortex from which neuronal recordings were obtained in each monkey. Sites in peri-arcuate cortex were stimulated through the same electrode used to record neuronal activity. The stimulation consisted of a train of 0.2 msec biphasic pulses at a rate of 350 pulses/sec delivered by an optically isolated pulse stimulator (AM Systems). The output of the stimulator was gated by a computer generated TTL level so as to be synchronized with other trial events. The current threshold for evoking saccades was determined by stimulating during a fixation task (26). The threshold was defined as the current level at which involuntary saccades were evoked on about half the stimulation trials (2). The mean threshold was 43 uA.
For all sites, electrically evoked saccades were almost always contraversive and showed a mediolateral gradation of amplitudes (2). In addition, the evoked saccade direction rotated systematically as the depth of the electrode changed. These features of the saccade amplitude and direction map are characteristic of the FEF. A structural MRI for one monkey is provided in the Supplemental Material.
After collecting data for the memory-guided saccade task, we switched to the speed categorization task (Fig. 1). In this task, subjects viewed a random dot motion stimulus and categorized its speed as “slow” or “fast”. The response was indicated by making a saccadic eye movement to either a red or green response target. Subjects learned to associate the response category with the target color (“slow” = red; “fast” = green). A correct response was reinforced with a few drops of water or fruit juice and a high auditory tone. An incorrect response was signaled with a low tone.
The stimulus speed was selected at random on every trial from a set of eight speeds (2, 4, 6, 8, 10, 12, 14, 16 deg s−1). Subjects classified the speed of motion as “slow” or “fast” according to boundaries (reference speeds) that were learned by trial and error. The subjects learned two boundaries (Fig. 1B). One boundary was between 4 and 6 deg s−1 (“slow” reference), the other was between 12 and 14 deg s−1 (“fast” reference). On any given trial, only one boundary was used, and the boundary was either fixed for a block of trials or selected at random on every trial. In 46 recording sessions, the category boundary was changed roughly once per 100 trials (“blocked” condition). In 50 sessions, the category boundary was selected randomly on each trial (“random” condition). The boundary was indicated by a cue presented at the beginning of the trial. A yellow circle indicated that the slow reference was in effect and a blue square indicated the fast reference.
Subjects indicated their categorical decision (“slow” or “fast”) by making a saccadic eye movement to one of two response targets. The targets were red and green and the subject learned the rule that “slow” = red and “fast” = green. The positions of the response targets were randomized so that there was no systematic relationship between the “slow” and “fast” categories and the direction of the eye movement.
It should also be noted that the boundary positions split the stimulus set into unequal parts; for each boundary, there were two stimulus speeds in one category and six in the other. This potentially affects the prior probability of each stimulus category. For example, given the slow reference, the subject might be able to respond “fast” on every trial, regardless of the stimulus, and be correct 75% of the time. To bring the guessing rate back down to 50%, we altered the stimulus probability. For the “slow” reference, speeds of 2 and 4 deg s−1 were presented three times as frequently as 6, 8, 10, 12, 14 and 16 deg s−1. For the “fast” reference, 14 and 16 deg s−1 were presented three times as frequently as the other speeds. Hence, for each reference speed, both response categories were equally likely to be correct. Over all trial conditions, the extreme speeds (2, 4, 14, 16 deg s−1) were presented twice as often as the intermediate speeds (6, 8, 10, 12 deg s−1).
The task timing was as follows: The subject had 800 ms to acquire the initial fixation target. The cue was then presented for 800 msec. Immediately after the cue, the motion stimulus and response targets were presented for 800 msec; this was the “decision” period. The subject could respond at any time during this interval, but the reward was not given until after the decision period had elapsed. Hence, the subject could not speed the reward by responding more quickly.
The geometry of the display was such that the response targets were always presented to either side of the random dot stimulus. The direction of dot motion was aligned with the axis orthogonal to the axis defined by the response targets. Both directions of motion along this axis were used and the direction was chosen randomly on each trial.
The speed categorization task had 64 conditions comprising different combinations of stimulus direction and speed, boundary speed, and response target position. Except for the reference speed, all stimulus conditions were presented interleaved randomly within a block of trials.
For analysis of neural activity, each trial of the speed categorization task was divided into four time epochs: (1) fixation interval: 100 ms before cue onset; (2) cue interval: 800 ms after the onset of the cue and prior to the stimulus presentation; (3) decision interval: the time after stimulus and response target onset and prior to the onset of the choice saccade; (4) post-saccadic interval: 100 ms after the end of the saccade. The average firing rate was computed within each time window. The number of repetitions of each trial condition was typically 10 or more. For the purposes of this paper, only neuronal activity during the decision interval is considered.
This research was supported by NIH-MH59244, Gatsby Institute, and NARSAD. We would like to thank Jeffery Schall, Roger Ratcliff, Franco Pestilli, Jack Grinband, Tobias Teichert, Crista Barberini, and Matthew Phillips for comments on a preliminary version of this manuscript.
Author ContributionsV.P.F., M.Y. and C.C. designed the experiments, analyzed the data, and wrote the manuscript. M.Y. and C.C. carried out the experiments.