|Home | About | Journals | Submit | Contact Us | Français|
We examined frontal cortex activity as monkeys performed a duration-discrimination task. Two stimuli, one red and the other blue, appeared sequentially on a video screen—in either order. Later, both stimuli reappeared, and to receive a reward the monkeys had to choose the stimulus that had lasted longer during its initial presentation. Some neurons encoded stimulus duration, but a larger number of cells represented their relative duration, which was encoded in three ways: whether the first or second stimulus had lasted longer; whether the red or blue stimulus had lasted longer; or, less commonly, as the difference between the two durations. As the monkeys’ choice approached, the signal encoding which stimulus (red or blue) had lasted longer increased as the order-based signal dissipated. By representing stimulus durations and relative durations—both bound to stimulus features and event order—the frontal cortex could contribute to both temporal perception and episodic memory.
Temporal perception plays an important part in the life of primates, especially people. Calculations about a traffic light’s duration, for example, carry as much weight as perception of its color and cognizance of the conduct it commands. Although no sensory receptors or cortical fields function solely in time perception, temporal factors underlie central aspects of perception, movement, sequences of perceptions and actions, and attempts to anticipate events. Temporal information thus transcends the sensory and motor systems of the brain. We can perceive the durations of acoustic and visual stimuli in common terms and have little difficulty integrating the times when sounds, sights, and actions occur. This feature of temporal information processing is indispensable to both an integrated perception of time and the temporal component of episodic memory.
Neuropsychological (Harrington et al., 1998; Mangels et al., 1998) and neuroimaging (Onoe et al., 2001; Rao et al., 2001) studies have implicated the frontal cortex in temporal perception, along with the cerebellum, basal ganglia, and posterior parietal cortex (Lejeune et al., 1997; Maquet et al., 1996; Nenadic et al., 2003). Onoe et al. (2001), for example, observed a timing deficit after injecting bicuculline, a GABAA antagonist, into the dorsolateral prefrontal cortex (PFdl) of monkeys. Accordingly, the present study focused on temporal information processing in the frontal cortex.
Despite the neuropsychological and neuroimaging studies, the neural mechanisms of temporal perception remain poorly understood. Since Niki and Watanabe (1979) first suggested that cortical neurons encode event durations, analyses of temporal processing have been reported for neurons in both parietal (Janssen and Shadlen, 2005; Leon and Shadlen, 2003) and frontal cortex (Genovesio et al., 2006; Lebedev et al., 2008; Lucchetti and Bon, 2001; Mita et al., 2009; Ohmae et al., 2008; Oshio et al., 2006; 2008; Brody et al., 2003; Roux et al., 2003; Sakurai et al., 2004; Tsujimoto and Sawaguchi, 2007). Like psychophysical studies on temporal perception, several of these neurophysiological experiments—including the present one—involved comparing the durations of two stimuli. None of the previous studies, however, simultaneously assessed each of three issues underlying such a comparison: (1) whether frontal neurons encode the duration of each stimulus or their relative duration; (2) whether, for relative duration, frontal neurons encode which stimulus lasted longer or their difference; and (3) whether frontal neurons encode duration based the order in which stimuli appeared or based on the features of a stimulus. Traditional psychophysical paradigms preclude the simultaneous assessment of these three issues because they typically involve a standard first stimulus that subjects compare with a second, variable stimulus. The present task, in contrast, allowed us to address all three issues simultaneously by freely interchanging two stimuli, their order of presentation, and their durations.
Two rhesus monkeys performed a duration-discrimination task. On each trial, the monkey viewed two visual stimuli, presented sequentially on a video screen, and later pressed a switch to report which of them had lasted longer. The monkey sat before three switches within arm’s reach (Figure 1A1–7).
The monkey began each trial by touching the central switch, which produced a fixation spot at the center of the video screen (Fig. 1A1). The monkey then fixated that spot and continued doing so until the “Go” signal, much later in the trial (Fig. 1A6). After fixation began and a pre-stimulus period elapsed, two visual stimuli appeared in succession at the fixation point, a blue circle and a red square, in either order. The first stimulus, called S1, lasted 100–3200 ms (Fig. 1A2). A delay period, called D1, followed (Fig. 1A3). After the D1 period ended, the second stimulus (S2) appeared (Fig. 1A4). Its duration always differed from that of S1 and ranged from 50–2400 ms according to two schemes, explained below. After S2, a second delay period (D2) usually followed (Fig. 1A5), although in one-third of the trials no such delay occurred. The red and blue stimuli then reappeared, one to the left of the screen center, the other to the right (Fig. 1A6), in either configuration. This event served as the “Go” signal, and to receive a reward the monkey had to touch the switch below the stimulus that had lasted longer on that trial (Fig. 1A7). Contact with the incorrect switch terminated the trial without reward. Although the monkey had 6 s to touch the left or right switch, in practice each one did so in less than 500 ms (Suppl. Fig. 1).
Note that the monkey could not select its motor response during either the S2 or D2 period because there was no way of knowing where the longest-lasting stimulus would appear: left or right (Fig. 1A6). At those times, the monkey could decide whether the red or blue stimulus had lasted longer, but not whether a leftward or rightward hand movement should report that decision. Therefore, the neural activity recorded prior to the “Go” signal could not reflect the ultimate choice of either the left or right response target or its motor concomitants.
Stimuli occurred in two sets of durations, termed the ‘square’ and ‘V’ sets because of their appearance on scatter plots (Fig. 1C). Each block of trials used one and only one stimulus set. For the ‘V’ set (Fig. 1C, right), S2 had an equal probability of being twice or half as long as S1—for all S1 durations less than 1.6 s. This means that—except for trials with S1 durations of 1.6, 2.4, or 3.2 s—the monkeys could not predict whether S2 would be longer or shorter than S1 based on S1’s duration. (Although these three long S1 stimuli were used for the recording sessions, the behavioral and neurophysiological analysis presented here excluded them, which eliminated prediction or classification confounds. For example, when S1 lasted 2.4 s, the monkey could have classified S1 as “long”, predicted that S2 would be shorter, and decided on a blue or red choice before S2 appeared). A disadvantage of the ‘V’ set resulted from its limited range of duration differences. The ‘square’ set of durations (Fig. 1C, left) provided that finer gradation, but permitted the monkey to predict the likely relative duration of S2 prior to its occurrence. For example, when S1 lasted 400 ms, 80% of S2s lasted longer.
Figure 1D shows that for each S1 duration in the ‘square’ set, the probability of the monkey reporting that S2 was longer increased with S2 duration. Overall, both monkeys performed the task accurately, with better scores and faster reaction times for easier discriminations (Suppl. Fig. 1). Monkey 1 performed at a mean of 81% correct for the ‘square’ set of stimulus durations and 80% correct for the ‘V’ set. Monkey 2 scored 80% and 77% correct, respectively. Suppl. Fig. 2 gives behavioral results for the ‘V’ set, and Suppl. Fig. 3 shows fits to a psychonometric (pseudologistic) function for the ‘square’ set.
The neuronal sample comprised 1720 neurons:509 from monkey 1 and 1211 from monkey 2. We used the ‘V’ set of durations for 511 cells and the ‘square’ set for 1209 cells. Because we used both sets for only 110 cells (6%), no separate analysis of this small subpopulation was attempted. The largest sample, 1286 cells, came from the periarcuate cortex (PA), and 434 neurons came from PFdl (Fig. 1B).
We measured activity during the S1 and S2 periods (from 200 ms after stimulus onset until stimulus offset, except where noted), during the D1 and D2 delay periods (from 80–400 ms after stimulus offset), and during the choice and action period (from the “Go” signal until contact with a reporting switch).
During the D1 period of each trial, the monkey needed to remember the duration of S1 in order to compare it later with S2. Many frontal cells encoded the duration of S1 during the latter part of the S1 period, during the D1 period, or both.
In all, a quarter of the neuronal sample showed a significant effect of S1 duration on D1-period discharge rates by ANOVA (437 of 1720 cells, 25% in PA and 26% in PFdl). Figure 2 shows two examples, and Suppl. Table 1 gives the breakdown by cortical area and stimulus set. To measure S1-period activity, we used trials with S1 durations of 600 ms or more, in order to exclude short latency visual responses. About a fifth of the sample showed duration effects during the last (333-ms) part of the S1 period (301 of 1720 cells, 18% in PA and 19% in PFdl). Some of the neurons selective for the duration of S1 in the D1 period also encoded its color (red or blue), as well as interactions between stimulus color and duration (Suppl. Table 1). In accord with the results from ANOVA, between a fifth and a quarter of the sampled neurons had significant linear correlations between D1-period activity and S1 duration (Suppl. Table 2).
Duration coding during the S1 and D1 periods displayed certain systematic interrelationships. As shown in Figure 3A, neurons with activity rates that correlated positively (β>0) with stimulus duration during the latter 333 ms of the S1 period, continued that significant positive correlation into the D1 period (blue points, upper right quadrant). Figure 2A shows an example of such a neuron. Similarly, neurons with negative correlations (β<0) between late S1-period activity and S1 duration continued to show a significant negative correlation in the subsequent D1 delay period (blue points, lower left quadrant). About a third (38%) of the cells encoding S1 duration during the D1 period also did so during the last part of the S1 period (101 of 264, 38% in PA; 38 of 104, 37% in PFdl). The discharge rates of other neurons (red points in Fig. 3A) correlated with S1 duration only after its offset, and Figure 2B shows an example.
Trends in S1-period activity (i.e., ‘declining’ or ‘climbing’ discharge rates) often predicted duration coding during the subsequent D1 period. To document this property, we calculated two indices for each neuron, based on a comparison of discharge rates during early (E, 200–533 ms), middle (M, 533–866 ms) and late (L, 866–1200 ms) segments of S1 periods of 1200 ms. The first index measured declines or increases from the early to the middle parts of S1 (M – E), normalized to the mean activity rate; the other measured activity trends from the middle to the late parts of S1 (L – M), normalized in the same way. Cells with negative correlations (β<0) between D1-period activity and S1 duration (green bars) tended to have declining activity (indices<0) during the preceding S1 period, whereas cells with positive correlations (red bars, β>0) tended to “ramp up” their activity (indices>0) during the S1 period (Fig. 3B). The indices of the populations with positive and negative correlations (red vs. green bars) differed from each other in both PA and PFdl (Kolmogorov-Smirnov two-sample test, p<0.001).
The relatively large number of durations used for S1 allowed us to evaluate duration tuning during the D1 memory period, which we classified by the method of polynomial contrasts. This method adds power terms for the independent variable, S1 duration (d), to a model to test whether d2 and d3 factors, in that order, significantly increase the variance accounted for. Of the neurons with duration effects, 29% had a significant linear relationship without significant quadratic or cubic relations, like the example illustrated in Figure 2A. Another 26% of the duration coding neurons had a significant quadratic relationship but not a cubic one (like the cell in Fig. 2B), and 31% had a significant cubic regression. Suppl. Table 3 gives the breakdown by cortical area and stimulus set. Thus, although many frontal cortex cells showed a significant linear relationship between discharge rate and stimulus duration, a larger number had better fits to nonlinear functions. Correlation coefficients for linear, quadratic, and cubic regressions were similar in PA and PFdl, accounting for 10%–25% of the variance in discharge rates (Suppl. Table 4).
Note that for the ‘V’ distribution of stimuli (Fig. 1C, right), the monkey could not predict whether an upcoming S2 stimulus would last half as long as S1 or twice as long, at least not for any of the data that contributed to the present analysis. On that basis, we can be confident that the correlations described so far reflected temporal information about S1 rather than predictions about S2, at least for the cells tested with the ‘V’ distribution (see Suppl. Tables 1–3 for the numbers tested). Suppl. Figure 4 illustrates this point explicitly for a PA neuron: its activity during the D1 period reflected a correlation with the duration of the previous stimulus (S1)—higher discharge rates after longer stimuli—without reference to any prediction about the upcoming one (S2). Likewise, the monkeys could not perform the task correctly by categorizing S1 as ‘short’ or ‘long’. Instead, the monkeys had to remember the duration of S1 in order to compare it to an unpredictable S2 duration that came later. The finding of duration coding during the D1 period indicates that PA and PFdl neurons encoded the duration information needed for later comparison and maintained it in short-term memory. Many frontal neurons encoded the duration of the red stimulus, others encoded the duration of the blue stimulus, and still others encoded the duration of the S1 stimulus regardless of color (Suppl. Table 1).
During the S2 and D2 periods, many frontal neurons conveyed information about the duration of S2, often linked to its color. These properties resembled the encoding of S1 duration during the S1 and D1 periods, and Suppl. Figure 5 shows an example neuron. This cell had activity at the end of the S2 period that correlated with S2 duration, and this property continued into the D2 period. Like Fig. 3B for the S1 and D1 periods, Suppl. Figure 6 shows for the S2 and D2 periods that neurons with climbing activity during the stimulus period had a positive correlation between activity and stimulus duration during the subsequent delay, whereas neurons with declining activity during the stimulus had a negative correlation.
Although duration encoding during the S2 and D2 periods resembled that during the S1 and D1 periods, these properties were usually observed in different neurons. Only 15% of the cells that encoded S1 duration did so for S2, and only 25% of the cells that encoded S2 duration did so for S1. Along with indicating largely separate processing channels for S1 and S2 duration, this result shows that the neural activity reflecting stimulus duration in D1 and D2 did not simply reflect low-order visual responses, such as “off” responses or rebound effects, which should not differ for S1 and S2, especially when they are separated by a minimum of 400 ms (see Materials and Methods).
Suppl. Table 5 shows the results of an analysis for the D2 period in the format of Suppl. Table 1 for the D1 period. Like duration coding during the D1 period, neurons during the D2 period encoded stimulus color as well as interactions between a duration factor and color. Suppl. Table 5 is not exactly analogous to Suppl. Table 1, however, because the duration factor is more complex for D2 than for D1. For the ‘square’ distribution of stimuli, a long S2 period meant that there was also a greater probability that S2 was the relatively longer of the two stimuli. Accordingly, some of the duration effects in Suppl. Table 5 could reflect the relative duration of S1 and S2 rather than simply S2 duration. (The ‘V’ distribution presented its own impediments to distinguishing stimulus-duration and relative-duration coding). In the next section, we use the ‘square’ set of durations to differentiate these and other duration factors by using a multiple, stepwise-regression procedure.
Relative-duration coding occurred in several forms. Most notably, many cells encoded whether the first (S1) or second (S2) stimulus had lasted longer, but not by how much. Others encoded whether the red or blue stimulus had lasted longer, but not by how much. Encoding of the magnitude of the difference was less common. Next, we present the results of ANOVA and multiple-regression analysis based on single-neuron data, followed by population-level measures.
In the overall neuronal sample, 30% of the cells showed activity that depended on whether S1 or S2 had lasted longer, regardless of whether the red or blue stimulus had appeared first (512 of 1720 tested cells, two-way ANOVA, 30% in both PA and PFdl). Suppl. Table 6 gives the breakdown by cortical area and stimulus set, and Figure 4A shows an example neuron. An additional 25% of the neuronal sample had D2-period activity that depended on whether the red or blue stimulus had lasted longer, regardless of their order (431 of 1720 tested cells, 26% in PA and 23% in PFdl), and Figure 4B shows an example of that property. In addition, a substantial proportion of cells (12–22%) showed combinations of these properties (Suppl. Table 6), and Figure 5 shows an example of such a combination. This cell showed a preference for trials with a longer S2 stimulus (or, equivalently, a shorter S1 stimulus) early in the D2 period (green arrows), but ~200 ms into the D2 period the cell switched (blue arrows) to a preference for trials with a longer blue stimulus (or a shorter red stimulus).
A stepwise, multiple-regression analysis confirmed the results from ANOVA. This procedure factored out relative duration and the duration of the individual stimuli. We tested, on a cell-by-cell basis, the predictive value of four factors to evaluate whether they accounted significantly for the observed variance. (In practice, it was rare for more than one of the four factors to contribute significantly). Monte Carlo analysis evaluated whether the observed number of neurons for each factor occurred more frequently than expected by chance.
One multiple regression used four temporal variables as factors: S1 duration, S2 duration, the S1–S2 difference, and whether S1 or S2 had lasted longer. The results confirmed that frontal neurons significantly (p<0.05) encoded both the relative duration of S1 and S2 and the duration of S2 (Fig. 6A). S1 duration was not significantly represented (p=0.99 in PA and p=0.48 in PFdl).
A similar but separate analysis was performed for four different factors: blue duration, red duration, the blue-red duration difference, and whether the blue or red stimulus lasted longer. Frontal neurons robustly encoded whether the blue or red stimulus had lasted longer, with slightly above chance-level representation of the other factors (Fig. 6B).
A larger, 8-factor regression model combined the two 4-factor models into one, and obtained comparable results (Suppl. Fig. 7). This similarity was to be expected because stimulus color and presentation order were orthogonal factors (e.g., a longer S1 was equally likely to have been blue or red). We also evaluated whether the coding of relative duration was better captured by the difference between the two stimuli or their ratio, and Suppl. Figures 8 and 9 show that both predictors produced the similar results.
As illustrated in Figure 6B, more cells encoded which stimulus (blue or red) had lasted longer than encoded their differences in duration (χ2=9.6, p<0.05 in PFdl; χ2=47.0, p<0.001 in PA). Similarly, as illustrated in Figure 6A, more cells encoded whether S1 or S2 had lasted longer than encoded their difference, which was a significant difference in PA (χ2=37.1, p<0.001) but not in PFdl (χ2=1.7, p=0.19, n.s.)..
An extension of the multiple-regression analysis into the choice and action period (see Fig. 1A7) showed that, by this time during each trial, order-based duration coding was infrequent (Fig. 6C). The encoding of blue-stimulus duration, red-stimulus duration or their difference was also rare (Fig. 6D). All of these declines, compared to the D2 period, reached statistical significance (χ2 test, p<0.05), with the exception of neurons encoding S1 duration, which was at chance levels in both periods. A larger percentage of cells, about 21% in PA and 12% in PFdl, encoded whether the blue or red stimulus had lasted longer (Fig. 6D, rightmost pair of bars), which was the information that the monkeys reported during the choice and action period. This predominance was statistically significant for both PFdl (χ2=15.6, p<0.001) and PA (χ2=154.2, p<0.001).
We also quantified the strength of relative-duration coding at the population level and did so in two ways: ROC analysis and population averages. For computing ROC values for each neuron, we used mean firing rates just before and during the D2 period. The ROC values reflect the ability to decode a signal based on activity during a single trial, without being affected by a cell’s overall activity level or its dynamic range, with the area under the ROC curve serving as a measure of relative-duration selectivity. For each selected longer-blue-stimulus trial, for example, we compared its activity to two pools of trials, longer-blue-stimulus trials and longer-red-stimulus trials. A value of 0.5 corresponded to no selectivity and a value of 1 corresponded to a complete selectivity in which all longer-blue-stimulus trials had higher activity than any longer-red-stimulus trial or vice versa. A separate, but analogous analysis was based on the order of stimulus presentation using longer-S1 and longer-S2 trial pools.
Figure 6E and F show the results of the ROC analysis, based on trials with a D2 period of 800 ms. Approximately 300–400 ms into the D2 period, the encoding of whether S1 or S2 lasted longer decreased (Fig. 6F, green and blue curves) just as the encoding of whether the red or blue stimulus lasted longer reached a peak (Fig. 6F, brown and orange curves). Note that 400 ms after S2 offset was one likely time for a “Go” signal, even though that was not the case on these trials because they all had an 800-ms delay period. The individual cell contributions to this population ROC analysis can be appreciated from Figure 6E. The differences observed were statistically significant: ROC values 300–500 ms after S2 offset were significantly higher for red vs. blue comparisons than for S1 vs. S2 comparisons (Kruskal-Wallis test, p<0.05, for both PA and PFdl), but there was no significant difference at the time of S2 offset (±100 ms).
Figure 7 shows results based on population averages, which confirmed the ROC analysis. Figure 7A shows average activity, drawn from the same neurons and trials used for the brown and orange curves in Figure 6F. These neurons encoded whether the red or blue stimulus had lasted longer or their difference, which means that they had higher activity when one of these stimuli had done so. That was defined as the preferred stimulus-duration (black curves), with the remaining one defined as anti-preferred (gray curves). For correctly executed trials (solid lines), the cells showed their preference before cue offset and reached a peak ~300 ms after cue offset, followed by a plateau. (The reason for the signal’s appearance before cue offset is taken up, below). The preferred vs. anti-preferred activity levels differed significantly in the D2 period (two tailed t test, t223=16.1, p<0.001 in PA; t58=5.6, p<0.001 in PFdl).
Figure 7B comes from the cells and trials used for the blue and green curves in Figure 6F: neurons that encoded whether S1 or S2 had lasted longer or their difference. The cells’ preferred and anti-preferred averages differed significantly during the D2 period (t232=14.8, p<0.001 in PA, t69=6.3, p<0.001 in PFdl), but by a lesser amount and for a shorter time after S2 offset than the population illustrated in Figure 7A. In accord with the ROC analysis, the difference between preferred and anti-preferred stimulus-durations dissipated ~400–500 ms after S2 offset. Unlike the neural signal indicating whether the red or blue stimulus had lasted longer (Fig. 7A), the one indicating whether S1 or S2 did so did not persist throughout the entire 800-ms D2 period (Fig. 7B).
During error trials (dashed lines in Fig. 7), the monkeys reported incorrectly that the shorter stimulus had lasted longer. We sorted trials according to the preferred stimulus-duration on correct trials, regardless of which stimulus the monkeys ultimately chose. In the D2 period of error trials, the difference between the cells’ preferred and anti-preferred averages was significant (t223=7.6, p<0.001 in PA, t58=3.3, p<0.05 in PFdl), but encoded the wrong (anti-preferred) stimulus for that trial. This finding indicates that the neuronal population represented which stimulus the monkey reported as lasting longer, independent of whether it had done so. Figure 8A shows an example neuron. When the monkey performed correctly, this neuron preferred trials in which the blue stimulus had lasted longer (bottom pair of raster and spike density displays). On error trials, the cell preferred trials with shorter blue stimuli (top pair of displays), which accorded with the monkeys erroneous choice of the blue stimulus, even though the red stimulus had lasted longer.
Figure 7B shows error-trial data for cells encoding whether S1 or S2 had lasted longer. Unlike the activity for correct trials, on error trials the cells’ preferred and anti-preferred averages did not differ significantly (t232=0.9, n.s. for PA, t69=1.6, n.s. for PFdl), which indicates that on error trials this population failed to encode whether S1 or S2 had lasted longer and also failed to encode the monkey’s subsequent choice.
Figures 8A and 8B differ from previous illustrations in that the neural activity is aligned on what we call the decision point, which refers to the instant when an ideal observer could make a decision, not to when a decision was actually made. As illustrated by the bars above each raster, when S2 was shorter (right column), an ideal observer could not decide until S2 offset, for obvious reasons. When S2 was longer (left column), at some time during its presentation its duration exceeded that of S1. Thus, the decision point was either when S2 first became longer than S1 (left column) or at S2 offset when S2<S1 (right column). The cell illustrated in Figure 8A developed activity that reflected a longer blue stimulus at some time after the decision point, more abruptly for the discrete sensory event of S2 offset (right column) and more gradually for the less precisely marked event of S2 exceeding S1 duration (left column).
Figure 8B shows population averages aligned on the decision point. The signal indicating whether the red or blue stimulus lasted longer developed near the decision point in both PA and PFdl. (Note the 100-ms moving average used to smooth the curve). Soon after the decision point, the differences between the preferred and anti-preferred curves became significant (t223=6.4, p<0.001 for PA, t58=3.4, p<0.05 for PFdl, in the first 100 ms). This finding explains why relative-duration coding appeared during the last part of the S2 period, before S2 offset (Figs. 6F and and7).7). This signal came from trials in which S2 duration exceeded that of S1 some time during the S2 period.
We studied the representation of stimulus durations in the frontal cortex in a task that involved the comparison of two stimuli, S1 and S2, both followed by delay periods, D1 and D2, respectively. After the D2 period ended, the monkeys reported whether a red or blue stimulus had lasted longer, regardless of their order of presentation. Four main findings emerged: (1) frontal cortex neurons encoded stimulus durations based on both stimulus features (e.g., color) and their order of presentation; (2) they encoded both the durations of individual stimuli and their relative duration, also bound to features and order; (3) relative-duration coding mostly indicated which stimulus was longer or shorter, and less frequently indicated how much the difference had been; and (4) the signal that the red or blue stimulus had lasted longer became predominant as the time for a report approached (Figs. 6F and and7).7). In addition, we found that duration-encoding neurons had both linear and nonlinear tuning functions and that S1-duration signals dissipated some time before the end of S2.
A substantial subpopulation of cells encoded stimulus duration during the D1 period and showed climbing or declining activity during the S1 period. Cells selective for long stimuli tended to increase their activity during S1 and vice versa (Fig. 3B). These two groups of cells might function as accumulators, modulating activity for the elapsed duration during stimulus presentation. One modeling study suggested that climbing activity might lead to a phasic activity increase in neurons postsynaptic to these cells, which could read out accumulated temporal information (Durstewitz, 2004). The cells that encoded stimulus duration only after the stimulus had ended (i.e., only during the D1 period) could correspond to this readout signal.
Of particular note is the substantial population of neurons encoding whether the red or blue stimulus had lasted longer (Fig. 6B), especially its predominance during the choice and action period (Fig. 6D). This finding likely reflects the main task requirement: to receive a reward the monkeys had to report that information, without reference to the order of stimulus presentation. Although the monkeys did not report whether S1 or S2 had lasted longer, this information might have served as an intermediate step in the computations leading to a decision. Population analysis supports this idea: the signal encoding whether S1 or S2 had lasted longer dissipated as the D2 period progressed, while the signal encoding whether the red or blue stimulus had lasted longer reached a peak and maintained its strength (Figs. 6F and and77).
The finding that there was no significant representation of S1 duration during the D2 period also merits a comment. In a sense, this finding is not surprising: the S1 event had long since ended by the beginning of the D2 period, with both the D1 and S2 periods intervening, and knowledge about S1 duration was no longer needed to perform the task. Yet S2 duration was well represented during the D2 period, as was the relative duration of S1 and S2 and combinations of order- and feature-based coding (Suppl. Fig. 9A), all of which were also unnecessary for task performance. These findings suggest that the frontal cortex retained information about certain unnecessary factors for computational purposes, but dispensed with S1 duration some time during the S2 period.
Activity during error trials proved especially informative. During error trials, the population encoding whether the red or blue stimulus had lasted longer reflected the stimulus chosen by the monkey, not the stimulus that had actually lasted longer (Fig. 7A). In contrast to this result, the signal indicating whether S1 or S2 had lasted longer was absent on error trials (Fig. 7B). These results show that the two processes had distinctive properties, notwithstanding the finding that 30% of these neurons carried a combination of the two signals (Suppl. Fig. 9A).
The fixation requirement ruled out interpretations in terms of visual stimuli or their locations in retinocentric or extrinsic coordinates, and the fact that the monkeys could not formulate a motor plan until after the D2 period ended ruled out motor interpretations and attention to a movement target.
We used two sets of stimulus durations because of their specific advantages. The ‘square’ set had the advantage of a finer gradation of duration differences, which allowed an assessment of tuning functions and relative-duration coding that would not have been possible with fewer stimulus durations. The ‘V’ distribution addressed the issue of predictability. For all of the trials contributing to either the behavioral or neurophysiological analysis, each stimulus of a pair was always either half or double the duration of the other, with equal frequency. Accordingly, the monkeys could not predict the duration of the second stimulus on the basis of the first. Because all of the fundamental findings reported here were confirmed with the ‘V’ set of stimulus durations (see Suppl. Tables 1–6), it seems likely that for the ‘square’ set, as well, the observed correlations reflect information about previous durations rather than predictions about future ones.
Some of the activity during D1 and D2 appears, at first glance, to be in the nature of an “off” response or a rebound effect of the sort observed in the retina and in low-order visual areas (Duysens et al., 1996). The observation that the vast majority of frontal cortex neurons encoded duration after either S1 or S2, but not both, argues against simple, low-order visual responses for the most part. Furthermore, 38% of the cells that encoded stimulus duration during the delay period also did so before stimulus offset (Figs. 2A and and3A;3A; Suppl. Fig. 5). In the stimulus period, no rebound or offset effect can have taken place. Nevertheless, such duration-dependent rebound effects could contribute to the duration signal observed in frontal cortex and would constitute one among many mechanisms for encoding durations. It is to be expected that the frontal cortex takes advantage of that information in encoding stimulus durations. Also likely is some interplay between visual responses and duration coding, mostly for sub-second intervals, as suggested by psychophysical studies. This possibility might be addressed in future neurophysiological studies, for example by comparing the neural activity related to filled versus empty intervals (i.e., intervals bounded by two lights or tones).
Our previous study used a saccade task with three different delays to study the representation of elapsed time (Genovesio et al. 2006). Many PFdl neurons showed phasic increases in activity that depended on the duration of the preceding delay period. However, in that experiment the monkeys were not required to make any temporal judgments or report on stimulus duration. The present study, a temporal discrimination task, had that requirement and also expanded the explored cortical region to include PA.
Three previous neurophysiological studies investigated the activity of PFdl (and basal ganglia) neurons using a task like the present one (Chiba et al., 2008; Oshio et al., 2006, 2008). These investigators concluded that although cells in PFdl encoded whether S1 or S2 had lasted longer on a given trial, there was no coding of relative duration based on stimulus features or of duration differences (Oshio et al., 2006). We found, instead, that PFdl (and PA) neurons encoded all of these factors. Task differences probably account for these discrepancies. In their first report on PFdl (Oshio et al., 2006), these authors used stimuli that allowed S2 duration to be predicted from that of S1. “Long stimuli” ranged from 1.2–1.6 s, whereas “short stimuli” never exceeded 1.0 s. The absence of feature-based and duration-difference coding in their neural data probably reflects the absence of any task requirement to encode that information and remember it over the D1 period. Indeed, a decision could be made as soon as S1 ended. In a separate paper on the basal ganglia (Chiba et al., 2008), the same group of investigators used stimulus durations that differed from the ones they used in their first PFdl report (Oshio et al., 2006). In their basal ganglia experiment—as in the present study—the monkeys could not predict S2 duration from that of S1. Although Chiba et al. (2008) did not observe a relative-duration signal based on stimulus features, they did report some properties in the basal ganglia that they had not observed in their cortical study. Chiba et al. (2008), for example, found the coding of S1 duration in the D1 period, like the results reported here for frontal cortex. Contrary to the conclusions of Chiba et al (2008), the present results suggest that the differences between their PFdl and basal ganglia results more likely depend on task differences than upon any difference between PFdl and the basal ganglia.
Another previous study used a match-to-sample design to examine duration coding in PFdl (Sakurai et al., 2004). In contrast to the present results, Sakurai et al. reported that very few neurons encoded relative duration, called “comparison neurons” in their report. In our task, many PFdl (and PA) neurons encoded relative duration, as shown for example in Figure 6A–D. This discrepancy probably also results from task differences. As in the work of Oshio et al. (2006), the monkeys studied by Sakurai et al. could simply categorize stimuli as short or long, rather than encode S1 duration. Moreover, the use of only two durations (0.5 and 2.0 s) severely limited their analysis. Another factor could be their definition of comparison neurons, which in their view “seem to reflect comparison between the lengths of the sample [S1] and comparison [S2] stimuli”. But the basis for that conclusion is unclear from the data they present. Rather than performing comparison functions, these neurons could instead represent a stimulus-order signal, indicating that a particular stimulus occurred second, with no relevance to relative duration or any other timing information. These neurons resemble the rank-order neurons described in PFdl for a task requiring the memory of object order (Ninokura et al., 2004). And although not described here, we likewise observed many frontal cortex neurons with the order-encoding properties described by Ninokura et al. (2004)
Recently Mita et al. (2009) studied the neural mechanisms involved in the generation of specific time intervals, using a time-production task. Their monkeys released a key after an interval of time that was cued by a visual instruction. A majority of neurons in both the pre-supplementary and supplementary motor areas signaled the initiation of action in a time-selective manner. Future studies might evaluate the role of the neurons described by Mita et al. in perceptual versus motor tasks.
There have been three published neurophysiological studies of temporal processing in parietal cortex, all from the lateral intraparietal area (LIP). Leon and Shadlen (2003) focused mainly on activity during S2 presentation, in a task that required fixed spatial responses to report whether S2 durations were ‘shorter’ or ‘longer’ than one of two standard S1s. LIP activity during S2 predicted the probability of a given report, as calculated from psychometric response functions. Jansen and Shadlen (2005) found a representation of hazard functions, the combination of elapsed time and the probability that a ‘go’ signal was imminent. Maimon and Assad (2006) found a proactive timing signal that increased until it reached a threshold for movement. Other neuropsychological and TMS studies, focusing especially on the timing of visual events having durations similar to those used in our study, also point to a role of parietal cortex (Battelli et al. 2008). Unlike the tasks used in previous neurophysiological studies, the present one separated the motor response, along with its spatial and attentional components, from decisions about durations. Another importance difference is our emphasis on delay-period activity rather than activity during stimulus presentation, as is more common in visual neurophysiology. And unlike the parietal cortex studies cited above, the present task required a comparison of two variable durations instead of a simpler contrast between a test (S2) duration and one or two standard (S1) durations. Accordingly, a detailed comparison of frontal and parietal activity in timing tasks must await more comparable experimental designs, preferably applied to the same monkeys.
Although PA and PFdl showed similar properties, the results from PA were much more robust. The caudal part of PA, less sampled in the present study, is known as dorsal premotor cortex (PMd). PMd has been implicated in timing functions, as has the supplementary and pre-supplementary motor areas (Coull et al., 2004; Mita et al., 2009; Rao et al., 2001), and it has recently been shown that an elapsed-time signal can be decoded from PMd as monkeys delay a key release for a defined interval (Lebedev et al., 2008). The finding of elapsed time and relative-duration coding in these regions should not be surprising considering that they are involved in attentional (Lebedev and Wise, 2001) and other nonmotor functions (Hanakawa et al., 2002), as well as motor ones. Note, however, that most of the PA data in the present study come from cortex rostral to the arcuate sulcus.
Our results do not distinguish among the various models of duration perception, which include oscillators and internal clocks, delay lines, and intrinsic dynamics of local circuits (Reutimann et al., 2004), and they appear compatible with both intrinsic mechanisms operating along sensory pathways as well as with a dedicated neural network for timing (Ivry and Schlerf, 2008). According to one of these models, the scalar expectancy theory (Gibbon et al., 1984), a pacemaker generates pulses, which are summed by a neural integrator. In the framework of this model, the neurons encoding whether the red or blue stimulus lasted longer could operate as components of a decision module. In line with clock models, our data show that the PA and PFdl neurons participate in the comparison of which stimulus lasted longer and might be involved in the accumulation and memory of “clock pulses”.
Timing impinges on many aspects of behavior, including perception and memory. In the frontal cortex, neurons reflect the binding of temporal information to representations of both stimulus features and their order of presentation. The latter finding suggests that the order of events aids judgments about their relative duration. Understanding when, in what order, and for how long specific events occurred depends on binding timing information to other representations and their order of occurrence. A role in the short-and long-term memory of such temporal information could be a key function of frontal cortex, working in coordination with intermediate-term temporal memory mechanisms of the hippocampal system (Brasted et al., 2003; Charles et al., 2004) to subserve both temporal perception and episodic memory.
The two adult, male rhesus monkeys (Macaca mulatta) used in this study weighed 8.5 kg and 8.0 kg, respectively. During training and task performance, they sat head-fixed in a primate chair, 29 cm from a video screen. As illustrated in Figure 1 (not to scale), the three infrared switches measured 3 × 2 cm each and were located within comfortable reach of the monkeys, beneath the video screen, separated horizontally by 7 cm (center to center). The fixation spot was a 0.6° white circle, which had to be fixated within ±7.4°; the blue stimulus was a 3° (diameter) circle; and the red stimulus was 3°×3° square. The pre-stimulus period lasted either 0.4 or 0.8 s; the D1 period 0.4, 0.8, or 1.2 s; and the D2 period 0.0, 0.4 or 0.8 s. As the “Go” signal, the two stimuli reappeared 7.8° to the left or the right of screen center. All of the variable parameters were pseudorandomly selected on each trial. The monkeys performed their tasks for fluid reinforcement using their left hands. Errors were signaled by acoustic feedback, and intertrial intervals were 300 ms.
For neural analysis, we used SPSS (SPSS Inc, Chicago, IL) and MatLab (MathWorks Inc, Natick MA).
Procedures followed the Guide for the Care and Use of Laboratory Animals (1996, ISBN 0-309-05377-3) and were approved by the NIMH Animal Care and Use Committee.
Using aseptic techniques and isofluorane anesthesia (1%–3%, to effect), recording chambers were implanted over the exposed dura mater of the left frontal lobe, along with head restraint devices. Monkey 1 had two 18 mm (diameter) circular chambers; monkey 2 had one 27 × 36 mm chamber.
Eye position was monitored with an infrared oculometer (Arrington recording, Scottsdale, AZ USA). Single-cell potentials were isolated with quartz-insulated platinum-iridium electrodes (0.5–1.5 MΩ at 1 KHz) advanced by a 16-electrode microdrive (Thomas Recording, Giessen, Germany) through a concentric recording head (518 μm electrode spacing). Spikes were discriminated online using Multichannel Acquisition Processor (Plexon, Dallas, TX) and confirmed with Off Line Sorter (Plexon) based on principal component analysis, interspike intervals, and clearly differentiated waveforms.
For the D1 period, we used two-way ANOVA (α=0.05) with S1 duration and stimulus color as factors. For cells with significant duration effects, we used ANOVA with orthogonal polynomial contrasts to describe the relationship between average activity and S1 duration. The quadratic polynomial tested whether quadratic relationships exceeded linear ones; the cubic polynomial tested whether cubic fits exceeded quadratic ones.
For the D2 period, a two-way ANOVA (α=0.05) tested for relative-duration effects. The two factors were relative duration based on the order of stimulus presentation (S1 or S2 longer) and based on stimulus features (red or blue longer). A separate analysis of D2-period activity, based on two stepwise regressions, examined the encoding of each stimulus duration, which stimulus lasted longer, and by how much. Stepwise regression analysis has the advantage of finding the most parsimonious model, i.e., the one with the minimum number of variables that can predict neural activity. The first 4-factor multiple regression was based on the order of stimulus presentation, with the durations of S1 and S2 regressed along with their relative duration (S2 – S1; S2 longer or shorter):
where Z is mean firing rate during the D2 period. A separate 4-factor stepwise regression was based on stimulus color, with the durations of the red and blue stimuli substituted for S1 and S2, respectively, in Eq. 1. The stepwise-regression analysis began with no predictors in the regression equation. After testing each factor, the predictor variable that had the highest correlation with the average neural activity then entered into the model first, if it was significant at the p<0.05 level. Having removed the variance from the first predictor, the remaining variables were then tested. If, for example, a second variable was selected based on the highest remaining partial correlation, it also entered the equation if statistically significant. For each new variable entering the model, the variables already in the model were examined for potential removal. If they did not meet the p=0.1 level or better, they were removed from the model. We used this less stringent p value for removal to avoid underestimating the number of cells showing a combination of effects. In practice, however, this concern proved to be unwarranted because only rarely did more than one factor reach significance. To test whether the number of neurons selective for each factor exceeded that expected by chance, we performed a Monte Carlo analysis. After shuffling the averages neuronal activity corresponding to different pairs of stimulus durations, the selectivity of a neuron for each factor was reanalyzed by stepwise regressions. We repeated this procedure 1000 times for each neuron and obtained a null distribution, against which the observations were tested.
Duration coding was also evaluated with the ROC (receiver operating characteristic) analysis (Green, 1966). ROC values were calculated in 200 ms bins from 300 ms before S2 offset to 900 ms after S2 offset in 20-ms steps.
Electrolytic lesions (15 μA for 10 s, anodal current) were made at selected locations. After 10 days, the animal was deeply anesthetized, then perfused through the heart with formaldehyde-containing fixative. We plotted recording sites on Nissl-stained coronal sections by reference to the recovered electrolytic lesions and the marking pins inserted at the time of the perfusion. Notwithstanding the locations of electrode-entry sites (Fig. 1B), PA recordings were predominantly taken from the prearcuate cortex, mainly area 8. The remainder were mostly in area 46 of PFdl, with a smaller population in area 12. Figure 1B shows the dividing line between the PFdl and PA recording sites.
This work was supported by the Division of Intramural Research of the National Institute of Mental Health (Z01MH-01092). We thank Dr. Andrew Mitz, Mr. James Fellows, and Ms. Ping Yu for technical support.
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.