PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
Nat Neurosci. Author manuscript; available in PMC 2010 April 1.
Published in final edited form as:
Published online 2009 September 13. doi:  10.1038/nn.2398
PMCID: PMC2847499
NIHMSID: NIHMS140200

Coding of stimulus sequences by population responses in visual cortex

Abstract

Neuronal populations in sensory cortex represent the time-changing sensory input through a spatiotemporal code. What are the rules that govern this code? We measured membrane potentials and spikes from neuronal populations in cat visual cortex (V1), through voltage-sensitive dyes and electrode arrays. We first characterized the population response to a single orientation. As response amplitude grew, population tuning width remained constant for membrane potential responses and became progressively sharper for spike responses. We then asked how these single-orientation responses combine to code for successive orientations. We found that they combine through simple linear summation. Linearity, however, is violated after stimulus offset, when responses exhibit an unexplained persistence. Thanks to linearity, the interactions between responses to successive stimuli are minimal. We demonstrate that higher cortical areas may reconstruct the stimulus sequence from V1 population responses through a simple instantaneous decoder. In area V1, therefore, spatial and temporal coding operate largely independently.

The computations performed by sensory cortex involve large neuronal populations whose activity evolves over time to code an ever changing sensory input1, 2. We wish to understand the rules that govern this dynamical code. Such rules would describe the computations performed by the underlying circuits. We also wish to understand the strategies required to decode these responses, as these strategies must be followed by downstream neurons that interpret V1 population activity.

In some neural systems, the spatial and temporal aspects of the neural code are inextricably linked3. For example, in the insect olfactory system the population activity that codes for a sequence of two odors differs not only from the representation of either odor alone but also from a mixture of the two4. Similarly, in the mammalian motor cortex, the temporal evolution of the population activity that codes for a movement describes a complex trajectory in state space5. An analogous situation may occur in rodent somato-sensory cortex, where responses to sequences of whiskers differ substantially from responses to individual whiskers6.

We asked if such a linkage of spatial and temporal coding is present also in visual cortex, and specifically in the representation of stimulus orientation by the primary visual area (V1). Orientation selectivity in V1 is a paradigmatic example of cortical elaboration of thalamic sensory afferents, providing a testbed for theories of cortical function7, 8.

Studies of the dynamics of V1 population activity concentrated on membrane potential responses to individual stimuli, and found a dissociation of spatial and temporal coding. Using voltage-sensitive dye imaging, these studies found that the orientation selectivity of the population remained constant over the course of the response9, 10. This invariance is consistent with tuning width measurements made from the membrane potential of single neurons: this tuning width typically remains constant during the course of a response11.

Do the spike responses to individual stimuli show a similar dissociation of spatial and temporal coding? The dynamics of V1 spike responses to oriented stimuli have been studied extensively in single neurons. Over the course of the response, the orientation bandwidth of neurons has been found to sharpen1216, to broaden17 or to remain largely constant8, 18. This variety may stem in part from differences in experimental procedures or in data analysis. Given the variety of results observed even within individual studies, however, it remains unknown whether the spike response to a single orientation of V1 populations shows any dynamics such as sharpening of selectivity over time.

More generally, it is not known if the population response to individual orientations can predict population responses to stimulus sequences. The interactions between responses to subsequent orientations could be complex. Indeed, the spike responses of individual V1 neurons to pairs of orientations display effects ranging from suppression to a repulsion of tuning curves1922. We don’t know to what degree these interactions may affect the overall population responses.

Similarly, it is not known how population activity evolves once the stimulus drive has been removed. Some V1 neurons give prolonged responses at stimulus offset23, perhaps providing support to perceptual phenomena of visual persistence24. These offset responses may reflect intracortical interactions that reverberate activity, possibly causing attractors in the dynamics of cortical activation25, 26. Attractors would explain why spontaneous activity in V1 favors patterns that resemble those elicited by oriented stimuli25, 27. Perhaps, once the responses attain one of these patterns, they linger there even after the initial drive is removed.

Finally, we don't know how stimulus sequences coded into population responses should be decoded by later stages of the visual system. Simple rules have been proposed for the decoding of stimulus orientation from V1 populations1, 2, 2831. These rules, however, have been rarely tested on actual population responses, and in particular they have never been applied to dynamically changing population responses. Can the time-evolving population activity of area V1 be decoded using simple rules?

Results

We monitored the dynamics of population activity in visual cortex, measuring both spikes and membrane potentials in identical stimulus conditions. To measure a population’s spikes we recorded from 10×10 electrode arrays32. To measure a population’s membrane potentials we used voltage-sensitive dye (VSD) imaging33. VSD imaging reveals the membrane potential responses in superficial layers34 with <10 ms temporal resolution9, 10, 33. In V1, its signals reflect largely the responses of complex cells10.

Seeking to activate multiple subpopulations of neurons in rapid sequence, we used an “orientation noise” stimulus. In this stimulus, gratings of random orientation (each flashed for 32 ms in random phase) are presented in sequence14,8. To study how cortical responses transition between resting and stimulated states and vice versa, we interleaved at random times a fraction (~30%) of blank frames, uniform gray screens that had the same mean luminance as the gratings. This orientation noise stimulus elicited lively spike responses35 and strong VSD signals (Supplementary Fig. 1).

Population responses to an oriented stimulus

The orientation noise stimuli yielded detailed maps of orientation preference (Fig. 1a,b). We measured the average activation profile of the membrane potentials following a given orientation (Fig. 1a). Activation generally emerged ~45 ms after stimulus onset and assumed the characteristic patchy structure of orientation maps36. As expected37, stimuli with lower contrast elicited slower responses (Supplementary Fig. 2). Responses to orthogonal stimuli tended to be complementary, so the single-orientation maps could be combined in a map of orientation preference36, where hue labels the preferred orientation and intensity reflects the selectivity of the response (Fig. 1b).

Fig. 1
Population responses to an oriented stimulus. (a) Average maps of VSD fluorescence triggered on the appearance of a 90° grating, at various delays from grating onset. Scale bar indicates 1 mm. (b) Combining maps obtained with multiple stimulus ...

To investigate the selectivity of these responses we expressed them as a function of preferred orientation (Fig. 1c,d). Having assigned to each pixel a preferred orientation (Fig. 1b), we summarized the activity of the population (Fig. 1a) by averaging together the responses of pixels with similar orientation preference9 (Fig. 1c). For this analysis, further, we averaged responses to stimuli of different orientations, expressed relative to the difference between stimulus orientation and preferred orientation. The shape of these responses remained constant: once responses at different times are normalized by their amplitude, their profiles become very similar (Fig. 1d).

We applied a similar analysis to the spike responses and obtained fairly different results (Fig. 1e,f). We identified the preferred orientation of each site in the electrode array, and averaged together responses of sites with similar preference (Fig. 1e). Unlike the membrane potential responses, the spike responses did show a dependence of tuning width on time, with population activity being more narrowly distributed at the peak of the response than at earlier or later times (Fig. 1f).

To compare the population responses measured from spikes and membrane potentials, we fitted them with circular Gaussian functions38 (Fig. 2a–b). These provided excellent fits (Fig. 1c,e) and yield two parameters: amplitude and tuning width. For membrane potential the amplitude peaked 74±2 ms (n=8) after stimulus onset (Fig. 2a, blue). For spikes the amplitude peaked earlier, at 50±2 ms (n=6) (Fig. 2a, red). Tuning width (half-width at half-height) for membrane potential remained fairly constant during the course of the responses, and averaged 31.2°±0.3° (Fig. 2b, blue). For spikes, instead, tuning width decreased by 27%, from close to 30° to almost 20°, before broadening again (Fig. 2b, red).

Fig. 2
Properties of single-orientation responses. (a) Time course of response amplitude (from Gaussian fits in Fig. 1c,e) for membrane potential (dark gray) and spikes (light gray). Shaded regions indicate ±1 s.e. (n=8 hemispheres for membrane potential, ...

The sharpening observed in population responses measured from spikes is seen also in individual neurons (Fig. 2c–d). We studied the spike responses of 32 well-isolated single units within the population. Similar to the multiunit spike activity, these single-unit responses peaked 50±2 ms after stimulus onset (Fig. 2c) and decreased in tuning width from close to 30° to almost 20° before broadening again (Fig. 2d). This clear sequence of sharpening and broadening during the course of the response was seen in all neurons, with no exceptions.

The difference in tuning width between membrane potential and spikes agrees with intracellular measures in single neurons, and is largely due to the spike threshold38. The difference in timing is less expected. It is likely due to two factors. First, neurons tend to fire during the rising phase of the underlying membrane potential39. Second, an early untuned depolarization8, 9, 11 may help the potential reach spike threshold in the initial portion of the responses. The properties of this depolarization are described below.

The population response to a single oriented stimulus could be decomposed into the sum two components, one untuned and one tuned8, 9, 11 (Fig. 2e,f). The untuned component is the mean activity across preferred orientations; it varies only in time (Fig. 2e,f, curves). The residual tuned component varies both in time and in preferred orientation (Fig. 2e,f colored panels). Consistent with earlier reports8, 9, 11, the untuned component led in time the tuned component.

Responses to a sequence of orientations

Having characterized the population response to single oriented stimuli, we asked whether a sequential application of this elemental population response can predict the responses to the full stimulus sequence.

We first considered the membrane potential responses, and found that we could predict them based on simple summation (Fig. 3). We expressed the stimulus as a function of orientation and time (Fig. 3a), and predicted responses by repeatedly summing the elemental population response (Fig. 3b) appropriately shifted in time and orientation (a two-dimensional convolution). The result is a linear prediction of the responses (Fig. 3c). Aside from occasional discrepancies, seen especially in overall amplitude, the predicted responses resembled the actual responses (Fig. 3d). Both predicted and actual responses showed peaks of activity that shifted to coincide with the appropriate stimulus orientation. Distributions of predicted vs. actual responses cluster along a curve that deviates from linearity only for large responses (Fig. 3e). We could indeed improve on the predictions of the model by fitting a mildly compressive nonlinear function to these distributions and applying it after the summation40. The resulting model explained 98±1% of the variance of the responses (s.d., n=8). The correlation between predicted and actual responses was high, at 0.69±0.06 (s.d., p<0.005., n=8).

Fig. 3
Predicting the membrane potential responses of the population to the full stimulus sequence. (a) The stimulus expressed as an image, with ones (dots) indicating the sequence of stimulus orientations as function of time, and zeros elsewhere (gray). For ...

We obtained similar results for the spike responses (Fig. 4). The summation model (Fig. 4a,b) captured the transient nature of these responses and the sequence of peak activations (Fig. 4c,d). Just as for membrane potential, the major deviations occurred for high and low values of the response (Fig. 4e). Thus, the model performance was improved by including a compressive nonlinearity, yielding a correlation between predicted and actual responses of 0.58±0.06 (s.d., p<0.005., n = 6, Fig. 4e).

Fig. 4
Predicting the spike responses of the population to the full stimulus sequence. Format as in Fig. 3. Panels a–d are from experiment 75-5-16.

These results indicate that population responses to the stimulus sequence are simply the sum of consecutive responses to the individual elements in the sequence. On the other hand, there is a noticeable scatter in the relation between predicted and actual population responses, both for membrane potential (Fig. 3e) and for spikes (Fig. 4e). Are the deviations systematic? To address this question we focused on two key stimulus conditions: the transitions from orientation to orientation and the transitions from stimulus to blank.

Effect of recent history of stimulation

We asked if the summation model could explain how the population’s activity depends on the orientations seen in the immediate past (Fig. 5). We focused on four “orientation jumps”, in which consecutive gratings differed in orientation by: 0° (the same orientation), 90°, +45°, and −45°.

Fig. 5
Predicting the interactions between population responses to successive orientations. (a) Average membrane potential responses of the population to two successive stimuli of the same (top panel) or different orientations. The schematic at left describes ...

The membrane potential responses of the population represented faithfully the end points of the orientation jumps, but tended to interpolate through neurons with intermediate orientation preferences during the transitions (Fig. 5a, e). For 0° changes, the membrane potential response showed a single prolonged activation (Fig. 5a, row 1): there was no clear demarcation between the responses to the two stimuli even though in 75% of the cases the transition between stimuli involved a change in spatial phase. For a 90° jump, conversely, the population response showed a distinct transition (Fig. 5a, row 2), involving in sequence two separate subpopulations with orthogonal orientation preferences. The responses to ±45° jumps, finally, showed a mixture of these behaviors (Fig. 5a, rows 3,4); responses show two distinct peaks, but these peaks are joined by a response passing through the intermediate orientations. The profiles of the membrane potential responses during the transition showed a marked attraction between response to first and second stimulus: the peak of the population response shows an average shift of 20.7±2.6° (s.e., n=6) toward the preceding stimulus orientation (Fig. 5e). Similar results were obtained when examining smaller orientation jumps (23° or 30°, not shown).

Similar though less marked effects were seen in the spike responses of the population (Fig. 5c, f), which followed more faithfully the transitions between orientations. Though the responses to 0° changes showed a single prolonged activation similar to that seen in membrane potential (Fig. 5c, row 1), responses to jumps by 90°, 45° and −45° showed clearly distinct peaks, with little response in neurons with intermediate orientation preferences (Fig. 5c, rows 2–4). Just as for membrane potential, however, the population responses exhibited a marked attraction towards the orientation of the previous stimulus, with a shift of 6.3±2.8° (s.e., n=5) toward the preceding stimulus orientation (Fig. 5f).

All these effects were explained accurately by the simple summation model. From the population responses predicted by the model we computed average responses to orientation jumps (Fig. 5b,d). They closely resemble the data, explaining 91±3% (s.d., n = 8) of the variance for membrane potential (Fig. 5b), and 94.3±2% (s.d., n = 6) of the variance for spikes (Fig. 5d). Specifically, the model accurately predicts the attraction in population responses that follows a ±45° jump, both in membrane potentials and in spikes (Fig. 5e,f, dotted lines). Therefore, even when the population responses appear to be traveling waves that involve neurons with orientation preference intermediate between two successive orientations, these responses are more usefully described as the sum of two elemental waves – the successive responses to the individual orientations.

Persistence of population responses

We then asked how the population response evolves once the stimulus drive has been removed. We measured the population activity that follows stimulus offset and tried to explain it with the simple summation model (Fig. 6).

Fig. 6
Unexplained persistence of population responses after stimulus offset. (a) When the visual stimulus is removed (stimulus-to-blank conditions, as shown on top by the contrast change), membrane potential activity in the population persists for well over ...

The membrane potential responses showed a persistence that could not be explained by the summation model. We averaged all cases when a stimulus was followed by a blank (Fig. 6a), and compared them to the model prediction (Fig. 6c). The measured responses clearly outlasted the prediction (Fig. 6g, red vs. black). The population responses persisted 48±3 ms (n=8) longer than the predicted responses, with the peak difference occurring 72±24 ms after stimulus offset (Fig. 6i). Membrane potential responses, therefore, persisted well beyond what would be expected from the elemental response to a given orientation.

The inadequacy of the model in predicting the offset responses is not due to erroneous choice of model parameters. The model involves convolution with a filter, the average population response to a single orientation. This filter is biased toward stimulus-to-stimulus transitions (70% of the data). To assess the impact of this bias we computed the filter using only stimulus-to-blank transitions. By definition, the resulting model did a perfect job at predicting the offset responses. However, it did significantly worse in describing the average orientation-to-orientation transitions: explained variance dropped from 91% to 75%. Therefore, the summation model could not explain population responses to both kinds of transition: from stimulus to stimulus, and from stimulus to blank.

We observed a similar persistence in the population responses measured from spikes. The spike responses to a stimulus followed by a blank (Fig. 6b) had a longer tail than the prediction of the summation model (Fig. 6d). The responses during the blank stimulus persisted 30±4 ms (n=6) longer than predicted (Fig. 6h, red vs. black), and the peak difference occurred 64±31 ms after stimulus offset (Fig. 6j).

The failure of the summation model in predicting persistence is not simply due to an inadequacy in dealing with blank stimuli: the model encountered no difficulties in predicting the responses to stimuli following a blank. We saw no consistent difference between model predictions and actual responses in onset latency, peak amplitude, and width of the response. Thus, cortical patterns of activity produced by a stimulus are similar whether they follow another response or a period of spontaneous activity.

Two observations suggest that the origin of response persistence is cortical. First, the persistence is tuned for orientation: it involves the same subpopulation of neurons that were initially activated by the stimulus (Fig. 6e,f). Indeed, the averaged response to a stimulus followed by a blank was separable, both for membrane potential responses (Fig. 6, compare e to a) and for spike responses (Fig. 6, compare f to b). Second, the persistence is most pronounced in the superficial layers of cortex. In a control experiment using multiprobe electrodes spanning the depth of cortex, we computed field potentials and the associated profile of current source density. We compared this profile with the predictions of the summation model, and found persistence in the superficial layers but not in deeper layers (Supplementary Fig. 3). Together, these results suggest that response persistence is not inherited from the thalamus. Its causes may lie in the circuitry of cortex.

Decoding population responses

Finally, we asked how the spike responses of the population should be decoded to reconstruct the stimulus sequence. We have seen that at any instant, population responses depend not only on the present stimulus, but also on the recent history of stimulation. This dependence was explained by the linear summation of elemental responses to individual orientations. In principle, therefore, to estimate the present stimulus orientation, a decoder should know the rules of summation, the shape of the elemental response, and the recent history of stimulation. Could a decoder neglect these aspects and still interpret the population responses instantaneously and accurately?

We implemented a simple Bayesian decoder1, 28, 31 that operates on instantaneous population firing rates, with no knowledge of previous stimuli and responses or of the summation rule (Fig. 7a,b). This decoder quantizes time in 8 ms intervals. For each interval, it measures the likelihood that the present stimulus has orientation θ given a population response. This likelihood is the product of the probabilities of observing each of the responses in the bins of orientation preference, conditioned on θ. We obtained these probabilities from the averaged population response to a single orientation (Fig. 7a), by measuring the mean and variance across stimulus presentations (Fig. 7b).

Fig. 7
Decoding stimulus orientation from the population responses. (a) The average response to an individual stimulus orientation. The dotted lines indicate the interval when this response is significantly higher than baseline. (b) Probability that a spike ...

Such a simple decoder predicted accurately the sequence of stimulus orientations (Fig. 7c–g). To be realistic, the decoder must operate on population responses measured in single-trials (Fig. 7d), which can be considerably noisier than responses averaged over trials (Fig. 7c). The decoder predicts a distribution of stimulus likelihoods (Fig. 7e). The maxima of this distribution are the decoded orientations. They matched very closely the actual stimulus orientations (Fig. 7f), with a circular correlation of r = 0.68 in the example data set (Fig. 7g) and of r = 0.67 ± 0.03 (s.e. p < 10−6) across n = 5 experiments.

Because of response persistence, during blank intervals the decoder continued to predict the presence of the previous orientation (Fig. 7h). To study this “perceived persistence”, we averaged all conditions when a stimulus was followed by a blank. Consistent with the persistence of population responses at stimulus offset (Fig. 6f), the distribution of stimulus likelihoods during a blank interval was not flat, but rather resembled the distribution obtained during the preceding stimulus (Fig. 7h). In other words, during the blank intervals, the decoder indicates that the stimulus has the orientation of the preceding stimulus. These perceived orientations simply reflect the fact that the population responses during a blank interval maintain a trace of the previous stimulus.

We performed this formal analysis on the spike responses as these constitute the output of area V1 and are read by subsequent visual areas. An informal analysis indicates that it would be equally easy to decode the population responses measured from membrane potential (Supplementary Fig. 1 and Supplementary Movie).

We conclude that it is not necessary for a decoder of population activity in area V1 to know the relationship between temporal and spatial encoding. We have seen that temporal and spatial encoding in V1 are linked to each other in two ways. First, the elemental population responses to a single orientation measured from spikes show sharpening over time. Second, the population responses retain a trace of previous stimuli, interacting with each other through summation. These properties of the coding mechanisms, however, can be safely ignored by a subsequent decoder. Indeed, a decoder with no knowledge of any linkage between temporal and spatial encoding could successfully predict the sequence of input stimuli.

Discussion

Using electrophysiology and imaging, we have studied how large populations of neurons in area V1 code for stimulus orientation. We described how population activity changes over time to reflect a rapidly changing sensory input, and we showed that the dynamics of this activity are summarized by a set of simple rules.

We first investigated the population responses to a single orientation. When measuring membrane potential, we found that the width of the population profile remained constant during the course of the responses. This result is consistent with earlier measurements obtained in populations with periodic stimuli9, 10, and in single neurons with random stimuli11. When measuring spike responses, however, we found that activity exhibited a pronounced sharpening during the course of the response. This sharpening was present not only in the overall population profile, but also in all individual neurons in the sample, with no exceptions.

The uniformity of sharpening across neurons is surprising given the variety of observations reported in previous single-neuron studies1217,8, 18. This variety may be explained by multiple factors. First, by differences in stimulus attributes such as size: large stimuli like ours lead to clearer sharpening41. Second, by differences in cell sample, including layers and location in the orientation preference map42. Third, by differences in data analysis, e.g. the way responses are normalized16, and whether one’s measure of orientation selectivity discounts an untuned response8, 9, 11, 14.

We then investigated the interactions between oriented stimuli presented in succession, and asked if they could be predicted from the elemental responses to single orientations. We found that simple summation was surprisingly successful.

Our data reveal that strong interactions between successive orientations are prevalent in the population responses, both in membrane potential and in spike activity. Population activity is attracted towards the orientation of the preceding stimulus, and such attraction corresponds to repulsion of tuning curves of individual neurons. These results agree with studies in single neurons, where suppression near the conditioning stimulus19 can accompany an enhancement at other orientations, resulting in the repulsion of tuning curves away from the conditioning stimulus2022.

With the exception of one study21, previous reports of interactions between subsequent orientations in single neurons did not ask whether such interactions would be explained by simple summation. Our data support this interpretation. Specifically, summation could predict with high accuracy the population responses to changes in orientation. These responses can appear as traveling waves, peaking at a range of intermediate orientations between the stimulus orientations. However, the summation model reveals that they really are the sum of two elemental waves.

The fundamental linearity that we observed in population responses may be surprising given the nonlinearities seen in individual V1 neurons. The summation model is linear: population responses are a weighted sum of past stimulus orientations. The static nonlinearity at the output of the sum does not alter this fundamental linearity40. By comparison, complex cells, which constitute the bulk of the signal in our imaging experiments10, are markedly nonlinear in their integration of spatiotemporal inputs43, 44. Even simple cells, whose spatial summation properties are more linear43 exhibit strong temporal nonlinearities45. Perhaps the orientation noise stimulus places the cortex in a regime where it operates more linearly (possibly because contrast is mostly constant and fairly high). Indeed, as a first approximation the spike responses of individual neurons to this stimulus can be predicted by the summation model35.

Yet, linear summation cannot be a complete description of V1 population responses. For instance, linear summation would not explain population responses to stimuli of different contrast. Linearity would predict that responses scale proportionally with contrast, while V1 responses vary nonlinearly with contrast, both in amplitude and in time course37, 43 (Supplementary Fig. 2). Accounting for these nonlinearities would likely require extending the model to include gain control mechanisms present at all stages of the early visual system43.

Indeed, we found a novel deviation from linearity: activity persists once the stimulus drive is removed (Fig. 6). This prolonged activation could be seen as a prolongation of the response to the stimulus (“persistence”) or a response to the disappearance of the stimulus (“off-response”). We favor the first description for two reasons. First, the lag between an off-response and the preceding on-response would resemble the stimulus duration (32 ms), whereas the prolonged activation peaked at a later time. Second, an “off-response” is de facto already included in the average stimulus-triggered population responses: the stimulus disappears (on average half of the pixels change gray level) every time there is a change in orientation.

We argued that this neural persistence is likely to be due to cortical mechanisms, possibly implicating the strong recurrent excitation that characterizes cortical circuits46. Recurrent circuits in area V1 may favor states that signal a single orientation7, 8, 2527, 47. Such states might act as attractors25, 26. The stimulus before the blank might drive the network in one of these attractors; when the drive is removed, activity persists because it is transiently trapped in the attractor. Attractors are probably weak compared to the feedforward drive, or they would lead to striking hallucinations47. However, their influence might be strong at low contrast or during a blank stimulus, when the feedforward drive is weaker or absent.

Perceptually, the neural persistence that we have observed may be a correlate of phenomena of visual persistence, which is in turn related to iconic memory24. Such correlates had been previously found in higher visual areas4850. Smaller but similar effects have been seen in area V123. The dynamics of these responses, however, had been studied only in few neurons and had been characterized qualitatively. Our results indicate that persistence is robustly present in the population responses of area V1, and indicate that this persistence is not due to the off-responses that are generally expected from a receptive field.

Finally, we found that the time-evolving spike responses of a V1 population can be read-out through a simple instantaneous decoder. Multiple methods have been proposed for decoding stimulus orientation from V1 populations1, 2, 2831. These methods had been rarely tested on actual population responses, and in particular they had not been applied to reconstruct the attributes of stimulus sequences from dynamically changing population responses. We implemented a simple decoder that operates instantaneously, and found it to do an excellent job in predicting the stimulus orientation.

The good performance of the decoder is surprising for a number of reasons. First, it is typically assumed that a decoder needs to take into account the large amount of variability that is shared among neurons2; our decoder ignores this covariance and yet it performs very well. Second, our decoder ignores a number of factors that would appear to be crucial: the rules of interaction between subsequent elemental responses, the shape and duration of the elemental response, and the recent history of stimulation. We suggest that the linearity of summation that we have demonstrated reduces the importance of these factors. Indeed, these factors are essential to decode the population activity in systems where the spatial and temporal aspects of the neural code are inextricably linked3.

In conclusion, we have provided a set of rules that govern the dynamical code by which populations of V1 neurons represent the attributes of a rapidly changing stimulus. Such rules ultimately describe the computations performed by the underlying circuits. We have also provided a simple example of the strategy that downstream neurons could follow to decode the population responses. By elucidating both aspects – coding and decoding – these findings may ultimately help relate neural activity to perception.

Methods

Physiology

We report results measured in 7 cats (8 hemispheres) for the VSD experiments and in additional 4 cats (6 hemispheres) for the electrical recordings. Young adult cats (2–4 Kg) were anesthetized first with Ketamine (22 mg/kg i/m) and Xylazine (1.1 mg/kg i/m) and then with Sodium Penthotal (0.5–2 mg/kg/hr i/v) and Fentanyl (typically 10 µg/kg/hr i/v), supplemented with inhalation of N2O (typically 70/30 with O2). A 1 cm craniotomy was performed over area V1 (usually area 18, occasionally area 17). The eyes were treated with topical atropine and phenylephrine, and protected with contact lenses. A neuromuscular blocker was given to prevent eye movements (pancuronium bromide, 0.15 mg/kg/hr, i.v.). The animal was artificially respirated, and received periodic doses of an antibiotic (Cephazolin, 20 mg/kg IM, twice daily), an anti-edematic steroid (Dexamethasone, 0.4 mg/kg daily), and an anticholinergic agent (atropine sulfate, 0.05 mg/kg, i/m, daily). Fluid balance was maintained by intravenous infusion. The level of anesthesia was monitored through the EEG. Additional physiological parameters that were monitored include temperature, heart rate, end-tidal CO2, and lung pressure. Experiments typically lasted 48–72 hours. Procedures were approved by the Institutional Animal Care and Use Committee.

Stimuli

The stimulus consisted of full-field stationary gratings flashed in random sequence for 40 ms each14. The gratings had one of 4 spatial phases and one of 4–8 orientations. Their contrast was typically 50%, and the spatial frequency was the optimal one as assessed by a preliminary experiment (typically, 0.2 cpd). Sequences were broken into 4–8 segments lasting 6 s each. Randomly interleaved with the gratings were blank frames (also lasting 40 ms), which occurred with a probability of about 30%. An additional 6 s control segment consisted entirely of blanks. Segments were presented in random order and each block of segments was generally presented 10 times. Stimuli were viewed monocularly with the eye contralateral to the hemisphere being imaged.

Imaging

Methods for VSD imaging were developed by Grinvald and collaborators9, 33 and described in our previous work10. We stained the cortex with the VSD RH-1692 and imaged its fluorescence in 15–30 mm2 of V1. The dye was circulated in a chamber over the cortex for 3 hours, and washed out with saline. We acquired images with a CMOS digital camera (1M60 Dalsa, Waterloo, Ontario), as part of the Imager 3001 setup (Optical Imaging Inc, Rehovot, Israel). Images were acquired at a frame rate of 110 Hz, with spatial resolution of 0.028 mm per pixel. Additional spatial filtering was performed offline (bandpass, 0.2–2.2 cycles/mm), except when measuring overall activation (Fig. 2a). Frame acquisition was synchronized with the respirator. Illumination from a 100 W halogen light was delivered through two optic fibers. Excitation and emission filters were bandpass, at 630±10 nm and 665±10 nm.

Population responses were measured by grouping pixels in 24 bins according to their preferred orientation and averaging their responses to obtain a single data point for each preferred orientation bin.

To analyze the VSD responses to orientation noise, we subtracted the response to the blank 6-s segment from the response to each 6-s segment; this subtraction removed artifacts due to respiration, which was synchronized with sequence onset. The resulting responses were then bandpass filtered between 1 and 25 Hz.

Z-scores were computed by averaging the responses across the 10 repeats, and dividing responses by the s.d. computed across these repeats.

Array recordings

We implanted a 10 × 10 electrode array (0.4 mm separation and 1.5 mm electrode length) in the same patch of cortex. To avoid excessive cortical damage, the arrays were inserted at high speeds (around 8 m/s) using a pneumatic insertion device. Insertion depths were about 0.8–1 mm. Due to the curvature of the cortex the depth of penetration varied across sites. The array and surrounding tissue was covered in 1.5% agar to improve stability. Signals that crossed a specified threshold in each electrode were saved to disk for spike sorting at a later stage. These threshold crossing events are defined as multi-unit activity. Well tuned multi-unit activity was typically recorded from most of the 96 electrodes. Many electrodes also contributed well isolated single unit recordings. Traces were acquired at 12 kHz, and firing rates were obtained by low-pass filtering the spike trains with a cutoff at 25 Hz.

Untuned response component

The untuned component of the membrane potential and spike responses is the mean activity across preferred orientations. In VSD imaging, signals of such low spatial frequency are dominated by noise. We removed this component from population responses because it reflects the activation of the entire region regardless of preferred orientation.

Event-related analysis

To compute the average response to a generic oriented stimulus, we used event related analysis and averaged the responses in a 200 ms window around the time of occurrence of a given oriented grating. This procedure was applied each time a grating was presented and the data sorted and averaged according to the stimulus orientation.

Conditional average responses (second-order interactions) were computed with the same algorithm. Event related analysis was performed on those conditions in which a stimulus was preceded by (1) an identical stimulus (2) a stimulus with 45° orientation difference (3) 90° orientation difference, or (4) −45° orientation difference.

Predicted population responses

As illustrated in Fig. 3a,b, the linear predicted responses Lθ(t) are computed by convolving the average response to a stimulus Fθ(t) with the sequence of stimuli S([var phi],t):

Lθ(t)=0πdφ0S(φ,t)Fθφ(tT)dT.

Here, the stimulus S([var phi],t) is 1 if the orientation of the stimulus presented at time t was equal to [var phi], and 0 otherwise, and θ indicates the preferred orientation of a subpopulation.

Finally, the predicted responses Rθ(t) are computed by passing the linear prediction through a static nonlinearity f to account for the discrepancy between linear model and prediction at high and low values of the responses: Rθ(t)=f(Lθ(t)). The static nonlinearity is fitted to the data so as to be optimal40.

Percentage of variance

The percentage of variance for orientation jumps was computed as

<[1t,θ(Rt,θPt,θ)2t,θ(Rt,θRt,θt,θ)2]>S1S2,

where R and P are responses and prediction for a given orientation jump. The sums, t,θ, and the average, left angle bracket right angle brackett,θ, are computed for time t between 50 ms and 150 ms after stimulus onset, an interval during which the response amplitude is significantly above the baseline noise. The average left angle bracket right angle brackets1→s2 is done over all stimulus transitions in 8 hemispheres.

Bayesian decoder

To decode population responses we computed firing rate over 8 ms time bins, and for each of these bins we estimated the probability that the stimulus had a certain orientation [var phi]. Given a population response R = (r1,r2,…,rn), and a set of orientations {[var phi]1,[var phi]2,…,[var phi]n} with equal probability of presentation p([var phi]), the probability of stimulus [var phi]j is given by

p(φj|r1,r2,,rn)=p(r1|φj)p(r2|φj)p(rn|φj)p(s)/i=1np(r1|φj)p(r2|φj)p(rn|φi)

Here we make the simplifying assumption that p(ri|[var phi]) and p(rj|[var phi]) are independent. In reality they are not, but in separate analyses (not shown) we have also computed the covariance matrix and included it in the model, and found that doing so did not improve the quality of the decoding.

For each response ri (i = 1,…n; n = 24) we computed the distribution p(ri|[var phi]j) as:

p(ri|φj)=1σ2πe(r¯(φj)ri)22σ2

where [r with macron]([var phi]f) is the mean across stimulus presentations of the response to the stimulus [var phi]j, and σ is the associated standard deviation. We obtained the n = 24 tuning curves used to derive the distributions p(ri|[var phi]j) (Fig. 7a,b), by doing 24 circular shifts of the average elemental response along the orientation axis. Only the time average of the elemental response (within a time interval centered on the peak time, Fig. 7a, dotted lines) is used to construct the tuning curves.

Supplementary Material

Acknowledgments

We thank I. Nauhaus, R.A. Frazor, L. Busse, and S. Katzner for help with data acquisition. We thank WT Newsome, WS Geisler, and G Felsen for helpful discussions. Supported by a Scholar Award from the McKnight Endowment Fund for Neuroscience (MC) and by National Institutes of Health grants EY017396 (MC) and EY018322 (DLR). MC holds the GlaxoSmithKline / Fight for Sight Chair in Visual Neuroscience.

Footnotes

Author contributions

AB and MC performed the experiments, AB analyzed the data, and all authors contributed to the intellectual development of the project and to the writing of the manuscript.

References

1. Pouget A, Dayan P, Zemel RS. Inference and computation with population codes. Annu Rev Neurosci. 2003;26:381–410. [PubMed]
2. Averbeck BB, Latham PE, Pouget A. Neural correlations, population coding and computation. Nat Rev Neurosci. 2006;7:358–366. [PubMed]
3. Buonomano DV, Maass W. State-dependent computations: spatiotemporal processing in cortical networks. Nat Rev Neurosci. 2009;10:113–125. [PubMed]
4. Broome BM, Jayaraman V, Laurent G. Encoding and decoding of overlapping odor sequences. Neuron. 2006;51:467–482. [PubMed]
5. Yu BM, et al. Mixture of trajectory models for neural decoding of goal-directed movements. J Neurophysiol. 2007;97:3763–3780. [PubMed]
6. Civillico EF, Contreras D. Integration of evoked responses in supragranular cortex studied with optical recordings in vivo. J Neurophysiol. 2006;96:336–351. [PubMed]
7. Ferster D, Miller KD. Neural mechanisms of orientation selectivity in the visual cortex. Annu Rev Neurosci. 2000;23:441–471. [PubMed]
8. Shapley R, Hawken M, Ringach DL. Dynamics of orientation selectivity in the primary visual cortex and the importance of cortical inhibition. Neuron. 2003;38:689–699. [PubMed]
9. Sharon D, Grinvald A. Dynamics and constancy in cortical spatiotemporal patterns of orientation processing. Science. 2002;295:512–515. [PubMed]
10. Benucci A, Frazor RA, Carandini M. Standing waves and traveling waves distinguish two circuits in visual cortex. Neuron. 2007;55:103–117. [PMC free article] [PubMed]
11. Gillespie DC, Lampl I, Anderson JS, Ferster D. Dynamics of the orientation-tuned membrane potential response in cat primary visual cortex. Nat Neurosci. 2001;4:1014–1019. [PubMed]
12. Shevelev IA, Sharaev GA, Lazareva NA, Novikova RV, Tikhomirov AS. Dynamics of orientation tuning in the cat striate cortex neurons. Neuroscience. 1993;56:865–876. [PubMed]
13. Volgushev M, Vidyasagar TR, Pei X. Dynamics of the orientation tuning of postsynaptic potentials in the cat visual cortex. Vis.Neurosci. 1995;12:621–628. [PubMed]
14. Ringach DL, Hawken MJ, Shapley R. Dynamics of orientation tuning in macaque primary visual cortex. Nature. 1997;387:281–284. [PubMed]
15. Chen G, Dan Y, Li CY. Stimulation of non-classical receptive field enhances orientation selectivity in the cat. J Physiol. 2005;564:233–243. [PubMed]
16. Ringach DL, Hawken MJ, Shapley R. Dynamics of orientation tuning in macaque V1: the role of global and tuned suppression. J Neurophysiol. 2003;90:342–352. [PubMed]
17. Celebrini S, Thorpe S, Trotter Y, Imbert M. Dynamics of orientation coding in area V1 of the awake primate. Vis Neurosci. 1993;10:811–825. [PubMed]
18. Mazer JA, Vinje WE, McDermott J, Schiller PH, Gallant JL. Spatial frequency and orientation tuning dynamics in area V1. Proc Natl Acad Sci USA. 2002;99:1645–1650. [PubMed]
19. Nelson SB. Temporal interactions in the cat visual system I. Orientation-selective suppression in visual cortex. J. Neurosci. 1991;11:344–356. [PubMed]
20. Müller JR, Metha AB, Krauskopf J, Lennie P. Rapid adaptation in visual cortex to the structure of images. Science. 1999;285:1405–1408. [PubMed]
21. Felsen G, et al. Dynamic modification of cortical orientation tuning mediated by recurrent connections. Neuron. 2002;36:945–954. [PubMed]
22. Dragoi V, Sharma J, Miller EK, Sur M. Dynamics of neuronal sensitivity in visual cortex and local feature discrimination. Nat Neurosci. 2002;5:883–891. [PubMed]
23. Duysens J, Orban GA, Cremieux J, Maes H. Visual cortical correlates of visible persistence. Vision Res. 1985;25:171–178. [PubMed]
24. Coltheart M. Iconic memory and visible persistence. Percept Psychophys. 1980;27:183–228. [PubMed]
25. Goldberg JA, Rokni U, Sompolinsky H. Patterns of ongoing activity and the functional architecture of the primary visual cortex. Neuron. 2004;42:489–500. [PubMed]
26. Ben-Yishai R, Lev Bar Or R, Sompolinsky H. Theory of orientation tuning in the visual cortex. Proc. Natl. Acad. Sci. 1995;92:3844–3848. [PubMed]
27. Kenet T, Bibitchkov D, Tsodyks M, Grinvald A, Arieli A. Spontaneously emerging cortical representations of visual attributes. Nature. 2003;425:954–956. [PubMed]
28. Oram MW, Foldiak P, Perrett DI, Sengpiel F. The 'Ideal Homunculus': decoding neural population signals. Trends Neurosci. 1998;21:259–265. [PubMed]
29. Salinas E, Abbott LF. Vector reconstruction from firing rates. J Comput Neurosci. 1994;1:89–107. [PubMed]
30. Seung HS, Sompolinsky H. Simple models for reading neuronal population codes. Proc Natl Acad Sci U S A. 1993;90:10749–10753. [PubMed]
31. Geisler WS, Albrecht DG. Bayesian analysis of identification performance in monkey visual cortex: nonlinear mechanisms and stimulus certainty. Vis. Res. 1995;35:2723–2730. [PubMed]
32. Normann RA, Maynard EM, Rousche PJ, Warren DJ. A neural interface for a cortical vision prosthesis. Vision Res. 1999;39:2577–2587. [PubMed]
33. Grinvald A, Hildesheim R. VSDI: a new era in functional imaging of cortical dynamics. Nat Rev Neurosci. 2004;5:874–885. [PubMed]
34. Petersen CC, Grinvald A, Sakmann B. Spatiotemporal dynamics of sensory responses in layer 2/3 of rat barrel cortex measured in vivo by voltage-sensitive dye imaging combined with whole-cell voltage recordings and neuron reconstructions. J Neurosci. 2003;23:1298–1309. [PubMed]
35. Ringach DL, Malone BJ. The operating point of the cortex: neurons as large deviation detectors. J Neurosci. 2007;27:7673–7683. [PubMed]
36. Hübener M, Bonhoeffer T. Optical imaging of functional architecture in cat primary visual cortex. In: Payne BR, Peters A, editors. The cat primary visual cortex. New York: Academic Press; 2002. pp. 1–137.
37. Dean AF, Tolhurst DJ. Factors influencing the temporal phase of response to bar and grating stimuli for simple cells in the cat striate cortex. Exp. Br. Res. 1986;62:143–151. [PubMed]
38. Carandini M, Ferster D. Membrane potential and firing rate in cat primary visual cortex. J Neurosci. 2000;20:470–484. [PubMed]
39. Azouz R, Gray CM. Dynamic spike threshold reveals a mechanism for synaptic coincidence detection in cortical neurons in vivo. Proc Natl Acad Sci USA. 2000;97:8110–8115. [PubMed]
40. Chichilnisky EJ. A simple white noise analysis of neuronal light responses. Network. 2001;12:199–213. [PubMed]
41. Xing D, Shapley RM, Hawken MJ, Ringach DL. Effect of stimulus size on the dynamics of orientation selectivity in Macaque V1. J Neurophysiol. 2005;94:799–812. [PubMed]
42. Schummers J, et al. Dynamics of orientation tuning in cat V1 neurons depend on the location within layers and orientation maps. Frontiers in neuroscience. 2007;1:145–159. [PMC free article] [PubMed]
43. Carandini M, et al. Do we know what the early visual system does? J Neurosci. 2005;25:10577–10597. [PubMed]
44. Touryan J, Lau B, Dan Y. Isolation of relevant visual features from random stimuli for cortical complex cells. J Neurosci. 2002;22:10811–10818. [PubMed]
45. Tolhurst DJ, Walker NS, Thompson ID, Dean AF. Nonlinearities of temporal summation in neurones in area 17 of the cat. Exp. Br. Res. 1980;38:431–435. [PubMed]
46. McCormick DA, et al. Persistent cortical activity: mechanisms of generation and effects on neuronal excitability. Cereb Cortex. 2003;13:1219–1231. [PubMed]
47. Carandini M, Ringach DL. Predictions of a recurrent model of orientation selectivity. Vis. Res. 1997;37:3061–3071. [PubMed]
48. Kovacs G, Vogels R, Orban GA. Cortical correlate of pattern backward masking. Proc Natl Acad Sci U S A. 1995;92:5587–5591. [PubMed]
49. Rolls ET, Tovee MJ. Processing speed in the cerebral cortex and the neurophysiology of visual masking. Proceedings. 1994;257:9–15. [PubMed]
50. Keysers C, Xiao DK, Földiák P, Perrett DI. Out of sight but not out of mind: the neurophysiology of iconic memory in the superior temporal sulcus. Cognitive Neuropsychology. 2005;22:316–332. [PubMed]