|Home | About | Journals | Submit | Contact Us | Français|
Mounting evidence suggests that understanding how the brain encodes information and performs computations will require studying correlations between neurons. The recent advent of recording techniques such as multielectrode arrays and two-photon imaging has made it easier to measure correlations, opening the door to detailed exploration of their properties and contributions to cortical processing. Studies to date, however, have reported discrepant findings, providing a confusing picture. Here, we briefly review these studies and conduct simulations to explore the influence of several experimental and physiological factors on correlation measurements. Differences in response strength, the time window over which spikes are counted, spike sorting conventions, and internal states can all dramatically affect measured correlations and systematically bias estimates. Given these complicating factors, we offer guidelines for interpreting correlation data and a discussion of how best to evaluate the impact of correlations on cortical processing.
Understanding how populations of neurons encode information and guide behavior is a major focus of systems neuroscience. Cortical neurons respond with variable strength to repeated presentations of identical stimuli 1, 2. This variability is often shared among neurons, and such correlations in trial-to-trial responsiveness can have a substantial impact on the amount of information encoded by a neuronal population2-6. Early studies, for instance, showed that correlations reduce the signal-to-noise of a pooled population response, since shared fluctuations in response cannot be averaged away 6. More recently, it has been shown that attentional modulation of correlations accounts for more of its impact on sensory coding than modulation of firing rate 7, 8. Determining how correlations are affected by stimulus drive 9-12, by learning or experience 10, 13, 14, or by changes in behavioral context 7, 8, 15-17 is thus likely to be as important as understanding how these factors affect the firing rates of individual cells 18.
Correlations can also provide important information about the functional architecture of neuronal networks. Correlation analysis has been used to infer connectivity within the retina 19, between the visual thalamus and cortex 20, and between neurons within cortex 9, 21. For a given circuit, changes in correlations under different stimulus or behavioral conditions can provide a signature of network function or computations that may be difficult to discern from measurements of individual neuronal firing rates 10, 12, 14-18. For instance, when an animal actively explores its environment, responses in sensory cortex are desynchronized, even when sensory input is disrupted and neuronal firing rates are unchanged 16.
Studying correlations is, however, inconvenient. It requires large amounts of data from simultaneously recorded neurons, and raw correlation values can be difficult to interpret. The recent advent of recording techniques such as multielectrode arrays and two-photon imaging has made obtaining such recordings easier, but has also highlighted the need to understand the experimental and physiological factors that can lead to different estimates and conclusions.
Correlation is a normalized measure of covariation. It has commonly been used to refer to two distinct phenomena. One use refers to tuning similarity, measured as the correlation in the mean responses of two neurons to an ensemble of stimuli (termed signal correlation or rsignal, see Figure 1A,C and Box 1). The second use of correlation—and our focus here—is as a measure of the degree to which trial-to-trial fluctuations in response strength are shared by a pair of neurons. This is typically quantified as the Pearson correlation of their spike count responses to repeated presentations of identical stimuli under the same behavioral conditions (spike count correlation or rSC, also called noise correlation; Figure 1A,B).
Co-fluctuations in the responses of a pair of neurons can arise over a range of timescales 8, 12, 22, 23, from the precise temporal alignment of spikes (i.e. synchrony) to slower changes in excitability (Box 1). The timescale over which correlated activity affects the responses of downstream neurons is unknown, but membrane time constants suggest that it is tens of milliseconds or less. However, most work on the relationship of spike count correlations to population coding and behavior has been based on responses measured over the duration of a stimulus presentation or behavioral trial (typically hundreds of milliseconds).
Over the past two decades, spike count correlations have been measured in many cortical areas under a variety of behavioral and stimulus conditions (Table 1). These studies have reported a range of values, but correlations are typically small and positive. They tend to be highest for pairs of neurons that are near each other 24-26 and have similar functional properties or tuning (high rsignal; 6, 7, 12, 13, 15, 25-31). Pairs recorded from opposite hemispheres have very low correlations7. These properties suggest that correlations reflect co-fluctuations in the responses of restricted subsets of neurons, rather than global fluctuations that affect all cells.
Correlation strength is also likely to depend on local circuitry or architecture. For instance, rSC is weak in the input layers of primary sensory cortex (32; M.A. Smith and A. Kohn, personal communication; J. Hansen and V. Dragoi, personal communication). Correlations in motor areas seem to be consistently lower than those in sensory cortex (see Table 1).
The influence of distance, tuning similarity, and architecture can explain some of the variability across studies, but even studies that sample similarly can arrive at quite different estimates of correlation strength. We show here that many of these discrepancies can be explained by differences in other factors that can systematically bias correlation estimates—namely, response strength, the time period for counting spikes, spike sorting, and fluctuations in internal states.
Why is it important to understand the influence of these factors and, more generally, differences in estimates across studies? It is not because the mean strength of correlations is a particularly critical quantity. Even very weak correlations can substantially affect the information encoded by a population of neurons, and the structure of correlations can have a much stronger influence than their mean strength2-6. Furthermore, other properties of the distribution of correlation coefficients, such as its variance, may be more informative about the underlying circuit than the mean 33. However, understanding differences in correlations across brain regions or in different stimulus or task conditions is critical both for elucidating their role in sensory processing and for making inferences about the circuitry and mechanisms that generate them. For this reason, we explore below the way that different experimental and physiological factors affect measurements of correlations and discuss guidelines for interpreting correlation data.
Correlations in pairs of neurons that fire few spikes per trial are weaker than in pairs that respond more strongly7, 8, 28, 34. Mathematically, correlations between discrete variables such as spike counts need not depend on their magnitude. Binary variables can be perfectly correlated, uncorrelated, or correlated to any intermediate degree. The dependence of rsc on spike count is therefore not a mathematical given but a biological and experimental phenomenon, as described below.
The number of spikes produced by a neuron is determined by its underlying firing rate and the time window over which responses are measured, and both factors vary considerably across studies. Firing rates depend on the stimulus, the animal's cognitive state, and the neurons’ tuning. In some studies, the stimulus is tailored to the tuning preferences of the neurons under study6, 12, 29. In others, a common stimulus is used to drive a large number of simultaneously recorded cells7, 26, 28, yielding weaker responses on average. In addition, neurons in some cortical areas, such as V1 and MT, are easier to drive than those in areas where stimulus preferences are less understood. The time window used to count spikes is purely an experimental decision, and thus also varies across studies. Below we show that both low firing rates and brief measurement windows can lead to lower measured values of rSC.
The relationship between rSC and firing rate depends largely on the proportion of subthreshold events that are masked by the spiking threshold34. Correlations in spiking responses arise because of cofluctuations in synaptic input, which give rise to correlated membrane potential fluctuations in pairs of neurons16, 33, 35-37. The degree to which shared membrane potential fluctuations are observable in spiking responses depends on the firing rates of the cells: when the mean membrane potential is far below threshold, responses are weak and many of the shared membrane potential fluctuations are unobservable in the spiking responses34, 37, 38.
This relationship is illustrated with a simple simulation shown in Figure 2 (similar to 34). We simulated correlated membrane potentials by picking values for each of two neurons from a bivariate Gaussian distribution. The membrane potential produces a spike response defined by the non-linearity shown in the middle panel. The shape of the nonlinearity is not critical; ours was chosen so that the variance of the spiking response is nearly the same as the mean, for both weak and strong responses39.
We set the membrane potential correlation to 0.2 in all of our simulations, but the measured spike count correlation depended on response strength. When the mean membrane potential is above threshold (Figure 2A), the correlation in spiking responses is similar to that of the subthreshold response. However, when the mean membrane potential is far below threshold (Figure 2B), rSC is markedly lower than between the subthreshold responses. This masking of correlated activity cannot be overcome by making more observations, which reduces the variance of correlation measurements but does not alter the mean. If membrane potential correlations are the same for stimuli that drive weak and strong responses, the spiking responses for the latter will thus be more correlated (Figure 2C), reaching asymptote at the strength of the underlying membrane potential correlation.
The dependence of correlation on response strength is typically assessed by comparing rSC to the geometric mean response of the two neurons7, 8, 28, 34. However, rSC will be reduced if either neuron responds weakly. For example, a pair in which one neuron has a mean response of 0.01 sp/s and the other 100 sp/s has the same geometric mean response as a pair whose mean responses are both 1 sp/s, but the measured rSC of first pair is only 15% of the underlying correlation compared with 85% in the second pair. Figure 2D shows rSC as a function of the response strength of the two neurons. A dependence of rSC on the geometric mean response would appear as diagonal stripes from the top-left to bottom right. Instead, Figure 2D has vertical and horizontal bands, indicating that the magnitude of rSC depends more on the minimum response of the two neurons than their mean.
The number of spikes a neuron fires also depends on the time window over which responses are measured. The studies in Table 1 use windows that range from tens of milliseconds to multiple seconds. Counting spikes over short epochs can lead to weaker observed values of rsc, even if both neurons are sufficiently responsive to avoid the effect of thresholding described in Figure 2.
We ran additional simulations to illustrate the dependence of rSC on measurement window (Figure 3). To do so, it was necessary to use a framework in which we could control the timescale of correlation. Since the simulations of Figure 2 do not specify a timescale, we instead imposed correlations by adding a small number of common spikes to the otherwise independent (Poisson) spike trains of two simulated neurons (Figure 3A,B; see also 40).
This simulation illustrates that correlations are systematically underestimated if the counting window is shorter than the jitter in the timing of the common spikes. If the common spikes occur at the same instant (Figure 3A), the resulting synchrony will be evident in spike count correlations based on responses measured over arbitrarily small windows (line marked ‘no jitter’ in Figure 3C). If the times of these common spikes are jittered (Figure 3B), however, the resulting spike count correlation will only be fully evident when it is calculated from responses during longer response epochs (lines labeled 5-80 ms in Figure 3C). For instance, when spike times are jittered using a Gaussian distribution with a standard deviation of 80 ms, a window of several hundred milliseconds is needed to capture the full strength of correlation. A similar dependence on measurement window is observed for Poisson distributed spikes conditioned on correlated underlying firing rates (see 22 and Supplementary Material for analytical description). Note that in this scenario correlations do not arise from nearly synchronous spikes, yet the observed correlations are still smaller for brief measurement windows.
Several studies have measured the timescale of correlation in cortex, providing estimates ranging from tens22 to a few hundred milliseconds8, 12, 23. Measurements using response windows briefer than these timescales are thus almost certain to yield weaker correlations than those using longer response windows.
Issues of recording quality or spike sorting can artifactually increase or decrease measurements of rSC. In general, errors that add independent variability to the responses of one neuron will bias estimates of correlations toward zero, while errors that involve combining the responses of multiple cells will increase the magnitude of measured rSC.
One spike sorting error that can lead to consistent overestimation of correlations is mistaking multiunit activity as spikes from a single neuron. Combining several units into one effectively averages out variability that is independent to each cell, so the correlation between two clusters of multiunit activity will be larger than between pairings of the constituent neurons. To quantify this influence, we simulated multiunit activity by grouping the responses of individual neurons produced from the simulation in Figure 2. We computed rSC between increasingly large groupings of neurons, made by simply summing the responses of individual units whose pairwise correlations ranged from very small (0.001) to more typical (0.1) values (Figure 4).
These simulations show that when pairwise correlations are weak, the measured value of rSC grows slowly with the number of units grouped together. Specifically, the value of rSC between multiunit clusters (rSC-measured) is given by:
where n is the number of units grouped together and rsc-pair is the pairwise correlation, which in this simulation was the same between all units, whether they were in the same or different groupings 41, 42. For nrsc-pair<<1, rSC thus grows linearly with the number of units n contributing to the multiunit clusters (i.e. as nrsc-pair). If, for example, the underlying pairwise correlation were 0.01, one would need to record simultaneously from clusters of nearly 20 cells to obtain the rSC of 0.2 that has been reported in many studies. Recordings from such large groups of cells would be evident by a proportional increase in firing rate and thus easy to distinguish from single unit activity.
It is important to note that excessively restrictive criteria in spike sorting can lead to an underestimation of rSC. Sorting spike waveforms in extracellular recordings is essentially a decision about when noisy voltage traces are similar enough that they are likely to have come from a single neuron. We considered a simple scenario in which the waveforms from two neurons were recorded on separate electrodes and clearly distinct, but each waveform was corrupted by noise (Figure 5A,B). We simulated increasingly stringent spike sorting by discarding a proportion of waveforms from each neuron. In this simulation, no spikes are mistakenly assigned to the other unit; changing the threshold simply alters the proportion of events that are accepted as valid spikes. As fewer spikes are accepted (meaning that the criterion for acceptance becomes more stringent), the measured value of rSC decreases (Figure 5C). Such oversorting (discarding valid waveforms) decreases rSC because whether a spike is accepted as valid or not is a random event—dependent strictly on noise—and variability that is independent for the two cells weakens measured correlation.
The relationship between the measured rSC (rSC-oversort) and the proportion of spikes discarded is given by
where p1, p2 are the probabilities that a spike is discarded from neuron 1 and 2, respectively, and n1, n2 are the spike counts of those cells (see Supplementary Material for derivation). In our simulation, the probability of deletion, p, is the same for the two cells (p1=p2), the two cells have equal rates and the Fano factor equals 1, so<n1>=<n2>=var(n1) =var(n2). In this case,
Therefore, when half the spikes are accepted, measured rSC decreases twofold (from 0.20 to 0.10). When the probability of discarding waveforms is different for the two cells (p1≠p2) or when the Fano factors are different from 1, the decay with sort stringency will be different from that depicted in Figure 5A-C. Because the denominator in Equation 2 is always greater than 1, however, rSC-oversort will always underestimate rSC-original.
A second scenario in which spike sorting can reduce measured correlations is when waveforms belonging to a single neuron are mistakenly assigned to multiple neurons (Figures 5D-F). We simulated this scenario by randomly dividing the spikes from a single neuron into two units (different shades of red in Figure 5D). We then assessed the correlations between these units and a single unit (Figure 5E) measured on another electrode. This manipulation reduced rSC by roughly 30% compared to the true underlying correlation (Figure 5F, open symbols compared to filled). When more than two units are created from a single unit, the measured value of rSC falls further (Figure 5F).
Measurements of rSC are based on sets of trials in which the stimulus and behavioral conditions are held as constant as possible. Despite experimenters’ best efforts, however, internal factors such as arousal, attention, or motivation are bound to vary. Similarly, in experiments using anesthetics, the depth of anesthesia may vary over time. Such fluctuations could in principle comodulate the responses of groups of cells and thus directly contribute to measurements of rSC.
It is impossible to determine experimentally the degree to which this is the case, as internal variables, by definition, are not under experimental control. However, comparing the timescale of fluctuations in internal states with that of correlations can provide important constraints on their contribution. In the absence of salient changes in the visual scene, animals can only shift their attention approximately once every 400 ms43-45. Even shifts in exogenous attention take 100-200 ms following an abrupt stimulus change43, 44, 46-49. Fluctuations in other cognitive states like arousal or motivation or in anesthetic state likely occur even more slowly.
In contrast, correlations are dominated by fluctuations on shorter timescales. The timescale of correlation can be estimated within a trial by comparing rSC computed in time windows of different sizes or by using the spike train crosscorrelogram8, 12, 22. The contribution of fluctuations that occur across trials can be estimated by calculating rSC between responses measured on different but nearby trials (e.g. by shifting the trials of one of the two neurons)22, 28. These measures reveal that correlations are typically dominated by fluctuations on the timescale of tens to a few hundred milliseconds; correlations between responses measured in different trials are usually near zero (22, 28; Kohn and Smith, Cohen and Maunsell, unpublished observations). Thus, although variations in cognitive factors or anesthesia states may well contribute to measured correlations, these correlations likely arise in large part from fluctuations on faster timescales.
The majority of studies that have measured spike count correlations have recorded neurons in primate sensory cortex. These studies report mean rSC values in the range of 0.1-0.2 in pairs of similarly-tuned, well-driven, nearby neurons (Table 1). In an apparent exception, a recent study found mean correlations in primary visual cortex (V1) that were on average slightly positive but very close to zero28. The authors concluded that variability in cortical neurons is independent and that the non-zero rSC measurements in many other studies were due to a variety of experimental artifacts.
The authors offer two primary explanations for this discrepancy. First, the authors argue that measured correlations in other studies were inflated because of poor spike sorting, either by mistakenly swapping spikes between two neurons or by misidentifying multiunits as single units. The former error is only possible when correlations are measured between pairs of neurons recorded on the same electrode, the situation considered in their spike sorting simulations. Of the studies in Table 1, only one relies on pairs recorded this way6, so this explanation cannot explain the departure from most previous results. That higher correlations in previous studies were the result of clustering single units into multiunits is also untenable. Previous studies would have needed to have mistakenly grouped responses from roughly 10-20 units with the near-zero correlations reported by Ecker et al. to explain the difference in mean rSC values (Figure 4). It seems inconceivable that all previous studies made mistakes of this magnitude. Furthermore, paired intracellular recordings—for which spike sorting is not an issue—have shown significant membrane potential and spiking correlations 16, 35, 36.
Second, Ecker et al. suggest that the correlations in previous studies arise from small fluctuations in uncontrolled cognitive factors such as the animal's internal state or behavioral factors like fixational eye movements, which produce “artifactual” correlations. As discussed above, the timescale of correlations makes it unlikely that they arise primarily from fluctuations in internal states. Furthermore, unlike other studies that asked subjects to perform difficult behavioral tasks (providing some means of controlling and assessing cognitive state), the subjects in Ecker et al. performed an easy fixation task. Therefore, the internal states of their subjects likely varied as much as, or more than, in previous studies. Fixational eye movements are also unlikely to be the primary source of measured correlations because positive correlations are reported in studies that specifically remove trials containing detectable eye movements 8, 15, 50 and in those that use anesthetized, paralyzed animals 12, 26. Fixational eye movements would also be expected to produce anti-correlations in some pairs (e.g. cells with offset spatial receptive fields, so that an eye movement that reduces drive to one cell would increase it to the other), but correlations are typically near zero or slightly positive in such cases 26.
Why then are the data of Ecker et al. so different from previous findings? A striking feature of the data in this study is that the firing rates were unusually low. Precise values were not provided, but summary plots suggest that the mean firing rate was a few spikes per second, and many cells had rates as low as 0.1 sp/s. Thus, neurons typically fired either a single spike, or no spikes, on each trial. As our simulations show, such weak responses—whether because neurons were only weakly driven or because of overly stringent spike sorting criteria—result in low measured values of rSC. A second contribution may be the location of the recordings. Correlations in the input layers of V1 are weaker than in other layers (M.A. Smith and A. Kohn, personal communication; J. Hansen and V. Dragoi, personal communication). If the data were recorded in part from those layers51, correlations would be weaker than in studies targeting other layers or visual areas.
The experimental factors discussed here – in addition to the influences of distance, tuning similarity, and architecture – can explain much of the diversity of reported rSC values. For instance, for studies in the primate visual system, differences in mean rate can account for more than 33% of the across-study variance in reported values of rSC (Figure 6). It is clear that these factors must be considered when comparing results across studies, cortical areas, and stimulus or behavioral conditions, although some are more likely to affect conclusions than others. For instance, when conclusions are based on the relative magnitude of correlations across conditions in a single experiment, spike sorting errors are unlikely to play a critical role. The dependence of rSC on firing rates, on the other hand, can be a key issue. Behavioral and stimulus manipulations often affect firing rates, and neurons in different areas or circuits may vary in their responsiveness to experimental manipulations. Alternative, rate-independent metrics, can be useful when rates vary across conditions38, 52, but determining the effect of stimulus and behavioral manipulations on correlations is most straightforward in cases when firing rates do not change12, 15 or when correlations decrease while firing rates increase7.
To date, most studies of correlations have focused on their impact on population encoding and decoding, used their properties to make inferences about network architecture, or made comparisons across conditions to study neuronal correlates of perceptual or cognitive events. Ultimately, the goal of studying correlations should also be to understand how they affect computations in cortex and directly contribute to behavior. The effort to establish a link between correlated variability in populations of sensory neurons, the activity of downstream neurons, and perceptual decisions is in its early stages. However, the studies reviewed below suggest that correlated variability directly contributes to perception and provide promising avenues for future research.
The best evidence that correlated variability in sensory neurons affects behavior is the well-established observation that fluctuations in the responses of individual neurons are predictive of an animal's perceptual decision (‘choice probability’; 53-55). Choice probability has been observed in a range of tasks and cortical areas, as early as primary visual cortex (56; I. Kang and J.H.R. Maunsell, personal communication; H.N. and B.G. Cumming, personal communication). If the activity of sensory neurons were independent, the responses of any individual neuron would be measurably correlated with behavior only if very few neurons contributed to perceptual decisions, making the probability of encountering these cells very low. A more plausible explanation is that because the responses of neurons are correlated, each neuron's response reflects the shared activity of a large group of cells that contribute to perceptual decisions.
Fluctuations in non-spike based measures of brain activity such as scalp recordings57, 58 and the BOLD signal measured in fMRI studies59-64 are also predictive of behavior. Fluctuations in such coarse signals presumably reflect the summed activity of many cells, suggesting that they are driven by correlated variability. Their relationship to decisions suggests that the underlying neuronal fluctuations affect behavior.
The relationship between behavior and fluctuations in neural activity suggests that correlated variability in sensory neurons affects perceptual decisions. The correlated variability underlying choice probability was originally thought to reflect noise in a bottom-up (i.e. sensory) signal, but recent evidence suggests that choice probability at least partially reflects top-down influences65. This raises the possibility that correlations in sensory neurons reflect rather than drive decisions, although it seems more plausible that the purpose of top-down signals is to influence decisions through their effect on pools of sensory neurons. Examining the time course of choice probability reveals that sensory neurons are correlated with decisions, and therefore with each other, during nearly the entire response to the stimulus65-68. Furthermore, both experimental69, 70 and theoretical studies71-74 indicate that sensory information is integrated until just before a decision is reported. Together, these results suggest that the correlated variability is present during, and not solely after, the period when perceptual decisions are made. However, establishing a causal link between correlated variability and perceptual decisions will require further research, and may require the development and application of techniques that allow experimenters to manipulate correlated activity directly.
Pearson's correlation is a simple, descriptive statistic. Its simplicity and the fact that it can be estimated from experimentally tractable amounts of data has made it the metric of choice for both experimentalists and theoreticians to explore issues of population coding and network function and connectivity. However, it is important to note that pairwise correlations provide a limited view of complex, population responses.
Recent theoretical work has begun to offer tools for providing a more complete and statistically-principled approach to measuring interactions among cells. Multivariate point process models provide a full description of joint response distributions, incorporating information about receptive field properties, spiking history, and interactions among cells 75-78. These methods provide means of simulating population responses, assessing the relative importance of single-cell properties and network influences on measured responses, and performing model-based decoding. Models of this type have been used to show that many of the observed fluctuations in single retinal ganglion cells 79 or neurons in sensorimotor cortex 80 can be accounted for by the current and past responses of other cells in the network. Future applications of these methods will undoubtedly provide important insights into issues of population coding and network structure.
Despite the difficulties in measuring and interpreting correlations, we have made important strides in characterizing their properties and how these depend on network state, behavioral experience and cognitive state81. Understanding which properties of correlations are most important—for instance, the most relevant timescales and sets of neurons—will ultimately depend on determining how population responses are interpreted by downstream networks, how they affect the computations performed there, and how they guide behavioral decisions. With recent improvements in experimental techniques and computational methods, this understanding feels very much within reach.
Signal correlation (rsignal) measures the correlation coefficient between mean responses to different stimuli. This measure is often used to quantify the extent to which a pair of neurons has similar tuning or other functional properties. Decreases in this type of correlation, for instance through adaptation 82 or contextual modulation 83, can lead to sparsening of population responses.
Spike count correlation (rSC, also called noise correlation) is the Pearson's correlation coefficient of spike count responses to repeated presentations of identical stimuli, under the same behavioral conditions. Spike counts are typically measured over the timescale of a stimulus presentation or a behavioral trial, which range from a few hundred milliseconds to several seconds (see Table 1). Spike count correlations are proportional to the integral under the spike train cross-correlogram (CCG; 22).
Synchrony measures the extent to which the timing of individual spikes is precisely aligned, typically on the timescale of one or a few milliseconds. It is typically quantified as a measure of a sharp peak in the CCG. Dependencies in the timing of individual spikes can also be measured in the frequency domain, using spike-spike coherence.
Long timescale correlation (rLT) measures the extent to which a neuron's response on one trial is correlated with a second neuron's response on trials in the future or past. It is used to quantify the influence of slow fluctuations in responsiveness on rSC. When measured, rLT has been found to be close to 0.
We are grateful to Peter Latham for analytical descriptions for our simulations and for the derivation of the relationship between measurement window and correlations that are not based on common spikes and to Ruben Coen-Cagli for the derivation relating correlation to the proportion of spikes discarded during spike sorting. We thank Ruben Coen-Cagli, Bruce Cumming, Mark Histed, Xiaoxuan Jia, Kresimir Josic, John Maunsell, Amy Ni, Hendrikje Nienborg, Alex Pouget, Odelia Schwartz, Seiji Tanabe, as well as Tony Movshon and Eero Simoncelli and members of their laboratories for helpful discussions and comments on an earlier version of the manuscript. This work was supported by NIH grants R01 EY016774 (AK) and K99 EY020844-01 (MRC).