PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
Vision Res. Author manuscript; available in PMC 2010 April 1.
Published in final edited form as:
PMCID: PMC2705991
NIHMSID: NIHMS124004

Linear and Nonlinear Systems Analysis of the Visual System: Why does it seem so linear?

A review dedicated to the memory of Henk Spekreijse

Abstract

Linear and nonlinear systems analysis are tools that can be used to study communication systems like the visual system. The first step of systems analysis often is to test whether or not the system is linear. Retinal pathways are surprisingly linear, and some neurons in the visual cortex also emulate linear sensory transducers. We conclude that the retinal linearity depends on specialized ribbon synapses while cortical linearity is the result of balanced excitatory and inhibitory synaptic interactions.

Keywords: Retina, visual pathway, contrast, linearity

Introduction

Linear and nonlinear systems analysis are tools that can be used to study communication systems. While it may seem that these methods are more appropriate for systems engineered by humans, in fact powerful insights about the biological mechanisms of sensory systems--especially vision--have been obtained through the use of systems analysis. This review paper makes this point by examples drawn from experiments inspired by systems analysis applied to the vertebrate retina and to the mammalian primary visual cortex (V1). The experimental results together with systems analysis reveal that the retina has specialized cellular and molecular mechanisms that enable the retina to provide a faithful, linear transformation of the visual image. After examining the retinal transformation, we will consider how V1 cortex reconstructs and then selectively distorts the neural image.

Many sensory neurons can be understood as stimulus-response transducers that are driven by sensory stimulation from the outside world. Such a neuron is quiet or in a background state in the absence of stimulation. Then, when presented with an appropriate stimulus, the neuron is either activated above its background level of activity or in some cases suppressed below background in a more or less consistent manner from one stimulus presentation to the next. This property is called staionarity. When the stimulation ceases, the sensory neuron’s activity relaxes back to the background state. This property is called finite memory. A response that is stationary and of finite memory applies to most sub-cortical sensory neurons that have been studied and so we can consider them as stationary, finite-memory transducers. In the sensory areas of the cerebral cortex there are neurons that behave as sensory transducers according to how we have defined the term here, though not all cortical neurons fit this description. A cortical neuron that is involved in memory or decisions or the initiation of action will have some activity that is not stimulus driven, and therefore such a neuron will not fit neatly into the definition of a transducer neuron. Linear and non-linear systems analysis techniques that we will be discussing in this paper are only applicable to neurons of the transducer type. Nevertheless, there are many neurons that can be understood as transducers and it is worth analyzing them in order to understand how neuronal networks can explain aspects of behavior.

Analysis of the visual system leads to the surprising conclusion that linearity is a rare and (apparently) prized commodity in neural signal processing. One reason that nonlinearity in neural information processing is the default is that neural communication is mainly through synaptic transmission, and most synapses are very nonlinear. The retina has very special synapses, the ribbon synapses that I will discuss later, and these specialized synapses appear to enable the retina to make use of linearity of signal transmission. The visual cortex seems to adopt a different approach because it does not have the luxury the retina has of handling continuous signals transmitted through ribbon synapses. The visual cortex must deal with the nonlinearity imposed by the spiking mechanisms of spiking neurons that feed it visual input. As I will describe, the cortex developed an intricate signal-balancing act to reconstitute a linear visual signal in the cortex. Cortical linearity is not simply the default result of convergence of excitatory inputs but rather requires extensive cortical computations. Thus the simplicity and elegance of linear systems are created in the visual system, and presumably also in other sensory pathways, by special efforts--by specialized synapses, or specially balanced networks. At the conclusion of this review paper I will offer some ideas about why the visual system works so hard to create, and then to reconstitute, a linearly filtered version of the visual world.

Linearity and nonlinearity in the vertebrate retina

Henk Spekreijse was a leader and an innovator in the application of systems analysis techniques to vision. He realized very early the importance of characterizing sensory transducers as linear or nonlinear. His early paper on linear and nonlinear analysis of visual responses in the goldfish retina (Spekreijse, 1969) applied an insightful method of linearizing neuronal responses with auxiliary signals to overcome the spike threshold nonlinearity of spiking neurons in the retina, the retinal ganglion cells. Figure 1 from his 1969 paper summarizes many of his results on the spike-rate responses of goldfish ganglion cell to sinusoidal light modulation. But before we consider Spekreijse’s specific findings and their implications, we will discuss briefly why he used sinusoidal modulation of signals to study linearity and non-linearity in retinal ganglion cells.

Figure 1
Ganglion cell responses in the goldfish retina (from Spekreijse 1969). The responses demonstrate the linearizing effect of an auxiliary signal (first column), internal noise (second column) and spontaneous spike discharge (third column). The calibration ...

Linear systems and sinewaves

We can learn the principles of linear systems analysis from the analysis of the simplest linear transducers: linear, single-input, single-output, or LSISO systems. An LSISO system will respond to a brief pulse of input of unit area with a response h(t), its impulse response. For an LSISO system, once we know the impulse response we know all there is to know about how the LSISO system will respond to any input. This is because linear systems obey the principle of superposition. Superposition means that the response of the cell to a sum of two stimuli, x+y, must equal the sum of the responses to the individual stimuli. Any realizable input can be decomposed into a sum of pulses at different times, of different heights. Based on superposition, the response of an LSISO system to this sum of pulses is simply the sum of its impulse responses to each of the pulses scaled appropriately. This process of summation is usually called convolution. Convolution is the basis for the linear synthesis of responses of LSISO transducers to any input (see Bracewell, 2000). This is why if we can measure h(t), the impulse response, then the LSISO system is understood completely.

Sinewaves are favorite stimuli to use in linear systems because they pass through unchanged in waveform and frequency, and are simply scaled and phase-shifted. One can prove this using convolution (Bracewell, 2000). Therefore, if a system’s response includes sinusoidal components not in the input, this is a certain indicator of some kind of nonlinearity, and is given the name “distortion”. If a system responds to a sinusoidal input with an undistorted sinewave output at the same frequency as the input, then it could be linear--at least such a system is emulating a linear system’s behavior to sinusoids. This is the reason Henk Spekreijse in 1969 was presenting sinewave-modulated light to the goldfish retina and monitoring the spiking rates of ganglion cells--to test for linearity or nonlinearity in retinal signal processing.

What Spekreijse (1969) reported was that usually there was a lot of distortion in the average spike-rate response of goldfish retinal ganglion cells (for example Figure 1B), but that under just the right conditions one could coax the retina to respond in a linear manner, that is to produce an undistorted sinewave output response (Figure 1C, F, H) to sinewave light modulation (Figure 1A). The nonlinearity he was exploring was the spike threshold of the ganglion cells, and the way he made the ganglion cells to yield linear responses was by using very small signals (Figure 1F) or by finding ganglion cells that had unusually high spontaneous spike-firing rates (Figure 1H) or by using an auxiliary signal added to the sinewave input as in Figure 1B. Hughes and Maffei (1966) had also found sinusoidal spike-rate responses in cat retinal ganglion cells. In the cat retina, the ganglion cells usually have moderately high spontaneous spike-firing rate, like the goldfish case in Figure 1H, and sinusoidal modulation of the firing rate was observable without the need for auxiliary stimuli.

Mechanisms of retinal linearity--ribbon synapses

There are many different implications of the ground breaking work of the Spekreijse (1969) paper, but for the purposes of this review paper I want to draw attention to the fact that it showed that the vertebrate retina could respond like a linear system to visual inputs all the way from the photoreceptors through to the retinal ganglion cells. Many vision scientists myself included have taken this result for granted and focused on the many interesting features of the nonlinear stages of signal processing in the retina, but the linearity exhibited in Figure 1 is most important for vision. Understanding what cellular and molecular mechanisms are needed for such linear signal processing remains a challenge for neuroscience.

What the retina is usually facing is a visual scene that has a mean light level and modulations above and below this steady level. In other words there is a continual bombardment of the retina by photons from the environment, and modulations of the rate of the photon flux comprise the visual message. In order to achieve a linear retinal response to modulations of the photon flux, the photoreceptors must be linear transducers around an operating point, and this they manage to do (Baylor et al., 1974; Tranchina et al. 1991).

The synapses from photoreceptors to bipolar cells and from bipolar cells to retinal ganglion cells must also operate in a linear manner around the mean level of synaptic release determined by the mean level of photon flux. I propose that the ribbon synapse depicted in Figure 2 (copied from DeVries et al 2006) is the crucial organelle that the retina uses to achieve linear signal processing (picking up a suggestion from Hochstein and Shapley, 1976a). The ribbon synapse is specialized for continual release of substantial quantities of glutamate packaged in synaptic vesicles (reviews in Witkovsky et al 2001; Heidelberger et al 2005; Heidelberger 2007). In the case of cone photoreceptors and their synapses, the light level at which cones usually operate is high enough that each cone will receive many photons/sec and the mean level of synaptic transmitter release will be determined by the mean light level. Then modulations of light intensity around this operating point, as in natural scenes or in laboratory experiments that mimic natural conditions, will modulate the amount of synaptic transmission above and below the mean level. In rod photoreceptor synapses the situation is different because the operating point is usually quite different from that of cones: lower mean light levels that usually provide less than 1 photon/sec to the rod. Then synaptic release is dominated by the dark current that depolarizes the rod (Hagins and Yoshikami, 1975), and photon events cause a large transient reduction in synaptic release. At the usual operating point of the rod synapse, nonlinearities in synaptic transmission do not matter because the retina is acting more as a photon counter, summing synaptic events coming from many different rods. Indeed, the rod--bipolar synapse is suspected of having a threshold nonlinearity (Sampath and Rieke, 2004) but the threshold may be low enough that even the rod synapse is approximately linear (Witkovsky et al, 2001; Robson et al 2004; Heidelberger et al, 2005). But for the normal operating range of the rod the amount of nonlinearity of the rod--bipolar synapse doesn’t much matter because the retina can operate in a linear manner by counting photon evoked events. So for the remainder of our discussion let’s consider the cone synapses and how they manage to transmit an undistorted signal at higher photon arrival rates.

Figure 2
Cone pedicle diagram (from DeVries et al 2006 with permission). Diagram of an invagination and surrounding region showing the ribbon (Rb), Horizontal cells (HCs), central invaginating contact (IC), and basal contacts (BC).

The photoreceptor ribbon synapse is presynaptic to both bipolar and horizontal cells. Evidence for linearity of synaptic transmission by means of sinusoidal driving of the photoreceptors by sinewave modulation of illumination has come mainly from experiments on horizontal cells because of the greater difficulties recording from bipolar cells though there are some data from bipolar cells that support the idea of the linearity of synaptic transmission from cones to bipolar cells (Marmarelis and Naka, 1973; Toyoda, 1974).

Spekreijse and Norton (1970) presented some of the earliest evidence for the linearity of the photoreceptor ribbon synapses in their study of horizontal cell responses in the goldfish retina. They used different temporal waveforms of illumination and observed horizontal cell response waveforms (Figure 3). For our purposes we will focus on the sinusoidal waveforms in Figure 3A. It is evident that the goldfish horizontal cells produced sinusoidal responses to sinewave modulation of the light, with little obvious distortion.

Figure 3
Goldfish horizontal cell (from Spekreijse and Norton, 1970). The top row demonstrates three types of periodic light stimuli: sinusoidal (A), squarewave (B), and (C) triangular-wave modulated light. The bottom row shows the corresponding responses for ...

Later, Tranchina et al (1981) performed another experiment using sinusoidally modulated light to study horizontal cells in the turtle retina (Figure 4). In Figure 4, the left panel shows the response waveform to a moderately high contrast (or modulation depth) of 0.5. As a test of how linear the transduction of the cone-horizontal cell synapse was, Tranchina et al. calculated the Fourier amplitude spectrum of the horizontal cell response--or in other words the best fitting sinewave at the modulation frequency (1st harmonic), at 2X the modulation frequency (2nd harmonic), at 3X the modulation frequency (3rd harmonic), and so on. The reason why they used the harmonics to test for nonlinearity is as follows. We can write an expression for the light stimulus in these experiments where we use the complex exponential to represent sinusoidal modulation:

I(t)=I0+I1cosωt.
Figure 4
Turtle horizontal cell responses (Tranchina et al 1981).

Here I0 is the mean retinal illumination, and I1 is the modulation, and I1/I0 is the contrast or modulation depth. The response of a linear system to this input will just be scaled and phase-shifted as:

R(t)=K0I0+K1(ω)I1cos[ωt+θ(ω])=R0+R1(ω)cos[ωt+θ(ω)]

But suppose we are dealing not with a linear system, but with a nonlinear system that is a square-law device, where R = MI2. The response of such a nonlinear system to a sinusoid will be:

RN(t)=(M0I0+M1(ω)I1cos[ωt+θ(ω])2==M02I02+2M1(ω)M0I0I1cos[ωt+θ(ω])+M1(ω)2I12cos2[ωt+θ(ω]

and the term on the right hand end of the equation

M1(ω)2I12cos2[ωt+θ(ω]=[1/2cos[2ωt+2θ(ω]1/2]M1(ω)2I12

is a sinusoid with a frequency twice the input frequency plus a constant term. In other words a square law device creates a 2nd harmonic--a harmonic distortion term-- plus a DC offset. A third power nonlinearity will generate the third harmonic, and so on (cf. Victor and Shapley, 1980).

The amount of harmonic distortion is therefore one quantitative measure of amount of nonlinearity, and this measure was used by Tranchina et al (1981) to assess cone--horizontal cell transmission. The relative heights of the harmonics in the turtle horizontal cell response are depicted in the middle graph of Figure 4. The 1st harmonic is tall and all the other harmonics are very short--consistent with a very linear transduction across the synapse. In other words, the cone ribbon synapse has very little harmonic distortion. In the right panel of Figure 4 the amplitudes of the 1st harmonic and 2nd harmonic responses are plotted vs stimulus contrast of the sinusoidal modulation. The 1st harmonic is proportional to contrast while the 2nd harmonic is negligible up to the highest contrast measured (around 0.5). In other experiments Tranchina et al. observed absence of harmonic distortion up to contrasts as high as 0.8.

There is some difference of opinion, possibly caused by species differences, in the precise mechanism of linear signal transduction at the cone ribbon synapse. Investigators of the goldfish retina and of the catfish retina have found evidence for the idea that horizontal--cone feedback modulates synaptic transmission at the ribbon synapse, and this modulation makes synaptic transmission more linear (Verweij et al 1996; Kraaij et al, 2000; Sakai and Naka 1987). Kraaij et al (2000) hypothesized that the early response of horizontal cells to a step increment of illumination was driven “open-loop” by the cones, while the later part of the step response was “closed-loop”, that is, influenced by horizontal--cone feedback. When they studied open-loop and closed-loop responses to different light intensities, they found that the ratio of horizontal/cone responses (what they called the gain of the cone synapse) was fairly linear--that is, proportional to light intensity--for the closed-loop response (Figure 5B), but quite nonlinear for the open-loop condition (Figure 5A).

Figure 5
Cone--horizontal transmission in goldfish retina (Kraaij et al 2000). The “open loop” early response (A) and “closed loop” from the later response (B) are gain-characteristic functions from cone to horizontal cells (HC) ...

Sakai and Naka (1987) found more evidence for horizontal--cone feedback’s influence on the linearity of synaptic transmission when they studied the size dependence of horizontal cell responses to noise-modulated light (Figure 6). They were recording the membrane potential of horizontal cells in the catfish retina, and stimulating the retina with a light stimulus that was a constant plus a Gaussian white noise (GWN) signal. Such a light stimulus appears to be flickering randomly in time. The use of such random signals to probe transducer properties is a powerful tool for studying linear and nonlinear systems. Schellart and Spekreijse (1972) reported one of the first experiments that used noise and cross correlation in their study of goldfish retinal ganglion cells. Sakai and Naka (1987) were using the Wiener kernel approach that had been pioneered by Marmarelis and Naka (1973). They calculated the first-order Wiener kernel by cross-correlation of the noisy input signal with the noisy neuronal response. The first-order Wiener kernel is the impulse response of the linear system whose response to the GWN signal is the best fit to the neuron’s response. Then Sakai and Naka made a test of how linear the cone--horizontal cell transmission was, by comparing the response of the best-fitting linear system with the response of the neuron. They did this comparison by calculating the power spectrum of the horizontal cell’s response, and comparing it with the predicted power spectrum of the response of the best-fitting linear system to the same GWN stimulus, as shown in Figure 6. In the figure the data are plotted with a solid line and the predicted response of the linear system is the dashed line. There were two experimental conditions: S means only a centrally located spot was stimulated by light and the rest of the visual field was dark; S/A means the spot was modulated in the presence of steady illumination of the surrounding region by an annulus of light that had the same mean retinal illumination as the spot. The data in Figure 6 show that in the S/A condition, the power in the horizontal cell’s response would be explained completely by the best-fitting linear system implying that the horizontal cell’s response was linear and also, therefore, cone--horizontal cell synaptic transmission was linear. But in the S condition when only the spot was illuminated, there was a significant discrepancy between the linear prediction and the measured response, implying significant nonlinearity in cone--horizontal cell transmission when only the spot was illuminated. It is known that horizontal cells have very large visual receptive fields and so it is reasonable to suppose that there was more sustained horizontal cell response in the S/A than in the S condition. Then the difference between the S and S/A conditions in linearity of cone--horizontal cell transmission could have been caused by the different amounts of horizontal cell response leading to different amounts of horizontal--cone feedback. A similar deviation between the linear prediction and measured power spectrum was reported for catfish bipolar cells (Sakai and Naka 1987).

Figure 6
Horizontal cell responses in the catfish retina (Sakai and Naka 1987). Power spectra of the (spot) light stimuli, of the spot-evoked responses (continuous lines), and of the linear models (dashed lines). Spectra marked by “S” were by the ...

The generality of the role of horizontal--cone feedback in linearity of synaptic transmission is called into question by the same kind of spot-annulus experiment applied to the turtle retina (Chappell et al 1985), as shown in Figure 7. As in the catfish retina, the presence of annular illumination affects the waveform of the measured first-order kernel (Figure 7A), and also the power spectrum of the horizontal cell’s response (Figure 7B). But in the turtle horizontal cell, the comparison of predicted power spectra with measured spectra (Figure 7B) yields a qualitatively different result: for the S condition as for the S/A condition, the linear prediction matches the measured power spectrum. This implies that the entire response of the turtle horizontal cell was consistent with linear transduction, whether the stimulus was a spot that did not engage much horizontal--cone feedback, or whether it was a modulated spot plus steady annulus that did evoke substantial feedback. Further work will be needed to clarify why there is a species difference between fish and reptiles (and other vertebrates) in the mechanisms of linearity at the cone ribbon synapses.

Figure 7
Horizontal cell responses in the turtle retina (Chappell et al 1985). A. First order kernels in response to Gaussian White Noise stimuli. F is for full field, S is for spot, S/A is for the same spot of light in the presence of a steady annular illumination. ...

One concern is that OFF-center (flat) bipolar cells that make synapses at the base of the cone pedicle might have different kinds of synaptic transmission from the ON-center bipolar cells that make invaginating synapses closely apposed to the synaptic ribbons, as diagrammed in Figure 2 (from DeVries et al 2006). Even if the ribbon synapses conferred linearity on the ON-pathway, one might wonder whether the OFF-pathway could have the same property of linearity if the synapses were not ribbon synapses. The results of De Vries et al (2006) put such concerns to rest. Investigating cone--bipolar transmission in the retina of the ground-squirrel, they found that both invaginating and flat bipolar cells are driven by transmitter released by ribbon synapses. The synaptic current is slightly delayed in the flat OFF bipolar cells compared to invaginating ON bipolar cells but this is almost compensated by the fact that metabotropic responses in invaginating ON bipolar cells are slightly delayed compared to the ionotropic responses in OFF cells. The single source of synaptic input to ON and OFF bipolar cells solves the problem of how there could be corresponding degrees of linearity of signal processing in the ON and OFF pathways.

Another finding about bipolar cells worth mentioning in the context of linearity of signal processing is the quasi-linearity of the membrane potential of isolated salamander bipolar cells driven by injected currents (Mao et al., 1998). While the bipolar cells have voltage gated currents that make them nonlinear transducers, around an operating point, a mean voltage level, their responses are quite linear in waveform and amplitude. The membrane nonlinearity reported by Mao et al (1998) may be involved in contrast adaptation or light adaptation--that is gain control processes that set the operating point around which the neurons may be modulated in a linear manner.

Retinal linearity--retinal ganglion cells

The linear transduction of the visual image would go to waste if signals were not transmitted without distortion to the retinal ganglion cells, the outputs of the retina. Now we will review the linearity of signal processing in cat retinal ganglion cells. The evidence suggests that linearity is preserved along some pathways all the way through the retina. A way to study this question is to examine the linearity of spatial summation of neural signals. A sinusoidal grating pattern, examples of which are shown in Fig. 8, is a useful tool for this task. If a ganglion cell is simply adding up neural signals, and there is no difference in the time course of the response from the different signal sources, then positions can be found at which introduction and withdrawal of the grating produce no response (left column, Figure 8; Enroth-Cugell and Robson, 1966). At these null positions the grating is placed so that introduction of the pattern produces as much net positive signal from one side of the cell’s receptive field as it produces net negative signal from the other side of the field; the two signals of equal magnitude but opposite sign cancel when added. The X-cell data in Figure 8 are from an OFF-center X cell; they are average spike-rate responses as functions of time, at two peak and two null spatial phases. Null positions can be found for X retinal ganglion cells in the cat retina, but not for Y cells (right side of Figure 8) that show signs of nonlinear signal distortion.

Figure 8
Cat retinal ganglion cells (Enroth-Cugell and Robson, 1966, 1984). Firing-rate records from a cat retinal OFF-center X-cell (left) and an ON-center Y-cell (right) responding to the appearance and disappearance of a sine-wave grating in different positions. ...

The early experiments of Enroth-Cugell and Robson (1966) employed a temporal modulation signal that was a step of contrast from zero to a fixed value. The time period when the contrast was stepped to a non-zero value is indicated in Figure 8 as the horizontal line. But one can use other temporal modulation signals, and in particular Hochstein and Shapley (1976a) suggested the use of sinusoidal temporal modulation in order to test the linearity of the retina by using the kind of temporal waveform analysis we have been considering in this review article. Thus, if one chooses the spatiotemporal stimulus to be a contrast-reversal grating,

I(x,t)=I0+I1sin(ϕx+ξ)cos(ωt)

where ϕ=2πk where k is spatial frequency in cycles/deg; spatial phase ξ=Q where Q is spatial offset in a fraction of a spatial cycle; temporal modulation frequency ω=2πf where the units of the f are cycles/sec or Hz. One can examine the average spike-rate modulation as a function of time and test for harmonic distortion just as investigators studied harmonic distortion in horizontal and bipolar cells. In response to a sinusoidal contrast reversal stimulus, the responses of X retinal ganglion cells follow a sinusoidal function of spatial phase ξ as illustrated in Fig. 9 (from Enroth-Cugell and Robson 1984). The data are from an experiment on a cat OFF-center X retinal ganglion cell. The sinusoidal spatial phase (position) dependence is a consequence of linearity of spatial summation (Hochstein and Shapley, 1976a). But in keeping with our focus on temporal waveform, it is quite salient that the temporal waveform of the ganglion cell’s average spike-rate response was a sinewave at the modulation frequency of the stimulus, and second harmonic distortion was very small.

Figure 9
Cat X retinal ganglion cell (Enroth-Cugell and Robson, 1984). Experimental results from a cat retinal OFF-center X-cell showing that the amplitude of the 1st harmonic component of the cell’s response to a sinusoidally contrast-reversing sine-wave ...

The experimental outcome is very different in cat Y cells as shown in Figure 10 (Enroth-Cugell and Robson, 1984), where the second harmonic component of the response is as large as the 1st. The relative sizes of the 1st and 2nd harmonics in the responses of Y cells depend on visual stimulus parameters like spatial frequency k, contrast I1/I0, and mean illumination I0.

Figure 10
Cat (OFF-center) Y retinal ganglion cell (Enroth-Cugell and Robson, 1984). The filled symbols represent the amplitude of the 1st harmonic component of the cell’s response to a sinusoidally contrast-reversing grating as a function of the spatial ...

Figure 11 (Hochstein and Shapley, 1976a) illustrates the spatial frequency dependence of a cat Y cell’s 2nd/1st harmonic ratio, as well as the spatial-phase invariance of the 2nd harmonic response that is also evident in Figure 10. Much scientific effort went into analyzing the nonlinear retinal pathways that drive Y cells in order to explain among other things the spatial frequency dependence, and the spatial-phase invariance, of the 2nd harmonic responses (Hochstein and Shapley, 1976a,b; Victor and Shapley, 1979a,b, among others) but these do not concern us in this paper. It is important to note that ON-center and OFF-center X cells are comparably linear in their response waveforms as indicated by harmonic analyses. The X cell in Figure 9 was an OFF-center cell, while the X cell that provided the data for Figure 11 was ON-center. One piece of explanation of this is, as we wrote above, that both ON-center and OFF-center bipolar cells are receiving input from cones through ribbon synapses (DeVries et al 2006). Also, the results imply that bipolar--ganglion cell signal processing is equivalently linear in both ON- and OFF-center pathways.

Figure 11
Cat X and Y cells (Hochstein and Shapley, 1976a) Contrast sensitivity of Fourier components to alternating grating as a function of spatial phase. Contrast sensitivity was determined with harmonic amplitude as the criterion.

Similar analyses of retinal linearity revealed the existence of retinal ganglion cells that resembled cat X cells in the retinas of many other vertebrate species: eel (Shapley and Gordon, 1978); rabbit (Caldwell and Daw, 1978); frog (Gordon and Shapley 1979); goldfish (Levine and Shefner, 1978 Bilotta and Abramov, 1989); mudpuppy (Tuttle and Scott 1979). It is reasonable to conclude that in the vertebrate retina there usually is a linear pathway for visual signals all the way from photoreceptors to retinal ganglion cells. I suggest that the ribbon synapses in photoreceptors are one component, and the ribbon synapses between bipolar and ganglion cells (Dowling and Boycott, 1966) are another necessary component of this linear pathway.

Sum of sinusoids analysis of linearity in retinal ganglion cells

Jonathan Victor developed a different approach for studying linear and nonlinear systems, and applied it first to studying the retina (Victor et al 1977; Victor and Knight 1979; Victor and Shapley 1980). This is the use of a sum of sinusoids (SOS) as a temporal modulation signal. For instance, instead of studying X cells with a contrast-reversal grating where the spatio-temporal stimulus as above was

I(x,t)=I0+I1sin(ϕx+ξ)cos(ωt)

Victor used as a stimulus

I(x,t)=I0+I1sin(ϕx+ξ)jcos(ωjt)

where the index j=1,2,..8, and ωj =2πfj where the units of the fj are cycles/sec or Hz. The SOS stimulus is illustrated in Figure 12. The eight different sinusoids in the sum are drawn there, as well as the sum at the bottom. The frequencies {fj } were selected very carefully. In fact, usually Victor et al (1977) used frequency sets where fj = (2j+21)/T (where T was the period of stimulation, approx. 30 sec.) so that sums and differences of pairs of frequencies were all distinct. Distinct output frequencies were useful for the following reason. In a linear system, the response to SOS must be a weighted sum of the input frequencies fj. Therefore, SOS with distinct output frequencies is useful for systems analysis because, as illustrated in Figure 13, if one Fourier analyzes the response of a system to SOS, the linear part of the response will appear at the input frequencies fj but non-linear response components will appear at the sum frequencies fj +fk or at the difference frequencies fjfk. The sum and difference frequencies are intermodulation distortions caused by the same nonlinearities that cause harmonic distortions; indeed second harmonic distortion 2fj = fj +fj is a special case of intermodulation distortion. In a way, the use of SOS is an extension of Spekreijse’s concept of linearizing (Spekreijse, 1969), but with many auxiliary signals. Figure 14 illustrates how we constructed the temporal frequency response of the best fitting first-order system (awhat we called the first-order frequency kernel, Victor and Shapley, 1979a) from the neuronal response to SOS modulation of a spatial pattern.

Figure 12
Sum of sinusoids (SOS) signal (from Victor and Shapley 1980).
Figure 13
Fourier analysis of responses to SOS (Victor and Shapley, 1979a). Low-frequency portions of the Fourier transform of the sum-of-sinusoids signal and a hypothetical ganglion cell response. The Fourier transform of the input sum-of-sinusoids signal is shown ...
Figure 14
First order responses to SOS (Victor and Shapley, 1979a). Construction of the first-order frequency kernel. The amplitudes of the first-order components of the response (see Fig. 13) are plotted on log-log coordinates as a function of the input frequency. ...

The use of SOS allowed us to test for linearity of signal processing definitively. The experimental result is presented in Figure 15 (Victor et al 1977). Data from one cat X cell are displayed in the right hand column in the figure, and Y cell data are on the left. The first-order responses are the curves in the upper panels, and the second-order intermodulation responses are graphed as contour lines depicting equal height along surfaces in the lower panels. For the purposes of this paper I will focus on the X cell’s data. While we were able to measure a large first-order response (10 spikes/sec amplitude of response) for the X cell, there was negligible second-order response at any pairwise intermodulation frequency, either among the sum frequencies or among the difference frequencies. This contrasts with the Y cell data where there was a large hill of response in the sum frequency quadrant, and a smaller but still measurable hill of response in the difference frequency quadrant. These results were typical; across the X cell population we studied, first-order responses were 5–10X bigger in summed amplitude than second-order responses (Victor and Shapley, 1979a).

Figure 15
SOS analysis of X and Y cell responses (Victor et al 1977). Linear response and second-order frequency kernels obtained from a Y cell (Left) and X cell (Right). The spatial stimulus pattern was a sinusoidal grating (0.5c/deg) placed in a position that ...

Before completing the part of this paper devoted to retinal signal processing, I need to add qualifications. The X cell pathway is remarkably linear for a pathway in a neural network but the retina is by no means a linear network. What we have considered is the linearity of signal transmission around an operating point of average illumination and average contrast. But the retina is also equipped with adaptation mechanisms to adjust input-output transductions. The first-order response of X cells is affected in a non-linear way by mean illumination (reviewed in Shapley and Enroth-Cugell, 1984) and by mean contrast (Shapley and Victor, 1978; Victor, 1987; Benardete et al, 1992; Sakai et al, 1995). Under normal operating conditions in viewing real scenes, the retina is working around an operating point, and the retina’s linearity around its operating point is remarkable.

Linear signal processing in simple cells of the primary visual cortex (V1)

Neurons in the primary visual cortex are classified as simple or complex, depending on how they respond to visual stimuli. If the response of the cell depends on the stimulus in an approximately linear fashion, the cell is termed “simple” and if non-linear then it is called a “complex cell.” For instance, in response to visual stimulation by the temporal modulation of grating patterns, the linearity of simple cells includes: (1) a sensitive dependence on the spatial-phase of the grating as in X retinal ganglion cells, (2) very little 2nd harmonic distortion. The responses of complex cells are very different: (1) they are spatial-phase-insensitive, and (2) their responses are predominantly 2nd harmonic (DeValois et al 1982; Spitzer and Hochstein, 1985; Figure 16). The linear dependence on visual stimuli of the simple cell might be assumed to be a simple consequence of convergence of excitatory drive from lateral geniculate nucleus (LGN) cells (Hubel and Wiesel, 1962; Reid and Alonso, 1995; Figure 17). However, such a feedforward model fails because of the nonlinearities of the LGN cells.

Figure 16
Simple cell responses to grating contrast reversal from De Valois et al. (1982) (with author’s permission),
Figure 17
Classic feedforward model from LGN to simple cells in V1 cortex. Adapted with permission from Hubel and Wiesel (1962). Four LGN cells are drawn as converging onto a single V1 cell. The circular LGN receptive fields aligned in a row on the left side of ...

The situation of the visual cortex is not like that of the retinal ganglion cells that are only two ribbon synapses away from the visual image. Spiking neurons, the ganglion cells and the LGN cells, are in the pathway to the cortex. Rectification caused by the spike-firing threshold produces nonlinear distortion of LGN responses for stimulus contrast >0.2 (Tolhurst and Dean, 1990; Shapley, 1994). Therefore, it is an open and important question, how can there be simple cells in the visual cortex? Wielaard et al (2001) offered an answer to this question by studying a large-scale neuronal network model of layer 4Cα in macaque primary visual cortex, V1. The choice of lateral connectivity within the Wielaard model is motivated not by Hebbian-based ideas of activity-driven correlations (Troyer et al., 1998), but by an interpretation of the anatomical and physiological evidence concerning cortical architecture. The crucial distinguishing features of the model, derived from biological data, are that the local lateral connectivity is nonspecific and isotropic, and that lateral monosynaptic inhibition acts at shorter length scales than excitation (Fitzpatrick et al., 1985; Lund, 1987; Callaway and Wiser, 1996; Callaway, 1998). We have tried to make a distinction between orientation preference and orientation selectivity. In the model, orientation preference is conferred on cortical cells by the convergence of output from many LGN cells (Reid and Alonso, 1995), with that preference laid out in pinwheel patterns (Bonhoeffer and Grinvald, 1991; Blasdel, 1992a,b; Maldonado et al., 1997). McLaughlin et al. (2000) showed that the orientation selectivity of cells in such a model of 4Cα is greatly enhanced by lateral cortico-cortical interactions.

Wielaard et al (2001) found that neurons in the large-scale network model behaved like V1 simple cells and this was a result of the cancellation of nonlinear LGN excitation by cortico-cortical inhibition. The neurons in the model were integrate-and-fire neurons with synaptic excitatory and inhibitory conductances modeled after those in real cortical cells. Therefore, the model outputs included intracellular conductances and membrane potential as a function of time. Its results could be compared with intracellular recordings of the membrane potentials of visual cortical cells, for instance the results of Jagadeesh et al (1997; Figure 18). The intracellular results are impressive because they reveal that even the intracellularly recorded membrane potential contains very little 2nd harmonic distortion (Figure 18) even though such distortion is present in the LGN input (Figure 19) and the Wielaard et al (2001) model can account for the intracellular results too. Please note that Figure 18 from the original paper contains two cycles of temporal modulation, so the response is at the fundamental frequency, not the 2nd harmonic.

Figure 18
Intracellular responses of a cat simple cell to sine wave contrast reversal at 2 Hz, shown over two cycles, from Jagadeesh et al. (1997) (with permission). This figure represents only half a cycle of spatial phase. The temporal modulation waveform is ...
Figure 19
LGN input to V1 cells in the Wielaard et al (2001) model. From contrast reversal gratings (at preferred orientation, 100% contrast, optimal spatial and temporal frequencies). Responses to nine different grating spatial phases are shown. One thick curve ...

The nonlinearity in the model is caused by LGN rectification, initially. The spatial arrangement of LGN cell receptive field centers that provide input to each cortical cell is as segregated ON–OFF subregions as in the feedforward model (Reid and Alonso, 1995). The ON-OFF segregation confers an orientation preference on the input to each cortical cell, and this preference (in the model) is laid out in pinwheel patterns. Additionally, the center of the receptive field of each cortical cell (created through the aggregate LGN input) is randomized. This was done to account for diversity in the location of this receptive field center, and possible random variations in the spatial symmetry of the ON–OFF subregions. The spatial arrangement of the LGN input confers a preferred spatial-phase on the LGN input of each cortical cell. From cortical cell to cell this spatial-phase preference is distributed randomly over a broad range, as has been found in experimental measurements (DeAngelis et al., 1999).

In response to contrast reversal, the summed LGN drive in the model has (for 100% contrast modulation) the generic spatial phase and time dependence shown in Figure 19. Notice that for each phase, the sinusoidal shape is significantly distorted. The absolute maxima of response occur at either 1/4 cycle or 3/4 temporal cycle of modulation, and the orthogonal phase case (with the lowest peak heights of response) has two peaks per cycle, that is, frequency-doubling or 2nd harmonic response. The 2nd harmonic is a consequence of the rectification in LGN firing rate. It is this 2nd harmonic that has to be eliminated for V1 cells to be simple cells.

The way the 2nd harmonic in the excitatory LGN drive is eliminated in simple cells in the Wielaard et al. (2001) model is by strong cortico-cortical inhibition. Thus, simple-cell-like intracellular and spike-rate responses to contrast reversal stimuli occur in the model because the model’s cortico-cortical inhibitory conductances have significant 2nd harmonic modulations that cancel the 2nd harmonic coming from the input. This statement is borne out by examining the inhibitory conductances in model neurons, as shown in Figure 20, where the 2nd harmonic in the conductance waveform is obvious. But why are such 2nd harmonic modulations present in the model’s inhibitory conductance? The reason is that the jth model neuron receives spikes from many other cortical neurons, each of which is responding individually in a manner sensitive to the spatial-phase of its own LGN drive. This individual spatial-phase dependence arises because each of these cortical neurons is driven by LGN excitation, and each summed LGN drive will have its own temporal waveform that will be one of those sketched in Figure 19. The excitation of each LGN cell is maximal at 1/4 or 3/4 temporal cycle. Because the cortico-cortical input to the jth neuron is an average over many such spatial-phase-sensitive responses, some of which peak at 1/4, some at 3/4 cycle, this results in a total cortico-cortical conductance that peaks at 1/4 and 3/4 of the temporal cycle, and consequently has significant 2nd harmonic content. In summary, the cortico-cortical conductances have large, phase-insensitive, 2nd harmonic modulations because the isotropic cortical architecture of the model allows an averaging over the activity of many cortical neurons, and thus, indirectly averages over the many preferred spatial phases of the LGN input [as suggested by the results in DeAngelis et al. (1999)], which peak at 1/4 and 3/4 cycle. This “phase averaging” by the network is similar to that used in a model for complex cells (Chance et al., 1999). It should be emphasized that although we have invoked phase averaging as the mechanism for producing frequency-doubled cortico-cortical inhibitory conductance, this state of cortical activity arises from the dynamics of the system in a way consistent with its architecture.

Figure 20
Constituent conductances for neurons near and far from pinwheel centers, at the null spatial phase of contrast reversal (from Wielaard et al 2001). Note the large inhibitory components of the conductances and the predominant 2nd harmonic.

The result of the interplay between excitation and inhibition in the model is that model cortical cells, unlike their LGN input, behave like simple cells in the contrast reversal experiment. Figure 21a–c shows data from an excitatory model neuron located near a pinwheel center. In Figure 3a both the “in-phase” and “orthogonal phase” membrane potential responses are shown. Here, the spike and reset mechanism of this neuron has been turned off—blocked—so that the waveform of stimulus-modulated membrane potential, Vb, can be seen more easily and compared with the experimental data of Jagadeesh et al (1997) where spikes were filtered out). Thus, the averaged waveforms of the membrane potentials in Figure 21a should be compared with those shown in Figure 18. There is a good degree of similarity. Extracellular spike counts for this same neuron (spike and reset now on) are displayed in Figure 21, b and c, as cycle averaged histograms, and these are comparable to the simple cell data in Figure 16. Figure 21a–c shows that model neurons have the linearity seen experimentally in simple cells. The response at the peak spatial phase is predominantly at the 1st harmonic of temporal modulation. The spike rate is not modulated at the second harmonic when the stimulus is at the “null phase”, and the membrane potential shows very little 2nd harmonic component in its null phase response, consistent with experimental measurements. If this explanation of simple cell function is correct, it means that the cortex is emulating some of the behavior of a linear system without actually being a network of linear elements. The apparent linearity of response time course and spatial summation are the result of a balance between nonlinear excitation and nonlinear inhibition, according to this way of thinking.

Figure 21
Responses in the Wielaard et al (2001) model to in-phase and null phase, contrast reversal gratings for an excitatory model neuron near a pinwheel center. The left column (a–c) shows responses for a representative fully coupled neuron, and the ...

To study the effect of lateral corticocortical interactions in the model, we shut them off. Figure 21d–f shows the results of a simulation with all network interactions shut off but with the LGN input the same as in the full network simulation shown in Figure 21a–c. For the null phase condition, notice the large amplitude of the 2nd harmonic in both the spike-rate response and the membrane potential. This 2nd harmonic response is inherited from the LGN input (as seen in Fig. 19 in the null phase LGN responses). The responses of an uncoupled model neuron are much larger than seen in the living cortex, because of the removal of strong inhibition in the model. Another approach to cortical modeling is to choose different input and internal noise parameters for the uncoupled model neurons to fit the background and peak firing rates of the real cortex. We did this and investigated the responses of what we called a “feedforward” neuron with much weaker LGN drive than in the full model. The results of the simulation for the feedforward neuron are shown in Figure 21gi. Compared with both the responses of the feedforward and uncoupled neurons, the membrane potential of the fully coupled neuron has a much smaller 2nd harmonic component, because of corticocortical interactions.

Our view that simple cells must be created by network interactions, and are not simply the default result of excitatory convergence, is supported by many experiments that reveal that the “simple” property of simple cells, the linearity, can be affected by unbalancing excitation and inhibition in the cortical network. Cells could be shifted from simple to complex by say weakening inhibition. This was observed (Fregnac and Schulz, 1999; Murthy and Humphrey 1999). The data of Murthy and Humphrey (1999) are particularly relevant. They stimulated their cat simple cells with grating contrast reversal as in our modeling and observed marked frequency doubling in spike rates of simple cells when bicuculline (which weakens GABA-ergic inhibition) was infused. The opposite effect, namely transferring a cell from the complex to simple group by increasing inhibition, was reported by Bardy et al (2006).

Simple and complex cells were first discovered in cat visual cortex (Hubel and Wiesel, 1962), and their existence confirmed subsequently in macaque V1 (Hubel and Wiesel, 1968; DeValois et al., 1982). Simple cells have been found in the primary visual cortex of many other species of mammals: owl monkeys (O’Keefe et al., 1998), baboons (Kennedy et al., 1985), tree shrews (Kaufmann and Somjen, 1979), rats (Burne et al., 1984; Girman et al.,1999), mice (Drager, 1975), rabbits (Glanzman, 1983), and sheep (Kennedy et al., 1983). Where there is a visual cortex, there are simple cells.

The modeling work suggests that the linear behavior of simple cells arises as a consequence of network activity in the cortical network. Why is linearity the cortical network’s goal? One idea is that, for visual perception, cortical cells must resolve and represent key spatial properties such as surface brightness and color, and also the perceptual organization of a scene. The existence of simple cells that respond selectively to spatial phase and monotonically to signed contrast are a requirement for the representation of surface properties. The large corpus of work on spatial vision requires linear spatial mechanisms to explain how patterns are detected separately and in combination (Graham, 1989; Wandell, 1995). Moreover, theories of color vision implicitly assume the existence of simple cells whenever they postulate the necessity of numerical computations of (signed) edge contrast (Wandell, 1995). A different function of vision also requires cells like simple cells. Scene organization requires computation of depth order that in turn depends on computation of stereoscopic depth and also of pictorial occlusion. Both stereo (Anzai et al., 1999a,b) and occlusion (Anderson, 1997) computations appear to require cortical representation of signed edge contrast. Also, the perception of salient contours embedded in a noisy field of distractors has been shown to be sensitive to spatial-phase and thus contrast sign of the elements of the contour (Field et al., 2000). Such neural computations would seem to require the linearity that only simple cells provide. Visual perception needs simple cells for important, basic functions. The retina and the visual cortex strive to create linear spatio-temporal elements to perform those functions. Recent work by Dr. Patrick Williams in my laboratory (Williams and Shapley, 2007), suggested that V1 reconstructs linear-looking simple cells in the input layer 4C, and these cells respond in an approximately linear manner not only to sinusoidal temporal modulation but also to steps of contrast of the kind they might experience after eye movements. The work of Friedman et al (2003) and also Williams and Shapley (2007) shows that V1 proceeds to make nonlinear but contrast-polarity-sensitive neurons in the upper layers of V1. The ultimate design goal appears to be neuronal sensitivity to contrast polarity. Further investigations in my laboratory by Drs. Chun-I Yeh and Dajun Xing are revealing more about the nonlinear transformation from layer 4C to layer 2/3 but the results are still preliminary. But it appears from what V1 cortex does (and from the linearly filtered signals sent from retinal ganglion cells) that a stage of neuronal processing that emulates a linear spatio-temporal filter is useful for further visual image processing in the visual system.

Usually the determination of the linearity or nonlinearity of signal processing is a first step in linear and nonlinear systems analysis. Then the goal of systems analysis is to obtain a set of measurements that identify or characterize the system structure by comparing the filtering properties of a system (linear or nonlinear) with a model system (for instance, see Enroth-Cugell and Robson, 1966; Spekreijse, 1969; Schellart and Spekreijse, 1972). What I have tried to show here is that even the first step of systems analysis may be very revealing about the function of a neural system and may lead us to uncovering unsuspected functional constraints in the neuronal mechanisms of the retina and of the brain.

Acknowledgments

This work was supported by grants R01 EY01472 from the US National Eye Institute, and NSF 0745253 from the US National Science Foundation. Thanks also to Drs. Daniel Tranchina and Jim Wielaard for helpful comments on earlier versions of the paper.

Supported by grants from the US National Eye Institute (R01 EY-01472) and the Sloan and Swartz Foundations.

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

References

  • Anderson BL. A theory of illusory lightness and transparency in monocular and binocular images: the role of contour junctions. Perception. 1997;26:419–453. [PubMed]
  • Anzai A, Ohzawa I, Freeman RD. Neural mechanisms for processing binocular information. I. Simple cells. J Neurophysiol. 1999a;82:891–908. [PubMed]
  • Anzai A, Ohzawa I, Freeman RD. Neural mechanisms for processing binocular information. II. Complex cells. J Neurophysiol. 1999b;82:909 –924. [PubMed]
  • Bardy C, Huang JY, Wang C, FitzGibbon T, Dreher B. ‘Simplification’ of responses of complex cells in cat striate cortex: suppressive surrounds and ‘feedback’ inactivation. J Physiol. 2006;574:731–50. [PubMed]
  • Baylor DA, Hodgkin AL, Lamb TD. The electrical response of turtle cones to flashes and steps of light. J Physiol. 1974;242:685–727. [PubMed]
  • Benardete E, Kaplan E. The dynamics of primate M retinal ganglion cells. Vis Neurosci. 1999;16:355–368. [PubMed]
  • Bilotta J, Abramov I. Spatial properties of goldfish ganglion cells. J Gen Physiol. 1989;93:1147–69. [PMC free article] [PubMed]
  • Blasdel G. Differential imaging of ocular dominance and orientation selectivity in monkey striate cortex. J Neurosci. 1992a;12:3115–3138. [PubMed]
  • Blasdel G. Orientation selectivity, preference, and continuity in the monkey striate cortex. J Neurosci. 1992b;12:3139 –3161. [PubMed]
  • Bonhoeffer T, Grinvald A. Iso orientation domains in cat visualcortex are arranged in pinwheel like patterns. Nature. 1991;353:429–431. [PubMed]
  • Bracewell R. The Fourier transform and its applications. 3 New York: McGraw-Hill; 2000.
  • Caldwell JH, Daw NW. New properties of rabbit retinal ganglion cells. J Physiol. 1978;276:257–76. [PubMed]
  • Callaway E. Local circuits in primary visual cortex of the macaque monkey. Annu Rev Neurosci. 1998;21:47–74. [PubMed]
  • Callaway E, Wiser A. Contributions of individual layer 2 to 5 spiny neurons to local circuits in macaque primary visual cortex. Vis Neurosci. 1996;13:907–922. [PubMed]
  • Chance F, Nelson S, Abbott LF. Complex cells as cortically amplified simple cells. Nat Neurosci. 1999;2:277–282. [PubMed]
  • Chappell RL, Sakuranaga M, Naka K-I. Dynamics of turtle horizontal cell response. J Gen Physiol. 1985;86:423–453. [PMC free article] [PubMed]
  • DeAngelis G, Ghose R, Ohzawa I, Freeman R. Functional microorganization of primary visual cortex: receptive field analysis of nearby neurons. J Neurosci. 1999;19:4046–4064. [PubMed]
  • DeValois R, Albrecht D, Thorell L. Spatial frequency selectivity of cells in macaque visual cortex. Vision Res. 1982;22:545–559. [PubMed]
  • DeVries SH, Li W, Saszik S. Parallel processing in two transmitter microenvironments at the cone photoreceptor synapse. Neuron. 2006;50:735–48. [PubMed]
  • Dowling JE, Boycott BB. Organization of the primate retina: electron microscopy. Proc R Soc, B, (Lond) 1966;166:80–111. [PubMed]
  • Draeger UC. Receptive fields of single cells and topography in mouse visual cortex. J Comp Neurol. 1975;160:269 –290. [PubMed]
  • Enroth-Cugell C, Robson J. The contrast sensitivity of retinal ganglion cells of the cat. J Physiol (Lond) 1966;187:517–552. [PubMed]
  • Enroth-Cugell C, Robson JG. Functional characteristics and diversity of cat retinal ganglion cells. Basic characteristics and quantitative description. Invest Ophthalmol Vis Sci. 1984;25:250–67. [PubMed]
  • Field DJ, Hayes A, Hess RF. The roles of polarity and symmetry in the perceptual grouping of contour fragments. Spat Vis. 2000;13:51– 66. [PubMed]
  • Fitzpatrick D, Lund J, Blasdel G. Intrinsic connections of macaque striate cortex afferent and efferent connections of lamina 4C. J Neurosci. 1985;5:3329 –3349. [PubMed]
  • Fregnac Y, Shulz D. Activity-dependent regulation of receptive field properties of cat area 17 by supervised Hebbian learning. J Neurobiol. 1999;41:69–82. [PubMed]
  • Friedman HS, Zhou H, von der Heydt R. The coding of uniform colour figures in monkey visual cortex . J Physiol. 2003;548:593–613. [PubMed]
  • Girman SV, Sauve Y, Lund RD. Receptive field properties of single neurons in rat primary visual cortex. J Neurophysiol. 1999;82:301–311. [PubMed]
  • Glanzman DL. Spatial properties of cells in the rabbit’s striate cortex. J Physiol (Lond) 1983;340:535–553. [PubMed]
  • Gordon J, Shapley R. Contrast Sensitivity and Spatial Summation in Frog and Eel Retinal Ganglion Cells . In: Armington JC, Krauskopf J, Wooten BR, editors. Visual Psychophysics and Physiology. Academic Press; New York: 1978. pp. 315–329.
  • Graham N. Visual pattern analyzers. Oxford: Oxford UP; 1989.
  • Hagins WA, Yoshikami S. Ionic mechanisms in excitation of photoreceptors. Ann N Y Acad Sci. 1975;264:314–25. [PubMed]
  • Heidelberger R. Mechanisms of tonic, graded release: lessons from the vertebrate photoreceptor. J Physiol. 2007;585:663–7. [PubMed]
  • Heidelberger R, Thoreson WB, Witkovsky P. Synaptic transmission at retinal ribbon synapses. Prog Retin Eye Res. 2005;24:682–720. [PMC free article] [PubMed]
  • Hirsch J, Alonso JM, Reid R, Martinez L. Synaptic integration in striate cortical simple cells. J Neurosci. 1998;15:9517–9528. [PubMed]
  • Hochstein S, Shapley R. Quantitative analysis of retinal ganglion cell classifications. J Physiol (Lond) 1976a;262:237–264. [PubMed]
  • Hochstein S, Shapley R. Linear and non-linear spatial subunits in Y cat retinal ganglion cells. J Physiol (Lond) 1976b;262:265–284. [PubMed]
  • Hubel D, Wiesel T. Receptive fields, binocular interaction and functional architecture of the cat’s visual cortex. J Physiol (Lond) 1962;160:106 –154. [PubMed]
  • Hubel D, Wiesel T. Receptive fields and functional architecture of the monkey striate cortex. J Physiol (Lond) 1968;195:215–243. [PubMed]
  • Hughes GW, Maffei L. Retinal ganglion cell response to sinusoidal light stimulation. J Neurophysiol. 1966;29:333–52. [PubMed]
  • Jagadeesh B, Wheat H, Kontsevich L, Tyler C, Ferster D. Direction selectivity of synaptic potentials in simple cells of the cat visual cortex. J Neurophysiol. 1997;78:2772–2789. [PubMed]
  • Kaufmann PG, Somjen GG. Receptive fields of neurons in areas 17 and 18 of tree shrews (Tupaia glis) Brain Res Bull. 1979;4:319 –325. [PubMed]
  • Kennedy H, Martin KA, Whitteridge D. Receptive field characteristics of neurones in striate cortex of newborn lambs and adult sheep. Neuroscience. 1983;10:295–300. [PubMed]
  • Kennedy H, Martin KA, Whitteridge D. Receptive field properties of neurones in visual area 1 and visual area 2 in the baboon. Neuroscience. 1985;14:405– 415. [PubMed]
  • Kraaij DA, Spekreijse H, Kamermans M. The open- and closed-loop gain-characteristics of the cone/horizontal cell synapse in goldfish retina. J Neurophysiol. 2000;84:1256–65. [PubMed]
  • Levine MW, Shefner JM. X-like and not X-like cells in goldfish retina. Vision Res. 1979;19:95–7. [PubMed]
  • Lund JS. Local circuit neurons of macaque monkey striate cortex: Neurons of laminae 4C and 5A. J Comp Neurol. 1987;257:60 –92. [PubMed]
  • McLaughlin D, Shapley R, Shelley M, Wielaard J. A neuronal network model of macaque primary visual cortex (V1): orientation selectivity and dynamics in the input layer 4Ca. Proc Natl Acad Sci USA. 2000;97:8087– 8092. [PubMed]
  • Maldonado P, Godecke I, Gray C, Bonhoeffer T. Orientation selectivity in pinwheel centers in cat striate cortex. Science. 1997;276:1551–1555. [PubMed]
  • Mao BQ, MacLeish PR, Victor JD. The intrinsic dynamics of retinal bipolar cells isolated from tiger salamander. Vis Neurosci. 1998;15:425–38. [PubMed]
  • Marmarelis PZ, Naka KI. Nonlinear analysis and synthesis of receptive-field responses in the catfish retina. II. One-input white-noise analysis. J Neurophysiol. 1973;36:619–33. [PubMed]
  • Murthy A, Humphrey AL. Inhibitory contributions to spatiotemporal receptive-field structure and direction selectivity in simple cells of cat area 17. J Neurophysiol. 1999;81:1212–24. [PubMed]
  • O’Keefe LP, Levitt JB, Kiper DC, Shapley RM, Movshon JA. Functional organization of owl monkey lateral geniculate nucleus and visual cortex. J Neurophysiol. 1998;80:594–609. [PubMed]
  • Reid RC, Alonso J-M. Specificity of monosynaptic connections from thalamus to visual cortex. Nature. 1995;378:281–284. [PubMed]
  • Robson JG, Maeda H, Saszik SM, Frishman LJ. In vivo studies of signaling in rod pathways of the mouse using the electroretinogram. Vision Res. 2004;44:3253–68. [PubMed]
  • Sakai HM, Naka K. Signal transmission in the catfish retina. V. Sensitivity and circuit. J Neurophysiol. 1987;58:1329–50. [PubMed]
  • Sakai HM, Naka K. Response dynamics and receptive-field organization of catfish ganglion cells. J Gen Physiol. 1995;105:795–814. [PMC free article] [PubMed]
  • Sakai HM, Wang JL, Naka K. Contrast gain control in the lower vertebrate retinas. J Gen Physiol. 1995;105:815–35. [PMC free article] [PubMed]
  • Sampath AP, Rieke F. Selective transmission of single photon responses by saturation at the rod-to-rod bipolar synapse. Neuron. 2004;41:431–43. [PubMed]
  • Schellart NA, Spekreijse H. Dynamic characteristics of retinal ganglion cell responses in goldfish. J Gen Physiol. 1972;59:1–21. [PMC free article] [PubMed]
  • Shapley R. Higher order processing in the visual system, Ciba Symposium. Vol. 184. Chichester, UK: Wiley; 1994. Linearity and nonlinearity in cortical receptive fields; pp. 71–87.
  • Shapley R, Gordon J. The eel retina: Ganglion cell classes and spatial mechanisms. J Gen Physiol. 1978;71:139–155. [PMC free article] [PubMed]
  • Shapley RM, Victor JD. The effect of contrast on the transfer properties of cat retinal ganglion cells. Journal of Physiology. 1978;285:275–298. [PubMed]
  • Shapley R, Enroth-Cugell C. Visual Adaptation and Retinal Gain Controls. In: Osborne N, Chader G, editors. Progress in Retinal Research. Vol. 3. Pergamon; London: 1984. pp. 263–346.
  • Spekreijse H. Rectification in the goldfish retina: analysis by sinusoidal and auxiliary stimulation. Vision Res. 1969;9:1461–72. [PubMed]
  • Spekreijse H, Norton AL. The dynamic characteristics of color-coded S-potentials. J Gen Physiol. 1970;56:1–15. [PMC free article] [PubMed]
  • Spitzer H, Hochstein S. Simple- and complex-cell response dependences on stimulation parameters. J Neurophysiol. 1985;53:1244 –1265. [PubMed]
  • Tolhurst D, Dean A. The effects of contrast on the linearity of spatial summation of simple cells in the cat’s striate cortex. Exp Brain Res. 1990;79:582–588. [PubMed]
  • Toyoda J-I. Frequency characteristics of retinal neurons in the carp. Journal of General Physiology. 1974;63:214–234. [PMC free article] [PubMed]
  • Tranchina D, Gordon J, Shapley R, Toyoda J-I. Linear information processing in the retina: A study of horizontal cell responses. Proc Nat Acad Sci US. 1981;78:6540–6542. [PubMed]
  • Tranchina D, Sneyd J, Cadenas ID. Light adaptation in turtle cones. Testing and analysis of a model for phototransduction. Biophys J. 1991;60:217–37. [PubMed]
  • Troyer T, Krukowski A, Priebe N, Miller K. Contrast invariant orientation tuning in cat visual cortex with feedforward tuning and correlation based intracortical connectivity. J Neurosci. 1998;18:5908 –5927. [PubMed]
  • Tuttle JR, Scott LC. X-like and Y-like ganglion cells in the Necturus retina. Invest Ophthalmol Vis Sci. 1979;18:524–7. [PubMed]
  • Verweij J, Kamermans M, Spekreijse H. Horizontal cells feed back to cones by shifting the cone calcium-current activation range. Vision Res. 1996;36:3943–53. [PubMed]
  • Victor JD, Knight BW. Non-linear analysis with an arbitrary stimulus ensemble. Quarterly of Applied Mathematics. 1979;37:113–136.
  • Victor JD, Shapley R. Receptive field mechanisms of cat X and Y retinal ganglion cells. J Gen Physiol. 1979a;74:275–298. [PMC free article] [PubMed]
  • Victor JD, Shapley R. The non-linear pathway of Y ganglion cells in the cat retina. J Gen Physiol. 1979b;74:671–689. [PMC free article] [PubMed]
  • Victor JD, Shapley R. A method of non-linear analysis in the frequency domain. Biophys J. 1980;29:459–484. [PubMed]
  • Wandell BA. Foundations of vision. Sunderland, MA: Sinauer; 1995.
  • Wielaard J, Shelley M, Mclaughlin DM, Shapley RM. How Simple Cells Are Made in a Nonlinear Network Model of the Visual Cortex . J Neurosci. 2001;21:5203–5211. [PubMed]
  • Williams PE, Shapley RM. A dynamic nonlinearity and spatial phase specificity in macaque V1 neurons. J Neurosci. 2007;27:5706–18. [PubMed]
  • Witkovsky P, Thoreson W, Tranchina D. Transmission at the photoreceptor synapse. Prog Brain Res. 2001;131:145–59. [PubMed]