PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of frontintegneuroLink to Publisher's site
 
Front Integr Neurosci. 2010; 4: 6.
Published online 2010 March 22. Prepublished online 2010 January 15. doi:  10.3389/fnint.2010.00006
PMCID: PMC2861478

An Emergent Model of Multisensory Integration in Superior Colliculus Neurons

Abstract

Neurons in the cat superior colliculus (SC) integrate information from different senses to enhance their responses to cross-modal stimuli. These multisensory SC neurons receive multiple converging unisensory inputs from many sources; those received from association cortex are critical for the manifestation of multisensory integration. The mechanisms underlying this characteristic property of SC neurons are not completely understood, but can be clarified with the use of mathematical models and computer simulations. Thus the objective of the current effort was to present a plausible model that can explain the main physiological features of multisensory integration based on the current neurological literature regarding the influences received by SC from cortical and subcortical sources. The model assumes the presence of competitive mechanisms between inputs, nonlinearities in NMDA receptor responses, and provides a priori synaptic weights to mimic the normal responses of SC neurons. As a result, it provides a basis for understanding the dependence of multisensory enhancement on an intact association cortex, and simulates the changes in the SC response that occur during NMDA receptor blockade. Finally, it makes testable predictions about why significant response differences are obtained in multisensory SC neurons when they are confronted with pairs of cross-modal and within-modal stimuli. By postulating plausible biological mechanisms to complement those that are already known, the model provides a basis for understanding how SC neurons are capable of engaging in this remarkable process.

Keywords: visual-acoustic neurons, anterior ectosylvian sulcus, enhancement, suppression, inverse effectiveness principle, neural network modelling

Introduction

Superior colliculus (SC) neurons integrate inputs they receive from multiple sensory modalities, thereby enhancing and speeding their responses to spatiotemporally coincident cross-modal stimuli (Rowland et al., 2007b; Stein and Stanford, 2008). This increases the reliability and accuracy of SC-mediated behavioral responses (Stein et al., 1989; Gingras et al., 2009). Because each sense operates independently and transduces a different form of energy, multisensory integration yields more informative products than can be obtained from any single sense (Ernst and Banks, 2002). The SC is particularly interesting because it receives converging, topographically-aligned unisensory inputs from many subcortical and cortical areas (Edwards et al., 1979), but inputs from association cortex (AES and rLS in the cat) are requisite for the development (Jiang et al., 2007), maintenance (Wallace and Stein, 1994; Jiang et al., 2001, 2002; Jiang and Stein, 2003; Alvarado et al., 2007a), and expression (Wilkinson et al., 1996; Jiang et al., 2002, 2007) of multisensory integration. When these cortices are inoperative, multisensory SC neurons retain the ability to respond to multiple sensory modalities but lose the ability to integrate their signals. Unfortunately, our understanding of this highly adaptive capacity is limited by our ignorance of the specific biological mechanisms by which it operates.

In an effort to reproduce the observed physiology, multiple models have been proposed to describe multisensory integration in the SC (Anastasio and Patton, 2003; Patton and Anastasio, 2003; Knill and Pouget, 2004; Rowland et al., 2007a; Ma and Pouget, 2008; Magosso et al., 2008; Martin et al., 2009; Ursino et al., 2009). While it is commonly assumed that excitatory inputs from different senses converge directly on individual SC neurons, different models suggest different mechanisms by which converging inputs (especially inputs from cortex) alter other signals (Anastasio and Patton, 2003; Patton and Anastasio, 2003). Some suggest that integration observed at the single neuron is an emergent property of the SC network itself (Knill and Pouget, 2004; Ma and Pouget, 2008; Magosso et al., 2008; Ursino et al., 2009). Here we present a model that incorporates both perspectives, while also emphasizing the importance of the cortico-SC projection. The model is informed by the most recent anatomical and physiological data supporting the fundamental hypothesis according to which the activity of the SC is almost completely controlled by cortical areas AES/rLS. It reflects non-AES/rLS inputs only when these areas are inactive. In the model, cortical inputs from different senses facilitate one another, while other non-cortical inputs compete through a winner-take-all (WTA) mechanism. The proposal is consistent with empirical findings that: multisensory integration is not an innate feature of this circuit, its postnatal development is protracted, and it adapts to the statistics of the animal's experience with cross-modal events. In other words, the cortex “knows” that certain cross-modal cues belong to common events and imposes this knowledge on the SC so that they are integrated. Other cross-modal inputs compete in driving SC-mediated responses.

Materials and Methods

General model structure

We provide a brief introduction to the essential model structure before describing its equations and parameters in explicit detail. Figure Figure11 provides a scheme of the model architecture.

Figure 1
The general structure of the network (A) and its physiological counterpart (B). The four projection areas (AES and non-AES) make excitatory connections (arrows) with the SC and with interneurons. The interneurons work in concert to provide two competitive ...

Neuron model

The output (“activation”) of each unit in the network is a continuous variable that changes according to both its own internal time constant and its changing inputs. As such, each unit should be interpreted as representing the collective activity of an ensemble of similar neurons on a given trial, whose responses will be compared to a single neuron's activity averaged over multiple trials in evaluating the model performance. During the simulation the network engages attractor dynamics and settles into a steady state, with the “response magnitude” of each unit to a simulated stimulus corresponding to its output at this time (Magosso et al., 2008; Ursino et al., 2009, for more details).

Input neurons

The inputs to the cat SC are simplified and represented as arrays of 100 units grouped into four sensory regions: visual inputs from AES (subregion AEV), auditory inputs from AES (subregion FAES), all non-AES visual inputs (“ascending” visual inputs), and all non-AES auditory inputs (“ascending” auditory inputs). Each input unit is sensitive to restricted but overlapping regions of space, which are topographically organized, so that a simulated stimulus will activate a restricted population of adjacent units. Units within each region exchange connections with one another that, for the sake of simplicity, may be excitatory or inhibitory. Excitatory connections are made with nearby units and inhibitory connections with more distant units.

SC interneurons

The model SC contains four different populations of inhibitory interneurons, each receiving input from a single unit within a specific input region (i.e., there are four separate arrays of 100 interneurons). The inhibitory interneurons receiving input from AES (Hv, Ha in Figure Figure1)1) effectively eliminate the influence of non-AES excitatory inputs when they are active. The inhibitory interneurons receiving input from non-AES sources (Iv, Ia) project to and inhibit one another. This means that, in the absence of AES input, stimulation of more than one non-AES sensory region will invoke a competition so that the stronger input will overwhelm the weaker (i.e., a WTA competition).

SC multisensory neurons

The SC multisensory neurons are modelled as an array of 100 topographically-organized units. Each SC neuron receives weighted input from the four input regions and from one of the members of each of the interneuron populations described above. Inputs derived from the sensory afferent populations are aligned topographically, so that a given SC neuron receives inputs from units in different sensory regions that are sensitive to the same region of space. SC neurons also exchange lateral connections that are locally excitatory but inhibitory at greater distances.

Mathematical description

Notational conventions

The model contains four sensory input arrays, four arrays of SC interneurons, and a single array of SC multisensory neurons. Each of these nine different arrays are referenced as follows:

  • Ca (cortical auditory): auditory AES (FAES) neurons:
  • Cv (cortical visual): visual AES (AEV) neurons;
  • Na: non-FAES auditory neurons;
  • Nv: non-AEV visual neurons;
  • Hv: inhibitory interneurons which receive input from AEV;
  • Ha: inhibitory interneurons which receive input from FAES;
  • Ia: inhibitory interneurons which receive input from the non-FAES auditory region;
  • Iv: inhibitory interneurons which receive input from the non-AEV;
  • Sm (superior colliculus multisensory): multisensory neurons in the SC.

Single neurons are referenced with superscripts indicating their array and subscripts that indicate their position within that array (i.e., indicating their spatial position/sensitivity). u(t) and z(t) are used to represent the net input and output of a given neuron at time t, respectively. Thus, zih(t) represents the output of a unit receiving net input uih(t) at location i within array h at time t.

Each excitatory connection linking two neurons in different regions, both at the same position i, is denoted Wih,k, where the first superscript (h) represents the receiving region and the second superscript (k) the projecting region. Inhibitory connections adopt the same convention but are denoted by a capital K instead of W. The lateral (excitatory or inhibitory) connections linking two neurons in the same region (but with different spatial position) are denoted Li,jh, where h is the region and the subscripts i and j represent the position of the target and projecting unit, respectively.

Unit model

The output of each unit in the network at each simulated moment in time is computed with a first-order dynamics of its input, which is transformed by a sigmoidal function. Specifically, for a unit i in region s with time constant τs receiving net input uis(t) at a moment in time t, its output is determined by the following differential equation (Eq. 1):

τsddtzis(t)=zis(t)+φ(uis(t))
(1)

where [var phi](us(t)) is a sigmoidal function with parameters [theta]s (the central point) and ps, which sets the slope at the central point (Eq. 2):

φ(us(t))=11+e(us(t)ϑs)ps
(2)

Thus, in this model, unit activity is limited to the range (0 1) as a convention (i.e., all neuronal activities are normalized to a maximum of 1). All units are initialized to an output of zero. The response of the unit for comparison to empirical data is taken as its output when the network reaches steady state in response to an external stimulus (see below).

Unisensory input regions

For simplicity, each unisensory input area is represented by an array of 100 units that receive input from external stimuli as well as from intrinsic lateral connections. External stimuli generate inputs that are functions of space (x) and time (t) to which a particular input area s is sensitive: Is(x,t). The receptive field of a generic unit i in an input area s is defined by a Gaussian function of space having a default maximum amplitude R0s, center xi, and standard deviation σis:

Ris(x)=R0se (xxi)22(σRs)2
(3)

As a consequence of Eq. 3, a stimulus presented at a particular position xi maximally excites unit i but can also excite adjacent units. The input an external stimulus provides to a generic unit i in input area s, ris(t), is determined by summing the products of the receptive field and the input stimulus for each spatial location:

ris(t)=xRis(x)Is(x,t)Δx
(4)

Unisensory input units within an area s also receive input through intrinsic lateral connections. The net lateral input, lis(t), is defined by the sum of the products of the weights of the lateral connections and the output of the projecting units for each location:

lis(t)=jLi,jszjs(t);
(5)

Lateral connections are symmetric and their weights (Li,js) are defined by a “Mexican hat” function derived by subtracting an inhibitory Gaussian function (max amplitude = Lins, std = σins) from an excitatory one (max amplitude = Lexs, std = σexs):

Li,js=Lexse [dx2]2(σexs)2Linse [dx2]2(σins)2
(6)

In this equation, dx represents the distance between the projecting and target units. Units at the extreme ends of a linear array potentially might not receive the same number of connections as other units (e.g., there are no units to the “left” of i = 1), which can produce undesired border effects. To avoid this complication, the array is imagined as having a circular structure so that each unit within an area receives the same number of lateral connections:

dx={ |ij|                       if  |ij|N/2N|ij|                if  |ij|>N/2
(7)

The net input received by a unit at position i in a unisensory input area s, uis(t), is the sum of the inputs from the external stimulus (Eq. 4) and the intrinsic connections (Eq. 5):

uis(t)=ris(t)+lis(t)
(8)

The output of these units in each unisensory input area is determined by Eqs 1, 2 and 8 where s is either Ca, Cv, Na, or Nv.

Interneuron populations

The four interneuron populations, each an array of 100 topographically-organized units, receive input from specific sensory input sources and send projections to the SC multisensory neurons and (in some cases) each other. Interneurons that receive input from AES areas have net inputs defined by the product of the activity of the topographically-aligned AES unit and the weight of the connection:

uiHa(t)=WiHa,CaziCa(t);
(9)
uiHv(t)=WiHv,CvziCv(t);
(10)

Interneurons that receive input from non-AES areas also receive inhibitory input from each other. Their inputs are computed as follows (K denoting an inhibitory connection):

uiIa(t)=WiIa,NaziNa(t)  KiIa,IvziIv(t);
(11)
uiIv(t)=WiIv,NvziNv(t)  KiIv,IaziIa(t);
(12)

The output of these units is determined by these input equations and Eqs 1 and 2 where s is Ha, Hv, Ia, and Iv, respectively.

SC multisensory units

SC multisensory units receive three types of inputs: weighted excitatory inputs from unisensory input areas (Ca, Cv, Na, Nv), inhibitory inputs from interneuron populations (Ha, Hv, Ia, Iv), and inputs from other SC multisensory neurons via intrinsic lateral connections. The different sensory inputs converging on an SC neuron are assumed to be in spatial register with one another (Meredith and Stein, 1996; Kadunce et al., 2001), and here we assume that the inhibitory interneurons have a matching spatial topography.

The direct excitatory inputs from AES are not subject to inhibition and their net inputs are computed as the product of the synaptic weight and the output of the upstream unisensory neuron:

uiSm,Ca=WiSm,CaziCa
(13)
uiSm,Cv=WiSm,CvziCv
(14)

The non-AES inputs are subject to a multiplicative (“shunting”) inhibition (as is the case in GABAa-mediated inhibition, see Koch, 1998) from all of the interneuron populations with matching location, producing more complicated input equations:

uiSm,Na(t)=WiSm,NaziNa(t)(1KiSm,HaziHa(t))   ×(1KiSm,HvziHv(t))(1KiSm,IvziIv(t))
(15)
uiSm,Nv(t)=WiSm,NvziNv(t)(1KiSm,HaziHa(t))×(1KiSm,HvziHv(t))(1KiSm,IaziIa(t))
(16)

Finally, each SC multisensory neuron also receives lateral input from other SC neurons:

liSm(t)=jLi,jSmzjSm(t).
(17)

For the sake of simplicity, we assume these connections can be either excitatory or inhibitory, with strengths conforming to a Mexican hat disposition, as in the unisensory input areas (see Eqs 5 and 6 above, where s = Sm).

The net input to a multisensory unit i is computed as the sum of all of these inputs:

uiSm(t)=uiSm,Ca(t)+uiSm,Cv(t)+uiSm,Na(t)+uiSm,Nv(t)+liSm(t)
(18)

Its output is computed from this input using Eqs 1 and 2 where s = Sm.

Fixed parameters

The values of all model parameters for the majority of the simulations are shown in Table Table1.1. To avoid limiting the model to a description of just one particular neuron or experiment, and to enhance its usefulness to the general audience, we fixed some of these parameters for simplicity and (as described in the Results) adjusted others to describe their influence on the unisensory and multisensory response properties of the model. For the most part, the parameters describing the input/output transformations of the units themselves (i.e., in Eqs 1 and 2) are fixed for different populations, while the strengths of connections between different populations and the properties of the external stimulus are varied experimentally.

Table 1
Parameter values.

Fixed unit properties

The time constant (a few milliseconds) is fixed for all units and is consistent with those normally used in deterministic mean-field equations (Ben-Yishai et al., 1995) and the time constants of SC neurons measured in vivo (Grantyn and Lux, 1988). For the units in the input areas and the SC multisensory units, the central abscissa of neurons, [theta]s, is selected to produce a small amount of baseline activity (i.e., without any external stimulus), and the slope of the sigmoidal relationships, pS, is assigned so that there is a smooth transition from silence to saturation in response to different input magnitudes. For interneuron units, the slope and the central abscissa of the sigmoidal relationships have been assigned so that there is a fast transition from silence to saturation in response to inputs coming from unisensory input areas, but little baseline activity in the absence of any external stimulation. This allows the implementation of a strong competitive mechanism even in the presence of a moderate stimulation. The standard deviation of the visual receptive fields of input units (σRs, s = Cv, Nv) has been selected to be approximately 10° in diameter and the standard deviation for the auditory receptive fields (σRs, s = Ca, Na) is selected to be approximately 15° in diameter. R0s (s = Cv, Nv, Ca, Na) is set to 1 to fix a scale for the external input.

Lateral interactions within unisensory input areas

Extant data suggest that the simultaneous presentation of a stimulus in the best area of the SC neuron's unisensory receptive field with another within-modal stimulus outside the receptive field can suppress the response by as much as 40% (Kadunce et al., 1997). Furthermore, we found it necessary to set the inhibition strength of these connections sufficiently high to suppress uncontrolled propagation of excitation to the overall area. Finally, an arbitrary decision was made to restrict the parameters of these lateral connections so that the presence of an external stimulus produced an activation bubble of neurons which is approximately equal to the size of the input receptive field. These constraints fixed the values of LexsLins,σexs and σins.

Lateral interactions between SC multisensory neurons

Extant data suggest that two cross-modal stimuli placed within overlapping regions of their respective receptive fields produce enhanced responses (Stein and Meredith, 1993), while two within-modal stimuli in the same configuration yield no enhancement or even a marginal suppression at the boundary (Alvarado et al., 2007a,b). Furthermore, two cross-modal or within-modal stimuli placed far apart (i.e., one inside, another outside the receptive field) cause significant suppression (Kadunce et al., 1997, 2001). These observations were used to fix the values of LexSmLinSm,σexSm and σinSm.

Connections between non-AES inputs and SC multisensory neurons

The parameters of direct connections from the unisensory non-AES input areas to the SC multisensory neurons (i.e., WiSm,Nv and WiSm,Na) were fixed relative to the strengths of the AES-derived connections (see below) so that unisensory responses were 50% depressed when AES was deactivated, as recently reported in (Alvarado et al., 2007a,b).

Connections with and between interneuron units

Connections from unisensory input areas to their topographically-aligned interneuron units (i.e., WiHv,Cv, WiHa,Ca, WiIv,Nv, and WiIa,Na) were selected so that even moderate levels of activity would drive near-maximum activity in their target interneuron populations. Furthermore, the inhibitory influence of these interneuron populations stimulated by unisensory input areas was set to a maximum value (i.e., KiSm,s = 1 for s = Hv, Ha, Iv, Ia, see Eqs 15 and 16). Inhibitory connections between interneurons stimulated by non-AES inputs (KiIv,Ia and KiIa,Iv) were balanced to implement a WTA competition between the two at a given location i in the non-AES route so that, during cortical deactivation, the stronger non-AES unisensory input would overwhelm the weaker and its influence would be seen in the output of the multisensory SC neuron (see Eqs 11 and 12).

Results

After fixing many of the model parameters based on some of the available physiological data in order to codify the model's essential structure (see above), we now evaluate how the model reproduces other physiological data by adjusting the few remaining free parameters (e.g., stimulus efficacy and the strength of the connections between AES inputs and the SC, i.e., WiSm,Cv and WiSm,Ca).

The operation of the intact model

Dynamic range

An initial set of simulations was performed to verify that the model would replicate the normal behavior of a typical SC neuron in response to different simulated modality-specific and cross-modal stimuli having different efficacies and spatial configurations. In each case the results of the model simulations were compared to those obtained from physiological recordings of individual SC neurons. The first test compared the output of a model neuron to one described in detail by (Perrault et al., 2003), in which visual and auditory stimuli were presented either individually or simultaneously at the same location in space. For this particular neuron, the visual stimulus evoked a stronger response, and so we assigned a slightly higher value to WSm,Cv than WSm,Ca. In this experiment, the efficacy of each modality-specific stimulus was systematically manipulated to explore the response magnitudes evoked by each stimulus or stimulus complex (i.e., the “dynamic range”). The empirical observation was that, as the efficacy of the individual modality-specific stimuli increased, the multisensory response evoked by their combination was larger. However, measured proportionately, the enhancement evoked by their combination was largest when the individual modalities were weakest, otherwise known as the “principle of inverse effectiveness” (Meredith and Stein, 1986a; Stein and Meredith, 1993; Wallace et al., 1998; Perrault et al., 2003, 2005; Stanford et al., 2005; Stein et al., 2009). These observations are common among SC neurons.

As illustrated in Figure Figure2,2, the model robustly accounts for each result and produces responses very similar to those reported in the empirical literature: (a) the model produces the same pattern of multisensory enhancement; (b) multisensory responses evidence a greater dynamic range than unisensory responses (i.e., a single stimulus cannot lead the SC neuron to saturation) (c) the model transitions from a superadditive computation to an additive computation at higher levels of stimulus effectiveness, and (d) the model reproduces inverse effectiveness. Enhancement in the model is related to the presence of the sigmoidal relationship in SC neurons and the presence of two simultaneous inputs from different sensory modalities that interact synergistically. A small modality-specific input cannot be strong enough to produce a significant response in the sigmoidal function of the SC neuron, but if it is coupled with another weak stimulus, this combination could produce an appreciable result in the sigmoidal curve. This cannot be replicated by two within-modal stimuli because a saturation occurs within the unisensory input layers when two stimuli use the same channel. It should be noted that the “tail” of the response (i.e., at the very lowest levels of effectiveness) is unlikely to represent a statistically significant response from the perspective of the physiologist.

Figure 2
Behavior of the intact network – Dynamic Ranges (DRs). The figure shows the activity of SC neurons in response to different inputs in the case of the intact model (i.e., with AES active and all the membrane receptors working). In all simulations ...

Spatial configuration

Results from physiological experiments have shown that SC multisensory integration is highly dependent on the spatial configuration of the stimuli. Response enhancement is elicited when the cross-modal stimuli are within their respective receptive fields, which overlap one another in space and, thus, the stimuli are in close spatial correspondence. However, within the region of overlap, response magnitude is generally the same regardless of small shifts in spatial displacement (Kadunce et al., 2001). Multisensory enhancement is not a consequence of simple stimulus “redundancy,” as equivalent results are not obtained when two within-modal stimuli are placed within a receptive field. In the latter case there is marginal or no response enhancement (Alvarado et al., 2008). Nevertheless, when either a cross-modal or a within-modal stimulus pair is arranged so that one stimulus is within its receptive field and the other is outside its receptive field (so that the stimuli are spatially disparate), there is either no response enhancement or response depression. This spatial principle of SC multisensory integration has been repeatedly demonstrated both physiologically (Meredith and Stein, 1986b, 1996; Stein et al., 1993; Stein and Wallace, 1996; Wallace et al., 1996, 1998; Kadunce et al., 1997) and behaviorally (Stein et al., 1989; Wilkinson et al., 1996; Jiang et al., 2002; Burnett et al., 2004).

We conducted simulations in which single or multiple auditory (Figure (Figure3A)3A) or visual (Figure (Figure3B)3B) stimuli were presented alone or together in different spatial configurations, and found similar results. In each example, the auditory or visual stimulus was placed in the center of the receptive field. To this stimulus we added another (within- or cross-modal stimulus) at different spatial disparities (sometimes inside, sometimes outside the receptive field). All stimuli were sufficiently robust to evoke strong responses. When cross-modal stimuli were in spatial register (i.e., at 0°) there was substantial (100–150%) response enhancement (compare solid and dotted lines in Figure Figure3).3). When the cross-modal stimuli were spatially-displaced, but still within the overlapping receptive fields, the magnitude of the response enhancement was generally the same. However, when the second stimulus, regardless of modality, was located far outside its receptive field and clearly disparate from the first, it significantly depressed the SC response. Conversely, when two high intensity within-modal stimuli were in spatial register, the response did not change from that obtained by the presentation of a single stimulus (compare dashed and dotted lines in Figure Figure3).3). If the two within-modal stimuli are extremely weak (i.e., the output of the unisensory neurons lies at the very bottom portion of the sigmoidal relationship) one can actually observe some within-modal enhancement, but this remains subadditive. Moreover, results demonstrate that if two stimuli are moved away, enhancement progressively decreases with the distance, until it is converted into a depressed response when one of the two stimuli is outside the RF of the SC neuron. The model explains that the absence of within-modal enhancement is due to the fact that the saturation of unisensory input neurons precludes higher responses. Multiple within-modal stimuli compete with one another within the respective modality-specific input regions for access to the cortico-collicular input channel. On the other hand, lateral inhibitory synapses within the SC are responsible for the cross-modal depression and they, coupled with their counterparts in the unisensory input areas, account for within-modal depression.

Figure 3
Behavior of the intact network – Integration as a function of the position of two stimuli. The figures show the response of the intact network to paired stimuli in different spatial configurations. Simulations are made by stimulating the model ...

These observations demonstrate that the model functions quite well in the intact condition. It behaves in the same way as does the biological circuit when dealing with multiple stimuli of different modalities, different spatial configurations and different levels of effectiveness. However, an effective way of testing the functional viability of the model is to examine its reaction to critical conditions and compare it to those of the actual biological circuit. Perhaps the best such condition is the deactivation of AES.

The operation of the model under simulated deactivation of AES

Recent evidence suggests that not all inputs to the SC neuron factor equally in determining the multisensory response. Indeed, empirical data reveal that the deactivation of AES eliminates multisensory integration in SC neurons, but does not eliminate their unisensory responses (Wallace and Stein, 1994; Jiang et al., 2001; Alvarado et al., 2007a, 2009), although recent evidence suggests there is a reduction in their magnitude (Alvarado et al., 2007a). The same essential observation is made when individual subregions of AES are deactivated (e.g., AEV or FAES, see (Alvarado et al., 2009)). However, when individual subregions of AES are deactivated, only the responses that are sensitive to inputs from that region are affected (Alvarado et al., 2009). In the present case, deactivation was simulated by assigning a value of 0 to the appropriate input areas, be they derived from the simulated regions AEV, FAES, or both. This effectively silences the AES-derived input to both multisensory SC neurons and to the inhibitory interneurons.

Dynamic range

When a modality-specific input (e.g., auditory) is simulated in the model during cortical deactivation, a response is generated in the SC target neuron because its matching non-AES input is still active and no longer subject to AES-induced suppression. When the entire AES is deactivated, the unisensory responses are smaller (Figure (Figure4B),4B), with saturation at about 0.1–0.2 (10–20%) of the maximum activity, a finding that parallels the physiology. Also, and more importantly, multisensory enhancement to spatially concordant cross-modal stimuli is lost throughout the dynamic range: the multisensory response is not significantly greater than the response to the more effective of the two component stimuli.

Figure 4
Behavior of the network as function of AES cortex. These figures compare the activity of SC neurons in response to different inputs with AES active (A) or inhibited, fully (B) or only partially [AEV inhibited (C), FAES inhibited (D)]. In all simulations, ...

When only AEV (Figure (Figure4C)4C) or FAES (Figure (Figure4D)4D) was deactivated, the effects on the SC target neuron were modality-specific: deactivation of AEV affected its visual responses but not its auditory responses and the reverse occurred with deactivation of FAES. However, even subregional deactivation eliminated multisensory enhancement. Conceptually, one might describe this result as indicating a “synergy” between the unisensory inputs derived from the subregions of AES (Alvarado et al., 2009). In the present model this synergy is an emergent property of the circuit. In the complete absence of AES, the influence of non-AES inputs is exposed. Because modality-specific non-AES inputs contact inhibitory interneurons that suppress the influence of other modalities, cross-modal stimuli generate signals that inhibit one another, reaching a stalemate response that represents no enhancement. Thus, in the complete absence of AES, the competition results in a multisensory response no better than the response to one of the component stimuli. However, when either AES subregion is intact, it suppresses the influence of all non-AES inputs through the interneuron population. Thus, when a cross-modal stimulus is presented, one sees only the influence of the stimulus that corresponds to the non-deactivated modality.

It must be noted that there is always variability in the electrophysiological observations both within and across neurons, and the model can account for those as well. A slightly different behaviour can be obtained assuming a weaker competition. Figure Figure55 shows results of a sensitivity analysis, in which weaker competition is simulated by progressively reducing the strength of the inhibitory synapses between the interneurons in the non-AES pathways (i.e., parameters  KiIa,Iv and  KiIv,Ia in Eqs 11 and 12) during total cortical deactivation. In the case of strong competition, the SC response to cross-modal stimuli resembles the response to the stronger modality-specific stimulus. Conversely, assuming weak competition, both interneurons Ia and Iv display non-zero activity and inhibit the excitation from the ascending path of the different sensory modality. As a consequence, the SC response to cross-modal stimulation becomes even smaller than the stronger individual unisensory response. This apparently paradoxical result, which is a consequence of the competition in the ascending routes, has been observed empirically (see Figure 9 in Jiang et al., 2001).

Figure 5
Sensitivity analysis of the strength of inhibitory competition in the ascending path. The figure shows the activity of SC neurons (continuous lines) in response to different cross-modal inputs during total AES deactivation (the same case as in Figure ...

Spatial configuration

Observations that AES deactivation in the model leads to the prohibition of SC multisensory enhancement prompted additional simulated experiments to investigate whether other multisensory interactions, specifically those leading to depression, would be similarly affected. The empirical results in this case are less clear-cut, but the general observation is that in most cases the deactivation of AES yielded a slight reduction in the amount of depression that was induced when spatially disparate cross-modal stimuli were presented (Jiang et al., 2002).

Figure Figure66 shows the response of the model to two simulated stimuli at different spatial disparities (using the same design as in Figure Figure3)3) after deactivation of AES. The enhanced response to spatiotemporally concordant stimuli was lost, as expected (see also Figure Figure4),4), and the overall response was much weaker than in the intact condition. Although cross-modal and within-modal depression were still evident when the stimuli were spatially disparate, cross-modal depression was weaker than in the intact condition. This weak inhibition is due to the presence of lateral inhibition within the SC area. Since SC neurons are weakly activated by non-AES inputs, the competition between them is weakened.

Figure 6
The effect of AES on integration. The same simulations as in Figure Figure33 performed after inactivation of AES. Results show: (1) a reduction in the SC response both to a unisensory and to a multisensory stimulation; (2) the loss of multisensory ...

Effect of NMDA deactivation

One common idea on multisensory integration is that it depends on the temporal coincidence of stimuli from different senses. A component of biological circuits popularly conceptualized as engaged in coincidence detection is the NMDA receptor, which acts as a type of biological AND-gate. There are experimental results that suggest that SC multisensory integration depends on the functional integrity of NMDA receptors, as responses to cross-modal stimuli are reduced during the application of the NMDA receptor antagonist AP5 (Binns and Salt, 1996). Moreover, responses to visual stimuli are greatly reduced, whereas inconsistent results are reported for responses to auditory stimuli. These results suggest that NMDA receptors play a greater role in the transmission of visual than auditory information in the SC. These observations were incorporated in the model by assuming that deactivation of NMDA receptors greatly reduces the strength of all AEV-SC synapses (see Table Table1)1) but does not significantly affect FAES-SC synapses. This aspect of the model is purely speculative at present, although it works in reproducing experimental data. This issue deserves a detailed analysis in the laboratory.

The impact of this manipulation is similar to deactivating AEV, and the results of simulations where stimulus intensity has been varied are shown in Figure Figure7A.7A. By comparing these results with those in Figure Figure22 it is apparent that the cross-modal and visual responses are strongly reduced, but the auditory response is only moderately degraded. The reduction of the strength of the synapses from AEV produces a similar effect as deactivating it. Also of note is that the relationship between neuronal activity and stimulus intensity changes from the non-linear one in the intact condition, to a quite linear one after NMDA deactivation (see Figure Figure7B).7B). These results agree fairly well with the physiological results reported by (Binns and Salt, 1996) (see also Rowland et al., 2007a).

Figure 7
Behavior of the network with NMDA receptors deactivated. The upper panel shows the activity of SC neurons after deactivation of NMDA receptors, in the same simulations as in Figure Figure2.2. Deactivation of NMDA receptors causes a 43% decrease ...

A more direct comparison between model and physiological results is provided in Figure Figure8,8, where the model results are shown for input intensity level close to saturation of the unisensory neurons. The simulated visual response reduction was 43.4% (45 ± 9% in Binns and Salt, 1996); the auditory response reduction was 6.7% (−4 ± 21% in Binns and Salt, 1996); the multisensory response reduction was 62.6% (59 ± 7% in Binns and Salt, 1996); and the sum of the single modality responses was reduced by 27.9% (26 ± 10% in Binns and Salt, 1996).

Figure 8
Unisensory and multisensory responses with NMDA receptors active or inhibited. Activity was assessed by presenting to the network auditory (dark-grey bars), visual (light-grey bars) and multisensory (black bars) inputs (with a high level of intensity, ...

Sensitivity analysis

To gain a deeper understanding on model behavior, we preformed a sensitivity analysis on some parameter changes. In particular we analyzed (i) the WTA mechanism; (ii) the role of lateral inhibition in the SC and in unisensory areas; iii) the key role played by interneurons. Results can be summarized as follows:

WTA mechanism

The analysis was performed by varying the intensity of the two ascending inputs and the relative strength of the inhibitory synapses linking the interneurons Ia and Iv. Results show that in case of strong competition (as with the basal values reported in Table Table1),1), only the higher input survives and there is no enhancement. When the competition is weak, both ascending inputs survive and inhibit one another, producing a depressed response. In general, the mechanism is quite robust and works even in the presence of a small unbalance between the two inputs.

Lateral synapses

With the basal value of parameters (Table (Table1),1), lateral inhibition in the SC plays a greater role than lateral inhibition in unisensory areas, for what concerns both cross-modal and within-modal suppression. However, by changing the strength of lateral synapses, different situations can be mimicked (for instance, if stronger lateral inhibition is used in unisensory areas but weaker lateral inhibition in the SC area, one can observe within-modal suppression without cross-modal suppression, as experimentally found in some SC neurons, Kadunce et al., 1997).

Inhibitory interneurons

The mechanisms realized by means of interneuron populations are robust despite moderate small variations in their parameters. Conversely, if the inputs to inhibitory interneurons Ia and Iv are dramatically reduced the SC exhibits enhancement even in case of AES deactivation. Moreover, if the afferents to interneurons Hv and Ha are dramatically reduced, the model exhibits cross-modal enhancement even in case of subregional AES deactivation (only AEV or only FAES). Both results are clearly in contrast with experimental evidences, emphasizing the critical role played by all inhibitory connections to reproduce the correct behaviour of the SC.

Discussion

The present network model is able to account for and provide a mechanistic explanation for the major physiological observations pertaining to SC multisensory integration (see review of Stein and Stanford, 2008). This includes multisensory enhancement and suppression, inverse effectiveness, the effects of selective cortical deactivation, NMDA blockade, and the differing responses and underlying computations that characterize responses to pairs of spatially disparate cross-modal and within-modal stimuli. This was achieved by including a limited number of biologically realistic mechanisms, some of which are known to be in place in this circuit and others of which require physiological verification.

The proposed model presents a new perspective on multisensory integration in the SC. Some extant models have assumed that multisensory integration reflects a synergistic amplification of cross-modal signals at the level of the single neuron (Rowland et al., 2007a), while others assume that integration is an emergent property based on network dynamics (Patton and Anastasio, 2003; Magosso et al., 2008; Ursino et al., 2009). The former do not incorporate the fact that the individual SC neuron is embedded in a network in which there is the potential for interactions between units that can affect responses, while the latter do not incorporate the fact that different circuit components do not appear to play equal roles in multisensory integration. The present model joins this discussion by merging these two perspectives in an architecture in which multisensory integration is an emergent network property only when certain components of the circuit are engaged.

The basic hypothetical assumption of the model is that in the cat the operation of the SC is, under normal circumstances, almost entirely controlled by the sensory inputs derived from AES and rLS. One of the most important mechanisms included in the model is that of nonlinearity. All neurons exhibit a non-linear characteristic, with lower threshold and upper saturation. Another essential feature of the model is the presence of two competitive mechanisms that are expressed via separate sets of interneurons. One of these mechanisms is initiated by descending inputs from AES that can inhibit the non-AES (“ascending”) inputs, and is necessary to simulate the loss of multisensory enhancement occurring after deactivation of a single AES subregion (e.g., AEV or FAES, see Alvarado et al., 2009). The second mechanism assumes a strong competition between the two ascending sources, so that the dominant ascending input causes the near complete inhibition of the other in a “Winner Take All” dynamic (WTA). Although empirical support for two competitive mechanisms via two different sets of interneurons is still needed, inhibitory interneurons are common in the SC and have recently been shown to be involved in the AES projection to multisensory neurons (Fuentes-Santamaria et al., 2008, 2009). A parallel involving non-AES inputs remains to be determined, but the arrangement predicted by the model is reasonable as it provides the substrate for a competition that would reflect the physiological effects of AES deactivation: loss of multisensory integration and a multisensory response that becomes similar to that evoked by the dominant modality-specific stimulus (Wallace and Stein, 1994; Jiang et al., 2001; Alvarado et al., 2007a, 2009). WTA dynamics in the ascending path is also hypothetical at present. A more sophisticated “sensory fusion strategy,” able to exploit the presence of cross-modal inputs in spatiotemporal register, may be implemented only in the descending path after incorporating the experience with such correlated sensory input (Stein and Stanford, 2008).

Actually, although some model assumptions (such as the arrangement of inhibitory synapses, the presence of both additive and shunting inhibition, and the effect of NMDA deactivation) are hypothetical, they are already in common use in neural networks. It is remarkable that this is one of the functions of a model – to bring together what is known and help generate new hypotheses to guide future empirical studies (in this case physiological studies at both extracellular and intracellular levels, and anatomical studies documenting the underlying circuitry).

Despite the successes of the model, it should be understood that in addition to the simplifications and assumptions noted above, the role of rLS in this process was ignored. This was justified by the observation that many SC neurons depend solely on AES, and in others AES appears to be the more important mediator of SC multisensory integration (Jiang et al., 2001). Given the model's operational effectiveness, it may be that a parallel rLS-SC circuit exists that can work independent of, or in concert with, the proposed AES-SC circuit. The presence of parallel circuits that have the potential to expand to substitute for one another during early life would be in keeping with physiological observations. Unlike the deleterious effects of disrupting the integrity of either rLS or AES in adults, ablating them individually early in life has no obvious effects on the maturation of SC multisensory integration. Only their combined loss precludes the maturation of this capacity (Jiang et al., 2007).

A previous model developed by (Rowland et al., 2007a) also leans heavily on the roles of AES and interneurons. It assumes that multisensory SC neurons receive both direct ascending and descending AES inputs, as well as indirect projections from each of these sources via inhibitory interneurons. A fundamental difference between the two models, however, is that in the (Rowland et al., 2007a) model converging inputs descending from different sensory subdivisions of AES preferentially target the same electrotonic compartment of the SC target neuron, whereas the two ascending inputs preferentially target different compartments. Since in this model each compartment exerts a squaring function, the descending inputs exhibit a synergistic interaction. This choice allows the different role of descending and ascending inputs to be simulated using a single population of interneurons, whereas the present model assumes two interneuron populations with different inhibitory roles. The advantage of the present strategy is that it can account for the AES deactivation-induced reduction in the unisensory responses of multisensory SC neurons. In addition, assuming a chain of neurons with receptive fields at different positions interacting via lateral synapses (the Mexican hat formation), rather than a single neuron, allows the model to account for response depression with spatially disparate cross-modal (and even within-modal) stimuli. Furthermore, the synaptic configurations in the model are also likely to be highly sensitive to early sensory experience and can serve as the foundation for how these experiences alter the circuit essential for the emergence and maintenance of multisensory integration (Wallace and Stein, 1997; 2000, 2001; Stein, 2005).

Additional model predictions pertain to the nature of its internal inhibitory dynamics. Inhibitory interactions that lead to multisensory depression are likely to depend on lateral inhibition, a feature expressed in the model via a Mexican hat disposition for synapses within afferent sources of inputs as well as within the SC itself. This assumption can explain the initiation of suppression when cross-modal or within-modal paired stimuli are spatially disparate (Meredith and Stein, 1996; Kadunce et al., 1997). Although such an arrangement is well documented in cortex (Rolls and Treves, 1998), it is less well documented in subcortical structures. Nevertheless, a consequence of this assumption is the model's prediction that spatially disparate cross-modal and within-modal stimuli would produce response depression even after AES deactivation (see Figure Figure6).6). Although this prediction still requires validation, some support for it already exists. (Jiang and Stein, 2003) showed that deactivation of AES and rLS severely compromised, but did not eliminate multisensory depression from the population of SC neurons, and only minor effects were noted on unisensory responses (within-modal suppression was not examined). The authors speculated that “… there are several afferent sources that mediate multisensory depression in SC neurons.”

NMDA receptors are thought to play a substantial role in SC multisensory integration (Stein and Meredith, 1993; Binns and Salt, 1996; Rowland et al., 2007a), and potentially do so via a preferential effect on the visual channel (Binns and Salt, 1996). The effect of instantiating this in the present model was a reduction in synaptic strength of AEV-SC inputs during NMDA blockade that proved effective in simulating the percentage changes in SC responses to modality-specific and cross-modal stimulation, as well as the non-linear to linear shift in the multisensory responses. However, the specificity of the presumptive NMDA influence (one afferent source from one sensory modality) requires empirical validation.

A limitation of the present model is that it does not account for the temporal aspects of multisensory enhancement. More particularly, model dynamics is very rapid: this signifies that two stimuli must occur in very close temporal proximity to interact and induce enhancement or suppressive effects. Conversely, data in the literature suggest the existence of a wider temporal window (about 200 ms) to have multisensory integration (Meredith et al., 1987; Maruff et al., 1999; Holmes and Spence, 2005). In order to improve this aspect, model needs more sophisticated dynamics able to sustain the input for a longer period before its decay.

In conclusion, the incorporation of additional biophysical elements into this model represents a substantial advancement that provides a new appreciation of the complexity of the implementation of multisensory integration in this circuit. The longevity and ultimate utility of this model rely on several aspects. First, the model described here shows the ability to summarily describe, within a unique theoretical structure, the massive physiological observations on SC neurons; thus it may be of value to physiologists to help interpretation of unisensory and multisensory response properties of SC neurons. Second, the model can be manipulated to understand additional properties of this circuit that are already known. For example, the model described here assumes a particular pattern of connectivity between specific populations of neurons and interneurons; however how this architecture may come into being as a consequence of normal maturation and development have yet to be explored. This can be the focus of future research. Moreover, the present model can make testable predictions that can help guide future experiments in order to validate, reject, or modify the main hypotheses. For example, one fundamental assumption of the model is the pivotal role of AES-initiated inhibitory mechanisms in suppressing other tectopetal sensory inputs. A second pivotal hypothesis is the ability of the stronger of these non-AES tectopetal inputs to suppress the weaker of them when AES is rendered non-functional. The role of these inhibitory effects can be tested in an experiment that combined SC infusion of a GABA antagonistic, AES deactivation and SC recording during cross-modal stimulation. The model predicts that under these circumstances, AES deactivation would not preclude SC multisensory integration as is normally the case (Wallace and Stein, 1994; Jiang et al., 2001) because the intrinsic SC inhibition that normally block it under these circumstances has been removed. Furthermore, in cases in which only one of the unisensory AES tectopetal input is deactivated (either AEV or FAES) SC multisensory integration would not be eliminated as is normally the case (Alvarado et al., 2009), because the active AES cortex cannot inhibit all non-AES tectopetal inputs as would be required to preclude multisensory integration.

Conflict of Interest Statement

The authors declare that the research was conducted in absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

The research described was supported in part by NIH grants EY016716 (BES) and NS036916 (BES), and a grant from the Wallace Research Foundation (BES).

References

  • Alvarado J. C., Rowland B. A., Stanford T. R., Stein B. E. (2008). A neural network model of multisensory integration also accounts for unisensory integration in superior colliculus. Brain Res. 1242, 13–2310.1016/j.brainres.2008.03.074 [PMC free article] [PubMed] [Cross Ref]
  • Alvarado J. C., Stanford T. R., Rowland B. A., Vaughan J. W., Stein B. E. (2009). Multisensory integration in the superior colliculus requires synergy among corticocollicular inputs. J. Neurosci. 29, 6580–659210.1523/JNEUROSCI.0525-09.2009 [PMC free article] [PubMed] [Cross Ref]
  • Alvarado J. C., Stanford T. R., Vaughan J. W., Stein B. E. (2007a). Cortex mediates multisensory but not unisensory integration in superior colliculus. J. Neurosci. 27, 12775–1278610.1523/JNEUROSCI.3524-07.2007 [PubMed] [Cross Ref]
  • Alvarado J. C., Vaughan J. W., Stanford T. R., Stein B. E. (2007b). Multisensory versus unisensory integration: contrasting modes in the superior colliculus. J. Neurophysiol. 97, 3193–320510.1152/jn.00018.2007 [PubMed] [Cross Ref]
  • Anastasio T. J., Patton P. E. (2003). A two-stage unsupervised learning algorithm reproduces multisensory enhancement in a neural network model of the corticotectal system. J. Neurosci. 23, 6713–6727 [PubMed]
  • Ben-Yishai R., Bar O., Sompolinsky H. (1995). Theory of orientation tuning in visual cortex. Proc. Natl. Acad. Sci. U.S.A. 92, 3844–384810.1073/pnas.92.9.3844 [PubMed] [Cross Ref]
  • Binns K. E., Salt T. E. (1996). Importance of NMDA receptors for multimodal integration in the deep layers of the cat superior colliculus. J. Neurophysiol. 75, 920–930 [PubMed]
  • Burnett L. R., Stein B. E., Chaponis D., Wallace M. T. (2004). Superior colliculus lesions preferentially disrupt multisensory orientation. Neuroscience 124, 535–54710.1016/j.neuroscience.2003.12.026 [PubMed] [Cross Ref]
  • Edwards S. B., Ginsburgh C. L., Henkel C. K., Stein B. E. (1979). Sources of subcortical projections to the superior colliculus in the cat. J. Comp. Neurol. 184, 309–33010.1002/cne.901840207 [PubMed] [Cross Ref]
  • Ernst M. O., Banks M. S. (2002). Humans integrate visual and haptic information in a statistically optimal fashion. Nature 415, 429–43310.1038/415429a [PubMed] [Cross Ref]
  • Fuentes-Santamaria V., Alvarado J. C., McHaffie J. G., Stein B. E. (2009). Axon morphologies and convergence patterns of projections from different sensory-specific cortices of the anterior ectosylvian sulcus onto multisensory neurons in the cat superior colliculus. Cereb. Cortex 19, 2902–291510.1093/cercor/bhp060 [PMC free article] [PubMed] [Cross Ref]
  • Fuentes-Santamaria V., Alvarado J. C., Stein B. E., McHaffie J. G. (2008). Cortex contacts both output neurons and nitrergic interneurons in the superior colliculus: direct and indirect routes for multisensory integration. Cereb. Cortex 18, 1640–165210.1093/cercor/bhm192 [PMC free article] [PubMed] [Cross Ref]
  • Gingras G., Rowland B. A., Stein B. E. (2009). The differing impact of multisensory and unisensory integration on behavior. J. Neurosci. 29, 4897–490210.1523/JNEUROSCI.4120-08.2009 [PMC free article] [PubMed] [Cross Ref]
  • Grantyn R., Lux H. D. (1988). Similarity and mutual exclusion of NMDA- and proton-activated transient Na+ -currents in rat tectal neurons. Neurosci. Lett. 89, 198–20310.1016/0304-3940(88)90381-3 [PubMed] [Cross Ref]
  • Holmes N. P., Spence C. (2005). Multisensory integration: space, time and superadditivity. Curr. Biol. 15, R762–R76410.1016/j.cub.2005.08.058 [PMC free article] [PubMed] [Cross Ref]
  • Jiang W., Jiang H., Rowland B. A., Stein B. E. (2007). Multisensory orientation behavior is disrupted by neonatal cortical ablation. J. Neurophysiol. 97, 557–56210.1152/jn.00591.2006 [PubMed] [Cross Ref]
  • Jiang W., Jiang H., Stein B. E. (2002). Two corticotectal areas facilitate multisensory orientation behavior. J. Cogn. Neurosci. 14, 1240–125510.1162/089892902760807230 [PubMed] [Cross Ref]
  • Jiang W., Stein B. E. (2003). Cortex controls multisensory depression in superior colliculus. J. Neurophysiol. 90, 2123–213510.1152/jn.00369.2003 [PubMed] [Cross Ref]
  • Jiang W., Wallace M. T., Jiang H., Vaughan J. W., Stein B. E. (2001). Two cortical areas mediate multisensory integration in superior colliculus neurons. J. Neurophysiol. 85, 506–522 [PubMed]
  • Kadunce D. C., Vaughan J. W., Wallace M. T., Benedek G., Stein B. E. (1997). Mechanisms of within- and cross-modality suppression in the superior colliculus. J. Neurophysiol. 78, 2834–2847 [PubMed]
  • Kadunce D. C., Vaughan J. W., Wallace M. T., Stein B. E. (2001). The influence of visual and auditory receptive field organization on multisensory integration in the superior colliculus. Exp. Brain Res. 139, 303–31010.1007/s002210100772 [PubMed] [Cross Ref]
  • Knill D. C., Pouget A. (2004). The Bayesian brain: the role of uncertainty in neural coding and computation. Trends Neurosci. 27, 712–71910.1016/j.tins.2004.10.007 [PubMed] [Cross Ref]
  • Koch C. (1998). Biophysics of computation: Information Processing in Single Neurons. New York: Oxford University Press
  • Ma W. J., Pouget A. (2008). Linking neurons to behavior in multisensory perception: A computational review. Brain Res. 1242, 4–1210.1016/j.brainres.2008.04.082 [PubMed] [Cross Ref]
  • Magosso E., Cuppini C., Serino A., Di Pellegrino G., Ursino M. (2008). A theoretical study of multisensory integration in the superior colliculus by a neural network model. Neural Networks 21, 817–82910.1016/j.neunet.2008.06.003 [PubMed] [Cross Ref]
  • Martin J. G., Meredith M. A., Ahmad K. (2009). Modeling multisensory enhancement with self-organizing maps. Front. Comput. Neurosci. 3, 1–1010.3389/neuro.10.008.2009 [PMC free article] [PubMed] [Cross Ref]
  • Maruff P., Yucel M., Danckert J., Stuart G., Currie J. (1999). Facilitation and inhibition arising from the exogenous orienting of covert attention depends on the temporal properties of spatial cues and targets. Neuropsychologia 37, 731–74410.1016/S0028-3932(98)00067-0 [PubMed] [Cross Ref]
  • Meredith M. A., Nemitz J. W., Stein B. E. (1987). Determinants of multisensory integration in superior colliculus neurons. I. Temporal factors. J. Neurosci. 7, 3215–3229 [PubMed]
  • Meredith M. A., Stein B. E. (1986a). Visual, auditory, and somatosensory convergence on cells in superior colliculus results in multisensory integration. J. Neurophysiol. 56, 640–662 [PubMed]
  • Meredith M. A., Stein B. E. (1986b). Spatial factors determine the activity of multisensory neurons in cat superior colliculus. Brain Res. 365, 350–35410.1016/0006-8993(86)91648-3 [PubMed] [Cross Ref]
  • Meredith M. A., Stein B. E. (1996). Spatial determinants of multisensory integration in cat superior colliculus neurons. J. Neurophysiol. 75, 1843–1857 [PubMed]
  • Patton P. E., Anastasio T. J. (2003). Modelling cross-modal enhancement and modality-specific suppression in multisensory neurons. Neural. Comput. 15, 783–81010.1162/08997660360581903 [PubMed] [Cross Ref]
  • Perrault T. J., Jr, Vaughan J. W., Stein B. E., Wallace M. T. (2003). Neuron-specific response characteristics predict the magnitude of multisensory integration. J. Neurophysiol. 90, 4022–402610.1152/jn.00494.2003 [PubMed] [Cross Ref]
  • Perrault T. J., Jr, Vaughan J. W., Stein B. E., Wallace M. T. (2005). Superior colliculus neurons use distinct operational modes in the integration of multisensory stimuli. J. Neurophysiol. 93, 2575–258610.1152/jn.00926.2004 [PubMed] [Cross Ref]
  • Rolls E. T., Treves A. (1998). Neural Networks and Brain Functions. Oxford: Oxford University Press
  • Rowland B. A., Stanford T. R., Stein B. E. (2007a). A model of the neural mechanisms underlying multisensory integration in the superior colliculus. Perception 36, 1431–144310.1068/p5842 [PubMed] [Cross Ref]
  • Rowland B. A., Quessy S., Stanford T. R., Stein B. E. (2007b). Multisensory integration shortens physiological response latencies. J. Neurosci. 27, 5879–588410.1523/JNEUROSCI.4986-06.2007 [PubMed] [Cross Ref]
  • Stanford T. R., Quessy S., Stein B. E. (2005). Evaluating the operations underlying multisensory integration in the cat superior colliculus. J. Neurosci. 25, 6499–650810.1523/JNEUROSCI.5095-04.2005 [PMC free article] [PubMed] [Cross Ref]
  • Stein B. E. (2005). The development of a dialogue between cortex and midbrain to integrate multisensory information. Exp. Brain Res. 166, 305–31510.1007/s00221-005-2372-0 [PubMed] [Cross Ref]
  • Stein B. E., Meredith M. A. (1993). The Merging of the Senses. Cambridge, MA: MIT Press
  • Stein B. E., Meredith M. A., Huneycutt W. S., McDade L. (1989). Behavioral indices of multisensory integration: orientation to visual cues is affected by auditory stimuli. J. Cogn. Neurosci. 1, 12–2410.1162/jocn.1989.1.1.12 [PubMed] [Cross Ref]
  • Stein B. E., Meredith M. A., Wallace M. T. (1993). The visually responsive neuron and beyond: multisensory integration in cat and monkey. Prog. Brain Res. 95, 79–9010.1016/S0079-6123(08)60359-3 [PubMed] [Cross Ref]
  • Stein B. E., Stanford T. R. (2008). Multisensory integration: current issues from the perspective of the single neuron. Nat. Rev. Neurosci. 9, 255–26610.1038/nrn2331 [PubMed] [Cross Ref]
  • Stein B. E., Stanford T. R., Ramachandran R., Perrault T. J., Jr, Rowland B. A. (2009). Challenges in quantifying multisensory integration: alternative criteria, models, and inverse effectiveness. Exp. Brain Res. 10.1007/s00221-009-1880-8 [PMC free article] [PubMed] [Cross Ref]
  • Stein B. E., Wallace M. T. (1996). Comparisons of cross-modality integration in midbrain and cortex. Prog. Brain Res. 112, 289–29910.1016/S0079-6123(08)63336-1 [PubMed] [Cross Ref]
  • Ursino M., Cuppini C., Magosso E., Serino A., Di Pellegrino G. (2009). Multisensory integration in the superior colliculus: a neural network model. J. Comput. Neurosci. 26, 55–7310.1007/s10827-008-0096-4 [PubMed] [Cross Ref]
  • Wallace M. T., Meredith M. A., Stein B. E. (1998). Multisensory integration in the superior colliculus of the alert cat. J. Neurophysiol. 80, 1006–1010 [PubMed]
  • Wallace M. T., Stein B. E. (1994). Cross-modal synthesis in the midbrain depends on input from cortex. J. Neurophysiol. 71, 429–432 [PubMed]
  • Wallace M. T., Stein B. E. (1997). Development of multisensory neurons and multisensory integration in cat superior colliculus. J. Neurosci. 17, 2429–2444 [PubMed]
  • Wallace M. T., Stein B. E. (2000). Onset of cross-modal synthesis in the neonatal superior colliculus is gated by the development of cortical influences. J. Neurophysiol. 83, 3578–3582 [PubMed]
  • Wallace M. T., Stein B. E. (2001). Sensory and multisensory responses in the newborn monkey superior colliculus. J. Neurosci. 21, 8886–8894 [PubMed]
  • Wallace M. T., Wilkinson L. K., Stein B. E. (1996). Representation and integration of multiple sensory inputs in primate superior colliculus. J. Neurophysiol. 96, 1246–1266 [PubMed]
  • Wilkinson L. K., Meredith M. A., Stein B. E. (1996). The role of anterior ectosylvian cortex in cross-modality orientation and approach behavior. Exp. Brain Res. 112, 1–1010.1007/BF00227172 [PubMed] [Cross Ref]

Articles from Frontiers in Integrative Neuroscience are provided here courtesy of Frontiers Media SA