Inputs from AES have already reached the multisensory SC at birth, even before its constituent neurons become multisensory (McHaffie et al., 1988
). Presumably, the inability of neonatal SC multisensory neurons to integrate their cross-modal inputs is because the AES-SC synaptic coupling is not properly functional (just as those from superficial layers are not). This is only a supposition, for at this point we know little about how this projection changes over time. Some of the AES inputs to the SC certainly become functional at about one month of age, for as soon after individual SC neurons exhibit multisensory integration, this capability can be blocked by deactivating AES (see and (Wallace and Stein, 2000
)). These relationships strengthen over the next few
The developmental appearance of multisensory integration coincides with the development of AES-SC influences
This is also a period during which the brain is exposed to a variety of sensory stimuli, some of which are linked to the same event and some of which are not. Cross-modal cues that are derived from the same event often occur in spatiotemporal concordance, while unrelated events are far less tightly linked in space and time. Presumably, after sufficient experience, the brain has learned the statistics of those sensory events, which, via Hebbian learning rules, have been incorporated into the neural architecture underlying the capacity to integrate different sensory inputs. Such experience provides the foundation for a principled way of perceiving and interacting with the world so that only some stimulus configurations will be integrated and yield response enhancement or response depression.
Put another way, the experience leads the animal to expect that certain cross-modal physical properties covary (e.g. the timing and/or spatial location of visual and auditory stimuli) and this “knowledge” is used to craft the principles for discriminating between those stimuli derived from the same event and those derived from different events.
The first test of this hypothesis was aimed at determining whether experience is essential for the maturation of this process, visual-nonvisual experiences were precluded by rearing animals in darkness from birth to well after the maturation of multisensory integration is normally achieved (i.e., 6 months or more). Interestingly, this rearing condition did not prevent the development of visually-responsive neurons. In fact, in additional to unisensory neurons, each of the cross-modal convergence and response patterns characteristic of normal animals was evident in neurons within the SC of dark-reared animals, though their incidence was slightly different (Wallace et al., 2001
;Wallace et al., 2004
). This parallels the observations in monkey, which is born later in development than the cat but already has visual-nonvisual SC neurons. Visual experience is obviously not essential for the appearance of such neurons.
The receptive fields of these neurons in dark-reared cats were very large, more like neonatal SC neurons than those in the adult. Like neonatal neurons, they could not integrate their cross-modal inputs and their responses to cross-modal pairs of visual-nonvisual stimuli were no more vigorous than were their responses to the best of the modality-specific component stimuli (). As postulated, experience with visual-nonvisual stimuli proved to be necessary to develop the capacity to engage in multisensory integration. This is also consistent with observations in human subjects who had congenital cataracts removed during early life. Their vision seemed reasonably normal, but they were compromised in their ability to integrate visual and nonvisual cues, despite having years of experience after surgical correction (Putzar et al., 2007
Comparison between normal and dark-reared animals
The next test of this hypothesis was to rear animals in conditions in which the spatiotemporal relationships of cross-modal stimuli were altered from “normal” experience, in which they are presumably in spatiotemporal concordance when derived from the same event. If cross-modal experience determines the governing principles of multisensory integration, then changing it should change the principles. This possibility was examined after rearing animals in special dark environments in which their only experience with simultaneous visual and auditory stimuli were when they were spatially displaced (Wallace and Stein, 2007
). They were raised to 6 mos or more in this condition and then the multisensory integration characteristics of SC neurons were examined.
Similar to simply dark-reared animals, these animals possessed the full range of multisensory convergence patterns and there were many visual-nonvisual neurons. However, the properties of visually-responsive neurons were atypical: their receptive fields were very large and many were unable to integrate visual-nonvisual cues. There was, however, a sizable minority of visual-auditory neurons that were fully capable of multisensory integration, but the stimulus configurations eliciting response enhancement or no integration were significantly different from those of normally-reared animals (). Their receptive fields, unlike those of many of their neighbors, had contracted partially, but were in poor spatial alignment with one another. Some were totally out of register, a feature that is exceedingly rare in normal animals, but one that clearly reflects the early experience of these animals with visual-auditory cues. Most important in the current context, is that those neurons integrated spatially disparate stimuli to produce response enhancement–not spatially concordant stimuli. This is because their receptive fields were misaligned and only spatially disparate stimuli could fall simultaneously within them.
Rearing animals in environments with spatially disparate visual-auditory stimulus configurations yields abnormal multisensory integration
Taken together, the dark rearing and disparity rearing conditions demonstrate that not only is experience critical for the maturation of multisensory integration, but that the nature of the experience directs formation of the neural circuits that engage in this process. In both normal and disparity-reared animals, the basis for multisensory response enhancement is defined by early experience. Whether this reflects a simple adaptation to specific cross-modal stimulus configurations, or the general statistics of multisensory experience, is a subject of ongoing experimentation.
Parallel experiments in AES cortex revealed that multisensory integration develops more slowly in cortex than in the SC. These multisensory neurons in AES populate the border regions between its visual (AEV), auditory (FAES), and somatosensory (SIV) subregions. This is perhaps not surprising, as in general, the development of the cortex is thought to be more protracted than that of the midbrain. These multisensory cortical neurons are involved in a circuit independent of the SC, as they do not project into the cortico-SC pathway (Wallace et al., 1992
). Despite this, they have properties very similar to those found in the SC. They too fail to show multisensory integration capabilities during early neonatal stages, and develop this capacity gradually, and after SC neurons (Wallace et al., 2006
). Just as is the case for SC neurons, these AES neurons also require sensory experience and fail to develop multisensory integration capabilities when animals are raised in the dark (Carriere et al., 2007
Although the above observations suggest that the development of multisensory integration in the SC and cortex is dependent on exposure to cross-modal stimuli and its principles adapt to their configurations, they provide no insight as to the underlying circuitry governing its development and adaptation. However, for multisensory SC neurons, the cortical deactivation studies described above coupled with the maturational time course of the AES-SC projection suggests that AES cortex is likely to play a critical role.
Evaluating this idea began with experiments in which chronic deactivation of association cortex (both AES and its adjacent area rLS) was induced on one side of the brain for 8 weeks (between 4–12 wks postnatal) during the period in which multisensory integration normally develops (Wallace and Stein, 1997
;Stein and Stanford, 2008
), thereby rendering them unresponsive to sensory (in particular, cross-modal) experience. The deactivation was induced with muscimol, a GABAa agonist. It was embedded in a polymer that was placed over association cortex from which it was slowly released over this period. After the polymer released its stores of muscimol or was physically removed, the cortex became active and responsive to environmental stimuli. Animals were then tested behaviorally and physiologically when adults (1 yr of age), long after cortex had reactivated. These experiments are still ongoing, but preliminary results are quite clear.
Their ability of these animals to detect and locate visual stimuli was indistinguishable from that of normal animals, and was equally good in both hemifields. Furthermore, behavioral performance indicated that they significantly benefitted from the presentation of spatiotemporally concordant but task-irrelevant auditory stimuli in the ipsilateral hemifield (as do normal animals). However, in the contralateral hemifield, they were abnormal: responses to spatiotemporally concordant visual-auditory stimuli were no better than when the visual stimulus was presented alone. Apparently, deactivating ipsilateral association cortex during early life disrupted the maturation of multisensory integration capabilities in the contralateral hemifield. SC neurons in these animals also appeared incapable of synthesizing spatiotemporally concordant cross-modal stimuli to produce multisensory response enhancement. These data strongly support the hypothesis that the AES-SC projection is principally engaged in the instantiation of multisensory integration in the SC. The fact that the deficits in multisensory integration were observed long after the deprivation period, regardless of whether they were induced by dark rearing or chronic cortical deactivation, suggests that there is a ‘critical’ or ‘sensitive’ period for acquiring this capacity. Such a period would demarcate the period in which the capacity could be acquired.