|Home | About | Journals | Submit | Contact Us | Français|
Although responses to auditory stimuli have been extensively examined in the well-known regions of auditory cortex, there are numerous reports of acoustic sensitivity in cortical areas that are dominated by other sensory modalities. Whether in ‘polysensory’ cortex or in visual or somatosensory regions, auditory responses in non-auditory cortex have been described largely in terms of auditory processing. This review takes a different perspective that auditory responses in non-auditory cortex, either through multisensory subthreshold or bimodal processing, provide subtle but consistent expansion of the range of activity of the dominant modality within a given area. Thus, the features of these acoustic responses may have more to do with the subtle adjustment of response gain within a given non-auditory region than the encoding of their tonal properties.
At first blush, the notion of auditory responses in non-auditory, sensory cortices seems counterintuitive. In areas dominated by vision or touch, it would seem unlikely that the brain would direct resources toward extracting information from anything but those specific modalities, and that little insight into the organization and function of audition could be gained by searching for auditory responses in such regions. Nevertheless, over the last 40–50 years, a number of studies have appeared that have reported the presence of auditory responses in normal, adult non-auditory cortex.
Historically, investigations of auditory responses outside acknowledged regions of auditory cortex took place in association areas regarded at that time as ‘polysensory.’ Many of these initial forays took place in the cat, and regions of the anterior suprasylvian gyrus, middle suprasylvian gyrus and anterior ectosylvian gyrus were identified as showing evoked or single-unit responses to somatosensory, visual and/or auditory stimulation (e.g., see Albe-Fessard and Fessard, 1963; Thompson et al., 1963; Schneider and Davis, 1974; Robertson et al., 1975). Similar approaches were used to identify polysensory areas in non-human primates, such as the superior temporal sulcus and prefrontal areas (e.g., see Nelson and Bignall, 1973; Benevento et al., 1977; Hikosaka et al., 1988) or, in rodents, the parietotemporal cortices (e.g., see Di et al., 1994; Brett-Green et al., 2003). In general, these studies used conventional techniques to characterize auditory responsiveness, such as tuning curves, octave-band range, binaural properties, etc. While some papers reported coarser levels of response tuning than those observed in units from primary auditory cortex (Dow and Dubner, 1969; 1971; Irvine and Huebner, 1979), others reported substantially similar properties (Toldi et al., 1984). Either way, auditory responses in polysensory cortices showed features that were closely linked to the onset and quality of auditory stimulation. In other words, auditory responses in polysensory cortex were clearly sensory in nature.
Given that both hearing and sight deal with features of extrapersonal space, it is not surprising that auditory responses in lower-level visual cortical areas were also reported (Murata et al., 1965; Bental et al., 1968; Spinelli et al., 1968; Morrell, 1972; Fishman and Michael, 1973). Most of these studies did not document the sensory features (e.g., latency, sensitivity to changes in stimulus quality) of the responses they observed, and it has been suggested that the activity observed in these paralyzed, unanesthetized preparations actually represented attempts to execute motor programs toward acoustic stimuli (Allman et al., 2008a). The fact that such acoustic responses in visual cortex would appear to violate Hebbian rules for synapse formation and maintenance should have elicited a massive investigational interest. Instead, this issue received relatively little attention until recently. Sensitive anatomical studies revealed sparse auditory projections to visual cortices (Falchier et al., 2002; Rockland and Ojima, 2003; Budinger et al., 2006; Hall and Lomber, 2008). Despite the expectation that the result of these crossmodal projections would generate overtly auditory responses in their targeted visual areas, visual cortical neurons in awake, behaving monkeys failed to show overt, suprathreshold responses to acoustic qualities of auditory stimuli. Instead, auditory inputs were found to subtly modify the responses of the dominant modality: response latencies to visual-auditory saccade targets were faster than to visual-only targets (Wang et al., 2008).
A repeated theme in many of the historical of studies of auditory responses in non-auditory cortices has been the interpretation of that activity in a framework related to auditory processing. This occurred despite the obvious fact that such auditory responses were embedded in non-auditory regions that are likely to subserve non-auditory perceptual and behavioral functions. More recently, the investigation of these phenomena has turned from merely identifying the presence of auditory responses toward that of interpreting the presence of auditory influences in the context of the structure in which they were observed. That this conceptual bridge has already been crossed is evidenced by a variety of recent reports, including that by Wang et al. (2008) described above. Although the report was titled perhaps to appease more historical treatments, the findings fall squarely in the context of the auditory modality influencing the ongoing activity of the dominant non-auditory modality, and these effects are most likely to influence perceptual and behavioral involvements of the non-auditory host region.
As recognized in the early studies, auditory responsivity in non-auditory regions was indicative of ‘polysensory’ activity. This term has more recently been replaced by ‘multisensory,’ which is generally accepted to be a neuron whose activity is influenced by more than one sensory modality (e.g., see Stein and Meredith, 1993). The term ‘multisensory’ has come to be regarded as synonymous with ‘bimodal (or trimodal), given that these forms of multisensory neurons have been studied now for nearly 50 years (Horn and Hill, 1966). However, a handful of recent studies have documented neurons that are influenced by more than one sensory modality, yet are not bimodal (or trimodal). Neurons that are excited by stimuli from one modality, but have that activity modulated by an otherwise ineffective stimulus from another modality have been identified and termed subthreshold multisensory neurons (Allman and Meredith, 2007). This form of multisensory neuron has been identified in a variety of regions, including somatosensory areas SIV (Dehner et al., 2004) and rostral suprasylvian sulcal cortex (Clemo et al., 2007), auditory Field of the Anterior Ectosylvian sulcus (FAES; Meredith et al., 2006;Carriere et al., 2006; Meredith and Allman, 2009), and Posterolateral Lateral Suprasylvian (PLLS, Allman and Meredith, 2007; Allman et al., 2008b) and ferret Area 21 (Allman et al., 2008a) visual areas. The present investigation sought to evaluate the relative contribution of acoustically sensitive multisensory neurons (bimodal as well as subthreshold multisensory) in non-auditory areas to the dominant non-auditory activity of that area.
The studies reviewed here have been described in detail (Dehner et al., 2004; Meredith et al., 2006; Clemo et al., 2007; Allman and Meredith, 2007; Allman et al., 2008a and 2008b; Meredith and Allman, 2009) and were performed in compliance with the Guide for Care and Use of Laboratory Animals (NIH publication 86-23) and the National Research Council’s Guidelines for Care and Use of Mammals in Neuroscience and Behavioral Research (2003), approved by the Institutional Animal Care and Use Committee at Virginia Commonwealth University. The procedural elements of these studies are very similar to one another and, as such, permit a general comparison of their findings. Specifically, all animals were surgically prepared at least a week prior to data collection. As a consequence, extracellular single-unit recordings took place under conditions of light anesthesia and paralysis. Furthermore, in the most recent reports, data collection occurred at predetermined intervals along a penetration to reduce sampling bias. Perhaps most importantly, all neurons were presented virtually the same, electronically-gated, adequate stimuli presented separately (e.g., unimodal) and in combination (multisensory). Thus, up to a few hundred neurons within a volume of tissue were examined for their responses to the same adequate stimulus set that was presented separately as well as in combination. This is in contrast to most multisensory studies to date that selectively evaluate only those neurons that are well isolated and demonstrate suprathreshold responses to stimuli from more than one modality (e.g., bimodal, trimodal).
Sandwiched between the auditory field of the anterior ectosylvian sulcus (FAES) dorsally and auditory area AII laterally, the AEV resides in the ventral bank of the anterior ectosylvian sulcus (Mucke et al., 1982; Olson and Graybiel, 1983). This visual area lacks an overall retinotopy, exhibits large receptive fields that generally include the area centralis, and responds vigorously to moving visual stimuli (Olson and Graybiel, 1983). When examined using sinusoidal gratings, the AEV responds well to pattern motion, much like the MT area of the primate cortex (Scannell et al., 1996). Studies in which recording electrodes vertically approached the auditory FAES often crossed the border between the FAES and AEV, where bimodal (visual-auditory) neurons have been identified (Meredith, 2004; Carriere et al., 2006; Meredith and Allman, 2009). Bimodal neurons make up approximately 19% (n=13/70) of AEV neurons sampled, an example of which is provided in Figure 1. Like other bimodal neurons, this visual-auditory neuron responded with suprathreshold excitation when presented a variety of visual or auditory stimuli presented alone. When presented repeated, electronically-generated stimuli that were adequate to elicit independent visual and auditory responses (shown in rasters and histograms of Fig. 1B), this neuron also showed a modest response increase when those same stimuli were combined. For the population of bimodal neurons, the scatter plot (Fig. 1B-bottom left) shows that most had a modest response increase when the visual and auditory stimuli were combined. These response changes (shown in the bar graph Fig. 1B-bottom right) were relatively small and none exceeded a 50% change in activity relative to the best unimodal response.
In the AEV, search stimuli also identified neurons that appeared to be responsive only to visual stimulation, yet combinations of stimuli (adequate to achieve activation in both modalities in bimodal neurons) revealed significant subthreshold influences by the apparently ineffective auditory cues. In these neurons, under no circumstances were responses observed to any form of auditory stimulation presented alone. An example is provided in Figure 1C, where an AEV neuron that was excited by a visual stimulus but not by an auditory stimulus showed a significantly facilitated response when the stimuli were combined. Of the population of subthreshold multisensory AEV neurons, 6 showed facilitation and 2 showed suppression. These effects, represented small but consistent response changes when the stimuli were combined (see scatter plot in Fig. 1C-bottom left) and, as evidenced in the bar graph (Fig. 1C-bottom right), no interactions exceeded a 50% response change.
Located between somatosensory areas SV and SII anteriorly, the anterior lateral lateral suprasylvian (ALLS) visual area posteriorly, and the anterior auditory field (AAF) laterally (see Fig. 2A), it would seem that the lateral bank of the rostral suprasylvian sulcus (also called Rostral Lateral Suprasylvian (RLS) by some authors) would contain a wealth of multisensory neurons representing each of the different modality combinations. However, the modest number of bimodal neurons located in this region were largely (21%; n=200/938) auditory-somatosensory (Clemo et al., 2007). As depicted in Figure 2B, bimodal forms of multisensory convergence occurred that not only exhibited suprathreshold activation by auditory and somatosensory cues delivered independently, but also showed increased response levels when the stimuli were combined. Although the basic trend for bimodal neurons to exhibit a response increase to combined stimulation was evident in the population (Fig. 2B-scatterplot), few (approximately 29%) showed significant levels of enhancement and only 3 examples exceeded a response change of 100% (Fig. 2B-bar graph).
Of the non-bimodal neurons found in the rostral suprasylvian region (n=88), a few (10/88) were significantly influenced by stimulus combinations that included an otherwise ineffective auditory cue. These neurons were never observed to respond overtly to any form of auditory stimulation presented alone. As shown in Figure 2C, neurons that were clearly activated by stimulation in one sensory modality, but not by another, showed a significant response modulation in the presence of the second cue. These subthreshold multisensory neurons generally revealed modest response changes (when compared with that elicited by the effective stimulus alone), as depicted in the scatter plots and bar graphs (Fig. 2C-bottom). It is also interesting to note that absolute response levels for bimodal neurons were substantially higher than those for subthreshold multisensory neurons (compare plots in Figures 2B and 2C), suggesting these different neurons may play different functional roles in this region.
This visual area, identified by Palmer et al., (1978), resides on the lateral bank of the suprasylvian sulcus. It is bordered, on the medial bank of the same sulcus by the posteromedial lateral suprasylvian (PMLS) visual area and, lateral to it, by the dorsal zone (DZ) of auditory cortex (See Figure 3A). This retinotopically organized area primarily represents the upper portion of contralateral visual space. Bimodal neurons (9%; n=49/520), such as that depicted in Figure 3B, have been identified largely at the transition zone between PLLS and DZ and are, accordingly, associated with visual receptive fields in the periphery (Allman and Meredith, 2007). A total of 19 (39%; 19/49) bimodal neurons showed significant response enhancement. When responses of bimodal neurons were graphed as an x-y scatter-plot of visual response (uniformly the best single-modality response in this sample) versus the combined response, the vast majority of bimodal neurons (86%; 42/49) showed increased activity levels to the combined stimuli and, consequently, plotted above the line of unity (see Fig. 3B-bottom).
Bimodal neurons were largely absent in the portion of the PLLS representing the central 40 degrees of visual space (Allman and Meredith, 2007). However, the influence of auditory stimulation remained manifest in many of these neurons in the form of subthreshold effects. As illustrated in Figure 3C, neurons responsive to visual inputs (but not to auditory) had those visual responses significantly facilitated by the presence of an auditory cue. This subthreshold effect occurred in 16% of the population (n=37/233). When the visual and combined responses of the subthreshold multisensory neurons were graphed as an x-y scatter plot of visual versus combined-modality responses, all plotted above the line of unity (Fig. 3C-bottom left) but the magnitude of response change remained comparatively small (Fig. 3C-bottom right).
As illustrated in Figure 4A, the ferret auditory cortices are separated from visual Area 21 by the Suprasylvian visual area on the bank of the suprasylvian sulcus (initially designated as Area Ssy but recently renamed as PMLS by Manger et al., 2008). Ferret Area 21 is retinotopically organized and has been suggested to represent the homologue of the primate V4 (Manger et al., 2002). A recent investigation that made large tracer injections into ferret auditory cortex identified an anatomical projection that terminated in the lower visual field representation of Area 21 (Allman et al., 2008a). However, single-unit recordings (n=416) failed to identify any neurons that were activated by acoustic stimulation. To assess whether the crossmodal projection may have a subthreshold influence on visual processing, a total of 296 neurons were tested with single- and combined-modality visual and auditory stimuli. A typical example is provided in Figure 4B, where a neuron that responded vigorously to a visual stimulus showed no increase or decrease in activity when presented an auditory cue, and showed no response change when the visual and auditory stimuli were combined. The population of Area 21 neurons behaved in a similar fashion. For each neuron, the response to the visual stimulus alone was plotted against its response to visual-auditory combined (see scatter plot Fig. 4B), where nearly equal proportions fell slightly above (48%; n=142/296) or slightly below (52%; n=154/296) the line of unity. Furthermore, the average of the responses to visual stimulation alone was virtually the same as the average response to the combined stimuli (see Fig. 4B-bar graph). Collectively, these results indicate that crossmodal auditory effects were not largely evident at the neuronal or population level in visual Area 21.
Given that the presence of a crossmodal projection lacked a measurable effect, the possibility that auditory influences in Area 21 might be masked by inhibitory processes was examined in an additional set of neurons. The same sensory testing and recording techniques were used as before, but now in combination with microiontophoretic application of the GABA-A antagonist, bicuculline methiodide (BIC). The responses of an Area 21 neuron to single- and combined-modality stimuli before application of BIC (control) and following its ejection (post-BIC) are illustrated in Figure 4C. In the control condition, this neuron was activated by a visual stimulus, but failed to respond to an auditory stimulus presented alone. However, in the presence of BIC, this same neuron showed increases in spontaneous activity as well as responsiveness during both visual alone and combined visual-auditory stimulus conditions. Furthermore, and despite the continued lack of response to the auditory cue alone, the response to the combined visual-auditory stimulation was now significantly greater than the visual stimulation alone. Within this sample of neurons tested with BIC, a larger proportion showed significant levels of subthreshold facilitation in the post-BIC condition (n=4/23) than before BIC administration (n=0/23), and 87% (20/23) now exhibited responses that fell above the line of unity (Fig. 4C-scatter plot). In contrast, in the control condition, only 52% (n=12/23) of the responses plotted above the line of unity. Thus, the blockage of local inhibition resulted in a significant change in multisensory activity. As shown in Figure 4C (bar graph), response discharge rates to visual and to visual-auditory stimulation showed no significant difference but, in the BIC condition, responses to combined-visual auditory stimulation significantly exceeded those evoked by visual cues alone. Collectively, these comparisons indicate that inhibitory mechanisms can strongly regulate the expression of subthreshold auditory inputs into visual Area 21.
This brief survey of selected somatosensory and visual cortices reveals that auditory information has access, in some form, to each of them, as summarized in Table 1. This observation provides several important insights into the organization and function of sensory cortices, such as the hierarchical levels that receive crossmodal inputs as well as the nature and potential influence of those inputs.
As has been known for decades, higher-level association non-auditory association (e.g., ‘polysensory’) areas exhibit neurons that show suprathreshold excitatory responses to auditory cues. It should be pointed out that when similar areas were examined in the present study, auditory responses were uniformly accompanied, in the same neuron(s), by suprathreshold responses to other modalities: i.e., they were bimodal. Bimodal neurons provide a substrate for signaling not in one but two separate modalities as well as bear the potential for integrating auditory and non-auditory information (to be discussed further, below). In addition, each non-auditory region provided evidence for neurons that received subthreshold levels of auditory inputs. This class of multisensory neuron did not show overt responses to any auditory stimuli when presented alone, but revealed the influence of auditory inputs on the non-auditory activity of the neuron. Subthreshold multisensory processing, first described in 2002 (Meredith, 2002), has also been observed in primate prefrontal and superior temporal sulcal cortex (Barraclough et al., 2005; Sugihara et al., 2006) as well as rattlesnake optic tectum (Newman and Hartline, 1979) and cat anterior ectosylvian sulcal cortex (Dehner et al., 2004; Meredith et al., 2006; Carriere et al., 2006; Meredith and Allman, 2009).
Many of the cortices previously identified as ‘polysensory’ have subsequently demonstrated dominant, modality-specific properties. Specifically, the AEV has strong visual properties and is thought to represent the homologue of the primate MT area (Scannell et al., 1996). The PLLS, likewise, has strong visual properties and is demonstrated to be involved in visual motion processing (Rauschecker et al., 1987; Rauschecker, 1988). Thus, the notion that these specific regions are ‘polysensory’ has been replaced by the designation of ‘higher-level’ processing area for the dominant sensory modality. Both bimodal and subthreshold multisensory neuron types were identified in these higher-level non-auditory cortices. On the other hand, subthrehsold multisensory, but not bimodal neurons have been observed in the lower-level visual Area 21. Thus, each of these areas received some form of crossmodal auditory inputs, but bimodal neurons were observed only in the higher-level areas whereas subthreshold multisensory neurons were identified at each level (see also Barraclough et al., 2005; Sugihara et al., 2006). This difference in distribution may account for the long-held notion that multisensory neurons occur in higher-level association areas (Jones and Powell, 1970), while multisensory effects are not readily observable in lower-level modality-specific regions. In reality, it is evident that crossmodal effects do reach lower-level cortices (see also Schroeder et al., 2001; Schroeder and Foxe, 2002; Bizley et al., 2007; Ghazanfar and Schroeder, 2006), but exposing their presence requires different and more sensitive methods [such as inhibitory unmasking (Allman et al., 2008a); or detection of oscillatory patterns (Lakatos et al, (2005)] than that for higher-level regions.
Another issue is the intrinsic organization of multisensory properties within a cortical region. Multisensory bimodal neurons have the highest likelihood of occurrence the closer they are to a border with a representation of another sensory modality (Meredith, 2004; Wallace et al., 2004;2006). This makes connectional sense because these locations have the highest likelihood of their dendrites receiving afferent contacts targeting the adjoining area, and numerous observations support this possibility (e.g., Di et al., 1994; Khan and Krubitzer, 2002; Brett-Green et al., 2003; Meredith, 2004; Hunt et al., 2006; Allman and Meredith, 2007; Meredith and Allman, 2009). On the other hand, distributional principles for subthreshold multisensory neuron types are less clear at this time. In the visual PLLS, subthreshold multisensory neurons predominate in the representation of central regions of visual space (Allman and Meredith, 2007), raising the possibility that the distribution of this neuron type may be related to the functional role of the region. However, in somatosensory SIV, subthreshold multisensory neurons are distributed throughout the representation (Dehner et al., 2004), as also seems evident in RSS (Clemo et al., 2007), the FAES (Meredith and Allman, 2009) and the AEV (present study). Thus, it is obvious that the principles of multisensory organization within a given region require further investigation.
Each of the recent studies from our laboratory employed a method that was different from that traditionally used for examination of multisensory processing at the neuronal level. To date, most multisensory investigations studied only those neurons that responded overtly to more than one sensory modality, and subsequently used many different stimulus qualities to assess the highest possible integrative levels. In contrast, the present investigations examined the responses of all neurons (sampled at a predetermined interval) to the separate and combined application of a single set of adequate stimuli. In this way, all neurons could be examined for their responses to separately presented stimuli, and those that responded to more than one were defined as bimodal. In addition, some neurons that appeared sensitive to only one stimulus modality presented alone showed a significant response change when the stimuli were combined; these units were defined as subthreshold multisensory neurons. Although it might be suggested that such neurons might represent bimodal types whose effective stimuli were not presented, this possibility is unlikely because a wide array of search stimuli that were effective in activating other bimodal neurons were always presented before standardized testing commenced. In another distinction from previous multisensory studies, the present results were obtained not by changing the quality of the stimuli presented to elicit maximal multisensory effects, but used only a single set of adequate stimuli instead. Thus, the present experimental design more closely imitates that used in EEG and fMRI studies where volumes of neurons are tested for their responses to the same set of stimuli and minimal, near-threshold levels are difficult to use due to signal-noise issues. Therefore, it is hopeful that the results observed here at the neuronal level, using a basic set of adequate stimuli, might provide insight into the evaluation of multisensory responses obtained by other methods.
Collectively, the present results indicate that combinations of adequate stimuli from different sensory modalities predominantly evoke low levels of multisensory response integration. For the well-known bimodal neurons, responses to combined-modality stimulation rarely rose above 100% increase in discharge (over the most effective single-modality stimulus; see Meredith and Stein, 1986). Subthreshold multisensory neurons consistently produced even lower levels of response change. These observations lend support to the proposition that subthreshold multisensory neurons provide an intermediary, smoothing function to the collective integrative activity of a region involving bimodal (highest integrative levels) as well as unimodal neurons (no integration)(Meredith and Allman, 2009). Despite reports of dramatic levels of multisensory integration well in excess of 100%, especially in the superior colliculus (Meredith and Stein, 1986; Stein and Meredith, 1993), response changes of 40% or less seem to be the norm for the cortical neurons examined so far. In fact, in the only definitive study of the range of integrative effects (Perrault et al., 2005), 85% of superior colliculus bimodal neurons did not exhibit superadditive levels of multisensory integration. A ceiling effect in the cortical neurons sampled might account for the low integrative levels observed in the present studies, but it is clear from the scatter plots of the responses that many of the neurons retained the potential to fire at even higher discharge rates than elicited by the stimuli employed. Ultimately, it seems fair to suggest, given the role of cortex in perceiving subtleties in stimulus qualities, that these peripheral events might be encoded by subtle, not massive, response changes in the neurons activated by them. For example, rather than the integration of visible lip movement and speech that results in profound changes in neuronal activity, the net response change in cortical neurons may be relatively low compared to brainstem circuits that underlie survival phenomena like orientation, detection or escape.
Because modality-specific receptive field properties are preserved in integrated multisensory responses of superior colliuclus neurons (Stein et al., 1993), it seems possible that bimodal multisensory neurons could encode tonal features of acoustic stimuli in non-auditory cortical regions. These modality-specific auditory response properties, however, were not examined in the current experiments. Alternatively, auditory responses in non-auditory cortices may simply influence the response gain of the predominant sensory modality in a given region, as summarized in Table 1. When the subthreshold form of multisensory neuron is considered, this possibility becomes even more likely. For subthreshold multisensory neurons, it is difficult to imagine how frequency-specific auditory information might be preserved when integrated responses of these neurons to combined stimuli are so heavily dominated by the non-auditory modality. In this case it may be instructive to imagine how auditory inputs might compete with and survive among the surrounding non-auditory signals during activity-dependent periods of development. Under these conditions, features of auditory inputs that would promote their survival and maintenance might be signals related to properties that are co-active in the non-auditory activity. Some aspects of sensory stimulation that share common features among several sensory modalities (known as ‘amodal’ properties) are location, intensity, duration, number, etc. Thus, if auditory afferents to a non-auditory area signal amodal features of stimuli, their inputs would be co-active with the dominant non-auditory responses of the region and the auditory synapses would be reinforced and maintained. In this manner, auditory inputs can eventually contribute to non-auditory processing in a manner that might be more meaningful to non-auditory sensory modalities than, for example, the encoding of a tone in the visual or tactile pathways.
Supported by NIH grant NS039460.
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.