Summary of fMRI findings
shows maps of the different category-selective regions identified using fMRI for two monkey subjects: the left hemisphere for monkey S (based on 9,916 functional volumes, 74 blocks/condition), and the right hemisphere for monkey W (based on 12,580 functional volumes, 74 blocks/condition). As described previously (Bell et al., 2009
), we identified regions selective for each of four categories tested: faces, body-parts, objects, and places. These category-selective regions were concentrated in the inferior bank of the STS, but extended to the ventral surface of IT cortex. In the left hemisphere of monkey S (, left), we identified two face-selective regions: one was located anteriorly in area TE (centered at +17–18 mm anterior to the interaural axis) and the other was located posteriorly in/near area TEO (centered at +5–6 mm). These regions correspond to the “anterior” and “middle” face-selective regions previously identified by Tsao and colleagues (Tsao et al., 2003
; Tsao et al., 2006
) and Pinsk and colleagues (Pinsk et al., 2005
; Pinsk et al., 2009
); and to the “anterior” and “posterior” face-selective regions of others (Hadj-Bouziane et al., 2008
; Bell et al., 2009
; Rajimehr et al., 2009
). Also consistent with previous studies (Pinsk et al., 2005
; Tsao et al., 2003
; Bell et al., 2009
), two regions selective for body-parts were located immediately adjacent to the face-selective regions, centered at +17 and +6 mm anterior to the interaural axis, respectively. Occupying the majority of the cortex between the two face/body-part selective regions was a single, large object-selective region, spanning regions between+6 to +14 mm. A single place-selective region (not shown) was located along the ventral surface of IT cortex lateral to the occipitotemporal sulcus (OTS), centered at approximately 5 mm anterior to the interaural axis.
The data obtained from the second monkey (monkey W, , right) were more variable, likely due to the increased movement observed in this subject. However, we were nonetheless able to identity several statistically significant category-selective regions within its right hemisphere. We identified both an anterior and a posterior face-selective region within the inferior bank of the STS, centered at +14–17 and +7–9 mm anterior to the interaural axis, respectively. In addition, several smaller face-selective regions were identified, including one on the ventral surface of IT cortex (centered at +11 mm anterior to the interaural axis, lateral to the OTS) and another within the STS close to the temporal pole (centered at +22–23 mm anterior to the interaural axis). It is probable that some of these smaller regions represent subregions of the anterior and posterior face-selective regions that became isolated due to statistical thresholding. Two body-part selective regions were identified on the lateral edge of the STS; one was located anteriorly (centered at +12–14 mm anterior to the interaural axis) and the other was located posteriorly (centered at +5–6 mm anterior to the interaural axis). Surrounding the anterior face and body-part selective regions was an object-selective region, spanning +10–16 mm anterior to the interaural axis. Two anterior place-selective regions were identified, one located within the STS (centered at +20–22 mm anterior to the interaural axis) and the other located immediately ventral to this, near the anterior middle temporal sulcus.
To localize our electrode penetrations relative to the fMRI data, we collected several anatomical scans with electrodes positioned at strategic depths (e.g., , see METHODS for details). The approximate area accessible by our recording grid is indicated on the flattened fMRI category maps in as dashed ovals. shows a top-down view of the recording grid for each monkey. The colors indicate the fMRI-identified category-selectivity in the inferior bank of the STS, where all recordings were performed. The white circles indicate the grid holes from which we sampled neuronal data (26 for monkey S; 23 for monkey W). Given the shape and location of our recording grids, we were able to access the following fMRI-identified regions: the complete anterior face-selective region (in both monkeys), the anterior portion of the posterior face-selective region (in both monkeys), the anterior body-part selective region (in both monkeys), a large portion of the object-selective region (in both monkeys), and an anterior place-selective region (in monkey W).
Properties of category-selective neurons in IT cortex
We recorded activity from 1,272 individual neurons in the inferior bank of the STS (areas TE/TEO; von Bonin and Bailey, 1947
) from two monkeys (609 from monkey S; 663 from monkey W). Of these, 77% (975/1272) showed a significant response to stimuli from at least one of the four visual categories tested; only these 975 visually responsive neurons were considered for further analysis. Three types of response profile were identified (; ). Most neurons significantly increased their firing rate in response to the visual stimuli tested (“Excitatory Response”; e.g., ; 529/975, 54%). The second group decreased their firing rate in response to stimuli from at least one category tested (“Suppressed Response”; e.g., ; 250/975, 26%). The remaining 20% of neurons (196/975) increased their firing in response to stimuli from one (or more) category, and decreased their firing in response to stimuli from at least one other category (“Both Excitatory/Suppressed”; ).
Response distribution in IT cortex
shows the neuronal populations sampled from the two monkeys, separated according to those neurons that exhibited significant excitatory (left) and suppressed (right) responses. Neurons exhibiting both excitatory and suppressed responses, such as those shown in , appear in both panels. Individual rows show the responses to each of the 80 different stimuli presented (20 exemplars per category) for a single neuron. Responses have been normalized to the baseline firing rate of each neuron in order to reveal excitatory vs. suppressed responses. Individual neurons are sorted according to which category evoked the strongest response (or, in the case of suppressed responses, the weakest response). Below, all visually responsive neurons are referred to by their preferred category (i.e., that which evoked the strongest average response across all exemplars within that category): those that responded most strongly to faces are referred to as “face neurons”, neurons that responded most strongly to body-parts are referred to as “body-part neurons”, and so forth. Note that this does not imply that a given neuron is ultimately “selective” for that particular category, merely that of the four categories tested, this category evoked the strongest response.
Excitatory and suppressed responses to visual stimuli
To evaluate if a given neuron exhibited significant across-category selectivity, we performed a one-way ANOVA on each neuron, with stimulus category as the factor of interest (i.e., faces, objects, etc.). By this criterion, 73% (713/975) of visually responsive neurons were category-selective (main effect of category, p<0.05; ). illustrates one such neuron, showing a clear bias for face stimuli. By comparison, the neuron in was not category-selective; it showed approximately the same level of response suppression to stimuli from all four categories. A second ANOVA, using stimulus identity as the main factor of interest (i.e., face1, face2, etc.), revealed that only 23% of neurons exhibited stimulus-selectivity
within their preferred category (main effect of stimulus identity, p<0.05; ). In other words, if a given neuron responded robustly to a face stimulus (for example), it was very likely to respond robustly to all
face stimuli. Object and body-part neurons showed the greatest proportion of neurons with significant stimulus-selectivity (; 45% and 41%, respectively vs. 17% and 23% for faces and places, respectively), which may be due to the greater variation in visual appearance across the different exemplars within these two categories compared to faces and places (Bell et al., 2009
Summary of stimulus-selectivity among IT neurons
We also observed significant differences in the degree to which individual neurons were selective for their preferred category. show the average normalized responses for stimulus category for each subpopulation of neuron (i.e., face neurons, body-part neurons, etc.). In the case of the excitatory responses, face neurons had relatively weak responses (or none at all) to stimuli from the remaining three categories (, see for example). A similar trend was observed for place neurons (). By contrast, body-part and object neurons showed relatively robust responses to certain non-preferred categories (in particular, objects and body-parts, respectively).
Properties of neurons in IT cortex
To quantify these differences in category selectivity across the four subpopulations of neurons, we calculated a category selectivity index (CSI) on the raw responses for each neuron, which expresses the ratio of the average excitatory response to all stimuli from a neuron's preferred category to those from the remaining three categories (see METHODS). shows the average CSI values for all neurons that responded preferentially to each of the four categories (i.e., face neurons, body-part neurons, etc.). Face neurons were the most selective (for their preferred category of faces), with an average CSI of 0.29±0.01 (indicating that the average response to face stimuli was approximately 82% greater than the average response to the remaining three categories). Body-part neurons were the next selective (for body-parts), with an average CSI of 0.24±0.01 (~63%). Place neurons had an average CSI of 0.22±0.02 (~56%), and object neurons were the least selective, with an average CSI of 0.19±0. 01 (~47%). In other words, a neuron that responded most strongly to faces was not likely to respond strongly to stimuli from any other category, whereas a neuron that responded most strongly to objects was likely to show robust responses to stimuli from other categories.
Similar disparities were observed for the suppressed responses (). Neurons whose activity was most suppressed by faces or places showed little suppression to stimuli from the remaining categories (average CSI: −0.36±0.01, ~−53% and −0.30±0.02, ~−46%, respectively). Thus these suppressive effects were strongly category-specific for these two categories; in fact, the responses to the other categories were often above baseline. On the other hand, neurons most suppressed by body-parts or objects tended to also show decreases in activity in response to stimuli from other categories (objects and body-parts, respectively) (average CSI: −0.28±0.02, ~−44% and −0.24±0.02, ~−39% respectively).
Finally, neurons also exhibited differences in response latency related to their category preferences (). Face neurons had significantly shorter response latencies (average response latency: 110±1 ms) compared to neurons that preferred each of the other three categories (123±1 ms for body-part neurons; 124±2 ms for object neurons; 125±2 ms for place neurons; P's<0.05).
These data highlight several differences among category-selective neurons in IT cortex of monkeys, independent of their location relative to the fMRI regions. Specifically, face neurons were: 1) more selective, and 2) had shorter response latencies, compared to the other three neuron types. These observations, together with the disproportionately large number of neurons selectively suppressed by faces, suggest that faces (and face neurons) represent a special category of visual stimuli/neuron (see DISCUSSION).
Spatial distribution of category selectivity in IT cortex
We next compared the distribution of all visually responsive neurons, relative to the location of the individual fMRI-identified category-selective regions. shows the fMRI-identified category maps for monkeys S and W. We subdivided the maps into 4 and 5 subdivisions for monkeys S and W, respectively, corresponding to the individual fMRI-identified category-selective regions located in the inferior bank of the STS accessible from our recording chambers. Below these are the corresponding distributions of all visually responsive neurons for each subdivision, separated into excitatory and suppressed responses. We chose to include all visually responsive neurons in this analysis (as opposed to just neurons that exhibited a certain level of selectivity for a particular category) based on the assumption that the MR signal would be correlated with the overall distribution of actively firing neurons, and not with the distribution of a select subgroup. We conducted individual χ2 tests on each distribution to assess whether neurons that preferred each of the four categories were evenly distributed within each subdivision. These tests revealed that the majority of subdivisions showed a significantly biased distribution (p<0.05). In the case of the excitatory responses, this bias matched the category-selectivity identified by fMRI. For example, in the case of monkey S, the fMRI-identified body-part selective region (subdivision #1, ) contained 56% body-part neurons. Immediately adjacent to this region was the anterior fMRI-identified face-selective region (subdivision #2, ), which contained 52% face neurons. No such pattern was observed for the suppressed responses: in almost all cases, neurons suppressed maximally by faces comprised the largest proportion of suppressed responses, regardless of the selectivity predicted by the fMRI data. Thus, unlike the excitatory responses, the distribution of category selectivity for the suppressed responses showed little correspondence to the fMRI-identified regions. From these data, we cannot infer that a relationship existed between the fMRI signal and the processes associated with suppressed spiking responses in this experimental context. Therefore the remaining analyses were restricted to the excitatory responses.
Distribution of category-selectivity across IT cortex
compares the proportion of neurons that preferred a given category found within (IN), near (NEAR) (located between 1–4 mm from the edge), and outside (OUT) (greater than 4 mm from the edge) the corresponding fMRI-identified category-selective region. These three zones are represented in the accompanying grid maps in (IN: colored according to the selective category; NEAR: gray boundaries; OUT: all remaining sampled locations. Note that sites defined as OUT for the anterior face region do not include those found in the posterior face region, and vice-versa). In all but one case (the object-selective region for monkey W), the greatest proportion of neurons that preferred a given category was found within recording sites that targeted the corresponding fMRI region. Further, in the case of the face-selective regions, a greater proportion of face neurons were found in the anterior face-selective regions as compared to the posterior face-selective regions. In the majority of cases (all but the face-selective regions in both monkeys and the object-selective region in monkey W), the next greatest proportion was found nearby (NEAR: gray bars), and the lowest proportion was found in recording sites located farthest from the fMRI region. This relationship between fMRI and neuronal distribution was most pronounced for face and body-part regions (in which the proportion of face- and body-part neurons ranged from 41–69%), and weaker for object and place regions (which contained 20–35% object and place neurons, respectively). As illustrated in , these biased distributions within the individual fMRI-selective region were significantly different from chance (χ2, p<0.05) in all cases except the object-selective and place-selective regions in monkey W.
Comparing neuronal distributions within vs. outside fMRI-identified category-selective regions
Overall, these data show that category-selective voxels identified with fMRI correspond to a local increase in the proportion of neurons that prefer that category, and that this concentration decreases further from the borders of these regions. This relationship was specific to neurons that increase their firing in response to the relevant stimuli (i.e., excitatory responses) and was more pronounced for faces and body-parts and weaker (or absent) for places and objects.
Contrasting fMRI and neuronal responses
shows the fMRI time-series and the corresponding spike-density functions for two different fMRI-identified category-selective regions. In the first example (anterior face-selective region from monkey W), the fMRI activation in response to faces was almost twice that to the next most active category (objects). The corresponding neuronal distribution was strongly biased towards faces (69%) and the population response was highly selective for faces. By contrast, in the second example (object selective region from monkey S), the fMRI activation was only weakly selective for objects, as was the underlying neuronal distribution. Furthermore, there was very little bias in the population response – all four categories evoked robust responses among the population of neurons found within this region.
To quantify this relationship between the selectivity of the fMRI response profiles within individual category-selective regions and those of the underlying neuronal populations, we correlated the selectivity indices for each category response within each fMRI-identified region for the fMRI and neuronal populations (). This analysis showed that as the strength of the neuronal response to faces (as an example) increased relative to the responses to non-faces, so too did the strength of the corresponding fMRI activation. Thus in this experiment, we might infer on the basis of this analysis that the fMRI signal predicts the “preferred-to-non-preferred” ratio of the responses of the underlying neuronal population. Note however, while this analysis revealed modest correlation values, it was only marginally significant (p=0.04) and failed to reach statistical significance when evaluated with a non-parametric analysis method (Spearman's Rank Correlation Coefficient).
Nonetheless, based on these examples, it is tempting to conclude that fMRI activations might correlate with the response magnitudes of the neuronal populations. However, caution must be taken when contrasting fMRI activation with spiking activity. For example, while the majority of neurons within a given region might respond most strongly to a particular category, this does not necessarily imply that the remaining neurons respond weakly to stimuli from another category (e.g., consider 100 face neurons each firing 10 sp/s to faces, compared with 10 object neurons each firing 100 sp/s to objects). Furthermore, peak-firing rate is only one method of quantifying a neuronal response. Because the haemodynamic response operates on a much longer time scale than neuronal activity, it is possible that weak but sustained responses might have a greater impact on the fMRI signal. Given these caveats, it was not surprising that the magnitude of the fMRI response to each category within a given region did not correlate significantly with the corresponding population neuronal response ().
Comparing neuronal properties inside vs. outside fMRI-identified category-selective regions
In addition to a correspondence between fMRI selectivity and the spatial distribution of neurons, we also investigated whether neurons found within the fMRI-identified regions are functionally different from those located outside these regions. Specifically, we compared the average category-selectivity (CSI) for the excitatory responses of neurons found inside (IN) vs. outside (OUT) the corresponding fMRI-identified category-selective regions (). Interestingly, both monkeys showed the identical trend: greater selectivity was found among face- and body-part neurons within the category-selective regions, as compared to those found outside. This trend was statistically significant in monkey W (p<0.05), but failed to achieve statistical significance in monkey S (p>0.05). Thus, in addition to indicating a concentration of category-selective neurons, fMRI-identified category-selective voxels may also reflect an increase in the selectivity of those neurons found within those voxels.
Properties of neurons found within vs. outside fMRI-identified category-selective regions