PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of cercorLink to Publisher's site
 
Cereb Cortex. 2010 August; 20(8): 1780–1787.
Published online 2009 December 16. doi:  10.1093/cercor/bhp244
PMCID: PMC2901016

Connectivity Analysis Reveals a Cortical Network for Eye Gaze Perception

Abstract

Haxby et al. (Haxby JV, Hoffman EA, Gobbini MI. 2000. The distributed human neural system for face perception. Trends Cogn Sci. 4:223–233.) proposed that eye gaze processing results from an interaction between a “core” face-specific system involved in visual analysis and an “extended” system involved in spatial attention, more generally. However, the full gaze perception network has remained poorly specified. In the context of a functional magnetic resonance imaging study, we used psychophysiological interactions (PPIs) to identify brain regions that showed differential connectivity (correlation) with core face perception structures (posterior superior temporal sulcus [pSTS] and fusiform gyrus [FG]) when viewing gaze shifts relative to control eye movements (opening/closing the eyes). The PPIs identified altered connectivity between the pSTS and MT/V5, intraparietal sulcus, frontal eye fields, superior temporal gyrus (STG), supramarginal gyrus, and middle frontal gyrus (MFG). The FG showed altered connectivity with the same areas of the STG and MFG, demonstrating the contribution of both dorsal and ventral core face areas to gaze perception. We propose that this network provides an interactive system that alerts us to seen changes in other agents’ gaze direction, makes us aware of their altered focus of spatial attention, and prepares a corresponding shift in our own attention.

Keywords: attention, effective connectivity, face perception, fMRI, social attention

Introduction

In their model of face processing, Haxby et al. (2000) highlighted the contribution of 3 occipitotemporal regions to visual analysis of faces—the inferior occipital gyrus (IOG), the lateral fusiform gyrus (FG; or fusiform face area; Kanwisher et al. 1997), and the posterior superior temporal sulcus (pSTS). Together, they form a “core system” involved in the visual analysis of different facial properties. Haxby et al. (2000) proposed that IOG underlies the early structural encoding of faces and that its output projects to both the pSTS and FG. Perception of “changeable” or dynamic facial characteristics, such as gaze, facial expression, and lipspeech was attributed to the pSTS, whereas the recognition of “invariant” facial features that change slowly across time, such as facial identity, was assigned to the FG.

Central to this model is the idea that processing particular facial properties (i.e., gaze, identity, expression, etc.) is achieved by the combined efforts of the core system, involved in visual analysis, and an “extended system” underlying multiple aspects of cognition. For example, perception of seen gaze and subsequent orienting of attention toward the location indicated by the gaze (Friesen and Kingstone 1998; Driver et al. 1999; Langton and Bruce 1999) involves the pSTS and brain areas implicated in attention. However, the extended network for gaze processing has remained poorly specified. There is evidence that it includes the intraparietal sulcus (IPS; Puce et al. 1998; Hoffman and Haxby 2000); however, other areas of the attention system are rarely discussed. Recent meta-analyses (Grosbras et al. 2005; Nummenmaa and Calder 2009) suggest more widespread involvement of the brain's attention circuits in gaze perception, but it remains unknown if these regions comprise a cortical network, and how they are functionally connected.

Regarding the attention network, Corbetta et al. (2008) have proposed that the IPS operates together with the frontal eye fields (FEFs) as a dorsal frontoparietal system involved in both goal-directed and stimulus-driven (i.e., involuntary) orienting of attention, with a particular role in attentional target selection. In addition, they propose that a separate ventral attention system, including the inferior supramarginal gyrus (SMG), posterior superior temporal gyrus/sulcus (STG/pSTS), lateral prefrontal cortex, frontal operculum, and anterior insula, acts as a “circuit breaker,” interrupting ongoing processing so that attention can be reoriented to behaviorally salient events (Corbetta et al. 2008). They also suggest that the middle frontal gyrus (MFG) acts as point of convergence, facilitating communication between the 2 systems. Shifts in another agent's gaze direction are both behaviorally salient and cause involuntary orienting of attention toward the gazed-at location (Friesen and Kingstone 1998; Driver et al. 1999; Langton and Bruce 1999). Hence, although research to date has concentrated on the IPS (i.e., dorsal attention system) in gaze perception, it is possible that components of the ventral attention system also contribute to the extended gaze network.

Given that the STG is part of the attention system, it is relevant that studies of patients with focal brain lesions have emphasized the contribution of this region to both spatial attention and gaze perception. With regard to spatial attention, Karnath et al. (2001) showed that, although hemispatial neglect is generally attributed to parietal damage, the right STG constituted the area of maximal lesion overlap in a large group of patients with neglect when those with visual field defects were excluded. This is in accord with previous animal research showing that damage to the superior temporal regions, but not the inferior parietal regions, gives rise to the sorts of behavioral deficits seen in human neglect patients (Luh et al. 1991; Watson et al. 2001).

In the case of gaze perception, Akiyama, Kato, Muramatsu, Saito, Nakachi, and Kashima (2006) reported a case study of a patient with selective damage to the right STG who showed impaired discrimination of gaze direction in the form of rightward bias, such that left gaze was perceived as direct and direct gaze as right; the same patient also showed impaired attentional orienting from gaze but not arrow cues (Akiyama, Kato, Muramatsu, Saito, Umeda, and Kashima 2006). Similarly, impaired gaze discrimination has been reported in 3 patients with damage to the left STG and left inferior parietal lobule (Boddaert et al. 2004) Hence, it is possible that the STG might also form part of the extended gaze network.

To characterize the network for gaze perception, we used functional magnetic resonance imaging (fMRI) together with a form of connectivity analyses known as psychophysiological interactions (PPIs) to identify brain regions that interacted with components of the core face network. Connectivity analysis is particularly suited to this issue because it inherently addresses the interaction or communication between brain regions, rather than isolated regional effects. Specifically, we identified areas that showed a change in connectivity with components of the “core” face-specific system, the pSTS, FG, or IOG, when participants viewed faces displaying shifts in gaze direction relative to opening and closing the eyes without a change in gaze direction.

PPIs measure the variation in physiological connectivity between 2 brain regions as a function of psychological context (Friston et al. 1997). An advantage of PPIs over other methods to assess effective connectivity (e.g., dynamic causal modeling or structural equation modeling) is that it does not require prior specification of an anatomical model but can identify likely regions of the extended network because of the condition-specific changes in connectivity. In our current study, the PPI analysis assessed how the connectivity with each of 3 “source” regions (pSTS, FG, and IOG) was “changed” as a function of viewing horizontal gaze shifts relative to opening/closing the eyes. In this way, regions were identified not because their activity is correlated with one of the source regions, or the presence/absence of gaze shifts, but rather the interaction between these 2 variables. Based on the role of pSTS in gaze perception and the tendency for humans to orient their attention toward the direction signaled by others’ gaze (Friesen and Kingstone 1998; Driver et al. 1999; Langton and Bruce 1999), we predicted significant PPIs between the pSTS and areas of the dorsal and ventral attention systems discussed above. Investigation of PPIs with the FG and IOG enabled us to address the extent to which other areas of the core face network are involved in gaze perception. In particular, Haxby et al. (2000) proposed that the FG is primarily involved in the perception of invariant facial features, and only marginally involved in perception of changeable facial properties, such as gaze and expression. Contrary to this, recent work has shown FG involvement in facial expression perception, but it remains to be determined whether it is involved in processing other changeable properties, such as gaze.

Materials and Methods

Subjects

Nineteen right-handed healthy volunteers (4 males; aged: 18–30 years; mean age = 24 years) with normal or corrected to normal vision participated in the study. Individuals with a history of neurological or psychiatric disease or currently taking medication affecting the central nervous system were excluded. All provided written informed consent as part of a protocol approved by The Suffolk Research Ethics Committee.

Experimental Design

Stimuli and design are summarized in Figure 1. Full-face computer images of 5 males and 5 females identities were generated with the DAZ Studio software (DAZ Productions, Draper, UT). Subjects viewed alternating 21-s epochs containing 6 gaze shifts or 6 open/close eye events; each was intermixed with 6 null events. A single trial comprised a 1000-ms presentation of an eye movement followed by a low contrast central cross (750 ms). Gaze shift events comprised 2 consecutive 500-ms frames showing leftward gaze followed by rightward gaze or vice versa; this produced a strong illusion of a dynamic gaze shift. Similarly, events consisting of opening or closing the eyes comprised consecutive 500-ms presentations of open eyes followed by closed eyes faces or vice versa; again producing an illusion of movement. Null events comprised a 1750-ms presentation of a low-contrast cross. The gender, identity, and direction of eye movement (i.e., left-to-right or right-to-left for gaze shifts condition and open-to-closed or closed-to-open for open/closed condition) were fully randomized. Half of the participants were instructed to make a button-press response whenever they saw a male face and half when they saw a female face. The total task duration was 15 min. Participants practiced the task outside the scanner prior to starting the fMRI experiment.

Figure 1.
Experimental design and sample stimuli. The experiment consisted of alternating 21-s blocks of horizontal gaze shifts (A) and control eye movements (opening/closing the eyes; B). The individual trials consisted of 1000-ms presentation of a gaze shift ...

Twenty epochs of each stimulus condition were presented; a total of 240 face trials (120 gaze shifts and 120 open/close eyes). The order of the stimuli during each epoch was pseudorandomized with respect to trial type (face or null), such that no more than 3 consecutive trials were of the same type. This pseudo-randomization enhanced design efficiency while preserving the unpredictability of stimulus onsets in naïve participants.

A separate functional localizer scan comprising blocks of face, house, and fixation events was used to localize the 3 core face areas (IOG, FG, and pSTS). Face and house stimuli were presented in alternating 16-s blocks separated by 16-s rest periods. An event in each block contained an 800-ms presentation of a face or house followed by a 200-ms blank interstimulus interval. Participants performed a 1-back task and pressed the button whenever the same image was presented consecutively (4% of trials). A total of 4 blocks of each stimulus category was presented. All stimuli were presented via an angled mirror above the participant's eyes, which reflected images back projected onto a translucent screen in the bore of the magnet behind the participant's head.

fMRI Acquisition and Analysis

MR imaging was performed with a 3-T Tim Trio magnetic resonance imaging scanner (Siemens, Germany) with a head coil gradient set at the MRC Cognition and Brain Sciences Unit, Cambridge. Whole-brain data were acquired with T2*-weighted echo-planar imaging (EPI), sensitive to blood oxygen level–dependent (BOLD) signal contrast (40 axial slices, 3-mm slice thickness; time repetition = 2424 ms; time echo = 30 ms; field of view = 192 mm; voxel size: 3 × 3 × 3 mm). The first 3 volumes were discarded to allow for equilibration effects. T1-weighted structural images were acquired at a resolution of 1 × 1 × 1 mm.

Data were preprocessed and analyzed using SPM5 software (www.fil.ion.ucl.ac.uk/spm/). The EPI images were sinc interpolated in time to correct for slice-time differences and realigned to the first scan by rigid-body transformations to correct for head movements. EPI and structural images were coregistered and normalized to theT1 standard template in Montreal Neurological Institute (MNI) space (MNI—International Consortium for Brain mapping) using linear and nonlinear transformations and smoothed with a Gaussian kernel of full width at half maximum 8 mm.

Analysis of Regional Effects

A random effects model was implemented using a 2-stage process (first and second level). This random-effects analysis assessed effects on the basis of intersubject variance and thus allowed inferences about the population that the participants were drawn from. For each participant, we used a general linear method (GLM) to assess regional effects of task parameters on BOLD indices of activation. The model included 2 experimental conditions (gaze shifts and open/close eyes) and effects of no interest (realignment parameters) to account for motion-related variance. Low-frequency signal drift was removed using a high-pass filter (cutoff 128 s) and AR(1) modeling of temporal autocorrelations was applied. The individual contrast images were generated using the contrast gaze shifts versus open/close eyes. These are contrasts images (of the voxel-wise difference in beta estimates for gaze shifts vs. open/close eyes) but not statistical images. The second-level analysis used these contrast images in a new GLM from which generated statistical images, that is, SPM t-maps. With balanced designs at first level (i.e., similar events for each subject, in similar numbers), this second-level analysis closely approximates a true mixed-effects design, with both within- and between-subject variance.

PPI in the GLM

The physiological connectivity between 2 brain regions can vary with the psychological context (Friston et al. 1997) known as a PPI. PPIs can be identified by GLMs sensitive to contextual modulation of task-related covariance. In contrast with dynamic casual modeling or structural equation modeling of network connectivity, GLMs do not require a specified anatomical model. Rather, one starts with a source region and identifies any other “target” voxels/clusters with which that source has context-dependent connectivity. Target regions need not correlate with the task or context alone but the interactions between these factors. Significant PPIs do not in themselves indicate the direction or neurochemistry of causal influences between source and target regions, nor whether the connectivity is mediated by mono- or poly-synaptic connections, nor changes in structural neuroplasticity from block to block. However, they do indicate interactions between regional systems and the results of PPIs accord with other connectivity methods such as dynamic causal modeling (Passamonti et al. 2008).

The right IOG, FG, and pSTS were used as the source regions for the analyses. Subject-wise local maxima in these regions were identified from the faces versus houses contrast from the face localizer. Next, spherical regions of interests (ROIs) with an 8-mm radius were generated around the individual local maxima for each source region. In other words, the center of each source region was the voxel with the highest statistical significance in the respective cluster, such that the position of the ROI was slightly different across individuals. A group-based analysis showed that the MNI average coordinates for the ROIs across participants were as follows: IOG: (44, −72, −6), FG (44, −46, −16), and pSTS (44, −56, 16).

The time series for each participant was computed by using the first eigenvariate from all voxel time series in the ROI. This BOLD time series was deconvolved to estimate a “neuronal time series” for this region using the PPI-deconvolution parameter defaults in SPM5 (Gitelman et al. 2003). The PPI term (PPI regressor) was calculated as the element-by-element product of the ROI neuronal time series and a vector coding for the main effect of task (i.e., 1 for gaze shifts and −1 for open/closed eye movements). This product was reconvolved by the canonical hemodynamic response function (HRF). The model also included the main effects of task convolved by the HRF, the neuronal time series for each source, and the movement regressors as effects of no interest.

Subject-wise PPI models (Friston et al. 1997) were run, and contrast images were generated for positive and negative PPIs. The identified regions have greater or lesser change in connectivity with the source region according to context (i.e., gaze shifts vs. open/close eyes). The contrast images were then entered into second-level GLM analyses for contrasts of interest, and SPM t-maps generated using Gaussian random field theory to make statistical inferences. Two approaches to statistically threshold maps were applied. First, for small volume corrections (SVCs) within a priori ROI proposed in the model by Haxby et al. (2000), p. 231 (IPS and FEF) as well as the STG implicated in spatial attention (Karnath et al. 2001) and gaze perception (Akiyama, Kato, Muramatsu, Saito, Nakachi, and Kashima 2006), the threshold was set at P < 0.05 family-wise error corrected (Worsley et al. 1996). For the IPS, we defined an 8-mm sphere using as center the local maxima from a previous study assessing the role of IPS across various attentional tasks (32, −47, 56; Wojciulik and Kanwisher 1999). For FEF, we used the same sphere size and took coordinates (35, −4, 47) from a meta-analysis on the location of the human FEF (Paus 1996). The coordinates reported above are in MNI space and were converted from Talairach space with the tal2icbm_spm transform (Lancaster et al. 2007). The bilateral STG ROIs were defined using the WFU pick atlas (Maldjian et al. 2003) and AAL (Tzourio-Mazoyer et al. 2002) atlas. To explore other possible regions, which were not predicted, a threshold of P < 0.001, uncorrected (unc.), with a minimum of 10 contiguous voxels was used.

Results

Behavioral Results

Mean accuracy for the gender detection task was 96% (standard deviation [SD] = 5.2) with a mean reaction times (RTs) of 568 ms (SD = 48 ms). There were no significant differences in RTs (P > 0.32) or accuracy (P > 0.20) for the gaze shifts and open/close eyes condition.

Face Localizer

The FG, pSTS, and IOG were used as source regions for the PPI analyses. They were identified by contrasting activation to faces versus houses from the face localizer. Consistent with the right-hemisphere bias for face perception (Rhodes 1985; Luh et al. 1991), all 3 regions could be identified in all participants in the right hemisphere; the same areas were found in the left hemisphere in just 9 of the 19 participants at P < 0.05, unc. Consequently, the PPIs examined connectivity arising from the right hemisphere only.

PPIs as a Function of Gaze

As predicted, the right pSTS showed a positive change in connectivity for viewing gaze shifts versus opening/closing the eyes with parietal, frontal, and temporal regions involved in attention and programming eye movements (Paus 1996; Corbetta and Shulman 2002; Grosbras et al. 2005; Corbetta et al. 2008); see Table 1 and Figure 2. In other words, the difference in the respective correlations between the activity in the source (pSTS) and following target regions for viewing gaze shifts and open/close eye stimuli is positive—right IPS (26, −44, 60, T = 3.57, P = 0.05, SVC), right FEF (32, −10, 48, T = 3.31, P = 0.05, SVC), bilateral STG (left: −64, −16, 10, T = 5.08, P < 0.005, SVC; right: 56, −30, 18, T = 4.12, P < 0.005, SVC) and adjacent SMG (left: −58, −28, 38, T = 4.95; right: 68, −28, 32, T = 4.99, P's < 0.001, unc.), and right MFG (48, 48, 8, T = 4.00, P < 0.001, unc.). Additionally, the pSTS showed a positive change in connectivity with the motion-sensitive area MT/V5 (left: −60, −60, 0, T = 4.18, right: 54, −64, 2, T = 4.30, P's < 0.001, unc.). Other regions that survived our a priori threshold are summarized in Table 1.

Table 1
Brain areas showing positive change in coupling with the right pSTS and FG while viewing gaze shifts versus opening/closing the eyes (P < 0.001, unc.)
Figure 2.
Brain regions showing positive change in coupling with the right pSTS (red to yellow) and FG (blue to turquoise) while viewing gaze shifts versus opening/closing the eyes. Areas that showed a change in coupling with both FG and pSTS are shown in green. ...

The PPI using the right FG as the source region also showed a positive change in connectivity with right STG (48, −28, 10, T = 4.53, P = 0.007, SVC) and the right MFG (36, 52, 26, T = 5.14, P < 0.001, unc.). Figure 2 shows that these areas overlapped with the same areas identified using the pSTS as the source region. The IOG did not show gaze-dependent changes in connectivity, even at reduced threshold (P < 0.01, unc.). No brain region showed a “negative” change in connectivity with any source region as a function of gaze shifts versus opening/closing the eyes (P < 0.01, unc.). In other words, for no brain region was the coupling for the gaze shifts condition less positive (or more negative) than the coupling for the eyes open/closed condition.

Main Effect of Gaze Shifts Versus Opening/Closing the Eyes

As the pSTS has been repeatedly associated with gaze perception (e.g., Engell and Haxby 2007; e.g., Puce et al. 1998; Hoffman and Haxby 2000; Pelphrey et al. 2004), we verified its involvement in viewing gaze shifts versus opening/closing the eyes. We defined a 6-mm ROI around the group-level pSTS maximum (44, −56, 16) from the faces versus houses localizer and found significant activation in the corresponding area (44, −52, 18, T = 2.94, P < 0.05, SVC) in the gaze shift versus opening/closing the eyes contrast as well. Other regions showing a differential response at the a priori threshold are summarized in Table 2 and include bilateral SMG (extending to STG on left), right inferior frontal gyrus, and left lingual gyrus.

Table 2
Brain regions showing greater response to gaze shifts versus opening/closing the eyes (P < 0.001, unc.)

Discussion

Our study provides the first application of connectivity analysis to understanding the brain network underlying gaze perception. When viewing gaze shifts versus open/close eye movements, pSTS showed significant changes in connectivity with components of both ventral (SMG and STG) and dorsal (IPS and FEF) attention networks that are thought to play respective roles in attentional capture by behaviorally salient events and orienting of attention more generally (Corbetta et al. 2008). The pSTS also showed altered connectivity with MFG that forms a point of convergence between both dorsal and ventral attention systems (Fox et al. 2006; Corbetta et al. 2008). It is striking that the FG showed a change in connectivity with the same areas of STG and MFG, indicating that perception of gaze shifts affects connectivity with both ventral (FG) and dorsal (pSTS) components of the core face network. Figure 3 provides a schematic summary of these results.

Figure 3.
The extended cortical network for eye gaze perception. FEF, FG, IOG, MFG, IPS, SMG, STG, pSTS.

The interpretation of a significant PPI is that there is differential engagement of anatomical connections as function of psychological context; in this case viewing gaze shifts or opening/closing the eyes. In Figure 3, we do not specify the directionality of causal influences in this schema, as this cannot be inferred from the PPI method alone. However, it is likely that the PPIs we observed reflect changes in the engagement of direct anatomical connections between the seed and target regions (effective connectivity, Friston et al. 1997) because such direct anatomical connections between the pSTS and STG and SMG are supported by tracing studies in other primates (Seltzer and Pandya 1978; Rozzi et al. 2006). Similarly, altered connectivity between the pSTS and dorsal attention system accords with anatomical tracing data showing connections between pSTS and both FEF (Barbas and Mesulam 1981) and IPS (Maioli et al. 1998) in monkeys. Finally, altered effective connectivity between the motion-sensitive area MT and pSTS fits with the idea that the former conveys dynamic facial information, such as gaze shifts or expressions, to the latter (O'Toole et al. 2002). The role of MT/V5 in gaze processing is also supported by a magnetoencephalographic study that showed an MT/V5 response to gaze cues within 160-ms poststimulus onset (Watanabe et al. 2006).

The FG

Although Haxby et al. (2000) emphasized the role of the pSTS in processing changeable facial cues (e.g., gaze and expression) and the FG in facial identity, a recent meta-analysis found that FG is also engaged during gaze processing and that this cannot simply be attributed to a response to facial stimuli alone (Nummenmaa and Calder 2009). It is therefore of note that the current study found that the FG showed altered connectivity with the same areas of STG and MFG identified using the pSTS as the source region (Fig. 2). This highlight points of convergence between the ventral (FG) and superior temporal (pSTS) face areas’ contribution to gaze perception and demonstrates that the cortical network for gaze perception is more distributed than previously assumed.

Previous research has shown increased FG engagement for perception or monitoring of direct gaze (Kawashima et al. 1999; George et al. 2001; Hooker et al. 2003). Hence, it is possible that the observed positive change in connectivity between the FG and each of the STG and MFG arose for a different reason to that observed between the pSTS and these same regions. For example, the FG may have shown negative correlations with each of the STG and MFG for the gaze shift and open/close eyes conditions but with a more negative correlation for the open/close eye movements (which contained a brief exposure of direct gaze). By contrast, the pSTS may have shown a greater “positive” correlation with activity in the STG and MFG to gaze shifts relative to open/close eye movements. Both patterns would have resulted in a positive change (i.e., difference) in connectivity for gaze shifts versus open/close eyes as measured by PPI but are clearly different. To address this, we categorized data points to one or other condition (either gaze shift or open/close) using a 6-s shift in the transitions between conditions to reflect the delay to peak BOLD response (after Stephan et al. 2003) and computed subject-wise regression coefficients (betas) for the responses between the FG and pSTS seeds and the respective STG and MFG target regions. Note that the formal modeling of the PPI in GLMs incorporates the full temporal profile of the evoked HRF, accounting for the temporal delay. As shown in Figure 4, both regions showed the same pattern—a greater positive correlation between the source regions (pSTS or FG) and each of the STG and MFG for gaze shifts relative to open/close eye movements. Thus, both the pSTS and FG show similar patterns of altered connectivity in response to viewing gaze shifts with these regions.

Figure 4.
Means and standard errors of PPI coefficients between responses in the pSTS and FG seed regions and the respective STG and MFG target regions for gaze shift and open/close eye movement trials.

With respect to the involvement of the FG in gaze processing, it is of interest that our study is not alone in finding that the FG contributes to the perception of changeable facial characteristics. Indeed a number of recent studies have identified a significant role for the FG in coding facial expressions (Ganel et al. 2005; Fairhall and Ishai 2007; Tsuchiya et al. 2008; Fox et al. 2009); for a review, see Calder and Young (2005). This suggests that the ventral face perception system may contribute to processing changeable facial features and that the posited functional dissociation between the roles of the dorsal and ventral core face perception areas may be less clear-cut than often assumed. However, it remains possible that the FG and pSTS process different aspects of gaze shifts; for example, their visual form in the case of the FG and their motion in the case of the pSTS. Consistent with this proposal, the pSTS but not the FG showed increased connectivity with area MT.

The STG and Spatial Awareness

As highlighted in the introduction, the STG has been implicated in both attention and gaze perception. Karnath et al. 2001 showed that damage to the STG gives rise to a left visuospatial neglect and suggested that in conjunction with subcortical structures (putamen, caudate, and pulvinar) the STG gives rise spatial awareness. In the current context, its connectivity with pSTS and FG may therefore reflect increased awareness of others’ attentional focus from their gaze. Consistent with this proposal, a patient with damage to the right STG showed impaired discrimination of gaze direction in the form of a tendency for gaze to be perceived as more “rightward” than its actual physical direction (Akiyama, Kato, Muramatsu, Saito, Nakachi, and Kashima 2006). The contribution of the STG to gaze processing is further underlined by impaired gaze perception following damage to the left STG and inferior parietal cortex, including SMG, in 3 patients (Boddaert et al. 2004). It is also of note that whereas Akiyama, Kato, Muramatsu, Saito, Nakachi, and Kashima (2006) patient showed a rightward bias in gaze perception and had recovered from left hemispatial neglect, neurologically intact volunteers show a “leftward” bias in gaze perception (i.e., a tendency to mistake right gaze for direct and direct gaze for left; Calder et al. 2008). This accords with the leftward spatial bias on line bisection and similar spatial tasks, or “pseudo neglect,” observed in healthy participants (McCourt et al. 2001; Chokron 2002). Thus, converging evidence suggests that brain mechanisms underlying spatial awareness affect gaze perception.

Although the supramarginal and STG form part of the ventral attention system, it is worth noting that this system usually incorporates the more posterior angular gyrus as well (Corbetta et al. 2008). However, it is important to keep in mind that the PPI analysis identifies areas showing “altered connectivity” with a source region (in this case the pSTS) rather than regional areas of activation that are identified by standard fMRI paradigms showing angular gyrus involvement. Moreover, altered connectivity between the pSTS and the supramarginal and STG accords with the existence of anatomical connections between these regions in macaques (Seltzer and Pandya 1978; Rozzi et al. 2006). Hence, it seems likely that the activation in these areas we have observed reflects engagement of the ventral attentional system.

Altered connectivity between the MFG and both pSTS and FG in the current study is interesting in light of the proposal by Corbetta et al. (2008) that the MFG constitutes a point of interaction between the dorsal and ventral attention systems. This interaction enables the dorsal system to restrict ventral system activation to behaviorally important events and allows the ventral system to interrupt dorsal system activity when behaviorally important events are detected. Since changes in another's gaze direction are salient behavioral cues, they may cause the ventral system to alert the dorsal attentional network.

Overt and Covert Orienting of Attention to Eye Gaze

Haxby et al. (2000) emphasized a role for the IPS in the extended system for gaze processing but proposed that FEF could also be involved. The IPS and FEF form the dorsal attention system that is considered to underlie attentional target selection. FEF has an established role in transforming visual input into instructions for eye movements (Schall 1995). Its involvement in the preparation of covert attention shifts (thought to be predecessors of eye movements) is also underlined by recent studies showing that they are accomplished by the FEF's role in programming, but not executing, eye movements (Awh et al. 2006). Hence, altered connectivity between the pSTS and both FEF and IPS in the current study could potentially reflects a covert or overt shift in attention toward the direction indicated by the gaze.

Our study did not include a behavioral measure of attentional orienting or eye movements, thus we do not have any direct evidence that the gaze shift stimuli evoked shifts in attention. However, given that numerous independent studies have demonstrated covert (Friesen and Kingstone 1998; Driver et al. 1999; Langton and Bruce 1999) and overt (Deaner and Platt 2003; Mansfield et al. 2003) shifts of attention toward the direction of seen gaze, the absence of eye movement data should not be viewed as a problem for interpretation. Overt, as well as covert, shifts in attention are a natural response to viewing shifts in gaze direction, even when the gaze direction of a face is unattended as in the present study (Mansfield et al. 2003; Nummenmaa and Hietanen 2006). However, it seems unlikely that the changes in connectivity we have observed can be accounted for overt eye movements alone. Only the pSTS seed showed changes in connectivity with the FEF and IPS. If this was only due to saccades triggered by viewing other's eye movements, and then changes in connectivity with FEF might also have been expected with the FG seed region as well. Similarly, the contrast of gaze shifts versus open/close eyes examining changes in regional activation should also have identified FEF. Given the absence of these effects, there is no evidence that eye movements alone can explain our findings.

Other Components of the Gaze Perception Network

We found no evidence that the IOG showed altered connectivity with other brain regions as a function of gaze, even at a reduced threshold (P < 0.01, unc.). However, this could arise for a number of reasons. For example, the information conveyed from the IOG to the pSTS and FG may be relatively generic (i.e., does not discriminate between gaze shifts and open/close eye movements). Alternatively, connectivity between the IOG and pSTS or FG may be equally responsive to the gaze and open/close eye movements used in the current study. Either explanation would result in no differential effect of gaze. A third consideration is that projections between MT and pSTS (rather than the IOG and pSTS) may be critical for coding the sorts of dynamic facial stimuli used here. Consistent with this, altered connectivity between these regions was found as a function of gaze versus opening/closing the eyes.

Whether the extended network includes regions, in addition to those delineated by the present study, remains to be established. For example, areas such as medial prefrontal cortex (mPFC) and amygdala have also been implicated in gaze perception (see review in Nummenmaa and Calder 2009). So, why did these regions not show gaze-dependent connectivity changes with IOG, FG, and pSTS in the present study? One potential answer is that during gaze perception, regions such as amygdala and mPFC serve a function that was not engaged by our experimental task. For example, amygdala is typically engaged when explicit gaze (or gaze contact) monitoring is required (Kawashima et al. 1999; Hooker et al. 2003), and it may serve a general role in encoding behavioral salience or affective arousal evoked by others’ gaze. The mPFC is recruited during “mind reading” or social cognitive reasoning based on eye-gaze direction (Calder et al. 2002; Williams et al. 2005; Bristow et al. 2007). Accordingly, it is likely that such higher order (and potentially volitional) social processes were not recruited during the incidental viewing of the gaze shifts versus opening/closing the eyes during a gender detection task, hence neither changes in connectivity nor regional effects in amygdala or mPFC were observed. Therefore, it is possible that the extended network for gaze perception could potentially include amygdala, mPFC, and other “social” brain regions as well, but this needs to be verified in future studies.

Conclusions

In conclusion, our study constitutes the first application of connectivity analysis to delineating the brain network for gaze perception. Viewing gaze shifts was associated with specific increases in connectivity between MT and pSTS; between pSTS and components of the dorsal frontoparietal attention network involved in attentional target selection; and between pSTS and STG, previously implicated in both spatial awareness and gaze perception. The FG also showed changes in connectivity with overlapping areas of 2 of these regions (STG and MFG), highlighting a role for both superior (pSTS) and inferior temporal face areas (FG) in the gaze perception network; for summary, see Figure 3. In concert, we propose that this network alerts us to a change in other people's gaze (ventral attention network), produces an awareness of the spatial direction of the gaze (STG), and initiates a corresponding overt or covert change in our own focus of attention (dorsal attention network). Future research should determine whether connections in this network are similarly engaged by other social attention cues, such as head direction or pointing gestures.

Funding

Academy of Finland (119088 to L.N.); Alfred Kordelin's Foundation (2006 to L.N.); AivoAalto Grant from the Aalto University and the Medical Research Council (project code U.1055.02.001.00001.01 to A.J.C.).

Acknowledgments

We thank our volunteers for participating in this study, and MRC Cognition and Brain Sciences Unit radiographers for their help in data acquisition. Conflict of Interest: None declared.

References

  • Akiyama T, Kato M, Muramatsu T, Saito F, Nakachi R, Kashima H. A deficit in discriminating gaze direction in a case with right superior temporal gyrus lesion. Neuropsychologia. 2006;44:161–170. [PubMed]
  • Akiyama T, Kato M, Muramatsu T, Saito F, Umeda S, Kashima H. Gaze but not arrows: a dissociative impairment after right superior temporal gyrus damage. Neuropsychologia. 2006;44:1804–1810. [PubMed]
  • Awh E, Armstrong KM, Moore T. Visual and oculomotor selection: links, causes and implications for spatial attention. Trends Cogn Sci. 2006;10:124–130. [PubMed]
  • Barbas H, Mesulam MM. Organization of afferent input to subdivisions of area 8 in the rhesus monkey. J Comp Neurol. 1981;200:407–431. [PubMed]
  • Boddaert N, Chabane N, Gervais H, Good CD, Bourgeois M, Plumet MH, Barthelemy C, Mouren MC, Artiges E, Samson Y, et al. Superior temporal sulcus anatomical abnormalities in childhood autism: a voxel-based morphometry MRI study. Neuroimage. 2004;23:364–369. [PubMed]
  • Bristow D, Rees G, Frith CD. Social interaction modifies neural response to gaze shifts. Soc Cogn Affect Neurosci. 2007;2:52–61. [PMC free article] [PubMed]
  • Calder AJ, Jenkins R, Cassel A, Clifford CWG. Visual representation of eye gaze is coded by a nonopponent multichannel system. J Exp Psychol Gen. 2008;137:244–261. [PubMed]
  • Calder AJ, Lawrence D, Keane J, Scott SK, Owen AI, Christoffels I, Young AW. Reading the mind from eye gaze. Neuropsychologia. 2002;40:1129–1138. [PubMed]
  • Calder AJ, Young AW. Understanding facial identity and facial expression recognition. Nat Rev Neurosci. 2005;6:641–651. [PubMed]
  • Chokron S. On the origin of free-viewing perceptual asymmetries. Cortex. 2002;38:109–112. [PubMed]
  • Corbetta M, Patel G, Shulman GL. The reorienting system of the human brain: from environment to theory of mind. Neuron. 2008;58:306–324. [PMC free article] [PubMed]
  • Corbetta M, Shulman GL. Control of goal-directed and stimulus-driven attention in the brain. Nat Rev Neurosci. 2002;3:201–215. [PubMed]
  • Deaner RO, Platt ML. Reflexive social attention in monkeys and humans. Curr Biol. 2003;13:1609–1613. [PubMed]
  • Driver J, Davis G, Ricciardelli P, Kidd P, Maxwell E, Baron-Cohen S. Gaze perception triggers reflexive visuospatial orienting. Vis Cogn. 1999;6:509–540.
  • Engell AD, Haxby JV. Facial expression and gaze-direction in human superior temporal sulcus. Neuropsychologia. 2007;45:3234–3241. [PubMed]
  • Evans AC, Collins DL, Mills SR, Brown ED, Kelly RL, Peters TM. IEEE Nuclear Science Symposium and Medical Imaging Conference, pt 3. ed. 1994. 3D statistical neuroanatomical models from 305 MRI volumes; pp. 1813–1817.
  • Fairhall SL, Ishai A. Effective connectivity within the distributed cortical network for face perception. Cereb Cortex. 2007;17:2400–2406. [PubMed]
  • Fox CJ, Moon SY, Iaria G, Barton JJ. The correlates of subjective perception of identity and expression in the face network: an fMRI adaptation study. Neuroimage. 2009;44:569–580. [PMC free article] [PubMed]
  • Fox MD, Corbetta M, Snyder AZ, Vincent JL, Raichle ME. Spontaneous neuronal activity distinguishes human dorsal and ventral attention systems. Proc Natl Acad Sci USA. 2006;103:10046–10051. [PubMed]
  • Friesen CK, Kingstone A. The eyes have it! Reflexive orienting is triggered by nonpredictive gaze. Psychon Bull Rev. 1998;5:490–495.
  • Friston KJ, Buechel C, Fink GR, Morris J, Rolls E, Dolan RJ. Psychophysiological and modulatory interactions in neuroimaging. Neuroimage. 1997;6:218–229. [PubMed]
  • Ganel T, Valyear KF, Goshen-Gottstein Y, Goodale MA. The involvement of the “fusiform face area” in processing facial expression. Neuropsychologia. 2005;43:1645–1654. [PubMed]
  • George N, Driver J, Dolan RJ. Seen gaze-direction modulates fusiform activity and its coupling with other brain areas during face processing. Neuroimage. 2001;13:1102–1112. [PubMed]
  • Gitelman DR, Penny WD, Ashburner J, Friston KJ. Modeling regional and psychophysiologic interactions in fMRI: the importance of hemodynamic deconvolution. Neuroimage. 2003;19:200–207. [PubMed]
  • Grosbras MH, Laird AR, Paus T. Cortical regions involved in eye movements, shifts of attention, and gaze perception. Hum Brain Mapp. 2005a;25:140–154. [PubMed]
  • Haxby JV, Hoffman EA, Gobbini MI. The distributed human neural system for face perception. Trends Cogn Sci. 2000;4:223–233. [PubMed]
  • Hoffman EA, Haxby JV. Distinct representations of eye gaze and identity in the distributed human neural system for face perception. Nat Neurosci. 2000;3:80–84. [PubMed]
  • Hooker CI, Paller KA, Gitelman DR, Parrish TB, Mesulam MM, Reber PJ. Brain networks for analyzing eye gaze. Brain Res Cogn Brain Res. 2003;17:406–418. [PubMed]
  • Kanwisher N, McDermott J, Chun MM. The fusiform face area: a module in human extrastriate cortex specialized for face perception. J Neurosci. 1997;17:4302–4311. [PubMed]
  • Karnath H-O, Ferber S, Himmelbach M. Spatial awareness is a function of the temporal not the posterior parietal lobe. Nature. 2001;411:950–953. [PubMed]
  • Kawashima R, Sugiura M, Kato T, Nakamura A, Hatano K, Ito K, Fukuda H, Kojima S, Nakamura K. The human amygdala plays an important role in gaze monitoring—a PET study. Brain. 1999;122:779–783. [PubMed]
  • Lancaster JL, Tordesillas-Gutierrez D, Martinez M, Salinas F, Evans A, ZilleS K, Mazziotta JC, Fox PT. Bias between MNI and Talairach coordinates analyzed using the ICBM-152 brain template. Hum Brain Mapp. 2007;28:1194–1205. [PubMed]
  • Langton SRH, Bruce V. Reflexive visual orienting in response to the social attention of others. Vis Cogn. 1999;6:541–567.
  • Luh KE, Rueckert LM, Levy J. Perceptual asymmetries for free viewing of several types of chimeric stimuli. Brain Cogn. 1991;16:83–103. [PubMed]
  • Maioli MG, Squatrito S, Samolsky-Dekel BG, Riva Sanseverino E. Corticocortical connections between frontal periarcuate regions and visual areas of the superior temporal sulcus and the adjoining inferior parietal lobule in the macaque monkey. Brain Res. 1998;789:118–125. [PubMed]
  • Maldjian JA, Laurienti PJ, Kraft RA, Burdette JH. An automated method for neuroanatomic and cytoarchitectonic atlas-based interrogation of fMRI data sets. Neuroimage. 2003;19:1233–1239. [PubMed]
  • Mansfield EM, Farroni T, Johnson MH. Does gaze perception facilitate overt orienting? Vis Cogn. 2003;10:7–14.
  • McCourt ME, Freeman P, Tahmahkera-Stevens C, Chaussee M. The influence of unimanual response on pseudoneglect magnitude. Brain Cogn. 2001;45:52–63. [PubMed]
  • Nummenmaa L, Calder AJ. Neural mechanisms of social attention. Trends Cogn Sci. 2009;13:135–143. [PubMed]
  • Nummenmaa L, Hietanen JK. Gaze distractors influence saccadic curvature: evidence for the role of the oculomotor system in gaze-cued orienting. Vision Res. 2006;46:3674–3680. [PubMed]
  • O'Toole AJ, Roark DA, Abdi H. Recognizing moving faces: a psychological and neural synthesis. Trends Cogn Sci. 2002;6:261–266. [PubMed]
  • Passamonti L, Rowe JB, Ewbank M, Hampshire A, Keane J, Calder AJ. Connectivity from the ventral anterior cingulate to the amygdala is modulated by appetitive motivation in response to facial signals of aggression. Neuroimage. 2008;43:562–570. [PMC free article] [PubMed]
  • Paus T. Location and function of the human frontal eye-field: a selective review. Neuropsychologia. 1996;34:475–483. [PubMed]
  • Pelphrey KA, Viola RJ, McCarthy G. When strangers pass: processing of mutual and averted gaze in the superior temporal sulcus. Psychol Sci. 2004;15:598–603. [PubMed]
  • Puce A, Allison T, Bentin S, Gore JC, McCarthy G. Temporal cortex activation in humans viewing eye and mouth movements. J Neurosci. 1998;18:2188–2199. [PubMed]
  • Rhodes G. Lateralized processes in face recognition. Br J Psychol. 1985;76:249–271. [PubMed]
  • Rozzi S, Calzavara R, Belmalih A, Borra E, Gregoriou GG, Matelli M, Luppino G. Cortical connections of the inferior parietal cortical convexity of the macaque monkey. Cereb Cortex. 2006;16:1389–1417. [PubMed]
  • Schall JD. Neural basis of saccade target selection. Rev Neurosci. 1995;6:63–85. [PubMed]
  • Seltzer B, Pandya DN. Afferent cortical connections and architectonics of the superior temporal sulcus and surrounding cortex in the rhesus monkey. Brain Res. 1978;149:1–24. [PubMed]
  • Stephan KE, Marshall JC, Friston KJ, Rowe JB, Ritzl A, Zilles K, Fink GR. Lateralized cognitive processes and lateralized task control in the human brain. Science. 2003;301:384–386. [PubMed]
  • Tsuchiya N, Kawasaki H, Oya H, Howard MA, III, Adolphs R. Decoding face information in time, frequency and space from direct intracranial recordings of the human brain. PLoS One. 2008;3:e3892. [PMC free article] [PubMed]
  • Tzourio-Mazoyer N, Landeau B, Papathanassiou D, Crivello F, Etard O, Delcroix N, Mazoyer B, Joliot M. Automated anatomical labeling of activations in SPM using a macroscopic anatomical parcellation of the MNI MRI single-subject brain. Neuroimage. 2002;15:273–289. [PubMed]
  • Watanabe S, Kakigi R, Miki K, Puce A. Human MT/V5 activity on viewing eye gaze changes in others: a magnetoencephalographic study. Brain Res. 2006;1092:152–160. [PubMed]
  • Watson RT, Valenstein E, Day A, Heilman KM. New insights into the functions of the superior temporal cortex. Nat Rev Neurosci. 2001;2:568–576. [PubMed]
  • Williams JHG, Waiter GD, Perra O, Perrett DI, Whiten A. An fMRI study of joint attention experience. Neuroimage. 2005;25:133–140. [PubMed]
  • Wojciulik E, Kanwisher N. The generality of parietal involvement in visual attention. Neuron. 1999;23:747–764. [PubMed]
  • Worsley KJ, Marrett S, Neelin P, Vandal AC, Friston KJ, Evans AC. A unified statistical approach for determining significant signals in images of cerebral activation. Hum Brain Mapp. 1996;4:58–83. [PubMed]

Articles from Cerebral Cortex (New York, NY) are provided here courtesy of Oxford University Press