Culture, gaze and the neural processing of fear expressions

doi:10.1093/scan/nsp047

Soc Cogn Affect Neurosci. 2010 Jun-Sep; 5(2-3): 340–348.

Published online 2009 December 17. doi: 10.1093/scan/nsp047

PMCID: PMC2894663

Culture, gaze and the neural processing of fear expressions

Reginald B. Adams, Jr,¹ Robert G. Franklin, Jr,¹ Nicholas O. Rule,² Jonathan B. Freeman,² Kestutis Kveraga,³ Nouchine Hadjikhani,^3,⁴ Sakiko Yoshikawa,⁵ and Nalini Ambady²

¹Department of Psychology, The Pennsylvania State University, University Park, PA 16802, USA, ²Department of Psychology, Tufts University, Medford, MA 02155, USA, ³Athinoula A. Martinos Center for Biomedical Imaging Massachusetts General Hospital, Harvard Medical School, Charlestown 02129, USA, ⁴Brain Mind Institute, EPFL, CH-1015 Lausanne, Switzerland and ⁵Kyoto University, Department of Cognitive Psychology in Education, Kyoto University, Yoshida-honmachi, Sakyo, Kyoto, 606-8501, Japan

Corresponding author.

Correspondence should be addressed to Reginald B. Adams, Jr., The Pennsylvania State University, 544 Moore Building, University Park, PA 16802, USA. E-mail: radams/at/psu.edu.

Received March 25, 2009; Accepted October 19, 2009.

This article has been cited by other articles in PMC.

Abstract

The direction of others’ eye gaze has important influences on how we perceive their emotional expressions. Here, we examined differences in neural activation to direct- versus averted-gaze fear faces as a function of culture of the participant (Japanese versus US Caucasian), culture of the stimulus face (Japanese versus US Caucasian), and the relation between the two. We employed a previously validated paradigm to examine differences in neural activation in response to rapidly presented direct- versus averted-fear expressions, finding clear evidence for a culturally determined role of gaze in the processing of fear. Greater neural responsivity was apparent to averted- versus direct-gaze fear in several regions related to face and emotion processing, including bilateral amygdalae, when posed on same-culture faces, whereas greater response to direct- versus averted-gaze fear was apparent in these same regions when posed on other-culture faces. We also found preliminary evidence for intercultural variation including differential responses across participants to Japanese versus US Caucasian stimuli, and to a lesser degree differences in how Japanese and US Caucasian participants responded to these stimuli. These findings reveal a meaningful role of culture in the processing of eye gaze and emotion, and highlight their interactive influences in neural processing.

Keywords: eye gaze, facial expression, face perception, cross-cultural psychology, amygdala

INTRODUCTION

Dating back to Darwin’s seminal book, On the expression of the emotions in man and animals (1872/1965), cross-cultural research has served a critical role in establishing the universality of how we express and recognize basic emotional displays. Following this tradition, contemporary research efforts by Paul Ekman and colleagues employed cross-cultural methodology to examine the neurocultural theory of emotion (Ekman, 1972), in which they proposed that a universal affect program underlies the experience and expression of certain basic emotions (i.e. anger, fear, sad, happy, disgust, surprise; see Ekman and Friesen, 1971). According to this theory, basic emotional displays are universal and thus expressed and recognized uniformly across all cultures.

In a recent meta-analysis, it became apparent that measurable intracultural advantages also exist in basic emotion recognition (Elfenbein and Ambady, 2002). Elfenbein and Ambady (2003a; 2003b) posited the existence of a specific affect program, one that exerts subtle variation, like accents, on both the expression and perception of otherwise universal emotional displays (see also Marsh et al., 2003). In addition to such nuanced influences, there exist explicitly learned display rules (Ekman, 1972) and decoding rules (Matsumoto, 1989) that vary across cultures. These rules motivate individuals to override standard universal responses in overt attempts to, for example, mask, neutralize, or emphasize expressions according to what is deemed culturally appropriate. Of particular relevance to the current inquiry are cultural rules governing emotional expression and eye gaze behaviors.

Culture and emotion communication

In one example of culturally learned display rules, male Japanese participants were found to smile more often than male US Caucasian participants when viewing unpleasant film clips in the presence of others (Friesen, 1972; cited in Matsumoto, 2001). In Japanese society, overt displays of many negative emotions tend to be discouraged and thus smiling behavior in this context was considered an attempt to mask negative emotion. In US culture, however, overt displays of many negative emotions are often considered appropriate (Matsumoto, 1990). Such norms in turn influence inferences people make when reading the emotional expressions of others. For instance, when the norm of a culture is to express overt emotion, such as in the USA, less intensity is read into such displays, whereas when the norm is to attenuate overt expression, such as in Japan, more intensity is read into these displays (Matsumoto, Kasri, and Kooken, 1999).

Evidence for cultural variation in emotion processing at the neural level remains limited. One study examined such differences, reporting distinct regions of activation between Caucasian and Japanese participants when viewing fearful expressions (Moriguchi et al., 2005). Specifically, when viewing fear faces, US Caucasian participants recruited more posterior cingulate, amygdala and supplementary motor cortex as compared to Japanese participants, who recruited more inferior frontal cortex, premotor cortex and insula. These findings were interpreted as reflecting different culturally learned styles for decoding emotion. In a more recent study, the interaction of stimulus and observer culture was explored in Japanese and US Caucasian participants. Specifically, evidence was found for cultural tuning in neural responses to fear expressions (Chiao et al., 2008), with greater activation in the bilateral amygdalae in response to viewing same- compared to other-culture fear expressions (this effect was consistent across both cultural groups; see Adams et al., in press, for similar intracultural effect in bilateral posterior superior temporal sulci during complex mental state decoding). Chiao et al. reported similar effects in the left hippocampus, right superior temporal sulcus, right caudate, left superior frontal gyrus, right parahippocampual gyrus and right middle frontal gyrus.

Culture and eye gaze behavior

As with expression, eye gaze represents another nonverbal behavior with great social significance. Although eye gaze appears to be a universally important social cue, it can exert different meanings across cultures. Argyle and Cook (1976), for instance, suggested that in Western cultures, eye gaze is generally seen as a sign of respect. A failure to make eye contact may therefore be interpreted as suspicious. In East Asian cultures, making eye contact can be impolite and seen as threatening under certain circumstances, whereas averted eye gaze, especially downward shifts, can be seen as respectful (Knapp and Hall, 2002). Consequently, US Caucasian participants tend to look a same-culture experimenter in the eye when answering a question and Japanese participants tend to look away from a same-culture experimenter (McCarthy et al., 2006).

To date, the influence of culture on neural responses to gaze information has not been examined. However, one study examined the influence of gaze on responses to viewing same- versus other-race faces (Richeson et al., 2008). In this study, when contrasting responses in Caucasian participants to direct- versus averted-gaze Black faces, more right amygdala activation was evident, as well as bilateral insula, left suprmarginal gyrus, right hippocampus, right middle frontal gyrus, left inferior occipital gyrus, right precentral gyrus, and medial orbital gyrus. Similar responses were not apparent when viewing White faces, leading the authors to conclude that the threatening, approach-oriented signal stereotypically ascribed to Black faces was compounded by direct gaze (see also Adams and Kleck, 2003, 2005; Ambady and Adams, in press, for rationale of compound social cues).

Eye gaze and emotion

Eye gaze also affects the perception of emotional expression. Using speeded reaction time tasks and self-reported perception of emotional intensity, Adams and Kleck (2003, 2005) found that direct relative to averted gaze affected perceptions of approach-oriented emotional facial expressions (e.g. anger and joy) by facilitating speed of processing, increasing recognition accuracy, and augmenting perceived intensity. Averted relative to direct gaze, on the other hand, exerted a similar influence on the perception of avoidance-oriented emotions (e.g. fear and sadness). Such interactive influences of gaze and emotion perception has since been replicated and extended by Sander et al. (2007) using dynamic threat displays, by Hess, Adams, and Kleck (2007) examining reactive responses to displays in observers, and by Graham and LaBar (2005; see also Bindeman et al., 2008) who demonstrated that the relative salience of eye gaze and emotion moderates such interactive influences.

Similarly, emotional expression has been found to influence the processing of eye gaze information. Studying individuals with high anxiety, for instance, Fox et al. (2008) found that fear expressions facilitated reflexive orienting to averted eyes, whereas anger facilitated attention capture to the face. Similarly, using a speeded reaction time task, Adams and Franklin (2009) found that facial expressions of anger facilitated the recognition of direct eye gaze and facial expressions of fear facilitated the recognition of averted eye gaze. Finally, another recent study revealed that observers tend to misperceive eye gaze as looking directly at them more often when presented on happy and angry faces as compared to neutral and fearful faces (Lobmaier et al., 2008).

The interaction between eye gaze and emotion is also apparent at the neural level. Adams et al. (2003) reported evidence for an interaction between eye gaze and emotion processing such that the left amygdala responded more when participants viewed ambiguous threat-gaze pairings (direct-gaze fear and averted-gaze anger) compared to congruent threat-gaze pairings (averted-gaze fear and direct-gaze anger). This finding fits with previously reported patterns of amygdala activation to fear and anger faces that were presented with only direct gaze (e.g. Whalen et al., 2001). To explain these previous results, Whalen (1998; see also Davis and Whalen, 2001) proposed that the amygdala may be instrumental in processing threat-related ambiguity, and thus the level of response may be directly proportional to the level of ambiguity surrounding the source of a threat. Adams et al.’s findings corroborate this hypothesis, replicating the original pattern found by Whalen and his colleagues in direct-gaze faces (i.e. greater amygdala activation to fearful faces than to angry faces), while revealing the reverse pattern for averted-gaze faces (i.e. greater amygdala activation to angry faces than to fearful faces).

Despite evidence for amgydala responses to threat-ambiguity as described above, there also exists recent evidence for greater amygdala responsivity to congruent versus ambiguous threat-gaze pairs. In one study, which presented predominantly US Caucasian faces to US Caucasian perceivers, greater amgydala response was found to averted- relative to direct-gaze fear faces (Hadjikhani et al., 2008), as well as greater superior temporal sulcus, intraparietal sulcus, fusiform, inferior occipital gyrus, hypothalamus, pallidum, visual area MT, premotor/motor cortex, caudate, somatosensory cortex, and anterior fusiform. Similarly, in a study presenting Japanese faces to Japanese perceivers, greater amygdala activation was found in response to peripherally presented anger expressions looking in toward a fixation cross (i.e. oriented inward toward the observer) relative when looking away from a fixation cross (i.e. oriented outward away from the observer; Sato et al., 2004). Thus, whereas Adams et al.'s (2003) findings seem to implicate a top-down process involved in deciphering ambiguity, these more recent findings (Sato et al., 2004; Hadjikhani et al., 2008) seem to indicate a more reflexive alerting response to salient threat-gaze combinations. Moderators of these opposing effects remain undetermined, but variations in stimulus clarity and temporal factors are likely candidates (see also Mogg, Garner, and Bradley, 2007). For the purposes of the current examination, in which we focused on examining cross-cultural influences on the role of gaze and in early threat perception, we employed a modified version of Hadjikhani et al.’s paradigm, using rapid (300 ms) presentations, as opposed to longer presentations previously employed by our own research team (i.e. 2 s; Adams et al., 2003).

The current study

The aim of the current study was to extend previous research examining the role of gaze in emotion processing by focusing on the impact of culture on the role of gaze in basic fear perception. Specifically, we were interested in examining responses to rapidly presented fear expressions, following Hadjikhani et al.’s (2008) findings of greater amygdala responsivity to clear versus ambiguous fear-gaze pairings when using short exposures. Given the literature reviewed above, our primary predictions were that same-culture faces would yield greater amygdala activation for averted- versus direct-gaze fear. In accordance with Chiao et al.’s (2008) finding for culturally tuned amygdala responses to fear expressions, we predicted that this effect would be greater in response to same- versus other-culture faces. Finally, considering recent evidence for greater amygdala responses to other-race faces when displaying direct- versus averted-gaze (Richeson et al., 2008), we were also able to explore whether similar group differences might extend to cultural group memberships as well, which predicts greater direct- relative to averted-gaze amygdala responses to other-culture faces. Notably, these predictions all represent intracultural influences in which similar responses across cultures would vary as a function of the relationship between the stimulus culture and the participant culture. Given evidence for explicit differences in cultural display rules regarding emotion and gaze behavior, we were also interested in exploring potential intercultural variation in neural responses to emotional expression and eye gaze behavior.

METHOD

Participants

Thirty-four participants were recruited using online postings or paper advertisements. Sixteen native Japanese students (eight women) and 18 Caucasian students (nine women) participated in the study for financial compensation. The Japanese students were visiting the USA as part of a university summer English language program. None of these students had spent more than a month in the USA prior to participating in this study. All participants were right handed, had normal or corrected-to-normal vision, and had no known medical, neurological, or psychiatric history. Two Japanese participants (one man, one woman) were dropped from the analysis because of excessive movement during the trials or anatomical features that impeded normalization into standard brain space.

Stimuli

We selected eight images (four women, four men) of Caucasian models displaying fearful faces from the Pictures of Facial Affect (Ekman and Friesen, 1978). These faces were ones used by Adams et al. (2003), and had been previously shown using a speeded reaction time paradigm to yield faster recognition for averted versus direct gaze fear when viewed by US Caucasian participants. In each of these images, we digitally altered eye gaze using Adobe Photoshop to generate faces with direct- and laterally averted (left and right) gaze. We selected Japanese faces from a larger set constructed at Kyoto University (Yoshikawa, 2009), which were similarly digitally manipulated to create direct and averted eye gaze. In order to match these faces with the Caucasian set, we chose eight models (four women, four men) that similarly yielded faster responding to averted than direct gaze fear when viewed by Japanese participants.

Procedure

As noted earlier, we adopted a version of the paradigm used by Hadjikhani et al. (2008), which consisted of two runs of passive viewing in an ABA block design alternating between direct and averted gaze fear faces showing either Japanese or Caucasian individuals across four block types (Caucasian-Averted, Caucasian-Direct, Japanese-Averted, Japanese-Direct). Each block type was repeated four times to yield 16 blocks per run. The culture of the faces alternated every eight blocks, and was counterbalanced across runs. The order of runs was also counterbalanced across participants. Within each block, sixteen faces were displayed for 300 ms, with a 1200 ms fixation cross displayed between stimuli, totaling 24 s. Each run lasted 6 min and 24 s.

fMRI data analysis

fMRI scanning employed a 3-T Siemens Tim Trio scanner with a standard 12 channel headcoil for data acquisition. Scanning consisted of two experimental functional runs and a T₁-weighted anatomical scan collected for coregistraton and normalization. Functional scans consisted of 192 dynamic whole-brain T2* weighted images collected in an oblique axial orientation (TR 2s, TE 30ms, 90° flip angle, voxel size, 3.125 × 3.125 × 5 mm, 32 interleaved slices, 192 volumes, 5 mm slice thickness with 1 mm slice gap). High-resolution T1-weighted (MP-RAGE, 128 sagittal slices 1.33 mm thick, 256 × 256 matrix) anatomical images were collected for each participant for coregistration with the participants’ functional data and display of individual activations. Foam padding around the head was used to minimize head movement. Images were viewed using a back-projection system.

Data were pre-processed and analyzed using SPM5 software. Images were realigned to the first functional image using a six-parameter rigid rotation. Each participant’s anatomical image was coregistered to a mean image of each participant's; functional images. Anatomical images were normalized according to a T1 template image using a standard space defined by the Montreal Neurological Institute (MNI) and these normalization parameters were applied to the functional images, in order to allow for group comparisons. Functional images were then smoothed using an 8 mm FWHM Gaussian kernel.

Contrasts were initially estimated using a fixed effects model within each subject. Stimulus conditions were modeled as delayed boxcar functions convolved with a standard hemodynamic response function. Low frequency signal components less than 0.007 Hz were filtered out using a high-pass filter. To make population inferences, contrast images from each subject were then entered into a mixed-effects model representing summary measures of subject responses. Statistical comparisons were conducted using a mass-univariate GLM approach comparing blocks of direct eye gaze with blocks of averted eye gaze.

For the whole-brain analysis we employed a set threshold of P < 0.001 (voxelwise), uncorrected, with an extent-threshold of six voxels. First, we computed a voxel-wise 2 (culture of participant) ×2 (culture of face) mixed effects analysis of variance (ANOVA). We extracted parameter estimates (beta values) for the peaks of each cluster defined by this analysis for each treatment condition to inspect the nature and direction of the main effects and the interaction effects found (Table 1). Since each eye gaze condition served as baseline for the other using this paradigm, the dependent variable of interest was computed as direct minus averted gaze, with positive BOLD signal thereby indicating more activation to direct gaze and negative to averted gaze.

Table 1

Regions of activation associated with the main effect of stimulus culture and the interaction of participant and stimulus culture

Based on our a priori hypotheses we also ran a region of interest analysis on bilateral amygdalae. To do this, we first extracted mean beta values from the entire anatomically defined region of interest using the AAL atlas (Tzourio-Mazoyer et al., 2002) and the WFU Pickatlas (Maldjian, Laurienti, Burdette, and Kraft, 2002) and submitted these to a subject-wise 2 (laterality: left versus right) by 2 (culture of participant) by 2 (culture of stimulus) mixed factorial ANOVA. We then conducted single-sample t-tests (subject-wise) within each treatment condition on the mean beta values extracted from each cluster that survived a threshold of P < 0.01 (voxelwise), uncorrected, extent-threshold of five voxels (as in Demos et al., 2008) in order to explicate the nature and direction of these interaction effects. For illustration purposes, all group contrast images were overlaid onto a representative T₁ template anatomical image using MRIcron (http://www.sph.sc.edu/comd/rorden/mricro.html).

RESULTS

Whole-brain analysis

We computed a 2 (culture of participant) ×2 (culture of stimulus) mixed effects analysis of variance (ANOVA), which yielded main effects of culture of participant and culture of stimulus, and an interaction between the two (Figure 1). In order to interpret the direction of these effects, we extracted peak beta values for all significant clusters and have displayed these by treatment condition (Table 1).

Fig. 1

(A) Top half of figure depicts global activations associated with main effects and interaction at P < 0.001, uncorrected, superimposed on the rendered brain from four perspectives, top, bottom, left, and right. (B) Bottom half of figure depicts (more ...)

Only one significant cluster, the right angular gyrus, was revealed in the main effect of participant culture. Inspection of the peak beta values for each treatment condition shows this effect was driven largely by relatively greater activation to direct versus averted gaze in Japanese participants and averted versus direct gaze in US Caucasian participants viewing fear faces (Table 1, top panel). Several regions were revealed in the main effect of stimulus culture including bilateral inferior temporal gyri, left cerebellum, left lateral occipital gyrus, bilateral angular gyri extending into the intraparietal sulcus, and bilateral fusiform gyri. Upon inspection of these peak beta values, all showed similar patterns of activation. Japanese stimuli tended to elicit greater activation for direct- versus averted-gaze fear faces, whereas US Caucasian faces tended to elicit greater activation for averted- versus direct-gaze fear faces (Table 1, middle panel).

For the interaction between gaze and emotion, significant clusters included bilateral thalamus, bilateral cerebellum, bilateral fusiform, left insula, left ventrolateral prefrontal cortex, left inferior frontal gyrus, left dorsolateral prefrontal cortex, right midbrain, and right parahippocampal gyrus extending into the right amygdala. Inspection of the peak beta values for these regions showed similar patterns of activation across stimulus conditions reflecting relatively greater activation to averted- versus direct-gaze fear for same-culture faces and greater activation to direct- versus averted-gaze fear for other-culture faces (Table 1, bottom panel).

Region of interest analysis

As indicated in the whole-brain voxelwise analysis, there was a cluster of parahippocampal gyrus extending into the right amygdala apparent for the interaction of stimulus and participant culture. In order to isolate and examine amygdala activation specifically, given that it was our primary region of interest, we extracted beta values from all voxels within the anatomically defined region of bilateral amygdalae and submitted these to a 2 (laterality: left versus right) by 2 (culture of stimulus: Japanese versus US Caucasian) by 2 (culture of observer: Japanese versus US Caucasian) ANOVA (subject-wise). This analysis revealed only the predicted two-way interaction between culture of stimulus and culture of participant, F(1, 30) = 5.55, P = 0.025.

In order to examine this interaction more closely, we extracted mean beta values for clusters within the anatomically defined region that survived a threshold of P < 0.01 (voxel-wise), uncorrected, extent-threshold: 5 voxels (Demos et al., 2008). At this threshold, clusters in both the left (x = −28, y = −4, z = −12; peak voxel, P < 0.005) and right (x = 30, y = −6, z = −12; peak voxel, P < 0.001) amygdalae were apparent (bottom Figure 1). We then submitted these mean beta values to one sample t-tests comparisons (subject-wise) within each treatment condition to explicate the direction and nature of the interaction. These comparisons revealed relatively greater activation to direct- versus averted-gaze fear faces in both cultural groups, when viewing other-culture faces in both the left and right amygdalae (t’s > 2.3, P’s < 0.05, two-tailed). Further, US Caucasian participants showed significantly greater activation to averted- versus direct-gaze fear when viewing same-culture fear expressions in the right amygdala [t(17) = 2.21, P < 0.05, two-tailed], a pattern that was evident but not significant in the left amygdala [t(17) = 1.58, P = 0.16]. Japanese participants showed no significant differences in either the right or left amygdala when viewing direct versus averted same-culture faces.

DISCUSSION

The current study revealed the predicted interaction between culture of participant and culture of stimulus in a number of regions involving face, gaze and emotion processing, including the bilateral fusiform gyri, left posterior cingulate, bilateral thalamus, left insula, right caudate, left inferior frontal gyrus, and left ventro- and dorsolateral prefrontal cortices. Helping explain the interaction effects within these regions was the relative increase in activation to direct- versus averted-gaze fear expressions in both Japanese and US Caucasian participants when viewing other-culture faces, and the relative increase in activation to averted- versus direct-gaze fear expressions in both cultural groups when viewing same-culture faces. This general pattern was also apparent when we examined activation within the anatomically defined region of bilateral amygdalae. Thus, this analysis replicates Hadjikhani et al.’s (2008) finding for greater averted- versus direct-gaze fear activation in US Caucasian participants viewing same-culture fear faces. For Japanese participants, however, no differences were found in processing averted- and direct-gaze same-culture fear faces in the amygdala. Given evidence for differences in neural activation that favor direct- versus averted-gaze in Japanese participants and stimuli, it may be that the differences found here were attenuated for Japanese participants due to the cultural meaning applied to direct gaze in Japanese culture. Direct eye gaze could have been construed as a threatening social cue when viewed in the context of a fear expression. Notably, both Japanese and US Caucasian participants showed similar amygdala responses to other-culture faces, with greater responsivity to direct- relative to averted-gaze fear faces. This latter effect is consistent with evidence for more amygdala activation in response to direct- relative to averted-gaze displays when viewing other-race faces (Richeson et al., 2008).

There were also a number of regions related to face and eye gaze processing activated by the main effect of stimulus culture, including bilateral fusiform gyri, bilateral inferior temporal gyri, and bilateral angular gyri. Physiognomic differences in prototypical race appearance, such as eye size (see Zebrowitz, 1997) and brow height (Keating, Mazur, and Segall, 1977), could arguably be a source of such variation; however, our dependent variable of interest was computed as the direct comparison of direct and averted eye gaze on otherwise identical faces and expressions, thereby ruling out such explanations. The differences in activation necessarily reflect different processing of gaze behavior in the context of fear faces across the two cultures. Specifically, all participants showed greater activation to direct- relative to averted-gaze when displayed on Japanese faces, whereas the opposite was true for US Caucasian faces. This finding suggests that both Japanese and US Caucasian participants appear to share a common understanding of the distinct cultural meanings associated with gaze behavior and thereby process gaze in a similar manner. In this case, participants from both cultural groups showed more activation to incongruous eye gaze behaviors, based on what is generally considered most culturally appropriate. This is a unique finding suggesting that social cues related to culture may be transmitted and processed consistently across observers from different cultures.

The current investigation showed only one significant effect of gaze direction on the processing of fear expressions as a function of culture of participant, in the left angular gyrus. Given the known role of the angular gyrus/intraparietal sulcus in spatial orienting and gaze perception (Hoffman and Haxby, 2000; Sato et al., 2008), this finding may represent a fundamental difference in gaze perception for Japanese and US Caucasian individuals—particularly in the context of processing fear. Inspection of the means revealed that this effect was largely driven by greater responses to direct- relative to averted-gaze in Japanese participants and to averted- relative to direct-gaze in US Caucasian participants. Given the cultural differences in display rules associated with gaze behavior, this finding may reflect a different role for direct versus averted gaze in these cultural groups. Given that this was not an a priori region of interest, however and that the whole-brain analysis yielded just this one relatively small cluster of activation showing this pattern, this finding must be taken with caution and warrants future corroboration.

The activations reported here show considerable overlap and consistency with regions previously reported (Table 1). The current work extends previous findings by demonstrating a robust intracultural effect within these regions for the role of gaze in fear perception. Of particular interest was the finding that amygdala responses to threat-gaze pairings varied as a function of whether participants viewed same- versus other-culture faces. This finding may help inform current inconsistencies reported across studies for role of gaze on fear processing at the neural level, with some work showing greater amygdala responses to ambiguous threat-gaze pairs (i.e. Whalen et al., 2001; Adams et al., 2003) and other work showing greater amygdala responses to clear threat-gaze pairs (Sato et al., 2004; Hadjikhani et al., 2008). In the current study the pattern of these responses cannot be attributed simply to a particular task or stimulus feature, and thus although the current work does not resolve this issue, the findings reported here are uniquely informative to ongoing research efforts examining this issue.

These findings also contribute to basic emotion theory. Recall that the specific affect program (Elfenbein and Ambady, 2003) highlights subtle culturally driven variation in expressive output as well as perceptual attunements to such variation. The current work suggests that culture heightens attunement to subtle, universally relevant aspects of expressions as well, in this case the mere millimeter shifts of irises and pupils that constitute changes in gaze direction. Recent work examining the integration of gaze in threat perception by women at different points along the menstrual cycle offers similar insight (see Conway et al., 2007). This work found that women high in progesterone perceived fear and disgust expressions as more intense when coupled with averted relative to direct gaze, whereas those low in progesterone showed the opposite pattern. Given that both fear and disgust signal the presence of an external contagion or threat in the environment, gaze is ecologically relevant to detecting its source. Thus, they argued that because pregnancy is also marked by high progesterone, the greater integration of averted gaze in the perception of fear and disgust in this study was likely due to an enhanced attunement triggered by increased threat vigilance. Continued inquiry into intrapersonal influences such as those found by Conway et al., and cross-cultural influences such as those revealed by the present work, on compound threat cue processing is likely to yield important new insights into the perception of what are otherwise often regarded as obligatory responses to threat displays.

In sum, the dependent variable of interest employed in this examination represented a contrast between direct- and averted-gaze on otherwise identical faces posing identical expressions. Thus, every face and corresponding expression served as its own control. The neural differences we found were due to responses to the few millimeter shifts in pupils and irises that represent changes in gaze direction. Demonstrating any differences with such a small manipulation underscores just how powerful a social cue gaze is, despite being physically subtle. Gaze also holds different meanings cross-culturally and holds different meanings when conveyed by same- versus other-culture group members. That we found pronounced cultural variation in the influence of gaze on the neural processing of fear expressions highlights the extent to which both culture and gaze meaningfully influence what are often assumed to be obligatory neural responses to threat (see also Chiao et al., 2008). Taken together, these findings suggest that we must consider how various social cues are processed in combination with one another for our understanding of the processing of each type of cue to be fully realized. Moreover, these findings underscore how cross-cultural examination is likely to advance a basic understanding of the mental and neural operations underlying social and emotional perception.

Acknowledgments

This research was supported by a Social Science Research Institute research grant, the Pennsylvania State University to RBA, Jr., a University Graduate Fellowship to RGF, Jr., Pennsylvania State University, a National Science Foundation grant (BCS-0435547 to N.A.); and a National Science Foundation graduate research fellowship (to N.O.R).

REFERENCES

Adams RB, Jr., Franklin RG., Jr Influence of emotional expression on the processing of gaze direction. Motivation & Emotion. 2009;33:106–12.
Adams RB, Jr., Gordon HL, Baird AA, Ambady N, Kleck RE. Effects of gaze on amygdala sensitivity to anger and fear faces. Science. 2003;300:1536. [PubMed]
Adams RB, Jr., Kleck RE. Perceived gaze direction and the processing of facial displays of emotion. Psychological Science. 2003;14:644–7. [PubMed]
Adams RB, Jr., Kleck RE. Effects of direct and averted gaze on the perception of facially communicated emotion. Emotion. 2005;5:3–11. [PubMed]
Adams RB, Jr., Rule N, Franklin RG, Jr., et al. Cross-cultural reading the mind in the eyes. An fMRI investigation. Journal of Cognitive Neuroscience. in press [PubMed]
Ambady A, Adams R.B., Jr. Us versus them: the social neuroscience of perceiving outgroups. In: Todorov A, Fiske ST, Prentice D, editors. Social Neuroscience: Toward Understanding the Underpinnings of the Social Mind. Oxford University Press; in press.
Argyle M, Cook M. Gaze and mutual gaze. Cambridge, UK: Cambridge University Press; 1976.
Bindemann M, Burton AM, Langton SRH. How do eye gaze and facial expression interact? Visual Cognition. 2008;16:708–33.
Chiao JY, Iidaka T, Gordon HL, et al. Cultural specificity in amygdala response to fear faces. Journal of Cognitive Neuroscience. 2008;20:2167–74. [PubMed]
Conway CA, Jones BC, DeBruine LM, et al. Salience of emotional displays of danger and contagion in faces is enhanced when progesterone levels are raised. Hormones and Behavior. 2007;51:202–6. [PubMed]
Darwin C. The Expression of the Emotions in Man and Animals. Chicago: The University of Chicago Press; 1872/1965. (Originally published, 1872)
Davis M, Whalen PJ. Vigilance and emotion. Molecular Psychiatry. 2001;6:13–34. [PubMed]
Demos KE, Kelley WM, Ryan SL, Davis FC, Whalen PJ. Human amygdala sensitivity to the pupil size of others. Cerebral Cortex. 2008;18:2729–34. [PubMed]
Ekman P. Universals and cultural differences in facial expressions of emotion. In: Cole J, editor. Nebraska Symposium on Motivation, 1971. Vol. 19. Lincoln: University of Nebraska Press; 1972. pp. 207–82.
Ekman P, Friesen WV. Constants across cultures in the face and emotion. Journal of Personality and Social Psychology. 1971;17:124–9. [PubMed]
Elfenbein HA, Ambady N. On the universality and cultural specificity of emotion recognition: a meta-analysis. Psychological Bulletin. 2002;128(2):203–35. [PubMed]
Elfenbein HA, Ambady N. Universals and cultural differences in recognizing emotions. Current Directions in Psychological Science. 2003a;12(5):159–64.
Elfenbein HA, Ambady N. When familiarity breeds accuracy: cultural exposure and facial emotion recognition. Journal of Personality and Social Psychology. 2003b;85(2):276–90. [PubMed]
Graham R, LaBar KS. Garner interference reveals dependencies between emotional expression and gaze in face perception. Emotion. 2007;7:296–313. [PubMed]
Hadjikhani N, Hoge R, Snyder J, de Gelder B. Pointing with the eyes: the role of gaze in communicating danger. Brain and Cognition. 2008;68:1–8. [PMC free article] [PubMed]
Hess U, Adams R.B., Jr., Kleck RE. Looking at you or looking elsewhere: the influence of head orientation on the signal value of emotional facial expressions. Motivation and Emotion. 2007;31:137–44.
Keating CF, Mazur A, Segall MH. Facial gestures which influence the perception of status. Sociometry. 1977;40:374–8.
Knapp ML, Hall JA. Nonverbal Communication in Human Interaction. Belmont, CA: Wadsworth; 2002.
Lobmaier JS, Tiddeman BP, Perrett DI. Emotional expression modulates perceived gaze direction. Emotion. 2008;8:573–7. [PubMed]
Marsh AA, Elfenbein HA, Ambady N. Nonverbal “accents”: cultural differences in facial expressions of emotion. Psychological Science. 2003;14:373–6. [PubMed]
Matsumoto D. Cultural influences on the perception of emotion. Journal of Cross-Cultural Psychology. 1989;20:92–105.
Matsumoto D. Cultural similarities and differences in display rules. Motivation and Emotion. 1990;14:195–214.
Matsumoto D. Culture and emotion. In: Matsumoto D, editor. Handbook of Culture and Psychology. New York: Oxford University Press; 2001. pp. 171–94.
Matsumoto D, Kasri F, Kooken K. American-Japanese cultural differences in judgements of expression intensity and subjective experience. Cognition & Emotion. 1999;13:201–18.
McCarthy A, Lee K, Itakura S, Muir DW. Cultural displays rules drive eye gaze during thinking. Journal of Cross-Cultural Psychology. 2006;37:717–22. [PMC free article] [PubMed]
Mogg K, Garner M, Bradley BP. Anxiety and orienting of gaze to angry and fearful faces. Biological Psychology. 2007;76:163–9. [PMC free article] [PubMed]
Moriguchi Y, Ohnishi T, Kawachi T, et al. Specific brain activation in Japanese and Caucasian people to fearful faces. NeuroReport. 2005;16:133–6. [PubMed]
Richeson JA, Todd AR, Trawalter S, Baird AA. Eye-gaze direction modulates race-related amygdala activity. Group Processes and Intergroup Relations. 2008;11:233–46.
Sander D, Grandjean D, Kaiser S, Wehrle T, Scherer KR. Interaction effects of perceived gaze direction and dynamic facial expression: evidence for appraisal theories of emotion. European Journal of Cognitive Psychology. 2007;19:470–80.
Sato W, Yoshikawa S, Kochiyama T, Matsumura M. The amygdala processes the emotional significance of facial expressions: an fMRI investigation using the interaction between expression and face direction. NeuroImage. 2004;22:1006–13. [PubMed]
Whalen PJ. Fear, vigilance, and ambiguity: Initial neuroimaging studies of the human amygdala. Current Directions in Psychological Science. 1998;7:177–88.
Whalen PJ, Shin LM, McInerney SC, Fischer H, Wright CI, Rauch SL. A functional MRI study of human amygdala responses to facial expressions of fear versus anger. Emotion. 2001;1:70–83. [PubMed]
Yoshikawa S. Young Adult Japanese Facial Displays. Unpublished stimulus set, Japan: Kyoto University; 2009.
Zebrowitz LA. Reading Faces: Window to the Soul? Boulder, CO: Westview Press; 1997.

Articles from Social Cognitive and Affective Neuroscience are provided here courtesy of
Oxford University Press