|Home | About | Journals | Submit | Contact Us | Français|
In this study, we compared the effects of using neutral face masks vs non-face pattern masks on amygdala activity to masked fearful faces. Twenty-seven subjects viewed 18 s blocks of either fearful or happy faces masked with either neutral faces or patterns, while their brain activity was measured using functional magnetic resonance imaging. Results replicated increased amygdala activation to face-masked fearful vs happy faces. In the pattern mask condition, the amygdala discriminated between masked fearful and happy faces, but this effect manifested as a decrease in activation to fearful faces compared to happy faces. This interactive effect between facial expression and mask stimulus shows that amygdala responses to masked fearful faces are influenced by the fearful stimuli per se as well as their interaction with the mask stimulus.
A number of human neuroimaging studies have demonstrated the relative automaticity of amygdala activation to fearful faces by using backward masking to mitigate the subjective awareness of study participants (Morris et al., 1998; Whalen et al., 1998; Rauch et al., 2000; Etkin et al., 2004; Pessoa et al., 2006; Williams et al., 2006). In backward masking experiments, the target stimulus (e.g. fearful face) is presented for a brief period of time (e.g. 17 ms, 33 ms) and then immediately replaced with a mask stimulus (e.g. neutral face). Even though study participants report being unaware of the target stimulus, studies have shown increased amygdala activity to fearful face targets (Morris et al., 1998; Whalen et al., 1998; Rauch et al., 2000; Williams et al., 2006). Experiments employing the backward masking paradigm also have been applied to the study of psychopathological groups and have shown abnormally exaggerated amygdala response in disorders such as post-traumatic stress disorder (Rauch et al., 2000) and depression (Sheline et al., 2001).
So far, studies employing this experimental technique have used neutral faces to mask the presence of the fearful expressions. Thus, even though the subjects were unaware of the presence of masked fearful faces, they were aware of viewing numerous neutral faces in the experimental context. The fact that backward masking studies of fearful expressions have used neutral faces as mask stimuli leads to an important question concerning the nature of amygdala activity—are amygdala responses to masked fearful faces dependent upon their presentation within a face context? In other words, would the amygdala show preferential activity in response to masked fearful faces, even when they are presented in a non-face context? Answering this question may help us understand whether the amygdala is responding to masked fearful faces per se, or to neutral faces that are perceptually primed by fearful target faces.
We sought to better understand amygdala responses during backward masking by comparing the masking of fearful faces with neutral faces to the masking of fearful faces with a non-face stimulus (i.e. pattern mask). If the amygdala were solely sensitive to the masked fearful face, then we would expect to see a similar increase in amygdala activity regardless of the type of the mask. However, if amygdala responses during backward masking are influenced by the neutral face mask stimulus, then we would expect to observe differential amygdala activity in the neutral face mask condition when compared with the pattern mask condition.
Twenty-seven healthy volunteers (18 women, 19.7 ± 0.99 years of age, 26 right-handed) participated in the current study. All of the subjects were screened for current or past psychiatric illness (Axis I or II) using an abbreviated version of the Structured Clinical Interview for DSM-IV (First et al., 1997). No subjects had ever taken psychotropic medications. Handedness was determined with the Edinburgh Handedness Inventory (Oldfield, 1971). After the functional magnetic resonance imagining (fMRI) scanning sessions, each subject’s anxiety level was assessed with the State Trait Anxiety Inventory (STAI-S, STAI-T) (Spielberger et al., 1988) self-report questionnaires. In addition, the subjects completed the Beck Depression Inventory (BDI) (Beck et al., 1961). The study protocol was approved by the Committee for the Protection of Human Subjects at Dartmouth College. Written, informed consent was obtained from the participants prior to the experiment.
Faces with fearful, happy and neutral expressions from six different individuals (three males and three females) were used (Ekman and Friesen, 1976). The faces were normalized for size and luminance. The pattern mask image was developed after testing many configurations resulting in the current image which was found to mask fearful faces as effectively as the neutral faces (i.e. similar low levels of subjective report across subjects compared to the neutral face mask condition; Figure 1). All of the stimuli were back-projected (Panasonic PT-D4000U DLP) onto a screen, which the subjects viewed using a mirror that was mounted on the head coil. Stimulus presentation time was carefully assessed with a photodiode-oscilloscope system (Tektronix TDS 2012) by averaging 100 trials, and verified that all target stimuli were presented between 16–17 ms (<2 refresh rate in a 60 Hz display).
For consistency with our previous work (Whalen et al., 1998), subjects were asked to passively view blocked presentations of masked images that appeared on the screen during three functional scans. During each scan, masked fearful and happy faces were presented separately in alternating 18 s blocks, interleaved with 18 s blocks showing a single crosshair at the middle of the screen. Within each 18 s face block, a total of 36 masked fearful or happy faces were presented on a black background. Each face was shown on the screen for 17 ms, and was immediately followed by a mask stimulus (neutral faces or pattern images) that was presented for 183 ms, with a fixed inter-stimulus interval of 300 ms (Figure 1). For the face mask condition, a different face identity was used for each fearful target and corresponding neutral mask, consistent with our previous study (fully counterbalanced across target and mask position across all six face identities; Whalen et al., 1998). The order of the faces within a block was pseudo-randomized to ensure that the same face was not presented more than twice in a row. The pattern mask was designed and piloted to produce similarly effective masking compared to the face masks, and this one pattern mask was used throughout the experiment. Thus, there were four types of blocks: (i) face-masked fearful, (ii) face-masked happy, (iii) pattern-masked fearful and (iv) pattern-masked happy. The order of the blocks was counterbalanced across subjects. Each run was 5 min and 14 s long.
Subjective awareness was assessed through post-scan interview sessions. Immediately after the fMRI scanning sessions, subjects were asked to describe what was presented on the screen during the experimental session. Next, the subjects were asked to comment on the emotional expressions of the faces. And finally, they were asked to report if they had seen any fearful or happy faces or any part of these expressions (e.g. smiles, wide eyes, etc.) during the fMRI scanning sessions. If a subject reported seeing even a single fearful or happy face, they were regarded as being subjectively aware of the target stimuli and thus were removed from the analysis.
After this post-scan interview, the participants were debriefed and told that there were in fact fearful or happy faces before each mask stimulus. With this knowledge, subjects were exposed to a total of 40 experimental blocks again (10 pattern-masked fearful, 10 pattern-masked happy, 10 face-masked fearful, 10 face-masked happy), and were asked to actively search for the masked faces. This post-experimental test was performed outside the MRI scanner using an LCD display with 60 Hz refresh rate that matched capabilities of the projector that was used during the scanning sessions, also verified using the photodiode-oscilloscope system (Tektronix TDS 2012). Subjects rated blocks instead of individual trials since these data could be more readily related to blocked stimulus presentations in the scanner. After each block, subjects were asked to report whether the masked faces were fearful or happy in a two alternative forced choice task. This allowed us to assess their objective awareness—the ability to correctly discriminate whether the masked faces were fearful or happy even without subjective awareness (Etkin et al., 2004; Whalen et al., 2004; Pessoa et al., 2006). Objective awareness was quantified based on signal detection theory by calculating a sensitivity index (d’) based upon the percentage of trials a masked stimulus was correctly identified when presented (hits) adjusted for the percentage of trials a masked stimulus was ‘identified’ when not presented (false alarms), using the following formula: [d’ = z-score (% hits) – z-score (% false alarms), with chance performance = 0 ± 1.74] (Whalen et al., 2004).
All subjects were scanned on a 3.0 Tesla Philips Intera Achieva Scanner (Philips Medical Systems, Bothell, WA) equipped with a SENSE birdcage head coil. Anatomical T1-weighted images were collected using a high-resolution 3D magnetization-prepared rapid gradient echo sequence, with 160 contiguous 1mm thick sagittal slices [echo time (TE) = 4.6 ms, repetition time (TR) = 9.8 ms, field of view (FOV) = 240 mm, flip angle = 8°, voxel size = 1 × 0.94 × 0.94 mm]. Functional images were acquired using echo-planar T2*-weighted imaging sequence. Each volume consisted of 36 interleaved 3 mm thick slices with 0.5 mm interslice gap (TE = 35 ms, TR = 2000 ms, FOV = 240 mm, flip angle = 90°, voxel size = 3 × 3 × 3.5 mm).
Anatomical and functional images were processed using Statistical Parametric Mapping software (SPM5, Wellcome Department of Imaging Neuroscience, London, UK). Raw functional data were preprocessed following standard procedures, starting with correcting for head movement. None of the subjects had head movement more than 1.5 mm in any direction. Functional images were then normalized to standard space using the Montreal Neurological Institute (MNI)-152 template. Spatial smoothing was applied to the normalized functional images using a Gaussian kernel of 6 mm full width at half maximum. By using a boxcar function convolved with the hemodynamic response function and covariates of no interests (a session mean, a linear trend for each run, and six movement parameters derived from realignment corrections), linear contrast maps [emotion (fearful, happy)] × [mask type (non-face pattern, face)] were generated for each subject. Contrast maps were then entered into a random effects model, which accounts for inter-subject variability and allows population based inferences to be drawn.
To assess the relationship between anxiety measures and amygdala blood oxygen level dependent (BOLD) signal increases to masked fearful vs happy faces, voxelwise correlation analyses on the contrast maps (face-masked fearful vs happy faces and pattern-masked fearful vs happy faces) were performed with STAI scores as a regressor. Separate voxelwise correlation analyses were performed for trait and state anxiety measures.
Given the current study’s focus on the amygdala, we imposed a significance threshold of P < 0.05 corrected for multiple comparisons over the amygdala volume (~4500 mm3, defined using the Automated Anatomical Labeling atlas; Maldjian et al., 2003), as determined by Monte Carlo simulations implemented in AlphaSim within AFNI software (Cox, 1996), a strategy we have implemented in previous studies (Kim et al., 2003; Johnstone et al., 2005; Davis et al., 2009). Based on the findings of previous backward masking studies (Whalen et al., 1998, 2004), we first sought to identify voxels in the amygdala that showed significantly increased BOLD signal to face-masked fearful vs happy faces. Then, we planned to use these voxels as a region of interest to examine the effects of using pattern masks on amygdala activity.
Post-scan interviews revealed that out of 27 subjects, five had seen at least one masked face during the fMRI experiment (i.e. subjective awareness). Of these five subjects, three had reported seeing masked faces in both pattern-mask and face-mask blocks, and two had reported seeing masked faces only in the pattern-mask blocks. Therefore, these five subjects were excluded from further analysis consistent with our previous study (Whalen et al., 1998) which also reported a subjective detection rate of ~20% of subjects.
Three out of 22 subjects demonstrated above chance performance (i.e. objective awareness; d’ > 1.74) in discriminating fearful and happy target faces on the two-alternative forced choice task. Of these three subjects, one showed above chance performance in the pattern-mask blocks, one in the face-mask blocks, and one showed above chance performance in both the face and pattern conditions. Since we have previously shown that post-scanning objective awareness does not impact brain activations observed in the previous passive viewing session (Whalen et al., 2004, see Supplementary data), these subjects were included in the initial analysis. There were no significant differences in the level of objective awareness between the face mask (0.024 ± 0.913) and pattern mask conditions (0.209 ± 0.688; t(21) = 1.26, P = 0.22).
Descriptive statistics for self-report measures were as follows: STAI-S = 33.73 ± 8.70; STAI-T = 35.32 ± 8.47; BDI = 2.95 ± 3.53. These results show that all scores for anxiety and depression were within the normal range.
Activation in the right amygdala [MNI 18, −3, −18; t(21) = 3.09, P < 0.05 corrected, cluster size = 297 mm3] was significantly increased to face-masked fearful vs happy faces (Figure 2A). No significant increase in amygdala activity was observed to pattern-masked fearful vs happy faces. However, the mean extracted values from the same voxels in the right amygdala showing the fearful > happy effect in the face-masked condition, revealed significantly decreased activation to pattern-masked fearful vs happy faces [t(21) = −2.48, P < 0.05; Figure 2B].
In the Supplementary data section we provide results in a separate group of 11 subjects who viewed blocks of neutral faces and pattern masks devoid of the fearful and happy target stimuli (Supplementary Figure 1). These data show that the pattern mask stimuli do not by themselves produce significant amygdala signal increases or decreases. Thus, the effect observed to pattern-masked fearful vs happy faces appears to be an active signal decrease in response to the fearful target stimuli (as opposed to a signal increase to happy faces).
No other amygdala voxel clusters were found in the pattern-masked fearful vs happy contrast. There were no significant differences in right amygdala activation between males and females [face-masked: t(20) = −0.05, P = 0.97, pattern-masked: t(20) = 0.03, P = 0.98]. The results remained unchanged when the three subjects who could objectively identify the target stimuli were removed from the analysis. Moreover, objective awareness (d’) to face and pattern mask conditions was not significantly correlated with right amygdala activity to face-masked (r = 0.09, P = 0.68) and pattern-masked fearful vs happy faces (r = 0.02, P = 0.92), respectively.
Voxelwise correlation analyses yielded no significant correlations between either state or trait anxiety scores and amygdala activity in response to either face- or pattern-masked fearful vs happy faces.
In this article, we demonstrated that the amygdala differentially responds to masked fearful and happy faces and that this discrimination was markedly different depending on the context within which the masked faces were presented. The current data replicate findings from previous backward masking studies showing increased amygdala activation to fearful faces when masked with neutral faces (Morris et al., 1998; Whalen et al., 1998; Rauch et al., 2000; Etkin et al., 2004; Pessoa et al., 2006; Williams et al., 2006). Furthermore, we have extended these findings by showing a selective decrease in amygdala activation to fearful faces when they were masked with non-face pattern images. The present data extend the results of our previous backward masking study (Whalen et al. 1998) in two ways: First, we demonstrated a significant increase in amygdala activation to face-masked fearful vs happy faces that were presented for 17 ms (compared to 33 ms in our original study), and second, we demonstrated this effect in a cohort consisting of male and female subjects, whereas our previous report studied only male subjects.
Our data from the pattern-masked condition highlight the fact that the amygdala is differentially activated by masked fearful vs happy faces, but the nature of this response is dependent on the type of mask stimulus used. This interactive effect of mask type and target face expression on amygdala activity was unexpected. Clearly, the basis of amygdala responses in the pattern-mask condition must begin with the fearful faces, but the observed signal decreases might reflect some interaction with other neural systems responding to the pattern mask. Similarly, amygdala signal increases observed in the neutral face mask condition may be influenced by both the fearful target face as well as the neutral face mask. However, the fact that we observed that the same area of the amygdala was responsive to both face- and pattern-masked fearful faces suggests that there are shared underlying neural processes involved in both conditions, which implies that the amygdala may be sensitive to masked fearful faces per se regardless of mask type.
Amygdala responses in the present experiment could be related to different proposed mechanisms of backward masking. One mechanism suggests that masking works via stimulus substitution (see Bachmann and Allik, 1976; Bachmann et al. 2005 for extensive discussion). By this account, the mask substitutes for the target stimulus at some level of neural representation and, thus, the first target stimulus never reaches the level of subjective awareness (Bachmann and Allik, 1976; Rolls and Tovee, 1994; Di Lollo et al., 2000). Such an account would predict similar neural responses to the emotional target stimulus per se regardless of the mask stimulus.
An alternative proposed mechanism is known as stimulus integration or amalgamation (Bachmann and Allik, 1976; Bachmann et al. 2005). By this account the target stimulus is amalgamated with the mask, perceived as a single object, and is therefore not reported. This account would suggest that neural responses to masked fearful faces should depend on the mask being a stimulus that can be interpreted differently based on the presence of these hidden targets (e.g. a neutral face).
Though neither of these theories necessarily implicates amygdala involvement, the differential amygdala responses observed in the face vs pattern mask condition could be consistent with the stimulus integration account, as amygdala signal increases were observed to masked fear only in the face-masked condition. This interpretation is consistent with a recent report showing that masked fearful faces can influence the interpretation of the face mask stimulus. Specifically, surprised faces were used as the mask stimulus and were interpreted more negatively when they were used to mask fearful faces compared to happy faces (Li et al., 2008). Future studies could seek to extend this effect to neutral face masks.
However, the stimulus integration account of the present effects is complicated by the fact that amygdala activity did discriminate between the fearful and happy conditions in the pattern mask condition. Specifically, we observed a decrease in amygdala activation to pattern-masked fearful vs happy faces, compared with the baseline level of activity (i.e. fixation blocks) supporting the notion that the amygdala is sensitive to the masked fearful face stimuli per se.
The observed BOLD signal decreases are, of course, not an unambiguous response pattern. We can say with certainty that amygdala activity discriminated between fear and happy without the benefit of a neutral face mask. One possibility is that the amygdala activation to pattern-masked fearful faces becomes actively suppressed, perhaps because this initial signal does not make sense in the non-face context. That is, the mismatch between the information that was being processed with awareness (pattern masks) and without awareness (fearful faces) may have led to the suppression of amygdala activation. Such an account accords with models of backward masking supposing that target and mask stimuli produce direct neural competition (Keysers and Perrett, 2002). It is not clear that BOLD signal decreases necessarily reflect diminished neuronal activity. For example, Maier and colleagues (Maier et al., 2008) have demonstrated that cortical BOLD signal decreases in area V1 dissociate from neuronal activity under certain psychological states (e.g. decreased BOLD but sustained neuronal activity was observed during perceptual suppression in monkeys). If this effect in cortex can be generalized to a subcortical structure like the amygdala, our data suggest the possibility that a subpopulation of neurons in the amygdala that are responsive to masked fearful faces may show sustained neuronal activity but exhibit decreased BOLD signal to pattern-masked fearful faces. If this phenomenon is related to the mismatch between the information from the mask and the target, future studies could examine the selective decrease in amygdala BOLD signal to pattern-masked fearful faces while manipulating the degree of congruency between the faces and the masks.
The observed effects are not likely due to any difference in the detectability of the fearful stimuli in the face vs pattern mask condition, since (i) subjectively aware subjects were excluded from the analyses, (ii) exclusion of subjects who were objectively aware did not change the results and (iii) the degree of objective awareness did not predict the strength of amygdala activity in either condition. We would concede though that since we deliberately chose to assess amygdala responses to masked stimuli in naïve subjects during passive viewing (rather than in subjects who are made aware of the presence of the masked faces and are instructed to actively search for the target faces during scanning; e.g. Pessoa et al., 2006), we cannot rule out the possibility that some level of awareness across both conditions could have impacted our results. We assume that our objective test of awareness following scanning is a reasonable metric for identifying which individual subjects were more likely to have been aware during the earlier naïve presentations and as noted above, these data were unrelated to amygdala responses to masked fearful faces.
Also, it should be noted that we did not observe significant correlations between amygdala activity to masked fearful faces and anxiety measures or the degree of objective awareness, though previous studies have observed such relationships (Etkin et al., 2004; Pessoa et al., 2006). This discrepancy may well be due to differences in the experimental designs (block vs event-related, passive viewing vs active task, inclusion of non-masked conditions), and is open to further scientific inquiry.
Taken together, the present data show that amygdala activity is influenced by the fearful target stimulus as well as the interaction of the fearful face with neutral face mask. More generally, the current findings show that implicit amygdala BOLD responses to crude representations of biologically relevant stimuli can interact with the explicit processing of contextual stimuli (e.g. mask stimuli, present study; additional task demands, Pessoa et al., 2006). In terms of amygdala function, the present backward masking data are consistent with other experimental techniques, namely binocular rivalry (Williams et al., 2004), continuous flash suppression (Jiang and He, 2006), chimerical faces (Morris et al., 2002) and low spatial frequency information (Vuilleumier et al., 2003) that also support a fundamental and automatic role for the amygdala in the assessment of biologically relevant predictive stimuli (LeDoux, 1996).