|Home | About | Journals | Submit | Contact Us | Français|
The evidence for amygdala processing of emotional items outside the focus of attention is mixed. We hypothesized that differences in attentional demands may, at least in part, explain prior discrepancies. In the present study, attention was manipulated by parametrically varying the difficulty of a central task, allowing us to compare responses evoked by unattended emotion-laden faces while the attentional load of a central task was varied. Reduced responses to unattended emotional stimuli may also reflect an active suppression of amygdala responses during difficult non-emotional tasks (cognitive modulation). To explicitly assess cognitive modulation, an experimental condition was used in which subjects performed the central task without the presence of irrelevant emotional stimuli. Our findings revealed that amygdala responses were modulated by the focus of attention. Stronger responses were evoked during a sex task (when faces were attended) relative to a bar-orientation task (when faces were unattended). Critically, a valence effect was observed in the right amygdala during low attentional demand conditions, but not during medium or high demand conditions. Moreover, performing a difficult non-emotional task alone was associated with signal decreases in a network of brain regions, including the amygdala. Such robust decreases demonstrate that cognitive modulation comprises a powerful factor in determining amygdala responses. Collectively, our findings reveal that both attentional resources and cognitive modulation govern the fate of unattended fearful faces in the amygdala.
There is good evidence that the processing of emotion-laden information is prioritized: it is fast (Globisch et al., 1999) and it interferes with perception (Hartikainen et al., 2000; Tipples and Sharma, 2000). However, does it occur independently of attention? This question has been addressed by determining responses to emotional stimuli which are either attended or unattended as a result of manipulating spatial attention, object-based attention, or task instructions. The evidence for the processing of emotional items outside the focus of attention is mixed. Some studies have reported that amygdala responses are not modulated by attention (Vuilleumier et al., 2001; Anderson et al., 2003). Other studies, however, have found the opposite result (Pessoa et al., 2002; Eimer et al., 2003; Holmes et al., 2003; Bishop et al., 2004; Ishai et al., 2004; Krolak-Salmon et al., 2004). In fact, strong valence by attention interactions have been observed when attention is manipulated, such that differential responses to fear are not only reduced, but eliminated (e.g., Pessoa et al., 2002).
We hypothesized that differences in attentional demands may, in part, explain prior discrepancies. It has been suggested that when attention is not fully consumed, spare processing capacity is utilized for the processing of unattended items (Lavie, 1995). Thus, a critical variable in understanding the extent of unattended processing is the attentional load of a task. Indeed, studies revealing that attention modulates the processing of unattended emotional stimuli employed demanding competing tasks that may have largely consumed processing capacity. At the same time, studies that observed little or no effect of attention used less demanding tasks. In the present study, attention was manipulated by parametrically varying the difficulty of a central task, allowing us to compare responses evoked by unattended emotion-laden faces while the attentional load of a central task was varied.
Another factor that may determine amygdala activation relates to the type of non-emotional (“attended”) task employed. It has been suggested that cognitive and emotional systems interact in a reciprocal fashion (Drevets and Raichle, 1998; Mayberg et al., 1999). Thus, certain cognitive tasks may affect amygdala responses more than others, possibly in a suppressive manner. Hence, reduced responses to unattended fearful faces may reflect an active suppression of amygdala responses during difficult tasks that do not involve emotional stimuli – we refer to this potential contribution as cognitive modulation. In the present study, we addressed the concern of task-related differences by comparing responses evoked by unattended faces when subjects were engaged in the same task while attentional load was varied. Critically, to explicitly probe cognitive modulation, a condition was used in which subjects performed the central task without the presence of irrelevant emotional stimuli. By comparing activity during this condition relative to baseline, it was possible to isolate the effects of the main non-emotional task on amygdala activation. In particular, decreases of activation relative to fixation would be indicative of suppressive effects. In sum, the present design allowed us to probe both attentional and cognitive-modulation effects on amygdala responses during the viewing of emotional faces.
Twenty volunteers (7 females) aged 20-40 years participated in the study, which was approved by the Institutional Review Board of both Brown University and Memorial Hospital of Rhode Island. All subjects were in good health with no past history of psychiatric or neurological disease and gave informed consent. Subjects had normal or corrected-to-normal vision.
There were five experimental conditions: sex task, bar-orientation task at easy, medium, and hard difficulty levels, and the bars-only condition. During sex and bar-orientation conditions, central faces with neutral or fearful expression were presented centrally along with peripheral bars (Fig. 1). The experimental conditions were presented in a blocked fashion and were separated from each other by a fixation condition lasting approximately 15 s (14900 ms). For the sex and bar-orientation conditions, blocks lasted approximately 45 s (44700 ms) and contained an initial instruction screen and 14 trials (7 fearful faces and 7 neutral ones); blocks for the bars-only condition were shorter and lasted approximately 21 s (20860 ms) and contained an initial instruction screen and 6 trials. Individual trials within a block lasted 2980 ms. During trials of the sex and bar-orientation tasks, an initial green fixation cross was shown for 450 ms and followed by a 200-ms display containing a central neutral or fearful face (approximately 4 deg vertically) and two peripheral bars to the right and left of fixation (presented at 6 deg eccentricity). After this stimulus display, a white fixation cross was shown for 2330 ms. Subjects were instructed to respond both rapidly and accurately. Subjects were explicitly told that fixation should be maintained during the presentation of the main stimulus display. The brief presentation of the stimulus, as well as the symmetrical positioning of the bars to the left/right of fixation, were aimed at essentially eliminating the occurrence of deliberate saccades; indeed, the timing was insufficient to allow subjects to gaze to one side and then the other to successfully perform the bar-orientation task. During sex-task trials subjects indicated whether faces were male or female. During bar-orientation trials, subjects indicated whether the orientation of the bars was the same or not; for such trials, 50% were matches and 50% were non matches. For the easy, medium, and hard blocks, task difficulty was manipulated via a staircase procedure that adjusted the angular difference of the bars during non-match trials such that performance was maintained at the desired difficulty level. That such control of task difficulty was successful was evidenced by the performance levels obtained (averaged across participants): 92%, 84%, and 67%, for the easy, medium, and hard conditions, respectively. For the bars-only condition, only the right and left bars were shown, together with a central fixation cross, and task difficulty was targeted to be equivalent to that of the hard level of the bar-orientation condition (average across participants: 68%).
Each block of trials was cued by an instruction display that indicated the type of trial as well as the difficulty of the task for bar-orientation trials. Blocks contained trials in which face type (neutral or fearful) and bar orientations were randomly chosen, but visual stimuli were identical for both sex and bar-orientation conditions (in different random orders), such that only the focus of attention alternated between faces and bars. Overall our design was hybrid, containing a general block structure and an event-related structure (facial expression) within each block. Events of interest were repeated 63 times or more, depending on the total number of runs performed by each subject (range: 9-12).
Face stimuli were obtained from the Ekman set (Ekman and Friesen, 1976), a set recently developed by Ohman and colleagues (KDEF, Lundqvist, D., Flykt, A., and Ohman, A.; Karolinska Hospital, Stockholm, Sweden), as well as a set developed and validated by Alumit Ishai (Ishai et al., 2004) at NIMH (Bethesda, USA).
fMRI data were collected using a Siemens 1.5 Tesla scanner. Each scanning session began with the acquisition of a high-resolution MPRAGE anatomical sequence (TR = 1900 ms, TE = 4.15 ms, TI = 1100 ms, 1-mm isotropic voxels, 256 mm field of view). Gradient echo echo-planar images were acquired with a TE of 38 ms and a TR of 2980 ms. Each volume consisted of 37 axial slices with slice thickness of 3 mm and in-plane resolution of 3 mm × 3 mm.
fMRI data were analyzed using AFNI tools (Cox, 1996) (http://afni.nimh.nih.gov/afni), unless indicated otherwise. Initially, both anatomical and functional data were normalized to the standard space defined by the Montreal Neurological Institute by using the BET and FLIRT tools from the FSL package (http://www.fmrib.ox.ac.uk/fsl/). For the functional data, the first 3 volumes of each run were discarded to allow for equilibration effects. The remaining volumes were then spatially registered to the first functional volume (i.e., volume acquired closest in time to the particular subject's high-resolution anatomy). Next, each volume was spatially smoothed with an 8-mm Gaussian filter (FWHM). Each subject's data were then analyzed with standard multiple regression methods (Friston et al., 1995). The linear models included constant and linear terms (for each run) that served as covariates of no interest (these terms controlled for drifts of MR signal). We optimized our design to allow for adequate separation of responses to different trial types, which amounted to choosing an experimental sequence that minimized the standard error associated with the statistical test of interest (e.g., comparing two experimental conditions); see (Birn et al., 2002). Such “optimal” experimental sequence was obtained by randomly generating a large number (i.e., 105) of experimental sequences and choosing the best 12 (each sequence was used for a separate run).
The main goal of the present study was to determine the effects of attentional load and cognitive modulation on amygdala responses. However, we also performed a whole-brain voxel-wise analysis to investigate general task-related activations, as well as to further investigate amygdala responses. A standard two-stage mixed-effects analysis was performed. The first (fixed) level involved determining the regression coefficients of regressors of interest, which modeled the effects of each experimental condition and facial expression (e.g., fearful faces during the hard bar-orientation task). The second level treated subject as a random factor and tested for task-related differences via paired t tests. As random-effects analysis may be fairly conservative in the context of fMRI data (Worsley et al., 2002), we employed a threshold of p < 0.001 (uncorrected), although activations survived stricter thresholds (e.g., see Fig. 2).
For the region of interest (ROI) analysis, for every individual, a site in the amygdala was chosen based on the contrast of fearful vs. neutral faces when they were attended (i.e., during the sex task). This condition was employed as the selection criterion because the associated differential responses were also observed at the individual level; see Anderson et al. (2003) for a similar strategy. Because we smoothed individual data with an 8-mm filter, the regression coefficients estimated via linear regression were taken from the peak voxel of the above selection contrast as representative coefficients for the ROI. We then interrogated the ROI at the group level in a random-effects manner for effects of valence and attentional load during the bar-orientation task when faces were unattended by performing pre-planned paired t tests.
To determine “blocked” average time courses for the amygdala (Fig. 2, middle and bottom rows), we treated our experiment as a blocked design and averaged the responses for each of the five block types (sex task, easy/medium/hard bar-orientation task, and bars-only), and expressed responses in terms of percent increase relative to responses during fixation. Note that for the blocks involving faces, the order of the fearful and neutral faces were not fixed across blocks, but instead were randomized. Although responses due to fearful and neutral faces were thus mixed together, averaging was used to summarize blocked activity for the associated condition.
There were five main experimental conditions: sex task, easy, medium, and hard bar-orientation, and bars-only condition. The mean reaction time (RT) during the sex task was 672 ms, which was significantly faster than the mean RT of 772 ms during the bar-orientation task, and significantly faster than the mean RT of 761 ms during the bars-only condition (in both cases, p < 0.05). We also investigated the three levels of difficulty within the bar-orientation task in a separate repeated-measures analysis of variance. As expected, attentional load significantly affected RTs (p < 0.0001), with faster responses during the easy condition (mean: 725 ms), intermediate responses during the medium condition (mean: 776 ms), and slower responses during the hard condition (mean: 815 ms). No significant main effect of valence or load by valence interaction was observed (p > 0.05 in both cases).
Initially, we contrasted the sex task and the bar-orientation task (collapsed over load levels) to determine brain regions more strongly recruited by the two tasks. Stronger responses during the bar-orientation task were observed in the superior parietal lobule, anterior intraparietal sulcus, frontal eye field, as well as the middle occipital gyrus in visual cortex. The parieto-frontal regions largely overlapped with a set of regions recruited during attention tasks (Kastner and Ungerleider, 2000; Corbetta and Shulman, 2002; Pessoa et al., 2003), consistent with the fact that the bar-orientation task was, in general, more difficult than the sex task. In concordance with faces being task-relevant during the sex task, stronger responses during this task relative to the bar-orientation task were observed in the fusiform gyrus, superior temporal sulcus, orbitofrontal cortex, and amygdala (Fig. 2A); for a complete set of sites see Table 1. In particular, stronger responses evoked in the amygdala during the sex task relative to the bar-orientation task replicates our previous findings (Pessoa et al., 2002).
We investigated the effect of attentional load on amygdala activity by performing an ROI analysis (Methods). Fig. 3 plots average response strength during the bar-orientation task as a function of load (easy, medium, and hard) and the expression of the unattended face (neutral and fearful); for comparison, we also illustrate the responses during the sex task when fearful and neutral faces were attended (this contrast served as the selection criterion for the ROIs; see Methods). Amygdala responses evoked during the bar-orientation task were largely reduced relative to responses during the sex task. Pre-planned contrasts revealed that, for the right amygdala, responses to fearful and neutral faces differed significantly during the easy condition (p < 0.05). Critically, no such difference was observed during the medium or hard conditions. Thus, a valence effect was observed during the low-load bar-orientation task but was eliminated during the higher-load conditions. For the left amygdala, no significant differences were observed during the bar-orientation task. It should be noted that some of the observed signal changes were relatively small. Nevertheless, they were statistically reliable at the group level.
To understand the role of cognitive modulation, we compared responses evoked during the bars-only condition relative to fixation and determined regions with significant decreases of activation. Deactivations were observed in a network of brain regions (Table 2) that included several sites along the medial surface (ventromedial prefrontal cortex anteriorly; posterior cingulate cortex and precuneus, posteriorly), as well as angular gyrus, anterior insula, and amygdala (Fig. 2B). Interestingly, in the amygdala, the peak of such decrease was observed in a more ventral portion of the amygdala (left: z = −27; right: z = −29) relative to the peak of the contrast between sex task vs. bar-orientation task (left: z = −19; right: −21; compare Figs. 2A and B). Again, although some of the signal changes were small, they were statistically reliable. For instance, although activations during the bars-only task decreases relative to fixation by approximately 0.1%, such decrease in the ventral amygdala was very reliable (p < 10-4).
To further probe such deactivation, we determined the “blocked” average time course as a function of experimental condition, focusing on three amygdala sites: (1) the peak location of deactivation obtained from the group map of bars-only vs. fixation; (2) the peak location of the group map contrasting the sex task vs. bar-orientation task; (3) individual-based peak voxels obtained by the contrast of fearful vs. neutral faces during the sex task (i.e., in the same manner as done for the ROI analysis; see Fig. 2C); such participant-based selection was preferable because, for this contrast, there was large variability in the locations of activation across subjects. For the first site (bars-only vs. fixation), no conditions exhibited significant increases relative to fixation (Fig. 2E and H). For the second site (sex task vs. bar-orientation task), significant increases of activation were only observed during the sex task (Fig. 2D and G). Finally, for the third site (fearful vs. neutral faces), activation increases were observed for all conditions, except during the bars-only condition (Fig. 2F and I).
In the present study, we investigated the role of attention in the processing of emotion-laden stimuli by parametrically manipulating task difficulty. Consistent with our previous findings (Pessoa et al., 2002), amygdala responses were modulated by the focus of attention: stronger responses were evoked during the sex task (when faces were attended) relative to the bar-orientation task (when faces were unattended). For unattended faces, a valence effect was observed during low attentional demand conditions, but not during medium or high demand conditions. Our results also revealed that performing the difficult bar-orientation task alone (bars-only condition) was associated with signal decreases (relative to fixation) in a network of brain regions, including the amygdala.
In the present study, to investigate the effect of attentional resources, we parametrically varied the difficulty of a non-emotional task, which involved matching the orientation of two peripheral bars (bar-orientation task). Our reasoning was that such manipulation would allow us to vary the amount of distractor processing (Lavie, 1995), thereby revealing the potential contribution of attentional resources. Consistent with the notion that attentional load modulates amygdala processing, differential responses to fearful faces in the right amygdala were observed only during low-load conditions. These findings demonstrate that amygdala responses are modulated by an attentional manipulation even when the task is fixed and only difficulty changes. Thus, in the present case, task differences cannot account for the modulation of the responses.
Prior studies disagree on whether or not the processing of emotion-laden visual stimuli is modulated by attention. In one study, spatial attention was manipulated by having subjects fixate a central cue and match either two faces or two houses presented eccentrically (Vuilleumier et al., 2001). The contrast of fearful and neutral faces was not modulated by the focus of attention, consistent with the view that the processing of emotional items does not require attention. A second study investigated this question by manipulating object-based attention while leaving spatial attention constant (Anderson et al., 2003). “Double-exposure” images containing both (semi-transparent) faces and buildings were used and subjects were instructed to make either a male/female judgment (attend to faces) or an inside/outside judgment (attend to places). In the amygdala, similar responses were evoked for both attended and unattended fearful or neutral faces. A third study, however, found the opposite result (Pessoa et al., 2002). Like in the present study, spatial attention was manipulated by having subjects, on some trials, indicate whether a central face was male or female and, on other trials, whether two peripheral bars had the same orientation or not. The bar-orientation task was made very difficult in an attempt to consume most attentional resources, leaving little for the processing of the unattended faces. During the sex task, fearful faces evoked stronger activity than neutral ones in a network of brain regions including the amygdala. Critically, such differential activation was not observed when subjects performed the difficult bar-orientation task.
We suggest that present and past findings can be integrated once the concept of attentional load is considered, as hypothesized in the introduction. Accordingly, attentional effects on amygdala signals are most robustly observed when resources are largely consumed. In this context, it is worth noting that in the study by Anderson et al. (2003) in which amygdala responses to attended and unattended fearful faces were the same, responses to unattended disgust faces were, paradoxically, increased. Thus, it appears that during conditions of relative inattention, only coarse affective properties would be registered, such as overall valence or stimulus arousal.
It is important to note that, in the present study, non-zero responses were observed during all levels of attentional load (see Fig. 3). This situation is unlike our previous study in which amygdala responses appeared to be largely eliminated during the unattended condition (Pessoa et al., 2002). However, one important difference between the two studies is that, in our previous study, both bars were located above fixation, possibly making the task even harder. In fact, average RTs during the present hard-orientation task were 190 ms faster than previous average RTs. Moreover, shifting attention to the periphery as in the previous study may be, in general, linked with a more complete elimination of resources available to process central unattended stimuli.
The present design also allowed us to probe the contributions of general task-related factors to amygdala activation. To do so, we compared responses evoked during the bars-only condition relative to fixation. Signal decreases were observed in a network of brain regions, including the precuneus, angular gyrus, posterior cingulate cortex, anterior insula, ventromedial prefrontal cortex, as well as amygdala. These regions strongly overlap with the “resting state” network described by Gusnard and Raichle, which is observed as a deactivation when goal-directed or “active” conditions are compared to low-level baselines (Gusnard and Raichle, 2001; Raichle et al., 2001), and involves the ventro- and dorsomedial prefrontal cortex, posterior medial cortices (posterior cingulate, precuneus, and retrosplenial cortices), and posterior lateral cortices.
It has been suggested that cognitive and emotional systems engage in mutually suppressive interactions such that when either system is recruited, the other will be relatively suppressed (Drevets and Raichle, 1998; Mayberg et al., 1999). Thus, during an emotional situation, limbic regions would inhibit cognitive centers, thereby, at times, compromising cognitive processing (Gray, 2001; Gray et al., 2002). Conversely, limbic regions would be inhibited during the performance of demanding cognitive tasks, which would constitute a form of emotional blunting during cognitive conditions. The present deactivation of the amygdala, as well as ventromedial and posterior cingulate, is consistent with the idea of the deactivation of emotion systems during cognitive processing. Although our bar-orientation task is perhaps more perceptual in nature than cognitive, because it did not involve emotional stimuli, we interpret our results as a form of “cognitive modulation”.
In the amygdala, we observed effects of valence during the sex-condition and during the easy bar-orientation task, as well as decreases of activation during the bars-only task. Based on these results, we suggest that the fate of unattended fearful faces is determined by both attentional resources and cognitive modulation. On the one hand, paying attention to faces was associated with increased fMRI responses, especially in the dorsal amygdala (Fig. 2A). On the other hand, performing a challenging task that did not involve emotional stimuli led to decreased activation in a more ventral portion of the amygdala (Fig. 2B). Importantly, the blocked activity associated in the more ventral site never exhibited increases relative to fixation, not even during the sex task (Fig. 2E; compare with Figs. 2D and F). Furthermore, consideration of the average blocked activity in Figs. 2D-F suggests that cognitive modulation occurred at both dorsal and ventral sites, as evidenced by signal decreases during the bars-only condition in all cases. Thus, cognitive modulation constitutes an important factor in determining amygdala activation. Interestingly, for dorsal sites (Figs. 2D and F), the effect of cognitive modulation and of viewing faces combined additively in determining fMRI activation. For these sites, signals during the hard bar-orientation task in which faces were unattended were nearly identical to the response during the sex task plus the (negative) response during the bars-only task in which the difficulty was also hard. Thus, amygdala responses during the hard bar-orientation task could be obtained by subtracting the magnitude of the bars-only response from sex-task responses.
The present dorsal/ventral distinction is interesting in view of the suggestion by Whalen and colleagues that the dorsal amygdala might be more directly involved with arousal and the ventral amygdala might be more important for the processing of valence (Kim et al., 2003). In this scenario, the suppression of more inferior portions of the amygdala would be consistent with a stronger modulation of valence processing during a non-emotional task.
While cognitive modulation is an important factor in explaining amygdala activation, it is not sufficient. During unattended conditions, a valence effect was observed only during the easy condition. Thus, in general, the fate of unattended fearful faces in the amygdala is determined by both attentional resources and cognitive modulation. We suggest, however, that a third factor should also be considered when accounting for amygdala activation. Ambiguity has been shown to influence amygdala responses (Whalen, 1998) and it is likely that during conditions of inattention, a stimulus would be more ambiguous. Thus, although the valence-related component of the stimulus would be expected to be less effective when unattended, ambiguity would increase, possibly leading to increased fMRI responses. This scenario is consistent with the present findings as the responses to neutral faces in the right amygdala during medium- and high-load conditions were approximately the same as responses to a fearful face during the low-load condition (see Fig. 3). As stated, such apparently paradoxical increases were also observed by Anderson et al. (2003) during the processing of unattended disgusted faces. In summary, a more comprehensive understanding of amygdala processing will necessitate the elucidation of how the interplay of several factors – valence, attention, cognitive task, and ambiguity, among others – sculpts the response profile within this complex structure.
We thank Mrim Boutla and Leticia Oliveira for insightful comments on the manuscript and Alumit Ishai for making available a set of emotional faces. This work was supported by NIH grants 1R01 MH071589-01, 1P20 RR16457-03, and by the Ittleson Foundation.
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.