|Home | About | Journals | Submit | Contact Us | Français|
It has been hypothesized that the amygdala mediates the processing advantage of emotional items. In the present study, we employed functional magnetic resonance imaging (fMRI) to investigate how fear conditioning affected the visual processing of task-irrelevant faces. We hypothesized that faces previously paired with shock (threat faces) would more effectively vie for processing resources during conditions involving spatial competition. To investigate this question, following conditioning, participants performed a letter-detection task on an array of letters that was superimposed on task-irrelevant faces. 7 Attentional resources were manipulated by having participants perform an easy or a difficult search task. Our findings revealed that threat fearful faces evoked stronger responses in the amygdala and fusiform gyrus relative to safe fearful faces during low-load attentional conditions, but not during high-load conditions. Consistent with the increased processing of shock-paired stimuli during the low-load condition, such stimuli exhibited increased behavioral priming and fMRI repetition effects relative to unpaired faces during a subsequent implicit-memory task. Overall, our results suggest a competition model in which affective significance signals from the amygdala may constitute a key modulatory factor determining the neural fate of visual stimuli. In addition, it appears that such competitive advantage is only evident when sufficient processing resources are available to process the affective stimulus.
A popular view of visual processing suggests that multiple items compete for limited processing resources (Desimone and Duncan, 1995; Kastner and Ungerleider, 2000). Such competition can be biased by both bottom-up and top-down factors, such as stimulus salience and selective attention, respectively. Whereas directed attention to a stimulus facilitates its processing, both perception and the underlying neural responses of objects outside of the focus of attention are attenuated when attentional resources are consumed (Rees et al., 1997). At the same time, the perception of emotion-laden visual stimuli, such as facial expressions and emotional scenes, is thought to be prioritized, and to proceed under some conditions of inattention (Pessoa, 2005; Vuilleumier, 2005). In attempting to understand whether emotional perception depends on attention, investigators often focus on the amygdala, a subcortical structure that is believed to be important for the processing of biologically relevant stimuli (Davis and Whalen, 2001). Differential responses in the amygdala for unattended stimuli (Vuilleumier et al., 2001; Anderson et al., 2003), as well as for stimuli that subjects are potentially unaware of (Whalen et al., 1998; Whalen et al., 2004), are viewed as evidence that emotional perception is “automatic” (Öhman, 2002). On the contrary, other studies suggest that emotional perception is, in fact, dependent on attention (Pessoa et al., 2002a; Pessoa et al., 2005; Eimer et al., 2003; Holmes et al., 2003, Hsu and Pessoa, 2007; Silvert et al., 2007; Bishop et al., 2007) and other top-down factors (Ishai et al., 2004), as revealed by robust valence by attention interactions in the amygdala (and other brain regions). Because previous studies suggesting that emotional perception depends on attention have employed stimuli that are not strongly arousing (e.g., facial expressions of fear or anger), it is conceivable that the effects of attention (i.e., lack of “automaticity”) were due to the usage of “less arousing” emotional stimuli.
To address this possibility, in the present study, we investigated the role of visual attention during the processing of emotion-laden visual stimuli by initially pairing a subset of our emotional faces with mildly aversive electrical stimulation. This form of aversive conditioning is dependent on the amygdala in both non-human species and humans (Davis and Whalen, 2001; Phelps and LeDoux, 2005). During conditioning, a stimulus acquires affective significance (conditioned stimulus, CS+) when paired with an aversive stimulus (unconditioned stimulus, US). We reasoned that such pairing would potentiate the processing of shock-paired faces. For instance, emotional words are less likely to be missed under attentionally demanding conditions (Anderson and Phelps, 2001; Anderson, 2005). Collectively, a body of findings suggests that emotionally salient events are associated with enhanced perception (Vuilleumier, 2005). Overall, one of the goals of our experimental manipulation was to increase the affective significance of visual stimuli, such as facial expressions, commonly employed in studying the interaction between attention and emotional processing (Vuilleumier et al., 2001; Pessoa et al., 2002a; Anderson et al., 2003; Eimer et al., 2003; Williams et al., 2005; Bishop et al., 2007).
In the present study, we investigated how fear conditioning affected the visual processing of task-irrelevant faces. We hypothesized that stimuli with increased affective significance would have a competitive advantage relative to other stimuli during conditions involving spatial competition. Because of the role of the amygdala in emotional perception and the role of the fusiform gyrus in the processing of faces (Kanwisher et al., 1997; Haxby et al., 2001; Allison et al., 1999), we were particularly interested in evaluating responses evoked in these structures. We tested our hypothesis in two complementary ways, which involved a letter-detection task and a subsequent implicit-memory task. Following an initial conditioning phase, participants performed a letter-detection task on an array of letters that was superimposed on task-irrelevant faces (Fig. 1). Attentional resources were manipulated by having participants perform an easy or a difficult search task. We predicted that faces previously paired with shock would be potentiated, leading to increased resistance to the effects of attention. For instance, fearful faces that were previously paired with shock would be expected to evoke stronger responses relative to unpaired faces in the amygdala and fusiform gyrus during both easy and hard attentional conditions – i.e., fearful faces paired with shock would resist the effects of attention, which in some studies have eliminated emotion-related differential responses (Bishop et al., 2004; Pessoa et al., 2005; Mitchell et al., 2007; Silvert et al., 2007). Subsequent to the letter-detection task, participants performed an implicit-memory task during which behavioral and fMRI-related priming measures were assessed. We predicted that the initial aversive conditioning of faces would allow them to be better (implicitly) encoded in memory during the letter-detection task relative to faces not paired with shock, even though faces were task-irrelevant during this task. Again, we were particularly interested in probing responses evoked by the amygdala and fusiform gyrus. Overall, our design allowed us to test the hypothesis that affective significance would be linked to increased competitive advantage in two separate ways (i.e., via the letter-detection and implicit memory tasks).
Twenty-nine right-handed subjects (17 male), aged 18–34 years, were recruited. All subjects were in good health with no history of neurological or psychiatric disorders and gave written informed consent, as approved by the Institutional Review Boards of Brown University and Memorial Hospital of Rhode Island. Subjects had normal or corrected-to-normal vision. Eight subjects were excluded from data analysis: five subjects were excluded due to excessive head motion during scanning (> 2 voxels); one subject was excluded due to chance performance during both the easy (52% correct) and hard (48% correct) letter-detection trials; two subjects were excluded because of insufficient signal quality in the amygdala (see below).
Face stimuli were taken from the Karolinska Directed Emotional Faces (KDEF) (Lundqvist et al., 1998), the Ekman set (Ekman and Priesen, 1976), the Ishai-NIMH set (Ishai et al., 2004), and the Nimstim Face Stimulus Set (MacArthur Foundation Research Network on Early Experience and Brain Development). All faces were converted to black and white, and both contrast and brightness were adjusted to maintain consistency across the face sets. Our combined face set consisted of 144 identities (72 males and 72 females): 24 identities employed during the conditioning phase; 80 identities employed during the letter- detection task; 40 identities employed during the implicit memory task. In addition, 12 separate identities were used during “filler” blocks during the letter-detection task and were not included in data analyses (see below). Two emotional expressions were used (neutral, fearful) and, for each emotion, different identities were assigned randomly. All faces were presented at fixation, subtended 5.6° × 7.5° of visual angle, and were shown on a gray background. During the letter-detection task, a letter array (5° horizontally) was overlaid on the faces (Fig. 1). Letters were adjusted in size to approximately compensate for cortical magnification (Rovamo and Virsu, 1979). As shown in Figure 1, the letter array was displayed in red and contained one target letter (N or X) and five non-target letters (five O’s during the easy condition and H, K, M, W, and Z during the hard condition) arranged in a random order (Jenkins et al., 2002; Jenkins et al., 2005). A black fixation cross appeared in the middle of the letter array. Stimuli were presented through a liquid-crystal display projector (EPSON PowerElite 175C) onto a rear- projection screen located near the feet of the subject.
The experiment comprised three phases (Fig. 2A): conditioning (1 run), letter-detection (6 runs), and implicit memory (1 run). The experimental schedule of stimulus presentation, triggering of shock, and measurement of reaction time (RT) were controlled by Presentation software (Neurobehavioral Systems, CA). For all three tasks, subjects were asked to respond both rapidly and accurately.
The conditioning employed a 2 TYPE (safe, threat) by 2 EXPRESSION (neutral, fearful) experimental design (Fig. 2B). Subjects performed a gender discrimination task (male/female). Male or female fearful faces were designated as conditioned stimuli (CS+) and mild electrical stimulation served as the unconditioned stimulus (US). Thus, the conditioned stimuli involved a combination of both facial expression and face gender (e.g., only male fearful faces were paired with shock). The gender of the faces paired with shock was counterbalanced across subjects. Threat fearful faces were followed by a US according to a 50% partial reinforcement schedule in a delayed conditioning paradigm. Thus, threat faces refer to both neutral and fearful faces of the gender paired with shock; safe faces refer to both neutral and fearful faces of the gender not paired with shock. The shock was administered for 500 msec (at 50 Hz) to the distal phalange of the third and fourth fingers of the right hand through a stimulator (E13–22; Coulbourn instruments, PA), which included a grounded RF filter, and MR-compatible leads and electrodes (BIOPAC systems, CA). Before the experiment, subjects were instructed of the contingency rule, but were not informed about the probability of US delivery. The intensity of the “highly unpleasant but not painful” electric shock (range: 1.4 – 4.0 mA) was determined for each participant while the subject was prepared for MRI scanning.
A slow event-related structure was employed during the conditioning phase. On each trial, following a 400-ms fixation cross and a 100-ms blank screen, a face was presented for 2 s and followed by a 10-s inter-trial-interval (ITI). Thirty-four faces were presented in random order with the constraint that no more than two faces of the same type were presented successively: 6 threat fearful faces paired with shock (CS+ with US); 6 threat fearful faces without shock (CS+); 6 threat neutral faces; 6 safe fearful faces; 6 safe neutral faces; 4 habituation trials (see below). The US was delivered 1500 ms after the onset of a CS+ stimulus and co-terminated with the CS+. On each trial, subjects were asked to make a button response reporting whether the face was male or female. At the end of the run, subjects were asked to indicate how much they feared a US on a 4-point scale (from “not at all” to “highly fearful”).
During MRI data acquisition, skin conductance responses (SCRs) were recorded with the MP-150 system (BIOPAC systems, CA) and MRI-compatible Ag/AgCl electrodes placed on the distal phalange of the index and middle finger of the non-dominant (left) hand. SCR was amplified and sampled at 250 Hz and the analysis of SCR waveforms was conducted using AcqKnowledge software (BIOPAC systems, CA). Recorded SCR waveforms were detrended and smoothed with a median filter over 50 samples to filter out MRI-induced noise. To reduce novelty effects, the first four “habituation” trials, defined as the first trial occurrence of each trial type (2 TYPE × 2 EXPRESSION), were excluded from the analyses. Trials involving shock delivery were excluded from further analyses, too. On each trial, the SCR was calculated by subtracting a baseline (average signal between 0 and 1 s) from the peak amplitude during the 1–6 s time window following stimulus onset (Prokasy and Raskin, 1974).
Following the conditioning phase, participants performed a series of letter-detection runs, during which they were asked to detect a target letter (N or X) in a letter array that was superimposed on the faces. Potential targets occurred equally often, and were shown randomly at one of the 6 spatial locations. Participants were instructed to maintain fixation and respond indicating the identity of the target letter (N or X), while ignoring the underlying faces. Letter-detection trials began with a green fixation cross shown for 450 ms, followed by a 50-msec blank screen. A display containing the letter array and the faces was then presented for 200 ms (the short duration minimized the possibility of saccades during stimulus presentation). The ITI was 1,800 ms.
As shown in Figure 2C, the letter-detection task was blocked by TYPE (safe, threat) and attentional LOAD (easy, hard), with a 10-s fixation separating them. No shock was actually administered during these blocks. In addition, we also included “filler” blocks in which 1–2 shocks were administered. Filler blocks, which were otherwise identical to the main blocks, were included to minimize the extinction of conditioned responses; because they actually contained shocks, they were excluded from further data analysis. Each run contained 8 main blocks (2 easy/safe, 2 easy/threat, 2 hard/safe, 2 hard/threat) and one filler block. Each block type was cued by an initial 5-s instruction display and contained 10 letter-detection trials (5 neutral faces and 5 fearful faces shown in random order). The instruction display indicated the gender of the faces (male or female; i.e., safe or threat) and task difficulty (easy or hard).
At the end of each run, a short “booster” segment consisting of 5 trials was presented to the participants. Such segment followed the same structure of the conditioning phase and was employed to maintain fear conditioning (i.e., to avoid extinction). The segment was signaled by a 5-s instruction display. Faces were shown for 2 s and were followed by a 10-s ITI. Subjects indicated whether faces were male or female. Of the 5 trials (shown in random order), 4 were not paired with shock (safe/neutral, safe/fearful, threat/neutral, threat/fearful) and one was paired with shock (threat/fearful). Finally, subjects were asked to indicate how much they feared a US on a 4-point scale (from “not at all” to “highly fearful”).
In summary, the letter-detection runs had a hybrid structure: a blocked structure for attentional load and stimulus type, and an event-related structure for emotional expression within each block. Each event of interest was repeated 60 times. Because of our goal to test implicit memory (see below), different sets of 10 identities were employed for each of the 8 conditions (2 TYPE × 2 LOAD × 2 EXPRESSION) during the letter-detection phase. In addition, trials during the initial conditioning phase, filler blocks or booster segments utilized a different set of faces. Faces employed during letter-detection blocks were repeated a total of 6 times (once per run).
Subjects participated in a final experimental phase in which they performed an expression discrimination task and were asked to indicate whether faces were neutral or fearful. They were instructed that no shock delivery would occur during this final phase. Each trial begun with a green fixation cross lasting 450 ms, followed by a 50-ms blank screen, and a face stimulus shown for 100 ms. The ITI was 2,500 ms. Blocks containing 24 trials were separated from each other by 10-s a blank screen. The implicit-memory task employed 80 old faces (shown in random order), which were shown during the letter-detection phase and 40 new faces (10 safe neutral faces, 10 safe fearful faces, 10 threat neutral faces, 10 threat fearful faces; as before, safe/threat faces corresponded to male/female faces). Because one participant did not finish the final implicit memory task due to fatigue, related analyses included 20 participants.
One anatomical scan and 8 functional scans were acquired using a Siemens 1.5 T Magnetom scanner (Siemens Medical Systems, Erlangen, Germany) with a standard head coil. Structural images were acquired first with a high resolution MPRAGE anatomical sequence (TR = 1900 ms; TE = 4.15 ms; TI = 1100 ms; 1-mm isotropic voxel; 256-mm field of view). Next, blood oxygenation level-dependent (BOLD) contrast functional images were acquired with gradient-echo echo-planar T2*-weighted imaging. Each functional volume consisted of twenty-nine axial slices (TR = 2500 ms; TE = 38 ms; FA = 90°; field of view: 24 cm; 64 × 64 matrix; 3.75 m thickness; interleaved acquisition order).
Analysis of fMRI data was performed using AFNI tools (Cox, 1996), unless indicated otherwise. The first four functional volumes of each run were removed to account for equilibration effects. The following processing steps were applied: slice-time correction, motion correction, normalization to the Montreal Neurological Institute (MNI) space (using the BET and FLIRT tools from the FSL package, http://www.fmrib.ox.ac.uk/fsl/), Gaussian spatial smoothing (full width at half maximum: 6 mm), and intensity normalization (each voxel’s mean was set to 100). Each individual’s functional data were analyzed via standard multiple linear regression. The linear models included constant and linear terms (for each run) that served as covariates of no interest (these terms controlled for drifts of MR signal). A two-stage mixed-effects analysis was performed in which regression coefficients for each condition of interest (fixed factor, e.g., threat, fearful face during hard task) were tested across subjects (random factor) via t tests or repeated-measures ANOVAs. We employed a .005 statistical threshold (uncorrected) and a minimum cluster extent of 5 voxels (3 voxels for amygdala). To reduce the number of statistical tests, the above tests were performed on voxels that exhibited significant task-related activation relative to fixation (i.e., significant activation during any of the 8 conditions [2 TYPE × 2 LOAD × 2 EXPRESSION]; p < .05). Activation coordinates are reported using MNI coordinates.
The main focus of our data analysis concentrated on regions of interest (ROIs) and was performed as follows. The local-maximum voxel (at the group level) of the contrast threat fearful vs. safe fearful during letter trials (collapsed across load conditions) was used as the center of a 5-mm radius sphere. The time series of voxels significantly activated (p < .05) within the sphere were averaged and submitted to additional tests that probed for effects during the easy or hard conditions and during the implicit-memory task. We investigated the following ROIs (based on the average of k voxels): right fusiform gyrus (5 voxels; x = 45, y = −57, z = −15); right superior parietal lobule (13 voxels; x = 39, y = −54, z = 57); left/right middle frontal gyrus (15/8 voxels; x = −33, y = −3, z = 51; x = 45, y = 7, z = 48); left/right inferior frontal gyrus (15/18 voxels; x = −48, y = 15, z = 24; x = 48, y = 27, z = 30); anterior cingulate cortex (12 voxels; x = 6, y = 27, z = 33); and right amygdala (5 voxels; x = 27, y = −4, z = −15; note that for the amygdala, voxels superior to z = −11 were not included in the ROI).
Because of susceptibility artifacts in the amygdala, signal quality (Thulborn, 2000) was inspected in this region and participants with low signal-to-noise ratios were excluded (N = 2). To do so, the standard deviation (SD) of the functional time series was assessed. Participants with 10 or more voxels in the amygdala (defined based on each individual’s anatomy) with an SD exceeding 1.5% of the mean (i.e., ~32% of the time series contained deviations around the mean that exceeded 1.5%) were excluded from the analyses.
Our experiment involved three phases (Fig. 2A): 1) conditioning, 2) letter-detection task, and 3) implicit-memory task. During the conditioning phase, we employed a 2 TYPE (safe, threat) by 2 EXPRESSION (neutral, fearful) fear conditioning paradigm during which either male or female fearful faces were paired with a mildly aversive electrical shock (US). Conditioning thus produced two types of faces: threat fearful faces with high affective significance, in addition to safe neutral, safe fearful, and threat neutral faces, all of which had lower affective significance. During the letter-detection task, we manipulated attentional load by changing task difficulty. During the low-load (easy) condition, subjects performed a relatively easy detection task involving the detection of a singleton within an array of letters. During the high-load (hard) condition, the detection task was more demanding, and involved searching for the target letter among an array of non-uniform distractor letters. We reasoned that, whereas few attentional resources would be consumed in the low-load condition and leave spare processing capacity to process the task-irrelevant faces, a more significant portion of the resources would be utilized during the high-load condition, thereby leaving fewer processing resources for the faces (Jenkins et al., 2005). Finally, during the implicit-memory task, participants viewed both new and old faces and were asked to indicate their emotional expression. Behavioral priming and fMRI repetition effects were probed to investigate how both attentional load and affective significance modulated the depth of processing of the faces during the preceding letter-detection task.
To verify the effectiveness of the conditioning procedure, we submitted SCR measures during gender-decision trials to a 2 (PHASE: conditioning, letter-detection) × 2 (TYPE: safe, threat) × 2 (EXPRESSION: neutral, fearful) repeated-measures ANOVA (during the letter-detection phase, gender-decision trials occurred during booster trials at the end of the run). The main effects of TYPE and EXPRESSION were statistically significant [TYPE: F(1,20) = 13.92, p < .001, η2 = .41; EXPRESSION: F(1,20) = 17.97, p = .001, η2 = .47]. As shown in Figure 3, SCR evoked during threat fearful faces were stronger than during all the other conditions, consistent with a significant TYPE by EXPRESSION interaction [F(1, 20) = 10.17, p < .01, η2 = .34]. No significant main effect of PHASE was observed [F(1, 20) = .76, p = .39, η2 = .04], indicating that conditioning did not extinguish during the main letter-detection phase. Subsequent, separate simple-effect analyses revealed that the SCR evoked by CS+ faces (threat fearful) was significantly larger than those evoked by all CS- stimuli (threat neutral, safe neutral, and safe fearful) (all p’s < .05). In terms of fMRI, during the conditioning phase, threat fearful faces evoked stronger responses relative to safe fearful faces in the right amygdala (x = 29, y = −4, z = −17; p < .05), as revealed by the direct contrast of these conditions, consistent with previous imaging studies of conditioning (LaBar et al., 1998; Cheng et al., 2003).
The mean accuracy during easy and hard conditions was 90.0 ± 2.3% and 72.9 ± 2.2%, respectively. A 2 (TYPE: safe, threat) × 2 (LOAD: easy, hard) × 2 (EXPRESSION: neutral, fearful) repeated-measures ANOVA on accuracy revealed only a significant main effect of LOAD [F(1, 20) = 84.74, p < .001, η2 = .81], indicating that the manipulation of attentional load was successful.
A similar repeated-measures ANOVA on reaction time (RT) revealed a significant main effect of LOAD [F(1, 20) = 107.52, p < .001, η2 = .84], again attesting to the effectiveness of the load manipulation. The average median RTs for the easy and hard conditions were 625.33 ± 22.08 ms and 880.46 ± 32.35 ms, respectively. In addition, the ANOVA also revealed a significant TYPE by EXPRESSION interaction [F(1, 20) = 5.63, p < .05, η2 = .22], such that we observed an effect of EXPRESSION (i.e., slower RT for fearful relative to neutral) for threat faces, but not for safe faces. This interaction pattern, together with the lack of a significant three-way interaction, indicated that no significant RT differences were observed when threat fearful vs. safe fearful faces were compared.
The overall mean rating for how much participants feared an aversive shock was 2.50 ± .16 (standard error of the mean) on a 4-point scale (from “not at all” to “highly fearful”). Critically, a repeated-measures ANOVA on the ratings, which were gathered at the end of the conditioning phase (2.67 ± .16) and at the end of each letter-detection run (2.48 ± .17), showed no significant main effect of time [F(6,120) = 1.37, p = .23, η2 = .06], suggesting that participants maintained their aversive reaction toward the CS+ stimuli during the session. Thus, no evidence of fear extinction was observed according to this self-report scale.
We initially analyzed whole-brain fMRI data in terms of a 2 (TYPE: safe, threat) × 2 (LOAD: easy, hard) × 2 (EXPRESSION: neutral, fearful) repeated-measures ANOVA. A network of brain areas, including bilateral superior parietal lobule (SPL; left: x = −23, y = −58, z = 48; right: x = 27, y = −65, z = 48), bilateral middle frontal gyrus (MFG; left: x = −25, y = 5, z = 54; right: x = 30, y = 3, z = 54), bilateral inferior frontal gyrus (IFG; left: x = −45, y = 9, z = 27; right: x = 47, y = 6, z = 30), and anterior cingulate cortex (ACC; x = 1, y = 23, z = 35), exhibited a main effect of LOAD, with greater responses observed during hard relative to easy conditions. These regions are known to overlap with a fronto-parietal “attentional network”, which has been proposed to engage top-down attentional effects (Kastner and Ungerleider, 2000; Corbetta and Shulman, 2002; Pessoa et al., 2003). However, we did not observe voxels that exhibited a significant main effect of TYPE or EXPRESSION.
Because a central objective of the present study was to probe how affective significance interacted with attention, we proceeded to a more focused set of analyses. To do so, we probed a set of ROIs that exhibited significant differences between responses evoked during threat vs. safe faces (pooled across load conditions). Because differential responses were very weak or absent when neutral faces were viewed, the ROIs were defined via the contrast of threat vs. safe (pooled across load conditions) when fearful faces were shown. Significant differences were observed in all ROIs (see Methods) during the easy condition but not during the hard condition (Fig. 4, left). Indeed, a significant affective significance (safe/threat) by attention (easy/hard) interaction was observed in all ROIs, although for the amygdala the p value just missed significance (p=.057).
To further probe our results, we contrasted threat vs. safe faces for each level of attentional load and expression in a voxel-wise manner. As illustrated in Table 1, robust differences between threat and safe faces were observed only for fearful faces during easy trials. Differential responses were observed along visual cortex (including the fusiform gyrus), fronto-parietal regions (including the MFG and the ACC), and amygdala (Fig. 4, right). For the same fearful faces, stronger responses to threat vs. safe were not observed during hard trials, consistent with the ROI analysis above. It is noteworthy that no differential responses between threat neutral and safe neutral faces were observed during the easy task for critical regions involved in face/affective processing, such as the fusiform gyrus and amygdala, suggesting that the conditioning effect did not strongly generalize across male/female faces – findings that are consistent with the pattern of SCR results (Fig. 3).
Following the letter-detection task, subjects performed an implicit-memory task, which allowed us to gauge the degree of face processing during letter-detection trials by assessing priming effects. During the implicit-memory task, subjects viewed both new and old faces (the latter were shown six times during easy and hard letter-detection conditions) and performed a neutral/fearful expression task. In the context of the implicit-memory task, safe and threat faces refer to the gender of the face originally paired with shock (Fig. 1A). A 2 (TYPE: safe, threat) × 3 (EXPOSURE: old easy, old hard, new) repeated-measures ANOVA on median RTs (neutral and fearful faces were pooled together to increase statistical power) revealed a significant main effect of EXPOSURE [F(2, 38) = 16.60, p < .001, η2 = .47] and a significant interaction between TYPE and EXPOSURE [F(2, 38) = 4.21, p < .05, η2 = .18]. As illustrated in Fig. 5A, responses to old threat faces viewed during the easy letter-detection task exhibited the fastest RT, which was significantly faster than the corresponding new threat faces [t(19) = −5.38, p < .001], revealing a behavioral priming effect. In addition, the same old threat faces viewed during the easy letter-detection task exhibited faster RT compared to the corresponding old safe faces [t(19) = −1.99, p = .06], consistent with the increased processing of threat relative to safe faces during the easy letter-detection task.
The neuroimaging data of the implicit-memory task were analyzed by performing 2 (TYPE: safe, threat) × 3 (EXPOSURE: old easy, old hard, new) repeated-measures ANOVAs on the right amygdala and right fusiform gyrus ROIs. We considered these regions because of their central role in face processing and previous priming-related findings in the literature (Vuilleumier et al., 2005; Henson and Mouchlianitis, 2007). For the right amygdala ROI (Fig. 5B), we observed a significant interaction between TYPE and EXPOSURE [F(2, 38) = 3.95, p < .05, η2 = .17] without significant main effects. Subsequent, simple-effects analyses were employed to examine repetition and affective significance, much like with the RT data. Responses evoked by old threat faces viewed originally during the easy letter-detection task were weaker relative to the corresponding new faces [t(19) = −3.71, p< .01], revealing a repetition effect. In addition, the same old threat faces exhibited decreased evoked responses compared to corresponding old safe faces [t(19) = −2.80, p< .05], again consistent with increased processing of threat relative to safe faces during the preceding letter-detection task. The same overall pattern of results was observed in the right fusiform gyrus ROI analysis (Fig. 5C). A 2 × 3 ANOVA showed a significant TYPE by EXPOSURE statistical interaction [F(2, 38) = 5.92, p < .01, η2 = .24] in the absence of main effects. As in the case of the right amygdala, responses evoked by old threat faces viewed originally during the easy letter-detection task were weaker relative to the corresponding new faces [t(19) = −2.96, p< .01], consistent with a repetition effect; old threat faces also exhibited decreased evoked responses compared to corresponding old safe faces [t(19) = −2.51, p< .05], again indicating increased processing of threat relative to safe faces during the preceding letter-detection task. Collectively, these results suggest that affectively significant stimuli (CS+ faces) were more strongly encoded (leading to stronger suppression) than safe stimuli (CS−) during the letter-detection task, but that such differential encoding occurred only when the load of the letter-detection task was low (easy condition).
In the present study, we manipulated affective significance by pairing fearful faces of a specific gender with aversive electrical stimulation. Attention was manipulated by having participants perform an easy or a hard search task. Contrary to our predictions, our findings revealed that shock-paired stimuli were not more resistant to top-down attentional control. Specifically, whereas threat fearful faces evoked stronger responses in the amygdala and fusiform gyrus relative to safe fearful faces during the easy letter-detection task, no evidence of differential responses was observed during the hard condition. Consistent with the increased processing of shock-paired stimuli during the easy condition, such stimuli exhibited increased priming relative to unpaired faces during a subsequent implicit-memory task.
Because processing resources are limited, multiple stimuli compete with each other in the visual system (Grossberg, 1980; Desimone and Duncan, 1995; Knudsen, 2007). Multiple factors affect the competition, including top-down (goal-driven) and bottom-up (stimulus-driven) influences. Due to such limited processing capacity, the extent of attention paid to an attended target stimulus will have an impact on the extent of distractor processing. It is believed that, if the main task is highly demanding, few processing resources will be available to be deployed to process distractors. Consequently, the processing of distractors will be hindered and decreased relative to that during less demanding conditions (Lavie, 1995).
Emotional content is an additional dimension that can modulate visual processing. Indeed, emotional perception is believed to be prioritized (Pessoa, 2005; Phelps, 2006; Vuilleumier, 2005). Emotion-laden visual stimuli are processed more effectively and faster (Öhman et al., 2001; Ishai et al., 2004). During fast serial presentations leading to an “attentional blink”, emotional words are less likely to be missed than neutrals ones (Anderson, 2005). Emotional words are also better identified than neutral ones during a challenging two-choice perceptual task (Zeelenberg et al., 2006). In one study, Smith et al. (2006) showed that an aversively conditioned item (an initially neutral visual stimulus that had been paired with an aversive sound) impaired the detection of a subsequent target, suggesting that they capture attention and cause an attentional blink. Several investigators have proposed that such processing advantage is dependent on the amygdala, consistent with the findings by Anderson and Phelps (2001), who showed that while emotional words are less likely to be missed under attentionally demanding conditions, such competitive advantage is lost in patients with bilateral amygdala lesions. In the present study, we reasoned that, if the amygdala biases visual competition in favor of emotion-laden items, then stimuli that undergo fear conditioning should have a competitive advantage. Our prediction was based on strong extant evidence that fear conditioning is dependent on the amygdala in both non-human species and humans (Davis and Whalen, 2001). Note, however, that because our findings are correlational, while they are consistent with amygdala mediation, additional studies, likely involving patients with a compromised amygdala, would be required to establish a stronger amygdala role.
The potential competitive advantage conferred by affective significance was tested in our experiment via two tasks: letter-detection and implicit memory (see below). During the letter-detection task, threat fearful faces evoked stronger amygdala responses than safe fearful faces during easy trials, consistent with the role of the amygdala in the processing of affective information (Davis and Whalen, 2001). A similar pattern of results was observed in the fusiform gyrus, a region that has been shown many times to be driven by stimulus valence (e.g., Vuilleumier et al., 2001; Pessoa et al., 2002b; Palermo and Rhodes, 2007). Interestingly, several fronto-parietal regions also revealed greater evoked responses to threat fearful vs. safe fearful faces during the easy condition. Because of the central involvement of these regions (including SPL, MFG, and ACC) in attentional functions, and much less so in the processing of the stimuli used in our experiment (faces and letters) (Haxby et al., 2000; Vinckier et al., 2007), we hypothesize that additional attentional resources may have been required to perform the letter-detection task when the faces were threatening relative to when they were safe. If this reasoning is correct, to maintain a similar level of performance (both accuracy and RT) when task-irrelevant threat faces were shown relative to the level observed with safe faces, greater “attentional effort” was required.
Contrary to our expectation, differential responses for threat vs. safe faces were not observed during hard trials – in this context, it is worth reiterating that we observed a main effect of attentional load (hard > easy) during the letter-detection task, confirming the effectiveness of the attentional manipulation. Equivalent responses were observed in the amygdala, fusiform gyrus, in addition to fronto-parietal regions. Our findings are in line with recent studies that have explicitly investigated the role of attentional load on the processing of emotional items (Erthal et al., 2005; Pessoa et al., 2005; Bishop et al., 2007; Hsu and Pessoa, 2007). For example, Bishop and colleagues showed that both state and trait anxiety modulated the response to fearful-face distractors during low, but not high, perceptual load. The present study goes beyond the previous ones by showing that the requirement for attention occurs even for items with increased affective significance (via conditioning). Whereas our results do not preclude the possibility that stimuli with more potent affective significance will exhibit some resistance to attentional manipulations, together with the previous findings, they argue against stronger versions of the view that affective information is processed “automatically” (Öhman and Mineka, 2001; Dolan and Vuilleumier, 2003). Finally, note that optimized scanning protocols for the amygdala may be able to improve signal quality in this region and may change the observed pattern of results.
Stimulus repetition is often accompanied by decreased responses in ventral visual cortex and may underlie behavioral priming effects (Wiggs and Martin, 1998). Both repetition suppression (i.e., reduced fMRI responses due to stimulus repetition) and behavioral priming have been suggested to occur without attention to or awareness of the stimuli during the initial exposure phase (Parkin et al., 1990; Szymanski and MacLeod, 1996; Stone et al., 1998). Recent evidence has challenged this view, however, and suggests that repetition suppression is modulated by task demands (Ishai et al., 2004; Yi et al., 2004; Henson and Mouchlianitis, 2007); see also Jenkins et al. (2005).
In the present study, in a final experimental phase, participants viewed new and old faces – the latter were shown during the preceding letter-detection task and occurred during the easy or hard attentional load condition. Analysis of the RT data revealed a behavioral priming effect. Even though the faces were task-irrelevant during the letter-detection task, new faces exhibited longer RTs relative to old ones. Behavioral priming was modulated by face type (safe vs. threat) during the easy condition, but not during the hard condition. These behavioral effects are thus consistent with the idea that behavioral priming occurs in an implicit fashion, while at the same time revealing that priming is modulated by attention and affective significance. Analysis of the fMRI data revealed repetition effects for threat faces. Comparison of responses evoked by old relative to new faces shown during the easy condition revealed that the former evoked weaker responses in the amygdala and the fusiform gyrus. Such repetition effects were not observed for faces shown originally during the hard condition.
The present fMRI repetition effects are consistent with previous studies that showed stronger repetition suppression (i.e., larger decreases) for repeated fearful faces relative to neutral faces in the amygdala (Ishai et al., 2004; Strauss et al., 2005). In our study, during the easy letter-detection task, threat fearful faces exhibited stronger activation relative to safe fearful faces in the amygdala, which we interpret as a reflection of the increased affective significance of these faces acquired through aversive conditioning. Subsequently, when old threat faces were viewed during the implicit-memory task, faces originally presented during the easy condition displayed robust repetition suppression, while old safe faces did not. Collectively, these findings suggest that amygdala responses reflect both affective significance and depth of processing, which are both modulated by attention.
The amygdala is centrally involved in aversive conditioning (Davis and Whalen, 2001; Phelps and LeDoux, 2005) and is also important for the processing of emotional faces (Pessoa et al., 2002a; Pessoa and Ungerleider, 2004; Vuilleumier, 2005; Palermo and Rhodes, 2007). In the present study, we hypothesized that the affective significance (manipulated via aversive conditioning) of task-irrelevant faces would provide a competitive advantage to such stimuli under conditions of spatial competition. Indeed, threat fearful faces evoked stronger responses than safe fearful faces during the letter-detection task. In addition, analysis of behavioral and fMRI-related priming effects during a final implicit-memory task corroborated the notion that threat fearful faces were more strongly encoded during the preceding letter-detection task relative to safe fearful faces. Although such competitive advantage was evident for faces viewed during low-load attentional conditions, it was not observed during high-load conditions, indicating that even visual stimuli with increased affective significance are under the control of attention. Overall, our results suggest a competition model in which affective significance signals from the amygdala may constitute a key modulatory factor determining the neural fate of visual stimuli. In addition, it appears that such competitive advantage is only evident when sufficient processing resources are available to process the affective stimulus.
This study was supported by the National Institute of Mental Health (award 1R01 MH071589 to L.P.) and the Ittleson Foundation.
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.