|Home | About | Journals | Submit | Contact Us | Français|
Often memory for emotionally arousing items is enhanced relative to neutral items within complex visual scenes, but this enhancement can come at the expense of memory for peripheral background information. This ‘trade-off’ effect has been elicited by a range of stimulus valence and arousal levels, yet the magnitude of the effect has been shown to vary with these factors. Using fMRI, this study investigated the neural mechanisms underlying this selective memory for emotional scenes. Further, we examined how these processes are affected by stimulus dimensions of arousal and valence. The trade-off effect in memory occurred for low to high arousal positive and negative scenes. There was a core emotional memory network associated with the trade-off among all the emotional scene types, however there were additional regions that were uniquely associated with the trade-off for each individual scene type. These results suggest that there is a common network of regions associated with the emotional memory tradeoff effect, but that valence and arousal also independently affect the neural activity underlying the effect.
Researchers often have found that, relative to neutral information, emotional information can attract attention more rapidly and also enhance memory strength. This prioritization of mental resources toward processing emotional information can lead to enhancements in memory for emotional information, and for its associated context [reviewed by Hamann, 2001; Levine & Edelstein, 2009; e.g., “flashbulb” memories, (Brown & Kulik, 1977), emotional words, (D’Argembeau & Van der Linden, 2004; Doerksen & Shimamura, 2001)]. However, it also can come at a cost for attending to and later remembering neutral peripheral or contextual information in visual images containing emotional items (Burke, Heuer, & Reisberg, 1992; Christianson, Loftus, Hoffman, & Loftus, 1991; Kensinger, Garoff-Eaton, & Schacter, 2007a; Kensinger, Piguet, Krendl, & Corkin, 2005) (reviewed by (Reisberg & Heuer, 2004), but see (Heuer & Reisberg, 1990; Libkuman, Nichols-Whitehead, Griffith, & Thomas, 1999) for circumstances of item and contextual memory enhancement). Within complex visual scenes, emotion-induced memory trade-off effect describes enhanced memory for emotional relative to neutral items that is accompanied by a decrement in memory for the backgrounds that are associated with those emotional items; memory for the background context is “traded off,” in favor of memory for the emotional items.
Behavioral evidence has begun to elaborate upon the specific characteristics of emotional scenes that lead to a trade-off in memory. The magnitude of this effect has been shown to vary with stimulus arousal level (how calming/soothing or exciting/agitating) [(Burke et al., 1992; Kensinger et al., 2007a; Libkuman, Stabler, & Otani, 2004) see (Russell, 1980) for discussion of these dimensions]. Previous studies of memory for emotionally arousing visual stimuli have shown a pattern of neural activity during encoding that is related to subsequently enhanced memory for emotional stimuli and poor memory for contextual details or for perceptually peripheral information (Kensinger & Schacter, 2006a; Mather et al., 2006). For example, a study by Mather et al. (2006) found that emotion did not uniformly enhance memory; although participants had better memory for emotionally arousing pictures than neutral, their spatial source memory for the arousing pictures was impaired. These behavioral patterns corresponded with greater activation at encoding within temporo-occipital visual processing regions for the high arousal than lower arousal pictures. The authors inferred from these results that encoding-related coupling of item and source information is disrupted for high arousal emotional items because of the focus of visual attention upon those stimuli; in other words, participants devoted their mental resources toward the processing of the emotionally arousing item and not toward other features of its presentation, such as its location on the computer screen (see (Mather, 2007) for further discussion). Further evidence to support this conclusion was gathered in a series of studies revealing that amygdala activation at encoding was related to subsequent memory for images of emotional items and the precise visual details of those items, but was not related to memory for information not directly tied to the perceptual nature of the emotional item, such as the semantic task performed with the item during the encoding phase (Kensinger & Schacter, 2006a; Kensinger, Garoff-Eaton, & Schacter, 2007c).
Although this research is often presented as demonstrating the disruptive effects of ‘emotional arousal’ on memory, the effects of negative arousing information have been studied to a much greater extent than the effects of positive arousing information, leaving some question about whether it is the arousal level or the negative valence of the stimuli that yields the effects. Some studies have shown that negative information leads to attention narrowing while positive information leads to attention broadening (Levine & Bluck, 2004; Libkuman et al., 2004). By this account, only negative stimuli should elicit memory trade-offs, while positive stimuli should not yield such effects. Yet in some cases, positive and negative information have both led to memory trade-offs [(Mather, Gorlick, & Nesmith, 2009; Waring & Kensinger, 2009); reviewed by (Reisberg & Heuer, 2004; Levine & Edelstein, 2009)], however the possibility that the dimensions of valence (how positive or negative) and arousal differently affect the underlying neural mechanisms of the trade-off has not been investigated.
The present study more directly examined the processes underlying the selective memory for the emotional item in a scene, examining whether an encoding-related mechanism tied to changes in recruitment of emotion-processing regions and visual detail-processing regions underlies the trade-off in emotional scene memory. Specifically, this study probed how the dimensions of valence and arousal affect the neural processes predicting a trade-off in memory, and how these differ from those predicting successful memory for the item and background of the scene. We hypothesized that the trade-off effect would arise in large part as a result of the brain s response specifically to the emotionally salient features within a scene. Amygdala activation is frequently cited as a key part of the neural response to viewing stimuli with emotional salience, and also strongly associated with more accurate subsequent memory for emotional information (Adolphs, Tranel, & Buchanan, 2005; Adolphs, Denburg, & Tranel, 2001; Hamann, 2001; Hamann, Ely, Grafton, & Kilts, 1999; Kensinger & Schacter, 2008). Moreover, amygdala activation does not guarantee good memory for all episodic details (Kensinger & Schacter, 2006a; Kensinger et al., 2007c), and so it seemed likely that engagement of the amygdala during encoding could yield a trade-off in scene memory. Beyond the amygdala, investigations into the neural basis of emotional memory have shown that portions of the prefrontal cortex (PFC), including medial and inferior frontal regions, also evidence response patterns associated with successful subsequent memory for emotionally arousing stimuli (Dolcos, LaBar, & Cabeza, 2004; Kensinger & Schacter, 2006a, 2008). Additionally, research investigating the role of visual processing for emotionally arousing images has revealed activation within portions of the fusiform gyrus, in addition to amygdala activation (Duncan & Barrett, 2007; Mather et al., 2006; Vuilleumier, Armony, Driver, & Dolan, 2001). Although these regions have been implicated in good memory for emotionally arousing items, we hypothesized that the same areas of the brain that are responsive to emotionally arousing stimuli could also be associated with a trade-off in scene memory because the increased attention and memory resources allocated to the emotional item in a complex scene might come at the cost of such resources being devoted toward contextual processing. Such a trade-off could occur at an attentional level, with visual attention disproportionately directed to the emotional item, or at a more elaborative or conceptual level of processing, with enhanced elaboration of the emotionally arousing item coming at the cost of deep processing of the context.
Researchers have generally assumed that the effects of attention allocation are attributable to the arousing nature of the emotional component of the scene; this assumption dates back to Easterbrook’s (1959) proposal that arousal causes visual attention to be narrowed on to arousing elements of scenes, thereby restricting the likelihood that the surrounding information will become encoded in memory [see review by (Reisberg & Heuer, 2004)]. However, this is difficult to validate because it is impossible to know why trade-offs might sometimes be exaggerated for scenes with high-arousal elements as compared to those with lower-arousal elements: Is it because the processes triggering the trade-off differ in the two instances, or because the high-arousal scenes elicit more of the same types of processes that yield a trade-off for the lower-arousal scenes? This is a particularly important question to address, because although the trade-off is sometimes exaggerated when the emotional element within the scene is high in arousal, the trade-off can also occur even when information is lower in arousal (Waring & Kensinger, 2009). Though at first glance this pattern could seem inconsistent with the hypothesized importance of arousal in evoking a trade-off, it is also possible that the processes leading to the trade-off are different when information is high in arousal rather than lower in arousal. Similar ambiguities in interpretation can result when examining whether arousal-related effects are influenced by valence; even though trade-offs can occur for both positive and negative stimuli (Reisberg & Heuer, 2004; Waring & Kensinger, 2009), this does not mean that the same neural processes support the trade-off in the two instances. Neuroimaging provides a way to gain leverage on these essential questions. By comparing the processes that lead to a trade-off between scenes of differing valence and arousal levels, this research can yield important information about how the affective characteristics of an experience lead to memory trade-offs.
The present study used fMRI to examine whether the way in which mental resources were allocated during encoding would predict when the trade-off in emotional scene memory would arise. We measured neural responses to images of scenes containing emotional or neutral items, each viewed in the context of a neutral background. Participants then took a recognition memory test outside the scanner where the item and background components from scenes were presented independently. This design allowed us to investigate how the valence or arousal characteristics of the item within the scene are related to the neural processes supporting the trade-off effect and how the regions associated with the trade-off differed from those corresponding with successful memory for both the item and background of the scene.
Participants included nineteen young adults (7 men, 12 women; mean age = 24.05 years, range 19 to 36 years). Participants had between 14 and 20 years of education (mean = 16.7). All were right handed, native English speakers with normal or corrected to normal vision. Participants were screened for history of psychiatric or neurological disorder and for treatment with centrally-acting medication. At the time of the study, participants scored within the normal range on measures of depression [Beck Depression Inventory mean = 1.44, (Beck & Steer, 1993)] and anxiety [Beck Anxiety Inventory mean = 3.12, (Beck, Epstein, Brown, & Steer, 1988)]. Written informed consent was obtained from all participants prior to the study in accordance with the study protocol approved by the Boston College and Massachusetts General Hospital Institutional Review Boards, and the study was performed in accordance with the standards of the 1964 Declaration of Helsinki. One participant was excluded from all neuroimaging analyses due to problems extracting a reliable signal (attributed to scanner spiking), leaving 18 participants included in imaging analyses; all participants are included in behavioral analyses.
The stimulus set included 272 emotional items, 136 neutral items, and 300 backgrounds. Scenes included an emotional or neutral item placed onto a background. The stimulus set was comprised of items and scenes from prior studies (Kensinger et al., 2007a, 2006, 2007b), those gathered from photo clipart packages (Hemera Technologies Inc., 2002, Canada), and from the International Affective Picture System [IAPS (Lang, Bradley, & Cuthbert, 1997)]. We binned items by valence into groups of positive, negative, and neutral items. For positive and negative items, arousal level was rated. Items and scenes from prior studies had been previously rated for stimulus arousal level by young and older adults. An additional 10 young and 6 older adults rated new items for arousal level using a 5-point scale, with low numbers indicating soothing or subduing items and high numbers signifying exciting or agitating items. Positive and negative items were further subdivided into higher and lower arousal item groups using a median split (pos. median arousal = 2.39, neg. median arousal = 3.86), resulting in 68 positive lower-arousal (m=1.85), 68 positive higher-arousal (m=2.96), 68 negative-lower arousal (m=3.24), and 68 negative-higher arousal (m=4.30) items (means are average of previous and new ratings). All neutral items were judged to be low in arousal (m = 2.24). The higher arousal items were rated significantly more arousing than the lower arousal items for each valence (t(270)=10.74, p<.0005). Items with arousal ratings falling exactly on the median split value, or ranked two items above or below that value were excluded from use. Scenes were visually matched by experimenters for size and location of foreground item, and were carefully constructed with strong consideration for the plausibility of each item in the context of the background with which it was paired (e.g. a chipmunk would not be paired with a dining room). Item and background congruency within a scene were assessed by a separate group of 6 raters, showing no significant difference in congruency between the 5 scene types (F(4,471)=1.14, p=.34), with scenes showing moderate levels of congruency (m= 3.28, SD=1.26, on a scale of 1–5); participants of the current study were not instructed to evaluate these factors.
Each participant saw 150 composite scenes (e.g., a snake by a river. See Fig. 1A), with each scene incorporating an item (60 positive, 60 negative, 30 neutral) in the context of a neutral background [backgrounds taken from (Kensinger et al., 2007a), and supplemented from IAPS (Lang et al., 1997) and clipart packages (Hemera Technologies Inc., 2002, Canada)] to create composite scenes. The valence of the item placed onto each of the backgrounds was varied across participants (i.e., one participant saw a snake [negative item] by a river, another saw a kitten [positive item] by a river, another saw a chipmunk [neutral item] by a river; see Fig. 1A) to isolate effects attributed to emotionality of items, and to avoid any confounds related to background of the scenes. Note that each participant saw a particular item or background only once; the valence of the item paired with each background was varied across participants. Half of the positive items and half of the negative items that participants viewed were higher arousal (below the median for that valence type), and the remaining half were lower arousal. Items were selected so that, across emotional valence types, they did not differ in size or category membership (e.g., people, animals, buildings). We evaluated the frequency of people, animals, or buildings appearing within the 5 types of scenes (positive lower-arousal, positive higher-arousal, negative lower-arousal, negative higher-arousal, neutral) and found there were no significant differences in the frequency of their inclusion (all F(4, 325)<1.32, p>.11).
The order in which scenes were presented within study lists was varied across participants to avoid order effects (see Fig. 1B). Each list contained 30 lower arousal positive items (e.g., flowers, kitten, board game), 30 higher arousal positive items (e.g., piñata, wedding dress, candy), 30 lower arousal negative items (e.g., toilet plunger, worms, casket), 30 higher arousal negative items (e.g., tarantula, bloody knife, roadkill), and 30 neutral items (e.g., microwave, postage stamp, potted plant), with each item displayed on a neutral background. The study lists were matched for equivalent ratings of arousal level (Fs(3,119)<.48). Composite scenes from the study sessions were separated into their isolated item and background components and each shown independently in a later recognition memory test (e.g. a kitten shown independently from a river background; see Fig. 1C). Scene components were counterbalanced across participants for studied/novel status at test.
Participants were told that they would see a series of photographic scenes presented inside the scanner, and they were asked to rate each scene according to whether they preferred to approach or retreat from the scene. Using a 1–3 scale, they chose the corresponding key on a button-box indicating their response, where 1= approach, 2= stay at present location, and 3= move away from the scene. Everyone had the opportunity to practice this task prior to entering the scanner. This encoding task was specifically chosen because in prior studies it had been shown to produce a reliable trade-off effect in young adults (Kensinger, Gutchess, & Schacter, 2007; Waring & Kensinger, 2009). Inside the scanner, scenes were presented for 3 sec., followed by a fixation cross (+) for an additional second, so participants had 4 sec. to view each scene and make a response before the program automatically advanced. This was enough time for participants to respond to each scene. Task instructions did not encourage participants to consider speed when making their responses and scenes were all presented for 3 sec., regardless of reaction time in responding. Following the approach or retreat decision and 1 sec. fixation cross, another fixation cross was presented at variable inter-stimulus interval (ISI) durations (mean ISI = 4 sec.; range = 2–16 sec.) to provide the jitter required to isolate the hemodynamic response to each stimulus necessary in an event-related design (Dale, 1999). The encoding task was divided into 3 functional runs of 5 min. each. The studied stimulus set was counterbalanced across participants during the scanning session, and scenes were presented in a pseudorandom order for each participant, as determined by the program optseq (written by Doug Greve), to optimize jittering within the fMRI environment.
Outside of the scanner and approximately 10 min. after viewing the scenes, participants took a surprise recognition memory test. Items and backgrounds from the composite scenes were presented independently from one another and intermixed with novel items and backgrounds. Test items included 150 ‘old’ items (30 lower arousal positive, 30 higher arousal positive, 30 lower arousal negative, 30 higher arousal negative, and 30 neutral) and 150 ‘old’ backgrounds (30 had been presented with a lower arousal positive item, 30 with a higher arousal positive item, 30 with a lower arousal negative item, 30 with a higher arousal negative item, and 30 with a neutral item), as well as 258 ‘new’ items (152 emotional [38 lower arousal positive, 38 higher arousal positive, 38 lower arousal negative, 38 higher arousal negative] and 106 neutral1) and 150 ‘new’ backgrounds (by definition, all neutral), for a combined total of 708 items and backgrounds. The orientation (i.e. horizontal or vertical) of each ‘old’ item presented at test was set to match its orientation in the studied scene; items were never rotated for presentation at retrieval. Participants had 3 sec. to view each test item or background component and 7 sec. to select the appropriate key to indicate if it was a component of one of the scenes viewed within the scanner (‘old’) or had not been seen previously within a scene (‘new’). There was a short practice test given before beginning the actual test to assure participants fully understood the meaning of “old” and “new” scene components.
Images were acquired on a 1.5T Siemens Avanto MRI scanner. Detailed anatomical data were collected using a multi-planar, rapidly acquired gradient-echo (MP-RAGE) sequence. Functional images were acquired using a T2*-weighted echo-planar imaging sequence (TR=2000msec, TE= 40msec, FOV=200mm, flip angle = 90°). Twenty-six axial-oblique slices (thickness=3.2mm, skip factor =0.6mm), aligned in a plane along the anterior/posterior commisure line, were acquired in an interleaved fashion.
All pre-processing and data analysis was conducted in SPM5 (Wellcome Department of Cognitive Neurology). Standard preprocessing was performed on functional data, including slice-timing, rigid body motion correction, normalization to the Montreal Neurological Institute (MNI) template (resampling at 2mm cubic voxels), and spatial smoothing (using a 7.6mm full-width half-maximum isotropic Gaussian kernel).
An event-related analysis was conducted on a voxel-by-voxel basis for each participant, with all instances of a particular event type modeled through convolution with a canonical hemodynamic response function. Analyses were conducted using a subsequent memory design, modeling data in a subject-specific, fixed-effects, event-related model. Participant data were then entered into second-order random effects analyses, contrasting activation as a function of recognition response (“old”, “new”), emotion type (positive lower arousal, positive higher arousal, negative lower arousal, negative higher arousal, neutral), and component type (item, background). Back-sorting the encoding-related activity based upon the memory test data in this way allowed examination of the four possible memory outcomes: remembering only the item, remembering only the background, remembering both item and background, or forgetting both the item and background from a scene. Analyses contrasting activation were calculated applying a threshold of p<.005, with a 12 voxel extent, yielding a corrected p-value of p<.05 as determined by Monte Carlo estimates (Slotnick, Moo, Segal, & Hart, 2003). MNI voxel coordinates for local maxima were translated into Talairach coordinate space; these coordinates reflect the most significant voxel within the cluster of activation in cortical regions, and the most significant voxels more than 6mm apart within the subcortical clusters (Talairach & Tournoux, 1988).
To further understand the relation between neural activity and subsequently remembered emotional scene information, region-of-interest (ROI) analyses were performed, using the MarsBar toolbox (Brett, Anton, Valabregue, & Poline, 2002). An ROI approach was chosen to most directly examine how the scenes’ emotional content affected the encoding-related activation associated with remembering only an emotional item while forgetting its paired background in the scene, within regions that were active for successful memory for all types of emotional items. ROIs were defined from a whole brain analysis that contrasted remembering only the item within an emotional scene with remembering only the item in neutral scenes (i.e. [lower arousal positive item only + higher arousal positive item only + lower arousal negative item only + higher arousal negative item only] > neutral item only). Because this contrast collapsed together activity from all emotional scenes, the regions defined by this analysis were unbiased with regard to whether they would show a difference in activation between the four types of emotional scenes. Event-related time-courses were extracted from active clusters by creating ROIs that included all significant voxels within a 5mm radius of the maximum voxel. A hemodynamic response function was calculated within each ROI for each individual participant and condition, as a function of peristimulus time (0–16 sec.). One-way ANOVAs testing the factor of emotion (lower arousal positive item only, higher arousal positive item only, lower arousal negative item only, higher arousal negative item only) were calculated on the average signal change within peristimulus time 6–8 sec in each ROI. Figures depict regions located in neurological space.
The term emotional enhancement in item memory is applied to instances where emotional items in a scene are recalled more frequently than are neutral items. Similarly, the term decrement in memory for backgrounds indicates that backgrounds from scenes containing emotional items are remembered less often than backgrounds from scenes containing neutral items. When both of these conditions (i.e., both enhancement in item memory and decrement in memory for backgrounds) arise within memory for scenes of a particular emotional type, a tradeoff effect has occurred. The magnitude of the trade-off in memory observed for emotional scenes is in relation to one’s memory ability for neutral scenes, respective of one’s overall memory ability. In this way, calculation of the trade-off effect takes into account individual variations between those persons with overall ‘good’ or ‘poor’ memory ability, and the magnitude of the trade-off observed is not driven by the circumstance of some persons having an overall more limited memory capacity, which would be present across all types of scenes.
We first established the presence of a trade-off effect in the behavioral results. Then an interaction contrast on the neuroimaging data identified the pattern of activation corresponding with an emotional enhancement in item memory and decrement in memory for backgrounds, thereby revealing the regions predicting a trade-off in memory for emotional scenes. Closer examination of the activation in regions identified within this contrast allowed further illustration of the effects of emotion upon the memory trade-off. Neuroimaging techniques additionally permit isolation of the encoding-related neural activity that specifically predicts selective memory for an emotional item without its background in emotional scenes, compared to the activity predicting good memory for both components from emotional scenes. These analyses identify regions exclusively associated with selective item memory as distinct from regions more broadly responding to the emotional item within the scene, irrespective of memory for the background.
Although the background components from scenes were all neutral in valence and not inherently arousing, the type of emotion assigned to backgrounds within analyses are defined by the identity of the item with which they had been paired in a scene during a prior study session (e.g., “high arousal background” indicates a background, such as a river, that had been studied as part of a scene containing a high arousal item, such as a snake).
Behavioral responses were analyzed using corrected recognition scores of hits minus false alarms (reported in Table 1). Corrected recognition rates were calculated to consider differences in participants’ biases to say “old” to test items. There was no significant interaction between the magnitude of the trade-off and gender (Fs(1,17)<1.00), so data from men and women were combined in all analyses2.
To establish to presence of a trade-off in behavioral memory performance, we conducted an ANOVA with factors of component (items, backgrounds) and scene type (lower arousal positive, higher arousal positive, lower arousal negative, higher arousal negative, neutral) on the corrected recognition scores. This ANOVA produced significant main effects of component (F(1,18)=113.75, p<.0005, partial η2=.86) and emotional scene type (F(4,15)=6.45, p<.003, partial η2=.63) qualified by an interaction between component and scene type (F(4,15)=18.17, p<.0005, partial η2=.83). The interaction between component and scene type reflects a trade-off effect, with an enhancement in memory for emotional items and a decrement in memory for backgrounds presented within emotional scenes relative to neutral scenes (Fig. 2), replicating the pattern of behavioral results reported in Waring & Kensinger (2009). Additionally, we computed T-tests contrasting memory for emotional scene components each compared to neutral (e.g., positive lower arousal items compared to neutral items, negative low arousal backgrounds compared to neutral backgrounds). There was a significant enhancement in memory for four types of emotional items compared to neutral (all t(18)>3.16, p<.005). There was a significant decrement in memory for backgrounds from emotional scenes compared to neutral scenes (all t(18)>2.31, p<.03) except for positive higher arousal scenes for which the numerical difference did not reach significance (t(18)=1.19, p=.25). Taken together these results show that the enhancement in memory for emotional items is present in all scene types, but the trade-off in memory is less likely to arise in scenes containing positive higher arousal items.
We analyzed the pattern of response times for the study phase approach/retreat decision between emotional scenes to find whether the effect of scene valence or arousal on the magnitude of the trade-off could be explained by the amount of time taken to consider the approach/retreat decision during the study phase. These response times reflect the amount of time to make the decision but not the total amount of time spent viewing the scene; participants viewed all scenes for a uniform time period (3 sec.), regardless of speed of responding. An ANOVA of response times with factors of scene type (lower arousal positive, higher arousal positive, lower arousal negative, higher arousal negative, neutral), component (item, background) and memory (remembered, forgotten) showed a main effect of scene type (F(4,15)=18.62, p<.0005, partial η2=.83), reflecting that response times were fastest for higher arousal negative scenes (m= 1.44 sec), slower for lower and higher arousal positive scenes and neutral scenes (all m = 1.59 sec), and slowest for lower arousal negative scenes (m=1.65 sec). The ANOVA also revealed an interaction between the factors of component and memory (F(1,18)=6.17, p<.03, partial η2=.26), where response times for items were unaffected by whether the item was subsequently remembered or forgotten (remembered items m = 1.56 sec, forgotten items m = 1.58 sec), but response times to scenes for which the background was subsequently forgotten were faster than to those scenes for which the background was subsequently remembered (forgotten backgrounds m = 1.54 sec, remembered backgrounds m = 1.61 sec). Importantly, there were no significant 2- or 3-way interactions between scene type and component or memory in the pattern of response times data (all p>.05), indicating that the response times did not affect memory for scene components significantly differently between the types of scenes. These results suggest that response time differences could contribute to the occurrence of selective item memory in general, but that across scene types, variability in response times for the encoding task cannot explain why there would be differences in the tradeoff effect between different types of emotional scenes.
Having demonstrated a trade-off in memory for all four types of emotional scenes, we wanted to examine the neural processes that led to those effects. We conducted an interaction contrast on the neuroimaging data to identify the regions predicting a trade-off in memory for emotional scenes, revealing the pattern of activation corresponding with an emotional enhancement in item memory and a decrement in memory for backgrounds. This contrast for correctly recognized scene components was defined as: (emotional – neutral when only the item remembered) – (“emotional” – neutral when only the background remembered); i.e., ([(lower arousal positive item only + higher arousal positive item only + lower arousal negative item only + higher arousal negative item only) – (neutral item only)] – [(lower arousal positive background only + higher arousal positive background only + lower arousal negative background only + higher arousal negative background only) - (neutral background only)]). Results of this interaction contrast showed that there is activation common to the trade-off among the four types of emotional scenes within a network of regions frequently associated with successful emotional memory (Adolphs et al., 2001; Dolcos et al., 2004; Kensinger & Schacter, 2008; Mather et al., 2006). These regions included the inferior and medial frontal gyrus, anterior cingulate cortex (ACC), left and right amygdala, thalamus, hippocampus, superior temporal gyrus, fusiform gyrus, and a large area along the middle temporal gyrus that was particularly left lateralized (see table 2).
We performed additional analyses within six regions-of-interest identified from the interaction contrast that are part of the traditional emotional memory network, in order to better understand the effects of emotion upon the memory trade-off within these regions. Regions of the inferior frontal gyrus (−46, 16, 14), right amygdala (26, 1, −12), left amygdala (−26, −5, −13), fusiform (−42, −42, −15), hippocampus (−32, −29, −4), and temporal pole (34, 1, −12) were selected for these further analyses (all coordinates refer to Talairach coordinates). We computed ANOVAs for each ROI to assess effects of 4 emotional scene types (lower arousal positive, higher arousal positive, lower arousal negative, higher arousal negative) upon the encoding–related activation that was associated with remembering an emotional item while forgetting its paired background in the scene. There were no significant effects of scene emotional content upon the encoding-related activation that was associated with remembering an emotional item while forgetting its paired background in the scene (Fs(3,15)<2.2, p>.13, partial η2< .31). These results indicate that a core set of regions is associated with the trade-off for emotional scenes, and that these regions are not involved differently based upon specific scene arousal or valence characteristics.
The interaction contrast reported above was chosen to mirror the way in which the tradeoff is behaviorally defined: as a comparison between the successful memory for elements of emotional compared to neutral scenes. Yet part of the reason why trade-offs are typically defined in relation to neutral scenes is because there is no behavioral measure of what happens when an item is remembered and a background is forgotten within a single scene; the proportion of time that this occurs is only meaningful in relation to a neutral scene baseline. By contrast, neuroimaging techniques permit the comparison of the neural signatures predicting successful item memory when the background of the scene is later remembered versus later forgotten. Therefore, contrasting the neural activity during encoding that predicts selective memory for the emotional item without its background compared to the activity predicting good memory for both components from emotional scenes (i.e., item only > item + background) can be additionally informative. In other words, is there a set of regions exclusively associated with the trade-off effect, or does this pattern of activation more generally reflect the response to the emotional item within the scene, irrespective of whether the background from the scene was remembered or forgotten? To answer this question we contrasted activation associated with remembering only the emotional item while forgetting the background versus remembering the emotional item and background that is common among all four emotional scene types ([lower arousal positive item only + higher arousal positive item only + lower arousal negative item only + higher arousal negative item only] > [lower arousal positive item and background + higher arousal positive item and background + lower arousal negative item and background + higher arousal negative item and background]). This contrast revealed that activation uniquely associated with selective item memory is found mainly within temporo-parietal regions, such as the inferior parietal lobule and middle temporal gyrus (see table 3). The regions identified here indicate the areas that are necessary for selective attention upon an emotional item in the scene, to the detriment of memory for the background of the scene. These regions are not all present in the interaction contrast because this contrast did not specify the regions exclusively associated with selective attention for items within emotional scenes; the same regions revealed here may be necessary for allocation of attentional resources toward items within neutral scenes. The regions identified in this contrast, but not in the interaction contrast previously described, are the areas necessary for attention to emotional items when the background is later forgotten versus later remembered, however they are not exclusive of those regions that may also be active for attention to neutral items.
Further, we examined whether there are certain regions associated with selective memory for emotional items depending upon the specific valence and arousal characteristics of the item within the scene, beyond those regions commonly activated among all 4 types of emotional scenes. We examined how the set of regions associated with selective item memory in each of the four types of emotional scenes is distinct from the other three types (e.g. positive lower arousal item only > [positive higher arousal item only + negative lower arousal item only + negative higher arousal item only]). These analyses identified the set of regions supporting selective memory for items from each of the four emotional scene types.
These four contrasts indicated that in addition to the core network of regions associated with the trade-off among all types of emotional scenes, there are additional networks of regions active for selective item memory in each of the four types of emotional scenes. The pattern of activation supporting selective memory for positive lower arousal items showed a small set of regions including portions of the medial and middle frontal gyri, anterior cingulate, and middle occipital gyrus (see table 4). The selective memory for positive higher arousal items was associated with a larger set of regions widely distributed across the prefrontal cortex (BA 6, 8, 9 10, 11, 46, 47), as well as cingulate gyrus and bilateral regions of the fusiform (see table 5). The selective memory for negative lower arousal items was predicted by activation mainly with middle occipital gyrus, cuneus, and middle frontal gyrus (see table 6). Lastly, the negative higher arousal item selective memory corresponded with activation in the left fusiform (see table 7). As depicted in Figure 3, these regions were largely unique and non-overlapping, with small areas of common activation among positive and negative lower arousal scenes in the left middle frontal gyrus and left cuneus/middle occipital gyrus, and among positive and negative higher arousal scenes within the left fusiform.
The purpose of this study was to elucidate how encoding-related neural activity relates to the trade-off in memory for scenes containing emotional items across a range of valence and arousal levels. Past research investigating the effect of emotion upon the coupling of stimulus features in memory provided evidence that emotionally salient features can differentially attract visual attention at encoding, thereby reducing integration of central aspects with peripheral contextual information into a unified mental representation (Reisberg & Heuer, 2004; Mather et al., 2006; Mather, 2007). Results of the current study provided evidence that such selective item memory can arise from the engagement of emotion-specific processes during encoding and that there is associated neural activity that is common to the trade-off among all emotional scene types. Results also showed that activation in other regions varies with the dimensions of valence and arousal of the central item in the scene.
We found that the encoding activation predicting the trade-off across all emotional scenes greater than neutral includes a set of limbic, prefrontal, and temporal regions often found to be active in response to emotional information and also associated with successful subsequent memory for emotional information (Kensinger & Schacter, 2006a; Kensinger et al., 2007c; Talmi, Anderson, Riggs, Caplan, & Moscovitch, 2008). These regions predict memory for the emotional item and not for its paired background. Closer examination of the timecourse of activation within the left and right amygdalae, hippocampus, fusiform, temporal pole, and inferior frontal gyrus showed that these areas are generally important for selective memory for emotional information, however there is no significant difference between activation corresponding with the specific valence or arousal characteristics of the emotional item in the scene.
However, results of this analysis were not limited to regions that were only predictive of a trade-off in memory, and could also be inclusive of regions engaged when the background would later be remembered. For this reason, we also examined how activation predicting a trade-off in memory differed from activation predicting later remembering both the emotional item and the background to isolate regions that were specific to the trade-off effect from those generally responsive to the presence of the emotional item in the scene. This contrast indicated that there were regions mainly within the middle temporal gyrus and inferior parietal lobe showing stronger correspondence with the trade-off effect than when remembering both scene components across all types of emotional scenes. This pattern of activity is in line with the literature showing that these regions often correspond with focused visual detail processing or directed visual attention (Garoff, Slotnick, & Schacter, 2005; Pessoa, Kastner, & Ungerleider, 2002). The limbic regions previously identified for their correspondence to the trade-off were not revealed in this analysis, suggesting that these limbic regions are important for the emotional enhancement in item memory irrespective of whether or not the background is remembered. As this is the first study to probe the underlying neural activity associated with selectively remembering the emotional component of a scene, while forgetting the background context, it may be that the encoding-related activation reported previously for complete images including an emotional component in fact represents the neural processes for selectively encoding the emotional item. These results show that there is a core network of regions whose activation generally predicts good memory for the emotional item in the scene, but that there are also areas within the temporal and parietal lobes whose activation specifically predicts later forgetting the background of the scene. However, without eye-tracking data, inferences about whether this represents additional visual processing or elaborative, conceptual processing would be merely speculative.
Prior literature has often reported that arousing stimuli evoke scene memory trade-offs, however the conclusions about valence effects have been less clear. In this study, we found that items across a range of emotional valence and arousal levels led to memory narrowing. The trade-off in memory observed here for positive and negative scenes is in contrast to some prior studies, which suggested that only negative items serve as “attention magnets” leading to memory narrowing, while positive items lead to a broadening of attention and memory focus to include peripheral or contextual information (Libkuman et al., 2004). Yet our results are consistent with other findings (Kensinger & Schacter, 2006a; Mather et al., 2006; Waring & Kensinger, 2009) suggesting that the trade-off can occur for any high-arousal content, regardless of valence, and extend this work to show that less arousing scenes can also promote a scene memory trade-off.
We were also particularly interested to find how the valence and arousal characteristics of the emotional item within the scene modulate the neural activity associated with selective memory. Each of the four scene valence and arousal combinations (positive lower arousal, positive higher arousal, negative lower arousal, negative higher arousal) had a unique pattern of underlying activation associated with the effect, showing that valence and arousal must be considered when evaluating the impact of emotion upon memory. Although there is a robust core network predictive of the trade-off for all emotional scene types, these additional regions identified for each individual scene type may reflect that items of different valence and arousal combinations are processed in distinct ways, and some of this processing may play a role in commanding affective-attention processes.
The fact that there were so few regions uniquely active for the higher arousal negative scenes relative to other emotional scene types suggests that the encoding of higher arousal negative items is mainly reliant upon the regions that are shared among emotional scene types. In other words, the regions supporting the encoding of negative higher arousal items are those generally necessary for encoding of visual detail processing across a range of emotional scenes. In contrast, selective item memory for lower and higher arousal positive scenes and lower arousal negative scenes only occurs when additional encoding processes are recruited. The recruitment of additional fusiform areas for the encoding of negative higher arousal items is in line with literature showing that arousing negative images evoke activation mainly within visual processing regions during successful encoding (Kensinger, Garoff-Eaton, & Schacter, 2007c; Mickley & Kensinger, 2009). In the same vein, the large span of areas active across the prefrontal cortex for encoding of positive higher arousal items is supported by the previous literature showing that encoding of positive images is associated with processing within more anterior regions (Dolcos et al., 2004; Mickley & Kensinger, 2009).
Alternatively, the unique sets of regions associated with selective memory for each type of item may be indicative that stimuli of differing valence or arousal levels are qualitatively different. Although the valence or arousal level of affectively-laden stimuli is often characterized upon continuous scale, ranging from values signifying less arousing (more calming, subduing) to more arousing (more exciting, agitating), and from more positive to more negative, there are a number of lines of research that have suggested that there may be a qualitative shift when items move from being lower in arousal to being higher in arousal, or between positive and negative items matched for ratings of arousal level. There often are different cognitive and neural processes associated with the processing and retention of information with varying levels of arousal (Kensinger, 2004; Mickley & Kensinger, 2009; Talmi & Moscovitch, 2004), and items rated lower in arousal may evoke different types of emotions from those rated higher in arousal. For example, lower-arousal items are more likely to elicit sadness or calmness, whereas emotional items rated higher in arousal level may be more fear-inducing or surprising. The fact that the neural signatures associated with selective memory between scene types present patterns largely unique from one another may fit more in line with expectations when the characteristics of emotional stimuli are considered in this light. Future studies where participants record their affective responses to the scenes presented could provide interesting information about the correspondence between specific emotions and the trade-off effect.
As each of the emotional scene types corresponded with a differing behavioral effect magnitude, we can only speculate whether this may have had a meaningful impact upon the extent of regions engaged. Although there is not necessarily a correspondence between magnitude of the trade-off and extent of activation, the association between these factors is unclear. Is the set of regions predicting a trade-off for negative higher arousal scenes limited because there is also small behavioral effect, or is this evidence that negative higher arousal items are playing a very strong role in the activation related to the trade-off in this scene type and thus not observable when contrasted with activation predicting that both item and background will be remembered? Future studies that are able to match the level of behavioral performance between conditions may be able to resolve this question.
The methodology employed here could not provide further information about the relationship between neural activity and longer-term effects of consolidation of emotional information, yet it did grant novel insight into the relationship of controlled visual attention and emotional arousal to short-term emotional memory in young adults. This outcome provides evidence that emotional stimuli engender selective memory for certain portions of stimuli rather than a universal enhancement for all stimulus features (Adolphs et al., 2005; Kensinger & Schacter, 2006a; Kensinger et al., 2007c). Although conclusions that can be drawn from fMRI results regarding visual attentional process are limited, future studies employing eye-tracking technology could lend more direct evidence of how visual attention is allocated within emotionally arousing scenes. Results from those investigations could then be associated with subsequently remembered scene components in order to better understand the contributions of visual attention versus consolidation stage processes to the trade-off in scene memory.
Though it often has been argued that trade-offs in memory for emotional scenes are the result of encoding-related processes, this study is the first to provide direct evidence of a link between the encoding processes engaged during scene processing and the likelihood of a tradeoff in memory. The present study makes clear that there can be enhanced memory for an emotional focus at the expense of memory for the background context, and that the existence of this memory trade-off can be related to activity elicited during encoding. The present results also reveal that key emotional memory regions such as the amygdala, hippocampus, fusiform, temporal pole, and inferior frontal gyrus, are commonly active for the trade-off among the range of emotional scene types, but that the exact pattern of neural activity corresponding with this effect varies with a scene’s valence and arousal properties. These results provide further evidence that activation in several regions can correspond with enhanced memory for some scene details, but simultaneously can be related to bad memory for other elements of scenes.
This research was supported by grants from the National Science Foundation (grant BCS 0542694) and the National Institute of Mental Health (MH080833) to E.A.K. Portions of this paper were included in the Master’s thesis of JDW. The authors thank Jessica Payne, Daniel Schacter, Maya Tamir, and Scott Slotnick for helpful discussions; Keely Muscatell and Kristin Hricko for assistance with data collection; and Jacqueline Grant and Chris Brown for assistance with stimuli development.
1Memory was tested for a large number of neutral novel items in an attempt to present similar numbers of novel emotional and neutral items at test, and because the novel backgrounds would be all inherently neutral.
2There was no main effect of gender upon our results, and there was no interaction observed between gender and arousal (Fs(1,17) < 1.00). There was a small effect of an interaction between component and gender because across scene types women remembered a greater number of items than men, and fewer backgrounds (F(1,17)=4.92, p<.04). Because this effect was present across scene types (i.e. affecting emotional and neutral scenes), we do not discuss this effect further.
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.