|Home | About | Journals | Submit | Contact Us | Français|
It seems obvious that what you see influences what you feel, but what if the opposite were also true? What if how you feel can shape your visual experience? In this experiment, we demonstrate that the affective state of a perceiver influences the contents of visual awareness. Participants received positive, negative, and neutral affect inductions and then completed a series of binocular rivalry trials in which a face (smiling, scowling or neutral) was presented to one eye and a house to the other. The percepts “competed” for dominance in visual consciousness. We found, as predicted, that all faces (smiling, scowling, and neutral) were dominant for longer when perceivers experienced unpleasant affect compared to when they were in a neutral state (a social vigilance effect), although scowling faces increased their dominance when perceivers felt unpleasant (a relative negative congruence effect). Relatively speaking, smiling faces increased their dominance more when perceivers were experiencing pleasant affect (a positive congruence effect). These findings illustrate that the affective state of a perceiver serves as a context that influences the contents of consciousness.
It is well known that perceivers contribute to perception (referred to top-down influences). A few milliseconds after a stimulus is presented, a perceiver’s brain begins making predictions about what an object is and how to act on it (Kverga, Ghuman, & Bar, 2007). Top-down feedback projections play a key role in human vision in all but the simplest circumstances. Even so-called bottom-up structures, such as the midbrain’s superior colliculus, and thalamic nuclei such as the pulvinar and mediodorsal, are influenced in a top-down manner (Abramson & Chalupa, 1985; Casanova, 1993; Webster, Bachevalier, & Ungerleider, 1993; Webster, Bachevalier, & Ungerleider, 1995).
The affective state of the perceiver is an important source of top-down influence in vision (Barrett & Bar, 2009). When perceivers are briefly exposed to affectively evocative faces or images, they are momentarily more sensitive to changes in low spatial frequency visual information (Bocanegra & Zeelenberg, 2009; Phelps, Ling, & Carrasco, 2006). Affect leads people to overestimate how far it is to the ground from a balcony ledge, to perceive the size of objects on the ground as larger (Stefanucci & Proffitt, 2009), and to overestimate the steepness of a hill (Stefanucci, Proffitt, Clore, & Parakesh, 2008).
Whereas other studies have examined how the affective state of the perceiver influences the accuracy or efficiency of visual processing, in this paper we examine whether affect influences what perceivers are aware of seeing. Early perceptual systems process significantly more information than that which reaches consciousness. Visual consciousness, then, is the information that people are aware of seeing (for reviews on the content of visual consciousness see Lamme, 2000; 2004). To examine whether the affective state of a perceiver influence what information is selected for visual awareness, we used a phenomenon known as binocular rivalry (for review see Blake, 2001). Binocular rivalry occurs when perceptually different images are presented to each eye (e.g., a face to one eye and a house to the other eye) and compete for perceptual dominance. Visual input from one eye is dominant (and seen) while the other image is suppressed (and remains unseen). Eventually, individuals experience the two images as alternating over time. By measuring the amount of time that each image is dominant (or suppressed), it is possible to determine which visual input the brain is selecting for conscious experience. Voluntary control and controlled attention do not influence which image is consciously seen (Meng & Tong, 2004), although imagining an object increases its dominance (Pearson, Clifford, & Tong, 2008).
In the present study, we were interested in whether the affective state of the perceiver would affect the visual dominance of affective and neutral faces during binocular rivalry. We considered that the perceiver’s affective state might influence perception in two different ways. First, affect might increase visual awareness for socially relevant information, consistent with the social brain hypothesis (Dunbar, 1998), promoting a social vigilance effect. With social vigilance, social percepts (such as faces) are more affectively relevant than non-social percepts (such as houses), leading faces to be more dominant to perceivers experiencing a hedonically charged state. Alternatively, the affective state of the perceiver might produce an affective congruence effect. A perceiver in a pleasant affective state might be more visually aware of smiling faces, whereas a perceiver in an unpleasant affective state would be more visually aware of scowling faces. We examined the role of pleasant and unpleasant affect on visual consciousness because positive and negative affect has been shown to exert differential influences during cognitive processing (Storbeck & Clore, 2005; Schwarz & Clore, 2007).
In addition, we examined the salience of smiling and scowling faces in visual awareness when perceivers were in a neutral affective state, allowing us to test whether emotional faces are prioritized for visual consciousness. Prior research indicates that perceivers experience emotional faces as perceptually dominant (Alpers & Gerdes, 2007; Alpers & Pauli, 2006; Bannerman, Milders, De Gelder, & Sahraie, 2008). In these studies, the affective state of the perceiver was neither directly controlled nor measured, and the affective images themselves were assumed to dominate because of their intrinsic affective content. But affective stimuli are only impactful in as much as they alter the state of the perceiver. A scowling face is said to be negative by virtue of its ability to make the perceiver feel momentarily unpleasant. By controlling the affective state of the perceiver directly (by inducing a neutral state), it was possible to determine whether emotional faces themselves were guiding visual awareness (an affective salience hypothesis).
Participants were 50 (15 male) naïve young adults ranging in age from 17 to 32 (Mean = 20.64 years). Nine participants were excluded from analysis because of extreme eye dominance (e.g., reported seeing the image presented to only their right or left eye). All participants reported normal or corrected-to-normal acuity.
Instructions and stimuli were presented using E-Prime Version 2 running on a Dell Optiplex 725 and a 17-inch Dell LCD flat-screen monitor (1280 X 1024). Participants sat with their head fixed with a chin rest and viewed stimuli through a mirror stereoscope at a distance of approximately 55 cm.
To manipulate participants’ affective state, we presented images from the International Affective Picture System (IAPS; Lang, Bradley, & Cuthbert, 2008). Ten images were selected from the IAPS set for each affect induction condition according to their normative ratings (for list of images used see Table 2 in the online supporting information). In an induction block, five images of the same type were presented with each IAPS image being presented for 2 s with a 300 ms ISI. Following a block of 5 IAPS images, participants completed 3 binocular rivalry trials (see below). To ensure that visual dominance effects were due to valence and not arousal, we manipulated pleasant and unpleasant affective states at both high and low levels of arousal at six points (blocked) in the experiment for a total of 24 ‘affective’ induction trials. These blocks were separated by two ‘neutral’ affect induction trials (neutral IAPS images) for a total of six neutral induction trials, creating 30 total trials. After each induction condition, participants reported their core affective state in terms of valence and arousal using a 9-point scale.
Following each block of five IAPS images, participants viewed three binocular rivalry test trials where participants were presented with a face to one eye and a house to the other eye (counterbalanced across trial). Faces depicted a neutral, smiling, or scowling face (randomized across the three trials). Stimuli were either grayscale photographs or line drawings (from Alpers & Gerdes, 2007), for the analysis presented here we collapsed across stimuli type. All photographs were matched on luminance and contrast to a single face target using Adobe Photoshop CS2’s color match tool.
Rivalrous stimuli subtended approximately 1.8 × 1.4 degrees of visual angle which pilot work showed is large enough to clearly perceive the stimuli but small enough to reduce blended percepts (Blake, O’Shea, & Mueller, 1992). A frame was placed around each stimulus to facilitate fusion of the two images. Each binocular rivalry trial began with a 300 ms fixation immediately followed by the 10 s face-house pair presentation. There was a 1 sec interval between each trial. Participants were instructed focus on the central fixation cross and to press and hold the ‘1’ key when they perceived a face, ‘9’ when they perceived a house, and to hold down both keys if they saw both a house and a face or a blend of the two (response keys were counterbalanced across participants). Participants were instructed to keep their fingers on the keys at all times during the task. Each face-house pair was presented 10 times, for a total of 90 binocular rivalry presentations.
As a manipulation check, participants rated all 50 IAPS images used on a 9-point scale for valence (1=unpleasant and 9=pleasant) and arousal (1=low arousal and 9=high arousal). Each image was displayed for 1 s.
To assess whether the perceiver’s affective state led to greater visual awareness of faces (affective vigilance) or to greater awareness of valence-congruent faces (affective congruence), and whether emotional faces dominate in visual awareness when a perceiver is in a neutral state (affective salience), we employed Hierarchical Linear Modeling (HLM) (Raudenbush, Bryk, Cheong, & Congdon, 2000). This allowed us to avoid aggregation and model the trial-by-trial responses nested within several different affective states within the same participant (for an example, see Barrett & Niedenthal, 2004). This approach has advantages over traditional methods of analyzing repeated measures data (like ANOVA), including simultaneous estimation of within-subject and between-subject variance, more efficient estimation of effects, lower Type-1 error rates (Kenny, Korchmarcos, & Bolger, 2003). We used a multivariate data set up (e.g., Barrett & Niedenthal, 2004), and tested each hypothesis using the multivariate hypothesis testing procedures (Raudenbush et al., 2000).i This is equivalent to the use of targeted, planned contrasts and allowed us to precisely test each hypothesis. Because we did not predict an effect of arousal, we collapsed across arousal conditions to examine our hypotheses for pleasant, unpleasant, and neutral affective states. For each individual observer, on each trial, we summed the amount of time a face was visible (face dominance index) as well as the time a house was visible (face suppression index).ii Blended percepts were excluded from analysis so trials on which participants reported only seeing blends did not contribute to the data reported here.
Participants’ self-reported feelings of pleasantness confirmed that the affect manipulations successfully influenced participants’ affective state, F(2, 98) = 43.05, p <.001. After viewing negative IAPS images, participants reported their experience as significantly less pleasant than following neutral t(49) = 7.104, p < .001, or positive IAPS images t(49) = 7.008, p < .001. And after viewing pleasant IAPS images, participants reported feeling more pleasant than following neutral IAPS images t(49) = 3.974, p < .001.
Table 1 presents the average duration that faces dominated in visual consciousness for all affect induction conditions. Across all affect induction conditions, smiling faces dominated in visual consciousness more than neutral or scowling faces, χ2 (1) =10.88, p = .001. This might have occurred because smiling faces contain more variations in contrast (light and dark patches) than do neutral and scowling faces, and such low level visual properties cause images to be more visually dominant (Blake, 2001). Even with the increased dominance of smiling faces, however, we found evidence that a perceiver’s affective state influenced visual consciousness in a top-down fashion.
When perceivers were in an unpleasant affective state, we found evidence for the social vigilance hypothesis. Specifically, all faces dominated in visual consciousness for a longer time when perceivers were in an unpleasant affective state compared to when they were in a neutral state, χ2 (1) =5.54, p < .018. Nonetheless, scowling faces dominated significantly longer when perceivers were unpleasant than when they were neutral (the change was 439.4 ms longer) and this was statistically significant, χ2 (1) =7.28, p < .007. The change was not significant when comparing the dominance of scowling faces in a pleasant versus a neutral affective state, however. Because scowling faces were not more dominant in in absolute terms when perceivers were feeling unpleasant, this serves as evidence of a relative negative congruence effect.
When perceivers were in a pleasant affective state, we found evidence of a relative positive congruence effect. Specifically, after viewing pleasant IAPS images, smiling faces dominated for 310.07 ms longer than did scowling faces, χ 2 (1) =4.26, p < .037 and 382.22 ms longer than did neutral faces, χ 2 (1) =8.79, p < .003. Thus, in an absolute sense, smiling faces dominated longer but smiling faces dominated across all conditions, and this was not unique to a pleasant affective state. Still, smiling faces dominated significantly longer than did neutral faces only when perceivers were in a pleasant affective state; this comparison was not statistically significant when perceivers were in a neutral state, χ 2 (1) = 1.53, p < .21. Social vigilance was not observed for pleasant affective states, χ 2 (1) =1.07, p < .30.
Finally, when the perceiver’s affective state was set to neutral, we found no evidence of an affective salience effect. When perceivers were in a neutral state, smiling and scowling faces did not dominate in visual consciousness longer as compared to neutral faces, χ 2 (1) =.03, p > .50.
In this experiment, we provide the first direct evidence that a perceiver’s affective state helps to select the contents of consciousness. When individuals were in an unpleasant affective state, scowling, smiling, and neutral faces dominated in visual awareness (as opposed to a house). Furthermore, scowling faces showed the largest increase in dominance when perceivers were in an unpleasant affective state (as compared to when in a neutral state), demonstrating a relative negative congruence effect. These findings are consistent with work suggesting that cognitive processes are tuned to meet the situational requirements signaled by the perceiver’s affective state (Schwarz, 2002) and that individuals are reflexively vigilant to negative social information, perhaps as a way monitor their environment for potential danger (Pratto & John, 1991). In all conditions, smiling faces were prioritized in visual consciousness, but relatively speaking, smiling faces dominated significantly more than did neutral faces when perceivers were in a pleasant state (when in a neutral state, this comparison was not significant). This relative positive congruence effect is consistent with research in the attention literature showing that positive affect biases attention toward rewarding information (e.g., Neidenthal & Halberstadt, 2000; Tamir & Robinson, 2007), presumably in the service of approach-related decision-making and behavior (Frederickson, 2001). The fact that we observed differential perceptual outcomes for pleasant and unpleasant affect is consistent with a number of findings that positive and negative valence can exert distinct influences across a broad array of cognitive processes (Schwarz & Clore, 2007; Storbeck & Clore, 2005).
The fact that smiling faces dominated in visual consciousness more than other types of faces is not surprising. Smiling faces tend to have stronger contrast (dark and light patches) and visual images with stronger contrast enjoy enhanced predominance rivalry (Blake, 2001; Hollins, 1980; Whittle, 1965) Despite low level visual effects, we still observed that the affective state of the perceiver influenced the dominance of faces in visual consciousness. Our findings are an example of the power of top-down, perceiver based influences over bottom up, stimulus driven effects in perception.
When perceivers were in a neutral state, emotional faces did not dominate when compared to neutral faces (i.e., there was no evidence of an affective salience effect). This finding fails to replicate previous findings that emotional information dominates over neutral material in binocular rivalry (Alpers & Gerdes, 2007). There are several possible reasons for this. First, in the majority of binocular rivalry studies perceivers were asked to indicate whether they saw an emotional or neutral object. Task instructions can serve as a context to bias how visual information is selected (Pearson, Clifford, & Tong, 2008; Schyns & Oliva, 1999). Specifically, an emotion word like “happy” engenders imagery of a particular facial configuration (Smith, Gosselin, Cottrell, & Schyns, 2005; Study 2), and imagery influences dominance (Pearson, Clifford, & Tong, 2008). Second, we did not include startled looking (fearful) faces in our study that might dominate more dramatically because of low-level visual features. Finally, it is possible that over time, emotional faces induced an affective change in the state of the perceiver, and this state influenced visual awareness for emotional faces, as the present study shows.
Taken together, our findings suggest that the affective state of the perceiver exerts a top-down influence in vision. This is consistent with our hypothesis that affect is a source of attention in the brain that directly and indirectly modulates the firing of neurons in visual cortex (for a review, see Barrett & Bar, 2009; Duncan & Barrett, 2007). Brain areas involved in the brain’s affective workspace (such as the amygdala and orbitofrontal cortex or OFC) receive direct projections from visual processing areas, including areas in the ventral visual stream whose activation are correlated with turning the external environment into an internal, meaningful representation (see Barbas, 1988, 2000; Carmichael & Price, 1995; Freese & Amaral, 2005; Ongur, Ferry, & Price, 2003) and the amgydala (particularly the basal nucleus) projects back to directly modulate activation in areas of the ventral visual stream (Amaral, Behnlea, & Kelly, 2003) Furthermore, affective brain sites project to nuclei in the brainstem and basal forebrain (Mesulam, 2000), as well as selected nuclei within the thalamus (Zikopoulos & Barbas, 2007), all of which influence the formation of neural assemblies that underlie conscious percepts (Edelman & Tononi, 2000). Finally, the lateral OFC projects to lateral prefrontal cortex, the source of goal-based or executive attention (Barbas, 2000; Miller & Cohen, 2001). Via these pathways, affect has the potential to tune sensory processing and prioritize some visual information for visual consciousness.
Preparation of this manuscript was supported by the National Institutes of Health Director’s Pioneer Award (DP1OD003312) and by the U.S. Army Research Institute for the Behavioral and Social Sciences (contract W91WAW-08-C-0018) to Lisa Feldman Barrett. The views, opinions, and/or findings contained in this article are solely those of the author(s) and should not be construed as an official Department of the Army or DOD position, policy, or decision.
iAlthough we did not predict any relation between affective state and rivalry rate, we also calculated the number of percepts seen per trial. Very brief percepts (less than 100 ms) were excluded from this analysis because we took them to reflect slight differences in reaction time for pressing or releasing both keys to report blended percepts. We found no effect for the number of percepts reported on each trial.
iiSome binocular rivalry studies record the first image to dominate on a particular trial (called “first percept”) and use first percept data as a measure of dominance in their analyses. Visual features of the stimulus such as luminance or contrast are more influential in determining which percept resolves “first” in visual consciousness. Since our interest was in the top-down influences on perception, we did not predict that a perceiver’s affective state would influence a stimulus driven variable like first percept and, in fact, it did not.
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Eric Anderson, Northeastern University.
Erika H. Siegel, Northeastern University.
Lisa Feldman Barrett, Northeastern University, Massachusetts General Hospital/Harvard Medical School.