|Home | About | Journals | Submit | Contact Us | Français|
The development of functional neuroimaging of emotion holds the promise to enhance our understanding of the biological bases of affect and improve our knowledge of psychiatric diseases. However, up to this point, researchers have been unable to objectively, continuously and unobtrusively measure the intensity and dynamics of affect concurrently with functional magnetic resonance imaging (fMRI). This has hindered the development and generalizability of our field. Facial electromyography (EMG) is an objective, reliable, valid, sensitive, and unobtrusive measure of emotion. Here, we report the successful development of a method for simultaneously acquiring fMRI and facial EMG. The ability to simultaneously acquire brain activity and facial physiology will allow affective neuroscientists to address theoretical, psychiatric, and individual difference questions in a more rigorous and generalizable way.
Since the advent of functional neuroimaging, thousands of empirical articles and a dozen or so specialty journals have appeared to support the dissemination of this important work. In fact, a recent PubMed search of “emotion and fMRI” (functional Magnetic Resonance Imaging) yielded over 3000 citations. This work has spanned the neuroscience of basic affective processes to inquiries into how emotion processing and its regulation are disordered in psychopathology. It has been suggested, for example, that this work has the potential to improve psychiatric treatment such that rates of recovery following psycho- or pharmaco-therapeutic interventions will be higher with the ability to uncover biological endophenotypes for the various psychiatric disorders (Insel, 2009).
Unfortunately, the validity and generalizability of affective neuroscience has suffered from an inability to objectively and unobtrusively measure affect as participants are simultaneously being scanned during emotion-related tasks. This is not the case in cognitive neuroscience where reaction time, item-selection, and memory performance are all easily measurable responses from the subject. Thus, a reliable and valid measure of affect could be used to not only confirm successful emotion induction, but could also be used to examine the neural correlates of individual differences in emotion reactivity and regulation. The ideal measure would have the following characteristics: a) objective, b) reliable, c) continuous, d) unobtrusive, and e) valence-specific (i.e., capable of distinguishing between negative and positive emotional states). Development of such a method would greatly facilitate research progress in affective neuroscience.
Despite the need for objective measures of affect, the majority of publications to date have either not measured whether affect was induced (relying instead on normative ratings of affective stimuli), or have relied on subjective self-reports to claim that emotion was successfully induced. The few others have used electrodermal activity (Delgado et al., 2008; EDA, measuring sweat gland activation), pupil dilation (Siegle et al., 2003; Johnstone et al., 2007), cardiac activity (Critchley et al., 2005), or startle (Neuner et al., 2010) to infer changes in emotional state. There are several problems with these methods (see Table 1). 1) Subjective self-reporting of emotion can be biased and inaccurate (Kahneman and Klein, 2009). Furthermore, requesting nuanced, graded self-reports within trials such as, how positive to negative a stimulus is perceived, or how successfully a participant thought he regulated his emotion engages subjects in a secondary self-reflective task. Engagement in such secondary, self-reflective tasks may contaminate brain imaging results, leading to activation in brain areas potentially distinct from those actually involved in emotion or its regulation. 2) While measurements of EDA from the forefingers, pupil dilation and heart-rate are continuous, objective and unobtrusive measures, they are not valence-specific measures – subjects show changes in EDA (Delgado et al., 2008), pupil-dilation and heart-rate in non-specific ways that do not distinguish between the valence of an elicited emotion (Cacioppo and Gardner, 1999). Thus, these measures are thought to reflect general states of arousal rather than specific emotions, per se. 3) Recently, a few publications have reported simultaneous acquisition of startle eye-blink with fMRI (Anders et al., 2004). While the use of this measure in the scanner is promising (the measure is objective and sensitive), startle provides neither a continuous nor unobtrusive measure of emotion.
One method which has not yet been used concurrently with fMRI is facial electromyography (EMG; Cacioppo et al., 2000). Starting with the suggestion by Darwin that the face is central to the experience of emotion (Darwin, 1872) and continued in the psychophysiological tradition, scientists have shown that facial EMG is a robust, unobtrusive and objective measure of emotion (Cacioppo et al., 1986; Cacioppo and Tassinary, 1990). An additional virtue of facial EMG is that it is also specific – of all the measures of emotion, facial EMG is the only objective and unobtrusive measure which can differentiate among various emotions (e.g., happiness, anger, disgust) and which is valence-specific (Lang et al., 1993; Larsen et al., 2003). However, because facial EMG measures micro-volt level changes in muscle activity, it is susceptible to electromagnetic noise. Because of this, to date, successful simultaneous acquisition of fMRI and facial EMG data has not been demonstrated.
To this end, we developed a method of simultaneously acquiring facial EMG – an objective, unobtrusive, and sensitive measure of affect – with Blood Oxygenation Level Dependent (BOLD) fMRI. The development of such methods promises to enhance the rigor and interpretability of neuroimaging research on emotion. Thus, we elected to use a highly robust and well replicated task to examine the simultaneous acquisition of corrugator EMG and BOLD fMRI. In this task, subjects passively viewed negative and neutral images from the International Affective Picture System set (Lang PJ, 2005). Negative slides from this set have been shown to elicit greater corrugator EMG activity (Larsen et al., 2003), greater amygdala activity (Hariri et al., 2003) and more negative valence ratings than neutral slides (Lang et al., 1993). It should be noted that for this methodological demonstration, we did not incorporate the EMG variable into the fMRI analysis. This report was intended to demonstrate that corrugator EMG can be simultaneously acquired with fMRI and to describe the methods used for such a study.
16 (15 right handed, 1 left-handed) subjects (7 female, mean age: 22.9), recruited from the Madison, WI area participated in the study. All subjects were recruited via the use of flyers posted in public places around the Madison, WI area. Subjects reported no current Axis I disorder and were not currently taking any psychotropic medication. This research was approved by the University of Wisconsin-Madison Health Sciences Institutional Review Board, and all participants provided written informed consent. Subjects passively viewed a set of 200 standardized pictures during scanning. Half of the pictures were “negative” and have been reliably shown to induce negative affect (Larsen et al., 2003); the other 100 pictures have been shown to be “neutral” and induce little or no affect. Pictures were presented for 4 seconds, with an 8 second inter-trial interval. Upon stimulus onset, subjects made a two-button forced choice response indicating whether the image was negative or neutral.
FMRI data were collected using a “bunched slice acquisition sequence” in which each EPI volume was collected in 1.5 sec, while the effective TR was 2.5 sec (with 1 second of silent time between volumes). The 1 sec silent time between volumes was used as the time window for EMG signal analysis (see below). 30 slices were collected, with a native resolution of [3.75×3.75×5 mm]; TE=25, flip angle=60. Given the sluggishness of the BOLD response, we reasoned that the 1.0 sec of quiet time would not disrupt our ability to detect regional brain activity. Data were slice-time corrected, motion corrected, and analyzed using a boxcar function in AFNI. For group analysis, data were normalized to the MNI152 T1 template using FSL’s FLIRT/FNRIT algorithm, resampled to a voxel size of 2mm3, and smoothed with a 6mm Gaussian kernel. We applied an anatomically defined bilateral amygdala region of interest (ROI) based on the Juelich atlas (in which there was a minimum of a 50% probability that the voxel was indeed within the amygdala)(Amunts et al., 2005) to examine whether there was a main effect of amygdala activity. FMRI results were thresholded at p<.05 corrected for multiple comparisons across the whole brain using AFNI’s program AlphaSim.
Facial EMG was recorded from the corrugator supercilii muscle using 4 mm electrodes. The corrugator muscle is involved in the frowning response and is increased during the experience of negative affect. Corrugator EMG data were recorded using a Biopac MP150 recording system and EMG100C electromyogram amplifier with MECMRI cable and filter components for MRI installation. EL254RT Ag-AgCl radio translucent electrodes were applied to the corrugator muscle separated by ~1 cm using adhesive collars and electrolyte gel. To minimize wire movement due to scanner noise and motion, leads were affixed to a foam tube exiting the bore. Grounding was provided EDA sensors located on the index and middle fingers. EMG amplifier gain was 1000 with 1 Hz highpass and 500 Hz lowpass filtering. Sampling rate was 1000 Hz with a TTL pulse from the scanner recorded on one channel for precise timing of the start of each TR.
Biopac EMG data were read into a Matlab program for hand scoring of data between TRs (TR intervals were automatically scored as bad using the TTL pulse channel from the scanner). Each run was divided into 1 sec intervals and power spectral density PSD for each interval computed using Welch’s method on 0.1 sec windows with 50% overlap. A threshold of 15microV^2/Hz was used to eliminate any 1 sec intervals exceeding this value. Corrugator EMG was estimated as the mean value for 45–200 Hz excluding 60 Hz (i.e. 45–48; 62–200 Hz). Linear interpolation was used to estimate picture epoch (-1 to +12sec) 1 sec values from the 1 sec recording intervals with a maximum of 0.5 sec to good recorded intervals. Corrugator EMG values were log10 transformed for normalization and are expressed in log10(uV2/Hz) units.
As stated previously, participants viewed 100 negative and 100 neutral images. Corrugator EMG data was not analyzed for the entire 4 second picture presentation. This was because we found that as the scanner acquired EPI data (1.5 out of every 2.5 seconds), the EMG signal was overwhelmed by the electromagnetic noise induced by the collection of EPI images. Thus, 1.5 out of every 2.5 seconds was automatically scored as bad (during collection of EPIs) and removed from further analysis. However, we were able to reconstruct an average time course of EMG signal during negative and neutral trials because trial onset was jittered with respect to EPI onset. For example, on some trials, the scanner acquired EPI data for the initial 1.5 seconds, and then from seconds 2.5 to 4, leaving corrugator EMG to be analyzed at seconds 1.5 to 2.5. On other trials, EPI data would have been acquired from seconds −0.5 to 1 and 2 to 3.5 and corrugator EMG would be analyzed from seconds 1 to 2 and 3.5 to 4. On other trials, EPI data would be acquired from seconds 1–2.5 and from 3.5 to the end of the trial, allowing corrugator EMG data to be analyzed from seconds 0 to 1 and 2.5 to 3.5. The average number of seconds corrugator EMG was analyzed per trial was 1.67s, with a range of 1 – 2 seconds. This is the case because trial presentation was jittered randomly with respect to TR onset. By averaging corrugator EMG magnitude across trials (but within condition), the jittering of EPI data acquisition with respect to trial onset allowed us to reconstruct the time course of corrugator EMG activity for negative and neutral trials. An example snapshot of the raw corrugator EMG data can be seen in Figure 3.
The initial analysis examining whether there was a main effect of valence on corrugator EMG magnitude was performed in accordance with previous studies examining the main effect of valence on corrugator EMG magnitude (Larsen et al., 2003). We averaged corrugator EMG magnitude during the 4,000-ms stimulus presentation and performed a paired-sample t-test (negative – neutral) to test for a main effect of valence. If the main effect of valence was indeed significant, we sought to follow up the first test and examine which specific time-points of the 4 seconds drove the main effect. For this test, we performed paired-sample t-tests for each of the 4 seconds.
At the end of the scan session, subjects were re-presented the same visual stimuli outside the scanner and asked to rate (on a 1–9 scale, negative – positive) the valence of the image. One subject did not perform post scan ratings, and therefore only 15 subjects were used to compare negative and neutral post-scan ratings.
Post-scan ratings confirmed that the negative stimuli were perceived as more negative than the neutral stimuli (t(14) = 9.34, p < .001, Figure 1). FMRI analyses also replicated previous reports and confirmed our predictions – bilateral Amygdala activity was significantly higher for negative as compared to neutral stimuli (max x,y,z: [18, −2, −20], t(15) = 5.31, p < .001, Figure 2).
Our initial analysis examining whether there was a main effect of valence on corrugator EMG magnitude was performed in accordance with previous studies examining the main effect of valence on corrugator EMG magnitude (Larsen et al., 2003). We averaged corrugator EMG magnitude during the 4,000-ms stimulus presentation. We found that corrugator EMG, acquired simultaneously with BOLD fMRI differentiated negative from neutral trials (t(15) = 1.77, p= 0.04 [one-tailed]). Further, and in accordance with previous reports using facial EMG, examining the first four scan runs yielded a more robust difference between negative and neutral slides (Dimberg, 1990). This yielded a significant main effect of valence on corrugator EMG activity (Figure 4a; t(15) = 2.26, p = 0.04). We then examined which specific time-points drove the main effect. This was driven by significant differences at 2000- and 3000-ms post stimulus onset (Figure 4b; time point 2: t(15) = 2.14, p = 0.04; time point 3: t(15)=2.45, p = 0.02). Thus, we have demonstrated the ability to simultaneously acquire brain activity with a continuous, objective, and unobtrusive measure of affect, namely facial EMG.
We report a novel method for simultaneously acquiring facial EMG with BOLD fMRI. Significantly, all of our manipulation checks – amygdala activity as well as subjective ratings of valence indicated that our manipulation was consistent with prior reports in the literature. The ability to concurrently acquire facial physiology with brain activity opens up new avenues for research. We discuss a few important areas to explore below.
First, basic affective neuroscience has shown associations between amygdala activity and negative affect, and arousal (Davis and Whalen, 2001). However, it would be illuminating to examine with what other physiological measures amygdala activity co-varies. This is also the case with other brain regions associated with the limbic circuit: the nucleus accumbens, for example, has been shown to be involved in reward-related processing (Delgado et al., 2000), but up to this point there has been no way (other than self-report) of objectively examining whether nucleus accumbens activity actually correlates with magnitude of emotional response. To objectively measure positive affect, for example, investigators would place EMG electrodes over the zygomaticus major muscle. The zygomaticus major muscle is involved in the smiling response and has been shown to be related to positive affect (Cacioppo et al., 1986; Lang et al., 1993; Dimberg et al., 2000; Larsen et al., 2003). Relating a continuous, unobtrusive and objective measure of affect with fMRI would add a level of interpretability and generalizability that has not yet existed in our field.
Second, with the much higher temporal resolution of facial EMG, researchers can begin to examine how the temporal dynamics of emotion relate to brain activity. As emotions are dynamic events, the ability to concurrently track the chronometry and regulation of an emotion with brain activity will likely be useful to a better understanding of the nature of the biological bases of affect (Davidson et al., 2000). Further, the ability to track temporal dynamics of emotion and emotion regulation may have consequences for the burgeoning field of clinical neuroscience. It is known that individuals with depression and other forms of psychopathology display abnormalities in the time course of their emotional responses. The ability to track abnormalities in the temporal dynamics of affect simultaneously with fMRI may allow for a more sensitive and robust clinical neuroscience.
Some limitations of this study should be noted. Corrugator EMG was not acquired outside the scanner. Within the same participants, it would be helpful for future work to acquire corrugator EMG both inside and outside the scanner. This would allow examination of the relationship between corrugator EMG acquired both inside the scanning environment. In addition, with this method, corrugator EMG is not examined continuously – it is only examined between EPI volume acquisitions, and therefore some data are lost in the process. It may be possible with future work to mathematically remove the electromagnetic noise induced by the scanner leaving a continuous EMG signal intact. However, given that current alternative approaches, such as fear potentiated startle, require the use of an obtrusive loud auditory stimulus, and that fear potentiated startle only yields a single point-estimate of affect, we believe that this approach is in many ways superior.
Lastly, it is thought that the examination of individual differences in emotion and emotion regulation is of fundamental importance to affective neuroscience (Davidson et al., 2002). With the development of methods to continuously, unobtrusively and objectively measure affect in the scanner, we can now examine how these individual differences in facial physiology are reflected in brain activity. These methods thus have import for addressing long-standing questions about the neural bases of individual differences in affective responding and emotion regulation. Moreover, the ability to objectively measure the time course of emotion in patients with psychopathology will be helpful in understanding abnormalities in emotion regulation in such patients.
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Author Contributions:A.S.H. conceived, designed, implemented, analyzed the data, and prepared the manuscript; L.L.G. analyzed the data; A.H. coordinated the project and prepared the data; M.J.A. contributed to the design of the setup; R.J.D. conceived and supervised the project.