|Home | About | Journals | Submit | Contact Us | Français|
Schizophrenia patients display impaired performance and brain activity during facial affect recognition. These impairments may reflect stimulus-driven perceptual decrements and evaluative processing abnormalities. We differentiated these two processes by contrasting responses to identical stimuli presented under different contexts. Seventeen healthy controls and 16 schizophrenia patients performed an fMRI facial affect detection task. Subjects identified an affective target presented amongst foils of differing emotions. We hypothesized that targeting affiliative emotions (happiness, sadness) would create a task demand context distinct from that generated when targeting threat emotions (anger, fear). We compared affiliative foil stimuli within a congruent affiliative context with identical stimuli presented in an incongruent threat context. Threat foils were analysed in the same manner. Controls activated right orbitofrontal cortex (OFC)/ventrolateral prefrontal cortex (VLPFC) more to affiliative foils in threat contexts than to identical stimuli within affiliative contexts. Patients displayed reduced OFC/VLPFC activation to all foils, and no activation modulation by context. This lack of context modulation coincided with a 2-fold decrement in foil detection efficiency. Task demands produce contextual effects during facial affective processing in regions activated during affect evaluation. In schizophrenia, reduced modulation of OFC/VLPFC by context coupled with reduced behavioural efficiency suggests impaired ventral prefrontal control mechanisms that optimize affective appraisal.
Patients with schizophrenia have deficits in identifying affective facial intent, and these deficits relate to negative symptom severity (Gur et al., 2007) as well as global outcome (Brekke et al., 2005). These affective evaluation deficits have been attributed to abnormalities in affective processing neurocircuitry in limbic and frontotemporal regions (Gur et al., 2007). However, abnormalities in basic visual processing (Butler et al., 2001) or in the ability to integrate visual information into visual objects (Doniger et al., 2002) may also contribute to affective identification deficits in schizophrenia (Leitman et al., 2005, 2008; Das et al., 2007; Fakra et al., 2008). Studies using backward-masking paradigms have suggested that schizophrenia patients have automatic or implicit processing deficits in facial affect detection, linked to subcortical dysfunction (Das et al., 2007). Most fMRI studies examining deficits in facial affect processing (Phan et al., 2002; Gur et al., 2002; Murphy et al., 2003; Baas et al., 2004) employ standard affect identification paradigms. These paradigms make it difficult to asses whether activation abnormalities in prefrontal cortex (PFC) executive regions indeed reflect independent deficits in the controlled evaluation of facial affect, or instead are purely stimulus driven, reflecting a cascade of dysfunction stemming primarily from basic sensory/perceptual disturbances.
A study by Gur et al. (2007) employed a hybrid (block and event-related) paradigm in which subjects were asked to identify a target emotion within a series of non-target foils that were themselves the targets of ensuing blocks. This design permits us to consider the impact of task demands on affective appraisal, and ask whether dysfunction reported in PFC and associated with evaluation is independently present when sensory/perceptual processes are held constant. We hypothesize that task instructions to identify the emotions of fear and anger create an affective context for the detection of threat (TH) within anger and fear target blocks, while instructions to identify happiness and sadness create a distinct affiliative (AF) context in their respective blocks. While happiness and sadness differ in terms of positive and negative valence, they are both considered ‘AF’ emotions, as they serve to increase inter-personal empathy (Miller and Eisenberg 1988; Eisenberg et al., 1989; Knutson 1996; Hess et al., 2000) and strengthen social bonds (Lewis et al., 2008). Given that the same stimuli were used as foils across blocks, contrasting presentations of TH foil stimuli within TH blocks (context congruent) with identical stimuli within AF blocks (context incongruent) could provide an estimate of the contextual influence on prefrontal activity. This effect of context should be unaffected by sensory aspects of affect processing such as facial feature perception and integration because the stimuli themselves are exactly the same.
We hypothesized that context incongruities would lead to increased activity within VLPFC and OFC. Ochsner and colleagues (2005; Wager et al., 2008) suggest that VLPFC is central to the cognitive regulation of emotion and the affective appraisal of stimuli. Other studies have identified VLPFC (Haxby et al., 2000; Mobbs et al., 2006; Guyer et al., 2008) and OFC (Haxby et al., 2000) as involved in affective evaluation and influenced by contextual framing effects. Patients display deficits in explicit emotion processing (van’t Wout et al., 2007) and fail to integrate contextual cues when making social judgments (Green et al., 2007, 2008); therefore, we predicted that schizophrenia patients would have reduced activation to incongruence in these regions, relative to controls. This is the first study to examine the effects of context on affective facial appraisal in schizophrenia, where the possible sensory processing antecedents of affective facial recognition are held constant.
This analysis is based on data collected in a previously published study (Gur et al., 2007). Therefore, the characterization of the studied population, image acquisition parameters and image analysis details are briefly summarized here.
The original sample included 16 patients (12 men), who met DSM-IV criteria for schizophrenia or schizoaffective disorder, and 17 healthy controls (12 men). As described in Table 1, the patients were somewhat older on average (t2,31 = 2.73, P = 0.011), and, as expected, had lower education (t2,31 = 3.72, P = 0.0008). However, they had comparable parental education (t2,31 = 1.95, P = 0.061). At the time of imaging, all patients, except one unmedicated patient, were on stable doses of antipsychotics: two received first-generation (CPZequiv = 542 ± 292/day) (Davis, 1976), 11 second-generation (OLZequiv = 18.2 ± 2.8/day) and two both [CPZequiv = 16.7/day, OLZequiv = 11.3/day (Kohler et al., 2003)]. After complete description of the study, written informed consent was obtained. Clinical ratings are detailed in Table 1.
The face emotion identification task included four conditions, presented in a counterbalanced order, each with a specific target facial expression: happy, sad, anger or fear (Figure 1). Each condition included four 90-s blocks of emotion identification, separated by 24 s of rest during which a scrambled face with a central cross-hair for fixation was displayed. Each 90-s identification block contained 8 target faces (e.g. 8 fear), 12 foil faces (e.g. 4 happy, sad and 4 angry) and 10 neutral faces. Thus, a condition included a total of 120 faces: 32 targets, 48 emotional foils and 40 neutral foils in a pseudorandom sequence. Faces appeared for 3 s and participants endorsed ‘target’ or ‘other’ using the two-button response pad. Abbreviated response instructions remained visible throughout the task. The same faces were cycled through the four conditions serving as targets or foils depending on the condition. Each condition (time series) lasted 8 min with total task duration ~32 min.
Detailed image acquisition and processing methods were described previously (Gur et al., 2007). Briefly, data were acquired on a 4T gradient-echo (GE) Signa Scanner (Milwaukee, WI, USA), employing a quadrature transmit and receive head coil. Structural images consisted of a sagittal T1-weighted localizer, followed by a T1-weighted acquisition of the entire brain in the axial plane (24-cm FOV, 256 × 256 matrix, resulting in voxel size of 0.9375 × 0.9375 × 4 mm). This sequence was used for spatial normalization to a standard atlas (Talairach, 1988) and for anatomic overlays of the functional data. Functional imaging was performed in the axial plane using a 16-slice, single-shot GE echo-planar sequence (TR/TE = 1500/21 ms, FOV = 240 mm, matrix = 64 × 40, slice thickness/gap = 5/0 mm). This sequence delivered a nominal voxel resolution of 3.75 × 3.75 × 5 mm. The 5-mm slice thickness was a compromise to permit optimal visualization of the amygdala with minimal sacrifice in brain coverage. Total slices per volume were also limited by a 1.5-s TR that was selected to provide two volume acquisitions per stimulus exposure (3 s per face). The slices were acquired from the superior cerebellum up through the frontal lobe. Inferiorly, this corresponded to a level just below the inferior aspect of the temporal lobes and superiorly to approximately the level of the hand-motor area in the primary motor cortex.
Gradient echoplanar images can be degraded in the presence of non-uniform magnetic fields. Therefore, shimming was performed manually in a region of interest (ROI) containing the anterior medial temporal lobe (Webb and Macovski, 1991). After shimming, pilot echoplanar images were obtained and these images were visually inspected for quality prior to fMRI acquisition. Following this inspection, images were corrected for residual geometric distortion (Jezzard and Balaban, 1995) based on a magnetic field map acquired with a 1-min reference scan.
Analysis of Affective Foils. Analyses were limited to foil stimuli within blocks. Target stimuli were excluded from the analysis because they confounded the assessment of task-driven context effects. Analysis of targets was presented previously (Gur et al., 2007). Analysis of behaviour examined detection efficiency (integrating both accuracy and reaction time) using Multivariate Analysis of Variance, with factors for stimulus type, context and group. Efficiency was defined as:
where Z reflects Z-transformed scores relative to the healthy controls’ performance across all conditions (Gur et al., 2001). This efficiency measure was calculated for the contrasts of interest described below. Selected fMRI contrasts reflect blood oxygen level dependent (BOLD) signal change in response to emotional face foils and cross-hair. For each time series, in addition to the foil regressor of interest, neutral foils and target faces were modelled but were not of interest here. Our analysis focused on the effects of context by contrasting identical stimuli under differing task demands. This classification resulted in four new conditions: (i) AF stimuli (happy and sad foils) in AF context—i.e. happy or sad target conditions (AFAFcontext), (ii) AF stimuli in TH context—i.e. fear and anger target conditions (AFTHcontext), (iii) TH stimuli in TH conditions (THTHcontext) and (iv) TH stimuli in AF conditions (THAFcontext). Note that this categorization resulted in 32 stimuli in context-congruent conditions (,AFAFcontext, THAFcontext) and 64 stimuli in context-incongruent conditions (THAFcontext, AFTHcontext). The potential difficulty in contrasting numerically different categories, however, is offset by the large number of stimuli sampled. Prior study within our lab has illustrated that contrasts containing much lower numbers of stimuli than 32 yield stable estimates of condition activation. These data were submitted to a mixed effects model, containing factors for hemisphere [right hemisphere (RH), left hemisphere (LH)], stimulus type [STIM (AF or TH)] and target condition type [CONTEXT (AFcontext or THcontext)], as well as a between groups factor for diagnosis (GROUP).
We hypothesized that activation to affective stimuli would vary as a function of context and that such variation in activation would be reduced in schizophrenia patients. We thus were primarily interested in STIM × CONTEXT and STIM × CONTEXT × GROUP interactions. Other complex interactions were beyond our scope and hence not tested. For pairwise contrasts, consistent with our hypothesis, we tested AFAFcontext vs AFTHcontext and THTHcontext vs THAFcontext within both hemispheres, and within and between groups. This analysis was conducted offline using percent signal change data extracted from our a priori ROIs in VLPFC/OFC.
We first constructed a VLPFC/OFC structural ROI (comprising Brodmann’s areas 11 and 47 bilaterally) from the Wake forest university pickatlas (Maldjian et al., 2003). This ROI was further constrained by a functional mask of areas showing robust activation (P < 0.00005 uncorrected) to all foil stimuli vs cross-hair across both groups. The resulting ROIs contained 986 and 586 voxels (2 × 2 × 2 mm voxel dimensions) for RH and LH, respectively (Figure 2). Given that our estimate of activation (see below) incorporates the number of activated voxels, we decided to examine laterality differences only when they significantly interacted with diagnosis. An exploratory whole brain voxelwise analysis was also conducted to examine contrast activation that occurred outside our ROIs.
Subject-level time series statistical analysis was carried out using functional magnetic resonance imaging of the brain (FMRIB)'s improved linear model with local autocorrelation correction (Woolrich et al., 2001). A second-level within-subject fixed effects analysis across all four blocks was then conducted for each subject. The resulting single-subject contrast estimates were then submitted to a third-level between-subjects (group) analysis employing FMRIB's local analysis of mixed effects (Beckmann et al., 2003), which models inter-session or inter-subject random effects components of the mixed-effects variance using Markov chain Monte Carlo sampling to estimate the true random effects variance and degrees of freedom at each voxel (Woolrich et al., 2004). Statistical significance was based on both voxel height and spatial extent in the whole brain, using Analysis of Functional NeuroImages AlphaSim to correct for multiple comparisons by Monte–Carlo simulation (10 000 iterations, voxel height threshold P < 0.01 uncorrected, cluster probability P < 0.01). This whole-brain correction required a minimum cluster size of 294 2 × 2 × 2 voxels.
Finally, for our measure of activation, we used ‘energy’ which takes into account both the magnitude and spatial extent of the activation (Gur et al., 2007). This index is calculated as:
Repeating the analysis using mean percent signal change or spatial extent separately yielded comparable results to those obtained using energy. Statistical analyses used a two-tailed alpha criterion of P < 0.05, except where noted otherwise. Statistical analyses were conducted using Statistical Analysis Software (Gary, Indiana).
A multivariate analysis of efficiency, which incorporates both the accuracy and speed of affective foil detection, indicated that patients were overall less efficient than healthy subjects (GROUP: F1,31 = 5.8, P = 0.022) (Figure 2, see Supplementary Table 1 for accuracy and reaction time data). Across groups, efficiency was reduced for all stimuli presented in TH contexts (CONTEXT: F1,31 = 12.1, P = 0.002), but no overall difference for stimulus type was observed (P = 0.83). Critically, a two-way interaction between STIM and CONTEXT was observed (F1,31 = 30.4, P < 0.0001), indicating that activation to foil stimuli varied as a function of both stimulus type and context. However, neither GROUP × STIM nor GROUP × CONTEXT interactions were observed (all P > 0.13). A three-way interaction of GROUP × STIM × CONTEXT was at a trend level (P > 0.07).
An examination of our a priori contrasts of interest, however, revealed the following: when contrasting identification performance for AF foils in an incongruent TH context (AFTHcontext) vs identical AF foils in a congruent AF context(AFAFcontext), patients showed a decrement in efficiency nearly twice the magnitude of that seen in controls. [Controls: 0.70 ± 0.19; Schizophrenia: 1.30 ± 0.27 (F1,31 = 3.2, P < 0.04 one-tailed)]. No such efficiency differences were observed in contrasting TH foils presented in incongruent AF contexts (THAFcontext) as compared with identical TH foils in a congruent TH context (THTHcontext) (P = 0.56).
Overall, our model fit was significant (χ35 = 288.6, P < 0.0001) with patient activation significantly lower than healthy controls across all affective foil stimuli (GROUP: F1,31 = 6.9, P < 0. 01; Figure 3).
Across groups and conditions, activation was higher in RH (HEM: F1,31 = 46.2, P < 0.0001). However, given that our ROI size varied across hemispheres main effects of hemisphere are difficult to interpret. On the other hand, schizophrenia patients did display significantly reduced RH laterality in activation as compared with healthy controls (GROUP × HEM interaction: F1,31 = 11.6 P = 0.0019). No significant main effects for either STIM (P = 0.27) or CONTEXT (P = 0.09) were observed, indicating that activation did not vary as a function of the stimulus type or context alone.
Consistent with our a priori hypothesis, we found a STIM × CONTEXT interaction (F1,31 = 28.2, P < 0.0001), a STIM × CONTEXT × GROUP interaction (F1,31 = 5.7 P = 0.027) and an HEM × STIM × CONTEXT × GROUP interaction (F1,31 = 4.7, P = 0.038). These interactions revealed that activation to foil stimuli varied as a function of both context and hemisphere, and that this variation differed for patients and controls. Specifically, within RH (but not LH), healthy subjects’ activation to AF foils in an incongruent TH context (AFTHcontext) was higher than activation to identical AF foils in a congruent AF context (AFAFcontext) (t1,31 = −3.49, P = 0.0015). However, we observed no significant difference in THAFcontext vs THTHcontext activation (P = 0.09).
In contrast to controls, patients’ activation to stimuli displayed no significant modulation by context within either hemisphere (P > 0.11), explaining the significant STIM × CONTEXT × GROUP interaction. Finally, whereas in RH patient activation was lower than controls for all foil conditions (all P < 0.01) except for AFAFcontext (P > 0.14), for LH no significant differences were seen in any of the four conditions (all P > 0.1).
Our previously published analysis of target emotions (Gur et al., 2007) revealed correlations of flat affect severity and BOLD responses. We therefore examined correlations between negative symptom severity and indices of the impact of context on foil activation defined as AFincong: AFTHcontext − AFAFcontext and THincong: THAFcontext − THTHcontext. within RH. Negative symptom severity was measured on the scale for the assessment of negative symptoms (SANS) (Andreasen, 1984). We found a significant correlation between global scores of affective flattening and RH THincong (rs = −0.52 P = 0.04). No other correlations were significant. There were no significant correlations between anti-psychotic medication dose or subject age and behaviour or activation values (all P > 0.31).
Psychologists and cognitive neuroscientists have often found it useful to fractionate cognitive processes into sensory-perceptual components and evaluative–executive components. While most cognitive processes, such as affective appraisal, undoubtedly result from the interaction of both components, isolating the neural underpinnings of these components can prove challenging in neuroimaging experiments. For clinical neuroscientists who wish to identify the locus of a specific neurocognitive abnormality, examining processes such as facial affect in terms of evaluative and executive processing vs sensory and perceptual processing can be especially informative. Within schizophrenia, a leading hypothesis attributing neurocognitive deficits to dopaminergic-based hypofrontality favours evaluative and executive explanations (Weinberger and Berman 1988; Carter et al., 1998; O'Reilly et al., 2002; Bach et al., 2008; Phillips et al., 2008). In contrast, glutamatergic (Javitt, 1996) or GABAergic (Lewis and Moghaddam, 2006) hypotheses emphasize more widespread neural dysfunction that also encompasses basic sensation and perception processes. Studies exploiting the high temporal resolution of event-related potentials (ERP) have indicated that facial affect perception is associated with reductions in early visuosensory components such as P1 and N1, while other studies found reductions only for latter stage ‘integration’ components such as the N170 and N250 (see Turetsky et al., 2007 for review). The presence of early ERP abnormalities in schizophrenia suggests basic sensory deficits, without ruling out the possibility that additional evaluative–executive deficits also contribute to impairment in facial affect identification. Studies by Van’t Wout and colleagues (2007) show that patients’ recognition of emotions such as fear is significantly impaired in explicit but not implicit emotion processing tasks. This disjunction suggests that task demand and affective evaluation may reflect impairment beyond pure sensory encoding of faces and their expressions. Similarly, studies by Green and colleagues (2007, 2008) examining gaze direction patterns have suggested that patients have difficulty integrating contextual information when making emotional and social judgments. Such affective evaluative impairment may reflect a more general executive impairment in utilizing context that has been long linked to frontal hypofunction (e.g. MacDonald et al., 2005).
The low temporal resolution of fMRI can make it difficult to disentangle relative contributions of sensory and executive processing abnormalities to affective appraisal. However, the current study’s hybrid (block and event-related) design afforded us the opportunity to examine evaluation aspects of face processing while holding sensory and perceptual effects (stimulus characteristics) constant. We compared identical faces that were foils in an emotion identification experiment under differing task demand contexts, hypothesizing that blocked trials in which subjects were asked to detect happy and sad emotions would form an AFcontext, while anger and fear target blocks would form a TH context. Prior research into the effects of context on affective evaluation had implicated VLPFC and OFC brain regions (Haxby et al., 2000; Mobbs et al., 2006; Guyer et al., 2008), hence we focused our analysis on this system.
Our finding of reduced ventral PFC activation to foils in schizophrenia is consistent with prior studies of affective appraisal, and more general findings of hypofrontality in the illness. Consistent with our hypothesis, we found that incongruent AF foils in the context of TH conditions produced less efficient behavioural responses and also elicited greater bilateral activation in VLPFC–OFC compared with their identical context-congruent counterparts. These finding suggest that task demands of TH detection exert significant contextual effects on affective evaluation. Within RH, this context modulation was present in healthy subjects but not in schizophrenia patients. Behaviourally, patients showed a nearly 2-fold greater reduction in efficiency than healthy subjects within the AFTHcontext − AFTHcontext contrast. Together, these behaviour and imaging findings illustrate abnormal evaluative processing deficits that are likely not directly attributable to sensory integration deficits. Both behavioural and activation effects were not related to medication dosage, indicating no direct role of antipsychotic medication on the observed context effects. Finally, in the patient group, reduced modulation of VLPFC–OFC activation to TH foils by incongruent vs congruent context was associated with greater affective flattening. Although no overall group differences were seen in this TH foil contrast, the symptom correlation suggests that patients with flat affect may be less likely to effectively employ contextual cues when appraising threatening facial stimuli under AF conditions.
A post hoc whole brain analysis (Supplementary Figure 1) of activation to AF foils in an incongruent TH context (AFTHcontext) vs activation to identical AF foils in a congruent AF context (AFAFcontext) revealed only two clusters that reached our statistical significance criteria. These clusters substantially overlapped with our a priori VLPFC/OFC ROI, yet the activation also extended to more dorsal aspects of PFC. In contrast, patients displayed only slight and subthreshold activation clusters in this contrast. No significant difference in activation was observed in the THAFcontext vs THTHcontext contrast within either group.
This study highlights the interpretative limitations of block designs, which are commonly employed in examining facial affect in clinical populations. Blocks with different task demands may induce context effects that alter response to otherwise identical stimuli, and group differences in block activation could reflect either context or stimulus effects. Thus, abnormal activation in schizophrenia during block-design tasks could reflect higher level deficits in context processing, while typically being interpreted as differences in response to stimulus features.
Our analysis of affective foils indicated that patients had reduced activation within our ROI to facial stimuli in general. This is consistent with prior work suggesting that schizophrenia patients have core deficits in face perception that extend beyond affective appraisal (Hooker and Park 2002; Leitman et al., 2008). The presence of these deficits in our sample indicate that despite our comparison of identical stimuli under differing contexts, we cannot completely rule out the possibility that the absence of context effects on the appraisal of AF stimuli seen in patients reflects an interaction between stimulus-driven sensory dysfunction and controlled evaluative processing deficits. Future studies directly accounting for differences in sensory–perceptual processing will be needed to settle this question definitively.
There are several limitations to our study. Task demands in the current study likely produce only weak contextual effects. Stronger contextual effects, such as those imposed in ‘correspondence bias’ and contextual framing paradigms, may induce even more robust changes in VLPFC/OFC. No significant difference within this ROI was found for incongruent TH foils in AF blocks vs their identical congruent counterparts. This disjunction suggests that TH-detection task demands create stronger contextual effects than those produced by task demands emphasizing affiliation. It is also possible that anger and fear are more closely aligned dimensionally in terms of TH than happiness and sadness are in terms of affiliation. Future studies should look at each emotion separately to explore emotion-specific contextual effects. Functional connectivity analysis could also help examine how context modulates interactions between frontal evaluative regions, amygdala and other neural nodes in the affective appraisal circuit. We did not directly assess subjective emotional responses to face stimuli, so we cannot rule out the possibility that patients experiences of stimuli as AF or threatening may vary somewhat from control subjects. We think this is unlikely to explain our context results, as prior studies indicate that schizophrenia patients have qualitatively similar subjective emotional responses to laboratory emotional stimuli (Kring et al., 1993, 1999; Kring and Neale, 1996); however, future studies should directly assess the afiliative vs threatening judgments in patients.
In daily life, affective appraisal takes place within situational contexts. Such contextual effects substantially shape memory encoding and recall, in some cases dramatically (Loftus and Pickrell, 1995; Loftus and Mazzoni, 1998). Context can also alter the perceptual threshold of stimuli, rendering detectable previously subthreshold stimuli (Cox et al., 2004; Bar et al., 2006). Such contextual information impacts directly on affect appraisal through executive control linked to VLPC–OFC (Haxby et al., 2000; Adolphs, 2002). Executive processing may facilitate affective appraisal by improving the efficiency and accuracy of TH prediction, for example, by constraining search within memory systems (Sahakyan and Kelley, 2002; Mobbs et al., 2006). Our study documented task-driven contextual effects on VLPFC–OFC processing of facial affect that were reduced in patients with schizophrenia. This suggests that patients may have difficulty utilizing prefrontal control mechanisms that optimize affective appraisal.
Supplementary data are available at SCAN online.
This work was supported in part by National Institute of Mental health (NIMH) grant MH060722 MH019112, and by National Alliance for Research on Schizophrenia and Depression NARSAD Young investigator Awards (to D.I.L. and D.H.W.). The authors thank Dr Amy Pinkham and Kosha Ruparel for their suggestions, and Dr Warren Bilker for his help with the statistical analysis.