Search tips
Search criteria 


Logo of jpnSubmit a ManuscriptEmail AlertsAbout JPNJournal of Psychiatry and Neuroscience
J Psychiatry Neurosci. 2010 May; 35(3): 185–194.
PMCID: PMC2861135

Multisensory integration of emotionally valenced olfactory–visual information in patients with schizophrenia and healthy controls



Patients with schizophrenia frequently have deficits in social cognition, and difficulties in the discrimination of emotional facial expressions have been discussed as an important contributing factor. We investigated whether this impairment is aggravated by difficulties relating the observed facial expression to contextual information, as is often provided by emotionally valenced crossmodal stimulation.


We investigated the effects of odorant primes on the accuracy and speed of emotional face recognition. Healthy controls and patients with schizophrenia were exposed to 2-second odorant stimuli: vanillin (pleasant), ambient air (neutral) and hydrogen sulfide (unpleasant). The odours were followed by an emotional face recognition task, in which participants determined if a face showed happiness, disgust or neutral affect.


Controls showed improved performance in the categorization of disgusted faces after all types of odour stimulation irrespective of the emotional valence. However, in controls, the response time for happy faces was slower after presentation of any odour. Schizophrenia patients showed an attenuated effect of olfactory priming on disgust recognition, which resulted in the increased performance differences between the groups. This effect was particularly strong for the unpleasant odour.


The study design did not allow us to fully differentiate between the effects of perceived odour intensity and valence. A possible contribution of cognitive deficits on the observed effects should be investigated in future studies.


Our results provide novel evidence for a special connection between the presentation of odorant cues and the accuracy of recognition of disgusted faces in healthy controls. This recognition advantage is disturbed in patients with schizophrenia and appears to contribute to the observed deficit in emotional face recognition.


The ability to recognize affect in the faces of others is an important component of social cognition. Deficits in this area are well established in schizophrenia and have been shown to be associated with symptom severity and neuropsychologic deficits.1,2 An increasing body of evidence indicates that at least part of the impairment is domain specific and is not secondary to a generalized cognitive deficit,3,4 but this suggestion is not accepted unanimously without contradiction (see Johnston and colleagues5 for an alternative viewpoint). A number of recent studies have identified structural and functional abnormalities specific to affective brain networks in schizophrenia; these networks appear to play a crucial role in the observed dysfunction.6,7

The majority of studies investigating processing of emotional facial expressions have presented face stimuli without contextual information. However, in natural settings, facial expressions are commonly preceded by events that trigger the emotional response. This process often engages several sensory channels simultaneously; for example, when a person wrinkles their nose at the presence of a pungent smell, an observer makes a connection between these 2 events, concluding that one is the cause of the other. Recent evidence shows that such a close succession of events considerably improves recognition performance in healthy controls relative to a single face stimulus and across different modalities.810 These mechanisms have been shown to be of high ecological relevance, influencing the perception of complex social scenes.11,12 Neuroimaging studies suggest that sensory convergence in higher brain structures plays a key role in relating different sensory stimuli to a common reference frame. A number of authors have proposed the existence of an adaptive system in which perceptual sensitivity in one modality can be modulated by input from another.13,14

Few studies have investigated these mechanisms in patients with schizophrenia. Evidence has been presented for impaired audiovisual integration in an emotional face categorization task,15,16 suggesting a deficit in automatic perceptual integration processes at the co-occurrence of crossmodal stimuli. To what extent these effects generalize to other modalities and stimulus types remains unclear. In light of substantial evidence that patients with schizophrenia have deficits in crossmodal integration in the cognitive domain,17 further investigation of such problems in emotional processing is warranted. We sought to address these questions using an emotionally valenced olfactory prime as a cue for an emotional face discrimination task.

A substantial body of research has shown that olfactory stimuli elicit emotions quickly and reliably because of the high degree of overlap between the brain structures involved in olfactory and emotional processing.1820 Interestingly, mood induction through odorants, when presented at suprathreshold intensities, has also been shown to modulate behaviour in patients with schizophrenia.21 This is despite the fact that an increasing body of research reports that patients show declarative deficits in olfactory functioning, affecting identification and hedonic judgment.2224

The aim of our study was to investigate whether cross-modal effects between olfactory and visual stimuli are impaired in patients with schizophrenia compared with healthy controls. We expected that, in line with previous studies,10 the odorants would influence the perception of emotional faces in the healthy participant group. A Stroop-like interaction pattern was expected, in which participants would show improved performance when matched face and odour valence were presented and impaired performance when conflicting face and odour valence were presented. On the basis of previously reported impairment in audiovisual integration of affective and cognitive stimuli, we further hypothesized that patients would be less affected by a crossmodal priming cue, showing less impairment during incongruent trials and less improvement during congruent trials.



In total, 24 patients with schizophrenia, all in- and outpatients of Rheinisch-Westfälische Technische Hochschule (RWTH) Aachen University Hospital, Germany, and 24 healthy volunteers participated in the study. All participants had normal or corrected to normal vision. They were matched by sex, age and parental education (Table 1).

Table 1
Sociodemographic and neuropsychological characteristics of patients with schizophrenia and healthy controls

All patients had a DSM-IV diagnosis of schizophrenia as determined by the Structured Clinical Interview for DSM disorders (SCID, German version25). Symptom severity was assessed with the Positive and Negative Syndrome Scale (PANSS26). The average PANSS scores for our group were slightly below the published norms on all subscales (Table 1). Both the SCID and PANSS were administered by experienced and clinically trained psychologists. Of the patients, 22 were taking atypical neuroleptics, 1 was taking both typical and atypical medication and 1 was unmedicated. Eight patients were receiving serotonin reuptake inhibitors in addition to neuroleptic treatment. We converted the medication dosages into chlorpromazine-equivalents as a means of comparing therapeutic doses across medications.

All participants completed a neuropsychological test battery that was administered by a qualified psychologist (J.S.). One control participant performed very poorly (2 standard deviations below average) on the face recognition task. When debriefed, this participant reported having allergies and stated this interfered with his ability to perform the task. We therefore replaced this participant with an equally matched healthy control.

All participants were administered a detailed screening questionnaire before enrolment in the study. We excluded patients and healthy controls with a lifetime diagnosis of addiction and current substance abuse as well as further psychiatric or neurologic comorbidities. We also excluded participants with a history of head trauma or injuries to the central nervous system, including loss of consciousness. We excluded healthy controls who had a first-degree relative with a psychiatric disorder. To control for possible confounding cognitive impairment, we administered a number of neuropsychological tests. Patients did not differ from controls in estimated premorbid intelligence as measured by a German version of a multiple choice vocabulary test, the Mehrfachwahl-Wortschatz-Intelligenztest (MWT-B27); both groups scored above the mean of the control sample. Patients performed worse than controls in measures of current functioning as assessed by the digit span and trail making tasks; and whereas the majority of patients showed impaired performance relative to published norms on the digit span, neither group performed below the control sample for the trail-making tasks (TMT). Also, an olfactory screening with the Sniffin’ Sticks test,28 a multiple forced-choice odour identification task, was performed to measure olfactory identification abilities. In line with research indicating impairment of odour discrimination abilities in schizophrenia, patients performed worse than controls (Table 1). Whereas only 1 healthy control had hyposmia, 5 of the schizophrenia patients had hyposmia. No participant from either group met the criteria for anosmia.

This protocol was approved by the institutional review board of the medical faculty of RWTH Aachen University and conformed to the ethical guidelines established by the Declaration of Helsinki (6th revision). Written informed consent was obtained from all participants after the procedures had been fully explained. A clinical interview was administered by an experienced clinician to determine if any of the patients were in an acute phase of illness at the time of testing. The patients’ cognitive abilities were assessed as part of the interview to determine that they were all capable of giving informed consent.

Olfactory stimulant delivery

Stimuli were delivered unirhinally to the right nostril in a standardized manner using a Burghart OM4 olfactometer operating at a constant temperature of 40°C. The odours were humidified to prevent any thermal irritation and drying out of the nasal mucosa. Airflow totaled six 1-minute deliveries (four 1-min deliveries of odorant and two 1-min deliveries of humidified air). This was held constant during the interval between pulses (six 1-min deliveries of humidified air).

Given the frequently observed impairment in odour assessment in schizophrenia, special attention was paid to the selection and presentation of odorant stimuli. We prepared the vanillin odorant by dissolving 1 g vanillin powder in 10 mL propylene glycol. The hydrogen sulfide (H2S) odorant consisted of hydrogen sulfide in nitrogen at a concentration of 20 ppm. We chose these odorants to minimize trigeminal impact because they have been shown to rarely be detected by people with anosmia who have intact trigeminal perception29,30 and to be clearly distinguishable from one another in their hedonic properties.31

The odours were presented at intensities that were clearly above threshold and differed enough in quality to require minimal odour discrimination skills. Upon completion of the task, each participant rated the intensity and pleasantness of each odorant on 9-point pictorial rating scales adapted from the Self-Assessment Manikin (SAM) Scale.32 The pleasantness scale consisted of 5 figures ranging from a widely smiling figure (pleasant pole) to a low-spirited one with the corners of the mouth pointing downwards (unpleasant pole); the middle figure indicated a neutral pleasantness. Rectangles located between the figures allowed participants to pick a valence level between 2 figures. The intensity scale consisted of 5 figures ranging from a very small figure indicating low intensity, to a very large figure indicating strong intensity. Again, the participants were able to pick rectangles between the figures. We did not use the arousal scale. For one healthy control, intensity and pleasantness ratings could not be obtained; therefore, we analyzed the data for only 23 controls and 24 patients.

Facial stimuli

The stimuli consisted of colour photographs of actors (50% women). The procedure for acquisition of these images has been described previously.6 To increase the difficulty of the task, we selected only a subset of mild-intensity facial expressions.

For the present study, we validated a subset of 50 happy, 50 disgusted and 50 neutral faces for accuracy of emotion recognition on a sample of 40 German participants. The validation procedure used the Emotional Self-rating Scale (ESR33) to describe the intensity of 5 specific emotions (happiness, anger, fear, sadness, disgust) on a 5-point unipolar intensity scale. Participants had to indicate to what extent they could recognize each of the 5 emotions in each of the faces. A value had to be given for each emotion to obtain a measure of discriminability. To reduce the ambiguity of the face stimuli, any face was excluded that did not achieve a minimum of a mean 1.5 point difference between the target emotion and the next-most likely emotion. Among the remaining face stimuli, we chose to include 30 happy, 30 disgusted and 30 neutral faces taken from the 43 depicted individuals that achieved the highest mean ratings for intensity of emotion expression. Familiarity was balanced across individuals, with 39 actors shown for 2 emotional facial expressions and 4 appearing for all 3 expressions. The included images were balanced for sex. Because all participants were white, we only included pictures of white faces to avoid effects of ethnicity.


The participants were seated 70 cm from a 150-inch IBM computer screen with the olfactometer’s nosepiece inserted into their nostril. They were instructed to breathe normally. The experiment was run at a resolution of 800 × 600 pixels. The visual angle of the face stimuli was 8.2° × 6.5°. Each trial started with a 2000 ms olfactory stimulus (vanillin, H2S or ambient air) followed by a 1000 ms interstimulus interval. During odour presentation and interstimulus interval, participants were shown a fixation cross. Then, an emotional face stimulus appeared for 4000 ms (Fig. 1). Participants were asked to identify the emotion as quickly and accurately as possible by pressing a button. The button layout varied across participants to avoid a button position bias; 3 different button sequences were used (disgust, neutral, happy; happy, disgust, neutral; and neutral, happy, disgust). Each odorant was paired with each emotional expression, resulting in 9 conditions with 20 trials each (180 trials total). This meant that every picture of our 90-picture set was presented twice to each participant during the course of the experiment. The trials were presented in a pseudorandom order, which was balanced for appearance of male and female faces per odorant. Also, no actor appeared more than twice with the same odorant, no more than 3 consecutive trials presented the same odorant, facial emotion or sex, and no consecutive trials showed an identical individual.

Fig. 1
Illustration of the experimental procedure. Three consecutive trials are shown (incongruent–unpleasant odour, congruent–neutral odour, incongruent–pleasant odour). During each trial, participants were presented with an odorant ...

Statistical analyses

We converted the intensity and valence ratings from pictures to values between 0 and 9 for the intensity ratings and −4 and +4 for the valence ratings. A 2 × 3 repeated-measures analysis of variance (ANOVA) was then performed, with group (control v. patient) as the between-subject factor and odorant type (hydrogen sulfide, vanillin, ambient air) as the within-subject factor. Sphericity tests showed that the data met the requirements for the assumption of equal variance between the repeated-measurement factors. Subsequently, we analyzed significant effects using Bonferroni-corrected Student t tests.

The analysis of our 2 main target parameters, emotion recognition accuracy and reaction time, required adjustments for distribution asymmetries. We chose 2 different parametric models to reduce the influence of skewness on the results.

We used conditional logistic regression to model the odds of correct recognition of facial expressions (happy, neutral, disgust) and how this was influenced by odour (pleasant, neutral, unpleasant) and group membership (patient, control). Nonresponses comprised less than 1% of trials overall, ranging from 0% to 1% for the controls and 0% to 5% for the patients. We chose to exclude nonresponses instead of counting them as wrong responses for methodological reasons. This decision did not affect our interpretation of the statistical results. We first modelled all 3 main effects of interest, all 2-way interactions and a 3-way interaction between emotion, odour and diagnosis. Although there was a group difference for the Sniffin’ Sticks score, there was no complete separation between the distributions. We therefore decided to use the Sniffin’ Sticks score as a covariate. To analyze significant between-group differences, we performed Bonferroni-corrected χ2 tests. We log-transformed the reaction times to approximate a normal distribution of responses.

We used a mixed effects linear model to assess the differences in reaction times. The model used an autoregressive correlation structure to account for the correlation between repeated observations from the same participants, as well as a random effect for match to account for the nonindependence within each matched pair. The fixed effects assessed were face, odour and group, as well as all possible 2-way and 3-way interactions. We analyzed separately the effects for neutral faces using a 2 × 3 mixed effects model. Between-group differences were analyzed using Bonferroni-corrected independent t tests. To further describe within-group response patterns, we performed exploratory repeated-measures t tests on mean accuracy and reaction time scores. Because of the exploratory nature of these tests, we choose a loose threshold of 0.05.

We conducted correlation analyses between accuracy measures and reaction times, respectively, and psychopathologic symptom severity, medication, cigarette consumption and neuropsychologic measures. A number of these parameters were not normally distributed; therefore, we chose to use a nonparametric correlation coefficient (Spearman-Rank correlation).

Also, we used correlation analyses to determine if the strength of the emotional reaction to odours or perceived intensity of odours covaried with accuracy or reaction time gains relative to the ambient air condition. For each facial expression, we computed difference scores of the reaction times and percent correct of neutral odour subtracted from pleasant and unpleasant odour, respectively, as a measure of how performance was affected by the presence of an odorant. We then correlated these with difference scores, subtracting the ratings for neutral odour from those for pleasant and unpleasant odors respectively. Because the distributions of these difference scores were right-skewed, we again chose to use Spearman-rank correlations. We performed the data analyses with SAS, version 9.1, and SPSS, version 15.0.


Rating scales

There was a significant main effect of odorant type on the intensity ratings (F2,90 = 35.74, p < 0.001). The main effect of group was significant, with healthy controls rating odours as more intense than did the schizophrenia patients (F1,45 = 4.38, p = 0.042). Pair-wise comparisons revealed that ambient air was judged to be significantly less intense than hydrogen sulfide (t46 = 7.55, p < 0.001) and vanillin (t46 = 3.32, p = 0.004). Vanillin was rated as less intense than hydrogen sulfide (t46 = 6.11, p < 0.001). None of the between-group t tests for individual odours yielded significant results.

Pleasantness scores revealed a significant main effect of odorant type (F2,90 = 79.38, p < 0.001). Pair-wise comparisons showed that vanillin was rated as more pleasant (t46 = −2.38, p = 0.023) and hydrogen sulfide was rated as less pleasant than ambient air (t46 = 9.20, p < 0.001). Vanillin was rated significantly more pleasant than hydrogen sulfide (t46 = 13.82, p < 0.001) (Fig. 2).

Fig. 2
Mean intensity (A) and pleasantness (B) ratings for vanillin, ambient air and hydrogen sulfide (H2S). Ratings were converted from the Self-Assessment Manikin32 rating scales filled out by the participants after completion of the task. Error bars indicate ...

There was no significant effect of group on the pleasantness ratings, either as a main effect of the ANOVA or in a post hoc t test.


The Sniffin’ Sticks score did not covary significantly with performance, so this factor was excluded from the model. Logistic regressions revealed a trend for a 3-fold interaction of odour, face and group (χ4 = 9.28, p = 0.054). There was also a significant 2-way interaction between face and odour (χ4 = 16.61, p = 0.002) and a main effect of face (χ2 = 17.64, p < 0.001). Between-group contrasts showed that for disgust faces, there was no performance difference between controls and schizophrenia patients under ambient air stimulation, but a difference emerged when disgust faces were presented with a pleasant (χ2 = 16.57, p < 0.001) or unpleasant (χ2 = 22.55, p < 0.001) odour (Fig. 3).

Fig. 3
Accuracy of emotion recognition among healthy participants and schizophrenia patients. We excluded nonresponses. The x-axis indicates the condition separately by odour and emotional face presented. Error bars indicate +/− 1 standard error. The ...

Exploratory post-hoc t tests showed that healthy controls were significantly more accurate in recognizing disgust expressions during the unpleasant (t23 = 2.25, p = 0.034) and pleasant (t23 = 5.41, p < 0.001) conditions than when no odour was presented. The pleasant and the unpleasant odour conditions did not differ significantly from each other. The recognition of happy and neutral faces was not significantly influenced by the presentation of an odorant stimulus. In the patient group, recognition of disgust faces during ambient air trials did not differ from either the pleasant or unpleasant odour condition. There was, however, a significant difference between the 2 odour stimulation conditions for disgust expressions (t23 = 2.51, p = 0.02), with patients performing worse at recognizing disgust faces during unpleasant olfactory stimulation than during than pleasant stimulation. No significant effect of odour was found for happy or neutral faces.

Reaction times

We did not find an effect of Sniffin’ Sticks score; we thus excluded this covariate from the model. Reaction times of the correct responses revealed a significant 3-fold interaction between face, odour and group (F4,8151 = 2.87, p = 0.022), as well as interactions between face and odour (F4,8151 = 5.99, p < 0.001) and between face and group (F2,8151 = 18.03, p < 0.001). We also found main effects of group (F1,8151 = 178.36, p < 0.001) and face (F2,8151 = 14.38, p < 0.001). Between-group contrasts revealed that patients were significantly slower than controls in all conditions (all below p = 0.005).

The separate 2 × 3 model for neutral faces revealed a significant interaction between odour and group (F2,2760 = 5.34, p = 0.005). There was also a significant main effect of group (F1,2760 = 39.76, p < 0.001).

Post-hoc t tests for mean accuracy ratings showed that in the healthy control group, responses to disgusted faces were decreased relative to ambient air stimulation when a pleasant (t23 = 2.07, p = 0.038) or an unpleasant odour (t23 = 2.12, p = 0.034) was presented. There was no significant difference between the odours. At the same time, response times to happy faces were increased relative to the ambient air condition when participants were exposed to a pleasant (t23 = 2.79, p = 0.005) or an unpleasant odour (t23 = 2.18, p = 0.029). Again, there was no difference between the 2 odours.

In the patient group, we observed faster response times for pleasant odours relative to neutral odours for the disgust faces (t23 = 2.82, p = 0.005). The reverse was true for neutral faces (t23 = 3.02, p = 0.003), where pleasant odour led to slower response times relative to the ambient air condition (Fig. 4).

Fig. 4
Log-transformed reaction times to the facial emotion recognition task (correct trials only). The left graph shows results for controls, the right graph shows results for patients. The x-axis indicates the condition separately by odour and emotional face ...

Influence of psychopathology and neurocognitive performance

There were no significant correlations of the PANSS scores with accuracy (positive: r = 0.042, p = 0.85; negative: r = −0.045, p = 0.84; general psychopathology, r = −0.048, p = 0.82; total score: r = 0.005, p = 0.98) or reaction times (positive: r = −0.19, p = 0.36; negative: r = −0.18, p = 0.41; general psychopathology: r = −0.013, p = 0.95; total score: r = −0.037, p = 0.86). There was no significant correlation between chlorpromazine equivalents and accuracy (r = −0.22, p = 0.37) or reaction time (r = −0.29, p = 0.23). Reaction time and accuracy were not significantly correlated with cigarette consumption (r = 0.03, p = 0.84 and r = −0.21, p = 0.16, respectively). We did not find any significant positive or negative correlation between MWT-B scores and reaction time or accuracy (r = 0.29, p = 0.06 and r = 0.21, p = 0.17). There was also no significant negative correlation of digit span score with reaction time (r = −0.25, p = 0.10) and a positive correlation with accuracy (r = 0.39, p = 0.01). The amount of time required to solve the TMT was positively correlated with reaction time, separately for TMT-A (r = 0.56, p < 0.001) and TMT-B (r = 0.55, p < 0.001). For accuracy, there was a significant negative correlation with TMT-A (r = −0.38, p = 0.008) but not with TMT-B (r = −0.21, p = 0.15). To rule out the possibility that cognitive deficits accounted for the observed findings, we reanalyzed the data with the TMT and digit span scores as covariates. For reaction times, the adjusted model with all 3 correlated test scores included showed significant effects for TMT-A, TMT-B and digit span. All main effects and interactions remained significant; however, only the 3-fold interaction between odour, face and group dropped to borderline significance (p = 0.054).

Correlation of valence and intensity ratings with reaction time and accuracy gains

Difference scores for pleasantness ratings averaged 1.13 (SD 1.84) for the pleasant odour and −2.91 (SD 1.86) for the unpleasant odour for the controls and 0.58 (SD 2.99) for the pleasant odour and −3.42 (SD 2.78) for the unpleasant odour for the patients. Because these values were computed from a bipolar scale, values higher than 0 indicate that the odour was rated more pleasant than the neutral odour, and values below 0 indicate that the odour was rated less pleasant than the neutral odour.

For intensity ratings, difference scores averaged 1.04 (SD 2.55) for the pleasant odour and 3.17 (SD 2.85) for the unpleasant odour in the controls and 1.04 (SD 2.35) for the pleasant odour and 2.96 (SD 2.77) for the unpleasant odour in the patients. Higher values indicate higher intensity ratings of odorants relative to ambient air.

For reaction time, a negative value of the difference score equaled a reaction time gain for odorant relative to ambient air, whereas for accuracy, a positive value was descriptive of a processing benefit.

For intensity ratings, there was no significant correlation for any odour and face combination for healthy controls or patients.

For pleasantness ratings, a number of significant and borderline significant correlations emerged. For neutral faces, we found a negative correlation among controls between reaction time difference score and H2S difference score; however this was not significant (r = −0.37, p = 0.07). This indicated that controls who rated H2S as more unpleasant relative to ambient air showed reaction time benefits during odour presentation relative to neutral faces. The reverse was observed for the schizophrenia patient group. Whereas the schizophrenia patients on average showed worse performance for neutral faces with an unpleasant odour, we found a positive correlation between the reaction time difference score and the H2S difference score (r = 0.42, p = 0.04), indicating that their difficulties were greater for H2S trials, the lower the perceived difference in pleasantness rating for neutral odour and hydrogen sulfide.

For disgusted faces, we found a nonsignificant correlation between difference scores for vanillin and accuracy difference scores in the control group (r = −0.35, p = 0.09). This indicated that the more participants preferred vanillin over the neutral odour, the more they were affected in their accuracy of recognizing disgust relative to the ambient air condition. Patients were affected in the reaction times, and their pattern matched that observed in the control group. It showed that the more patients preferred vanillin over ambient air, the longer it took them to respond to disgust trials in comparison to those primed with an ambient air stimulus (r = 0.62, p = 0.001).


The present study was designed to compare the effect of olfactory affective cues on emotional face identification in patients with schizophrenia and healthy controls. Although we found evidence for differences in response patterns between the 2 groups, these turned out to be more complex than our initial hypotheses of congruent and incongruent effects in healthy participants and diminished responsiveness in patients with schizophrenia. Rather, in healthy controls, our results suggest an effect of odour on disgusted facial expressions independent of odour valence. Patients with schizophrenia, on the other hand, showed a response modulation that differed between the pleasant and unpleasant odorant and resulted in greater performance differences between the groups especially for the congruent-negative condition.

Olfactory–visual interactions in healthy controls

Independent of the emotional quality of the olfactory prime, the presence of an odorant improved the recognition of disgusted faces but not happy or neutral faces in the healthy control group. Reaction times to disgust faces were affected in the same way, with both odours leading to accelerated responses. The reverse was true for happy facial expressions, where odours, independent of valence, caused slowed responses. No effects of odour on neutral face perception were found. Given that participants had no difficulty determining the emotional valence of the odorants on the presented rating scales, the absence of an influence of odour valence was unexpected.

Two explanations for our findings are considered here. The first assumes that chemosensory perception bears a close link to disgust processing, facilitating a behavioural link between odour and disgust expression, resulting in a processing advantage for odour–disgust pairings and a disadvantage for pairings with other emotional facial expressions. Rozin and colleagues34 showed that the main features of disgust are generally associated with smell (nose wrinkle) and taste (mouth gape, tongue extension). Also, the “nose wrinkler” is one of the main action units specified for disgust in the FACS coding system.35 These mimics are closely linked to “visceral” disgust (fear of contamination), a phylogenetically adaptive mechanism for the communication of rotten food.36

The second explanation is that the modulating factor is not generalizable to all olfactory–visual interactions in disgust but is influenced by the perceived sensory properties of the stimulus as measured by the individual intensity and pleasantness ratings to the olfactory stimulus. To test this hypothesis, we investigated whether interindividual differences in responsivity to the odorants could account for the strength of the observed priming effects. The lack of correlations observed between intensity ratings and performance makes this an unlikely influencing factor. However, priming of disgust faces with a pleasant odour was less effective when the odour was perceived as very pleasant, which suggests that perceived pleasantness might modulate the priming effect. It has previously been shown that crossmodal integration is most likely to occur under circumstances in which the information in one sensory channel is ambiguous (the principle of inverse effectiveness37). This may have been the case for the vanillin odorant presented in our study because the average pleasantness ratings were only moderately positive.

A similar observation was made for the neutral faces, which represent the most ambiguous type of face stimulus. Here the response was increasingly delayed the more negatively participants perceived H2S stimulation, indicating that the stimuli that were affectively more clearly defined may have exerted influence on the more ambiguous stimulus. Similar interactions are frequently encountered in social situations, in which one person might reevaluate their own affective response to an odorant upon seeing another person’s disgusted face. A candidate region to mediate these interactions between a person’s own disgust experience and emotional face perception on a neuronal level is the anterior insula. Wicker and colleagues38 compared brain responses for perceived disgust evoked by smells and presentations of disgusted facial expressions and found that both shared common neural substrates in this region. The authors therefore suggest the presence of a common mechanism for understanding disgust in others and feeling the same emotion in oneself. Our study indicates that interpersonal differences in the appraisal of the hedonic properties of an olfactory stimulus are relevant to describe the strength of such an olfactory–visual integration effect.

Olfactory–visual interactions in patients with schizophrenia

In line with previous studies,3,4,6 we found that patients with schizophrenia generally showed slower performance than healthy controls and were less accurate in their recognition of disgust. In contrast to the priming effects in the healthy control group, the effect of odour in patients with schizophrenia differed by odour valence. Our initial hypothesis of a general reduced susceptibility to odorant primes was not confirmed. Rather, the effect of a pleasant odour followed the same pattern of increasing accuracy and accelerating response time on disgust as observed in the control group. Similarly to the control group, we found that the effect of odour on accuracy was less pronounced when participants interpreted the vanillin odorant as highly pleasant. Pleasant odour increased the reaction time to neutral faces; the effect on happy faces, although insignificant, was in the same direction. The unpleasant odour had no effect on the behavioural measures at the group level; however, stronger aversive perception was associated with less response impairment for neutral faces. Whereas it has been shown that patients with schizophrenia have a large number of deficits related to olfactory functioning,24 our study suggests that patients’ responses to emotional faces can be modulated by the presentation of an olfactory stimulus in the presence of above-threshold and clearly distinguishable odours. The negative effect of pleasant odour priming on neutral faces and a trend in that direction for happy faces indicates that odour, like in the control group, seems to be closely associated with disgust perception and possibly incongruent with other emotional expressions in schizophrenia patients. Because the group differences in perceived intensity were of borderline significance, it seems likely that differences in absolute olfactory thresholds might account for some of the between group differences observed here. Further studies should specifically test this hypothesis. Also, an effect of the inability of patients with schizophrenia to benefit from greater intensity of emotional expression4 should be considered. Because crossmodal stimulation might lead to a greater intensity of perceived emotion in healthy controls,13 it remains to be determined to what extent this difference might account for some of the observed effects. A functional neuroimaging study investigating olfactory-related modulation of predominantly visual areas, such as the fusiform gyrus, is warranted to further investigate this aspect and is currently in preparation.

These established impairments of patients with schizophrenia, however, fail to account for the fact that no congruency effects were observed in the patient group for the negatively valenced stimulation. The largest performance differences between patients and controls were observed in the presence of a negatively valenced olfactory–visual stimulus pair. This was mainly accounted for by the substantial improvement in the control group and the absence of such an effect in the patients. It is also notable that for neutral faces, correlation analyses indicated a relation between better performance and strong unpleasantness ratings, suggesting that abnormal affective valence perception might aggravate performance deficits. The observed performance gap might have behavioural relevance in social situations, in which schizophrenia patients might appear particularly slow or inaccurate because of an inability to benefit from a behavioural cue to which healthy controls have a high level of responsiveness.

Failure to respond to negative primes has been attributed to increased vigilance of patients with schizophrenia to negative events, affecting processes of selective attention.39 Within this framework, it is assumed that participants would dedicate more resources to viewing such stimuli and hence delay their response, which would eliminate any priming effects otherwise observed. It therefore seems likely that not only impaired face recognition but also the observed unusual pattern of crossmodal integration for negatively valenced stimuli needs to be taken into account in characterizing the emotional and social difficulties that are frequently observed in patients with schizophrenia.


A number of issues will need to be addressed in future studies to further clarify the neuronal mechanisms behind the observed deficits and their specificity to olfactory–visual interactions.

One shortcoming of our study is related to the main effect of odour, which was observed in both the healthy controls and the schizophrenia group. Whereas the relative advantage of positive over negative emotion represents a well-established finding in the literature,4,10 it is possible that recognition rates over 95% indicate ceiling effects, which may have prevented the occurrence of facilitation effects in neutral and happy faces. We adjusted our statistical analyses to account for this problem, preventing a systematic overestimation of the effects present in the more difficult conditions. In addition, we found that odour had a detrimental effect on response times for happy face recognition but a positive effect for disgusted faces, which suggests that ceiling effects were not present here. Because the facilitation effects for disgust followed the same pattern in both target variables, it seems likely that our effects represent true variations between the individual emotion conditions, and that accuracy, under increased task difficulty, might show similar response patterns as reaction times for the other emotions. Also, ceiling effects do not account for the observed differences in performance patterns between healthy controls and patients. Future studies should, however, aim to increase recognition difficulty to confirm stability of the observed effects under conditions of higher task demand.

Furthermore, the effect of neuropsychologic deficits should be studied more closely, because executive functioning was able to account for a portion of the variance observed in this study. In our study, the interstimulus interval between odour and face was chosen with a following imaging study in mind, so that the neural correlates of odour presentation and the crossmodal priming effect could be disentangled. It should also be noted that the working memory task was not very complex, with only 2 different stimulus types and a retention period of 1 second. However, because working memory deficits have frequently been reported to be a core feature of schizophrenia,40,41 future studies should reduce the contribution of this factor as much as possible.

Our results further indicate that more sensitive measures may be required to assess the relation between schizophrenia symptoms and behavioural performance. Whereas previous studies have shown correlations with symptom-specific sub-scales of the Scale for the Assessment of Positive Symptoms42 and Scale for the Assessment of Negative Symptoms,43 we could not detect such a relation using the positive, negative and general symptom scales from the PANSS. A review of existent reports of correlations would be advisable to advance the understanding of the schizophrenia symptoms related to difficulties in social cognition and crossmodal integration. Such an undertaking appears essential for targeted intervention.

Finally, our results indicate a need for clarification of the differences in the effects resulting from emotional priming relative to mood-induction paradigms. The response patterns shown by healthy controls here differ from those found in the study by Leppänen and Hietanen,10 who used long-term odorant exposure and a between-subject design. Their results are explained in terms of a benefit for self or other mood congruency. Similarly, the study by Wicker and colleagues38 reported differential activations for pleasant and unpleasant odorants. As discussed above, our study showed only limited evidence for differential effects of odour valence. This indicates the short exposure and quick changes in odour stimulation, as used in the present study, may not be as suited to evoking positive emotions as a mood-induction method with more clearly separated and longer stimulation phases.


Our study provides interesting and novel evidence for a special connection between the presentation of odorant cues and the subsequent increased accuracy in recognition of disgusted faces in healthy controls. A similar response pattern was found for pleasant odours in schizophrenia patients. However, their impairment in the benefit from congruent negative stimulation seems to further point to the special role of negative emotion processing in schizophrenia.44 This may aggravate difficulties in social interaction and warrants more detailed investigation.


Competing interests: Dr. Seubert has received a scholarship from IRTG 1328 Schizophrenia and Autism and travel assistance from GlaxoSmithKline. None declared for Drs. Loughead, Kellermann, Boers and Habel and Ms. Brensinger.

Contributors: Drs. Seubert, Loughead, Boers and Habel designed the study. Dr. Seubert acquired the data, which all authors analyzed. Dr. Seubert wrote the article, which all other authors reviewed. All authors approved the final version submitted for publication.


1. Schneider F, Gur RC, Gur RE, et al. Emotional processing in schizophrenia: neurobehavioral probes in relation to psychopathology. Schizophr Res. 1995;17:67–75. [PubMed]
2. Kohler CG, Bilker W, Hagendoorn M, et al. Emotion recognition deficit in schizophrenia: association with symptomatology and cognition. Biol Psychiatry. 2000;48:127–36. [PubMed]
3. Calkins ME, Gur RC, Ragland JD, et al. Face recognition memory deficits and visual object memory performance in patients with schizophrenia and their relatives. Am J Psychiatry. 2005;162:1963–6. [PubMed]
4. Kohler CG, Turner TH, Bilker WB, et al. Facial emotion recognition in schizophrenia: intensity effects and error pattern. Am J Psychiatry. 2003;160:1768–74. [PubMed]
5. Johnston PJ, Devir H, Karayanidis F. Facial emotion processing in schizophrenia: no evidence for a deficit specific to negative emotions in a differential deficit design. Psychiatry Res. 2006;143:51–61. [PubMed]
6. Gur RE, McGrath C, Chan RM, et al. An fMRI study of facial emotion processing in patients with schizophrenia. Am J Psychiatry. 2002;159:1992–9. [PubMed]
7. Pinkham AE, Penn DL, Perkins DO, et al. Implications for the neural basis of social cognition for the study of schizophrenia. Am J Psychiatry. 2003;160:815–24. [PubMed]
8. Vroomen J, de Gelder B. Sound enhances visual perception: cross-modal effects of auditory organization on vision. J Exp Psychol Hum Percept Perform. 2000;26:1583–90. [PubMed]
9. Hietanen JK, Leppänen JM. Judgment of other people’s facial expressions of emotions is influenced by their concurrent affective hand movements. Scand J Psychol. 2008;49:221–30. [PubMed]
10. Leppänen JM, Hietanen JK. Affect and face perception: odors modulate the recognition advantage of happy faces. Emotion. 2003;3:315–26. [PubMed]
11. Spreckelmeyer KN, Kutas M, Urbach TP, et al. Combined perception of emotion in pictures and musical sounds. Brain Res. 2006;1070:160–70. [PubMed]
12. Van den Stock J, Grèzes J, de Gelder B. Human and animal sounds influence recognition of body language. Brain Res. 2008;1242:185–90. [PubMed]
13. Ethofer T, Anders S, Erb M, et al. Impact of voice on emotional judgment of faces: an event-related fMRI study. Hum Brain Mapp. 2006;27:707–14. [PubMed]
14. Kreifelts B, Ethofer T, Grodd W, et al. Audiovisual integration of emotional signals in voice and face: an event-related fMRI study. Neuroimage. 2007;37:1445–56. [PubMed]
15. de Gelder B, Vroomen J, de Jong SJ, et al. Multisensory integration of emotional faces and voices in schizophrenics. Schizophr Res. 2005;72:195–203. [PubMed]
16. de Jong JJ, Hodiamont PP, Van den Stock J, et al. Audiovisual emotion recognition in schizophrenia: reduced integration of facial and vocal affect. Schizophr Res. 2009;107:286–93. [PubMed]
17. de Gelder B, Vroomen J, Annen L, et al. Audio-visual integration in schizophrenia. Schizophr Res. 2003;59:211–8. [PubMed]
18. Zald DH, Pardo JV. Emotion, olfaction, and the human amygdala: amygdala activation during aversive olfactory stimulation. Proc Natl Acad Sci U S A. 1997;94:4119–24. [PubMed]
19. Rolls ET. Convergence of sensory systems in the orbitofrontal cortex in primates and brain design for emotion. Anat Rec A Discov Mol Cell Evol Biol. 2004;281:1212–25. [PubMed]
20. Habel U, Koch K, Pauly K, et al. The influence of olfactory-induced negative emotion on verbal working memory: individual differences in neurobehavioral findings. Brain Res. 2007;1152:158–70. [PubMed]
21. Schneider F, Koch K, Reske M, et al. Interaction of negative olfactory stimulation and working memory in schizophrenia patients: development and evaluation of a behavioral neuroimaging task. Psychiatry Res. 2006;144:123–30. [PubMed]
22. Strauss GP, Allen DN, Ross SA, et al. Olfactory hedonic judgment in patients with deficit syndrome schizophrenia. Schizophr Bull. Epub 2009 Feb. 17 ahead of print. [PMC free article] [PubMed]
23. Doop ML, Park S. On knowing and judging smells: identification and hedonic judgment of odors in schizophrenia. Schizophr Res. 2006;81:317–9. [PubMed]
24. Moberg PJ, Arnold SE, Doty RL, et al. Impairment of odor hedonics in men with schizophrenia. Am J Psychiatry. 2003;160:1784–9. [PubMed]
25. Wittchen HU, Zaudig M, Fydrich T. Strukturiertes Klinisches Interview für DSM-IV. Göttingen (Germany): Hogrefe; 1998.
26. Kay SR, Fiszbein A, Opler LA. The positive and negative syndrome scale (PANSS) for schizophrenia. Schizophr Bull. 1987;13:261–76. [PubMed]
27. Lehrl S. Mehrfachwahl-Wortschatz-Intelligenztest (MWT-B) Balingen (Germany): Spitta Verlag; 1977.
28. Hummel T, Konnerth CG, Rosenheim K, et al. Screening of olfactory function with a four-minute odor identification test: reliability, normative data, and investigations in patients with olfactory loss. Ann Otol Rhinol Laryngol. 2001;110:976–81. [PubMed]
29. Doty RL, Brugger WE, Jurs PC, et al. Intranasal trigeminal stimulation from odorous volatiles: psychometric responses from anosmic and normal humans. Physiol Behav. 1978;20:175–85. [PubMed]
30. Kobal G, Hummel T. Olfactory and intranasal trigeminal event-related potentials in anosmic patients. Laryngoscope. 1998;108:1033–5. [PubMed]
31. Seubert J, Rea AF, Loughead J, et al. Mood induction with olfactory stimuli reveals differential affective responses in males and females. Chem Senses. 2009;34:77–84. [PubMed]
32. Lang PJ. The cognitive psychophysiology of emotion: fear and anxiety. In: Tuna AD, Maser JD, editors. Anxiety and the anxiety disorders. Hillsdale (NJ): Erlbaum; 1985. pp. 131–70.
33. Schneider F, Gur RC, Gur RE, et al. Standardized mood induction with happy and sad facial expressions. Psychiatry Res. 1994;51:19–31. [PubMed]
34. Rozin P, Lowery L, Ebert R. Varieties of disgust faces and the structure of disgust. J Pers Soc Psychol. 1994;66:870–81. [PubMed]
35. Ekman P, Friesen WV. Facial action coding system: a technique for the measurement of facial movement. Palo Alto (CA): Consulting Psychologists Press; 1978.
36. Rozin P, Fallon AE. A perspective on disgust. Psychol Rev. 1987;94:23–41. [PubMed]
37. Meredith MA, Stein BE. Interactions among converging sensory inputs in the superior colliculus. Science. 1983;221:389–91. [PubMed]
38. Wicker B, Keysers C, Plailly J, et al. Both of us disgusted in My in-sula: the common neural basis of seeing and feeling disgust. Neuron. 2003;40:655–64. [PubMed]
39. Waters FA, Badcock JC, Maybery MT. Selective attention for negative information and depression in schizophrenia. Psychol Med. 2006;36:455–64. [PubMed]
40. Goldman-Rakic PS. Working memory dysfunction in schizophrenia. J Neuropsychiatry Clin Neurosci. 1994;6:348–57. [PubMed]
41. Silver H, Feldman P, Bilker W, et al. Working memory deficit as a core neuropsychological dysfunction in schizophrenia. Am J Psychiatry. 2003;160:1809–16. [PubMed]
42. Andreasen NC. Scale for the Assessment of Positive Symptoms (SAPS) Iowa City (IA): University of Iowa; 1984.
43. Andreasen NC. Scale for the Assessment of Negative Symptoms (SANS) Iowa City (IA): University of Iowa; 1983.
44. Habel U, Krasenbrink I, Bowi U, et al. A special role of negative emotion in children and adolescents with schizophrenia and other psychoses. Psychiatry Res. 2006;145:9–19. [PubMed]

Articles from Journal of Psychiatry & Neuroscience : JPN are provided here courtesy of Canadian Medical Association