Determining how schizophrenia patients visually decode emotional content from faces is crucial to understanding the nature of impaired emotion perception in schizophrenia. With the Bubbles procedure, we examined how schizophrenia patients employ visual information to judge emotional expression of faces. The amount of visual information revealed by the bubbles across trials was adjusted to maintain 75% accuracy for each participant, so that schizophrenia patients and healthy controls were matched in behavioral performance.
To correctly identify emotional expressions of faces in general, schizophrenia patients required more visual information compared with controls. Moreover, schizophrenia patients used a different strategy of collecting visual information to identify emotional expression. For identifying fearful emotion, schizophrenia patients did not utilize information from the eyes but instead relied on areas around the mouth, whereas healthy controls relied on eyes at high-spatial frequency and areas around the mouth at mid-spatial frequencies. To identify happiness, schizophrenia patients relied on the area around the mouth at high- and mid-range spatial frequencies and the eyes at mid-range spatial frequency, whereas healthy controls only used mid-range spatial frequency information around the mouth. This study is the first, to our knowledge, to identify potent information that is used to make decisions about emotional content of facial stimuli in schizophrenia.
Using the Bubbles technique, this study recreated the information that patients used to discriminate emotional expressions. Not only did schizophrenia patients use different facial features but they also utilized different spatial frequencies. Especially this study presented spatial frequency information across all spatial frequency bands simultaneously instead of one spatial frequency bandwidth at a time. In this sense, the current study differs from previous studies that manipulated spatial frequency information in schizophrenia.36
By presenting all spatial frequency bandwidths simultaneously, this study was able to identify which spatial frequency information is more critical at what facial regions. Schizophrenia patients showed an atypical usage of visual information, and this atypical strategy was more prominent in high-spatial frequency bandwidth. Previous studies on face processing suggested that schizophrenia patients have more difficulty processing configural facial information than featural face information.37,38
Considering the role of high-spatial frequency information in featural face processing,19,20
the current finding demonstrates that schizophrenia have abnormal featural processing when judging emotional expression of faces.
One major distinction between schizophrenia patients and controls is the usage of high-spatial frequency information from the eye regions. Schizophrenia patients did not use high-spatial frequency information from the eyes to identify fearful emotion, whereas this information was critical for controls. When processing fearful faces, healthy individuals tend to activate the amygdala, an effect that is associated with viewing the eye regions, as opposed to other parts of the face.39,40
A previous report that used the Bubbles technique showed that a patient with an amygdala lesion used less of the eye regions at high-spatial frequency when recognizing fear.41
Among schizophrenia patients, studies using functional magnetic resonance imaging found reduced activation associated with processing fearful faces in the amygdala.42,43
Our finding of reduced use of high-spatial frequency information around the eyes may be related to the reduced activation of the amygdala to fear in schizophrenia patients. Patients with autism also show an atypical strategy of using visual information to recognize fearful faces, which is similar to what is seen in schizophrenia patients. In those studies,44,45
autistic patients (as well as their unaffected parents) relied less on the eye areas and more on the mouth areas when judging fear. Furthermore, the parents who used this atypical search strategy to a greater extent were also more likely to be socially aloof, a personality characteristic related to autism.45
Future studies with larger samples will be able to determine whether this aberrant strategy is associated with other characteristics of schizophrenia patients.
Restricted visual scanning of emotional faces has been previously suggested as a possible mechanism of impaired emotional recognition in schizophrenia patients.6,46
In general, visual scanning measured through eye movements (i.e., visual scan paths) provides useful information about where on the face, and for how long, people look when judging emotions. However, these studies do not inform us how schizophrenia patients use visual information. Knowing where someone is looking does not tell us how he/she uses visual information. In contrast, with the Bubbles technique, this study showed which parts of facial features or what levels of spatial frequency were critical to make decisions about emotional expression of faces. Hence, this study provides information that is not available from studies using visual scan paths.
On the surface, the current study resembles previous studies on visual integration of schizophrenia patients.47
For example, both perceptual closure tests of visual integration and the Bubbles task include partially obscured stimuli. However, it is difficult to make any inference about visual integration from the current study because it is possible that participants were making decisions by using specific visual cues that do not require integration (e.g., making a decision based on the amount of white above the iris) instead of mentally filling in the rest of faces from bubbles (i.e., visual integration).
Fearful and happy faces were selected because they have been shown to be associated with the most distinct use of facial visual information.21
Because we only used 2 emotions; however, it is unclear if patients made their decisions based on the presence of the critical features of one emotion or the absence of the critical features of the other emotion. It is possible that this finding characterizes how patients differentiate fear from happy instead of how they recognize each emotion in isolation. Thus, it remains to be tested whether schizophrenia patients would show a similar use of visual information when asked to recognize fear and happiness among more alternatives. Reassuringly, healthy controls in this study showed the same search strategies for fear and happy that were found when people were asked to identify the 6 basic emotions plus neutral faces.21
Another limitation of this study is that we did not collect the stimulus presentation time during the Bubbles task. It remains to be determined whether schizophrenia patients need longer stimulus presentation time to collect necessary visual information.
In summary, we found that compared with controls schizophrenia patients collected different spatial frequency information from different facial regions when judging emotional expression of faces. The atypical usage of visual information in schizophrenia patients suggests an abnormal processing of featural facial information. This study helps us to better understand the underlying mechanism of impaired recognition of emotional expression in schizophrenia.