PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
J Child Psychol Psychiatry. Author manuscript; available in PMC 2013 December 1.
Published in final edited form as:
PMCID: PMC3505257
NIHMSID: NIHMS395695

The perception and identification of facial emotions in individuals with Autism Spectrum Disorders using the Let’s Face It! Emotion Skills Battery

Abstract

Background

Although impaired social-emotional ability is a hallmark of autism spectrum disorder (ASD), the perceptual skills and mediating strategies contributing to the social deficits of autism are not well understood. A perceptual skill that is fundamental to effective social communication is the ability to accurately perceive and interpret facial emotions. To evaluate the expression processing of participants with ASD, we designed the Let's Face It! Emotion Skills Battery (LFI! Battery), a computer-based assessment composed of three subscales measuring verbal and perceptual skills implicated in the recognition of facial emotions.

Methods

We administered the LFI! battery to groups of participants with ASD and typically developing control (TDC) participants that were matched for age and IQ.

Results

On the NameGame labeling task, participants with ASD (N = 68) performed on par with TDC individuals (N = 66) in their ability to name the facial emotions of happy, sad, disgust and surprise and were only impaired in their ability to identify the angry expression. On the Matchmaker Expression task that measures the recognition of facial emotions across different facial identities, the ASD participants (N = 66) performed reliably worse than TDC participants (N = 67) on the emotions of happy, sad, disgust, frighten and angry. In the Parts/Wholes test of perceptual strategies of expression, the TDC participants (N = 67) displayed more holistic encoding for the eyes than the mouths in expressive faces whereas ASD participants (N = 66) exhibited the reverse pattern of holistic recognition for the mouth and analytic recognition of the eyes.

Conclusion

In summary, findings from Let's Face It! Emotion Skills Battery show that participants with ASD were able to label the basic facial emotions (with the exception of angry expression) on par with age- and IQ-matched typically developing participants. However, participants with ASD were impaired in their ability to generalize facial emotions across different identities and a tendency to recognize the mouth feature holistically and the eyes as isolated parts.

Keywords: ASD, Computer-based assessment, Facial emotions, Perceptual skills, Social communication

Autism is a pervasive developmental disorder (PDD) involving impairment in reciprocal social interaction as well as impairments in verbal and non-verbal communication, a lack of imaginative play, and repetitive and restricted solitary activities. Autism is highly heritable and involves developmental differences in brain growth, organization and function. Autism presents with a range of severity and associated features, and its heterogeneity, is commonly referred to as Autism Spectrum Disorder (ASD). According to the current DMS-IV-TR standards, individuals diagnosed with autism display an impaired ability to understand the emotions and feelings of others and a “lack of social reciprocity.” This, in turn, creates difficulties in social interactions that Kanner (1943) has characterized as a “disorder of affective contact.” Although the social deficits of autism are likely to have multiple origins, the facility to recognize and interpret facial expressions during face-to-face interactions is critical is to normal social functioning. In everyday social encounters, a person’s facial expression is the outward manifestation of their internal emotional state or the emotional state that they wish to convey to the external observer (Ekman & Friesen, 1971). Therefore, success in social interactions relies on the capacity to recognize and interpret facial emotions in a social context. If individuals with ASD have difficulties perceiving facial expressions, it follows that they would be at a disadvantage when interpreting the emotional state and intention of others (Leppanen & Nelson, 2006) and this would invariably lead to difficulties in everyday social exchanges. Indeed, it has been speculated that deficits in face processing skills and abnormalities in the neural circuits that meditate these functions may play a causative factor in the social deficits of autism (Dawson, Webb, Wijsman, Schellenberg, Estes, Munson, et al., 2005; Schultz, 2005).

Emotion Recognition

Despite the clear link between facial expression and social function, the empirical results present a mixed picture as to whether individuals with autism are impaired in their ability to perceive and interpret facial emotions. For example, some studies show that individuals with ASD perform worse than control participants on tasks requiring the labeling of primary basic emotions, such as anger, disgust, and sadness (Bolte & Poustka, 2003; Celani, Battacchi, & Arcidiacono, 1999) and secondary social emotions, such as trustworthiness and jealousy (Adolphs, Sears, & Piven, 2001; Rump, Giovannelli, Minshew, & Strauss, 2009). In contrast, other studies indicate that perception of expression is not disrupted relative to control participants (Castelli, 2005; Grossman, Klin, Carter, & Volkmar, 2000; Ozonoff, Pennington, & Rogers, 1990).

There are multiple factors that might account for the presence (or absence) of expression recognition differences between ASD and TDC groups. The disparate findings across studies can be attributed to differences in the age of the ASD and control participants, the criteria on which the two groups are matched and the cognitive complexity of the tasks employed (Harms, Martin, & Wallace, 2010). With respect to age, it has been argued that expression recognition abilities develop more slowly and reach their peaks sooner in the ASD population than the typically developing population (Gepner, Deruelle, & Grynfeltt, 2001). Consequently, emotion recognition differences between ASD and typically developing children that are undetected in younger children may become apparent by adolescence and young adulthood (O'Connor, Hamm, & Kirk, 2005). Secondly, decisions on what factors used to match the participants can either enhance or mask facial expression variation in performance on facial recognition tasks between the ASD and TDC groups (Burack, Iarocci, Flanagan, & Bowler, 2004). Given that verbal IQ scores of persons with ASD are typically lower than their non-verbal IQ scores, several studies have shown emotional impairments when participants with ASD were matched on nonverbal intelligence, but no differences when the groups were matched for verbal intelligence (Fein, Lucci, Braverman, & Waterhouse, 1992; Ozonoff, Pennington, & Rogers, 1990). Third, expression tasks that are perceptually or cognitively more taxing are more likely to reveal ASD and TDC differences than simpler tasks. For example, some studies have shown that participants with ASD were not impaired in their recognition of basic emotions presented in static photographs (Castelli, 2005; Grossman, Klin, Carter, & Volkmar, 2000; Ozonoff, Pennington, & Rogers, 1990). By contrast, other studies have shown group differences when using dynamic, video stimuli (Rump, et al., 2009; Smith, Montagne, Perrett, Gill, & Gallagher, 2010) or when more subtle, secondary social emotions (e.g., trustworthiness) were tested (Adolphs, Sears, & Piven, 2001).

To investigate the expression recognition skills of participants with autism, we developed the Let’s Face It! (LFI!) Emotion Skills Battery. The LFI! Emotion Skills Battery is a compliment to the Let’s Face It! Skills Battery that assesses the recognition of facial identity (Wolf, Tanaka, Klaiman, Cockburn, Herlihy, Brown, et al. 2009). The LFI! Emotion Battery consists of three subscales: Name Game, Matchmaker Expression and Parts-Wholes Expression. The Name Game is a subscale that assesses the child’s ability to label basic facial emotions of happy, sad, anger, disgust, surprise and fear. Matchmaker Expression is a test in which the child is asked to match a target expression (e.g., happy) depicted in one person to the corresponding expression shown in one of three new faces (Celani, Battachi, & Arcidiacono, 1999). The purpose of the task is to assess the child’s ability to generalize expressions across identities. The Parts/Wholes Expression test assesses the child’s use of analytic and holistic encoding strategies to identify happy and angry expressions. On tests of facial identity, children with ASD tend to focus more on the mouths and less on the eyes than TDC children (Joseph & Tanaka, 2003; Wolf, et al., 2009). The Parts/Wholes Expression test examines whether a similar mouth strategy is employed during the perception of facial emotions. In the present study, we administered the emotion battery to a large sample of participants diagnosed with ASD (N = 85) and a control sample of TDC (N = 130) participants. To avoid potential age and IQ confounds, participants from the two groups were matched on each separate subscale in the battery for chronological age and full scale IQ. After controlling for age and IQ, we hypothesized that individuals with ASD should differ from TDC individuals in their ability to label emotions, to generalize emotions across identities and in their reliance on expression information in eye and mouth regions of the face.

Method

Participants

This study was approved by the institutional review boards of Yale University School of Medicine and the University of Victoria. All participants (or parents of minor participants) gave written informed consent after study procedures were fully explained to them.

Participants of the present study included 85 participants, adolescents, and young adults with ASD, and 130 typically developing participants, adolescents, and young adults. Participants in the ASD group were recruited on the basis of previous diagnoses of Autistic Disorder, Asperger’s Disorder, or Pervasive Developmental Disorder, Not Otherwise Specified (PDD-NOS), through presentations at schools and parent organizations, and through existing relationships with families of participants on the autism spectrum. TDC participants were recruited through word of mouth and through local churches and school systems. TDC participants were excluded if they had significant symptoms of a DSM-IV Axis I disorder (based on the Child Symptom Inventory; Gadow & Sprafkin, 1994). TDC and ASD participants were excluded if they had vision worse than 20–100 in both eyes, or if, in the judgment of an experienced clinician, they were unable to comprehend the instructions of the experimental tasks.

ASD diagnoses were confirmed based on DSM-IV criteria through use of the Autism Diagnostic Interview, Revised (ADI-R; Rutter, LeCouteur & Lord, 2003) and the Autism Diagnostic Observation Schedule – Generic (ADOS-G; Lord, Rutter, DiLavore & Risi, 1999) by a clinician trained in their administration, with more than five years of experience working with individuals with ASD. In some cases, ADOS-G or ADI-R data were missing (ADOS: 4 missing, ADI: 7 missing), or participants did not meet criteria for an autism spectrum disorder on one of these measures (ADOS: 16 did not meet; ADI: 7 did not meet; note that there is no overlap in these numbers; i.e. all participants met criteria on at least one of the two diagnostic measures). In these instances, a final diagnostic decision was made by consensus among two or more clinicians with more than five years of experience in the field of ASD, independent of any knowledge of how the child performed on the LFI! Skills Battery.

IQ scores were obtained for all participants using the Wechsler Abbreviated Scale of Intelligence (WASI; Wechsler, 1999), the Wechsler Intelligence Scale for Children, 3rd edition (WISC-III; Wechsler, 1991), the Wechsler Adult Intelligence Scale, 3rd edition (WAIS-III; Wechsler, 1997), or the Differential Abilities Scales (DAS; Elliott, 1990). In cases in which a participant had an IQ test administered clinically within the last year, an IQ measure was not re-administered, and scores from the previous administration were utilized for the purposes of the present study.

The TD control (TDC) group was composed of 130 participants (87 males and 53 females) with a mean age of 11.96 years (range: 5.10 to 18.10 years of age) and a mean full scale IQ of 113.28 (range: 81 to 119 points). The ASD group consisted of 85 participants (71 males and 14 females) with a mean age of 11.58 (range: 5.81 to 20.72 years of age) and a mean full scale IQ of 99.74 (range: 58 to 147 points). The ASD group comprised 36 individuals with Autistic Disorder, 21 with Asperger’s Disorder, and 28 with PDD-NOS. From this pool of participants, subsamples were created for each analysis in which the ASD and TDC groups were carefully matched on age and IQ. The ASD group contained a higher proportion of males (84% males) than the TD group (63% male). However, no gender differences were found between the males and females in the ASD group or males and females in the TD group on NameGame, Matchmaker Expression, Parts/Wholes Expression subtests, p > .05. Because some of the subtests in the LFI! Skills Battery were under development at the outset of the study, group matching was conducted separately for each of the measures, blindly with respect to dependent variables of interest. As shown in Table 1, for all analyses, groups were matched for both age and Full Scale IQ such that no means differed by more than 0.1.

Table 1
Group characteristics for the three subscales. Group matching was conducted separately for each analysis based on both age and full scale IQ. Note: No gender differences were found between males and females in the ASD or TD groups.

Procedure

Participants were administered the LFI! Skills Battery in addition to other neuropsychological and behavioral measures. The LFI! Skills Battery was administered over a two-day period. Half of the items were administered on Day 1 of testing, and a parallel set of items was administered on Day 2 of testing.

Description of Let’s Face It! Skills Emotion Battery

Name Game

This assessment examined the child’s ability to match a word label to its facial expression. At the beginning of each trial, an emotional face was centrally presented on a computer screen with a list of the six emotion names (happy, angry, sad, disgust, surprise, and frighten) shown on the right side of the display (See Figure 1a). The child’s task was to click on the emotion name that was depicted in the face. To minimize reading demands, the computer software spoke the emotion name when the participant scrolled their mouse over the label. The face remained on the screen until the participant made a response. After the response, there was a one second inter-trial interval followed by the presentation of the next item. The faces were color images from the NimStim set of facial expressions (Tottenham, Tanaka, Leon, McCarry, Nurse, Hare et al., 2009). There were a total of 66 trials composed of 11 items for each of the six emotions (happy, sad, angry, disgust, frighten, surprise). The face images subtended visual angles of 2.5° and 4° in the horizontal and vertical dimensions, respectively.

Figure 1
(a) Name game assessment: emotional face test. (b) Matchmaker expression assessment test using images from the NonStim face database.

Matchmaker Expression

This assessment examined the child’s ability to match emotional expressions across changes in facial identity without explicit verbal labels. A study face depicting a basic emotion (i.e., happy, angry, sad, disgust, frighten) was presented in front view for one second followed by three probe faces of different identities (see Figure 1b). In a three-alternative forced choice task, the participant was asked to select the probe face that matched the emotion depicted in the study face. Both the study face and probe faces remained on the screen until a response was made. After the response, there was a one second inter-trial interval followed by the presentation of the next item. There were 30 trials with six items per emotional expression (happy, angry, sad, disgust and frighten). The face stimuli were color images from the NimStim face database (Tottenham, et al., 2009) that subtended a visual angle of 2° in horizontal dimension and 3.5° cm in vertical direction.

Parts /Wholes Expression

This assessment examined the child’s use of featural and holistic strategies to recognize the facial expressions of happiness and anger. A study face depicting a consistent expression or inconsistent expression was presented for 2 seconds. The “consistent” expression depicted a face in which the top and bottom halves were the same expression. The “inconsistent” expressions were constructed by combining the top half of one expression (e.g., angry) with the bottom half of the other (e.g., happy) from the same individual (see Figure 2a and 2b for examples of consistent and inconsistent expressions). At test, the participant’s memory for a face part (eyes or mouth) was evaluated by presenting the face part and its foil either in isolation or in a whole face. In the whole face condition, the faces were identical with the exception of the target face part and its foil. For example, if the critical face part was the eyes, the target and foil eye parts were tested in isolation or embedded in a face with the exact same mouth and nose features. The measure of holistic processing is the difference between performance in the whole face test condition versus performance in the part test condition. The measure of emotional processing is the difference in performance between the consistent expression condition and inconsistent expression condition. Four faces were used in the assessment and were normed for expression from the NimStim set of facial expressions (Tottenham, et al., 2009). Recognition memory for eye and mouth features was tested in the part and whole conditions in both the consistent and inconsistent conditions. The factors of condition (part, whole), feature (eyes, mouth), consistency (consistent, strong inconsistent, weak inconsistent) yielded a total of 48 trials. The face images subtended visual angles of 2.5° and 4° visual angle in the horizontal and vertical dimensions, respectively.

Figure 2
Consistent and inconsistemt expressions from the parts/whole expression assessment test.

Results

Results (including effect sizes) from the Name Game, Matchmaker Expression and Parts/Wholes Expression subscales are reported in Table 2.

Table 2
Mean percentages of correct responding and effect sizes (Cohen’s d) for the Name Game, Matchmaker Expression and Parts/Wholes Expression subscales. Standard deviations are indicated in parentheses.

Name Game

A 2 × 6 ANOVA was conducted on the Name Game data, with group (ASD, TDC) as a between-group variable and Expression (happy, sad, disgust, angry, surprise, frighten) as a within-group variable. The main effect of group approached significance, F(1, 132) = 2.86, p = .09, with the TDC group demonstrating higher overall accuracy than the ASD group. Results demonstrated a significant main effect of expression, F(5, 660) = 77.52, p<.001, but no expression x group interaction, F(5, 660) = 1.43, p > . 05. In both groups, the highest accuracy was on happy expressions (where both groups reached near ceiling performance), and the lowest accuracy was on frighten expressions. Group comparisons on the individual expressions revealed that the ASD group performed reliably worse in labeling the angry expression, p < .01, but did not differ from the TDC group on the other expressions of happy, sad, disgust, surprise and frighten (see Figure 3a).

Figure 3
Comparative results between the ASD and TDC groups for the (a) (name game (b) matchmaker expression and (c) parts/whole expression assessment tests.

Matchmaker- Expression

A 2 × 5 ANOVA was conducted on the Matchmaker-Expression data, with group (ASD, TDC) as a between-group variable and expression (happy, angry, frighten, sad, disgust) as a within-group variable. Results demonstrated a significant main effect of expression (F(4, 524) = 76.49, p<.001), a significant main effect of group (F(1, 131) = 35.42, p<.001), and a significant expression x group interaction (F(4, 524) = 3.97, p<.01). The TDC group had significantly higher accuracy than the ASD group on all five expressions (happy: t(131) = 2.19, p<.05; angry: t(131) = 4.40, p<.001; fear: t(131) = 3.72, p<.001; sad: t(131) = 4.78, p<.001; disgust: t(131) = 2.84, p<.01 (as shown in Figure 3b).

Parts/Wholes Expression

A 2 × 2 × 2 × 2 ANOVA was conducted with the between-group variable of group (ASD, TD) and within-group variables of Condition (part, whole), Feature (eyes, mouth) and Consistency (consistent, inconsistent). A significant main effect of Condition was found, F(1, 131) = 18.94, p<.001, such that face parts were recognized better when tested in the whole face than in isolation. The main effect of Consistency, F(1, 131) = 59.26, p<.001, was also significant indicating that participants were more accurate when recognition was tested with study faces with consistent expressions (e.g., happy eyes, happy mouth) than with inconsistent expressions (e.g., happy eyes, angry mouth). Consistency also interacted with Condition, F(1,131) = 26.85, p < .01, indicating a stronger whole face advantage for consistent than inconsistent expressions. Finally, the main factor of Feature was reliable, F(1, 131) = 52.26, p<.001, showing that recognition of the mouth was better than the eyes.

Overall, there were no group differences between the ASD and TDC cohorts, F(1,131) = 1.35, p > .10. However, Group did significantly interact with Feature, F(1,131) = 10.82, p < .01, such that the ASD group performed reliably worse than the TDC group in their recognition of eyes, but performed as well as TDC participants on their recognition of the mouth. The three-way interaction between Group, Condition and Feature was also significant, F(1, 131) = 4.30, p< .05. Whereas the TDC group recognized the eye feature better when tested in the whole face than in isolation, p < .05, participants in the ASD group showed no whole face advantage for recognition of the eyes. However, the ASD group recognized the mouths better in the whole face than in isolation, p <.05; this was not the case for the TDC group. No other interactions were significant. These results are depicted in Figure 3c.

Correlational analyses

We also correlated performance on the LFI! battery with age, IQ and ADOS scores.

Age correlations

For the TDC group, age correlated with the three emotion tasks: Match Maker - Expression, r(119) = .268, p < .01; Name Game, r(123) = .318, p < .001; and Parts Wholes - Expression, r(119) = .314, p < .001. For the ASD group, age correlated with performance on Match Maker - Expression measure, r(83) = .291, p < .01. Age did not reliably correlate with performance on the Name Game, r(83) = .149 or the Part/Whole r(83) = .196, measures.

IQ correlations

For the TDC group, IQ was reliably correlated with overall performance on the Part/Whole – Expression subscale, performance on r(119) = .182, p < .05. IQ did not correlate with performance on the Name Game subscale, r(119) = .137, p > .10, or performance on the MatchMaker Expression subscale, , r(119) = .004, p > .10

For the ASD group, IQ reliably correlated with performance on all three measures in the LFI! – Emotion Battery. IQ correlated with MatchMaker Expression, r(83) = .324, p < .01 and the ability to label an expression as indexed by the Name Game test, r(83) = .436, p < .001. IQ also correlated with the specific expressions of angry, r(83)= .319, disgust, , r(83)= .408, frighten, r(83)= .257, sad, r(83)= .251, p < .02, and surprise, r(83)= .353, p < .01. Finally, there was a significant correlation between IQ and overall performance on the Part/Whole–Expression subscale, r(82)= .408, p < .01. The correlations between IQ and LFI! performance were significantly stronger for the ASD group than the TDC on the Name Game and MatchMaker Expression subscales, p < .05 (two-tailed) and marginally stronger on the Parts/Wholes Expression task, p < .09 (two-tailed).

ADOS

We also correlated social impairment as measured by Module 3 of the ADOS with performance on the LFI! battery. MatchMaker Expression scores reliably correlated with ADOS Total Score, r(53) = .394, p < .01, and the ADOS Social scores, r(53) = −.332, p < .05. The ADOS Social scores also correlated with Name Game performance, r(69) = −.274, p = .05.

Discussion

The goal of this study was to investigate the processing of facial emotions by individuals from ASD and TDC groups across three expression tasks. The study included a large sample of individuals from the ASD (n = 85) and the TDC (n = 130) group. Subsamples of participants were created by matching individuals from each group on chronological age and Full Scale intelligence. For the Name Game Task, participants were shown an expressive face and then asked to select the corresponding emotional label from a set of given alternatives of happy, angry, sad, disgust, frighten and surprise. On this measure, individuals from the ASD group showed a slight deficit, p <. 10, compared to individuals from the TDC group. Comparisons on the individual expressions revealed that participants with ASD performed reliably worse than typically developing participants only on their identification of the angry expression. Anger is considered a “top half” emotion where the majority of the expressive information is conveyed in the upper half of the face (Smith, et al., 2010). Given the tendency of individuals with ASD to avoid the eye region of the face in deference to information in the lower mouth region (Klin, Jones, Schultz, Volkmar, & Cohen, 2002; Riby, Doherty-Sneddon, & Bruce, 2009; Rutherford, Clements, & Sekuler, 2007; Wolf, et al., 2008), it was not surprising that perception of anger would be differentially compromised relative to the other facial expressions.

Our main Name Game finding of no major differences between the ASD and TDC groups is consistent with other null findings reported in the literature (Castelli, 2005; Grossman, Klin, Carter, & Volkmar, 2000; Ozonoff, Pennington, & Rogers, 1990). However, other studies have found expression naming impairments in autism (Bolte & Poustka, 2003; Celani, Battacchi, & Arcidiacono, 1999). The divergent results may be explained by differences in tasks demands (e.g., perceptual versus memory) and response characteristics (e.g., free naming versus recognition). In Name Game, response demands were minimized by presenting the basic emotion labels in written and auditory form and by allowing the study faces to remain on the screen until a response was made.

We found robust differences between the ASD and TDC groups on the Matchmaker Expression measure. This test assesses the child’s ability to generalize facial emotions across different facial identities. Consistent with previous findings (Celani, et al., 1999; Riby, Doherty-Sneddon, & Bruce, 2008), we found that the ASD group’s matching performance was significantly below the performance of the TDC group on all five of the tested expressions of happy, angry, sad, frighten and disgust, p< .05. The poor performance of the ASD group on the Matchmaker task is somewhat unexpected given the intact labeling performance assessed in the Name Game measure. Labeling would be an effective strategy for completing the Matchmaker task where study and probe faces could be matched according their emotional names. As discussed below, the strong correlation between IQ and Matchmaker Expression performance suggests that verbal labeling is a good predictor of expression recognition for individuals with ASD.

The perceptual strategies of ASD and TDC participants were further explored in the Parts/Wholes Expression subscale. ASD and TDC individuals differed in their abilities to recognize the eyes and mouth features in an expressive face. When presented with an emotional face, the TDC individuals recognized eyes holistically and mouths analytically. In contrast, the ASD individuals showed the opposite pattern, recognizing mouths holistically and the eyes analytically. These results are consistent with results from previous parts/wholes studies where ASD participants have shown deficits in eye recognition and equal or superior recognition of the mouth (Joseph & Tanaka, 2003; Wolf, et al., 2008).

Our correlational findings suggest a connection between social competency and impaired perception of facial emotion. We found that individuals with ASD who exhibited greater impairment in their ability to label facial emotion and to recognize expressions across different identities showed greater deficits in social function as assessed by Module 3 of the ADOS measure. These results support the claim that impaired face processing causes a cascade of negative consequences in social function (Dawson et al.,2005; Schultz 2005). However, it is equally plausible that impaired social functioning can produce deficits in attending to and interpreting facial emotions.

An unanticipated finding in our study was the robust correlation between IQ and performance on the LFI! Emotion Battery for individuals with ASD. Results from previous studies have shown that face recognition skills are not strongly linked to general cognitive abilities in autism (Campbell et al., 2006; Langdell, 1978). However, these studies have focused on the recognition of facial identity. On tests of emotion perception and recognition, IQ reliably correlated with the ability to label an expression (Name Game, r = .44), to match a target expression across changes in identity (MatchMaker Expression, r = .32) and to identify an expression part in isolation or in a whole face (Part/Whole Expression, r = .41). The strong correlation between IQ and emotion processing for individuals with ASD stands in contrast to the modest or nil correlations for typically developing individuals on these same measures (Name Game r = .14; MatchMaker Expression, r = .01; Parts/Wholes Expression, r = .18). Group differences in IQ-emotion correlations suggest that the two populations adopt divergent strategies to decode facial emotions. For individuals with ASD, recognizing emotions may tap into higher-level, intellectual processes involving more deliberate problem solving methods whereas typically developing individuals may employ affective rather than cognitive strategies (Grossman, Klin, Carter, & Volkmar, 2000). If the emotion tasks in the LFI! Battery were more taxing for the ASD than the TDC group, the obtained group differences might reflect differences in cognitive function that are not necessarily specific to emotional processing. To test the specificity of the impairment, the performance of ASD and TDC groups could be compared on non-face control tasks that are equal in their cognitive complexity as the LFI! measures.

Conclusion

The Let’s Face It! Emotion Skills Battery provides an informative look into emotional face processing of participants with ASD. Our findings show that participants with ASD are able to label the basic facial emotions (with the exception of angry expression) on par with age- and IQ-matched typically developing participants. However, participants with ASD are impaired in their ability to generalize facial emotions across different identities and show a tendency to recognize mouth holistically and perceive the eyes as isolated parts. Results from the Let’s Face It! Emotion Skills Battery have practical implications for designing effective interventions to improve the emotion processing skills of children with ASD (Golan, Ashwin, Granader, McClintock, Day, Leggett, et al. 2010; Tanaka, Wolf, Klaiman, Koenig, Cockburn, Herlihy, L., et al., 2010). Our findings suggest that treatment programs should promote the generalization of emotion recognition across a variety of people and social settings. Instruction should cue the child to emotional information conveyed in the eyes and its perceptual integration with information in the whole face. Fostering expression recognition abilities in individuals with ASD should enhance skills in social reciprocity and build confidence and competency in everyday social interactions.

Key points

  • Children with ASD have significant deficits in their ability to recognize and interpret facial expressions
  • The cognitive strategies underlying expression recognition in autism have not been well characterized in the literature.
  • The Let’s Face It! Emotion Skills Battery is a computer-based assessment that measures the perceptual and cognitive skills that children use in recognizing facial expressions.
  • Children with ASD recognize the basic facial expressions of happy, angry, sad and disgusted as well as age- and IQ-matched control children.
  • In contrast, children with ASD are impaired in their ability to recognize expressions across different people and to encode the eyes in expression recognition.
  • These findings suggest treatment approaches for training facial emotion skills in children with ASD.

Acknowledgement

This study was funded by grants from the NIH (Studies to Advance Autism Research and Treatment), the James S. McDonnell Foundation, the National Science Foundation (#SBE-0542013) and the National Science and Engineering Research Councils of Canada.

We wish to thank the participants and their families who made this research possible.

Footnotes

The authors have declared that they have no competing or potential conflicts of interest.

References

  • Adolphs R, Sears L, Piven J. Abnormal processing of social information from faces in autism. Journal of Cogntiive Neuroscience. 2001;13:232–240. [PubMed]
  • Bolte S, Poustka F. The recognition of facial affect in autistic and schizophrenic subjects and their first-degree relatives. Psychological Medicine. 2003;33:907–915. [PubMed]
  • Boraston Z, Blakemore SJ, Chilvers R, Skuse D. Impaired sadness recognition is linked to social interaction deficit in autism. Neuropsychologia. 2007;45:1501–1510. [PubMed]
  • Burack JA, Iarocci G, Flanagan TD, Bowler DM. On mosaics and melting pots: conceptual considerations of comparison and matching strategies. Journal of Autism and Developmental Disorders. 2004;34(1):65–73. [PubMed]
  • Campbell R, Lawrence K, Mandy W, Mitra C, Jeyakuma L, Skuse D. Meanings in motion and faces: developmental associations between the processing of intention from geometrical animations and gaze detection accuracy. Developmental Psychopathology. 2006;18:99–118. [PubMed]
  • Castelli F. Understanding emotions from standardized facial expressions in autism and normal development. Autism. 2005;9:428–449. [PubMed]
  • Celani G, Battachi MW, Arcidiacono L. The understanding of emotional meaning of facial expressions in people with autism. Journal of Autism and Developmental Disorders. 1999;29:57–66. [PubMed]
  • Dawson G, Webb SJ, Wijsman E, Schellenberg G, Estes A, Munson J, et al. Neurocognitive and electrophysiological evidence of altered face processing in parents of children with autism: implications for a model of abnormal development of social brain circuitry in autism. Developmental Psychopathology. 2005;17:679–697. [PubMed]
  • Ekman P, Friesen WV. Constants across cultures in the face and emotion. Journal of Personality and Social Psychology. 1971;17:124–129. [PubMed]
  • Elliott CD. Differential Ability Scales. San Antonio, TX: Psychological Corporation; 1990.
  • Fein D, Lucci D, Braverman M, Waterhouse L. Comprehension of affect in context in children with pervasive developmental disorders. Journal of Child Psychology and Psychiatry. 1992;33:1157–1167. [PubMed]
  • Gadow KD, Sprafkin J. Child Symptom Inventories manual. Stony Brook, NY: Checkmate Plus; 1994. Reliability and validity of a DSM-III-based checklist for disruptive behavior disorders.
  • Gepner B, Deruelle C, Grynfeltt S. Motion and emotion: a novel approach to the study of face processing by young autistic children. Journal of Autism and Developmental Disorders. 2001;31:37–45. [PubMed]
  • Golan O, Ashwin E, Granader Y, McClintock S, Day K, Leggett V, et al. Enhancing emotion recognition in children with autism spectrum conditions: an intervention using animated vehicles with real emotional faces. Journal of Autism and Developmental Disorders, 2010;40:269–279. [PubMed]
  • Grossman JB, Klin A, Carter AS, Volkmar FR. Verbal bias in recognition of facial emotions in children with Asperger syndrome. Journal of Child Psychology and Psychiatry. 2000;41:369–379. [PubMed]
  • Harms MB, Martin A, Wallace GL. Facial emotion recognition in autism spectrum disorders: a review of behavioral and neuroimaging studies. Neuropsychology Review. 2010;20:290–322. [PubMed]
  • Joseph RM, Tanaka J. Holistic and part-based face recognition in children with autism. Journal of Child Psychology and Psychiatry. 2003;44:529–542. [PubMed]
  • Kanner L. Autistic disturbances of affective contact. Nervous Child. 1943;2:217–250.
  • Klin A, Jones W, Schultz R, Volkmar F, Cohen D. Visual fixation patterns during viewing of naturalistic social situations as predictors of social competence in individuals with autism. Archives of General Psychiatry. 2002;59(9):809–816. [PubMed]
  • Langdell T. Recognition of faces: An approach to the study of autism. Journal of Psychology and Psychiatry. 1978;19:255–268. [PubMed]
  • Leppanen JM, Nelson CA. The development and neural bases of facial emotion recognition. Advances in Child Development and Behavior. 2006;34:207–246. [PubMed]
  • Lord C, Rutter M, DiLavore PC, Risi S. Autism Diagnosic Observation Schedule. Los Angeles, CA: Western Psychological Services; 1999.
  • Macdonald H, Rutter M, Howlin P, Rios P, Le Conteur A, Evered C, et al. Recognition and expression of emotional cues by autistic and normal adults. Journal of Child Psychology and Psychiatry. 1989;30:865–877. [PubMed]
  • O'Connor K, Hamm JP, Kirk IJ. The neurophysiological correlates of face processing in adults and children with Asperger's syndrome. Brain and Cognition. 2005;59(1):82–95. [PubMed]
  • Ozonoff S, Pennington B, Rogers S. Executive function deficits in high-functioning autistic individuals: relationship to theory-of-mind. Journal of Child Psychology and Psychiatry. 1990;32:1081–1105. [PubMed]
  • Riby DM, Doherty-Sneddon G, Bruce V. Atypical unfamiliar face processing in Williams syndrome: what can it tell us about typical familiarity effects? Cognitive Neuropsychiatry. 2008;13(1):47–58. [PubMed]
  • Riby DM, Doherty-Sneddon G, Bruce V. The eyes or the mouth? Feature salience and unfamiliar fae processing in Williams syndrome and autism. The Quarterly Journal of Experimental Psychology. 2009;62:189–203. [PubMed]
  • Rump KM, Giovannelli JL, Minshew NJ, Strauss MS. The development of emotion recognition in individuals with autism: An expertise approach. Child Development. 2009;80:1434–1447. [PMC free article] [PubMed]
  • Rutherford MD, Clements KA, Sekuler AB. Differences in discrimination of eye and mouth displacement in autism spectrum disorders. Vision Research. 2007;47(15):2099–2110. [PubMed]
  • Rutter M, Le Couteur A, Lord C. ADI-R: The Autism Diagnostic Interview-Revised. Los Angeles, CA: Western Psychological Services; 2003.
  • Schultz RT. Developmental deficits in social perception in autism: the role of the amygdala and fusiform face area. International Journal of Developmental Neuroscience. 2005;23:125–141. [PubMed]
  • Smith MJL, Montagne B, I PD, Perrett DI, Gill M, Gallagher L. Detecting subtle facial emotion recognition deficits in high-functioning Autism using dynamic stimuli of varying intensities. Neuropsychologia. 2010;48:2777–2781. [PubMed]
  • Tanaka JW, Wolf JM, Klaiman C, Koenig K, Cockburn J, Herlihy L, et al. Using computerized games to teach face recognition skills to children with autism spectrum disorder: the Let's Face It! program. Journal of Child Psychology and Psychiatry. 2010;51:944–952. [PubMed]
  • Tottenham N, Tanaka JW, Leon AC, McCarry T, Nurse M, Hare TA, et al. The NimStim set of facial expressions: judgments from untrained research participants. Psychiatry Res. 2009;168(3):242–249. [PMC free article] [PubMed]
  • Wechsler D. Wechsler Intelligence Scale for Children – Third Edition. San Antonio, TX: Psychological Corporation; 1991.
  • Wechsler D. Wechsler Adult Intelligence Scale – Third Edition. San Antonio, TX: Psychological Corporation; 1997.
  • Wolf JM, Tanaka JW, Klaiman C, Cockburn J, Herlihy L, Brown C, et al. Specific impairment of face-processing abilities in children with autism spectrum disorder using the Let's Face It! skills battery. Autism Research. 2008;1:329–340. [PubMed]