Search tips
Search criteria 


Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
Ann N Y Acad Sci. Author manuscript; available in PMC 2010 July 6.
Published in final edited form as:
PMCID: PMC2897139

The Need for a Cognitive Neuroscience of Naturalistic Social Cognition


Understanding the minds of others is one of the great challenges humans face. Accordingly, much work in cognitive neuroscience has explored the brain systems engaged when perceivers share and make inferences about the internal states of social targets. These studies, however, typically use divergent and highly simplified stimuli and methods, and as a consequence have produced largely non-overlapping sets of results that have motivated artificially constrained theories about the processes involved in perceivers' abilities to understand targets. Here we suggest that these difficulties may stem from two main sources: the lack of meaningful behavioral data about the brain bases of perceivers' actual accuracy in inferring target states, and qualitative differences between the social stimuli used in neuroimaging paradigms and the social information perceivers encounter in the real world. We advocate more focus on studies of naturalistic social cognition, which could overcome these limitations and complement current approaches, and discuss work in our lab that has demonstrated the feasibility and utility of such paradigms. Finally, we discuss the relevance of naturalistic social cognition to diagnosing and treating autism spectrum disorder. Overall, using naturalistic paradigms in neuroimaging will be critical to modeling the way the brain actually understands other minds.

Keywords: social cognition, empathy, mirror neurons, medial prefrontal cortex

One of the great challenges faced by the human mind is the need to comprehend the content of various other minds. Every individual's interaction partners, group members, and competitors provide complex, often-contradictory cues about what they are thinking and feeling (i.e., their internal – mental – states). Moreover, the mental states of such social targets often contain cues that are critical to a perceiver's planning of their own actions. For example, if a target looks in a terrified way at something behind me, I probably should consider attending to, and potentially running away from, that thing. If I wish to gain resources through social means – either by tricking a competitor or cooperating with a partner – understanding the mental states of others becomes central to attaining these resources. This is further underscored by illnesses such as autism spectrum disorder and schizophrenia, in which inabilities to read the mental states of targets cause severe deficits in social function.

Given the importance of understanding others, it is unsurprising that a rapidly increasing body of cognitive neuroscience research has sought to explore the neural bases of social cognitive function. By and large, this work has taken one of two main approaches, which have in turn motivated strikingly different theoretical approaches to the way we understand other minds. In this article we briefly review this work, consider its strengths and weaknesses, and then propose a new direction for research on interpersonal understanding that addresses some of the shortcomings of current work. Finally, we will discuss future directions and implications that research on interpersonal understanding may have on the study of autism.

Mental State Attribution

The first cognitive neuroscience approach to understanding others has concentrated on the neural systems involved in making complex inferences about others, especially when target and perceiver mental states diverge. Consider, for example, that in some situations a perceiver has access to knowledge a target does not. Imagine a case in which a target is looking for an object, such as a coffee mug, and you (the perceiver) know that it is hidden in a non-obvious location (i.e. in the refrigerator instead of the cupboard). In those cases, while trying to infer how the target will behave, you do important cognitive work, such as inhibiting your prepotent tendency to guess that the target will act with the knowledge you have (i.e. by looking in the refrigerator), forming mental representations of the targets' intentions and beliefs based on their observable behavior, and keeping representations of both your beliefs and those of the target in mind simultaneously.

This suite of cognitive processes – which typically are referred to collectively as involving mental state attribution (MSA) – is instantiated in a system of cortical regions outlined in Figure 1. Some of these regions, such as temporoparietal junction (TPJ) and posterior cingulate cortex (PCC) may be involved in allocating attention to salient cues in the environment and assessing their relevance to the self,1-3 while other regions, such as the medial prefrontal cortex (MPFC) may be involved in forming representations of internal mental states.4-7 Interestingly, the MPFC is also engaged in forming representations of perceivers' own internal mental states and qualities,7-9 suggesting that perceivers may use common cognitive processes when forming representations about either themselves or others.

Figure 1
Brain areas involved in MSA, and brief descriptions of their functions.

Behavioral data suggests that MSA has – at least in part – piggybacked on more generalized executive processes such as inhibition of prepotent responses. MSA develops in parallel with inhibitory functions during childhood,10, 11 and systematic biases in MSA are introduced when perceivers have to perform a concurrent task while making judgments about targets.12 There is evidence that some information about targets, especially trait attributions, may be processed and retained automatically by perceivers.13-15 Nonetheless, taken together, the data suggest that MSA is not always automatic, but may require controlled processing of cues about target states, especially when a targets' knowledge or mental states diverge from those of a perceiver.16 Data from developmental psychology, as well as cognitive neuroscience, suggest that these controlled processes are critical to accurately understanding a target's mental states, at least in these divergent situations for review, see 17.

Shared Representations

In contrast to the first approach to the cognitive neuroscience of interpersonal understanding, the second approach has focused on situations in which perceivers' experiences, sensations, or actions converge with those of targets. For example, imagine watching a friend accidentally burn himself while cooking. While seeing this, you (the perceiver) may vicariously share various aspects of this experience with your friend (the target), such as a general feeling of unpleasantness or anxiety, or even a localized feeling of pain in your own finger. You might also imitate some of your friend's motor actions, such as wincing or pulling your hand back as though it too had been burned.

Behavioral and psychophysiological evidence supports the idea that, in general, people behave as you did in the above example: perceivers tend to align their actions and sensory experiences in synchrony with those they observe in targets. Thus, perceivers become physiologically aroused both when receiving pain directly and when seeing someone else in pain,18 and non-consciously imitate the facial expressions19 and actions of targets.20 It is further possible that imitation could aid a perceivers' cognitive understanding of target states, which has been explored with respect to emotional facial expressions. Posing an expression (i.e. a smile) can lead to an experience of congruent affective states,21 and allow subjects to identify congruent affective states more rapidly,22 even when people are not aware of the expressions they are posing.23

Research beginning in the early 1990s has identified brain systems that could provide a neural substrate for these behavioral effects. The studies exploring these systems have used the logic of shared representations: the idea that perceivers may employ a common cognitive and neural coding to represent both their own states and those of targets. To the extent that common systems are involved in coding responses to one's direct experience and to experiences we observe in others, those systems are said to support shared representations (see Figure 2).

Figure 2
Brain areas involved in shared representations, and brief descriptions of their functions.

Several types of data support this basic notion. For example, single unit recordings in monkeys and neuroimaging studies in humans have identified subregions of sensory and motor cortex in the parietal lobe, premotor cortex, and inferior frontal gyrus that exhibit “mirror” properties: responding both when perceivers perform actions and when they observe targets performing those actions,24-28 especially if targets' motor intentions are clear.29, 30 More recent studies have demonstrated that areas responsive to the experience of affect – such as the anterior cingulate cortex (ACC) and anterior insula (AI) respond both to a perceiver's experience of pain, disgust, and emotional facial expressions, and to observation of a target experiencing those states.31-34

Although there is no direct behavioral evidence that shared representations instantiated in these systems aid in understanding internal states, theorists have claimed that they are involved in recognizing and responding to the experiences of others.35 More specifically, such theories posit that shared representations are a sensible candidate for supporting interpersonal understanding, especially understanding of basic sensory and affective states such as pain or disgust. Though mirror-like neural and physiological responses to target states are modulated by context,36-39 such systems could work quickly and automatically, for example in underlying non-conscious motor imitation and “contagion” of emotional states.40 The idea is that mechanisms allowing quick and automatic detection of target states through shared representations are most likely helpful in orienting perceivers towards especially salient information in a targets' behavior (i.e. fearful faces indicating an environmental threat).

The Importance of Naturalistic Social Cognition

Given that both MSA and shared representations have been advanced as the source of interpersonal understanding, the independence with which they have been discussed and studied is striking. For example, neuroimaging studies engaging brain regions involved in MSA rarely also engage brain regions underlying shared representations, and visa versa though functional connectivity between these systems has been demonstrated during certain tasks; see 41. This presents a somewhat confusing theoretical picture: these social cognitive processes, while ostensibly supporting the same outcome (allowing a perceiver to understand a target's mind), seem to be operating in relative isolation. As a consequence, a debate has emerged over whether perceivers understand targets through MSA or shared representations.35, 42 More recent viewpoints recognize, however, that it is likely that MSA and shared representations both support interpersonal understanding, and are probably deployed flexibly depending on the type of cognitive resources and social cues available to perceivers.43-45 That being said, there remains a dearth of direct evidence to support this idea.

We believe that two main problems have led to this confusion about the neural bases of interpersonal understanding. First, while studies of MSA and shared representations both claim to explore the basis of a perceivers' ability to share or understand target states, such studies rely almost exclusively on indirect evidence, usually without behaviorally measuring the sharing or understanding of internal states. For example, studies of shared representations usually ask perceivers to observe and imitate target movements, or have them directly experience sensory states and observe those states in targets – all without requiring participants to make any judgments about target states or in any way behaviorally demonstrate a clear understanding of those states. As such, it is impossible to infer whether activations in such studies actually support perceivers' understanding of targets. Similarly, neuroimaging studies of MSA ask perceivers to make judgments about targets presented in simple stimuli (i.e. pictures or cartoons), but these targets are fictional, and the judgments perceivers are asked to make are generally too easy (i.e., there is a ceiling effect) to create any variability in performance that could be taken as evidence that processes supporting understanding have been called into play to varying degrees. As such, these paradigms also fail to afford any direct measure of brain activity supporting interpersonal understanding.

Here it should be noted that studies of MSA and shared representations claiming to demonstrate the brain bases of psychological processes that, in actuality, they do not measure, are committing errors similar to those made by early neuroimaging studies of emotion: they treat a complex cognitive phenomenon as a quality of a stimulus, such as shape, size or color.46, 47 For example, showing people negative or gruesome pictures may cause them to experience negative affect, but without measuring subjective experience or any other behavioral index of emotional responding, it is impossible to know whether brain activity in response to such pictures actually corresponds with the response of interest (i.e, an emotional response), or with some other process engaged by pictures (e.g. subjects distracting themselves, something about processing the perceptual aspects of stimuli, and so on). Similarly, studies of social cognition that manipulate the presence or absence of internal states in stimuli and assume that these stimuli de facto cause social cognitive processing produce results that are inherently ambiguous.

The second problem is that extant studies of shared representations and MSA have most often used simplified stimuli that differ qualitatively from the types of social information perceivers must process in real life social interactions. Using stimuli that vary only along tractable dimensions is critical to achieving tight control over the cognitive processes studied in any experiment, and the use of such stimuli has allowed for crucial progress in mapping distinct social cognitive processes in the brain. However, experimental control can come at a cost, resulting in artificially constrained ideas about the psychological processes involved.

This is especially true for the study of inherently complex phenomena such as social cognition. For example, the neuroimaging studies of MSA and shared representations described above utilize simple stimuli and tasks (i.e. imitation of finger and facial movements or judgments about mental states from pictures), such that they engage single (or limited and circumscribed) sets of processes in relative isolation. This control over stimulus properties may make it unsurprising that such studies show engagement of neural systems responsible either for shared representations or for MSA in isolation from one another.

There is no question that this type of research has been critical in creating a taxonomy of discrete processes involved in social cognition. However, taking the next inferential step – deciding that one, another, or even a combination of processes studied this way account for our social cognitive abilities in the real world – may be less straightforward than current theoretical approaches have assumed. This is because real-life social information differs from such lab stimuli in at least three critical ways. First, cues about target states in the real world are multimodal and involve visual, semantic, and prosodic information. Second, they are dynamic, involving information that is presented serially or simultaneously that has to be integrated by perceivers over time. And third, they are contextually embedded in that perceivers may have access to information (e.g. a targets' beliefs or past behaviors) that can constrain their interpretation of cues about targets' internal states.

To make this concrete, imagine hearing a friend describe their performance on a recent exam. This social target may present semantic cues (i.e. “well, I didn't do that well), and visual cues (smiling) that on the surface are not congruent, but as a perceiver, you may have contextual information (e.g. knowing that your friend is describing her performance within earshot of someone else who failed the same exam) that helps guide your processing of the cues you perceive directly. As the conversation unfolds, your ideas about your friend's internal states will shift as you perceive and account for new information she provides through multiple modalities over time.

These differences between real-life social information and the types of stimuli used in previous studies of MSA and shared representations are salient in that they may produce not only quantitative, but also qualitative differences in associated neural and cognitive processing of social information. For example, integrating information over time (i.e. segmenting event structures, picking out salient environmental cues from a changing visual field) produces unique patterns of neural activity,48, 49 including activity in neural structures involved in motor control and MSA. Furthermore, activity in MSA-related structures during perception of naturalistic events predicts subsequent memory for these events.50 Similarly, access to contextual information can change both the judgments perceivers make about targets' states,51 and the neural activity associated with making such judgments.52

Integrating aspects of naturalistic social cognition into neuroimaging

Given these points, we believe that while extant neuroimaging research has done much to advance knowledge about the processes involved in understanding other minds, limitations in the ecological validity of this work suggest a new, complementary direction for the study of social cognition. What is needed are behavioral measures producing variance in performance – and more specifically, variance in the accuracy of judgments about attributes or the perceived intensity of a target's internal states – which can be used to meaningfully connect brain activity to social perceptions or behaviors. Additionally, naturalistic, dynamic stimuli should be employed to probe the neural bases of perceiving social cues that better approximate those encountered by perceivers in the wild.

One approach to expanding the cognitive neuroscience of social cognition in these ways has been developed in our lab over the last few years. Our starting point was empathic accuracy paradigms from social psychology,53, 54 which have two appealing features relevant to naturalistic social cognition. First, they involve complex stimuli that depict actual social targets experiencing internal states dynamically across time: in this case, stimuli are videotapes of individuals discussing emotional autobiographical events. Second, these paradigms critically allow for a continuous, variable measure of social cognitive performance. Just after being taped, targets watch the videos of themselves, and continuously rate how positive or negative they felt while discussing these events. In a subsequent paradigm, perceivers continuously rate how positive or negative they believe targets felt while talking, and time series correlations between perceiver inferences and targets' self ratings are used as a measure of empathic accuracy (for a diagrammatic view of this procedure, see Figure 3).

Figure 3
Diagram of empathic accuracy task used in our studies. A) Targets are videotaped while discussing emotional autobiographical events, and later watch the videos of themselves while rating how positive or negative they felt at each moment. In a second phase, ...

Research in social psychology has demonstrated that several factors are related to perceivers' empathic accuracy for targets, including but not limited to relationships between perceivers and targets, perceivers' motivation, the modalities of information available to perceivers, and the shared physiological arousal between perceivers and targets.54-59 Furthermore, empathic accuracy predicts social adjustment in adolescents,60 and is impaired in autism spectrum disorder, an illness characterized by deficits in social interactions,61 suggesting that this type of social cognitive performance may meaningfully relate to social functioning.

These aspects of empathic accuracy made the paradigms for studying it appealing for use in neuroimaging. In our first study of this type, we scanned perceivers while they rated several target videos and assessed their accuracy about targets' emotional states on a video-by-video basis. We then searched for brain regions whose activity tracked parametrically with a perceivers' accuracy; or in other words, regions that were selectively engaged during periods of accurate, as opposed to inaccurate inferences (see Figure 4). Results indicated that both regions classically involved in MSA, including the medial PFC and superior temporal sulcus, and parts of the mirror neuron system involved in shared representations, including the inferior parietal lobule and the premotor cortex, tracked with the accuracy of inferences made about these complex social stimuli (Figure 5, see also Zaki et al.62)

Figure 4
Diagram of parametric analyses used to assess neural correlates of empathic accuracy. Accuracy correlation scores from each perceivers' videos were entered as parametric modulators to predict neural activity. This was performed for each perceiver, and ...
Figure 5
Brain regions whose engagement was related to levels of empathic accuracy for perceivers showed about target affect in a given video.

This paradigm serves as a demonstration of methodological feasibility, and its results motivate further use of a naturalistic approach to social cognition. By employing a complex, naturalistic task, this study provides evidence that neural systems underlying shared representations and MSA work in concert while perceivers make inferences about targets' internal states. The use of a meaningful performance measure further confirmed that the engagement of both of these systems is related to empathic accuracy – in other words, that this activity tracks with a fully operationalized, empirically defined understanding of a target's mind.

Representing targets' dispositions in perceivers' brains

Another advantage of exploring naturalistic social cognition in the brain is that it allows for examination of social cognition as a truly interpersonal phenomenon. While social cognitive theories commonly argue that attributes of both a target and a perceiver contribute actively to that perceiver's inferences,63-65 neuroscience views of empathy and social cognition – in part because of methodological constraints – have focused exclusively on the cognitive processes perceivers engage in when thinking about other minds.

While this approach has advanced knowledge about intrapersonal aspects of social cognition in perceivers, it may ignore critical aspects of social inference processes as they occur outside the decidedly non-social space inside a scanner. For example, behavioral studies have shown that aspects of a targets' personality strongly predict outcomes such as empathic accuracy. Specifically, targets who report being high in dispositional emotional expressivity as indexed using measures developed by Ickes et al.66, 67 are also more affectively “readable,” producing higher levels of empathic accuracy regardless of the perceiver viewing them.64, 68 This may be because high expressivity targets produce social cues (i.e. emotional language and facial expressions) that telegraph corresponding internal states more clearly than cues given off by low expressivity targets.56, 59 Perception of these “high fidelity” cues could in turn influence the social cognitive processes perceivers typically engage to understand targets. In other words, while perceivers may employ common intrapersonal cognitive processes when seeking to understand the mental states present in a cartoon, picture, or real-life target, these processes do not occur in a vacuum, and may importantly be affected by interpersonal factors such as targets' behaviors and dispositions.

Using real social targets as stimuli in cognitive neuroscience studies affords researchers the ability to address this issue by monitoring exactly the ways in which differences between social targets can affect the neural processes of perceivers. To explore this, we used parametric analyses similar to the ones described above to search for perceiver brain regions tracking with the expressivity of a target they were viewing in a given video. Results demonstrated that target expressivity related to perceivers' engagement of several brain regions commonly associated with MSA, including large sections of dorsal MPFC and the premotor cortex, which is part of the mirror neuron system engaged during sharing of sensorimotor representations (see Figure 6). Conjunction analyses revealed that both the MPFC and premotor cortex areas tracking with target expressivity also tracked with perceivers' empathic accuracy. Together, this evidence suggests that expressive targets may produce high-fidelity social cues that in turn cause perceivers to more strongly engage neural and cognitive processes allowing them to be accurate about target states. This interpersonal account of social cognition in the brain provides another example of how naturalistic paradigms can better approximate real-life social cognition and foster exploration of its neural bases.

Figure 6
Brain regions whose engagement (in perceivers) was related to targets' levels of emotional expressivity, as measured by a self-report measure.

Future directions and ties to autism spectrum disorder

The use of naturalistic stimuli in neuroimaging is an exciting avenue of research that will allow investigators to probe the roots of cognitive processes that occur in the real world, but heretofore have not been possible to examine within the tight constraints of cognitive neuroscience paradigms. This is especially important to the study of social cognition, because exploring social cognitive processes using simplified stimuli, examining perceivers in isolation, and ignoring behavioral correlates of neural activity can produce research methods and results that may not map on to the way people understand each other in the richly complex social world.

New research employing a naturalistic approach to social cognition will be able to complement the tighter, more controlled work that has dominated the field until now. Two important challenges for this research will be using meaningful behavioral measures of social cognitive success or bias, and employing more realistic stimuli involving social cues that are multimodal, dynamic, and contextually embedded.

Existing work in shared representations has begun addressing the first of these challenges by demonstrating links between the intensity of pain a perceiver rates a target as experiencing and activity in that perceiver's pain matrix.69 Similar paradigms are being developed to monitor the behavioral effects of MSA using variance in reaction time or allocentric biases in spatial perspective taking.70, 71 For example, work in social psychology has demonstrated that perceivers with access to unique knowledge overestimate the extent to which naïve social targets will use that knowledge when making decisions.72, 73 Similarly, perceivers overestimate the extent to which an aspect of the environment that is emotionally salient to them (e.g. the Barry Manilow t-shirt the perceiver has been forced to wear) will be salient to others.74, 75 Presumably, the extent of these biases is not fixed, and perceivers attending most closely to targets' internal states may attenuate perceivers' bias in making such judgments. Adapting such paradigms to a neuroimaging context will allow researchers to understand the brain bases of not only social cognitive inferences, but also cognitive and neural predictors of the efficacy of such inferences.

The second important challenge to imaging naturalistic social cognition also has been addressed in new lines of research. In addition to the work described in this article, the feasibility of using dynamic, naturalistic stimuli has been demonstrated by paradigms employing complex stimuli such as videos successfully in neuroimaging research.48, 50, 76

Of course, even using realistic stimuli and performance measures are not enough to capture much of the richness of social cognition as it occurs in the real world. For example, as we pointed out above, oftentimes when perceivers consider the mental states of targets, they make inferences using not only the cues available to them at the moment, but also pre-existing contextual information they have that may constrain their expectations about targets' mental states. For example, knowing that you are seeing a friend the day after the football team he follows obsessively loses the superbowl, you may have a specific expectation that he will feel upset or dejected, and may see signs of those emotions that you would have otherwise ignored in his behavior. There is some evidence from social psychology and cognitive neuroscience that such contextual information indeed can impact the way perceivers process and judge basic social cues.51, 52, 77 However, the majority of this research has been conducted with static picture stimuli. Examining how contextual information changes judgments about more naturalistic social information will be an important future direction.

Another import factor as of yet unexplored in cognitive neuroscience is the way that interpersonal dynamics contextualize and change how perceivers make inferences outside the lab. In real life, if a perceiver is unsure about what inference to make about a target's internal states, they need not – and probably will not – sit back and passively ponder what a target is experiencing. Instead, they will actively pursue information about targets: asking them how they feel or what they are thinking, or indirectly probing for cues about these states. Further, the information they pursue will often be biased towards confirmation of their previously existing beliefs about social targets.78-80 Similarly, targets will constrain their behavior to match the social roles they wish to fulfill.81 This role will importantly vary based on the perceiver observing that target: imagine a college student's shifting behaviors as they interact with their professors, parents, roommates, and romantic partners. A target's behavioral shifts will in turn alter the way perceivers make inferences about that target, often allowing perceivers to be accurate about targets in some, but not other situations referred to as “circumscribed accuracy” by Swann65 Thus, the process of social cognition involves an interpersonal negotiation that will be quite difficult to capture in controlled experimental laboratory settings, let alone using neuroimaging. Nonetheless, we believe it is important to prioritize attempts at capturing as much of the social cognitive process as possible in experimental settings.

Finally, the use of naturalistic social cognition will be beneficial – and may be critical – to understanding illnesses involving social cognitive deficits. For example, autism spectrum disorder (ASD) is characterized in part by deficits in reciprocal social interactions, which have been long related to difficulties in accurate mental state attribution82, 83 and in spontaneous motor imitation.84, 85 Brain bases of such behavioral abnormalities have been reported more recently: even high functioning individuals with ASD show less activity in the mirror neuron system during imitation tasks,86 and less activity in MSA related regions such as the MPFC while making explicit inferences about target internal states.87, 88

This evidence has motivated several prominent theories concerning the neurocognitive bases of social deficits in ASD. Unfortunately, these theories have often been overly constrained, suggesting that ASD is an illness purely defined either by deficits in shared representations (i.e. the “Broken Mirror Hypothesis” of Ramachandran and Oberman89), or by deficits in explicit MSAs.90 In essence, neuroscientific theories about social cognitive dysfunction in ASD have often reproduced the problems of neuroscientific theories of normative social cognition, by hanging a richly complex problem in social interaction on abnormalities in single cognitive or motor processes. Compounding this problem is the fact that – as described above – the simplified methods used to study social cognitive processes in isolation may not serve as realistic proxies for the social world, social cognition, or its deficits in ASD.

Data support the idea that single-process models of social deficits in ASD are insufficient. For example, not all studies demonstrate problems in MSA when employing simplified tasks.91 Further, the few studies attempting to directly link behavioral deficits in MSA with social symptom severity assessed clinically often have failed to find such relationships.92, 93 Finally, interventions aimed at improving social cognitive performance in simplified tasks (i.e. by training people with ASD to recognize basic emotional facial expressions in pictures) often produce improvements on these tasks, without causing any improvements in clinically assessed social interaction abilities.94-96

These disparities underscore the qualitative differences between tasks used to assess social cognition and the types of social cognition necessary to real-life interactions. They also suggest that the social deficits in ASD may stem from inabilities to carry out inferences about complex, contextually embedded social cues. In fact, the two extant studies examining empathic accuracy in ASD, using paradigms similar to the ones described above, support such assertions. First, while these individuals performed normally on simplified emotion recognition tasks, they showed more severe impairments in naturalistic empathic accuracy tasks.61 Second, these deficits were only exhibited in certain situations: people with ASD were less accurate about targets engaging in an unstructured interaction with each other, but not about targets who were interviewing each other in a structured way asking each other questions, such as “what do you like to do in your spare time?” from a list; see Ponnet et al.97 This suggests that individuals with ASD may be better able to interpret especially clear and transparent cues about target internal states that do not require contextual information to decode. One intriguing possibility is that expressive individuals, who give off more frequent and direct cues about their internal states, could be more “readable” to perceivers with ASD, potentially through the increased engagement of MSA-related brain regions associated with viewing such targets. Were this to be the case, it could motivate a novel form of intervention for ASD, in which caregivers and family members of individuals with ASD could restructure their behavior to provide clear, readable cues about their internal states, thereby improving the ability of people with ASD to understand the minds of others not only in the lab, but in the perpetually complex social world.


The possibility of finding new diagnostic techniques and interventions for ASD highlights the fact that naturalistic paradigms can allow researchers to make headway not possible using current standard techniques for assessing social cognition. By moving towards paradigms that capture the complexity of the real social world, and assessing perceivers' abilities to make accurate inferences about targets, neuroimaging of social cognition can approach more ecologically valid theories about how minds understand each other.


The authors wish to thank L. Fuhrman and J. Spicer for helpful comments on this manuscript, and NIDA Grant 1R01DA022541-01 (to KO) and Autism Speaks Grant 4787 (to JZ) for support of this work.


1. Vogt BA, Vogt L, Laureys S. Cytology and functionally correlated circuits of human posterior cingulate areas. Neuroimage. 2006;29:452–466. [PMC free article] [PubMed]
2. Decety J, Lamm C. The role of the right temporoparietal junction in social interaction: how low-level computational processes contribute to meta-cognition. Neuroscientist. 2007;13:580–593. [PubMed]
3. Mitchell JP. Activity in right temporo-parietal junction is not selective for theory-of-mind. Cereb Cortex. 2008;18:262–271. [PubMed]
4. Fletcher PC, et al. Other minds in the brain: a functional imaging study of “theory of mind” in story comprehension. Cognition. 1995;57:109–128. [PubMed]
5. Goel V, Grafman J, Sadato N, Hallett M. Modeling other minds. Neuroreport. 1995;6:1741–1746. [PubMed]
6. Mitchell JP, Heatherton TF, Macrae CN. Distinct neural systems subserve person and object knowledge. Proc Natl Acad Sci U S A. 2002;99:15238–15243. [PubMed]
7. Ochsner KN, et al. Reflecting upon feelings: an fMRI study of neural systems supporting the attribution of emotion to self and other. J Cogn Neurosci. 2004;16:1746–1772. [PubMed]
8. Kelley WM, et al. Finding the self? An event-related fMRI study. J Cogn Neurosci. 2002;14:785–794. [PubMed]
9. Mitchell JP, Banaji MR, Macrae CN. The link between social cognition and self-referential thought in the medial prefrontal cortex. J Cogn Neurosci. 2005;17:1306–1315. [PubMed]
10. Carlson SM, Moses LJ. Individual differences in inhibitory control and children's theory of mind. Child Dev. 2001;72:1032–1053. [PubMed]
11. Carlson SM, Moses LJ, Claxton LJ. Individual differences in executive functioning and theory of mind: An investigation of inhibitory control and planning ability. J Exp Child Psychol. 2004;87:299–319. [PubMed]
12. Gilbert D, Pelham B, Krull D. On Cognitive Busyness: When Person Perceivers Meet Persons Perceived. Journal of Personality & Social Psychology. 1989;54:733–740.
13. Todorov A, Uleman JS. Spontaneous trait inferences are bound to actors' faces: evidence from a false recognition paradigm. J Pers Soc Psychol. 2002;83:1051–1065. [PubMed]
14. Todorov A, Uleman JS. The person reference process in spontaneous trait inferences. J Pers Soc Psychol. 2004;87:482–493. [PubMed]
15. Winter L, Uleman JS, Cunniff C. How automatic are social judgments? J Pers Soc Psychol. 1985;49:904–917. [PubMed]
16. Apperly IA, Riggs KJ, Simpson A, Chiavarino C, Samson D. Is belief reasoning automatic? Psychol Sci. 2006;17:841–844. [PubMed]
17. Saxe R, Carey S, Kanwisher N. Understanding other minds: linking developmental psychology and functional neuroimaging. Annu Rev Psychol. 2004;55:87–124. [PubMed]
18. Vaughan KB, Lanzetta JT. Vicarious instigation and conditioning of facial expressive and autonomic responses to a model's expressive display of pain. J Pers Soc Psychol. 1980;38:909–923. [PubMed]
19. Dimberg U, Thunberg M, Elmehed K. Unconscious facial reactions to emotional facial expressions. Psychol Sci. 2000;11:86–89. [PubMed]
20. Chartrand TL, Bargh JA. The chameleon effect: the perception-behavior link and social interaction. J Pers Soc Psychol. 1999;76:893–910. [PubMed]
21. Strack F, Martin LL, Stepper S. Inhibiting and facilitating conditions of the human smile: a nonobtrusive test of the facial feedback hypothesis. J Pers Soc Psychol. 1988;54:768–777. [PubMed]
22. Stel M, van Knippenberg A. The role of facial mimicry in the recognition of affect. Psychol Sci. 2008;19:984–985. [PubMed]
23. Niedenthal P, Brauer M. When did her smile drop? Facial mimicry and the influences of emotional state on the detection of change in emotional expression. Cognition & Emotion. 2001;15:853–864.
24. di Pellegrino G, Fadiga L, Fogassi L, Gallese V, Rizzolatti G. Understanding motor events: a neurophysiological study. Exp Brain Res. 1992;91:176–180. [PubMed]
25. Gallese V, Fadiga L, Fogassi L, Rizzolatti G. Action recognition in the premotor cortex. Brain. 1996;119(Pt 2):593–609. [PubMed]
26. Rizzolatti G, Fadiga L, Gallese V, Fogassi L. Premotor cortex and the recognition of motor actions. Brain Res Cogn Brain Res. 1996;3:131–141. [PubMed]
27. Fogassi L, et al. Parietal lobe: from action organization to intention understanding. Science. 2005;308:662–667. [PubMed]
28. Iacoboni M, et al. Cortical mechanisms of human imitation. Science. 1999;286:2526–2528. [PubMed]
29. de Lange FP, Spronk M, Willems RM, Toni I, Bekkering H. Complementary systems for understanding action intentions. Curr Biol. 2008;18:454–457. [PubMed]
30. Iacoboni M, et al. Grasping the intentions of others with one's own mirror neuron system. PLoS Biol. 2005;3:e79. [PubMed]
31. Carr L, Iacoboni M, Dubeau MC, Mazziotta JC, Lenzi GL. Neural mechanisms of empathy in humans: a relay from neural systems for imitation to limbic areas. Proc Natl Acad Sci U S A. 2003;100:5497–5502. [PubMed]
32. Morrison I, Downing PE. Organization of felt and seen pain responses in anterior cingulate cortex. Neuroimage. 2007;37:642–651. [PubMed]
33. Singer T, et al. Empathy for pain involves the affective but not sensory components of pain. Science. 2004;303:1157–1162. [PubMed]
34. Wicker B, et al. Both of us disgusted in My insula: the common neural basis of seeing and feeling disgust. Neuron. 2003;40:655–664. [PubMed]
35. Gallese V, Keysers C, Rizzolatti G. A unifying view of the basis of social cognition. Trends Cogn Sci. 2004;8:396–403. [PubMed]
36. Lamm C, Batson CD, Decety J. The neural substrate of human empathy: effects of perspective-taking and cognitive appraisal. J Cogn Neurosci. 2007;19:42–58. [PubMed]
37. Lanzetta JT, Englis BG. Expectations of cooperation and competition and their effects on observers vicarious emotional responses. J Pers Soc Psychol. 1989;56:543–554.
38. Singer T, et al. Empathic neural responses are modulated by the perceived fairness of others. Nature. 2006;439:466–469. [PMC free article] [PubMed]
39. Gu X, Han S. Attention and reality constraints on the neural processes of empathy for pain. Neuroimage. 2007;36:256–267. [PubMed]
40. Neumann R, Strack F. “Mood contagion”: the automatic transfer of mood between persons. J Pers Soc Psychol. 2000;79:211–223. [PubMed]
41. Zaki J, Ochsner KN, Hanelin J, Wager T, Mackey SC. Different circuits for different pain: Patterns of functional connectivity reveal distinct networks for processing pain in self and others. Social Neuroscience. 2007;2:276–291. [PMC free article] [PubMed]
42. Saxe R. Against simulation: the argument from error. Trends Cogn Sci. 2005;9:174–179. [PubMed]
43. Decety J, Jackson PL. The functional architecture of human empathy. Behav Cogn Neurosci Rev. 2004;3:71–100. [PubMed]
44. Olsson A, Ochsner KN. The role of social cognition in emotion. Trends Cogn Sci. 2008;12:65–71. [PubMed]
45. Singer T. The neuronal basis and ontogeny of empathy and mind reading: review of literature and implications for future research. Neurosci Biobehav Rev. 2006;30:855–863. [PubMed]
46. Ochsner K, Feldman-Barrett L. A multiprocess perspective on the neuroscience of emotion. In: M TJ, Bonnano G, editors. Emotion: Current Issues and Future Directions. (Guilford Press; New York: 2001. pp. 38–81.
47. Ochsner KN, et al. Top-down and bottum-up processes in emotion generation. Psychol Sci. under review.
48. Hasson U, Nir Y, Levy I, Fuhrmann G, Malach R. Intersubject synchronization of cortical activity during natural vision. Science. 2004;303:1634–1640. [PubMed]
49. Zacks JM, et al. Human brain activity time-locked to perceptual event boundaries. Nat Neurosci. 2001;4:651–655. [PubMed]
50. Hasson U, Furman O, Clark D, Dudai Y, Davachi L. Enhanced intersubject correlations during movie viewing correlate with successful episodic encoding. Neuron. 2008;57:452–462. [PMC free article] [PubMed]
51. Carroll JM, Russell JA. Do facial expressions signal specific emotions? Judging emotion from the face in context. J Pers Soc Psychol. 1996;70:205–218. [PubMed]
52. Kim H, et al. Contextual modulation of amygdala responsivity to surprised faces. J Cogn Neurosci. 2004;16:1730–1745. [PubMed]
53. Ickes W. Empathic Accuracy. Guilford Press; New York: 1997.
54. Ickes W, Stinson L, Bissonnette V, Garcia S. Naturalistic social cognition: Empathic accuracy in mixed-sex dyads. Journal of Personality & Social Psychology. 1990;59:730–742.
55. Gesn P, Ickes W. The development of meaning and contexts for empathic accuracy: Channel and sequence effects. Journal of Personality and Social Psychology. 1999;77:746–761.
56. Hall JA, Schmid Mast M. Sources of accuracy in the empathic accuracy paradigm. Emotion. 2007;7:438–446. [PubMed]
57. Levenson RW, Ruef AM. Empathy: a physiological substrate. Journal of Personality & Social Psychology. 1992;63:234–246. [PubMed]
58. Pickett CL, Gardner WL, Knowles M. Getting a cue: the need to belong and enhanced sensitivity to social cues. Personality & Social Psychology Bulletin. 2004;30:1095–1107. [PubMed]
59. Zaki J, Bolger N, Ochsner K. Unpacking the informational bases of empathic accuracy. Emotion. 2009 Aug;9(4):478–87. [PubMed]
60. Gleason K, Jensen-Campbell L, Ickes W. The role of empathic accuracy in adolescents' peer relations and adjustment. Personality and Social Psychology Bulletin. under review. [PubMed]
61. Roeyers H, Buysse A, Ponnet K, Pichal B. Advancing advanced mind-reading tests: empathic accuracy in adults with a pervasive developmental disorder. J Child Psychol Psychiatry. 2001;42:271–278. [PubMed]
62. Zaki J, Weber J, Bolger N, Ochsner K. Neural bases of empathic accuracy. Proc Natl Acad Sci U S A. 2009 Jul 7;106(27):11382–7. [PubMed]
63. Kenny DA, Albright L. Accuracy in interpersonal perception: a social relations analysis. Psychol Bull. 1987;102:390–402. [PubMed]
64. Snodgrass SE, Hecht MA, Ploutz-Snyder R. Interpersonal sensitivity: expressivity or perceptivity? Journal of Personality & Social Psychology. 1998;74:238–249. [PubMed]
65. Swann WB., Jr Quest for accuracy in person perception: a matter of pragmatics. Psychol Rev. 1984;91:457–477. [PubMed]
66. Gross J. The Berkeley Expressivity Questionnaire. In: Maltby J, Lewis CA, Hill A, editors. Commissioned reviews on 300 psychological tests. Edwin Mellen Press; Lampeter, Wales: 2000.
67. Gross J, John OP. Revealing feelings: Facets of emotional expressivity in self-reports, peer ratings, and behavior. J Pers Soc Psychol. 1997;72:435–448. [PubMed]
68. Zaki J, Bolger N, Ochsner K. It takes two: The interpersonal nature of empathic accuracy. Psychological Science. 2008;19:399–404. [PubMed]
69. Saarela MV, et al. The Compassionate Brain: Humans Detect Intensity of Pain from Another's Face. Cereb Cortex. 2007;17:230–237. [PubMed]
70. Choudhury S, Blakemore SJ, Charman T. Social cognitive development during adolescence. Soc Cogn Affect Neurosci. 2006;1:165–174. [PMC free article] [PubMed]
71. Samson D, Apperly IA, Kathirgamanathan U, Humphreys GW. Seeing it my way: a case of a selective deficit in inhibiting self-perspective. Brain. 2005;128:1102–1111. [PubMed]
72. Epley N, Gilovich T. The anchoring-and-adjustment heuristic: why the adjustments are insufficient. Psychol Sci. 2006;17:311–318. [PubMed]
73. Epley N, Keysar B, Van Boven L, Gilovich T. Perspective taking as egocentric anchoring and adjustment. J Pers Soc Psychol. 2004;87:327–339. [PubMed]
74. Epley N, Savitsky K, Gilovich T. Empathy neglect: reconciling the spotlight effect and the correspondence bias. J Pers Soc Psychol. 2002;83:300–312. [PubMed]
75. Gilovich T, Medvec VH, Savitsky K. The spotlight effect in social judgment: an egocentric bias in estimates of the salience of one's own actions and appearance. J Pers Soc Psychol. 2000;78:211–222. [PubMed]
76. Hutcherson CA, et al. Attention and emotion: does rating emotion alter neural responses to amusing and sad films? Neuroimage. 2005;27:656–668. [PubMed]
77. Aviezer H, et al. Angry, disgusted, or afraid? Studies on the malleability of emotion perception. Psychol Sci. 2008;19:724–732. [PubMed]
78. Meichenbaum DH, Bowers KS, Ross RR. A behavioral analysis of teacher expectancy effect. J Pers Soc Psychol. 1969;13:306–316. [PubMed]
79. Rosenthal R, Jacobson L. Pygmalion in the Classroom. Rinehart & Wilson; New York, NY: 1968.
80. Snyder M, Swann W. Hypothesis testing processes in social interactions. Journal of Personality and Social Psychology. 1978;36:1202–1212.
81. Goffman E. The Presentation of the Self in Everyday Life. Anchor Press; New York, NY: 1959.
82. Baron-Cohen S. Mindblindness. MIT Press; Cambridge, Mass: 1994.
83. Baron-Cohen S, Leslie AM, Frith U. Does the autistic child have a “theory of mind”? Cognition. 1985;21:37–46. [PubMed]
84. McIntosh DN, Reichmann-Decker A, Winkielman P, Wilbarger JL. When the social mirror breaks: deficits in automatic, but not voluntary, mimicry of emotional facial expressions in autism. Dev Sci. 2006;9:295–302. [PubMed]
85. Rogers SJ, Hepburn SL, Stackhouse T, Wehner E. Imitation performance in toddlers with autism and those with other developmental disorders. J Child Psychol Psychiatry. 2003;44:763–781. [PubMed]
86. Dapretto M, et al. Understanding emotions in others: mirror neuron dysfunction in children with autism spectrum disorders. Nat Neurosci. 2006;9:28–30. [PMC free article] [PubMed]
87. Baron-Cohen S, et al. Social intelligence in the normal and autistic brain: an fMRI study. Eur J Neurosci. 1999;11:1891–1898. [PubMed]
88. Wang AT, Lee SS, Sigman M, Dapretto M. Reading affect in the face and voice: neural correlates of interpreting communicative intent in children and adolescents with autism spectrum disorders. Arch Gen Psychiatry. 2007;64:698–708. [PMC free article] [PubMed]
89. Oberman LM, Ramachandran VS. The simulating social mind: the role of the mirror neuron system and simulation in the social and communicative deficits of autism spectrum disorders. Psychol Bull. 2007;133:310–327. [PubMed]
90. Baron-Cohen S, Wheelwright S, Hill J, Raste Y, Plumb I. The “Reading the Mind in the Eyes” Test revised version: a study with normal adults, and adults with Asperger syndrome or high-functioning autism. J Child Psychol Psychiatry. 2003;42:241–251. [PubMed]
91. Castelli F. Understanding emotions from standardized facial expressions in autism and normal development. Autism. 2005;9:428–449. [PubMed]
92. Fombonne E, Siddons F, Achard S, Frith U, Happe F. Adaptive behaviour and theory of mind in autism. Eur Child Adolesc Psychiatry. 1994;3:176–186.
93. Tager-Flusberg H. Evaluating the theory-of-mind hypothesis in autism. Current directions in psychological science. 2007;16:311–315.
94. Golan O, Baron-Cohen S. Systemizing empathy: teaching adults with Asperger syndrome or high-functioning autism to recognize complex emotions using interactive multimedia. Dev Psychopathol. 2006;18:591–617. [PubMed]
95. Hadwin J, Baron-Cohen S, Howlin P, Hill K. Can we teach children with autism to understand emotion, belief, or pretense. Development and Psychopathology. 1996;8
96. Hadwin J, Baron-Cohen S, Howlin P, Hill K. Does teaching theory of mind have an effect on the ability to develop conversation in children with autism? J Autism Dev Disord. 1997;27:519–537. [PubMed]
97. Ponnet K, Buysse A, Roeyers H, De Clercq A. Mind-Reading in Young Adults with ASD: Does Structure Matter. J Autism Dev Disord. 2007 [PubMed]