|Home | About | Journals | Submit | Contact Us | Français|
There has been considerable interest in identifying the neural correlates of action naming, but the bulk of previous work on this topic has utilized static stimuli. Recent research comparing the visual processing of dynamic versus static actions suggests that these two types of stimuli engage largely overlapping neural systems, raising the possibility that the higher-order processing requirements for naming dynamic and static actions might not be very different. To explore this issue in greater depth, we developed the Dynamic Action Naming Test (DANT), which consists of 158 video clips 3–5 sec in length, for each of which the participant is asked to produce the most appropriate verb. We administered the DANT to 78 brain-damaged patients drawn from our Patient Registry, and to a demographically matched group of 50 normal participants. Out of the 16 patients who performed defectively on the DANT, nearly all (15/16) had damage in the left hemisphere. Lesion analysis indicated that the frontal operculum was the most frequent area of damage in the 15 patients; also, damage to the posterolateral temporal-occipital sector (in and near MT) was specifically related to defective dynamic action naming. Most of the brain-damaged participants (n = 71) also received our Static Action Naming Test (SANT), and we found that performances on verb items that were common across the DANT and SANT were highly correlated (R = .91). Moreover, patients who failed the DANT almost invariably also failed the SANT. These findings lend further support to the hypothesis that there is considerable commonality in the neural systems underlying the use of verbs to orally name dynamic and static actions, a conclusion that is in turn compatible with the concept of “representational momentum.” Our results also contribute more generally to the rapidly growing field of research on embodied cognition.
In recent years there has been significant progress in understanding the neural basis for action-related knowledge. An important unanswered question, however, is whether there are significant differences in the brain systems underlying the observation and naming of actions that are presented statically (i.e., as snapshots or stills) versus dynamically (i.e., in real time). The primary objective of the study reported here was to explore this question with a neuropsychological approach that involved comparing the performances of brain-damaged patients on action naming tasks that utilized either static or dynamic stimuli. We begin by reviewing the literature relevant to these issues.
The observation of actions that unfold dynamically in real time is known to engage numerous neuroanatomical structures (for a review see Jeannerod, 2006). Here we restrict our discussion to three cortical areas that have been the focus of considerable research: (1) the posterolateral temporal-occipital region, including area MT; (2) the inferior parietal lobule, including the intraparietal sulcus; and (3) the premotor and primary motor cortices.
It is natural to start with area MT (or V5, also known as MT+ or hMT+), since it is the first cortical region that is reliably and specifically activated by all manner of visual motion processing (for a review see Malikovic et al., 2006). A number of studies with fMRI, PET, TMS, and other techniques have demonstrated MT to be fairly consistently located at the borders of lateral BA1 37 and 19, in the anterior part of the occipital lobe at the continuation of the inferior temporal sulcus (e.g., Dumoulin et al., 2000; Malikovic et al., 2006; Tootell et al., 1995; Watson et al., 1993; Zeki et al., 1991). Malikovic et al. (2006) specified cytoarchitectonic criteria for this region, and suggested the term “area hOc5.” Although MT does not itself appear to distinguish systematically between different types of object-associated motion, it projects to higher-level lateral temporal areas that clearly do. One processing stream extends from MT into areas of the posterior superior temporal sulcus (STSp) that respond preferentially to biological motion patterns, especially human ones (e.g., Bonda et al., 1996; Decety & Grézes, 1999; Grézes et al., 2001; Grossman & Blake, 2002; Grossman et al., 2000; Kable et al., 2006; Peelen et al., 2006; Pelphrey et al., 2003, 2005; Puce et al., 1998; Saygin et al., 2004b; Vaina et al., 2001; for reviews, see Blake & Shiffrar, 2007; Grossman, 2006; Puce & Perrett, 2003; for a neurophysiologically plausible computational model, see Giese & Poggio, 2003, and Giese, 2006). Another processing stream extends from MT into areas of the middle temporal gyrus (MTG) that respond preferentially to nonbiological motion patterns, especially tool ones (Beauchamp et al., 2002, 2003; Beauchamp & Martin, 2007; Martin, 2007; Martin & Weisberg, 2003; Weisberg et al., in press; for a review see Lewis, 2006). The different specializations of these two visual motion processing streams are nicely illustrated by an fMRI study by Beauchamp et al. (2002). Several regions in posterolateral temporal-occipital cortex, including MT, STSp, and MTG, responded more to human and tool motion than to simple moving gratings; however, STSp preferred the complex articulated motion characteristic of biological movements, whereas MTG preferred the simple unarticulated motion characteristic of tool movements.
The posterolateral temporal-occipital cortex projects to the inferior parietal lobule (IPL), and the latter region is also engaged during action observation (and has been shown to respond selectively to motion stimuli, e.g., Tootell et al., 1995). IPL activity has been reported when subjects watch certain kinds of non-object-directed actions, e.g., whole-body dance sequences, which engage the IPL in proportion to the observers’ ratings of their own ability to perform the movements (Calvo-Merino et al., 2006; Cross et al., 2006). However, the IPL usually responds even more strongly to the visual perception of bodily actions that are directed toward objects in peripersonal space. For example, Buccino et al. (2001) demonstrated that the observation of object-directed versus non-object-directed actions elicited bilateral parietal activations that were somatotopically organized insofar as they differentiated between mouth, hand, and foot movements. In addition, more recent research indicates that the anterior intraparietal sulcus (AIP) is not only recruited during the observation of object-directed hand actions (e.g., Shmuelof & Zohary, 2005, 2006), but may even represent the intended goal of the agent’s movement (Gazzola et al., 2007; Hamilton & Grafton, 2006; Tunik et al., 2005; see also Fogassi et al., 2005, and Tunik et al., in press, for review). Finally, activation in and around this same region is evoked when subjects watch videos of people using tools (Manthey et al., 2003), a result that converges with a large literature relating manipulation knowledge of tools with the IPL, predominantly in the left hemisphere (for reviews see Johnson-Frey, 2004, and Lewis, 2006; see also Goldenberg & Iriki, 2007).
The IPL projects to the premotor cortex, and ever since the seminal discovery of “mirror” neurons in area F5 of the macaque (Di Pellegrino et al., 1992), there has been a rapid accumulation of evidence that the premotor cortex contributes not only to action execution but also to action recognition (for recent reviews see Iacoboni & Dapretto, 2006; Keysers & Gazzola, 2006; Rizzolatti & Craighero, 2004). For instance, observation of mouth, hand, and foot movements triggers somatotopically organized premotor activation (Buccino et al., 2001; Sakreida et al., 2005; Wheaton et al., 2004), and the strength of this activation is modulated by the observer’s ability to perform the actions (Bangert & Altenmuller, 2003; Buccino et al., 2004; Calvo-Merino et al., 2006; Cross et al., 2006; Haueisen & Knosche, 2001; Stevens et al., 2000). In addition, the involvement of the primary motor cortex during the observation of dynamic human actions has been suggested by several studies utilizing a variety of techniques, including EEG (Caetano et al., 2007; Cochin et al., 1999; Oberman et al., 2007a, 2007b; Van Schie et al., 2004), MEG (Hari et al., 1998; Jarvelainen et al., 2004; Van Schie et al., in press), and TMS (e.g., Borroni et al., 2005; Clark et al., 2004; Fadiga et al., 1995b; Maeda et al., 2002; Montagna et al., 2005; for a review see Fadiga et al., 2005a). It is also noteworthy that a growing body of research points to direct top-down motoric influences on the visual perception of human motion (e.g., Bosbach et al., 2005; Casile & Giese, 2007; Jacobs et al., 2004; Jacobs & Shiffrar, 2005; Loula et al., 2005; for reviews see Blake & Shiffrar, 2007; Jeannerod, 2006; Wilson & Knoblich, 2005).
Taken together, these and other related findings have led to the idea that when we observe conspecifics performing dynamic bodily actions, we understand those actions partly through an automatic process of covert simulation. However, the central theoretical notion of “simulation” is currently being scrutinized and refined (e.g., Gallagher, in press), partly due to recent work on such topics as the self-other distinction (e.g., Jeannerod, 2006; Schütz-Bosbach et al., 2006) and the human-robot distinction (e.g., Gazzola et al, in press; Oberman et al., 2007a). Moreover, a crucial point is that if the ability to understand dynamic actions truly depends on the integrity of the so-called “mirror neuron system,” then disruption of that system’s normal operation should impair the ability. This important prediction has received some support (e.g., Iacoboni & Dapretto, 2006; Pobric et al., 2006; Saygin, in press; Saygin et al., 2004a), but the issue is still controversial (Mahon & Caramazza, 2005). In the current study, we sought to contribute data that would help inform this issue.
We turn now to research on the observation of static snapshots of actions—that is, still pictures that merely imply movement. An interesting place to begin is with the concept of “representational momentum,” set forth originally by Freyd (1983). This is a robust perceptual phenomenon whereby observers have a veritable mandatory tendency to extrapolate movement beyond what is actually perceived. This effect, which is maximal a few hundred ms post-stimulus onset (Freyd & Johnson, 1987), was first discovered for objects that appear to be rotating (Freyd & Finke, 1984), and it was later extended to linear paths (Freyd & Finke, 1985; Hubbard & Bharacha, 1988), spiral paths (Freyd & Jones, 1994), and centripetal force and curvilinear impetus (Hubbard, 1996). The phenomenon is also sensitive to apparent velocity (Freyd & Finke, 1985), acceleration and deceleration (Finke et al., 1986), and predictable changes in direction (Hubbard & Bharucha, 1988; Verfaillie & D’Ydewalle, 1991). These and other similar behavioral findings (for brief reviews see Wilson & Knoblich, 2005, pp. 464–466, and Gibbs, 2006, pp. 139–142) suggest that representational momentum involves perceptual inferences that reflect “the internalization in the visual system of the principles of physical momentum” (Kelly & Freyd, 1987, p. 369).
These considerations raise the intriguing possibility that the higher-order neural processing of static versus dynamic actions (at the level of V5, for example) might not be very different, or different at all, inasmuch as the brain appears to automatically extrapolate movement from stimuli even when no movement is present in the stimuli themselves. Although this issue remains largely unexplored, there are a few studies that support the notion of neural processing similarity for static versus dynamic stimuli. For example, several fMRI studies have shown that the MT region and surround is activated when subjects view static images of implied motion (David & Senior, 2000; Kourtzi & Kanwisher, 2000; Olson et al., 2003; Senior et al., 2000). Furthermore, transient disruption of this area by TMS eliminates representational momentum (Senior et al., 2002) (although Alford et al. [in press] did not replicate this finding in a recent study). Urgesi et al. (2006) found that motor networks are also engaged during the observation of snapshots of ongoing hand actions. In particular, compared with static images of either relaxed hands or hands suggesting a completed action, static images of hands executing a pincer grip induced an increase in the excitability of the muscles that would normally be used to perform the action; those muscles were not affected, however, by static images of nonbiological entities with (e.g., waterfalls) or without (e.g., icefalls) implied motion. Lesion studies provide additional evidence that understanding static versus dynamic actions depend on similar neural structures. For instance, we (Tranel et al., 2003) conducted a neuropsychological study in which subjects were required to retrieve knowledge for actions in a nonverbal format. Subjects evaluated attributes of photographed actions, and also compared and matched photographed actions. The regions of highest lesion overlap among the subjects with defective performance on these tasks were in (1) the white matter underneath the left MT region, (2) the left IPL and subjacent white matter, and (3) the left inferior premotor/prefrontal cortex and subjacent white matter.
It is also important to note, however, that there may be some important differences between the processing of static versus dynamic action stimuli. For example, an interesting dissociation between recognition of static versus dynamic stimuli was reported for the recognition of emotional facial expressions (Adolphs et al., 2003). Specifically, a patient with bilateral temporal lobe damage caused by herpes simplex encephalitis was found to be very impaired in recognizing nearly all basic emotions in static facial expressions; however, the patient was almost entirely normal (except for the expression of disgust) in recognizing basic emotions in dynamic facial expressions. A conceptual replication of this finding in normal participants was reported by Ambadar et al. (2005), who found that motion facilitated the deciphering and recognition of subtle facial expressions. A similar finding was reported by Steede et al. (2007), who found that a patient with developmental prosopagnosia was able to recognize dynamic facial identities but not static ones. Thus, there are at least a few hints that the higher-order neural processing of static versus dynamic stimuli may not overlap entirely.
Actions are typically denoted by verbs, and although there are non-trivial cross-linguistic differences in the nature of verb meanings (e.g., Aikhenvald & Dixon, 2006; Levinson & Wilkins, 2006; Majid et al., in press; McGregor, 2002; Newman, 1998, 2002; Slobin, 2004), it is clear that most languages have large inventories of verbs that allow speakers to refer to actions in a variety of ways for purposes of communication. Focusing just on English, Levin (1993) sorted over 3,000 verbs (most of which designate actions) into approximately 50 classes and 200 subclasses, based on common semantic and syntactic properties. The verbs in a given class collectively provide a richly detailed categorization—that is, a kind of semantic map—of the relevant conceptual space by making distinctions, often of a remarkably fine-grained nature, along a number of different dimensions. As just one example, English verbs of motion express different types of rapid motion (e.g., sprint, hurry, dart), leisurely motion (e.g., stroll, amble, wander), furtive motion (e.g., creep, sidle, tiptoe), and awkward motion (e.g., limp, lurch, stagger) (Slobin, 2000).
Studies employing mainly comprehension tasks suggest that distinct semantic components of the meanings of action verbs are subserved by partially distinct left hemisphere—and possibly right hemisphere (Neininger & Pulvermüller, 2003)—cortical regions in ways that roughly parallel the functional-anatomical organization described above. First, the visual manner-of-motion features of action verbs—e.g., the visual movement pattern specified by kick—may depend on the posterolateral temporal region anterior and dorsal to MT (e.g., Kable et al., 2002, 2005; Kemmerer et al., submitted; Noppeney et al., 2005; for convergent behavioral work, see Kaschak et al., 2005, and Zwaan et al., 2004). Second, the goal-oriented agent-patient spatial-interactive aspects of action verbs—e.g., the type of object-directed path specified by kick—may depend on somatotopically mapped regions of the IPL (Tettamanti et al., 2005). Third, the motor component of action verbs—e.g., the type of action program specified by kick—may depend on somatotopically mapped premotor and perhaps also primary motor regions (e.g., Aziz-Zadeh et al., 2006; Bak et al., 2006; Buccino et al., 2005; Hauk et al., 2004; Hauk & Pulvermüller, 2004; Kemmerer et al., submitted; Kemmerer & Tranel, 2003; Pulvermüller et al., 2005a, 2005b; Rüschemeyer et al., 2007; Tettamanti et al., 2005; for convergent behavioral work, see, e.g., Borreggine & Kaschak, 2006; Boulenger et al., 2006; Glenberg & Kaschak, 2002; Scorolli & Borghi, 2007; Zwaan & Taylor, 2006). Collectively, these findings bolster the nascent “embodied semantics” framework which, in accord with the other literature reviewed above (and as attested by many of the contributions to this special issue; e.g., Boulenger et al., van Elk et al., Roy et al., Simmons et al.) maintains that much, but not necessarily all, of linguistic meaning may be grounded in sensorimotor representations (e.g., Barsalou, 2005; Bergen, in press; Damasio et al., 2004; Gallese & Lakoff, 2005; Gibbs, 2006; Jeannerod, 2006; Kemmerer, 2006, in press).
When a person orally names an action, the meaning of the appropriate verb is accessed first, and then the verb’s phonological form is retrieved and articulated. Confrontation action naming is in fact one of the most commonly used methods for investigating verb knowledge and processing. However, it is noteworthy that the vast majority of research on this topic has employed static snapshots of actions as stimuli, either in the form of drawings or photographs. One of the most reliable findings is that the process of mapping the meanings of action verbs onto the corresponding phonological forms appears to depend on the left frontal operculum (FO)2 (e.g., Damasio et al., 2001; Sahin et al., 2006; Shapiro et al., 2000, 2006; Shapiro & Caramazza, 2003; Tranel et al., 2001; Tsapkini et al., 2002). In addition, consistent with the literature on verb semantics discussed above, a number of studies suggest that the left posterolateral temporal-occipital cortex in and around area MT contributes to the naming of static actions (e.g., Aggujaro et al., 2006; Damasio et al., 2001; Tranel et al., 2001, 2005), and there is also some evidence that the left IPL plays a role in naming certain types of static actions, especially transitive actions performed with tools (e.g., Damasio et al., 2001; see also Kemmerer et al., submitted).
These findings fit nicely with the previously mentioned studies suggesting that the observation of static actions engages many of the same brain regions that underlie the observation of dynamic actions. Still, it is not known whether significant differences might exist between the neural systems for naming the two types of action stimuli. We are only aware of a few studies that have directly addressed this issue, and the findings are rather mixed. Berndt et al. (1997) found that five brain-damaged patients were equally impaired at naming static pictures of actions and dynamic video clips of actions, a result that supports the notion that the neural systems for naming the two types of actions are to a large degree coextensive. On the other hand, Druks and Shallice (2000) reported a patient who had superior verb naming for actions that were acted out by the experimenter, compared to those same actions portrayed in static pictures, a finding that supports the alternative view that the neural systems for naming the two types of actions are at least partially distinct.
Picking up on some of the important unresolved questions in the literature, we investigated the following hypothesis: Naming (with verbs) actions from dynamic stimuli requires neural regions that are mostly or wholly coextensive with those required for naming (with verbs) actions from static stimuli. We tested two specific predictions: (1a) Brain-damaged patients would demonstrate highly correlated performances on tests of dynamic action naming and static action naming and, in a related vein, (1b) patients who failed one type of action naming test would fail the other type as well. (2) Impaired action naming would be associated with lesions in the left FO, the left posterolateral temporal-occipital region (in and near MT), and the left IPL. As an exploratory question, we looked for cases in which there was a dissociation between the two types of action naming, e.g., patients who had normal static action naming but impaired dynamic action naming or vice versa, and investigated whether there were systematic neural correlates for such dissociations.
The literature also suggests an additional question of interest: Are there differences in the neuroanatomical correlates of verbs encoding tool movement versus verbs encoding biological movement? We explored this question in the current study, using dynamic action naming stimuli and focusing on “tool” versus “biological” subcategories of verb naming.
We studied 78 brain-damaged patients drawn from the Patient Registry of the Division of Behavioral Neurology and Cognitive Neuroscience at the University of Iowa. The patients were included in the current study if they could participate validly in the experimental procedures, and an effort was made to cover a variety of different lesion loci, i.e., lesions to either hemisphere, and to different sites within a hemisphere. Under the auspices of their enrollment in the Patient Registry, the patients have been screened to be free of mental retardation, learning disability, psychiatric disorder, substance abuse, and dementia. The patients in our Patient Registry have been extensively characterized neuropsychologically and neuroanatomically, following standard neuropsychology (Tranel, 2007) and neuroanatomy (H. Damasio and Frank, 1992) protocols for our program. The neuropsychological, neuroanatomical, and experimental data were collected when the patients were in the chronic phase of recovery. The patients provided informed consent to participate in these studies, in accord with the Human Subjects Committee of the University of Iowa and Federal regulations.
To be enrolled in our Patient Registry, patients have to have focal, stable cerebral lesions. In the sample included in the current study, lesion etiologies were as follows: cerebrovascular disease (n = 42), anterior temporal lobectomy (n = 22), surgical intervention (benign tumor resection, n = 10; subdural hematoma resection, n = 3); or herpes simplex encephalitis (n = 1). Of the group, 50 patients had unilateral left hemisphere lesions, 26 had unilateral right hemisphere lesions, and 2 had bilateral lesions. Handedness was measured with the Geschwind-Oldfield Questionnaire, which has a scale ranging from full right-handedness (+100) to full left-handedness (−100), and in the current sample, this variable was distributed as follows: 66 subjects were fully right-handed (+90 or greater); 1 was primarily right-handed (+80); 2 were partially right-handed (+55, +40); 6 were fully left-handed (−100); 2 were primarily left-handed (−80); and 1 was partially left-handed (−30). The sample had a mean age of M = 51.3 years (SD = 12.8) and a mean educational level of M = 14.4 years (SD = 2.5). There were 35 men and 43 women in the sample. As noted, all of the participants were in the chronic recovery epoch, and they participated in the Dynamic Action Naming Test when they were a mean of M = 6.5 years post lesion onset (SD = 6.5).
The participants did not have basic impairments in intellectual functioning, language, visual perception, or attention that would confound their performances on the experimental tasks. Of note, some of the patients were recovered aphasics—in fact, given our focus on left FO and IPL (among other regions) as potential key anatomical regions, we tried to sample these regions reasonably well, and this meant that recovered aphasic patients were included. However, we only included such patients if they were capable of (1) understanding the directions for the tasks, and (2) providing valid responses to the experimental stimuli. We have extensive experience working with these patients, and thus we can ascertain carefully, for every case, whether they are able to participate validly in the experiments described herein. (We excluded 2 patients with severe global aphasia, who were unable to comprehend the tasks and produce valid responses.) We calculated summary data for key neuropsychological indices for the 78 subjects included in the study. In regard to intellectual functioning, IQ scores were as follows: WAIS-III Verbal IQ: M = 101.6 (SD = 14.6); Performance IQ: M = 101.3 (SD = 14.1); Full Scale IQ: M = 101.9 (SD = 13.8). On the Token Test (from the Multilingual Aphasia Examination), which provides an index of aural comprehension, the sample had a mean score of M = 40.8 (SD = 8.0), which falls in the normal range (33rd %ile). On the Benton Facial Discrimination Test, which provides an index of complex visuoperceptual discrimination, the sample had a mean score of M = 45.2 (SD = 4.1), which also falls in the normal range (52nd %ile).
We designed a Dynamic Action Naming Test (DANT) that included 158 short video clips (3 to 5 seconds) depicting movements and actions. The set of stimuli was designed to include items from many different classes of action verbs that vary along both semantic and syntactic parameters (Levin, 1993) and that may call on varied neural substrates. In addition, we attempted to include items that had the same verb name as the static items that were included in the Static Action Naming Test (SANT) published by Fiez and Tranel (1997), to facilitate direct comparisons between the two tests (dynamic versus static). Finally, there were practical considerations that came into play in our stimulus development, mainly pertaining to whether a video clip could show an action that would be reliably named with a specific verb. (It turned out that it was not possible to elicit some verbs reliably with video clips, e.g., trudging and plodding as manners of locomotion could not be elicited reliably.) Most of the video clips were developed in our laboratory, and we supplemented these with clips taken from movies or other media sources (no recognizable actors were shown).
The DANT was administered via PowerPoint. Each video clip was presented, one at a time. One replay of the clip was permitted if the subject provided an answer that was close but was not quite the target response, or if the subject requested to have the clip replayed (the determination of “close” was at the discretion of the examiner, and “close” responses were typically an approximate synonym or partial form of the target verb (e.g., “wiping” for “erasing”) or a noun form or uninflected verb form (e.g., “swimmer” or “swim” for “swimming”). The subject was instructed to provide the “one best word” that would describe what was happening in the scene. An example was presented, specifically of a person hopping: the video clip was played, and the subject was told that they should say “hopping” for this. The subjects’ answers were recorded verbatim. If subjects responded with multiple-word answers or short phrases, they were reminded to say the “single best word” that describes the action in each stimulus. Due to the dynamic nature of the stimuli, the correct verb in all cases was in the form marked by the suffix “-ing,” indicating progressive aspect, and subjects were reminded to use the “-ing” form if they provided uninflected or other inflected forms.
The DANT was administered to 50 normal participants prior to being used with the brain-damaged patients. The normal group had a mean age of M = 60.3 years (SD = 18.9), a mean education level of M = 15.4 years (SD = 2.5), and there were 20 men and 30 women in the sample. The normal participants were comparable to the brain damaged participants on basic demographic parameters. We contrasted the normal and brain-damaged groups statistically using t-tests, and it turned out that the normal group was slightly older than the brain-damaged group (t(126) = 3.03, p = 0.003), and slightly better educated (t(126) = 2.27, p = 0.025). However, these factors might be expected to operate in different directions (i.e., age favoring the brain-damaged group, education favoring the normal group); thus, we did not correct for these factors in the analyses of the experimental naming measures.
We analyzed the response data from the normal participants, and culled from the original set of 177 video clips a number of items (19 in all) that had poor name agreement, high non-response rates, or were redundant (i.e., elicited the same verb as another item); this left the set of 158 items that were scored for the current study. For 122 of these 158 items, all of the normal participants generated the same verb naming response, and this was designated as the correct answer and the patient responses had to match this verb in order to be counted as correct. For the remaining items (36), the normal participants tended to provide two (or in a couple of cases, three) synonymous verb names (e.g., “rowing”/”paddling”), and following the same criteria used by Fiez and Tranel (1997), we accepted two (or three) verb names as correct for these items.
In order to investigate the question regarding verbs encoding tool movement versus verbs encoding biological movement, items from the DANT were divided into two subcategories, “tool actions” and “biological actions.” For both subcategories, we focused on items that are performed with the hand/arm, so that the variable of body part would be constant across the subdivision of “tool” versus “biological.” The tool action subcategory included 23 items (e.g., “chopping,” “cutting,” “peeling”) that are performed with a tool, and the biological action subcategory included 41 items (e.g., “pointing,” “reaching,” “scratching”) that are not performed with a tool. The actions/verbs for these subcategories were not equated for nuisance variables such as frequency and word length, and this is a limitation in the design of our study.3 However, as a first approach to determining whether there might be different neural correlates for the subcategories, we felt that we could gain a preliminary notion of whether these subdivisions seem likely to be meaningful (akin to the approach we used previously in a detailed analysis of verb naming of static pictures, see Kemmerer & Tranel, 2000a, 2000b). It should also be noted that the “tool actions” in our study entailed video clips of tools being used by human actors, and these are not strictly comparable to tool actions used in some previous functional imaging studies where the tool movement was carefully isolated and not tied to any human actor (cf. Beauchamp et al., 2003).
The Static Action Naming Test (SANT), developed by Fiez and Tranel (1997), comprises 100 items, 75 of which are single, static pictures depicting an ongoing action, designed to elicit verbs in the progressive form (e.g., "walking"). The remaining 25 stimuli are picture pairs depicting some change in an object, designed to elicit verbs in the past tense (normally “-ed”) form. For the 75 ongoing actions, subjects were told to say a single word that best described what the person, animal, or object was doing. For the 25 completed actions, subjects were told to say the single word that best described what had happened to the object. The SANT was developed and standardized with 64 healthy participants drawn from undergraduate psychology classes, who were approximately 18 to 21 years of age and had on average one year of college education. All were native English speakers. Compared to the DANT, the healthy comparison group for the SANT is younger and less educated, factors that might be expected to operate in different directions vis-à-vis naming performances.
As in the DANT, during administration of the SANT to brain-damaged participants, if the subject provided an answer that was close but was not quite the target response (the determination of “close” responses was at the discretion of the examiner, and applied to certain items in the test that were designated in the standardization study as appropriate for prompting), the examiner prompted the subject to be more specific or to try a different verb, depending on the nature of the “close” response (e.g., an approximate synonym, naming the wrong part of the stimulus). In developmental work with these tasks (including both the SANT and DANT), when the procedures were being designed, we have examined how using only first responses (i.e., ones provided without prompts and without stimulus replays) affects the results, and have found that we tended to have more unreliable data and inflated error scores. Hence, we adopted procedures whereby “close” responses are prompted (for certain items on the SANT) or allowed a stimulus replay (on the DANT), to increase reliability and glean a more accurate picture of the patients’ naming capacities.
To test the hypothesis of the study, vis-à-vis the main objective of contrasting naming of dynamic action stimuli with naming of static action stimuli, we focused on a subset of action naming stimuli. Specifically, there were 60 items for which the exact same verb was the right answer on both the DANT and SANT (see Appendix 1 for a list of the items). Direct comparisons between dynamic and static action naming focused on these 60 items. Of the overall group of 78 brain-damaged subjects in the study, there were 71 who had complete datasets for both the DANT and SANT. Thus, the main contrast for comparing dynamic action naming to static action naming involved 71 subjects and 60 test items. The demographic and neuropsychological parameters for this group of 71 subjects were comparable to the overall group: Age, M = 51.6 years (SD = 13.0); Education, M = 14.4 years (SD = 2.6); Gender, 32 men/39 women; Chronicity, M = 6.9 years post lesion onset (SD = 6.7); WAIS-III Verbal IQ, M = 101.4 (SD = 15.0); Performance IQ, M = 101.3 (SD = 14.2); Full Scale IQ, M = 101.8 (SD = 14.0); Token Test, M = 40.8 (SD = 8.1); Benton Facial Discrimination Test, M = 45.3 (SD = 3.9). The group of 71 included 45 patients with left hemisphere lesions, 24 with right hemisphere lesions, and 2 with bilateral lesions.
Prediction 1a under the main hypothesis was tested by correlating the DANT and SANT performances of the 71 brain-damaged patients who had received both tests, using the 60 items in common across the tests. Prediction 1b was tested by determining which patients were impaired on the DANT, SANT, or both (using all 78 participants and the full stimulus sets for both tests). To define impairment, we used the conventional approach of classifying as “impaired” all scores that were 2 or more SD’s below the mean of normal performance, as in previous work (see H. Damasio et al., 2004, for a detailed discussion of the rationale for a 2 SD threshold for “impaired”). We looked at whether DANT and SANT failures were highly correlated, whether there were any patients who failed one test but not the other, and whether there were any lesion commonalities in patients who were impaired on only one or the other of the two tests. To investigate Prediction 2, we ascertained lesion overlaps in the group of brain-damaged participants who had impaired performance on the DANT (using all 78 participants).
To test the question about tool movement versus biological movement, we first calculated the performance of the normal group on the “tool action” versus “biological action” subcategories. We then looked through the brain-damaged patients for discrepancies between the two subcategories that indicated disproportionately impaired verb naming for one subcategory or the other. In parallel with the use of a 2 SD threshold for defining impairments in the overall tests, we defined “disproportionate” here as a difference between subcategories that exceeded 2 times the average between-category difference manifested by normal participants. The patients who demonstrated disproportionate impairment of tool action naming relative to biological action naming were grouped together, and their lesion commonalities were analyzed. The patients who demonstrated disproportionate impairment of biological action naming relative to tool action naming were grouped together, and their lesion commonalities were analyzed.
The neuroanatomical quantification and lesion analysis was based on magnetic resonance (MR) data obtained in a 1.5 Tesla scanner with an SPg sequence of thin (1.5 mm) and contiguous T1 weighted coronal cuts. (In a few patients, an MR could not be obtained due to metal in the head or claustrophobia, and in these cases the analysis was based on CT data.) Each subject's lesion was reconstructed in three dimensions using Brainvox (H. Damasio & Frank, 1992; Frank et al., 1997). Lesions were analyzed qualitatively on a case-by-case basis. Also, we conducted a lesion overlap analysis to address the prediction that action naming deficits would be associated with damage to particular neural regions. To do this, all lesions for a given set of patients (grouped according to behavioral performance, as described below) were transposed and manually warped into a normal 3-D reconstructed brain. This method produces a maximal overlap map relative to both surface damage and depth extension, and the number of patients contributing to the overlap can be calculated. This approach allows a determination of which areas of brain damage are most strongly associated with a particular behavioral defect.
We also calculated a lesion density map for patients with left hemisphere lesions, in order to provide a general overview of the lesion sampling in our study (Figure 1). This map focused on left hemisphere subjects, because the key anatomical predictions involved left hemisphere regions (FO, IPL, MT). As can be seen in Figure 1, the lesion sampling was highest in the anterior temporal region (especially BA38) and in the perisylvian region. There was fairly sparse sampling in the occipital lobe, and almost no sampling of very anterior and superior prefrontal regions. Importantly, the lesion density map shows that the main lesion results from our study (as reported below) can not be accounted for by systematic over-sampling of key anatomical sectors (especially left FO and MT).
The 50 normal participants obtained an average score of 90.8% correct (SD = 4.2) on the DANT. Scores ranged from a low of 79% correct to a high of 97% correct. (For the sake of comparison, on the SANT, reported in previous publications, the average percent correct for normal participants was 85.2% (SD = 5.0).) Thus, for the DANT, we effectively constructed a test that provides reasonable range and variability of performance in normal participants and does not have serious ceiling effect limitations.
We looked at the effect on DANT performance of demographic variables. There was no effect of sex, with men (M = 91.1%, SD = 3.9) and women (M = 90.5%, SD = 4.5) performing very similarly (t(48) = 0.51, n.s.). For education, we divided the sample into participants with ≤ 15 years of education (N = 28) versus those with > 15 years of education (N = 22). The two groups performed similarly on the DANT, with means of M = 91.1% (SD = 3.6) and M = 90.3% (SD = 5.0) in the lesser and greater educated groups, respectively (t(48) = 0.67, n.s.). For age, we divided the sample into participants aged < 70 (N = 28) and those aged ≥ 70 (N = 22). The younger age group (M = 92.5%, SD = 3.5) outperformed the older age group (M = 88.5%, SD = 4.1) on the DANT, and although this difference was statistically significant (t(48) = 3.76, p < .001), it is not practically very large in magnitude (4 percentage points, about 1 SD unit). Based on these results, together with the fact that the normal comparison and brain-damaged groups were demographically comparable, we did not adjust the action naming performances of the brain-damaged participants for any demographic factors.
In the 71 patients who had received both the DANT and SANT, the average performances on the two tests were very similar: DANT, M = 86.4% correct (SD = 13.9); SANT, M = 89.6% correct (SD = 15.4). We measured the performance differences between the two tests by calculating the difference score for each participant (SANT score minus DANT score), and this yielded an average difference of 3.2 points (SD = 6.3). Note that the direction of this result is positive, which reflects the fact that the participants tended to perform slightly better on the SANT. This is an interesting and perhaps somewhat counterintuitive result, i.e., that for the exact same 60 verbs, naming would be slightly superior for static depictions of actions versus dynamic depictions. We return to this point in the Discussion.
Performances on the DANT and SANT were highly correlated, with a Pearson correlation coefficient of R = 0.91 (R2 = 0.83) (see Figure 2). For exploratory purposes, we looked at the correlations separately for participants with left hemisphere damage versus those with right hemisphere damage. This revealed that the DANT-SANT correlation was very high in the LH group (R = 0.93, p < .0001), and much lower in the RH group (R = 0.30, n.s.). The right hemisphere group outperformed the left hemisphere group on both the DANT (RH, M = 91.9 [SD = 4.9]; LH, M = 83.2 [SD = 16.3] and SANT (RH, M = 96.3 [SD = 2.7]; LH, M = 85.8 [SD = 18.2]), and both differences were significant in a t-test (p-values = 0.010 and 0.005, respectively). Overall, the data indicate that action naming performance was highly comparable for stimuli presented dynamically compared to stimuli presented statically.
We looked through the scores of brain-damaged participants (on the 60 common items) for cases where the DANT versus SANT discrepancy was 10 or more percentage points. We chose this discrepancy threshold for a rough demarcation for what could be considered a meaningful difference between performances on the two tests, because 10 points is fairly substantial (more than 3 times the average DANT vs. SANT difference in the group as a whole, which was 3.2 points), but not so large as to miss out on cases where there might be interesting differences. It should be noted that this analysis was exploratory, and thus the threshold was intended to provide a general overview of the results rather than to provide a strict test of an a priori prediction. In 14 patients, there was a DANT v. SANT discrepancy of ≥ 10 points, and in 13, the difference favored better performance on the SANT. We conducted a lesion analysis of these 13 cases, and found no evidence of consistent lesion commonalities: 7 had left hemisphere lesions and 6 had right hemisphere lesions, and in neither hemisphere were there consistent lesion loci.
In regard to overall performances on the DANT and SANT (based on all 158 items for the DANT and all 100 items for the SANT), there was also a high congruence between performance on the two tests. The overall scores for the two tests were very similar in the brain-damaged participants, with the DANT yielding an overall score of M = 83.0% (SD = 13.5) and the SANT yielding an overall score of M = 87.4% (SD = 15.5) (N’s were 78 for the DANT and 71 for the SANT). We calculated the correlation between the full DANT (158 items) and full SANT (100 items) for the 71 participants who had received both tests, and this was very high (R = 0.93, p < .0001).
Scores on the DANT that were 82% correct or lower were classified as impaired (such scores are ≥ 2 SD’s below the mean of normal performance). Using this criterion, there were 16 patients in the current sample (of the 78 total) who were impaired on the DANT (based on all 158 items) (see Table 1). The mean performance in these 16 patients was M = 63.6% (SD = 17.8). Of the 16 patients who failed the DANT, 1 patient had not received the SANT. Of the remaining 15, 7 patients also failed the SANT, and 8 passed the SANT (on the SANT, scores of 75% or lower are ≥ 2 SD’s below the normal mean). Of the 8 patients who failed the DANT but passed the SANT, 4 (1621, 2456, 1783, 3202) had scores on the SANT that were barely above the cutoff (76%, 77%, 78%, 78%, for the 4 cases, respectively)—in other words, these 4 patients barely passed the SANT, and the discrepancy between tests in these 4 patients was in fact small. In the other 4 patients who failed the DANT but passed the SANT (2773, 3046, 3050, 3227), their DANT scores were not very far below the cutoff (77%, 78%, 78%, 78%, for the 4 cases, respectively)—that is, they failed the DANT by a fairly small margin. Nonetheless, these 4 patients had scores on the SANT that were well within the range of normal performance (93%, 84%, 89%, 90%, for the 4 cases, respectively), suggesting a non-trivial degree of DANT v. SANT discrepancy. We looked at the lesions in these 4 patients, and there was no evidence of a common lesion locus: 2 cases had left anterior temporal lesions, 1 had a left hemisphere lesion in the white matter underneath the inferior frontal operculum, and 1 had a right insular/inferior parietal lesion. Interestingly, there was no instance of a patient who failed the SANT and passed the DANT4 (this issue is taken up in the Discussion). Overall, these findings provide support for the prediction, and indicate that failure on the two types of action naming tests was strongly correlated; moreover, dissociated performances were not clearly associated with any particular lesion locus.
We also calculated the correlation between DANT and SANT performance, using the 60 common items, in the 15 participants who failed the DANT and for whom both DANT and SANT scores were available. The performances were highly correlated, with a Pearson correlation coefficient of R = 0.97 (p < .0001). (Not surprisingly given the more truncated range of scores, the DANT-SANT correlation in the 56 patients who were not impaired on the DANT, and for whom both DANT and SANT scores were available, was lower, viz., R = 0.38, albeit still statistically significant (p = .004).)
All but 1 of the 16 patients who were impaired on the DANT had a left hemisphere lesion. In the 15 left hemisphere patients, the mean performance was M = 62.8% (SD = 18.0). The most common lesion locus in these cases was the left frontal operculum (FO); 9 patients had lesions that overlapped in this region, in the cortex and/or underlying white matter (Table 1; Figure 3). Two patients had lesions that overlapped in the region in and near MT, in the lateral occipitotemporal sector and underlying white matter, at the junction of BA 37 and 19 (Table 1; Figure 4). We did not find patients with lesions restricted to IPL who were impaired on the DANT; there were 4 cases where left IPL was involved in the lesion, but all of these cases also had FO damage (these were counted in the 9 mentioned above). There were 4 patients with anterior temporal damage (mainly BA 38) who failed the DANT.
The specificity of the lesion findings was assessed by looking through the lesions of the other 35 patients with left hemisphere damage, to determine whether there were “false negative” cases vis-à-vis the FO, MT, IPL, and anterior temporal sectors—i.e., patients with lesions involving these regions who did not fail the DANT. In regard to the FO sector, there were 2 patients with FO damage who passed the DANT (but see Section 3.3 below). This compares to the 9 patients with FO damage who failed the DANT, indicating a high degree of specificity regarding the association between FO damage and impaired DANT performance. There were no patients with MT damage who passed the DANT, indicating that the MT finding was also specific. In regard to the IPL, there were 3 patients with damage to this region who passed the DANT, suggesting that IPL damage was not specific to failing the DANT. Finally, we found 12 patients with anterior temporal lesions who were normal on the DANT, indicating that damage to the anterior temporal region was not consistently associated with impaired DANT performance—that is, many more patients with damage to this region (n = 12) passed the DANT, compared to the many fewer with damage to this region (n = 4) who failed the DANT. Thus, we can discount the significance of the left anterior temporal region vis-à-vis dynamic action naming.
To summarize, the regions that came out as being strongly and specifically related to DANT performance—i.e., regions where damage was reliably associated with impaired naming of dynamic actions—were the FO sector and the posterolateral temporal-occipital sector (in and near MT). These findings, which partially support our prediction, are consistent with previous work (see Discussion). It is also important to mention that these findings provide a close replication of our previous neuropsychological study regarding action naming, which also pinpointed FO (and to some extent, the posterolateral temporal-occipital sector) as being strongly associated with action naming (Tranel et al., 2001). In terms of patients who had impaired action naming, the previous study included two of the same patients as the current study; thus, there were 17 patients in the Tranel et al. (2001) study and 13 in the current study who were unique to each study, indicating a strong replication, in a new group of patients, of our previous findings.
There are several important questions that arise in connection with the findings reported above, and we turn now to a consideration of these issues. Perhaps the most immediate question is the extent to which the action naming impairments in our patients are part of a more general impairment in naming (anomia). To address this, we calculated the average naming performance of the 15 patients with impaired DANT scores and left hemisphere lesions, on naming of concrete entities from the categories of animals, fruits/vegetables, tools/utensils, musical instruments, and vehicles. Across these 5 categories of non-unique concrete entities, there are 297 total stimuli (see H. Damasio et al., 2004, for details about stimuli and procedures related to concrete entity naming). In 3 patients, naming of concrete entities was clearly impaired (overall M = 42.4% correct), and the patients had impaired naming scores in all 5 individual categories. Two other patients had impaired naming scores in 4 of the 5 individual categories, and impaired overall naming scores (M = 78.1% correct). In the other 10 patients, however, naming of concrete entities was not notably impaired. Their overall naming scores (M = 91.3%, SD = 3.4, range: 87.7% to 98.0%) were close to or within the range of normal performance (~mid 90’s % correct). It is beyond the scope of the current study to investigate in detail the question of category-related or grammatical class-related naming deficits in these patients, but it can be said that the finding of impaired dynamic action naming performance does not appear to be part of a general anomia in the majority of the affected patients (10 out of 15). In the other 5, it could be argued that impaired dynamic action naming is part of a more general impairment in naming, and in fact this is not surprising as those are also the patients who tend to have the most significant residual aphasia profiles.
Another issue concerns the nature of the errors produced by the patients with impaired dynamic action naming. In 4 patients, “no response” was a fairly common error type (20%, 16%, 14%, and 6% of overall errors). In the other 12 patients, “no response” was very rare, and the most common error types tended to be verbs that were not specific enough (e.g., “jumping” for “bucking”) or were not the target verb (e.g., “muscling” for “flexing”), or descriptions that were not verbs (e.g., “hurts my back to look at it” for “hoeing”). Many of the wrong verb responses could be classified as semantic paraphasias, but other types of aphasic errors were uncommon. Overall, the nature of the predominant errors in the impaired patients hints at more of a lexical-phonological access deficit than a semantic knowledge/processing deficit, but we would be quick to acknowledge that our data are not sufficient to adjudicate this issue in a definitive fashion.
We also explored whether performance on the DANT in the 16 patients with impaired DANT scores was correlated with two neuropsychological measures, the Token Test (a measure of aural comprehension) and the Benton Facial Discrimination Test (a measure of visual perception). For the Token Test, the correlation with DANT scores was high (R = 0.84, p < .001); for the Facial Discrimination Test, the correlation with DANT scores was low (R = 0.22, p = .404). The Token Test result is not surprising, given the fact that a number of recovered aphasic patients were in the group with impaired DANT performance. In this context, though, it is important to note that 8 of the 16 impaired DANT performers had entirely normal Token Test scores (M = 43.4, SD = 1.1, range: 41 to 44, of 44 possible points), and 2 more had scores that were only slightly impaired (38, 36). It is also important to reiterate that none of the patients had comprehension (or visual perceptual) defects that precluded valid participation in the experiments.
Finally, we looked at the lesion etiologies for the patients with impaired DANT performances. The lesion etiologies in these 16 cases were as follows: cerebrovascular disease, n = 11; anterior temporal lobectomy, n = 4; surgical intervention (subdural hematoma resection), n = 1. In the 62 unimpaired patients, by comparison, the lesion etiologies were as follows: cerebrovascular disease (n = 31), anterior temporal lobectomy (n = 18), surgical intervention (benign tumor resection, n = 10; subdural hematoma resection, n = 2); herpes simplex encephalitis (n = 1). In both the impaired and unimpaired groups, the most common lesion cause was cerebrovascular disease (67% and 50%, respectively), and there do not appear to be any notable or systematic differences between the groups in terms of lesion etiology.
In the normal participants, naming performances were very similar in the tool action and biological action subcategories: tool M = 95.4% (SD = 4.3); biological M = 92.8% (SD = 4.9). The mean difference score (tool minus biological) for the normal group was M = 2.5 (SD = 5.4), indicating that normal subjects performed slightly better for tool actions than biological actions. Overall, the brain-damaged group showed a similar pattern: their performances were similar in the tool (M = 88.2%, SD = 15.2) and biological (M = 86.8%, SD = 13.9) subcategories, and the mean difference score (tool minus biological) was small and positive, M = 1.4 (SD = 8.4).
We analyzed in more detail the performances of brain-damaged patients who were impaired in naming for one or both of the tool and biological subcategories (using the ≥ minus 2 SD cutoff to define impairment). Overall, there were 21 such cases (12 of whom were also part of the group of 16 who failed the overall DANT, as specified in section 3.2.3. above). Of this group, 6 patients were impaired to a comparable extent in both subcategories, and the mean tool minus biological score in this group was 1.8 percentage points (SD = 6.1). (All 6 of these cases had failed the overall DANT.) The remaining 15 patients, however, had disproportionately poor performance in one of the two subcategories (as noted earlier, “disproportionate” was defined as an absolute difference between the tool subcategory and biological subcategory of more than 5 percentage points, i.e., twice or more the mean between-subcategory difference in the normal participants). More specifically, 6 patients were disproportionately impaired in naming biological actions, with a mean difference between tool minus biological scores of 17.8 percentage points (SD = 3.5). (One of these six cases had failed the overall DANT.) The other 9 patients were disproportionately impaired in naming tool actions, with a mean difference between biological minus tool scores of 11.8 percentage points (SD = 6.3). (Of these 9, 5 had failed the overall DANT, and 4 had not.) Detailed data for the 15 patients with disproportionate naming impairments in the tool subcategory or the biological subcategory are presented in Table 2.
We analyzed the lesions in these different subgroups of patients, and found some interesting patterns. In the disproportionately impaired biological naming group, 3/6 cases had right hemisphere lesions, 1 had a bilateral prefrontal lesion, and 2 had left hemisphere lesions. Although there was no clear lesion commonality in this group, it is interesting that none of them had damage to the left FO or MT regions (2957 had right FO damage). In the disproportionately impaired tool naming group, 8/9 cases had left hemisphere lesions, and of the 8 left hemisphere cases, 4 had lesions that involved the FO sector (2537, 3050, 3172, 3202), 2 had lesions that involved the posterior temporo-occipital sector (in and near MT) (1621, 2980), 1 had a mesial inferior occipital lesion (2439), and 1 had a left anterior temporal lesion (2496). Interestingly, none of these cases had a lesion that involved the IPL or superior temporal gyrus/sulcus regions. Also, 2 of these cases, both with FO lesions (2537, 3172), had been counted as “false negatives” in the overall DANT analysis, because their overall DANT score was not impaired (section 3.2.3. above). The finding that these 2 patients had impaired naming for the tool action subcategory strengthens the lesion-deficit association for the FO region, reducing the number of false negatives for this region to zero. (All of the 6 patients with comparable tool and biological subcategory impairments had left hemisphere lesions. Three of these cases had damage that mainly involved the IPL and posterior superior temporal gyrus, 2 patients had left FO lesions (which in one case extended into IPL), and 1 had a left anterior temporal lesion.) Overall, the patterns depicted by the patients reported in Table 2 indicate some intriguing consistencies with the functional imaging literature, and we pick up this theme in the Discussion.
Previous research comparing the visual processing of static and dynamic actions suggests that these two types of stimuli engage largely overlapping neural systems. We therefore hypothesized that naming dynamic actions (with verbs) would require neural regions that are mostly or wholly coextensive with those required for naming static actions (with verbs). The results of our study support two predictions which derive from this hypothesis. First, the 71 patients who received both the DANT and the SANT exhibited highly correlated performances in retrieving the same verbs to name dynamic and static presentations of essentially the same actions. In related fashion, those patients also tended to manifest similar overall performance profiles on the two tests, consistent with the findings of Berndt et al. (1997); in fact, all of the patients who failed the SANT also failed the DANT, and most of the patients who failed the DANT either also failed the SANT or were barely above the cutoff score for the SANT. Second, lesion analysis indicated that defective naming of both dynamic and static actions was associated with damage to brain structures that have previously been implicated in observing actions as well as understanding and producing action verbs, especially left FO and the left posterolateral temporal-occipital region in and near MT (and with less specificity, left IPL). The lesion findings replicate previous neuropsychological work using static actions (Tranel et al., 2001). In what follows, we discuss in greater detail these two major outcomes of our study, and attempt to situate the findings in the context of the extant literature, with an eye towards how the concept of representation momentum can help explain these findings.
As we mentioned in the Introduction, the notion of “representational momentum” may play a key role in explaining why static actions appear to be processed by the same neural structures that process dynamic actions. The basic phenomenon, originally described by Freyd (1983), is that recognition memory of still images suggesting motion is distorted forward along the implied trajectory (for brief reviews see Wilson & Knoblich, 2005, pp. 464–466, and Gibbs, 2006, pp. 139–142). This perceptual effect has been demonstrated in behavioral studies using snapshots of human figures as stimuli (Verfaille & Daems, 2002), and other work suggests that such stimuli engage not only the motion-sensitive posterolateral temporal-occipital cortex but also motor structures that would normally be activated during execution of the observed action (Urgesi et al., 2006). Furthermore, Shiffrar and Freyd (1990) found that biomechanically plausible illusions of human motion can be automatically evoked if two snapshots are rapidly presented at the appropriate temporal interval. For example, if two snapshots—the first one showing a woman with her left arm at the left side of her knee, and the second one showing the same woman with her left arm at the right side of her knee—are displayed within 550–750 ms (a velocity compatible with real movements), the observer reports seeing the woman’s arm move around her knee—a possible, indeed likely, trajectory; however, if the snapshots are displayed at a shorter interval of 150–350 ms, the observer reports seeing the woman’s arm move through her knee—an impossible trajectory.5 These considerations suggest that when subjects use verbs to name static actions, at least in the case of observing conspecifics performing bodily actions, they mentally simulate the implied movement—a process that appears to be subserved neurally by many of the same brain structures that contribute to the recognition of dynamic actions. The current study provides neuropsychological evidence in support of this idea.
Some significant dissociations between dynamic and static action naming did emerge, and they were all in the same direction—specifically, better performance on the SANT than on the DANT. Some other findings from our study suggest that the SANT may be slightly easier than the DANT: (1) For the 60 items with identical target verbs in the two tests, the 71 patients who received both tests performed better when the actions were presented statically (M = 89.6%) than dynamically (M = 86.4%); (2) Those 71 patients’ overall scores were also higher for the SANT (M = 87.4%) than the DANT (M = 83.0%); and (3) Of the 14 patients with performance discrepancies exceeding 10 percentage points between the two tests, 13 achieved higher scores on the SANT than the DANT. At first glance, it might seem counterintuitive that naming static actions would be easier than naming dynamic actions, because in spite of the seeming automaticity of representational momentum, the critical motion information is only implicit in the static stimuli but is completely explicit in the dynamic stimuli. On the other hand, the very fact that the images are temporally stable for static portrayals of actions may allow subjects to inspect them at their own pace and thereby consider more carefully which verb is most appropriate. Also, the static portrayals may depict the essence of a particular action more saliently than the dynamic portrayals, reminiscent of how line drawings and even caricatures can capture the essence and defining features of concrete entities more so than photographs. Further research is needed to explore these issues in greater depth. We would, however, like to underscore the finding that the patients who manifested superior performance on the SANT relative to the DANT did not have systematic commonalities in their lesion sites. This supports the idea that there is not a specific neural region that is reliably more important for naming dynamic than static actions; rather, as indicated above, the weight of evidence favors the conclusion that the two types of actions are processed by largely overlapping brain circuits, and possibly in similar ways.
As we predicted, the left FO was the most common lesion site among the 15 patients who failed both the DANT and the SANT. What cognitive operations might this region subserve, that are essential for naming actions with appropriate verbs? One function that has been attributed to the left FO is the mapping between the semantic structures and the phonological forms of verbs. We (Tranel et al., 2001) offered such an interpretation to account for the finding that a large group of patients with left FO lesions failed the SANT but nevertheless performed within the normal range on a test that probes knowledge of the kinds of action concepts that are typically encoded by verbs. It is possible, however, that some of the patients in that study had semantic deficits that were not detected by the conceptual knowledge test,6 and in fact, as described below, the left FO has been associated with both semantic working memory and the understanding of actions, particularly hand actions.
Regarding semantic working memory, within the left FO, BA45 has been linked with a particular type of operation that involves selecting appropriate representations from among competitors (Badre et al., 2005; Fletcher et al., 2000; Gold et al., 2006; Moss et al., 2005; Thompson-Schill et al., 1997, 1999; for a study focusing specifically on verbs, see Kemmerer et al., submitted). Thus, a possible consequence of damage to BA45 might involve difficulty identifying which verb meaning (or corresponding verb form), from among several co-activated alternatives, provides the best match to an action presented either as a static picture or as a dynamic video clip. Applied to the results of the current study, this could account for—or at least would be consistent with—the prevalence of semantic paraphasias in the errors produced by the patients with impaired action naming.
In addition, the left FO—especially BA44, which may be the human homologue of macaque F5 (for reviews see Arbib & Bota, 2006; Binkofski & Buccino, 2006)—has been found to be engaged when hand actions are planned (Johnson & Grafton, 2003), imagined (Grafton et al., 1996), remembered (Johnson-Frey et al., 2003), imitated (Iacoboni et al., 1999), or observed (Rizzolatti et al., 1996). A recent study also demonstrated that rTMS over this region disrupted the ability of participants to judge the weight of a box lifted by a person, but not the weight of a bouncing ball (Pobric & Hamilton, 2006). Consistent with this finding, several lesion studies have shown that damage to left FO—and, importantly, to surrounding prefrontal, premotor, and primary motor regions—frequently impairs the understanding of actions (Bak et al., 2006; Bak & Hodges, 2003; Kemmerer & Tranel, 2003; Saygin, in press; Silveri & Ciccarelli, in press; Tranel et al., 2003). Taken together, these considerations raise the possibility that in the current study some of the patients with left FO lesions may have failed both the SANT and the DANT because of underlying deficits involving the knowledge and/or processing of action concepts. Studies have also implicated left FO in the retrieval of motoric aspects of tool knowledge not only during actual or pantomimed tool use, but also during the observation of dynamic tool actions, and even during the naming of static tools (for review see Lewis, 2006; see also Ferrari et al., 2006; Goldenberg et al., 2007; Kan et al., 2006). Our study contributes to this literature insofar as 4 of the 9 patients with disproportionate impairment in naming tool actions (i.e., arm/hand actions performed with tools) relative to biological actions (i.e., arm/hand actions performed without tools) had damage in the left FO. As the data in Figure 3 show, however, we did not find a substantial extent of lesion overlap in the more dorsal aspect of the ventrolateral prefrontal/premotor cortices, which might have been expected based on previous work that has linked this sector of FO in particular to “motor cognition” involving hand actions (especially hand actions involving tool manipulation) (e.g., Buccino et al., 2001; Pulvermüller et al. 2005b). It is possible that our test materials (and/or lesion sampling) did not allow a good test of this specific issue, and further work is needed to help clarify these results.
In summary, the left FO may facilitate action naming by subserving three closely related functions—lexical access, semantic working memory, and action understanding. The patients who failed both the SANT and the DANT as a result of left FO lesions may have deficits involving various combinations of these functions. Future work will be needed to parse out the relative contributions of each of these functions, and the possible relationship to specific neural subsectors within FO. (And there are, of course, morphosyntactic considerations that likely play a role as well, as emphasized by Shapiro and Caramazza (2003), among others.)
We were somewhat surprised that we did not find specific associations between disproportionately impaired naming of tool actions and damage in the left IPL, since this brain region has been associated with tool knowledge, particularly “planning and preparation” components (for reviews see Johnson-Frey, 2004; Lewis, 2006; see also Glover, 2004; Imazu et al., 2007; Buxbaum et al., in press; Ebisch et al., in press). On the other hand, it is interesting that among the 15 patients who failed both the SANT and the DANT, 4 had lesions that encompassed the left IPL, and all of them were comparably impaired in naming tool actions and biological actions. The subcategory of biological actions included both intransitive (i.e., non-object-directed) and transitive (i.e., object-directed) arm/hand actions, so these lesion-deficit correlations are in keeping with evidence that the left IPL responds to the sight of both types of actions, intransitive (e.g., Calvo-Merino et al., 2006; Cross et al., 2006; Lui et al., in press) and transitive (e.g., Bonda et al., 1996; Buccino et al., 2001). This cortical region, however, tends to be more strongly engaged during the observation of transitive compared to intransitive actions, leading some researchers to propose that it plays an important role in representing the general semantic dimension of “manipulability,” which includes information about the multifarious ways in which people use different body parts to interact physically with objects in peripersonal space (e.g., Saccuman et al., 2006; Tettamanti et al., 2005; see also Assmus et al., 2007; Tranel et al., 2003).
Of the patients who failed both the SANT and the DANT, 2 had lesions that encompassed left MT, and both of them were disproportionately impaired in naming tool actions relative to biological actions. As we pointed out in the Introduction, functional imaging studies suggest that the visual motion patterns characteristic of tools are processed within a relatively specialized cortical pathway that extends from MT anteriorly into the MTG (for reviews see Martin, 2007; Beauchamp & Martin, 2007). Moreover, a PET study demonstrated that a region close to left MT is activated significantly more when participants use noun-verb homophones (e.g., comb) as verbs to name photographs of implied tool actions, compared to when participants use the same words as nouns to name the depicted tools themselves (Tranel et al., 2005).
A limitation of the current study is that action knowledge was only evaluated by means of naming tasks. Because other methods of probing action knowledge were not included, we cannot easily distinguish between naming deficits that reflect disorders of lexical access and naming deficits that reflect disorders of semantic structures/processes (cf. Caramazza & Hillis, 1990; Laine & Martin, 2006). Given their lesion sites, however, it is likely that some of the patients with impaired action naming have deficits of the first type, others have deficits of the second type, and still others have deficits of both types. An important goal of future research will be to investigate these issues in greater depth by using dynamic stimuli to evaluate not only the production but also the comprehension of action verbs, ideally controlling for semantic variables such as: the body part(s) used to perform the action; whether the action is intransitive or transitive; whether the action involves the use of a tool; whether the action involves an object undergoing a change of location; and whether the action involves an object undergoing a change of state. Employing dynamic stimuli to explore the neural underpinnings of these and many other semantic variables will help integrate cognitive neuroscience research on action verbs with recent advances in linguistic theory, and will also bring both approaches into closer contact with the rapidly expanding field of “embodied cognition.”
This study was supported in part by a grant from the National Institute for Neurological Diseases and Stroke (Program Project Grant P01 NS19632). We thank Joe Whitman for assistance with scoring the patient protocols, David Rudrauf, Joel Bruss, and the Grabowski Computational Neuroimaging Laboratory for assistance with the neuroanatomy figures, and Eduardo Vianna and Jocelyn Spoon for assistance in developing the dynamic action stimuli.
Supported by Program Project Grant NINDS P01 NS19632
1BA stands for Brodmann area, and we follow the conventional numbering system.
2To be precise, we use FO to refer to the cortex comprised by BA44/45, the inferior aspect of BA6, and underlying white matter; in other words, the ventral portion of tissue located anterior to the precentral sulcus, posterior and ventral to the inferior frontal sulcus, and dorsal to the ascending ramus of the Sylvian fissure.
3Adelman et al. (2006) have shown that the “contextual” diversity of words is a more potent factor influencing performance on lexical retrieval tasks, compared to standard factors such as word frequency. Equating stimulus sets for such nuisance variables is, under the best of circumstances, somewhat of a moving target.
4As noted previously, there were 7 patients who had been administered the DANT but not the SANT, and 1 of these (with a left anterior temporal lesion) failed the DANT. Of the other 6, 4 had LH lesions and 2 had RH lesions. The lesions in these cases were in varied loci. Thus, it is very unlikely that any of the main findings would be altered significantly by SANT data for the 7 missing cases; in fact, given the findings reported in section 3.2.1., we would expect the SANT scores to track very closely with the DANT scores for these 7 participants.
5See Nishitani and Hari (2002) for a MEG study providing additional data on how static images of implied motion trigger activity first in the posterolateral temporal cortex (especially STSp), then in the IPL, then in FO, and finally in primary motor cortex. Note also that the concept of “snapshot neurons” is integral to the computational model of biological motion perception developed by Giese and Poggio (2003; see also Giese, 2006).
6Most of the patients in Tranel et al.’s (2001) study also participated in a study by Kemmerer et al. (2001a, 2001b), which reported and analyzed performances on not just the SANT and the Picture Attribute Test, but also on 4 additional tests that evaluate, in different ways, the processing of action verbs and concepts (the entire test battery was originally described by Fiez & Tranel, 1997). Importantly, Kemmerer et al. present data for 10 patients who, as in Tranel et al.’s (2001) study, failed the SANT yet passed the Picture Attribute Test; however, of this group of 10 patients, 7 failed various combinations of the other 4 tests, suggesting that they do have at least some defects in the processing of action verbs and concepts.
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.