|Home | About | Journals | Submit | Contact Us | Français|
In this paper, we review the most recent and often conflicting findings on conventional measures of executive control in autism spectrum disorders. We discuss the obstacles to accurate measurement of executive control, such as: its prolonged developmental trajectory; lack of consensus on its definition and if it is a unitary construct; the inherent complexity of executive control; and the difficulty measuring executive control functions in laboratory or clinical settings. We review the potential of an ecological validity framework to address some of these problems, and describe new tests claiming verisimilitude, or close resemblance to “real life” demands. We also review the concept of veridicality, which allows for the measurement of the ecological validity of any task, and discuss the few studies addressing ecological validity in individuals with autism. Our review suggests that a multi-source approach emphasizing veridicality may provide the most comprehensive assessment of executive control in autism.
Executive control is a widely studied construct in the field of autism spectrum disorders, henceforth referred to in this paper as autism. Though the research findings are inconsistent (see below), most clinicians, teachers, and family members agree that individuals with autism have difficulties with various aspects of executive control in their daily lives. This review will explore this discrepancy by addressing the following three questions:
Our goal is to highlight novel areas of research in this field that could: advance the study of executive control in autism; improve its ability to capture the daily executive control obstacles that impair individuals with autism; and provide new targets for intervention.
Autism was first described as a frontal, executive dysfunction disorder in a seminal paper by Damasio and Maurer (1978), in which they described similarities in behavioral presentation between individuals with autism and individuals with frontal lesions. They argued that atypical functioning of the frontal lobes could drive the lack of social motivation, poor communication, and perseverative behavior commonly observed in autism. In the 30 years that have followed, almost 150 articles and three major reviews (Hill, 2004; Pennington & Ozonoff, 1996; Sergeant et al., 2002) have been published on executive control in autism.
Executive control is an umbrella term that captures a set of cognitive processes that direct behavior regulation and orchestration of attaining a future goal (Welsh & Pennington, 1988). These cognitive processes include working memory, inhibition, cognitive flexibility, monitoring, planning, and generativity (Rogers & Bennetto, 2000). While the neural basis of executive control cognitive processes includes multiple cortico-cortical and cortico-striatal loops (D’Esposito, 2007; Miller & Cohen, 2001; Robbins, 2007), the frontal lobes are thought to play a central role (Miller, 1999; Miller & Cohen, 2001; Stuss & Benson, 1984).
Although the claim of executive dysfunction as a causal factor in autism (Ozonoff et al., 1991; Russell, 1997) is controversial (Dawson et al., 2002; Griffith et al., 1999; Liss et al., 2001; Yerys et al., 2007), it is clear that executive dysfunction plays a role in the social and cognitive deficits observed in individuals with autism (e.g., Hughes, 1996; Lopez et al., 2005). In their extensive review, Pennington and Ozonoff (1996) reported larger effect sizes associated with cognitive inflexibility and poor planning in autism than for any executive dysfunction measures in other developmental disorders (i.e., attention deficit hyperactivity disorder, conduct disorder, and Tourette’s syndrome). Inhibition and working memory, on the other hand, were considered mostly intact in autism. Six years later, Sergeant et al. (2002) reported similar findings in their review. Hill’s (2004) review supported the argument for significant planning and cognitive flexibility deficits in individuals with autism, as well as suggesting difficulties with inhibiting a prepotent response (i.e., response inhibition), and generativity. However, Hill (2004) also highlighted the mixed results of deficits across different age bands and the various executive control cognitive processes. This work suggested that executive control laboratory tasks capture atypical cognition in individuals with autism, but that evidence for a unique executive control profile in autism, or evidence that executive control is the singular cause of autism, is weak.
A literature review entering the terms ‘autism’ with ‘executive function’, ‘working memory’, ‘set-shifting’, ‘planning’, ‘cognitive flexibility’, ‘inhibition’, or ‘cognitive control’ into common search engines revealed more than 40 new articles assessing executive control in individuals with autism in comparison to a control group subsequent to Hill’s (2004) review (see Table 1). Neuropsychological and neuroimaging investigations have yielded mixed results about executive control weaknesses in autism. For high-functioning children and adults, several neuropsychological studies continue to document planning and cognitive flexibility deficits in autism relative to typically developing controls (Geurts et al., 2004; Joseph et al., 2005a; Lopez et al., 2005; Ozonoff et al., 2004; Tsuchiya et al., 2005; Winsler et al., 2007); however, the evidence is not conclusive. Planning was originally considered the most robust deficit in autism with a pooled Cohen’s d greater than 2 (Pennington & Ozonoff, 1996), but Table 1 shows that deficits in planning were observed in five of the 10 studies published since 2004 with two studies showing mixed results and three showing no differences. When analyzed at the task level, however, these results are less contradictory. Early conclusions about planning abilities in autism were predominantly drawn from performance on standard forms of the Tower of London and Tower of Hanoi tasks. The findings of our review confirm deficits in autism groups, as compared to controls, on these and similar tower tasks (Geurts et al., 2004; Joseph et al., 2005a; Lopez et al., 2005; Verté et al., 2005; Pellicano et al., 2006). Recently, however, many investigators have used a computerized version of the tower task (typically the CANTAB Stockings of Cambridge) and have often failed to find deficits in individuals with autism (Ozonoff et al., 2004; Goldberg et al., 2005; Happé et al., 2006; Just et al., 2007; but see Landa & Goldberg, 2005; Sinzig et al., 2008). This difference in performance with computer versus human administration is an important theme that we will return to later in this review.
The same pattern occurs with cognitive flexibility. A number of recent investigations suggest that high-functioning individuals with autism do not differ from typically developing controls in this domain (Bogte et al., 2007; Goldberg et al., 2005; Landa & Goldberg, 2005; Edgin & Pennington, 2005; Yerys et al., 2007). Again, however, previously reported deficits on the traditional Wisconsin Card Sorting Test (human) administration are upheld. In contrast, the majority of negative findings involve a computerized measure of flexibility, the CANTAB Intra-dimensional/Extradimensional shift test.
Other executive control processes, such as inhibition and working memory, which were initially reported as being spared in autism (Ozonoff & Strayer, 1997; 2001), have been shown as deficient in recent studies with high-functioning children and adults with autism (Christ et al., 2007; Geurts et al., 2004; Joseph et al., 2005a; Luna et al., 2007; Steele et al., 2007; Williams et al., 2005). It is difficult to reconcile different findings regarding inhibition. Hill (2004) postulates that inhibition of prepotent responses is specifically impaired in autism, with sparing of other inhibitory functions. However, functioning even on a simple motor response inhibition task, Go/NoGo, is not consistently intact in recent reports (Christ et al., 2007). Variability in IQ between autism and comparison groups may play a role in these mixed findings. Other studies finding inhibition deficits employ a variety of tasks, each measuring inhibition in the context of disparate domain–specific demands (e.g., oculomotor saccades (Luna et al., 2007)).
Whereas performance on working memory tasks has historically been relatively under-reported in autism studies (for example, it is not addressed in Hill’s 2004 review), there has been an explosion in the number of recent reports on the topic, which generally indicate impaired spatial working memory in autism on four different tasks, with both computer and human administration formats (Geurts et al., 2004; Goldberg et al., 2005; Happé et al., 2006; Joseph et al., 2005a; Landa & Goldberg, 2005; Luna et al., 2007; Sinzig et al., 2008; Steele et al., 2007; Verté et al., 2005; Williams et al., 2005; Williams et al., 2006; but see Edgin & Pennington, 2005). At the same time, several studies indicate spared N-back, as well as digit, letter, and word span performance in autism (Koshino et al., 2005; Koshino et al., 2008; Lopez et al., 2005; Nakahachi, et al., 2006; Williams et al., 2005; Williams et al., 2006; Williams et al., 2008).
Perhaps with the exception of robust spatial working memory deficits in autism, it could be argued that our clarity regarding affected executive control subdomains in autism has actually attenuated as we accumulate more research in this area. There are many factors driving confusion in this literature. Some have to do with the complexity and heterogeneity of intellectual functioning and neuropsychological profiles in autism, which generally complicate assessment in this group. A primary problem involves the difficulty disentangling social cognition and motivation deficits in autism from performance on any socially administered neuropsychological task. An example of this is Ozonoff’s (1995) finding of improved flexibility on the Wisconsin Card Sorting Test when it was administered to individuals with autism using a computer, as opposed to a human examiner. A parallel finding can be inferred from the consistent evidence of autism-related impairment on human-administered tower tasks, but generally intact performance on computer-administered tower tasks (see above). Another problem is that autism is associated with high levels of comorbidity with symptoms of disorders such as attention deficit hyperactivity disorder and obsessive compulsive disorder (Leyfer et al., 2006), which present executive control deficits that can obscure the executive control profile. For example, Sinzig et al. (2008) found greater inhibition problems in children with autism and attention deficit hyperactivity disorder symptoms than in children with autism alone. Finally, as Hill (2004) points out in her review, variability in the intellectual level of individuals with autism affects the expression of executive control deficits on various tasks. There are also many problems in the general measurement of executive control that are interfering with our ability to come to a consensus on the executive control profile in autism. A number of these problems are discussed in the following section.
We struggle to capture executive control with traditional measures of the construct for a number of reasons, some of which relate to its prolonged developmental trajectory. Other major obstacles to defining the executive control profile arise from: a lack of consensus about the construct itself, such as its definition and whether it is unitary or fractionate; and the inherent complexity of EC, especially its goal setting and plan execution components, which are difficult to capture in traditional laboratory or clinical settings. Each of these issues is discussed below.
The protracted developmental trajectory of executive control through the first decades of life, reflecting the late development of prefrontal brain structures (Bunge et al., 2002; Casey et al., 1995; Durston et al., 2002; Lenroot et al., 2007; for review see Mesulam, 2002), complicates assessment techniques. As Hill (2004) and Jurado and Rosselli (2007) review, discrete executive control processes, such as cognitive flexibility and planning, reach maturity at disparate times in typical development. As a result, investigations of preschoolers with autism (Dawson et al., 1998; 2002; Griffith et al., 1999; Stahl & Pry, 2002; Yerys et al., 2007) may find intact cognitive flexibility and therefore report intact executive control in autism, while an investigation of school age children tapping later developing executive control abilities, such as planning, finds deficits, and therefore reports impaired executive control in autism (Ozonoff et al., 2004). In this case, its prolonged developmental trajectory contributes to conflicting findings, as well as a second important problem in the measurement of executive control: disagreement on its fundamental components.
At the most basic conceptual level, accurate assessment of executive control is challenged by a lack of consensus about the definition of this relatively new construct in neuropsychology (see Jurado & Rosselli, 2007 for review). This leads to problems in autism research when investigators sample different component processes of executive control and draw disparate conclusions about whether or not executive control is impaired. A related problem in the assessment of executive control is disagreement regarding whether it is a unitary or fractionate construct in both typical and atypical development. While some theories of executive control in developmental disorders emphasize impairment in specific, potentially fractionable, processes, such as inhibition (Barkley, 1997) or working memory and inhibition (Pennington & Ozonoff, 1996; Ozonoff & Jensen 1999), others describe executive control as a unitary concept related to goal-directed behavior (e.g., Duncan, 1986). The executive control tasks that have most reliably demonstrated impairments in autism are multi-factorial tasks that combine multiple executive control demands into one, such as the Wisconsin Card Sorting Test and the tower tasks. Even the spatial working memory tasks that have proven difficult for autism groups have strategic demands (Cambridge Cognition, 1996). These findings have traditionally been reported as indication of specific autism-related deficits in “flexibility,” “planning,” and “working memory,” but could in fact represent other weaknesses. In the case of the Wisconsin Card Sorting Test, cognitive flexibility, abstract reasoning, generation of problem-solving ideas, response to socially presented feedback, working memory, and other abilities are required (Gioia & Isquith, 2004). The traditional tower tasks require inhibition of prepotent responses, working memory, planning, generation of problem-solving ideas (Hill & Bird, 2006), as well as adherence to socially presented rules. It is possible that the autism literature is suffering from premature association of complex tasks with specific subdomains of executive control, such as flexibility or planning. In fact, weighing the evidence, the safest conclusion to draw from the plethora of studies may be that individuals with autism perform poorly on tasks that tap multiple executive control processes simultaneously.
This leads to the importance of acknowledging the complexity of the executive control construct. Major theories of executive control describe it as a “central process” that governs responses to complex problems and requires: determining where to focus and switch attention (Baddeley & Logie, 1999); correction of errors, sequencing of actions and multi-step problem-solving (Shallice & Burgess, 1991); and creation and maintenance of goal-related behavior (Burgess et al., 1998). Another aspect of the complexity of the executive control construct is that it is a domain-general function subserved by distributed networks throughout cortical and subcortical structures in the brain (Miller & Cohen, 2001; Royall et al., 2002). Thus, modality-specific demands (i.e., verbal versus visuospatial) of traditional laboratory tasks can complicate interpretation of domain-general executive control deficits (Burgess, 2006; Denckla, 2002). This has been a particular problem in the autism literature in regard to measuring executive control among those with language impairments which can complicate identification of executive control deficits (e.g., Joseph et al., 2005a; Bennetto et al., 1996).
Finally executive control related goal-setting and planning abilities, which include the execution, as well as the development, of plans (Chevignard et al., 2000), and the ability to engage and disengage actions in the service of overarching goals (Hill & Bird, 2006), are difficult to measure in the laboratory. In fact, there is a direct conflict between the parameters of clinical and laboratory assessment settings and the settings which demand higher levels of executive control. Both research and clinical assessments are usually conducted in a quiet room, with one highly supportive adult examiner prompting performance. In this structured arrangement, the examiner provides the plan, organizes the activities, gives cues regarding performance, probes for elaboration, presents tasks one at a time, and generally supports executive control (Gioia & Isquith, 2004; Bernstein & Waber, 1990). Children with autism are known to respond well to highly structured settings (Schopler et al., 1995) and explicit behavioral expectations (Lovaas, 1987). Thus, they often perform better on laboratory-based executive control measures than on executive control related tasks in more natural settings. This discrepancy has led to the investigation of ecologically valid, or “real world,” tasks to tap executive control in the settings that demand it for successful, efficient performance.
There are many assessment strategies that can help us address limitations in executive control measurement. First, the construction of comprehensive executive control batteries that tap multiple subdomains that can be aggregated into composite scores (e.g., the Behavior Rating Inventory of Executive Functions, (Gioia et al., 2000)) enables researchers to directly test unitary versus fractionate theories of executive control (Gioia et al., 2002c) and encourages measurement of multiple domains in any given study. Second, access to strategic plan execution is improved by techniques designed to reveal the processes involved in learning and problem-solving (e.g., Kaplan’s (1988) process approach; the microgenetic approach (Siegler & Crowley, 1991); and eye tracking techniques). It is further enhanced by the development of standardized strategy scores for neuropsychological tests (e.g., the Rey Osterrieth Developmental Scoring System (Bernstein & Waber, 1996); California Verbal Learning Test, (Delis et al., 2000)). Finally, the problem of capturing a complex process such as executive control is also addressed by an emphasis on ecological validity.
Ecological validity is increasingly valued in the evaluation of neuropsychological tests and research, especially in regard to measures of executive control. In their recent review, Chaytor and Schmitter-Edgecombe (2003) define ecological validity as the degree to which task performance corresponds to real world performance, and they argue that ecological validity does not describe a task but rather the inferences that are drawn from task performance. Following Kvavilashvili and Ellis (2004), Burgess et al. (2006) define ecological validity as a measure of the “representativeness” of the task, or the correspondence between the task and real life situations, and the “generalizability” of the task, or the degree to which task performance predicts problems in real life settings. In this paper, we will use the terminology from Franzen and Wilhem (1996), who refer to the “verisimilitude” of tasks, or their resemblance to demands in the every day environment. They further argue that the ecological validity of any neuropsychological task can be measured by its “veridicality,” or correlation with measures of everyday functioning.
The advent of brain imaging and other sophisticated diagnostic techniques has shifted neuropsychology’s role from diagnosis and lesion location to the definition of functional capacities at home, work, and school, and elevated the importance of ecologically valid measures of neuropsychological constructs (Chaytor & Schmitter-Edgecombe, 2003; Manchester et al., 2004). This has required the development of new tests, as most commonly used executive control tasks, such as the Wisconsin Card Sorting Test (Berg, 1948), were developed many years ago, when the goals of neuropsychology and the understanding of executive control were both quite different than they are today (see Jurado & Rosselli, 2007 for a review; Burgess et al., 2006). This development coincides well with the specific need for increasing the sensitivity of executive control assessment specifically (Goldberg et al., 2005). Tasks striving for verisimilitude get us out of the laboratory, at least figuratively speaking, and more readily contain the demands to integrate multi-dimensional information, determine priorities, set subgoals, and incorporate feedback that are typical of both the real world and executive control functions (Shallice & Burgess, 1991).
Ecologically valid executive control measures fall into two broad categories: task-based approaches or questionnaire-based approaches (see Table 2 for a list of ecologically valid executive control tasks). The most commonly used and comprehensive ecologically valid executive control task is the Behavioral Assessment of the Dysexecutive Syndrome. It is a battery of six measures, which tap planning, organization, shifting, inhibition, novel problem-solving, and temporal judgment by requiring the subject to engage in familiar activities such as searching for lost keys or planning a visit to the zoo. The Behavior Rating Inventory of Executive Function is a widely studied questionnaire that taps up to nine subdomains of executive control through self or informant (parent or teacher) report about executive control abilities in daily life.
Ecologically valid executive control measures vary in the amount of research supporting their validity, reliability, and specificity. Some measures provide normative data and thus have the advantage of being standardized to neurotypical individuals (the Behavioral Assessment of the Dysexecutive Syndrome, the Behavior Rating Inventory of Executive Function, the Dysexecutive Questionnaire, the Frontal Systems Behavior Scale, and Tests of Everyday Attention). Moreover, the Behavioral Assessment of the Dysexecutive Syndrome, the Behavior Rating Inventory of Executive Function, and the Tests of Everyday Attention all have child versions that provide developmental norms for studying ecologically valid executive control and enable the researcher to investigate most of the same executive control domains across much of the life span. This does not discount the potential contribution of other measures such as the Multiple Errands Shopping Test (Shallice & Burgess, 1991; Alderman et al., 2003), the Cooking Task (Chevignard et al., 2008), or the Test Taking Strategy Task (Kofman et al., 2008); however, additional research is needed to validate and standardize these measures.
Research on the validity of the ecologically valid executive control measures, such as the Behavioral Assessment of the Dysexecutive Syndrome, comes with its own set of criticisms. The Behavioral Assessment of the Dysexecutive Syndrome, like some traditional executive control tasks such as the Wisconsin Card Sorting Test, was developed to discriminate between neurotypical adults and adults with frontal lobe brain lesions (Manchester et al., 2004; Jurado & Rosselli, 2007). This logic assumes that all adults with frontal lobe lesions will exhibit executive control deficits, and will therefore perform poorly on executive control measures. However, the Behavioral Assessment of the Dysexecutive Syndrome, like the Wisconsin Card Sorting Test (Shallice & Burgess, 1991) has not always successfully discriminated between adults with frontal lobe lesions and neurotypical adults (Wood & Liossi, 2006). The validity of the Behavioral Assessment of the Dysexecutive Syndrome improves when it is combined with a secondary source of information, such as an informant-based questionnaire like the Dysexecutive Questionnaire (Dywan & Segalowitz, 1996). For example, if a relative completing the Dysexecutive Questionnaire identifies cognitive flexibility as an impaired executive control process for the patient, then the patient is more likely to exhibit deficits on the shifting test from the Behavioral Assessment of the Dysexecutive Syndrome (Adlerman et al., 2003). This type of convergent evidence may appear as a tautology, but it does highlight the need for observing executive control functioning from multiple sources. While many previous studies in autism use multiple lab measures that tap the same construct (Christ et al., 2007; Geurts et al., 2004; Happé et al., 2006; Kenworthy et al., 2005; Yerys et al., 2007), research efforts may be better served by combining lab or ecologically valid executive control measures with standardized informant reports, such as the Dysexecutive Questionnaire, the Frontal Systems Behavior Rating Scale, or Behavior Rating Inventory of Executive Function.
Another problem for ecologically valid executive control tasks is that despite their verisimilitude, or their resemblance to demands of the everyday environment, they sometimes lack veridicality, that is, they do not actually correlate with measures of everyday executive control functioning. Again the Behavioral Assessment of the Dysexecutive Syndrome is the best studied in this regard, with mixed findings. Wilson and colleagues (1998) report significant correlations between the Behavioral Assessment of the Dysexecutive Syndrome and the Dysexecutive Questionnaire, and Norris and Tate (2000) partially replicate these findings. In a study of four Behavioral Assessment of the Dysexecutive Syndrome subtests, Wood and Liossi (2006) found limited ecological validity in patients with severe head trauma, and Evans et al. (1997) report significant Behavioral Assessment of the Dysexecutive Syndrome-Dysexecutive Questionnaire relationships in patients with neurologic impairments, but not those with schizophrenia. Overall, Chaytor and Schmitter-Edgecombe (2003) conclude that there is some evidence of superior veridicality in tasks with verisimilitude as opposed to standard laboratory tasks, but clearly the association cannot be assumed. Further research regarding the relatively new ecologically valid executive control tasks described in Table 2 should address this issue directly.
Ecologically valid executive control questionnaires have a further liability. The validity of self-report for executive control function has been called into question (see Chaytor & Schmitter-Edgecombe, 2003 for a review of this issue), although Obhuda et al. (2005) report accurate self-report from one brain-injured group. Given common problems with self-awareness and self-monitoring in individuals with executive control impairments (Gioia et al., 2000), the consensus is that it is important to obtain informant report on executive control related questionnaires. A related concern is the influence of bias (in either direction) on the part of any informant.
Finally, by virtue of their reliance on everyday activities, ecologically valid executive control tasks frequently tap multiple abilities (e.g., language, processing/motor output speed), and can easily confound domain-general executive control functions with domain-specific functions (Jurado & Rosselli, 2007; Burgess, 1997). This confound may be unavoidable in the executive control domain. As described above, the traditional executive control tasks that are most sensitive in autism tap multiple processes simultaneously. From a neurodevelopmental perspective, Bernstein and Waber (2007) argue that what we call executive control relies not on functional modules but on functional neural networks that develop in the context of experience. This observation appears particularly relevant to autism, which has defied modular explanation at the genetic, neuroanatomical, neurofunctional, and behavioral levels, and is increasingly understood as a disorder of distributed networks in the brain (Müller, 2007). In any case, ecologically valid executive control tasks, even more than their traditional counterparts, must be interpreted cautiously lest we prematurely specify executive control as the culprit in deficient performance on a multi-dimensional task. In autism the problem is most acute when the task occurs in a social context. As noted above, findings of deficient executive control in autism on both the Wisconsin Card Sorting Test and tower tasks are attenuated with computer administration. These cautionary statements apply equally to executive control questionnaires, which are inherently tapping multiply determined behaviors, as they record observations in real life, uncontrolled settings.
A review of investigations in autism with tasks that claim ecological validity through verisimilitude reveals only 12 studies (see Table 3). To date, most independent studies of ecologically valid executive control measures in autism have used either the Behavioral Assessment of the Dysexecutive Syndrome or the Behavior Rating Inventory of Executive Function. Four studies using the Behavioral Assessment of the Dysexecutive Syndrome reveal mixed findings. Although Hill and Bird (2006) found executive control deficits on the Behavioral Assessment of the Dysexecutive Syndrome in a group of high-functioning adults, Rajendren and colleagues (2005) and Harris and colleagues (2008) failed to find deficits on the Behavioral Assessment of the Dysexecutive Syndrome in adolescents with autism, and Boucher and colleagues (2005) found intact performance on the Zoo Map subtest of the Behavioral Assessment of the Dysexecutive Syndrome in young adults with autism. Results from the Predicaments Test (Channon et al., 2001), another ecologically valid executive control task, have been reported in a study of adolescents with autism. Adolescents with autism were impaired compared to typically developing controls; however, this task entails significant social demands, which cannot be disentangled from its executive control aspects. It requires the participant to generate multiple possible socially appropriate solutions to awkward social situations.
In contrast to real life tasks, Table 3 shows that investigations of questionnaires filled out by primary caretakers provide convergent evidence of unique executive control impairments in children and adolescents with autism. Investigations using the Behavior Rating Inventory of Executive Function revealed executive control deficits in autism when compared to: normative data (Gilotty et al., 2002; Kenworthy et al., 2005; Mackinlay et al., 2006); typical controls (Gioia et al., 2002b; Winsler et al., 2007); and children with ADHD (Gioia et al., 2002b; Winsler et al., 2007). Comparing Behavior Rating Inventory of Executive Function profiles across groups of children with different developmental and acquired disorders, revealed unique deficits in flexibility for children with autism (Gioia et al., 2002b). Other measures, such as the Dysexecutive Questionnaire and the Behavioral Flexibility Rating Scale – Revised (Peters-Scheffer et al., 2008) have also identified difficulties in children with autism (Channon et al., 2001; Didden et al., 2008). While the Behavioral Flexibility Rating Scale–Revised is focused solely on children’s ability to shift from one activity to another, the Dysexecutive Questionnaire assesses a broader set of symptoms associated with the dysexecutive syndrome in adults.
In summary, with the exception of the Behavioral Assessment of the Dysexecutive Syndrome, the use of ecologically valid executive control tasks and questionnaires has yielded convergent findings that children and adults with autism struggle on unstructured tasks (or are rated as such by parents) that they encounter in their daily lives. However, the potential confound of social demands on these measures suggests that more work with other ecologically valid executive control tasks is needed to understand whether social deficits are the driving factor in identifying more ecologically valid deficits in autism.
Another approach to improving the ecological validity of executive control assessment in autism is to assess the veridicality of standard executive control tests and favor those tests which show a positive relationship with important everyday outcome variables (Chaytor & Schmitter-Edgecombe, 2003). As the experience with the Behavioral Assessment of the Dysexecutive Syndrome demonstrates, task veridicality can be population specific; thus, it should be established specifically for autism groups of different functioning levels across different ages. A measure’s veridicality is also influenced by the everyday outcome variable selected. Some investigations of veridicality have used general measures, such as job performance (Kibby et al., 1998) or adaptive behavior (Gilotty et al., 2002), while others use measures of behavior more specifically related to executive control, such as the Dysexecutive Questionnaire (Burgess et al., 1998) and the Behavior Rating Inventory of Executive Function (Mackinlay et al., 2006). Burgess and colleagues (1998) present evidence that in adult neurological patients executive control fractionates at the behavioral as well as at the cognitive level. He argues therefore for use of a nuanced executive control related measure such as the Dysexecutive Questionnaire when testing veridicality. The findings of Mackinlay and colleagues (2006) indicating correlations between some subdomains of the Behavior Rating Inventory of Executive Function, but not others, with a laboratory measure of multi-tasking, in a study of autism is consistent with Burgess and colleagues’ (1998) argument. At this point, however, data on the veridicality of specific tasks in any population, not to mention autism, is so sparse that conclusions are premature.
There has been more extensive, although still inconclusive, work in autism assessing the relationship between performance on traditional executive control tasks and number of symptoms. Significant correlations have been reported between autism symptoms and executive control measures, including verbal fluency (Bishop & Norbury, 2005), the CANTAB Intradimensional/Extradimenesional shift task (Yerys et al., in press); a composite score of the Wisconsin Card Sorting Test and Trail Making test (Lopez et al., 2005) and the Wisconsin Card Sorting Test (South et al., 2007).
Several conclusions can be drawn from the literature reviewed here. There are robust deficits in autism groups on two multi-dimensional traditional executive control tasks, the Wisconsin Card Sorting Test and tower tests. However, when the human examiner is replaced with computer administration, the magnitude of these findings erodes, indicating perhaps that it is the socially mediated response to feedback and adherence to socially presented arbitrary rules that is difficult for individuals with autism, as opposed to the executive control flexibility and planning processes that are usually associated with the Wisconsin Card Sorting Test and tower tasks. Recent investigations with traditional executive control measures also indicate that spatial working memory tests with strategic task components are consistently difficult for individuals with autism, whether administration is computerized or not.
Confidence in the definitiveness of these findings is limited by heterogeneity of cognitive and behavioral presentation within the autism spectrum and many problems inherent in the measurement of executive control in the laboratory. Regarding the former, one future approach involves studies with large sample sizes that allow for the explicit assessment of the role that intelligence, psychiatric co-morbidities (such as attention deficit hyperactive disorder), and other factors (such as language) play in executive control profiles in autism. Some of the measurement problems in executive control reflect its complex and prolonged developmental trajectory, which can be addressed only through longitudinal studies of executive control in autism, or at the least, consistent use of multiple tasks tapping different aspects of executive control across the life span, with a sensitivity to the typical developmental trajectories of specific component processes of executive control. Another difficulty involves the multifaceted character of executive control, and the confusion which arises when investigations report contradictory findings that are difficult to interpret because different aspects of executive control were measured. The development of comprehensive executive control batteries, such as the Behavioral Assessment of the Dysexecutive Syndrome, which allow interpretation of executive control as both a unitary and fractionated phenomenon, is one potential response to this difficulty. Use of comprehensive batteries across age bands and populations may resolve some of the present confusion in the literature.
Other challenges include the difficulty of capturing complex goal-setting/plan execution abilities in the highly structured laboratory setting. Process sensitive techniques, such as eye tracking and standardized strategy scores, can capture some aspects of these complex functions. In addition, fMRI and other neuroimaging techniques, whose overall contribution to understanding executive control in autism is beyond the scope of this paper, may elucidate issues surrounding goal setting/plan execution abilities in several ways. On the face of it, confining a subject to a narrow, noisy metal tube in which he or she cannot move would appear to be antithetical to tapping real world executive control abilities, but the advent of virtual reality technology improves the capacity of researchers to simulate daily living experiences. By revealing the neurobiological correlates of performance, neuroimaging can demonstrate the extent to which goal/plan tasks depend on the integration of multiple abilities, in the executive control domain or in other neuropsychological domains. This may restrain our tendency to prematurely draw overly precise conclusions about what abilities can be inferred from performance on complex tasks such as the tower. Neuroimaging has also demonstrated that even when the performance of an autism group is intact on a unitary executive control task, such as the inhibitory Go/No-Go test, atypical neural networks are recruited (Kana et al., 2007; Schmitz et al., 2006). This may be useful in both clarifying conflicting findings on inhibition tasks and elucidating the breakdown of performance on complex executive control tasks that require several abilities (e.g., inhibitory control and working memory) combined.
Finally, the development of a new generation of ecologically valid executive control tasks designed to mimic real life problem-solving is encouraging. Vigilance regarding the social and language demands that may be inherent in complex executive control tasks striving for verisimilitude is warranted, however. Disentangling impairments due to reduced social motivation and impaired social cognition from truly deficient executive control abilities on purported measures of executive control is a major challenge for the coming years. In this regard the computerized virtual reality tasks, such as the Virtual Errands Test (Law et al., 2006), are interesting, because they provide a modality for assessing real life goal setting and plan execution without imposing social requirements. Although not computerized, the Battersea Multitasking Paradigm (Mackinlay et al., 2006) was developed specifically for assessment in autism, and it was introduced in a paper that offers a useful model for future investigation of executive control in autism. MacKinaly et al. (2006) combined a rigorously designed laboratory measure of executive control with a comprehensive executive control informant questionnaire, which provides both an ecologically valid measure of executive control and a measure of the veridicality of the task findings. Such inclusive use of the multiple tools that are now available to query executive control function in autism will improve in our understanding of this complex neuropsychological construct in this equally complex population. In the last 30 years almost 150 peer reviewed articles on executive control in autism have been published, yet we are aware of only one peer reviewed report on an intervention to address executive control in autism (Fisher & Happé, 2005). It is our hope that improved understanding and measurement of executive control and its impact on daily functioning will encourage the further development of innovative interventions that target executive control, and enhance the everyday lives and independence of individuals with autism.
We thank Alex Martin for his review and helpful comments on this manuscript and Richard B. Roberson III and Kathryn F. Jankowski for assistance with its preparation. We also thank the people with autism and their families who have educated us over the years. The views expressed in this article are those of the authors and do not necessarily reflect the official position of the National Institute of Mental Health (NIMH), the National Institutes of Health, or any part of the U.S. Department of Health and Human Services. Mention of trade names, commercial products, or organizations does not imply endorsement by the U.S. government. LK is supported by an award from The Isadore and Bertha Gudelsky Family Foundation and the Intramural Research Program of the NIMH. LA is supported by awards from the Organization for Autism Research and The Isadore and Bertha Gudelsky Family Foundation. GLW is supported by the Intramural Research Program of the NIMH. BEY is supported by the Intellectual and Developmental Disabilities Research Center (NIH IDDRC P30HD40677).