Ecologically valid executive control measures fall into two broad categories: task-based approaches or questionnaire-based approaches (see for a list of ecologically valid executive control tasks). The most commonly used and comprehensive ecologically valid executive control task is the Behavioral Assessment of the Dysexecutive Syndrome. It is a battery of six measures, which tap planning, organization, shifting, inhibition, novel problem-solving, and temporal judgment by requiring the subject to engage in familiar activities such as searching for lost keys or planning a visit to the zoo. The Behavior Rating Inventory of Executive Function is a widely studied questionnaire that taps up to nine subdomains of executive control through self or informant (parent or teacher) report about executive control abilities in daily life.
A sampling of ecologically valid executive control tasks, including the associated age range and a brief description.
Ecologically valid executive control measures vary in the amount of research supporting their validity, reliability, and specificity. Some measures provide normative data and thus have the advantage of being standardized to neurotypical individuals (the Behavioral Assessment of the Dysexecutive Syndrome, the Behavior Rating Inventory of Executive Function, the Dysexecutive Questionnaire, the Frontal Systems Behavior Scale, and Tests of Everyday Attention). Moreover, the Behavioral Assessment of the Dysexecutive Syndrome, the Behavior Rating Inventory of Executive Function, and the Tests of Everyday Attention all have child versions that provide developmental norms for studying ecologically valid executive control and enable the researcher to investigate most of the same executive control domains across much of the life span. This does not discount the potential contribution of other measures such as the Multiple Errands Shopping Test (Shallice & Burgess, 1991
; Alderman et al., 2003
), the Cooking Task (Chevignard et al., 2008
), or the Test Taking Strategy Task (Kofman et al., 2008
); however, additional research is needed to validate and standardize these measures.
Research on the validity of the ecologically valid executive control measures, such as the Behavioral Assessment of the Dysexecutive Syndrome, comes with its own set of criticisms. The Behavioral Assessment of the Dysexecutive Syndrome, like some traditional executive control tasks such as the Wisconsin Card Sorting Test, was developed to discriminate between neurotypical adults and adults with frontal lobe brain lesions (Manchester et al., 2004
; Jurado & Rosselli, 2007
). This logic assumes that all adults with frontal lobe lesions will exhibit executive control deficits, and will therefore perform poorly on executive control measures. However, the Behavioral Assessment of the Dysexecutive Syndrome, like the Wisconsin Card Sorting Test (Shallice & Burgess, 1991
) has not always successfully discriminated between adults with frontal lobe lesions and neurotypical adults (Wood & Liossi, 2006
). The validity of the Behavioral Assessment of the Dysexecutive Syndrome improves when it is combined with a secondary source of information, such as an informant-based questionnaire like the Dysexecutive Questionnaire (Dywan & Segalowitz, 1996
). For example, if a relative completing the Dysexecutive Questionnaire identifies cognitive flexibility as an impaired executive control process for the patient, then the patient is more likely to exhibit deficits on the shifting test from the Behavioral Assessment of the Dysexecutive Syndrome (Adlerman et al., 2003
). This type of convergent evidence may appear as a tautology, but it does highlight the need for observing executive control functioning from multiple sources. While many previous studies in autism use multiple lab measures that tap the same construct (Christ et al., 2007
; Geurts et al., 2004
; Happé et al., 2006
; Kenworthy et al., 2005
; Yerys et al., 2007
), research efforts may be better served by combining lab or ecologically valid executive control measures with standardized informant reports, such as the Dysexecutive Questionnaire, the Frontal Systems Behavior Rating Scale, or Behavior Rating Inventory of Executive Function.
Another problem for ecologically valid executive control tasks is that despite their verisimilitude, or their resemblance to demands of the everyday environment, they sometimes lack veridicality, that is, they do not actually correlate with measures of everyday executive control functioning. Again the Behavioral Assessment of the Dysexecutive Syndrome is the best studied in this regard, with mixed findings. Wilson and colleagues (1998)
report significant correlations between the Behavioral Assessment of the Dysexecutive Syndrome and the Dysexecutive Questionnaire, and Norris and Tate (2000)
partially replicate these findings. In a study of four Behavioral Assessment of the Dysexecutive Syndrome subtests, Wood and Liossi (2006)
found limited ecological validity in patients with severe head trauma, and Evans et al. (1997)
report significant Behavioral Assessment of the Dysexecutive Syndrome-Dysexecutive Questionnaire relationships in patients with neurologic impairments, but not those with schizophrenia. Overall, Chaytor and Schmitter-Edgecombe (2003)
conclude that there is some evidence of superior veridicality in tasks with verisimilitude as opposed to standard laboratory tasks, but clearly the association cannot be assumed. Further research regarding the relatively new ecologically valid executive control tasks described in should address this issue directly.
Ecologically valid executive control questionnaires have a further liability. The validity of self-report for executive control function has been called into question (see Chaytor & Schmitter-Edgecombe, 2003
for a review of this issue), although Obhuda et al. (2005)
report accurate self-report from one brain-injured group. Given common problems with self-awareness and self-monitoring in individuals with executive control impairments (Gioia et al., 2000
), the consensus is that it is important to obtain informant report on executive control related questionnaires. A related concern is the influence of bias (in either direction) on the part of any informant.
Finally, by virtue of their reliance on everyday activities, ecologically valid executive control tasks frequently tap multiple abilities (e.g., language, processing/motor output speed), and can easily confound domain-general executive control functions with domain-specific functions (Jurado & Rosselli, 2007
; Burgess, 1997
). This confound may be unavoidable in the executive control domain. As described above, the traditional executive control tasks that are most sensitive in autism tap multiple processes simultaneously. From a neurodevelopmental perspective, Bernstein and Waber (2007)
argue that what we call executive control relies not on functional modules but on functional neural networks that develop in the context of experience. This observation appears particularly relevant to autism, which has defied modular explanation at the genetic, neuroanatomical, neurofunctional, and behavioral levels, and is increasingly understood as a disorder of distributed networks in the brain (Müller, 2007
). In any case, ecologically valid executive control tasks, even more than their traditional counterparts, must be interpreted cautiously lest we prematurely specify executive control as the culprit in deficient performance on a multi-dimensional task. In autism the problem is most acute when the task occurs in a social context. As noted above, findings of deficient executive control in autism on both the Wisconsin Card Sorting Test and tower tasks are attenuated with computer administration. These cautionary statements apply equally to executive control questionnaires, which are inherently tapping multiply determined behaviors, as they record observations in real life, uncontrolled settings.