|Home | About | Journals | Submit | Contact Us | Français|
Research examining set-shifting has revealed significant difficulties for adults with autism spectrum disorders (ASDs). However, research with high-functioning children with ASDs has yielded mixed results. The current study tested 6- to 13-year-old high-functioning children with ASD and typically developing controls matched on age, gender, and IQ using the Intradimensional/Extradimensional (ID/ED) Shift Test from the Cambridge Neuropsychological Test Automated Battery. Children with ASDs completed as many ED shifts and reversal ED shifts as controls; however, they made significantly more errors than controls while completing the ED reversal shifts. Analyses on a subset of cases revealed a significant positive correlation between ED reversal errors and the number of repetitive behavior symptoms in the ASD group. These findings suggest that high-functioning children with ASDs require additional feedback to shift successfully. In addition, the relationship between set-shifting and non-social symptoms suggests its utility as a potentially informative intermediate phenotype in ASDs.
Individuals with autism spectrum disorders (ASDs) struggle to shift their attention during daily activities (Gioia et al., 2002) and on laboratory measures (see Hill, 2004 and Pennington and Ozonoff, 1996 for comprehensive reviews). Set-shifting in laboratory tasks requires successful switching between two tasks or sorting the same stimulus by different features. Set-shifting is one of several cognitive components that fall within the executive function (EF) domain, which enables individuals to achieve goals efficiently (Pennington and Ozonoff, 1996). The present study examines: (1) set-shifting in 6- to 13-year-old children with and without high-functioning ASDs using the Intradimensional/Extradimensional (ID/ED) Shift Test from the Cambridge Neuropsychological Test Automated Battery, and (2) the relationship between set-shifting performance and core repetitive behavior/restricted interest symptoms in the ASD group.
Set-shifting is one cognitive process that holds promise as an intermediate phenotype in ASDs. Intermediate phenotypes include cognitive processes that serve as a link between the observed behavioral symptoms of neurodevelopmental and psychiatric disorders and their brain and/or genetic bases (Gottesman and Gould, 2003).This connection requires establishing an association between the cognitive process and core symptoms as well as brain and genetic indices. Several independent investigations report a positive relationship between set-shifting deficits and restricted interest/repetitive behavior symptoms in autism (Lopez et al., 2005; South et al., 2007). Furthermore, in a recent functional neuroimaging investigation of set-shifting in adults with ASDs, the activation patterns of anterior cingulate cortex and left intraparietal sulcus correlated negatively with the restricted interest/repetitive behaviors index from the Autism Diagnostic Interview–Revised (ADI–R: Shafritz et al., 2008).This finding connects brain function during a set-shifting task with core symptoms of ASDs. Taken together, these findings suggest set-shifting as a potentially valuable intermediary between ASD symptoms and neurobiological functioning.
To investigate this possibility further, a reliable measure of set-shifting with as few extraneous (cognitive and motor) demands as possible and with an established link to brain function is needed. Investigations of set-shifting in ASD have typically relied on two measures of flexibility: the Wisconsin Card Sorting Test and the ID/ED Test. The former is criticized for its lack of specificity to flexibility (Hill, 2004), social administration (Ozonoff, 1995), and language demands (Liss et al., 2001). In contrast, the ID/ED Test measures flexibility in a systematic fashion that allows for controlled increases in shifting demands (Cambridge Cognition, 1996). It consists of nine stages, which encompass three types of tasks: simple object discrimination; shifting within a single dimension (e.g. shape: intra-dimensional (ID) shift); and shifting attention from one dimension to another (e.g. ignore shape and attend to previously ignored line: extra-dimensional (ED) shift). Embedded in ID and ED shifting is a reversal shift that requires participants to maintain the same rule but select an alternate exemplar (see Figure 1). Success on ID/ED is measured both by the ability to pass a stage (number of stages completed) and by the number of errors made while passing a stage (errors to criterion). Experiments with monkeys have linked lesions to lateral and orbital regions of the prefrontal cortex with impaired performance on ED and ED reversal shifts, respectively (Dias et al., 1996). Atypical development of orbital (Girgis et al., 2007) and lateral (Carper and Courchesne, 2005) prefrontal regions in ASDs highlights the importance of assessing flexibility in ED shift and ED reversal shift conditions.
While ID/ED is a promising task for investigating flexibility in children with ASDs, studies to date have yielded mixed results. Some investigations report deficits (Hughes et al., 1994; Ozonoff et al., 2000; 2004) while others do not (Edgin and Pennington, 2005; Goldberg et al., 2005; Happé et al., 2006; Landa and Goldberg, 2005). Conflicting results may reflect variability in the participants’ age and the stages of the task which are analyzed. Studies focusing on younger and narrower age ranges of high-functioning children have not reported performance on ED reversal shifting and have not found ED shift deficits in ASD groups. Two of the three studies finding deficits on the ID/ED involve ASD groups with mean ages in the teens, and the third involves cognitively impaired children. In addition, both of the studies that analyzed ED reversal shifting found deficits in the ASD group. In summary, previous investigations document deficits in ED shifts and reversal shifts in high-functioning adolescents and adults with ASD, while investigations of primary school age children have failed to find deficits in ED shifts and have not reported on ED reversal shifts (see Table 1 for a summary of ID/ED studies in ASD).
The current study attempts to address this gap in primary school age children (6–13 years) by probing both ED shifting and ED reversal shifting in high-functioning children with ASDs and typically developing (TD) controls matched on age, gender ratio, and IQ. We focus on higher-functioning individuals because ASD-specific difficulties are best observed in individuals whose deficient performance is not confounded with general cognitive impairments (arguments made by Hill and Bird, 2006). In light of previous work with smaller samples in our age range we predict group differences in errors, but not in the number of stages completed (Goldberg et al., 2005; Landa and Goldberg, 2005). Consistent with this hypothesis, Luciana and Nelson (2002) report a lack of sensitivity of the stage success criterion beyond 9 years of age. Given conflicting evidence regarding increased errors during the ED shifting stage, we constrain our hypothesis to an expectation of increased errors in the ASD group on the ED reversal stage, as reported by Ozonoff and colleagues (2004). In addition, in line with recent evidence that shifting relates to restricted interests/repetitive behaviors (Lopez et al., 2005; South et al., 2007), we predict a positive relationship between the number of errors committed during these two stages and restricted interest/repetitive behavior symptoms in the ASD group.
Forty-two children with ASDs (35 with high-functioning autism and seven with pervasive developmental disorder not otherwise specified) were recruited for this study through a hospital clinic specializing in ASDs and neuropsychological assessment. Specific diagnoses were informed by diagnostic testing outlined below and made by experienced clinicians using DSM-IV-TR criteria (American Psychiatric Association, 2000) in combination with criteria outlined by the Collaborative Programs of Excellence in Autism (CPEA: Lainhart et al., 2006).The ASD diagnostic subgroups did not differ significantly from one another on demographic characteristics (age, IQ, and gender) or ID/ED performance (stages completed, mean errors across stages; data not shown). To increase power they were collapsed into an omnibus ASD group. One child was prescribed psychostimulant medication (methylphenidate) at the time of the study, but the parents withheld medication 24 hours prior to testing; one other child was on other psychotropic medications that were not discontinued (sertraline, mirtazapine, atomoxetine). All children but one (with a comorbid diagnosis of tic disorder not otherwise specified) in the ASD group had a history free from comorbid neurological disorders. Inclusion of this child’s ID/ED data did not alter the pattern of results; therefore, these data were retained in all analyses. See Table 2 for descriptive characteristics and ASD versus TD comparisons.
Eighty-four TD children were recruited from the community via advertisements and through studies at the National Institute of Mental Health (NIMH). Participants were screened and excluded if found to have developmental, learning, neurological, or psychiatric disorders as well as psychiatric medication usage.
Diagnoses in the ASD group were confirmed with the Autism Diagnostic Observation Schedule (ADOS–G: Lord et al., 2000) and the Autism Diagnostic Interview (ADI: Le Couteur et al., 1989) or Autism Diagnostic Interview–Revised (ADI–R: Lord et al., 1994). Children enrolled in the TD group did not receive diagnostic measures, but were screened through a brief parent-completed interview and questionnaire. For both groups, a full-scale IQ score of at least 80 was required for study entry and was derived from testing on a Wechsler instrument: Wechsler Abbreviated Scale of Intelligence, Wechsler Intelligence Scale for Children - Third Edition, Wechsler Intelligence Scale for Children - Fourth Edition (Wechsler, 1991; 1999; 2003).
(Cambridge Cognition, 1996). In this task, four empty rectangular boxes appear on a computer screen, and each trial starts with two stimuli in separate opposing boxes (left–right or top–bottom).The stimuli are abstract unfamiliar pink shapes or white line drawings. Children are instructed to select a stimulus and then induce a rule through computer feedback (‘correct’ displayed in green or ‘wrong’ displayed in red); after selecting correctly for six consecutive trials, the rule changes. Children must make six consecutive correct selections within 50 trials to successfully complete a stage, and the task ends when they fail a stage. The task has nine stages (see Figure 1). Stages 1–5 are discrimination stages which require the subject to distinguish between one of two shapes through trial and error learning, while ignoring distracting shapes. Stages 6 and 7 introduce ID shifting demands to apply the old rule to new stimuli. Specifically:
Stage 8 and 9 require ED shifting because the subject must attend to a previously ignored feature of the stimulus. Specifically:
Dependent measures include: (1) the number of stages completed, (2) the number of errors to criterion within a stage (hereafter referred to as errors), and (3) the total trials to criterion (errors + correct answers).
All participants were tested in a laboratory setting within the context of two separate larger studies. Parental consent and participant assent were obtained prior to testing in accordance with the requirements of the IRB from the Children’s National Medical Center and the National Institute of Mental Health. Children received monetary compensation for their time.
For the current analysis, we included the number of stages completed and errors for the ED and ED reversal stage; trials to criterion were not included for either stage because this measure is fairly redundant with errors to criterion. The trials-to-criterion variable provides information on whether children break cognitive set (i.e. change rules without negative feedback). Previous set-shifting findings suggest that individuals with ASD, regardless of functioning level, are more likely to perseverate than break cognitive set (Pennington and Ozonoff, 1996).
We first examined stage completion in an independent groups t-test and non-parametrically in stage completion (pass; fail) by group (ASD;TD) χ2 analysis for stages 8 and 9 to investigate overall set-shifting success. Previous studies have employed parametric analyses to document group differences (Goldberg et al., 2005; Luciana and Nelson, 2002); however, we also conducted a non-parametric analysis because of the discontinuous nature of the dependent variable. For all t-tests, a Welch t-test was used to correct for the unequal sample sizes between the two diagnostic groups.
We next examined the number of errors by group (2) and stage (2) with a repeated measures multivariate analysis of variance (RMANOVA). We focused on stages 8 and 9 which require ED shifting and ED reversal shifting, respectively. When children did not attempt a stage, they were assigned 25 errors as per the CANTAB manual (Cambridge Cognition, 1996).
We re-examined stage errors only among children who attempted all nine stages. This subset analysis excluded children who received a substitution score of 25 errors (chance performance) by not attempting or reaching stage 9. It is unknown whether children who fail to reach stage 9 would perform at chance or perform worse than chance by applying a previously reinforced rule. The final subset of 78 children (27 ASD; 51 TD controls) maintained matching on IQ (t(75) = 1.18, p = 0.24), age (t(76) = 0.86, p = 0.39), and gender ratio (χ2(2, N = 78) = 1.19, p = 0.28) that was observed in the full sample. For all t-tests, a Welch t-test was used to correct for the unequal sample sizes between the two diagnostic groups.
Substitution scores included for children not attempting a stage could increase variability and skew the distribution when conducting correlation analyses with symptoms from the ADI/ADI–R. Therefore, we conducted our correlation analyses on the subset of children with ASD that attempted all nine stages. Data from the ADI/ADI–R were used in the correlation analyses with ID/ED performance as the ADOS provides a narrow window (30–45 minute child interview) in which to observe symptoms while the ADI/ADI–R covers the child’s lifetime. We conducted both Pearson’s r and Spearman’s rho correlations to examine relationships with symptom scores from the ADI/ADI–R because these scores have not been established as continuous measures of symptom severity (Sears et al., 1999).
The ASD and TD groups did not differ in stages completed, t(88.11) = 0.55, p = 0.59 (see Table 2 for stages completed and stage errors). The χ2 analyses revealed no group differences in number of children completing stage 8 (χ2(2, N = 126) = 0.15, p = 0.70) or stage 9 (χ2(2, N = 126) = 1.92, p = 0.17).Twenty-seven of 42 children in the ASD group and 51/84 children in the TD group passed stage 8; 18/42 children in the ASD group and 47/84 children in the TD group passed stage 9. The group by stage RMANOVA on errors to criterion did not show a main effect of group (F(1, 124) = 2.02, p = 0.16). However, the analysis revealed a significant stage by group interaction (F(1, 125) = 9.08, p < 0.005). Follow-up t-tests revealed that children in the ASD group made significantly more errors than children in the TD group during stage 9 (t(124) = 2.17, p < 0.05) but not during stage 8 (t(124) = 0.49, p = 0.63) (see Figure 2).
The group by stage RMANOVA on errors to criterion on the subset of 78 children yielded a significant main effect of group (F(1, 76) = 13.77, p < 0.001), with children in the ASD group making significantly more errors than children in the TD group. Furthermore, the analysis also revealed a significant stage by group interaction (F(1, 76) = 10.64, p < 0.005). Follow up t-tests again showed that children with ASD made significantly more errors in stage 9 than children in the TD group (t(34.14) = 3.31, p < 0.01) and a trend in this direction in stage 8 (t(49.92) = 1.82, p = 0.07) (see Figure 3).
Correlation analyses revealed significant relationships between the ADI/ADI–R restricted interest/repetitive behavior symptom domain and ID/ED performance on stage 9, but not on stage 8 (see Table 3). This finding was significant using both Pearson’s r and Spearman’s rho correlations. Social and communication domain scores from the ADI/ADI–R were not significantly associated with either stage 8 or stage 9 ID/ED performance.
This study probed set-shifting abilities in high-functioning children with ASD by assessing both stage completion and errors on the ID/ED shift test. Consistent with previous reports, children with ASD completed as many stages as TD children; however, children with ASD made significantly more errors than TD children during the ED reversal stage.We did not observe group differences in the number of errors made during the ED shift stage. We also examined errors within a large subset of children who at least attempted all nine stages. This subset analysis was consistent with the full sample analysis in that children with ASD made significantly more errors during the ED reversal stage than did TD controls. Correlation analyses revealed a significant positive relationship between restricted interest/repetitive behavior symptoms on the ADI/ADI–R and errors completed during ED reversal shifting, but not during ED shifting.
The current investigation not only documents ED reversal shifting deficits but also, for the first time, links these deficits with restricted interest/repetitive behavior symptoms among high-functioning children with ASD. Our finding of an ED reversal shifting deficit for high-functioning children in the full and subset analyses supports the two previous studies which have reported on ED reversal shifting in ASD. The present study extends findings from a study of low-functioning children with ASDs (Hughes et al., 1994) and a study including a combined group of children and adults with high-functioning ASDs (Ozonoff et al., 2004). Our findings are consistent with, and expand on, previous investigations of high-functioning primary school age children with ASD, which reported intact ED shifting but did not report on ED reversal shift performance.
ED reversal involves reversing a valence assignment, switching the reward value from positive to negative for a stimulus, and the orbitofrontal cortex plays a key role in stimulus–reward assignments (Loveland et al., 2008). Associating a stimulus and reward within a social interaction (i.e. reward is meted out by a person) is impaired in young children with ASDs (Dawson et al., 2002). Loveland and colleagues reduced the level of social demands and still reported reversal deficits in primary school aged children (Loveland et al., 2008). The present findings extend this line of investigation by demonstrating ED reversal deficits in a computerized task that has no social demands, suggesting that the orbitofrontal-limbic network may also underlie non-social reversal learning deficits.
The correlation between ED reversal errors and restricted interest/repetitive behavior symptoms highlights the utility of reversal set-shifting as a potential intermediate phenotype for ASDs. As discussed above, intermediate cognitive phenotypes serve as a link between behavioral symptoms and brain and genetic assays. In line with past studies, the present findings confirm the relationship between set-shifting difficulties and restricted interest/repetitive behavior symptoms in children with ASDs (Kenworthy et al., 2008; Lopez et al., 2005; South et al., 2007). Further investigation of ED reversal shifts might include other groups of children with developmental disorders, however, in order to determine the specificity of these findings to ASDs alone. In addition, it would be important to investigate whether this relationship occurs across all ages and functioning levels of individuals with ASDs. Previous studies investigating ED reversal shifts did not investigate the relationship between ED reversal shifting performance and ASD symptoms (Hughes et al., 1994; Ozonoff et al., 2004).
One puzzling aspect of the current findings is the relatively unimpaired ED shifting but deficient ED reversal shifting in our high-functioning ASD group. Past studies of individuals with ASDs of similar ages and functioning levels also report successful ED shifting (Edgin and Pennington, 2005; Goldberg et al., 2005; Happé et al., 2006; Landa and Goldberg, 2005), but the one study probing ED reversal shifting found deficits in both ED shifting and ED reversal shifting (Ozonoff et al., 2004). Two potential interacting confounds may explain this apparent discrepancy of findings: (1) the inclusion of a substitution score of 25 errors for stages not attempted, and (2) the age groups tested in previous studies. The substitution score inflates variance and this may obscure group differences of modest effect size in the ED shift stage. Studies reporting no ED shifting deficits for ASD groups tested mostly primary school aged samples. Primary school aged samples may require a higher number of substitution scores for both ASD and TD groups during the ED shift stage than an older, particularly TD, sample. This discrepancy in the utilization of substitution scores may lead to reduced variability in the older sample which increases the power to detect differences of moderate effect size.
While the present study provides novel insights into the set-shifting abilities of high-functioning children with ASDs, several unanswered questions remain for future investigations. Examination of the relationship between these deficits and restricted interest/repetitive behavior symptoms was coarse due to utilization of a summary score (i.e. the restricted interest/repetitive behavior symptoms scale from the ADI/ADI–R), and would be better investigated using a continuous measure, such as the Social Responsiveness Scale (Constantino and Gruber, 2005). Future investigations of ED reversal shift deficits may also elect to segregate this behavioral symptom construct and probe whether ED reversal shifting deficits relate to higher-order cognitive rigidity, reflected in resistance to change, insistence on sameness, and rituals, or to lower-order repetitive and sensory behaviors, reflected in stereotypies and self-stimulation. Additionally, the high average IQ of our ASD group may limit generalizing the current results to lower-functioning children with ASDs.
Strengths of the study include the large, well-characterized ASD sample within a narrow age range, which provided adequate power to detect group differences and significant correlations. Children with ASDs were well matched group-wise (with a two-to-one ratio) for age, IQ, and gender ratio; this matching was sustained on the subset analysis. The current investigation also presents ID/ED data on the largest exclusively pediatric ASD sample published to date (n = 42), which allowed matching the ASD and TD groups on performance. While the correlation analysis was conducted on a subset of the total sample (n = 27), raising issues of power or sample stability, this sample size was equal to or larger than full samples reported earlier (Edgin and Pennington, 2005; Goldberg et al., 2005; Landa and Goldberg, 2005). Furthermore, the significant correlation observed between the number of ED reversal shift errors and restricted interest/repetitive behavior symptoms was robust and immune to the influence of outliers because we used non-parametric (Spearman’s rho) correlations. Thus, we avoided limitations of previous investigations by recruiting a large group of high-functioning children with ASDs over a narrow age range when dynamic developmental gains in set-shifting skills are observed (Luciana and Nelson, 2002), and by examining group differences in performance on both the ED shift and ED reversal shift tasks.
The present study documents inefficient ED reversal shifting among high-functioning children with ASDs, and reinforces the utility of the ID/ED test in parsing specific components of set-shifting difficulty among individuals with ASD. Furthermore, the current investigation establishes a link between ED reversal shifting and restricted interest/repetitive behavior symptoms used to diagnose ASDs. This finding strengthens the case for utilizing set-shifting as a potential intermediate phenotype for informing gene–brain–behavior models of ASDs.
We thank the Frederick and Elizabeth Singer Foundation, the Studies for the Advancement of Autism Research and Treatment (STAART: NIMH U54 MH066417), and the NIMH Intramural Research Program (IRP) for supporting data collection. We also thank the Intellectual and Developmental Disabilities Research Center at Children’s National Medical Center (NIH IDDRC P30HD40677) for supplementary support. A T-32 postdoctoral fellowship through the IDDRC (NIH T32HD046388) supported BEY and the NIMH IRP supported GLW, MJC and JNG. Thanks to Dr Laura Anthony for comments on an earlier draft. Finally, we thank the children and families that offered their time and energy for the current study.
Benjamin E. Yerys, Children’s National Medical Center, USA.
Gregory L. Wallace, National Institute of Mental Health, USA.
Bryan Harrison, Children’s National Medical Center, USA.
Mark J. Celano, National Institute of Mental Health, USA.
Jay N. Giedd, National Institute of Mental Health, USA.
Lauren E. Kenworthy, Children’s National Medical Center, USA.