This study set out to determine the contribution of EF to impairment in occupational functioning. A secondary aim was an examination of the relative utility of the two different methods of assessing EF (ratings vs. tests) in their capacity to predict such impairment. When examined individually and apart from the EF tests, self-ratings on the DEFS subscales were found to contribute significantly to all 11 occupational impairment measures, including self-rated work quality, the percentage of jobs on which these adults had experienced various behavioral and interpersonal problems or had been fired, employer ratings of overall work performance and impairment across a variety of work contexts, and clinician ratings of social and occupational adjustment on the SOFAS. Three DEFS were especially useful in these predictions, these being Self-Discipline (Inhibition), Self-Management to Time, and Self-Motivation with each contributing to five different occupational measures. The amount of variance in the occupational impairment measures accounted for by the DEFS ranged from just 5% to 22% except for the prediction of the clinician SOFAS rating (63%). These results indicate that EF as assessed by ratings of daily life activities makes some contribution to occupational impairments.
In comparison, when examined individually and apart from the EF ratings, just three EF test scores (two tests) made significant contributions to the occupational measures, the two most frequent being CPT Commission Errors and the 5-Point Test Unique Designs score. The first is typically interpreted as a measure of inhibition and contributed to various occupational impairments related to behavior, interactions with others, boredom, and hostility as well as to the number of jobs held since high school. It is understandable why problems with behavioral inhibition might contribute to such occupational difficulties. The CPT hit RT score is also viewed as a measure of response inhibition given its loading on the same factor as CPT Commission Errors in previous factor analyses (Murphy et al., 2001
). It likewise contributed to the number of jobs held by our participants. These findings are also consistent with the findings above from the DEFS that problems with inhibition, or self-discipline, in daily life contribute to these same occupational difficulties.
The second EF test of any utility was a measure of nonverbal working memory and fluency—the 5-Point Test. It made separate contributions to several occupational measures, specifically from the employer rating scale: Employer rated workplace impairment and work performance quality. These results are consistent with current views that working memory may contribute to some of the inattention symptoms seen in ADHD (Barkley, 1997/2001
) and thus to workplace problems related to inattention. Nonverbal working memory may be more impaired in ADHD than verbal working memory (Martinussen et al., 2005
) and thus may contribute more to inattention in ADHD which may explain the utility here of a nonverbal working memory test to predict workplace problems possibly associated with inattention.
When the EF ratings and tests were examined jointly to evaluate their unique contributions to the occupational measures, the EF ratings contributed significantly to 10 of the 11 occupational measures when entered first and to 9 of them when entered second in the regression analyses. The greatest of these contributions was to the clinician SOFAS rating where they explained 45%–67% of the variance depending on the order of entry. The contribution of the EF tests when entered second after the EF ratings was not significant on 10 of the 11 measures. But it did contribute to a small degree (10% additional variance) to the percentage of jobs on which the individual had difficulties with their own behavior and work performance. Recall that this was the only occupational measure related to IQ and so it may be the involvement of IQ in the EF tests that led to this contribution. When the EF tests were entered first, they contributed to just two additional occupational measures, whereas the EF ratings continued to contribute unique variance to the majority of those measures. These results suggest that a few EF tests add some unique variance to the prediction of occupational adjustment beyond that contribution predicted by EF ratings, but the contribution is relatively small. EF ratings, in contrast, predict significant variance in the majority of occupational adjustment measures and often do so beyond that variance captured by EF tests. We hypothesized that EF tests would show substantially weaker relationships with our various measures of occupational impairment than would the EF ratings. This hypothesis was confirmed. If predicting occupational impairment is an important aspect of the validity of EF measures, then EF ratings were superior to EF tests in doing so. These findings also agree with the study by Mitchell and Miller (2008)
that found that EF tests were only modestly related to both ratings and observations of daily functional activities.
These results provide further support for our earlier contention that EF tests should not be viewed as the only standard of evidence for establishing the presence of EF deficits, particularly in ADHD as prior research has stated or implied (Biederman et al., 2006
; Boonstra et al., 2005
; Hervey et al., 2004
; Jonsdottir et al., 2006
; Nigg et al., 2005
; Wilcutt et al., 2005
). As reported elsewhere (Barkley & Murphy, 2009
), EF deficits are present in the vast majority of adults with ADHD (89%–98%) when ratings of EF in daily life activities are used. Thus, where weak or no EF test deficits are found in those having ADHD relative to control groups or are not significantly related to ADHD severity, this should not be taken to indicate that EF deficits are not part of or related to ADHD, as some have concluded (Boonstra et al., 2005
; Jonsdottir et al., 2006
; Marchetta et al., 2008
; Wilcutt et al., 2005
). Deficits may not be apparent in tests evaluating EF while being ubiquitous using ratings. And if predicting impairment in major life activities is evidence of validity, then EF ratings are far superior to EF tests in doing so. Thus, although neither method alone should serve as the gold standard for determining the presence of EF deficits, EF ratings may be taken to have greater validity at predicting impairment in daily life activities, and particularly occupational adjustment.
Our results, however, should not be taken to mean that EF tests may not be valuable for assessing certain features of EF. We assert this despite the numerous concerns we have raised about these tests as the best indices of the construct of EF and despite their limited predictive utility concerning impairments studied here. Although there appears to be uniformity of opinion that the construct of EF is broad and comprises multiple components (Castellanos et al., 2006
), we believe that the nature of EF is also multileveled and hierarchical. Regardless of its definitional ambiguity, EF, like the prefrontal cortex that largely facilitates it, is most likely organized in a hierarchical system that allows smaller sequences of behavior to become clustered into more complex, nested sets of larger goal-directed actions that are sustained over longer intervals of time (Badre, 2008
). And those complex behavioral sets can be further arranged into even larger nested meta-sets to accomplish even longer term and larger goals spanning days, weeks, months, or even years (Botvinick, 2008
This hierarchical functional organization of the prefrontal lobes is highly consistent with the view that the EFs largely created by those lobes must likewise be hierarchically organized. The EFs should comprise increasingly higher and larger levels of behavioral organization with each comprised of longer and more varied and complex goal-directed actions. In our view, the nature of EF is likely to be similar to that of driving a motor vehicle that involves not only multiple cognitive processes at a basic level, but also several hierarchically organized levels of abilities (basic cognitive, instrumental, tactical, and strategic). Research on driving, like that on EF, has shown little or no relationship between clinic-based evaluations of basic cognitive abilities known to be necessary but not sufficient for driving (vision, RT, inhibition, motor coordination) or even simulator-based assessments of driving and measures of tactical and strategic driving levels involved in actually driving in real-world situations, such as may be measured by ratings or direct observations of actual driving behavior. And such basic cognitive clinic-based tests have little or no association with measures of adverse driving outcomes (citations, accidents; Barkley, Murphy, DuPaul, & Bush, 2002
). These results can arise when measures of the lowest level of a complex and multilevel domain are necessary, yet not sufficient, to represent higher level functions or abilities utilized in daily life situations and done to meet strategic goals.
EF likewise can be conceptualized as not just involving multiple components at a basic instrumental neurocognitive level as is likely assessed by EF tests, but also involving higher levels of more complex behavioral organization at a tactical level (daily activities, immediate social goals). And those tactical actions may be further organized upward into even more complex actions and for larger goals that can be considered a strategic level of EF (longer term social, economic, occupational, and other goals spanning weeks, months, and even years). Above this level may be one of ultimate utility that encompasses the individual's long-term welfare and achievements that is likely reflected in measures of impairment that capture the long-term consequences of EF deficits at the lower levels.
At each new level of this hierarchy additional abilities and skills come into play that are not represented in lower levels and yet contribute to mastering and effectively performing executively at that and higher levels of EF. Therefore, one should not be surprised that basic measures of more proximal, short-term instrumental EF constructs (EF tests) have little relationship with the tactical or strategic level of EF as represented in ratings of EF ascertained across months. It should also not be puzzling that such basic EF tests make little contribution to the ultimate outcomes of EF as indexed indirectly by impairment in domains of major life activities that span years of EF utilization (occupational, driving, financial, social, marital, educational, etc.) and are three levels or more removed in this hierarchy. All this is to say that the purpose of choosing any method for the assessment of EF should be dictated by the purpose of the research or clinical undertaking in contrast to pitting measures of EF against each other as if all assessed the same level.
The limitations of this study should be considered in evaluating the foregoing results and interpretations. First, our study is limited by the method used to create the EF scale. It is possible that had other items of EF been generated besides the 91 we tested additional dimensions of EF deficits might have been unearthed. Yet, we believe our initial efforts were sufficiently comprehensive to provide a first broad pass at determining the possible nature of EF deficits as evaluated by rating scales. It was certainly more comprehensive, theory-based, and empirically constructed than the brief 20-item EF scale created in the study by Burgess and colleagues (1998)
. Earlier findings (Barkley & Murphy, 2009
) also suggest that no matter what narrowband subscales of particular EF deficit studies may identify, such dimensions are highly inter-correlated and imply the possibility that there is a single over-arching meta-construct of EF shared across all these dimensions.
A second limitation was the relatively circumscribed battery of EF tests used here to assess the various constructs believed to characterize EF. Had other tests of these constructs been used or had additional constructs, such as planning and problem-solving, been assessed, some differences in these results might have been evident. However, other studies that have used the DEFS and examined its relationship with an entirely different battery of EF constructs did include such tests of planning and problem-solving as well as different measures of response inhibition and working memory. Those measures were collected at the adult follow-up of children with ADHD followed to adulthood and the study found very similar results to those discussed earlier—that being little or no relationship between EF ratings and those EF tests (Barkley & Fischer, 2010
). Thus, we believe that the relatively poorer showing of EF tests to predicting impairment here is not entirely a consequence of our selection of these EF tests.
A third limitation may have occurred in the occupational impairment measures we derived from self-reports of work history that may be affected by recall biases and other influences that may make them less than accurate in reflecting the actual employment problems experienced by these participants. We tried to overcome this by also obtaining the reports of employers about current workplace problems and performance but could only get consent to do so for a subset of participants. This does not completely eliminate the problems with the historical nature of the self-reported information. We also did not measure other problems in the workplace, such as actual productivity, use of sick leave, and workplace accidents that have been found to be associated with ADHD in adults in earlier studies (de Graaf et al., 2008). Our employer rating scale can only be considered a relatively crude index of actual workplace problems that might have been revealed by more thorough or detailed workplace measures or direct observations of working. But such measures can be highly intrusive into the occupations of research participants and cannot be undertaken easily or lightly in conducting research with psychiatric patients. As an initial attempt to study occupational impairment, we felt that our employer rating scale offered minimal intrusiveness into the important domain of work for our participants while still allowing us a cursory glimpse of their potential workplace problems.
A further limitation may have occurred in the procedure of collapsing the ADHD and clinical control groups in with the community group to study the relationships among the EF measures and impairment measures. Such a heterogeneous grouping would not be expected to resemble the distributions of these measures in a general population sample of adults, and the greater range of scores and over-representation of extreme (clinical) cases may have inflated the size of the relationships obtained. This would be expected to do as much for the EF tests as for the EF ratings and so would not necessarily account for the differences we found between those measures in their predictive utility of impairments. Even so, we concede that these findings may not be readily extrapolated to the general population or even to other clinical disorders not represented in the clinical samples used here.
With these limitations in mind, this study found that EF deficits make a significant contribution to the occupational problems of adults who vary as a function of the severity of their ADHD symptoms. It therefore seems to be the deficits in EF associated with ADHD that are in part contributing to the workplace problems documented here and in past studies of adults with ADHD and in children with ADHD followed to adulthood. In this regard, ratings of EF in daily life activities may be more predictive of impairments in occupational history and current workplace functioning than are EF tests. These results and those of prior studies indicate that these two methods of assessing EF are likely sampling different aspects and even different hierarchically organized levels of EF. Consequently, neither should be taken alone as the sole index of the presence of EF deficits in various clinical populations, and particularly in ADHD.