Systematic instruments are needed that can facilitate the complicated diagnostic process concerning ASD in adults. The current study is the first that examined the psychometric properties of ADOS module 4 in an independent sample of high-functioning adult males with an established clinical ASD diagnosis compared to meaningful and relatively homogeneous clinical and non-clinical groups. Our findings show that ADOS module 4 is a reliable instrument. At all levels (i.e. classification, domains and items) raters obtained substantial agreement. In addition, ADOS module 4 has good general criterion-related validity. It is able to correctly classify the majority of individuals and higher scores on the ADOS predict a higher probability of having a clinical ASD diagnosis. The high Areas under the Curve are further indications that ADOS scores can predict whether an individual actually has an ASD. Furthermore, group comparisons between the ASD and other groups show that the ADOS is valuable in differentiating between ASD, and psychopathy and typical development. The distinction between psychopathy and ASD even holds when only taking into account forensic individuals with ASD (although the group size was rather small to perform such an analysis). The finding that ASD and psychopathy are so well-discriminated by means of ADOS scores is promising for forensic psychiatric settings.
Another finding is the similarity between ASD and schizophrenia with respect to ADOS scores. Clearly, individuals with schizophrenia and marked negative symptoms show behavior that is very similar to ASD (Frith and Happé 2005
). Some patients with schizophrenia even have autistic-like symptoms that covary with negative symptoms (Sheitman et al. 2004
). In line with these data, we show that the degree of negative symptomatology correlates significantly with ADOS scores, in particular with items resembling negative symptoms, such as (lack of) directed facial expressions and shared enjoyment. This resemblance makes it difficult for an observational instrument such as the ADOS to differentiate these groups on that behavior (see Reaven et al. 2008
for a similar finding in children with childhood-onset schizophrenia). The findings underscore previous recommendations of using a comprehensive assessment that incorporates information on daily functioning and early development with direct observation to reach a clinical diagnosis (Lord et al. 1999
). Nevertheless, four items did show a difference between these groups: individuals with ASD use more stereotyped language, less reciprocal social communication, and display qualitatively poorer social responses and overall rapport. This suggests that core social items and stereotyped language discriminate individuals with ASD from those with schizophrenia.
Although findings are preliminary, the revised SARRB domain, which combines social, communication and repetitive behavior items, seems promising in this and other respects. It not only discriminates ASD from all other groups including schizophrenia, but also has high internal consistency, and does well in identifying ASDs: a higher score on this domain predicts a higher probability of a clinical ASD diagnosis with 33% per additional point. Another positive indication for the revised algorithm is the confirmation that stereotyped language fits better with the RRB factor than with the original communication domain. Notwithstanding the caution of interpreting ASDs in adults in exactly the same way as in children, the revised algorithm as developed for modules 1–3 seems promising for module 4 as well. More research is needed in a larger sample containing individuals with more severe autistic symptoms and lower levels of daily functioning to further investigate the revised algorithm.
A marked finding is the limited role of the original communication domain in the identification of ASDs in this sample. Despite group differences between ASD and psychopathy/typical development, the communication domain does not predict a clinical ASD diagnosis. Combined with its low internal consistency, the communication domain as such does not seem to add to the validity of ADOS module 4 in the current sample. However, when communication items are incorporated in the revised algorithm, a consistent scale (SA) emerges that is valuable in the diagnostic procedure for ASD. Similarly, although restricted and repetitive behaviors were rare in our ASD sample, their contribution to SARRB supports the distinction of ASD from schizophrenia. The relatively short duration of the ADOS interview naturally could have played a role in the paucity of RRBs (Lord et al. 1999
). However, combining these two findings also fits the general clinical picture: in adolescents and adults with ASD there is a greater prevalence of impairment in non-verbal communication and social reciprocity than in verbal communication or repetitive behaviors and stereotyped interests (Shattuck et al. 2007
). In fact, repetitive behaviors decline most strongly with age (Seltzer et al. 2003
). Apart from ageing, individuals in our sample might have had relatively more intact verbal skills from the outset as they were all considered to be high-functioning. Stereotyped language, however, does differentiate the ASD group from all other groups in our sample. This may be typical of our high-functioning group, because idiosyncratic language and language complexity are positively associated (Volden and Lord 1991
). Cultural differences in the use of gestures might also have played a role. Typically developing adults in our sample, for instance, used few emotional and only occasional descriptive gestures themselves.
The sensitivity in our sample was rather low (0.61), which means that not every individual with a clinical diagnosis of ASD obtained a concurrent classification on the ADOS. It is probable that the characteristics of our group played a role in this. Our sample consisted of high-functioning individuals that signed up for an extensive research project. They are probably situated at the milder end of the spectrum and some might have been able to (partly) compensate some behavior due to their high intelligence. Resulting relatively low scores make it difficult for the ADOS to identify these individuals. Our findings resemble the outcomes in ADOS modules 1–3, in which lower sensitivity (SE) was found for distinctions involving children with milder ASDs (module 3 by Lord et al. 2000
, SE = 0.80, versus later studies: de Bildt et al. 2009
, SE = 0.64; Gotham et al. 2008
, SE = 0.49; Gotham et al. 2007
, SE = 0.68). The high specificity (0.82), on the other hand, means that a positive ASD classification on the ADOS is a very strong indication for a clinician to consider diagnosing ASD. Sensitivity and specificity are tightly linked and the aim of the assessment determines which one is most important. High specificity is essential when one needs to be certain that the individuals selected actually have an ASD, for instance in autism research. High specificity can, however, lead to underinclusiveness. When the aim of the assessment is to screen for ASD, high sensitivity is crucial in order not to miss any potential case. For this purpose, lower thresholds could be considered at the expense of specificity. To prevent overinclusiveness, developmental history and current daily functioning should then be carefully reviewed. As this study included only a specific ASD group and specific control groups, further research is needed to establish the optimal cut-off points on the ADOS module 4.
This study has a number of limitations that should be taken into account when interpreting the results. First, compared to studies on the psychometric properties of modules 1–3 (de Bildt et al. 2009
; Gotham et al. 2007
; Oosterling et al. 2010
), our study has a small sample size (n
= 93). However, it is the first study examining module 4 in an adult population with ASD compared to specific and meaningful groups. Second, we are focused on high-functioning adult males with ASD, which means results cannot be generalized to the entire ASD population. Future studies on module 4 should comprise a larger sample, including individuals with lower levels of daily functioning, since the high-functioning character of our sample may have influenced the results. On the other hand, exactly these individuals are not always recognized during childhood. Therefore, increasing knowledge on module 4 seems most important for individuals showing milder autistic symptoms. In this light, it will also be important to include a group of high-functioning adult females, who run the risk of being undiagnosed because they might be especially good at compensating their behavior (Attwood 1999
; In ‘t Velt-Simon Thomas and Mol 2008
). Third, no standardized measures were available for the clinical diagnosis of ASD, which characterizes current practice in adult psychiatry. However, the normal clinical procedure included review of developmental history and current functioning and observation. In addition, most participants with ASD were recruited through a specialized centre. Fourth, we did use standardized measures to diagnose schizophrenia, but not to review early developmental history in this group. Therefore, we cannot eliminate the possibility that ASD was present before the onset of schizophrenia. However, this possibility is minimized by the fact that these individuals were extensively tested in a specialized psychosis centre and selected for this study by experienced clinicians. The control groups in the current sample were comparatively homogeneous and aimed to challenge the ADOS by comparing ASDs with other psychiatric groups with social deficits. For the investigation of ADOS’ value in differential diagnostics, examining different subtypes of schizophrenia and other diagnostic groups will be of great relevance as well (e.g. anxiety disorder, depression, ADHD, and OCD).
In summary, the ADOS module 4 is a reliable instrument that has good predictive value for ASD. It can adequately discriminate ASD from psychopathy and typical development in an adult population. With respect to schizophrenia, discrimination is more difficult due to behavioral overlap. These groups are, however, different on some core items. Although ADOS module 4 fails to classify ASD in a significant proportion of our higher functioning and more mildly affected ASD group, its ASD classification is a strong lead for a clinician to at least consider an ASD diagnosis. Explorative analyses of the revised algorithm indicate that a revision -in line with modules 1-3 and developments in criteria for ASD- could be beneficial for discriminating ASD from schizophrenia.