This section will provide a brief review of screening tools that have been administered to general population samples, include children less than 24 months of age, and have published psychometric information about the accuracy of detecting children with ASD. Any screening tool should have strong psychometric features to support its accuracy in identifying at-risk children who need further evaluation. Sensitivity (true positives), specificity (true negatives), positive predictive value, and negative predictive value provide particularly important information about the accuracy of screening tools. To be considered psychometrically sound, a screening tool would minimally need to report sensitivity and specificity. Meisels (1989)
recommended that both sensitivity and specificity be no less than 80% for developmental screening of young children; however, he noted that a “75% sensitivity ratio is considerably less favorable than a 75% specificity proportion” (p. 579). Adjusting cutoffs to increase sensitivity will decrease specificity and vice versa, and therefore, one should not be considered without the other. There are not recommended standards for positive and negative predictive value because they are related to the base rate of a disorder. That is, the higher the prevalence rate of the disorder, the greater the probability that a positive result will be correct and the higher the positive predictive value. In screening a general population for relatively low incidence disorders such as ASD, even an instrument with a sensitivity and specificity of .80 will yield a poor positive predicative value (Clark & Harrington, 1999
There are three autism-specific screening tools that have been used with a general population sample. The Checklist for Autism in Toddlers
(CHAT; Baird et al., 2000
; Baron-Cohen, Allen, & Gillberg, 1992
; Baron-Cohen et al., 1996
), consisting of 9 items reported by parents and 5 items observed by a health professional at the 18-month developmental checkup, was the first to be studied. Baird et al. (2000)
reported on a follow-up at age 7 years of 16,235 children screened with the CHAT at a mean age of 18.7 months. At follow-up at age 7 years, 94 cases of ASD were identified. The CHAT correctly identified 33 children, which is a rate of 2.03 per 1,000, well below the expected prevalence rates. These findings indicate that the CHAT has a specificity of 97.7% but a sensitivity of 35.1% and positive predictive value of 8.1% (Baird et al., 2000
), and missed more children at 18 months who were later diagnosed with ASD than it detected. The poor sensitivity and corresponding high false negative rate indicate that the CHAT is not a valid screening tool at 18 months. It should not be relied on as an accurate screener and likely does not merit the time in a pediatric practice.
The Modified Checklist for Autism in Toddlers
(M-CHAT; Robins, Fein, Barton, & Green, 2001
; Robins, & Dumont-Mathieu, 2006
; Kleinman et al., in press
) consists of 23 parent report questions using the original 9 items from the CHAT as a basis. The MCHAT can be downloaded from www.firstsigns.org
. The M-CHAT was initially studied on 1,122 children from a general pediatric sample at age 16–30 months and had 3 positive screens (Robins et al., 2001
). This is a rate of 2.7 per 1,000 based on the low-risk sample, which is only slightly better than the CHAT with children slightly older and still below the expected prevalence rates. They estimated that sensitivity was 97%, specificity was 99%, and positive predictive power was 80%, but more accurate measures cannot be determined until a follow-up study is conducted as with the CHAT. In a more recent replication study, Kleinman et al. (in press)
administered the M-CHAT to 3,309 children from a general pediatric sample at a mean age of 20.5 months. A telephone interview was administered to caregivers of children with a positive screen to review failed items at a mean age of 22.7. They reported that 189 children had a positive screen initially and 31 after the phone interview, with a positive predictive value of 11% for the M-CHAT and 65% for the M-CHAT combined with the telephone interview. They detected 20 children later diagnosed with ASD from the general pediatric sample. This is a rate of 6.0 per 1,000, which is near current ASD prevalent estimates.
In conclusion, the M-CHAT questionnaire alone without the telephone interveiw, even at a mean age of 20.5 months does not appear to have better positive predictive value than the CHAT at a mean of 18.7 months with a general pediatric sample. Kleinman et al. concluded that the M-CHAT should only be used in combination with an interview with a general pediatric sample in order to reduce false positives and avoid unnecessary referrals and parent concern. It is noteworthy that the M-CHAT was more promising with a high-risk sample at a mean of 24.3 months and the interview did not improve positive predictive value sufficiently to warrant the time with the high risk sample. It is premature to judge sensitivity and specificity of the M-CHAT until a more thorough follow-up study is conducted to carefully detect possible missed cases.
The Early Screening of Autistic Traits Questionnaire
(ESAT; Dietz, Swinkels, Daalen, Engeland, & Buitelaar, 2006
) is a 14-item two-stage screening instrument designed for use at 14–15 months of age. A pre-screening instrument with 4 ESAT items was designed for use at well-baby clinics as the first stage of screening. A Dutch screening study pre-screened 31,724 children selected from a random population sample at a mean age of 14.91 months and 370 children screened positive; 255 or 69% agreed to participate in a second screening stage during a home visit using the 14-item ESAT. They detected 18 children with ASD from the positive screens, indicating a positive predictive value of 25% and a rate of 0.57 per 1,000, well below the expected prevalence rates. The false positives included children with other developmental delays. These findings do not provide support for the validity of the ESAT as an ASD-specific screener for a general population sample. The low number of children with ASD detected may be partly due to the young age that they were screened. Some children with ASD may not show detectable features at 14–15 months as documented in prospective studies of younger siblings (Landa, Holman, & Garrett-Mayer, 2007
There is only one broadband screener that has been studied to detect children with ASD, albeit preliminary. The Infant-Toddler Checklist
(ITC; Wetherby & Prizant, 2002
; Wetherby et al., 2004
) is one component of the Communication and Symbolic Behavior Scales Developmental Profile (CSBS DP; Wetherby & Prizant, 2002
) and is designed as a broadband screener for communication delays. The ITC can be downloaded from www.firstsigns.org
. The ITC includes 24 questions with 3 to 5 choices about developmental milestones of social communication. It also asks the following question about concerns: “Do you have any concerns about your child’s development?
”, and if yes, to describe the concerns. The Flesch reading ease score is 84.0, which would be easily understood by an average 12-year old and the Flesch-Kincaid grade level is 4.9, which is based on a U.S. grade level and corresponds to an age of 10 to 11 years. The ITC is a standardized tool that, in addition to screening cutoffs, has standard scores at monthly intervals from 6 to 24 months based on a normative sample of over 2,188 children (Wetherby & Prizant, 2002
Wetherby et al. (2004)
reported on a preliminary study of 3,021 children from a general population sample screened with the ITC between 6 and 24 months through the longitudinal research of the FIRST WORDS®
Project. Children performing in the bottom 10th
percentile on the ITC and randomly selected children performing within normal limits were invited for a communication evaluation using the CSBS DP Behavior Sample during the second year of life. This is an interactive structured observation of social communication that is norm-referenced and was videotaped. Red flags of ASD were rated from the behavior samples of 36 children with communication delays, 18 who received a diagnosis of ASD at 3 years of age and 18 with developmental delay (DD) in which ASD was ruled out, and 18 children with typical development (TD). Seventeen of the 18 children in the ASD group or 94.4% had a positive screen on the ITC, 15 in the DD group or 83.3%, and 2 in the TD group or 11.1%. Sensitivity of the ITC was estimated at 88.9% when the ASD and DD groups were combined and increased to 94.4% when only the ASD group was examined with the TD group. Specificity was 88.9%. These results suggest that the ITC has high sensitivity and specificity (both 88.9%) for catching toddlers at risk for ASD and other developmental delays from a general pediatric sample. However, a follow-up study is needed to examine the validity of the ITC on a larger sample with more systematic surveillance methods to determine how many children with ASD may have been accurately detected or missed. The ITC is a broadband screener, and therefore, a positive screen indicates that the child is at-risk for a communication delay but does not differentiate a child with ASD from a child with other developmental problems.
There is a critical need for further research to develop and validate screening tools for ASD in very young children. These findings suggest that it may be more accurate to use a broadband screener followed by an ASD-specific screener to detect children with ASD at 18–24 months from a general pediatric sample. If a broadband screener is to be used as a first stage to an ASD-specific screener, then further research is needed on broadband screeners to document that they actually catch children with ASD along with children with other developmental disorders. The first aim of this study was to estimate the positive and negative predictive value of the ITC to detect children with communication delays, including children with ASD, from a general population sample of 5,385 children. The second aim was to document the percentage of positive screens and parents with concern reported on the ITC and the developmental characteristics of children later diagnosed with ASD from this general population sample.