PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of intjepidLink to Publisher's site
 
Int J Epidemiol. 2009 October; 38(5): 1224–1234.
Published online 2009 September 7. doi:  10.1093/ije/dyp261
PMCID: PMC2800781

Diagnostic change and the increased prevalence of autism

Abstract

Background Increased autism prevalence rates have generated considerable concern. However, the contribution of changes in diagnostic practices to increased prevalence rates has not been thoroughly examined. Debates over the role of diagnostic substitution also continue. California has been an important test case in these controversies. The objective of this study was to determine the extent to which the increased prevalence of autism in California has been driven by changes in diagnostic practices, diagnostic substitution and diagnostic accretion.

Methods Retrospective case record examination of 7003 patients born before 1987 with autism who were enrolled with the California Department of Developmental Services between 1992 and 2005 was carried out. Of principal interest were 631 patients with a sole diagnosis of mental retardation (MR) who subsequently acquired a diagnosis of autism. The outcome of interest was the probability of acquiring a diagnosis of autism as a result of changes in diagnostic practices was calculated. The probability of diagnostic change is then used to model the proportion of the autism caseload arising from changing diagnostic practices.

Results The odds of a patient acquiring an autism diagnosis were elevated in periods in which the practices for diagnosing autism changed. The odds of change in years in which diagnostic practices changed were 1.68 [95% confidence interval (CI) 1.11–2.54], 1.55 (95% CI 1.03–2.34), 1.58 (95% CI 1.05–2.39), 1.82 (95% CI 1.23–2.7) and 1.61 (95% CI 1.09–2.39). Using the probability of change between 1992 and 2005 to generalize to the population with autism, it is estimated that 26.4% (95% CI 16.25–36.48) of the increased autism caseload in California is uniquely associated with diagnostic change through a single pathway—individuals previously diagnosed with MR.

Conclusion Changes in practices for diagnosing autism have had a substantial effect on autism caseloads, accounting for one-quarter of the observed increase in prevalence in California between 1992 and 2005.

Introduction

A wide spectrum of studies conducted in numerous locales suggest that the measured prevalence of autism, a condition characterized by impairments in social interaction and communication, has increased markedly over the past 40 years.1 Consistent with previous studies, data from the California Department of Developmental Services (DDS) show that between 1987 and 2003 the number of autism cases handled by the California DDS increased by 634%.2 The general consensus is that these increases are striking. Equally striking is the absence of consensus regarding the degree to which changes in diagnostic standards, practice and procedures, and/or diagnostic substitution have contributed to increased prevalence rates.

Some scholars have suggested that diagnostic substitution plays a significant role in the increasing prevalence of autism.3–8 Diagnostic substitution occurs when an individual, net of changes to diagnostic standards, practices and procedures, or individual condition, is diagnosed with one condition at one time and subsequently with another condition at some further point in time. One reason why diagnostic substitution seems plausible is that autism is difficult to diagnose since there are no known biological markers and the symptoms are hard to assess, especially among persons with cognitive impairments. Evidence in support of the diagnostic substitution hypothesis arises from recent studies which have shown that increased autism rates are accompanied by concurrent declines in the prevalence of mental retardation (MR) and other developmental disabilities.3,5,7 At the same time, other studies have found no evidence of diagnostic substitution.9,10

Independent of diagnostic substitution, some scholars have argued that changes in diagnostic practices lie behind the increased prevalence of autism. Of course, these scholars also note that changing diagnostic practices and procedures may both accompany and be implicated in a process of diagnostic substitution. Since Kanner first described autism in 1943, diagnostic standards, practices and procedures have changed considerably.8,11,12 Some local studies have shown that changing diagnostic practices has a demonstrable effect on autism incidence rates, for example, in Olmstead County, Minnesota.13 A study applying current autism diagnostic criteria to adults who had a history of developmental language disorder, found that 32% (12/38) of adults qualified for an autism spectrum diagnosis under the new standards.5 Similarly, a recent study examining population incidence rates in California found that changes in diagnostic criteria may account for as much as one-third of the increased prevalence in the state.14 Hence, there is some evidence that both changes in diagnostic standards and diagnostic substitution are driving part of the observed increase in prevalence. What is less well understood is how these two processes are related and the effect they may have on the prevalence of autism.

Much of the controversy over diagnostic substitution and diagnostic change has focused on data from the California DDS. Analyses and subsequent re-analyses differ in the extent to which they contend that the increase in measured prevalence arises from diagnostic substitution and/or changes in diagnostic standards. Croen et al.15 initially suggested that much of the increase in prevalence could be attributed to diagnostic substitution. However, Blaxill et al.16 identified several concerns in the paper by Croen et al., including problematic analysis of trend information and evidence of ascertainment bias in younger cohorts. Rethinking these issues from the original paper, Blaxill et al. concluded that there was little evidence to support the idea that diagnostic substitution played a significant role in increasing autism prevalence.16 Croen et al. agreed, concluding, ‘[d]iagnostic substitution does not appear to account for the increased trend in autism prevalence’.17 Likewise, a separate study of special education records concluded that diagnostic substitution is not occurring in California, though it does appear to be happening in the majority of other states.7 In sum, although several studies have found that diagnostic substitution appears to be occurring in several locations, the evidence suggests that it is not occurring in California.

Comparing aggregated prevalence rates for autism and MR or other disabilities, rather than looking at changes in individual cases, has hampered previous studies. In fact, with only one exception,5 studies have looked at aggregated prevalence rates to identify diagnostic substitution, despite the fact that diagnostic substitution happens at the level of the individual. Comparing aggregated prevalence rates to identify diagnostic substitution can lead to erroneous conclusions. For instance, in contexts where the overall MR caseload was increasing and diagnostic substitution was occurring, a comparison of MR and autism caseloads would lead to the faulty conclusion that diagnostic substitution was not taking place. Unfortunately, one can learn little about the dynamics of diagnostic change by focusing solely on the macro level.

In this article, we consider the effect of diagnostic change at both the individual and macro levels. At the individual level, one form of diagnostic change is diagnostic substitution in which there is a switch in diagnosis (X → Y). A second form of diagnostic change at the individual level is what we identify as diagnostic accretion. Diagnostic accretion occurs when an individual initially diagnosed with one disorder subsequently acquires a second diagnosis, but retains the first diagnosis as a co-morbidity (X → X + Y). Diagnostic accretion, should it be occurring, would impact the autism caseload but would have no discernable affect on the MR population, thus making the process difficult to identify in the aggregate. At the macro level, diagnostic change refers to changes in diagnostic standards and practices that propel either substitution or accretion. Here too, we note that the aggregation of cases without disaggregating micro and macro level processes can obscure the effect that diagnostic changes may have on autism caseloads.

We seek to estimate the impact of diagnostic change on the prevalence of autism. Recall that diagnostic change captures diagnostic substitution and accretion, which may reflect changes in diagnostic practices. Changing diagnostic practices may include changes in diagnostic standards, such as a change to the Diagnostic and Statistical Manual of Mental Disorders (DSM), as well as the procedures or guidelines used when making a diagnosis. In many administrative systems, such as special education classifications, clients are assigned a single diagnosis. In contexts such as these, clinicians are allowed to check one box, and we would observe pure diagnostic substitution if persons who would have previously been diagnosed with one disorder are now diagnosed with autism. However, in the California DDS systems, clients may maintain multiple diagnoses. Thus, to observe diagnostic accretion, we can look for clients who enter the system with a single diagnosis and subsequently acquire a second diagnosis.

We are particularly interested in when these clients experience diagnostic substitution or accretion, as one of the central research questions that confronts us is whether or not changes in diagnostic practices impact individuals’ diagnostic state. For instance, if an individual had a sole diagnosis of MR under the DSM-III guidelines, but was later additionally diagnosed with autism under DSM-IV criteria, resulting in MR–autism co-morbidity, we would observe a diagnostic accretion as the result of a change in diagnostic standards. Similarly, if an individual was diagnosed with MR under one diagnostic regime and under a later diagnostic regime had a sole diagnosis of autism, this would be a diagnostic change (as a diagnostic substitution) catalysed by changing diagnostic standards or practices.

Relevant for this study are changes in diagnostic practices in California since 1987, that is, since publication of the DSM-III-R. Diagnostic standards were changed in 1994 (DSM-IV), 1998, 2000 (DSM-IV-TR) and 2001–02. The 2000 DSM-IV-TR changed the criteria for pervasive developmental disorder—not otherwise specified (PDD-NOS), which could influence autism diagnoses, though the revision did not specifically address the diagnostic criteria for autism. The 1998, 2001 and 2002 revisions were changes in practices specific to California. Beginning in 1998 and continuing through 2002, a series of changes in diagnostic practices were implemented by the DDS. In 1998, the first ‘Best Practices’ guide was published by the Department of Developmental Services and the California Department of Education. In 2001, increasing attention was directed to the diagnostic component of best practices as dictated by Assembly Bill 430 (effective August 2001), which required the DDS to develop evaluation and diagnostic procedures for the diagnosis of autistic disorder, as well as to create a training program for the utilization of the new diagnostic guidelines. The bill was passed on 10 August 2001 and required the DDS to develop evaluation and diagnostic procedures by 1 April 2002 and implement the training program by 1 July 2002. A rough proxy for the complexity of the diagnostic practice is the manual's length—it is 180 pages long.

Throughout this process the diagnostic practices and procedures underlying the autism spectrum as a whole have changed and the boundaries of autism spectrum disorders have blurred. In this article, we examine the increasing prevalence of autistic disorder (DSM 299.00). When references are made to autism spectrum disorders, including Asperger's Disorder and PDD-NOS, we will be explicit. In all other instances, autism refers to autistic disorder.

Methods

Source data

Data for this study come from the California DDS Client Development and Evaluation Report (CDER). The DDS coordinates diagnoses, services and support for persons with developmental disabilities, including MR and autism, living in California. The DDS only provides services to patients with autistic disorder. Services are not available based on a diagnosis of Asperger's disorder, childhood disintegrative disorder, Rett's disorder or PDD-NOS. MR diagnoses are based on standardized age-appropriate general cognitive aptitude tests and adaptive behaviour ratings. Although enrolment with the DDS is optional, it is estimated that 75–80% of persons with autism in California are enrolled.15

A CDER is maintained for all persons enrolled with the DDS. The DDS is unique in that it maintains computerized records on all of its clients. The CDER consists of two portions, a diagnostic element and an evaluation element. The diagnostic element contains information on a client's diagnosis, the level of severity of the primary disability, and potential etiological information. The evaluation element describes the client's level of functioning in five areas: motor skills, independent living, social interaction, cognition and communication. Client CDERs are required to be updated at least once every 3 years or when clients experience a significant change in state. Over the course of this study, 80% of cases were reviewed annually. CDER updates may or may not include a review of diagnosis but serve as a regular point of contact with a clinician who has the discretion to initiate a diagnostic review.

Study population

To examine the effect of changes in diagnostic standards on autism prevalence rates, we examined computerized DDS records of all clients with a CDER on file between 1992 and 2005. All clients were screened by trained diagnosticians upon entry to the system, as well as at repeated intervals. From the DDS records, we selected the 7003 clients born before 1987 who at any point in their case history received a diagnosis of autism, indicated by an autism code 1 (full syndrome) or code 2 (residual state) on the CDER. The study population was restricted to persons born before 1987, so we could observe them throughout the series of diagnostic changes that began to occur in 1994. Restricting our observations based on birth year had two advantages: (i) clients were all born into the DSM III-R diagnostic regime; and (ii) were at least 8 years of age in 1994 and therefore should have been diagnosed before the implementation of the DSM IV if they met the criteria under the previous diagnostic regime. Additionally, because of their age, patients’ conditions should be relatively stable and ascertainment bias should be minimal. In sum, our study examines all individuals with autism in the DDS system >8 years of age in 1994, and included 631 individuals who entered the DDS system with a sole diagnosis of MR but later received a diagnosis of autism.

Measures

Outcome measure

The CDER contains diagnostic information on all individuals in the DDS system. From the CDER we ascertained whether patients experienced a change in diagnostic status. Occasionally patients diagnosed with autism lost their diagnosis, but this only occurred in 89 cases. Thus, clients were roughly seven times more likely to acquire than lose an autism diagnosis. We modelled change in diagnosis from a sole diagnosis of MR to a diagnosis of autism or autism with MR co-morbidity. In 95% of cases, clients acquired an autism diagnosis, making their diagnosis autism–MR. Only 5% of clients experienced pure diagnostic substitution from a sole diagnosis of MR to a sole diagnosis of autism. By examining when patients experienced diagnostic substitution or accretion—i.e. acquired a diagnosis of autism—we gain a better understanding of the contribution of diagnostic change to increasing autism prevalence. The year of diagnostic change was captured by the date of the CDER first reporting a change in diagnostic status.

Changes in diagnostic practices

To assess the importance of changes in diagnostic standards for diagnostic change, period effects for each year were included. The reference group for the period effects was 2005. By using the last year available in our analyses we captured the year in which there was greatest awareness about the disorder and administrative diagnostic systems had the greatest opportunity to fully develop. By using period effects with a reference year of 2005, we ascertained whether in years in which the DSM or DDS diagnostic standards changed, the probability of observing individual diagnostic changes increased relative to 2005.

Control variables

Using data from the evaluation element of the CDER, we controlled for changes in evaluation relevant for an autism diagnosis including one-on-one interaction patterns, friendship formation and maintenance, repetitive behaviours, participation in group and social activities, whether social behaviours were acceptable, receptive and expressive language skills, and word usage. The measurement details for each of these elements are summarized in Appendix 1 (see IJE online). A lower score on these elements is consistent with the criteria for autism. To develop a composite evaluation score, we summed the scores for each of the elements listed above. The ability to include a measure of patient status is critical for the analysis since it allowed us to observe how changes in diagnostic standards net of changes in individual conditions related to changes in individual diagnosis. In addition, we included controls for race, institutional status, level of MR, sex and year of birth.

Analyses

We first conduct an empirical analysis to ascertain the likelihood that an individual obtained their diagnosis through diagnostic change. To predict the odds that an average individual in a given year would acquire their diagnosis through diagnostic change, associations are estimated using generalized estimation equations (GEE) with a logit link function. The GEE method is a form of logistic regression for panel data that improves standard error estimates and produces efficient estimates of coefficients in data in which individuals are observed repeatedly.18 An exchangeable correlation matrix was used. While this is a fairly strong assumption, we felt that it was appropriate given the nature of the data and would provide the most conservative estimates. To test the sensitivity of our results to the choice correlation structure, we conducted robustness checks with alternative specifications. The robustness checks produced identical results. We first calculated the odds of change with period effects only. Then the odds of change were adjusted for other potentially relevant variables, including evaluation scores. The period effects allow us to examine whether individuals were more likely to obtain their diagnosis through diagnostic change (for the vast majority of instances, through accretion) in years in which diagnostic criteria changed.

We then engage in a thought experiment designed to answer the question: if we could observe these effects filtering through the population born after 1987, what would be the cumulative effect of diagnostic change? To do so, we use a prediction equation to generalize the results for the population born before 1987. For the population born after changes in diagnostic criteria are underway, it is impossible to directly observe the effect that changes in criteria may have had, since persons who enter with an autism diagnosis are unlikely to be administered a standardized intelligence-quotient test, the foundation of an MR diagnosis. This is particularly true for children. Accordingly, directly estimating the effect of diagnostic change on the MR pathway from the population that enter with an autism diagnosis is impossible. Thus, we used a small portion of the MR population as a proxy for the unobserved population that would have been at risk for diagnostic change along the MR pathway. Thus, this calculation rests on the assumption that the processes that applied to a certain portion of the MR population born before 1987 would apply to a similar proportion of the population that do not enter the system until later. [Details of the calculation can be found in Appendix 2 (see IJE online.)] To model this process, we use the following equation:

equation image

Here the increase in caseload due to diagnostic change is calculated by multiplying the hypothetical population at risk for diagnostic change in each period by the probability of change in that period and summing across the years. The probability of change (pΔ) is derived from the prediction equation taking into account the period effect coefficients. In order to ascertain the increase in caseload due only to diagnostic change, we net out the influence of other factors in our analysis that might affect the probability of change, such as evaluation scores, by taking into account the non-period effect coefficients in our prediction equation (pΔtpΔtb), where pΔt is the probability of change including the period effects and pΔtb are the effects of all other variables, except the period effects. The subscript is used to designate the year. Summing across the years gives us the predicted increase in caseload due to changing diagnostic standards and practices.

The hypothetical population at risk for diagnostic change is calculated from the MR, autism and co-morbid caseloads reported by the DDS. In greater detail, the rate of co-morbidity at the last date in which a change in practices occurred (Aut2003|MR2003) was multiplied by the total population of persons with MR not yet diagnosed with autism (MR*) yielding an estimate of the population at risk for receiving an autism diagnosis at any given point in time. In 2003, after all changes in diagnostic practices had occurred, ~8% of the MR population had a diagnosis of autism (Aut2003|MR2003). Assuming changes in diagnostic practices now capture previously under-ascertained cases, the rate of co-morbidity under the most recent, and presumably most accurate, system of practices should more precisely reflect the true rate of co-morbidity in the population. Thus, an 8% rate of co-morbidity is used in our calculation of the size of the hypothetical population that would have previously been diagnosed with MR but now comes in directly with an autism diagnosis. The results of our experiment are influenced by the assumed rate of co-morbidity, which is unobservable. To the extent that it is too high or too low, we over- or underestimate the caseload that is attributable to diagnostic change. We later test the effect of this assumption on our results.

Since diagnostic practices change over time, the risk of receiving a diagnosis of autism changes over time as well. Seen positively, diagnostic change means that individuals’ true diagnostic state is uncovered as the practices expand to capture them. To account for the differential rate of case identification due to changing diagnostic practices, we multiplied the probability of change in a given year, which is derived from the prediction equation using the GEE estimates by the number of unrealized co-morbid cases. The unrealized co-morbid population is the number of co-morbid cases that will eventually be diagnosed but have not been captured at a given time and is calculated by multiplying (Aut|MR) × (MR*).

Results

Case records from the California DDS for the years 1992 until 2005 revealed that 25% (7003/27 697) of cases of autism occurred in patients born before 1987. Among this population, 9% (631/7003) of cases arose from diagnostic accretion or substitution. This figure accords with a medical record review of 75 children born between 1983 and 1985 in the DDS system, which found that 10% of children with a sole diagnosis of MR ‘qualified’ for a diagnosis of autism under new standards.15

Figure 1 reports the frequency of diagnostic accretion and substitution by year. The vertical arrows denote years in which the practices for diagnosing autism changed. A cursory inspection of Figure 1 suggests that in periods in which the diagnostic practices for autism changed, changes in diagnostic status occurred with greater frequency. Changes were most frequent between 1998 and 2003, the periods surrounding numerous changes in diagnostic practices in California. The cumulative probability of change over the period was 14.5%.

Figure 1
Number of diagnostic changes (accretions or substitutions) by year. Vertical arrows denote years in which diagnostic practices change

In Table 1, the unadjusted odds confirm the insight arising from visual inspection of Figure 1. The odds of change in years in which diagnostic practices change are significantly different—and higher—from the reference year. Controlling for changes in individual evaluation, sex, race, level of MR, aetiology of MR, institutional status and birth year, the adjusted odds ratios (ORs) in years in which the DSM diagnostic standards changed were 1.68 in 1994 (DSM-IV) and 1.58 in 2000 (DSM-IV TR). For the years in which the California diagnostic practices changed, we observe comparable increased odds of changed diagnosis, specifically, 1.55 (1998), 1.82 (2001) and 1.62 (2002). In the years surrounding the period of diagnostic change, 1999 and 2003, the odds of changes were also elevated, suggesting a modest lag between implementation of changes in diagnostic standards and changes in diagnostic status.

Table 1
Results of GEE estimation of odds of experiencing diagnostic change (accretion or substitution)

Examining the control variables, we see that the level of intellectual impairment of clients had a significant effect on the likelihood of observing diagnostic change. The relationship between severity and the odds of change appears to be non-linear with moderate and profound severity to be at greatest risk for diagnostic change. Changes in evaluation scores, which capture many of the requirements for an autism diagnosis, surprisingly had little discernable effect on the likelihood of diagnostic change [OR 1.02; 95% confidence interval (CI) 1.00–1.04]. Finally, race and year of birth were also significantly associated with the odds of change. Persons born in later years, who were younger, were more likely to experience diagnostic accretion or substitution. Finally, African–Americans were considerably less likely than Caucasians to have a change in diagnostic status.

Using a prediction equation, we ascertained the cumulative probability of an individual acquiring an autism diagnosis in a given year. The cumulative effect of changes in diagnostic practices is reported in Figure 2, which graphically represents the parameter estimates obtained from the GEE estimation reported in Table 1.

Figure 2
The cumulative adjusted probability of an average individual experiencing a change in diagnosis in a given year. The baseline probability, which takes into account changes in the underlying population distribution, is represented by the line marked with ...

The line at the bottom of Figure 2 marked with a triangle reports the predicted cumulative probability of change for an average person, not allowing for changes in diagnostic practices but adjusting for changes in the underlying population distribution (adjusted odds = 0.003–0.004). While diagnosis is relatively fixed in the absence of macro level diagnostic changes, there is still always some room for mobility, but it is very muted. In contrast, the solid line reports the cumulative probability of change for an average person allowing for the effects of changes in diagnostic practices as well as changes in the underlying population distribution. The probability of observing a change in diagnosis, as marked by the line with an asterisk, increases steeply to 0.13 once these cumulative effects are taken into account.

The effect of changing practices on caseload

We now turn to the effect of changing diagnostic practices on the overall autism caseload. Figure 3, based on the observed data for MR and autism populations in the dataset, illustrates changes that occurred in the autism and MR populations. As before, the focus remains on the pathway from MR to autism. The population with MR in California (and elsewhere) is significantly larger than the population with autism. Over time, the proportion of MR cases with autism increases. The resulting increased co-morbidity of autism among individuals diagnosed with MR has a substantial effect on the overall autism caseload. Figure 3 schematically represents these processes. Each circle is scaled to the observed number of cases. Thus, comparing 1992 with 2005, one can observe that the number of cases of autism and MR both expand, although the number of autism cases increased at a much more rapid rate. Simultaneously, the rate of co-morbidity of MR within the autism population decreased (from 0.73 to 0.37), though the number of co-morbid cases increased, as did co-morbidity within the population with MR (Aut|MR).

Figure 3
The changing diagnostic world in California between 1992 and 2005. The number of cases of autism and MR both expand, though the proportion of autism cases is increasing at a much greater rate. As this is happening, the rate of co-morbidity within the ...

Figure 4 illustrates a theoretical experiment designed to show what proportion of the autism caseload might be attributed to changing diagnostic criteria if we had been able to observe these changes taking place. This figure takes the results from the population born before 1987 and generalizes to the total population with autism. This is necessary since persons born after 1987 enter into the system as diagnostic changes are underway. From Figure 4 we can observe diagnostic change thus accounted for 26.4% (95% CI 16.23–36.48) of the increased autism caseload.

Figure 4
Autism caseload increase in California between 1992 and 2005. Shaded in grey is the observed increase in DDS autism cases. In black is the proportion of that increase that is due to diagnostic change. 95% CIs are denoted by white bars

These estimates take into account the effect of changing diagnostic practices on the population that may have previously been diagnosed with MR under prior diagnostic regimes.

Robustness checks

Our estimate is sensitive to the choice of the level of co-morbidity (Aut|MR) since the actual co-morbidity of autism within the MR population is not directly observable due to ascertainment bias. The observed co-morbidity in our data varies between 0.05 and 0.09. If the level of co-morbidity is set at the actual level of co-morbidity in any given year (Autt|MRt), rather than at the level observed after all changes in diagnostic practices had taken place (Autt13|MRt13), our results are virtually unaltered. Using the observed changing levels of co-morbidity, 25.9% of the caseload increase is accounted for by diagnostic change. Assigning a level of co-morbidity of 5 and 10%, to generate lower and upper bounds, produces estimates that account for 16.9 and 33.82% of the caseload increase, respectively. This provides further support for the idea that roughly one in four cases of autism are the product of diagnostic change operating on the MR pathway.

The appropriate population at risk for a change in diagnosis are MR patients who eventually received an autism diagnosis, since this population expressed symptoms consistent with an autism diagnosis under 2005 diagnostic standards and practices. However, using a broader conception of who is at risk for an autism diagnosis, one could argue that anyone with MR may be at risk for a change in diagnosis since differential diagnosis is difficult.19 This would assume that autism and MR diagnoses are interchangeable, which is not the case. However, since our analysis requires us to make several assumptions about the population at risk, we wanted to test the validity of our assumptions by conducting a supplementary analysis. To test whether the definition of the population at risk had any effect on our results, we conducted a supplementary analysis examining the probability of change from a sole MR diagnosis to autism or autism–MR diagnosis using the entire MR population. We then used the probabilities of change obtained from the GEE in a prediction equation to estimate the caseload increase arising from diagnostic change along the MR pathway. The results of this supplementary analysis, which appear in Appendix 3 (see IJE online), are similar to the results obtained using the population with autism. The probability of observing a change in diagnosis was heightened in years in which the criteria for diagnosing autism changed. Collectively, these changes in criteria could account for close to one-third of the observed increase in caseload (7410/22 767). These supplementary analyses confirmed the importance of changes in diagnostic practices for changes in diagnostic status.

Discussion

This study makes two different contributions. Most critically, using empirical data, we examined the likelihood that an individual obtained their autism diagnosis through diagnostic change. We found that in years in which the criteria for diagnosing autism change, it was more likely that an individual would obtain their diagnosis through diagnostic change. In the second part of the article, we engage in a thought experiment to assess the potential impact that diagnostic changes may have had as they filtered through the population. We estimate that one-quarter of the increase in measured autism prevalence is the result of diagnostic change. This of course leaves the remaining three-quarters unexplained. This study, however, has several limitations.

While diagnostic change likely affects prevalence rates throughout the USA, our estimate of the effect size of these process is limited to the state of California. California has an exceptionally well-developed service delivery programs and extensive diagnostic resources. The state’s diagnostic capacity, which includes continual re-evaluation of patients with developmental disabilities, could lead to higher rates of diagnostic change than would be observed elsewhere. Moreover, much of the diagnostic change modelled in this study is the result of state-specific policies. Arguably, the state’s vigilance at identifying autism cases should produce an upper bound estimate of the effect of diagnostic change. Selection issues may also be pushing our estimate upwards. It is estimated that at least 75–80% of the persons with autism in California are receiving services from the DDS.15 If the other 25% are less impaired, particularly less cognitively impaired, the resultant selection bias would lead to an overestimation of the influence of diagnostic change on the MR pathway. However, several studies have argued that diagnostic substitution is not occurring in California, though it accounts for much of the increased prevalence in other states. Against this background, California is a conservative test case. These countervailing tendencies make generalizing beyond the state difficult.

Furthermore, many of the changes in diagnostic practices have applied to the higher functioning end of the spectrum, particularly with the addition of Aspergers to the DSM in 1994. However, the DDS does not serve patients with a sole diagnosis of PPD-NOS or Aspergers. Thus, our attempts to produce an estimate of the effect of changing diagnostic practices applicable to the non-cognitively impaired portions of the spectrum are limited. Diagnostic substitution and accretion along the MR pathway is likely contributing to the lower functioning portion of the autism spectrum. Diagnostic substitution and diagnostic accretion along other pathways, such as developmental language disorder or other learning disabilities, may be contributing to an increase in higher functioning cases. In a study applying contemporary diagnostic standards and practices to persons with a history of developmental language disorder 21% (8/38) of the individuals met the criteria for autism and 11% (4/38) met the criteria for milder forms of ASD.5 Thus, there are multiple pathways to an autism diagnosis from multiple disorders that contribute to increases along various parts of the spectrum. In this article, we have considered only one pathway and one part of the spectrum.

Our ability to ascertain the increased risk associated with changes in practices was limited to the use of period effects. As in all analyses, period effects are sensitive to the choice of reference year. The frequency of diagnostic changes reported in Figure 1 provides an indication of the overall distribution of changes. Our analysis is, of course, sensitive to this distribution. However, at this basic level, the frequencies indicate that the likelihood of an individual acquiring an autism diagnosis was elevated in years in which the diagnostic practices for autism change. Period effects are also imperfect measures of what they are attempting to capture. Here, we use the year in which diagnostic practices change, as a proxy for the changing practices. Ideally, we would have had a more refined measure of the diagnostic changes.

Finally, our estimate of the increase in caseload attributable to diagnostic change uses the results from one population, persons born before 1987, to generalize to later cohorts. This is necessary since we cannot observe the entire diagnostic history of later birth cohorts. To the extent that the childhood population may differ from the adult population, caution is warranted in interpreting the estimated increase in caseload. Two additional caveats about this experiment may be warranted. First, the ‘true’ rate of autism in the MR population is unobservable. In our projection, we assume it to be 8%. To the extent that this is too high (or too low) we overestimate (underestimate) the impact of diagnostic change on caseload. However, the positive association between birth year and diagnostic change assuages our concerns about this assumption, as does the considerable number of instances of diagnostic accretion among later birth cohorts (n = 1101) despite the uneven and limited exposure to diagnostic change faced by these cohorts. As would be expected, given the censoring in this population, the likelihood of diagnostic accretion and substitution increase over time.

Taking these limitations into consideration, we have estimated that 26.4% (95% CI 16.25–36.48) of the increased autism caseload in California can be attributed to the effects of changing diagnostic practices and diagnostic accretion and substitution, which we refer to as diagnostic change. Diagnostic accretion is similar to diagnostic substitution in that persons who would have historically been included in one diagnostic category are now included in a different or additional diagnostic category. Both processes can have a substantial impact on caseload. Our findings are consistent with a recent study using data from California by Hertz-Picciotto and Delwiche which found that changes in diagnostic criteria may account for as much as one-third of the increased autism prevalence.14

We have estimated that one in four children who are diagnosed with autism today would not have been diagnosed with autism in 1993. This finding does not rule out the possible contributions of other etiological factors, including environmental toxins, genetics or their interaction to the increased prevalence of autism. In fact, it helps us to recognize that such factors surely play an important role in increasing prevalence. There is no reason to believe that any of these frameworks are wrong and many reasons to believe that the increase in autism prevalence is in fact the outcome of multiple self-reinforcing processes. However, this study demonstrates that subsequent explanations for the increased prevalence of autism must take into account the effect of diagnostic change.

Supplementary data

Supplementary data are available at IJE online.

Funding

NIH Director's Pioneer Award program, part of the NIH Roadmap for Medical Research, through grant number 1 DP1 OD003635-01; Robert Wood Johnson Health and Society Fellows Program and the Institute for Social and Economic Research and Policy at Columbia University.

Acknowledgements

The authors thank Lisa Bates, Christine Fountain, Gil Eyal, Julien Teitler and the anonymous reviewers for helpful comments on earlier drafts.

Conflict of interest: None declared.

References

1. Fommbonne E. The prevalence of autism. JAMA. 2003;289:87–89. [PubMed]
2. Services CDoD. Autistic Spectrum Disorders: Changes in the California Caseload: An Update: 1999 through 2002. Sacramento, California: Department of Health and Human Services Agency, Department of Developmental Services; 2003.
3. Coo H, Ouellette-Kuntz H, Lloyd J, Kasmara L, Holden J, Lewis S. Trends in autism prevalence: diagnostic substitution revisited. J Autism Dev Disorders. 2008;38:1036–46. [PubMed]
4. Gernsbacher MA, Dawson M, Goldsmith HH. Three reasons not to believe in an autism epidemic. Curr Dir Pyschol Sci. 2005;14:55–58.
5. Bishop D, Whitehouse A, Watt H, Line E. Autism and diagnostic substitution: evidence from a study of adults with a history of developmental language disorder. Dev Med Child Neurol. 2008;50:341–45. [PubMed]
6. Rutter M. Incidence of autism spectrum disorders: changes over time and their meaning. Acta Pediatr. 2005;94:2–15. [PubMed]
7. Shattuck P. Contribution of diagnostic substitution to the growing administrative prevalence of autism. Pediatrics. 2006;117:1028–37. [PubMed]
8. Wing L, Potter D. The epidemiology of autism spectrum disorders: is prevalence rising. Mental Retard Dev Disab Res Rev. 2002;8:151–61. [PubMed]
9. Gurney J, Fritz M, Ness K, Sievers P, Newschaffer C, Shapiro E. Analysis of prevalence trends of autism spectrum disorders in Minnesota. Arch Pediatr Adolesc Med. 2003;157:622–27. [PubMed]
10. Newschaffer C, Falb M, Gurney J. National autism prevalence trends from United States special education data. Pediatrics. 2005;115:277–82. [PubMed]
11. Control CfD. [March 2007, date last accessed]. Changes in Autism Diagnostic Criteria. http://www.cdc.gov/ncbddd/autism/overview_changes_diagnostic.htm.
12. Fombonne E. Is there an autism epidemic. Pediatrics. 2001;107:411–12. [PubMed]
13. Barbaresi W, Katusic S, Colligan R, Weaver A, Jacobsen S. The incidence of autism in Olmstead County, Minnesota, 1967–1997: results from a population based study. Arch Pediatr Adolesc Med. 2005;159:37–44. [PubMed]
14. Hertz-Picciotto I, Delwiche L. The rise in autism and the role of age at diagnosis. Epidemiology. 2009;20:84–90. [PubMed]
15. Croen LA, Grether JK, Hoogstrate J, Selvin S. The changing prevalence of autism in California. J Autism Dev Disord. 2002;32:207–15. [PubMed]
16. Blaxill M, Baskin D, Spitzer W. Commentary: Blaxill, Baskin, and Spitzer on Croen et al. (2002), the changing prevalence of autism in California. J Autism Dev Disord. 2003;33:223–26. [PubMed]
17. Croen L, Grether J. Response: a response to Blaxill, Baskin, and Spitzer on Croen et al. (2002), 'the changing prevalence of autism in California'. J Autism Dev Disord. 2003;33:227–29. [PubMed]
18. Zeger S, Liang K-Y. Longitudinal data analysis for discrete and continuous outcomes. Biometrics. 42:121–30. [PubMed]
19. Association AP. Diagnostic and Statistical Manual of Mental Disorders, DSM IV. Washington, DC: American Psychiatric Publishing; 1994.

Articles from International Journal of Epidemiology are provided here courtesy of Oxford University Press