PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
J Am Acad Child Adolesc Psychiatry. Author manuscript; available in PMC 2012 March 1.
Published in final edited form as:
PMCID: PMC3049293
NIHMSID: NIHMS261148

Cumulative Prevalence of Psychiatric Disorders by Young Adulthood: A Prospective Cohort Analysis from the Great Smoky Mountain Study

Abstract

Objective

No longitudinal studies beginning in childhood have estimated the cumulative prevalence of psychiatric illness from childhood into young adulthood. The objective of this study was to estimate the cumulative prevalence of psychiatric disorders by young adulthood and to assess how inclusion of not otherwise specified (NOS) diagnoses affects cumulative prevalence estimates.

Method

The prospective, population-based Great Smoky Mountains Study assessed 1420 participants up to 9 times between ages 9 and 21 from 11 counties in southeastern US. Common psychiatric disorders were assessed in childhood and adolescence (ages 9 to 16) with the Child and Adolescent Psychiatric Assessment, and in young adulthood (ages 19 and 21) with the Young Adult Psychiatric Assessment. Cumulative prevalence estimates were derived from multiple imputed datasets.

Results

By age 21, 61.1% of participants had met criteria for a well-specified psychiatric disorder. An additional 21.4% had met criteria for an NOS disorder only, increasing the total cumulative prevalence for any disorder to 82.5%. Males had higher rates of both substance and disruptive behavior disorders compared to females; therefore they were more likely to meet criteria for either a well-specified disorder (67.8 vs. 56.7%) or any disorder (89.1 vs. 77.8%). Children with an NOS disorder only were at increased risk for a well-specified young adult disorder compared to children with no disorder in childhood.

Conclusions

Only a small percentage of young people meet criteria for a DSM disorder at any given time, but most do by young adulthood. As with other medical illness, psychiatric illness is a nearly universal experience.

Keywords: epidemiology, prevalence, NOS disorders, psychiatric disorders

Introduction

The modern era of psychiatric epidemiology was ushered in by the development of highly specified operational criteria for the diagnosis of psychiatric disorders1-3 and structured diagnostic interviews specifically designed for use with both adults and children4, 5. Use of these methods in representative population samples has suggested that the annual prevalence of psychiatric disorders is around 30% in adults under 55 and projects a lifetime prevalence rate for psychiatric disorder by age 75 around 50%6.

Recently it has been recognized that the onsets of many disorders occur in childhood and adolescence. In the National Comorbidity Survey replication approximately half of all lifetime cases reported onsets by age 14, and three quarters by age 246. By extension, prospective studies of children and adolescents should find lifetime prevalence of psychiatric disorders to be around 25% by the mid teens, but they do not. Rather, the reported cumulative prevalence rates have been close to 40% by age 14-16 for studies that employ multiple prospective assessments7, 8. Furthermore, among young adults (ages 18-26) with psychiatric disorders, 78% had received a diagnosis before age 18, and 58% before age 159. Differences in prevalence rates of psychiatric disorders when using retrospective versus prospective data were recently illustrated in a study that derived prospective rates from the Dunedin Multidisciplinary Health and Development study, ages 18 to 32, and retrospective rates derived from three studies10. Across four disorder groups, the prevalence rates were approximately doubled in prospective as compared to retrospective reports. It is not surprising that prospective studies provide higher estimates for the early onset of psychiatric disorders than do cross-sectional studies of adults that use retrospective recall. People are highly prone to forget even high intensity medical events, 11 and recall of the timing of onset psychiatric symptoms is very imprecise12, with a tendency to remember events as having occurred later than they actually did13.

To date, prospective longitudinal studies have not provided cumulative estimates of the rates of lifetime psychiatric disorders from childhood through young adulthood. The data provided by Kim-Cohen et al. suggest that approximately half of the total number of their lifetime cases of disorder were first diagnosed after the age of 15. If the rate by mid adolescence were close to 40%, as reported in 8, that would imply that by age 26 80% of the sample had already suffered from a psychiatric disorder. Can such figures be correct? Our first aim is to answer this question.

Even estimates from prospective studies may be underestimates, because these studies typically only considered well-specified DSM diagnoses (e.g., major depressive disorder, antisocial personality disorder), despite the fact that many individuals with significantly impairing psychiatric symptoms present with “subthreshold” or mixed clinical pictures that are not well-captured by DSM diagnoses14, 15. For this reason the DSM-IV provides Not Otherwise Specified (NOS) categories which permit individuals who present with “subthreshold” or mixed clinical pictures to be classified according to their predominant symptomatology. There is no general agreement, however, on the rules for diagnosing NOS disorders. Consequently, NOS cases are rarely considered in epidemiological studies, although both the DSM and clinicians regard them as having psychiatric disorders. As many as half of clinically referred child psychiatric patients do not meet criteria for a well-specified DSM diagnosis, but have significant symptomatology and disability16-19, so this is not a minor problem.

Several years ago we coined the term “symptomatic impairment” to describe individuals with significant psychosocial impairment resulting from psychiatric symptoms who could reasonably be regarded as suffering from NOS disorders14. We found that the 3-month rate of symptomatic impairment was 9.4% in a representative community sample of 9-13 year olds, and that, on a variety of “caseness” measures (e.g., service use, risk for later diagnosis), these children deserved to be regarded as suffering from a psychiatric disorder. The second aim of this study is to examine how the inclusion of individuals with symptomatic impairment or NOS disorders affects estimates of cumulative prevalence by young adulthood.

Method

Participants

The Great Smoky Mountains Study (GSMS) is a longitudinal study of the development of psychiatric disorders and need for mental health services in rural and urban youth7, 20. A flowchart detailing sample recruitment is provided in Figure 1. A representative sample of three cohorts of children, age 9, 11, and 13 at intake, was recruited from 11 counties in western North Carolina. Potential participants were selected from the population of some 12,000 children using a household equal probability, accelerated cohort design. Youth with behavior problems were over sampled. A screening questionnaire was administered to a parent (usually the mother) of the first stage sample (N=3,896). The questionnaire consisted mainly of the externalizing (behavioral) problems scale of the Child Behavior Checklist21, because evidence from previous studies suggested that parental reports of behavioral problems best predict most types of psychiatric symptoms and service use 22, 23. All children scoring above a predetermined cutpoint (the top 25% of the total scores), plus a 1 - in - 10 random sample of the rest (i.e., the remaining 75% of the total scores), were recruited for detailed interviews. Ninety-five percent of families contacted completed the telephone screen. All participants were given a weight inversely proportional to their probability of selection. Consequently, the results presented are unbiased by the oversampling strategy and representative of the population from which the sample was drawn. Additional information about sampling or screening methodology is available from 24.

Figure 1
Flowchart of study recruitment.

About 8% of the area residents and the sample are African American, and fewer than 1% are Hispanic. American Indians make up only about 3% of the population of the study area, but because they are an understudied group they were oversampled from school records to constitute 25% of the study sample. This was done by using the same screening procedure but recruiting everyone irrespective of screen score. Of the 456 Indian children identified, screens were obtained on 96%, and 81% (N=350) participated in the study.

Measures

Psychiatric disorders were assessed using 1) the Child and Adolescent Psychiatric Assessment (CAPA)24 until age 16, and 2) the upward extension of the CAPA, the Young Adult Psychiatric Assessment (YAPA)25 at ages 19 and 21. Both the CAPA and YAPA are interviewer-based interviews. The goal of such interviews is to combine the advantage of clinical interviews with those of highly structured “epidemiologic” interview methods. All CAPA interviews were coded by a trained interviewer and each interview was then checked by a supervisor. A detailed glossary provides the operational rules for identifying clinically significant symptoms. Scoring programs for the CAPA and YAPA, written in SAS by the senior authors, combined information about the date of onset, duration, and intensity of each symptom to create diagnoses according to the fourth edition of the Diagnostic and Statistical Manual of the American Psychiatric Association. A symptom was counted as present if reported by either parent or child or both, as is standard in child and adolescent epidemiological studies, approximating the process of combining information from multiple informants in clinical practice. The time frame for determining the presence of most psychiatric symptoms was the past three months. Onset dates were assessed for all symptoms to allow for diagnosis when the DSM-IV criteria required duration of greater than 3 months.

Two-week test-retest reliability of CAPA diagnoses in children aged 10 through 18 is comparable to that of other highly-structured child psychiatric interviews25. Construct validity as judged by 10 different criteria including comparison to other interviews and ability to predict mental health service use is good to excellent26. Both a parent and the participating child were interviewed at ages 9-16; at and after age 19 only the participant was interviewed. Common psychiatric disorders used to derived cumulative prevalence estimates included anxiety disorders (separation anxiety, generalized anxiety, social phobia, specific phobia, agoraphobia, panic disorder, obsessive-compulsive disorder, and posttraumatic stress disorder), mood disorders (major depression, dysthymia, mania, and hypomania), behavioral disorders (conduct disorder, oppositional defiant disorder, attention-deficit/hyperactivity disorder, and antisocial personality disorder), and substance disorders (meeting DSM-IV abuse or dependence criteria for nicotine (dependence only), alcohol, cannabis, amphetamines, opioids, cocaine, hallucinogens, inhalants, or sedatives). Personality disorders other than antisocial personality disorder were not assessed.

Psychosocial impairment secondary to psychiatric symptomatology in 17 areas of functioning related to life at home, at school and elsewhere was also rated according to a series of definitions and rules specified in the CAPA glossary and the interview schedule. In general, some decrement in actual function had to be described for a positive rating to be given (see 27, for a full description of the concept of impairment implemented in the CAPA). Briefly, having completed the symptom part of the interview, the interviewer reviews with the subject the areas of positive symptomatology, which are divided into 19 symptom groups (e.g., ADHD symptoms or depressive symptoms). For each of these areas the subject is questioned about whether those symptoms have resulted in any of the 17 impairment categories (each impairment category is reviewed separately). NOS status or symptomatic impairment was met when individuals displayed psychosocial impairment resulting from psychiatric symptoms but did not meet diagnostic criteria. A single symptom might be the basis for an impairment coding related to that symptom's group, but this was uncommon. The average number of DSM-IV symptoms for those displaying symptomatic impairment but not meeting criteria for a formal diagnosis was 4.56 (SD=3.88).

The test-retest intraclass correlation coefficient for level of psychosocial impairment by child self-report was 0.77 (Angold and Costello, 1995). In a previous publication14, children with symptomatic impairment were elevated on the following markers of “caseness” when compared to children with no diagnosis: 1) rates of specialist mental health service use; 2) school-based mental health-related service use; 3) parental perceived burden; 4) perception by parent of child that the child had a problem; 5) parent or child perception of need for help, and 6) higher risk for a psychiatric diagnosis or further impairment one year later.

Procedure

Interviewers were residents of the study area. All had at least bachelor's level degrees. They received one month of training and constant quality control. Interviewers were trained by Department of Social Services staff in the State's requirements for reporting abuse or neglect, and all suspected cases were referred to the appropriate agency. Families were visited by two interviewers, either at home or in a location convenient for them. Before the interviews began, parents and adult participants signed informed consent forms; informed assent forms were signed by minor children. Approval for this protocol was obtained from the Duke University Medical Center Institutional Review Board.

Missing Data

Table 1 shows the assessment schedule for each of the three cohorts up to age 21. Of a total of 11168 total possible observations, 8806 were completed (81.9%, range 74% to 94% at any particular wave). In the GSMS, all participants contributed at least one first observation with over 75% of participants contributing at least 5 assessments and more than half of participants contributing at least 7 observations (median = 7; mean = 6.3, SD=2.1; range for completed assessments: 1 to 9). If missing observations were more or less likely to have a psychiatric disorder, prevalence estimates would be biased. To test for such effects, each individual's total number of missed assessments adjusted for the total number of expected assessments was regressed on the individual's psychiatric status at their first assessment. Initial psychiatric status was unrelated to later missingness (OR=1.1, 95%CI 0.98, 1.2, p=.12).

Table 1
Data collection by cohort in the Great Smoky Mountain Study: Number of subjects interviewed and participation rates

Even missing-at-random observations, however, bias cumulative prevalence estimates. Having had more observations was associated with higher cumulative prevalence estimates in the GSMS (OR=1.2, 95%CI 1.1, 1.3, p =.003). Therefore, any missing observation, even if missing-at-random, might lead to an underestimate of cumulative prevalence. To counter this problem, data were imputed using a logistic regression approach (the LOGISTIC option on the MONOTONE statement in SAS PROC MI)28, 29. Diagnostic status including NOS status was predicted by age and the sampling weight which incorporates information about race, sex, cohort, and pre-study screen status on the CBCL externalizing scale20. The resulting logistic regression equation generates a probability for each case with missing data. A Bernoulli draw is made for that probability, producing imputed values of 0 or 1. Twenty-five complete datasets were produced using this method to model the uncertainty of imputed values. Another unbiased approach to imputation of categorical data, discriminant function imputation, was also tested and led to similar results. Results for the discriminant function imputed data sets are available from the first author.

Analyses

Survival functions were generated from weighted Cox proportional hazard models using SAS PROC PHREG. To estimate prevalence, weighted logistic regression models were computed using the generalized estimating equations option in PROC GENMOD. Sampling weights were inversely proportional to selection probability. Robust variance (sandwich type) estimates were used to adjust the standard errors of the parameter estimates for the stratified design effects. Therefore, the resulting prevalence estimates are representative of the population from which the sample was drawn. Because multiple datasets had been imputed, SAS PROC MIANALYZE was used to read parameter estimates and associated covariance matrices for each imputed dataset and to derive valid statistical inferences and estimates30.

Results

Cumulative Prevalence of DSM-IV Disorders by Young Adulthood

Unimputed and imputed cumulative prevalence estimates are presented in Table 2. Unimputed data approximates the unlikely assumption that there would be no cases of psychiatric disorder in missing observations. Even with this conservative bias, close to half of individuals across the three cohorts met criteria for a well-specified psychiatric disorder and another 22.5% displayed an NOS disorder only. Together, over 7 of 10 subjects met criteria for a psychiatric disorder using unimputed data.

Table 2
Cumulative imputed and unimputed rates of psychiatric disorders by age 21

Imputed data approximate the assumption that missing observations would have rates of psychiatric disorders similar to those that were observed in non-missing cases. The cumulative prevalence estimate based on imputed data by age 21 was 61.1% for well-specified disorders. Figure 2 displays the cumulative prevalence for well-specified disorders by age 21 in each of the three study cohorts. Substance disorders were the most common followed by behavioral disorders, anxiety disorders, and mood disorders. An additional 21.4% of participants had met criteria for an NOS disorder only. This brought the cumulative prevalence for any disorder to 82.5% by age 21. Figure 3 displays the cumulative prevalence for any psychiatric disorder in each cohort by age 21. Not surprisingly, the youngest cohort (which had the highest number of possible assessments) had the highest cumulative prevalence.

Figure 2
Cumulative prevalence of meeting criteria for a well-specified psychiatric disorder by age 21 for each of the three Great Smoky Mountain study cohorts using imputed data. Note: All estimates represent results of 3-month assessments competed between ages ...
Figure 3
Cumulative prevalence of meeting criteria for any DSM-IV psychiatric disorder by age 21 for each of the three Great Smoky Mountain study cohorts using imputed data. Note: All estimates represent results of 3-month assessments competed between ages 9 and ...

Males were more likely than females to meet criteria for either a well-specified disorder (67.8 vs. 56.7%, OR=1.6, 95% CI=1.1, 2.4, p =.02) or any disorder (89.1 vs. 77.8%, OR=2.3, 95% CI=1.3, 4.1, p =.003). There were no sex differences in the likelihood of having had an NOS disorder only (male vs. female: 21.2 vs. 20.9%, OR=1.0, 95% CI=0.6, 1.8, p =.95). The sex differences in rates of well-specified disorders was accounted for by higher rates of disruptive behavior (29.8% vs. 16.9%, OR=2.1, 95% CI=1.3, 3.4, p =.004) and substance disorders (49.7% vs. 34.8%, OR=1.9, 95% CI=1.3, 2.8, p =.002) in males. Although cumulative prevalence rates of anxiety and depressive disorders were slightly higher in females, these sex differences were not significant (anxiety: 24.8% vs. 21.1%, OR=1.2, 95% CI=0.8, 1.9, p =.36; depression: 17.5% vs. 12.8%, OR=1.5, 95% CI=0.8, 2.5, p =.18). Depressive disorders were more prevalent in females compared to males after puberty, but more common in boys before puberty. Post-pubertal differences were not sufficiently large to produce an overall sex difference in the cumulative prevalence by age 21.

Prior diagnostic status in young adults with disorders

Inclusion of individuals with an NOS diagnosis only (here, defined as having symptomatic impairment) resulted in very high estimates of cumulative prevalence of psychiatric disorders. Is it reasonable to include such cases in cumulative prevalence estimates? We evaluated the predictive validity of NOS disorders by testing whether children with only an NOS disorder were at increased risk for meeting criteria for a well-specified disorder in young adulthood. The following exclusive groups were derived: participants with no childhood well-specified or NOS diagnosis, an NOS diagnosis only, or a well-specified disorder. The likelihood of having a young adult disorder was 17.9% (95% CI= 11.2, 24.5) for those with no childhood diagnosis, 28.1% (95% CI= 19.6, 36.6) for those with an NOS diagnosis only, and 42.9% (95% CI= 35.3, 50.4) for those with a well-specified disorder. In all groups subjects were most likely to not meet criteria for a disorder in young adulthood. As compared to those with no childhood diagnosis, however, both childhood well-specified diagnosis and those with an NOS disorder only were at increased risk for a young adult disorder (well-specified disorder: OR=3.5, 95% CI=2.0, 5.9, p < .001; NOS disorder only: OR=2.0, 95% CI=1.1, 3.6, p=.04).

Discussion

Estimates of cumulative prevalence of psychiatric disorders from childhood to adulthood have hitherto relied upon retrospective recall, despite evidence that forgetting is common for recall beyond 3-6 months31-33. Even with these concerns, lifetime prevalence estimates from such studies suggest that about half of adults will meet criteria for a DSM-IV disorder by age 756. We assessed psychiatric functioning over a limited time period at each assessment to minimize forgetting and other response biases, but completed up to nine assessments from childhood to young adulthood per person. Prevalence estimates at any 3-month assessment (averaging about 13%) were, if anything, on the low side in comparison with other studies of short-term prevalence34. Despite these modest 3-month prevalence estimates, 3 of 5 individuals had met criteria for a well-specified diagnosis and 4 of 5 met criteria for any diagnosis (including NOS disorders) by our final assessment, and using imputed data at age 21. In the youngest cohort (the cohort with the most assessments over time), the cumulative prevalence for any disorder was above 90%. This suggests that the experience of psychiatric illness is not merely common but nearly universal.

There are several methodological issues that must be considered when evaluating these estimates.

Representativeness

Although the GSMS sample was representative of children in the 11 counties studied, both African Americans and Latinos were underrepresented and Native Americans were overrepresented as compared to their respective proportions in the US population. Comparison of the GSMS to other studies, however, finds similar rates of childhood disorders (up to age 16) to representative samples involving higher levels of Hispanic and African American youth8, 26, 27 and the proportion of children receiving needed mental health care is similar to that in other areas of the US 35-39 and the UK 40. The child poverty rate in the GSMS sample was slightly lower than the national average (18.1% vs. 19.8% in 1996)

Primary Period

Participants and their parents were interviewed about the 3 months immediately preceding the interview to minimize forgetting. Although the participants were followed over the course of 12 years, cases will have been missed because participants may have met criteria for disorders prior to our study, between assessments, or after their last assessment. Even for the participants with 9 observations, information collected covered only about 2 years and 3 months (9 × 3 months) or about 11% of their life-times. Although our cumulative estimates may be accurate regarding the periods assessed, they will likely substantially underestimate true lifetime prevalence, particularly given recent evidence that psychiatric problems may begin much earlier in development than was assessed in this study41.

Taken together, methodological effects on our cumulative prevalence estimates would tend to bias our estimates downward, whereas the striking feature of our data is already how high the rates of early psychiatric disorders are. Most likely, the lifetime prevalence of psychiatric problems by age 21 well exceeds 80%, suggesting that the experience of psychiatric illness is nearly universal.

It is indisputable that close to 100% of people have experienced physical illness many times over by adulthood. As with mental illness, not all physical illnesses are chronic and severe. There are large classes of physical illnesses that are usually relatively mild, transient, but impairing (influenza, varicella, or conjunctivitis), others may be recurrent and impairing, but not life threatening (migraine headaches and back pain), and still other cases may fall on the periphery of established diagnostic thresholds (rheumatoid arthritis, asthma, diabetes, and hypothyroidism). Thus, physical illness involves a continuum with a large number of less severe or transient, albeit impairing, conditions on one end and a limited number of life threatening conditions on the other. Nevertheless, it is often the less severe conditions that, as a result of their common occurrence, account for the greatest costs to society in terms of bed days, work-loss days, productivity losses, and impairment in activities of daily living42, 43. Notably, even putative minor illnesses such as varicella and influenza may result in serious complications or death in vulnerable subgroups and consequently, there are widespread public health efforts to prevent complications even from these relatively mild physical illnesses. Psychiatric illness is no different. Inpatient units, psychiatric hospitals, and intensive treatment programs treat a small subset of cases that are critical or unremitting and severe. The overwhelming majority of individuals with psychiatric problems merits no such intensive care but still experience significant distress and/or impairment. This suggests that diagnostic thresholds, when applied without consideration of NOS disorders, significantly underestimate the public health burden of mental illness.

There remains a concern that regarding individuals with symptomatic impairment as being disordered risks inappropriately diluting the threshold for mental illness, and mislabeling the vicissitudes of normal life as pathology44. However, in a previous study, children with symptomatic impairment were found to be a significant burden on home and school resources, in need of psychiatric attention, and at increased risk of psychiatric problems at 1-year followup14. Here, children with symptomatic impairment in childhood who had never met full criteria for a psychiatric disorder were also at increased risk for a well-specified psychiatric disorder in young adulthood. Together, this evidence makes it difficult to avoid the conclusion that those with symptomatic impairment should be regarded as displaying a psychiatric disorder. The NOS category of disorders is a means to incorporate such individuals into the current taxonomy. We are willing to regard individuals infected with the influenza virus as ill, despite the fact that nearly everyone has had influenza at some time, so why should we demand a higher threshold in the psychological sphere? Indeed, the recognition that, by adulthood, we have almost all suffered a mental illness could go some way towards further destigmatizing psychiatric disorders.

The paradox of our findings is that while most young adults have experienced mental illness at some period in development, relatively few young people are ill at any one time. Mental illness, like physical illness in early life, is both nearly universal over time and relatively uncommon at any one time, and prevalence estimates depend on the time period considered and the thresholds applied. Only about one in three individuals with a well-specified psychiatric disorder received any treatment at all and even when treatment was obtained, it rarely conformed to best practice recommendations18, 19, 35. Thus, despite the almost universal experience of mental illness at some point before adulthood and the common reality of unmet need or insufficient treatment, many people go on to achieve the emotional, behavioral and social milestones of life, just as they do in the physical realm despite the universality of physical illness.

Acknowledgments

The work presented here was supported by the National Institute of Mental Health (MH63970, MH63671, MH48085), the National Institute of Drug Abuse (DA/MH11301), and the William T. Grant Foundation.

Footnotes

Disclosure: Dr. Copeland receives research support from the National Institute of Mental Health and the National Alliance for Research on Schizophrenia and Depression. Dr. Shanahan receives research support from the National Institute of Mental Health. Dr. Angold receives research support from the National Institute of Mental Health, the National Institute on Drug Abuse, and the National Institute of Child Health and Human Development. Dr. Costello and Dr. Angold are co-recipients of the 2009 National Alliance for Research on Schizophrenia and Depression Ruane Prize for Outstanding Child and Adolescent Psychiatric Research.

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

References

1. Robins E, Guze SB. Establishment of diagnostic validity in psychiatric illness: Its application to schizophrenia. American Journal of Psychiatry. 1970;126:107–111. [PubMed]
2. Feighner JP, Robins E, Guze SB, Woodruff RA, Jr, Winokur G, Munoz R. Diagnostic criteria for use in psychiatric research. Archives of General Psychiatry. 1972 Jan;26(1):57–63. [PubMed]
3. Spitzer RL, Endicott J, Robins E. Research diagnostic criteria: Rationale and reliability. Archives of General Psychiatry. 1978;35:773–782. [PubMed]
4. Robins LN. Diagnostic grammar and assessment: Translating criteria into questions. Psychological Medicine. 1989;19:57–68. [PubMed]
5. Wing JK. Measurement and Classification of Psychiatric Symptoms. Oxford: Oxford University Press; 1974.
6. Kessler RC, Berglund P, Demler O, Jin R, Merikangas KR, Walters EE. Lifetime prevalence and age-of-onset distributions of DSM-IV disorders in the national comorbidity survey replication. Archives of General Psychiatry. 2005 June 1;62(6):593–602. [PubMed]
7. Costello EJ, Mustillo S, Erkanli A, Keeler G, Angold A. Prevalence and development of psychiatric disorders in childhood and adolescence. Archives of General Psychiatry. 2003;60:837–844. [PubMed]
8. Jaffee S, Harrington H, Cohen P, Moffitt TE. Cumulative prevalence of psychiatric disorder in youth. Journal of the American Academy Child and Adolescent Psychiatry. 2005;44(5):406–407. [PubMed]
9. Kim-Cohen J, Caspi A, Moffitt T, Harrington H, Milne B, Poulton R. Prior juvenile diagnoses in adults with mental disorder: Developmental follow-back of a prospective-longitudinal cohort. Archives of General Psychiatry. 2003;60:709–717. [PubMed]
10. Moffitt TE, Caspi A, Taylor A, et al. How common are common mental disorders? Evidence that lifetime prevalence rates are doubled by prospective versus retrospective ascertainment. Psychological Medicine. 2009:1–11. First View. [PMC free article] [PubMed]
11. Cannell CF, Marquis KH, Laurent A. A summary of studies of interviewing methodology: 1959-1970. (2).Vital and Health Statistics. 1977:1–78. no.26. [PubMed]
12. Angold A, Erkanli A, Costello EJ, Rutter M. Precision, reliability and accuracy in the dating of symptom onsets in child and adolescent psychopathology. Journal of Child Psychology and Psychiatry. 1996;37:657–664. [PubMed]
13. Pickles A, Neale M, Simonoff E, et al. A simple method for censored age of onset data subject to recall bias: Mother's reports of age of puberty in male twins. Behavior Genetics. 1994;24:457–468. [PubMed]
14. Angold A, Costello EJ, Farmer EMZ, Burns BJ, Erkanli A. Impaired but undiagnosed. Journal of the American Academy of Child and Adolescent Psychiatry. 1999;38:129–137. [PubMed]
15. Lewinsohn PM, Shankman SA, Gau JM, Klein DN. The prevalence of comorbidity of subthreshold psychiatric conditions. Psychological Medicine. 2004;34:613–622. [PubMed]
16. Costello EJ, Shugart MA. Above and below the threshold: Severity of psychiatric symptoms and functional impairment in a pediatric sample. Pediatrics. 1992;90(3 September):359–368. [PubMed]
17. Bickman L, Guthrie PR, Foster EM, et al. Evaluating Managed Mental Health Services: The Fort Bragg Experiment. New York, NY: Plenum Press; 1995.
18. Angold A, Erkanli A, Farmer E, et al. Psychiatric disorder, impairment, and service use in rural African American and White youth. Archives of General Psychiatry. 2002;59:893–901. [PubMed]
19. Farmer E, Burns B, Phillips S, Angold A, Costello E. Pathways into and through mental health services for children and adolescents. Psychiatric Services. 2003;54(1):60–66. [PubMed]
20. Costello EJ, Angold A, Burns B, et al. The Great Smoky Mountains Study of Youth: Goals, designs, methods, and the prevalence of DSM-III-R disorders. Archives of General Psychiatry. 1996;53:1129–1136. [PubMed]
21. Achenbach TM, Ressscorla LA. Manual for the ASEBA School-Age Forms and Profiles. Burlington, VT: University of Vermont Research Center for Children, Youth, & Families; 2001.
22. Achenbach TM, McConaughy SH, Howell CT. Child/adolescent behavioral and emotional problems: Implications of cross-informant correlations for situational specificity. Psychological Bulletin. 1987;101:213–232. [PubMed]
23. Achenbach TM, Edelbrock CS. Behavorial problems and competencies reported by parents of normal and disturbed children aged four through sixteen. Monographs of the Society for Research in Child Development. 1981;46:1–82. [PubMed]
24. Costello EJ, Angold A, Burns BJ, et al. The Great Smoky Mountains Study of Youth: Goals, designs, methods, and the prevalence of DSM-III-R disorders. Archives of General Psychiatry. 1996;53:1129–1136. [PubMed]
25. Angold A, Costello EJ. A test-retest reliability study of child-reported psychiatric symptoms and diagnoses using the Child and Adolescent Psychiatric Assessment (CAPA-C) Psychological Medicine. 1995;25:755–762. [PubMed]
26. Angold A, Costello EJ. The Child and Adolescent Psychiatric Assessment (CAPA) Journal of the American Academy of Child and Adolescent Psychiatry. 2000;39:39–48. [PubMed]
27. Angold A, Prendergast M, Cox A, Harrington R, Simonoff E, Rutter M. The Child and Adolescent Psychiatric Assessment (CAPA) Psychological Medicine. 1995;25:739–753. [PubMed]
28. SAS/STAT® Software: Version 9 [computer program] Cary, NC: SAS Institute, Inc.; 2004. Version.
29. Allison PD. Imputation of Categorical Variables with PROC MI. Paper presented at: SUGI 30; Cary, North Carolina. 2005.
30. Rubin DB. Multiple Imputation for Nonresponse in Surveys. New York, NY: John Wiley and Sons, Inc.; 1987.
31. Angold A, Costello EJ. Toward establishing an empirical basis for the diagnosis of Oppositional Defiant Disorder. Journal of the American Academy of Child and Adolescent Psychiatry. 1996;35:1205–1212. [PubMed]
32. Pillemer DB, White SH. Childhood events recalled by children and adults. Advances in Child Development and Behavior. 1989;21:297–340. [PubMed]
33. Harlow SD, Linet MS. Agreement between questionnaire data and medical records: The evidence for accuracy of recall. American Journal of Epidemiology. 1989;129(2):233–248. [PubMed]
34. Committee on the Prevention of Mental Disorders and Substance Abuse Among Children Y, and Young Adults: Research Advances and Promising Interventions (J Costello, member) Preventing mental, emotional, and behavioral disorders among young people: Progress and possibilities. Washington, DC: National Research Council and Institute of Medicine; 2009. [PubMed]
35. Leaf PJ, Alegria M, Cohen P, et al. Mental health service use in the community and schools: Results from the four-community MECA study. Journal of the American Academy of Child and Adolescent Psychiatry. 1996;35(7):889–897. [PubMed]
36. Leslie DL, Rosenheck RA, Horwitz SM. Patterns of mental health utilization and costs among children in a privately insured population. Health Services Research. 2001;36(1 Pt 1):113–127. [PMC free article] [PubMed]
37. Horwitz SM, Leaf PJ, Leventhal JM, Forsyth B, Speechley KN. Identification and management of psychosocial and developmental problems in community-based, primary care pediatric practices. Pediatrics. 1992;89(3):480–485. [PubMed]
38. Staghezza-Jaramillo B, Bird HR, Gould MS, Canino G. Mental health service utilization among Puerto Rican children ages 4 through 16. Journal of Child and Family Studies. 1995;4(4):399–418.
39. Offord DR, Boyle MH, Szatmari P, et al. Ontario child health study: II. Six-month prevalence of disorder and rates of service utilization. Archives of General Psychiatry. 1987;44:832–836. [PubMed]
40. Vostanis P, Meltzer H, Goodman R, Ford T. Service utilisation by children with conduct disorders. European Child & Adolescent Psychiatry. 2003;12(5):231–238. [PubMed]
41. Egger HL, Angold A. Common emotional and behavioral disorders in preschool children: Presentation, nosology, and epidemiology. Journal of Child Psychiatry and Psychology. 2006;47(3/4):313–337. [PubMed]
42. Goetzel R, Long S, Ozminkowski R, Hawkins K, Wang S, Lynch W. Health, absence, disability, and presenteeism cost estimates of certain physical and mental health conditions affecting U.S. employers. Journal of Occupational and Environmental Medicine. 2004;46(4):398–412. [PubMed]
43. Druss B, Marcus S, Olfson M, Pincus H. The most expensive medical conditions in America. Health Affairs. 2002;21(4):105–111. [PubMed]
44. Parker G. Is depression overdiagnosed? Yes. BMJ. 2007 August 18;335(7615):328. [PMC free article] [PubMed]