PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of jcacapLink to Publisher's site
 
J Can Acad Child Adolesc Psychiatry. 2009 November; 18(4): 287–292.
PMCID: PMC2765380

Language: English | French

Stability/Change of DSM Diagnoses among Children and Adolescents Assessed at a University Hospital: A Cross-sectional Cohort Study

Sassan Ghazan-shahi, MD,1 Nasreen Roberts, MD, MRCPsych (UK), FRCPC,2 and Kevin Parker, PhD2

Abstract

Objectives:

This study’s aim was to examine changes or stability of DSM diagnoses in children and adolescents over the period from childhood to young adulthood and to discuss the instability in DSM diagnoses from a developmental perspective.

Method:

We used cross-sectional cohort design to assess the congruence of DSM diagnoses in children and adolescents who had a diagnostic assessment at least twice as inpatient and/or outpatient at a university hospital from age 5 to 22. Data analysis was conducted using kappa statistics

Results:

The hospital computerized database consisted of 264 patients who were born from 1983 to 1985 and had had a diagnostic assessment at least twice over a 17-year period. The highest percentages of stable cases were of Mood disorders and Psychosis. Behavioural disorders and Anxiety disorders had lower percentages of stable cases but significant Kappa values suggesting fewer cases were stable but also fewer new cases were added. Substance related disorders had very low percentages and non-significant Kappa value. When divided into three groups based on the delay between first and second diagnosis, stability of diagnosis degraded sharply with time.

Conclusions:

The results of this study show poor stability for all diagnoses, however the trend seemed to follow that reported in previous literature where moods disorders and schizophrenia showed more stability than other diagnoses. Explanations are provided for the results. A well-designed prospective longitudinal study utilizing structured diagnostic interviews to assign DSM-IV TR diagnosis from child hood to adulthood would improve the reliability of diagnoses and perhaps time for crystallization of psychopathology and clarification into more discrete diagnostic entities.

Keywords: stability of diagnosis, DSM diagnosis, child and adolescent psychiatry

Résumé

Objectif:

Examiner l’évolution des diagnostics du DSM chez les enfants et les adolescents, de l’enfance au début de l’âge adulte; analyser la stabilité des diagnostics du DSM du point de vue développemental.

Méthodologie:

Nous avons étudié, dans une cohorte transversale, la congruence des diagnostics du DSM chez des enfants et des adolescents âgés de 5 à 22 ans qui ont reçu au moins à deux reprises un diagnostic de TDAH dans un hôpital universitaire, à la suite d’une hospitalisation ou d’une consultation en clinique externe. Les données ont été caractérisées par analyse statistique Kappa.

Résultats:

La base de données informatisée de l’hôpital était constituée de 264 patients nés entre 1983 à 1985; ces patients ont été examinés au moins deux fois sur une durée de 17 ans. Les diagnostics des troubles de l’humeur et de psychose étaient les plus stables. Les diagnostics des troubles du comportement et de l’anxiété étaient moins stables, mais les valeurs de kappa élevées indiquaient une diminution du nombre des cas stables et des nouveaux cas. Les diagnostics de troubles liés à la consommation de substances étaient très peu nombreux, avec une valeur kappa non significative. La stabilité du diagnostic diminuait nettement lorsque les sujets étaient répartis en trois groupes formés en fonction du temps écoulé entre le premier et le second diagnostic.

Conclusion:

Tous les diagnostics sont peu stables; toutefois, la tendance va dans le sens de la littérature qui indique que les troubles de l’humeur et la schizophrénie sont des diagnostics plus stables que les autres. Les résultats font l’objet d’une explication. Une étude longitudinale prospective bien conçue - qui utiliserait les entrevues diagnostiques afin d’établir le diagnostic du DSM-IV TR et recueillerait des données allant de l’enfance au début de l’âge adulte - améliorerait la fiabilité des diagnostics, raccourcirait peut-être le temps de cristallisation de la psychopathologie et clarifierait les valeurs discrètes des diagnostics.

Introduction

Despite the acceptance of moderate and severe psychopathology among children and adolescents, the stability of psychiatric diagnoses in this population is less well established. Diagnostic stability is defined as the degree to which a diagnosis remains constant at subsequent patient assessments (Whitty et al., 2005). Stability of diagnosis is an essential validating feature of a disorder; it provides a basis on which to predict course and outcome, and offers insight into meaningful subdivisions within the disorder leading to improvement of the diagnostic system. More importantly, in children and adolescents, understanding diagnostic stability may help in clarifying the course, developmental changes, and long-term prognoses of psychiatric disorders (Mattanah et al., 1995). Thus research efforts have been directed at improving knowledge by conducting studies of the stability/change of DSM-IIIR/DSM-IV diagnoses over defined periods of time in children and adolescents. These studies have varied in design, sample size, age range and study period. Some have concentrated primarily on inpatient populations and others on community populations. One of the inpatient studies examined the stability of diagnosis across multiple psychiatric hospitalizations over a 9-year period from archival data of 815 children (mean age 13–14 years). A major strength of this study was that estimates of stability were calculated using both positive diagnostic concordance rates and Kappa statistic. The results showed low –moderate stability of diagnosis across episodes for both internalizing and externalizing disorders. Nonetheless the stability was higher for Bipolar disorder and Major depressive disorder, and for schizophrenia as opposed to psychosis NOS. The authors provided a number of explanations for the low stability; symptom-stressor model, wherein specific environmental stressors that vary over time produce different manifestations of psychopathology and greater developmental plasticity may reduce the stability of psychopathology over time (Pettit, 2005). Ten years earlier a 2-year post-discharge follow up study of 70 hospitalized adolescents showed low stability over diagnostic groups as evident in lower kappa values for most diagnoses. Nonetheless the authors stated that studies of stability provide opportunities to examine the effects of developmental change on the manifestation and evolution of psychopathologies in this population (Mattanah, 1995). Unlike these, some studies have examined specific diagnostic groups; one study of inpatients examined personality function and disorders in a group of discharged adolescents over a two year period and concluded that there was low-to-moderate stability of personality dysfunction in adolescents and that personality disorders may improve over time, and can potentially benefit from treatment (Grilo et al., 2001). Reviewing community based studies, a longitudinal community study of 1420 children aged 9 to 13 assessed prevalence and development of DSM-IV disorders annually till age 16. The results showed that some disorders such as depression and substance abuse increased in prevalence, whereas others, including separation anxiety decreased, further that those who had a previous diagnosis had a greater risk of later diagnoses and this risk was higher in girls than boys. Risk of continuity of the same disorder was significant for all disorders except specific phobias. Change from one diagnosis to another was significant from depression to anxiety and anxiety to depression, from ADHD to oppositional defiant disorder (Costello et al., 2003). Other prospective studies have utilized structured instruments for diagnosis for specific disorders such as Attention deficit disorder (August et al., 1998) and Oppositional Defiant Disorder (ODD) Ninety-two boys, aged 4–5.5 years, with a DSM-III-R diagnosis of ODD were followed over a 2-year period; 42 of them had co-morbid attention-deficit hyperactivity disorder (ADHD); multiple assessment procedures were used, including a modified version of the Diagnostic Interview Schedule for Children. Among 79 boys assessed 2 years later, 76% had ODD, ADHD, or both. Of those, 25% had other diagnoses as well, primarily anxiety and/or mood disorders. Conduct disorder was rare. Subjects with co-morbid ODD/ADHD at intake were significantly more likely to have a psychiatric disorder at follow-up, especially ADHD alone. The authors concluded that ODD in the preschool period is a clear indicator of high risk, especially when co-occurring with ADHD (Speltz et al., 1999). A two-year follow-up study of 51 subjects presenting with psychotic disorders according to DSM-IV criteria utilized standardized diagnostic assessments, including Structured Clinical Interview for DSM-IV, Diagnostic Interview for Children and Adolescents, pre-morbid Adjustment Scale, Schedule for Positive Symptoms and the Schedule for Negative Symptoms, Dissociative Experiences Scale, Children Global Assessment Scale and WISC-III at baseline, at one and two-year’s follow-up. The researchers concluded that, early-onset psychotic disorders can be reliably diagnosed using standardized assessments and are stable over a two-year period (McClellan & McCurry, 1999). Despite differing designs and diagnostic groups under study, all these studies come to similar conclusions that, the stability of most disorders in children and adolescent is, at best, low to moderate. Unfortunately there is a paucity of studies amongst children and adolescents that provide Kappa estimates for either temporal stability or across episode stability. Kappa is preferable as it provides a comprehensive estimate of stability and corrects for agreements due to chance. This was one of the rationale for conducting the present study, the other was that the duration of most past studies in children and adolescents have covered a short time period and a longer period of time would perhaps allow for the subjects to enter period of risk for some of the major psychiatric disorders and thus generate better estimates.

We have chosen to focus on temporal stability to enable us to capture both those disorders that have lengthy courses and those that are episodic and recurring. In addition, to include new cases in our final result, we tried to examine each subject’s diagnosis not only at their first clinical contact and at the end of study period, but also during the entire period covered by the study. Temporal stability is the most commonly assessed form of diagnostic stability, and refers to presence or absence of a disorder at two different time points. High temporal stability may reflect either a lengthy course of a disorder or an episodic, recurring course of a disorder. Fewer studies have investigated the “across-episode stability”, this refers to the recurrence of specific symptoms patterns over distinct episodes of psychopathology with intermittent periods of reduced or no psychopathology (Pettit et al., 2005).

Method

Study Design

This study used a “Cross-sectional Cohort” design to assess the congruence of diagnosis over time. Samples were drawn from a source population cross-sectionally and then outcome was assessed over a specified time period.

We considered threats to validity, such as non-ignorable exiting and measurement error from retrospective assessments however, we chose the cross-sectional cohort design as we felt it offers advantages over traditional designs where there is a long interval between exposures and outcomes, in which exposures and outcomes can be accurately assessed retrospectively, and in which the outcome does not accelerate any exiting from the population. Such situations often arise when studying chronic or episodic conditions with low mortality, such as psychiatric disorders (Hudson et al., 2005).

Thus, using our computerized database, we examined the stability or change of diagnosis up till the age of 22. This applied to the cohort born in 1983, 1984 and 1985 and who were seen as inpatients or outpatients at Child and Adolescent Psychiatry division and who had at least 2 visits between age 5 and 22.

Sample

As mentioned earlier, study sample included all children who were born in 1983, 1984 and 1985 and were seen as patients in Child and Adolescent Psychiatry Division. The sample was drawn from the records of children and adolescents who were either admitted to inpatient wards or were seen in the out-patient clinics from age 5 to age 22, a potentially 17 year follow-up period, at a Child and Adolescent Psychiatry Division. Only those patients who had more than one contact between age 5 and 22 were included in the study.

Procedures and materials

The hospital database contains the records of patients, their date of birth, hospital chart number, date of psychiatric contact, age at the time of visits, gender and DSM diagnosis codes assigned at each contact for each patient.

Patients who were seen in the outpatient clinics were assigned a diagnosis by the treating psychiatrist or by the combined assessments of psychiatrist and the multi-disciplinary team. On the inpatient unit patients were assigned a final DSM III/IV diagnosis at discharge generated by comprehensive assessments conducted by the attending psychiatrist and Multidisciplinary team (Psychologist, Occupational Therapist, Registered Nurses and Social Worker). Diagnoses were based on the DSM-III/1V criteria. Diagnoses, which were documented in patients’ charts and subsequently in computerized hospital record system, were used as reference in our study.

Data Management

The data, as described above, was extracted from the patients’ charts in the hospital medical records. Data was entered in a password-protected file and results were backed up on compact discs to guard against any technical mishap.

Data Analysis

SPSS software was used to analyse the data and explain the result.

Stability of diagnoses was estimated through calculation of positive and negative concordance rates and kappa coefficients. Positive concordance rates provide the proportion of cases that received the same diagnosis in consecutive hospitalizations. Negative concordance rates indicate the proportion of cases that did not receive a given diagnosis at either of two hospitalizations. Kappa provides an estimate of inter-episode agreement while correcting for agreement levels due to chance alone. Kappa values range from +1 to −1, with a value close to 1 indicating nearly perfect agreement. A value close to 0 indicates that agreement is no greater than would be expected by chance. In general, kappa values greater than 0.80 represent excellent stability, values between 0.60 and 0.80 represent good stability; those between 0.40 and 0.60 represent fair stability, and those less than 0.40 represent poor stability (Jeremy et al., 2005).

In order to show the effect of time on stability/instability of DSM diagnoses in children, we divided our sample of 264 patients, into three sub-groups of those with “short gap” (less than two years) between their first and last psychiatric assessments, those with “intermediate gap” (two or three years) and those with “long gap” (four or more years) between their first and last visit.

Also, in order to better highlight the effect of time on stability/instability of DSM diagnoses in children, we did not divide the potential 17-years time frame into three equal periods; instead, we divided them in a way that would demonstrate impact of time on stability of diagnoses more effectively.

Ethical Considerations

Ethics approval for this study was obtained from the Research and Ethics Board of Queen’s University, Canada.

Results

The primary sample yielded 427 patients who were born in 1983, 1984 or 1985. Of these, 264 subjects were assessed more than once, and they constituted the sample for analysis.

Analysis revealed a Male/Female ratio of 47/53; 21% of patients were born in 1983, 28% in 1984 and 51% in 1985. Fifty-three patients had their last visit before age of 18, 120 patients between age 18 and 20 and 91 patients had their last visit between ages 20 to 22. Thus of the 264 subjects of this study sample, 212 patients had their last assessment after age 18.

In order to better analyse the data, we categorized DSM-III/IV diagnoses into 6 general sub-groups for the major Axis I diagnoses and considered the rest of diagnoses (e.g. child-parental conflict, etc.) as “Others”.

Table 1 shows the percentage with a particular diagnosis at the first and at the last visit; such as Mood disorders were present in 39% at first and 45% at last assessment. With no personality disorder diagnosis at first visit in this sample, Kappa is indeterminate, so this diagnosis is dropped from subsequent analyses.

Table 1.
Prevalence of different DSM diagnostic groups among children and adolescents, at their first and last visits

Of the 264 patients who were considered as our final samples, 137 patients (52%) had an interval of less than two years between their first and last visit, 37 patients (14%) were considered to have “intermediate gap” with a 2 or 3 years interval and 90 patients (34%) had 4 years or more documented psychiatric follow-up (long gap).

Table 2 presents results, in descending order, of percentages & kappa significance of the diagnostic groups.

Table 2.
Stability of different DSM diagnostic groups among children and adolescentsDSM IV Diagnostic Groups

Table 3 reports the Kappa statistic computed for the three groups – short, medium and long gap. For mood disorders, behavioural disorders, other diagnoses and the combined table of all diagnoses, the shortest gap produced the highest Kappa, it was significantly greater than zero and it was greater in magnitude than the Kappa for the entire sample. Substance related disorders failed to show a significant Kappa with the combined sample, but the short gap Kappa (although not significant) was also greater than the combined sample. Psychosis showed a small negative Kappa for the short interval and a large positive Kappa for the medium interval. The probability that five of the six diagnoses would show their highest Kappa in the shortest interval simply by chance is less than .01 based on a randomization test.

Table 3.
Stability of diagnosis as a function of interval between assessments

In our study, Mood disorders and Psychosis had highest percentage of stable cases and a higher Kappa suggesting that these disorders are mostly stable in adolescences. Behavioural disorders and Anxiety disorders had lower percentages of stable cases but significant kappa values suggesting fewer cases were stable but also fewer new cases were added. Substance related disorders had very low percentages and non-significant Kappa meaning fewer cases were stable also fewer new cases were added. The latter result is in keeping with diagnostic practices in our division prior to 2000 when neither Personality disorder nor substance use diagnosis were used.

Conclusion

The present study examined temporal diagnostic stability from childhood to young adulthood amongst children and adolescent seen at a regional psychiatric service. The results of our study are in keeping with previous studies, which demonstrate poor to moderate overall stability of disorders through childhood and adolescences. However despite the longer duration of our study the diagnostic stability was poor with kappa values lower than those obtained in other studies. One explanation for the difference may be the study population; the studies with higher Kappa values and diagnostic stability were on severe disorders like schizophrenia (Hollis, 2000) or severity of illness requiring inpatient treatment (Pettit, 2005; Mattanah, 1995); These patients would have had a higher frequency of severe disorders which are more stable as noted both from clinical experience and past literature. Nonetheless the results of our study do follow the trends noted in previous studies, where the highest diagnostic stability and Kappa values were for Mood disorders and Psychotic disorders.

Similar to at least one previous study, the diagnostic stability for substance use disorders was poor in our study. This may be partially explained by the fact that substance abuse is very rarely the primary reason for psychiatric consultation and an infrequent primary diagnosis in our sample. The results for personality disorders are explicable on the basis of age cut-off, being above 18 years, so none of those under that age would have received this as a diagnosis. Somewhat surprising is the result on anxiety disorders, as one of the most enduring traits seems to be that of social inhibition, however diagnostic instability may be understandable when one considers that the manifestation of anxiety changes as a function of developmental stage and age. An additional factor explaining the poorer stability from our results is not only that there was change from DSM-III-R to DSM-IV in 1994 but that diagnostic practices prior to 1998 in the program loosely applied the DSM-III criteria whereas from 2000 onwards there was more stringent adherence to DSM-IV criteria for assigning diagnosis, albeit not all. Indeed, more recently, some of the Inpatient diagnoses were based on structured diagnostic interviews.

Another, often neglected factor, in comparing studies of diagnostic stability is the impact of the gap between the two diagnoses. It is quite understandable that the longer the gap, the more may have changed. The child’s development, treatment of the child’s disorders, the clinicians and their approach to diagnosis, and the diagnostic system can all change over the course of a long study.

Before discussing the implications we will address the limitations of this study. The first of these is that this is a retrospective study of archival data, which was not systematically gathered for research but for clinical records and hence is not uniform for all patients and has limited information. Secondly, diagnoses were based on clinical assessments conducted by a psychiatrist or a treatment team. This is considered to be a less reliable method of assigning diagnosis than structured interviews (Shaffer et al., 2000) and although we did not formally examine the inter-rater reliability, past literature has shown poor inter-rater reliability for diagnoses based on clinical interviews alone (Cantwell & Baker, 1989) as compared to structured diagnostic interviews. Another limiting factor that needs to be considered is the diagnostic reliability itself, despite the improvement in DSM-IV criteria, research shows poor to modest reliability depending on the diagnoses (Todd et al., 2008) and the method used to generate them. Finally, we considered only the main and primary diagnosis for each patient, given the inconsistency of our computerized data in recording second and third diagnoses, which was recorded for just few cases; this issue again, may have had impact on the calculated stability.

Nonetheless, our study has some strength one of which is the longer period of follow-up time of 17 years. The other is that patients are not assigned the same psychiatrist each time they are referred and are seen by different psychiatrists at different visits and depending on whether they are seen in outpatient, day program or admitted to inpatient ward. This would reduce the potential for bias and spurious stability of diagnosis.

From our study results, it would be reasonable to conclude that some of the major psychiatric disorders seem much more stable than others and that since the Bio-Psycho-Social domains of a child are still undergoing developmental changes, we would expect instability of some psychiatric diagnoses such as personality disorders, which crystallizes into more discrete entities with age. It is also apparent that some of the major Axis I diagnoses in childhood and adolescence will continue into adulthood. This forms the major implication of this study, that it is crucial we make every effort to ensure standardized diagnostic practices to improve the diagnostics of all childhood psychiatric disorders in an effort to minimize the long term effects of psychopathology and institute preventive interventions to reduce the extent of impairment and burden of suffering over a person’s life.

Acknowledgements/Conflict of Interest

We would like to thank Dr. David Streiner for his invaluable comments on our study’s proposal. The authors have no financial relationships or conflicts to disclose.

References

  • August GJ, Braswell L, Thuras P. Diagnostic stability of ADHD in a community sample of school-aged children screened for disruptive behaviour. Journal of Abnormal Child Psychology. 1998;26(5):345–56. [PubMed]
  • Cantwell DP, Baker L. Stability and natural history of DSM-III childhood diagnoses. Journal of the American Academy of Child and Adolescent Psychiatry. 1989;28(5):691–700. [PubMed]
  • Costello EJ, Mustillo S, Erkanli A, Keeler G, Angold A. Prevalence and development of psychiatric disorders in childhood and adolescence. Archives of General Psychiatry. 2003;60(8):837–44. [PubMed]
  • Grilo CM, Becker DF, Edellm WS, McGlashan TH. Stability and change of DSM-III-R personality disorder dimensions in adolescents followed up 2 years after psychiatric hospitalization. Comprehensive Psychiatry. 2001;42(5):364–8. [PubMed]
  • Hudson JI, Pope HG, Jr, Glynn RJ. The cross-sectional cohort study: an underutilized design. Epidemiology. 2005;16(3):355–9. [PubMed]
  • Pettit JW, Morgan S, Paukert AL. The stability of axis I diagnoses in youth across multiple psychiatric hospitalizations. Child Psychiatry and Human Development. 2005;36(1):53–71. [PubMed]
  • Mattanah JJF, Becker DF, Levy KN, Edell WS, McGlashan TH. Diagnostic stability in adolescents followed up 2 years after hospitalization. American Journal of Psychiatry. 1995;152(6):889–894. [PubMed]
  • McClellan J, McCurry C. Early onset psychotic disorders: diagnostic stability and clinical characteristics. European Child and Adolescent Psychiatry. 1999;8(Suppl 1):113–9. [PubMed]
  • Shaffer D, Fisher P, Lucas CP, Dulcan MK, Schwab-Stone ME. NIMH Diagnostic Interview Schedule for Children Version IV (NIMH DISC-IV): description, differences from previous versions, and reliability of some common diagnoses. Journal of the American Academy of Child and Adolescent Psychiatry. 2000;39(1):28–38. [PubMed]
  • Speltz ML, McClellan J, DeKlyen M, Jones K. Preschool boys with oppositional defiant disorder: clinical presentation and diagnostic change. Journal of the American Academy of Child and Adolescent Psychiatry. 1999;38(7):838–45. [PubMed]
  • Todd RD, Huang H, Todorov AA, Neuman RJ, Reiersen AM, Henderson CA, Reich WC. Predictors of stability of attention-deficit/hyperactivity disorder subtypes from childhood to young adulthood. Journal of the American Academy of Child and Adolescent Psychiatry. 2008;47(1):76–85. [PubMed]
  • Whitty P, Clarke M, McTigue O, Browne S, Kamali M, Larkin C, O’Callaghan E. Diagnostic Stability Four years after a first episode of psychosis. Psychiatric Services. 2005;56:1084–1088. [PubMed]

Articles from Journal of the Canadian Academy of Child and Adolescent Psychiatry are provided here courtesy of Canadian Academy of Child and Adolescent Psychiatry