PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptNIH Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
Int J Methods Psychiatr Res. Author manuscript; available in PMC Dec 1, 2013.
Published in final edited form as:
PMCID: PMC3591789
NIHMSID: NIHMS422519
Diagnostic validity across racial and ethnic groups in the assessment of adolescent DSM-IV disorders
Jennifer Greif Green,1 Michael J. Gruber,2 Ronald C. Kessler,2 Julia Y. Lin,3 Katie A. McLaughlin,4 Nancy A. Sampson,2 Alan M. Zaslavsky,2 and Margarita Alegria5
1School of Education, Boston University, Boston, MA USA
2Department of Health Care Policy, Harvard Medical School, Boston, MA USA
3Cooperative Studies Program Coordinating Center, VA Palo Alto Healthcare System, Mountain View, CA USA
4Division of General Pediatrics, Children’s Hospital Boston, Harvard Medical School, Boston, MA USA
5Center for Multicultural Mental Health Research, Cambridge Health Alliance, Somerville, MA USA
Reprint requests: ncs/at/hcp.med.harvarvd.edu. Correspondence: Ronald C. Kessler, Department of Health Care Policy, Harvard Medical School, 180 Longwood Avenue, Boston, MA 02115; 617-432-3587 (voice); 617-432-3588 (fax); kessler/at/hcp.med.harvard.edu
We examine differential validity of DSM-IV diagnoses assessed by the fully-structured Composite International Diagnostic Interview Version 3.0 (CIDI) among Latino, Non-Latino Black, and Non-Latino White adolescents in comparison to gold standard diagnoses derived from the Schedule for Affective Disorders and Schizophrenia for School-Age Children (K-SADS). Results are based on the National Comorbidity Survey Replication Adolescent Supplement, a national US survey of adolescent mental health. Clinicians re-interviewed 347 adolescent/parent dyads with the K-SADS. Sensitivity and/or specificity of CIDI diagnoses varied significantly by ethnicity/race for four of ten disorders. Modifications to algorithms sometimes reduced bias in prevalence estimates, but at the cost of reducing individual-level concordance. These findings document the importance of assessing fully-structured diagnostic instruments for differential accuracy in ethnic/racial subgroups.
Keywords: National Comorbidity Survey Replication Adolescent Supplement, Validity, Disparities
Racial/ethnic disparities in treatment of mental disorders are well-documented (Cuffe et al., 1995; Kataoka et al., 2002) and may be due at least in part to differences in the identification of mental disorders across racial/ethnic groups. Interviews designed to diagnose mental disorders are usually developed in general population samples rather than designed to be as effective in minority samples, with the result that these instruments are often less effective in identifying racial/ethnic minorities with mental health problems as compared to respondents in the majority population (Alegria et al., 2011; Alegria et al., 2009; Breslau et al., 2008; Eisen et al., 2006; Johnson et al., 2007). These differences may also help explain discrepancies in epidemiological studies of racial/ethnic differences in disorder prevalence. The current report presents results of a psychometric study designed to investigate this issue in a large epidemiological survey of mental disorders among youths in the US. Previous studies of race/ethnic differences in the mental health of US adolescents have yielded inconsistent results (Anderson and Mayes, 2010; McLaughlin et al., 2007), with some studies finding significant group differences in disorder prevalence (Minsky et al., 2006; Roberts et al., 1997) and others not (Angold et al., 2002). It is important to examine the extent to which lack of equivalence in the sensitivity or specificity of measures of mental health across racial/ethnic groups are involved in explaining these results for both research and service planning purposes.
There are several reasons to expect that the diagnostic validity of fully-structured diagnostic interviews may be lower among racial/ethnic minority adolescents than other adolescents. First, there may be racial/ethnic differences in comfort with the interview process, rapport built with interviewers (potentially related to interviewer race/ethnicity), and general familiarity with the task of responding to questions about mental health symptoms (Kessler and Ustun, 2004; Komiti et al., 2001). A second concern is that minority adolescents may have different interpretations of the structured interview questions, many of which are complex, use vague terminology, and include multiple clauses (Kessler and Ustun, 2004). Third, cultural context may influence adolescents’ and parents’ subjective interpretation of behaviors, particularly in judging whether symptoms reflect maladjustment or result in impairment (Alegria et al., 2009). Fourth, culturally-grounded conceptions of time (Alegria et al., 2009; Kessler and Ustun, 2004) may influence the accuracy of retrospective reports across racial/ethnic groups, particularly when asked questions about events that happened many years ago or when interviewers request information about the duration and recency of symptoms. Finally, it is possible that the phenomenology of disorders vary across racial/ethnic groups, with some cultures viewing behaviors as pathological that others consider normative (Lewis-Fernández et al., 2010). These variations may influence both endorsement of symptoms and reports of related impairment.
In this study we examine the validity of the Composite International Diagnostic Interview Version 3.0 (CIDI), administered in the US National Comorbidity Survey Replication Adolescent Supplement (NCS-A; Merikangas et al., 2009), for racial/ethnic minority adolescents. Prior research in the NCS-A using the aggregate youth sample showed strong concordance of individual-level CIDI diagnoses with diagnoses determined by blinded interviews carried out by clinicians using the Schedule for Affective Disorders and Schizophrenia for School-Age Children (K-SADS; Kessler et al., 2009c). Area under the ROC curve (AUC) ranged from .84–.94 for classes of disorders and was .87 for diagnosis of any disorder.
Although overall CIDI validity is quite good at identifying mental health needs for the general population of adolescents, it is unclear whether validity differs by race/ethnicity and, if so, whether it is possible to improve measurement precision for racial/ethnic minority youths by changing diagnostic thresholds. Prevalence estimates in the NCS-A indicate some significant racial/ethnic differences, particularly lower prevalence of externalizing disorders among minorities than non-Latino white youths, even when controlling for socio-economic status and other demographic covariates (Kessler et al., under review). Given inconsistencies in the literature about variation in disorder prevalence by race and ethnicity (Anderson and Mayes, 2010; McLaughlin et al., 2007), we investigate whether group differences in the NCS-A may be influenced by lack of measurement equivalence for certain racial or ethnic groups.
The NCS-A sample
The NCS-A was carried out between February 2001 and January 2004. Adolescents (ages 13–17) were interviewed face-to-face in dual-frame household and school samples (Kessler et al., 2009a, b). The household sample included 904 adolescents (25 who had dropped out of school) from households that participated in the National Comorbidity Survey Replication (NCS-R), a national survey of adults (Kessler and Merikangas, 2004). The conditional (on NCS-R participation) adolescent response rate was 86.8%. The school sample included 9,244 adolescents from a representative sample of 320 schools in the NCS-R counties. The conditional (on school participation) adolescent response rate was 82.6%. One parent or surrogate (henceforth referred to as parents) was asked to complete a self-administered questionnaire (SAQ) about the participating adolescent’s developmental history and mental health. The parent SAQ conditional (on adolescent participation) response rate was 82.5% in the household sample and 83.7% in the school sample. This report focuses on the 6,483 adolescent-parent pairs with data from both adolescent interviews and SAQs.
Although the proportion of initially selected schools that participated was low (28.0%), these were replaced with matched replacement schools. Comparison of household sample respondents who attended nonparticipating schools with school sample respondents from replacement schools found no evidence of bias in estimates of either prevalence or correlates of disorders (Kessler et al., 2009a).
Once the survey was completed, cases were weighted for variation in within-household probability of selection (household sample) and residual discrepancies between sample and population socio-demographic and geographic distributions. These weighting procedures are detailed elsewhere (Kessler et al., 2009a, b). The weighted composite sample socio-demographic distributions of the NCS-A closely approximate those of the 2000 Census population, with 65.6% of respondents being Non-Latino White, 15.1% Non-Latino black, 14.4% Latino, and 5.0% Other. The Others were not included in the analyses reported here because of their small numbers.
Written informed consent was obtained from parents or guardians before approaching adolescents to participate in the survey. Written adolescent assent was then obtained from adolescents before surveying either adolescents or parents. Each respondent was given $50 for participation. These recruitment-consent procedures were approved by the Human Subjects Committees of both Harvard Medical School and the University of Michigan.
The clinical reappraisal sample
A clinical reappraisal study was completed by telephone with a quota sample of 347 adolescent respondents from the school sample and their parents (Kessler et al., 2009c). Adolescents who met DSM-IV/CIDI criteria for one or more relatively uncommon disorders (e.g., bipolar I or II, agoraphobia) were oversampled relative to respondents who met criteria only for more common disorders and those with no disorders were sampled at an even lower rate to acquire a large enough sample of adolescents with each disorder to conduct disorder-specific analyses of concordance between diagnoses based on the CIDI and the K-SADS. The sample included 241 Non-Latino Whites, 56 Non-Latino Blacks, 32 Latinos, and 18 Others. Each adolescent and parent respondent was given a $50 incentive for participation in the clinical reappraisal survey (over and above the $50 incentive for participation in the main survey). (For details on the NCS-A clinical reappraisal study see Kessler et al., 2009c).
Measures
CIDI Diagnostic Assessment
Adolescents were administered a modified version of the Composite International Diagnostic Interview (CIDI), a fully-structured interview designed for use by trained lay interviewers (Kessler and Ustun, 2004). The DSM-IV disorders assessed include mood disorders (major depressive disorder [MDD] or dysthymia, bipolar I–II disorder [BPD]), anxiety disorders (panic disorder with or without agoraphobia [PD], agoraphobia without history of panic disorder, social phobia, specific phobia, generalized anxiety disorder [GAD], post-traumatic stress disorder [PTSD], separation anxiety disorder), behavior disorders (attention-deficit/hyperactivity disorder [ADHD], oppositional-defiant disorder [ODD], conduct disorder [CD], intermittent explosive disorder [IED], and eating disorders [anorexia nervosa, bulimia nervosa, binge-eating disorder]), and substance disorders (alcohol and drug abuse, alcohol and drug dependence with abuse).
In addition, parent questionnaires assessed disorders for which parent reports have previously been shown to play a large part in diagnosis: behavior disorders (Grills and Ollendick, 2002; Johnston and Murray, 2003) and depression/dysthymia (Braaten et al., 2001). Parent and adolescent reports were combined at the symptom level using an “or” rule (i.e., the symptom was considered present if endorsed by either respondent). All diagnoses were made using DSM-IV organic exclusion rules. All but two diagnoses were made using DSM-IV diagnostic hierarchy rules. The exceptions were ODD, which was defined with or without CD, and substance abuse, which was defined with or without dependence. Prevalence was assessed in three time frames: lifetime, past 12 months, and past 30 days. Our analyses of diagnostic concordance are focused on the concordance of lifetime diagnoses, as the K-SADS administration focused on a lifetime time frame. In order to address the problem that lifetime prevalence is under-estimated in retrospective cross-sectional surveys of adults (Kessler et al., 1998; Moffitt et al., 2010) and youth (Cohen et al., 1993; Costello et al., 2003; Kim-Cohen et al., 2003), special probing procedures were used to increase accuracy of memory search (Kessler and Ustun, 2004). These procedures have been shown experimentally to increase recall of lifetime mental disorders among adults (Kessler et al., 1998).
K-SADS Diagnostic Assessment
The K-SADS is a semi-structured research diagnostic interview that is designed to be administered by trained clinical interviewers. K-SADS questions are designed to elicit rich verbal responses that form the basis of interviewer ratings about the presence or absence of symptoms. The standard K-SADS was modified by deleting disorders not assessed in the NCS-A, focusing only on a lifetime time frame, and streamlining the initial screening section of the interview to include information about respondent endorsement of diagnostic stem questions in the earlier CIDI interview (for description, see Kessler et al., 2009c).
The disorders assessed in this version of the K-SADS included six DSM-IV anxiety disorders (panic disorder with or without agoraphobia, agoraphobia without a history of panic disorder, GAD, specific phobia, social phobia, PTSD), three mood disorders (bipolar spectrum disorder, MDD, dysthymic disorder), three disruptive behavior disorders (ADHD, conduct disorder, ODD), and four substance use disorders (alcohol abuse with or without dependence, illicit drug abuse with or without dependence, alcohol dependence with a history of abuse, illicit drug dependence with a history of abuse).
Socio-demographics
Adolescents were assigned to the following racial/ethnic groups based on self-reported responses to race and ethnicity questions: Non-Latino White, Non-Latino Black, Latino, and Other. As noted above, adolescents classified as “Other” were excluded from these analyses because of their small number and heterogeneity.
Analysis Methods
Analyses of concordance between diagnoses based on the CIDI and those based on the K-SADS were carried out after weighting the clinical reappraisal sample data to adjust for the under-sampling of CIDI non-cases and those with only common CIDI disorders. We investigated whether there were racial/ethnic differences in concordance with binary logistic regression equations that predicted K-SADS diagnoses from the main effects and interaction of race/ethnicity with CIDI diagnoses. Significant interactions indicated that concordance between diagnoses based on the CIDI and K-SADS differ significantly by respondent race/ethnicity. In cases of this sort, we investigated the nature of the discordance by testing whether CIDI sensitivity (SN) and specificity (SP) differed significantly by race/ethnicity based on the Rao-Scott χ2 test.
For each disorder with significant racial/ethnic differences in either SN or SP, we calculated McNemar χ2 tests to evaluate bias in prevalence estimates and area under the receiver operator characteristic curve (AUC) and Cohen’s κ (Cohen, 1960) to evaluate individual-level concordance between diagnoses based on the CIDI and K-SADS. We used the κ because it is the most commonly used measure of diagnostic concordance. But we also used AUC as an alternative measure because κ has the disadvantage of varying across populations that differ in prevalence even when the populations do not differ in SN or SP. The AUC does not have this problem. Although the AUC was developed to study the association between a continuous predictor and a dichotomous outcome, it can be used in the special case where the predictor is a dichotomy, in which case AUC equals (SN + SP)/2.
The final phase of analysis involved modifying the strategies for combining CIDI symptom-level data to generate diagnoses that would improve concordance with K-SADS diagnoses for different racial/ethnic groups. This was done by using regression analysis to predict K-SADS diagnoses from CIDI symptom-level items. Where possible, we used results from previous analyses of K-SADS/CIDI concordance in the full clinical reappraisal sample to guide the selection of criteria for CIDI modification. When the interaction of a CIDI item and the race/ethnicity of the respondents significantly improved prediction of K-SADS diagnosis, we modified the CIDI diagnostic algorithm and then compared modified CIDI diagnoses to the K-SADS diagnoses to see if they had higher concordance than did the original CIDI diagnoses.
Racial/ethnic differences in CIDI/K-SADS diagnostic concordance
With the exceptions of one low AUC value for Latinos (.66 for ADHD) and three low values for Non-Latino blacks (.53–.65 for panic disorder, PTSD, and ODD), AUC values are quite comparable across diagnoses among Latinos (.75–1.0), Non-Latino Blacks (.73–1.0), and Non-Latino Whites (.76–.94). (Table 1) Disaggregation of AUC into its two main components, SN and SP, finds significant race-ethnic differences in SN for four disorders (agoraphobia, panic disorder, PTSD, and ADHD) and in SP for one (agoraphobia). These significant differences encompass all cases where AUC is meaningfully lower among minorities with the exception of ODD. In the latter case, the low AUC among Non-Latino Blacks (.53) is not associated with significant race/ethnic differences in either SN or SP due to the small number of respondents with this disorder.
Table 1
Table 1
Race/ethnic differences in the sensitivity, specificity, and area under the ROC curve (AUC) of CIDI/DSM-IV diagnoses compared to the clinical gold standard diagnoses in the NCS-A clinical reappraisal sample (n = 347)
For agoraphobia, SN is significantly lower for Non-Latino Whites than others (χ21 = 4.0, p = .045) and SP is significantly lower for Non-Latino Blacks than others (χ22 = 8.0, p = .018). For panic disorder and PTSD, SN is significantly lower for Non-Latino Blacks (panic disorder χ21 = 9.1, p = .003; PTSD χ21 = 5.6, p = .018), whereas ADHD SN is significantly lower for Latinos than those in other race/ethnic groups (χ22 = 7.8, p = .020). These four diagnoses were the focus of subsequent analyses to investigate how to enhance the algorithms for detecting these disorders so as to remove differences in concordance between diagnoses based on the CIDI and K-SADS across racial and ethnic groups.
CIDI/K-SADS diagnostic concordance for agoraphobia
For agoraphobia, SP was strong for all race/ethnic groups; however, the relatively low SP for Non-Latino Black youths (95.9% vs. 97.3–99.4% for Latinos and Non-Latino Whites) led to over-identification of the disorder. (Table 2) When based on the K-SADS, the estimated prevalence of agoraphobia among Non-Latino Blacks was 2.6%; the estimate more than doubled when it was based on the CIDI (6.5%; χ22 = 19.5, p < .001). The CIDI also significantly over-estimated agoraphobia among Latino youths (CIDI 4.4% vs. K-SADS 1.8%, χ22 = 7.2, p = .007).
Table 2
Table 2
Race/ethnic differences in the diagnostic concordance of original and modified CIDI/DSM-IV diagnoses compared to the clinical gold standard diagnoses in the NCS-A clinical reappraisal sample (n = 347)
To improve specificity, we tested strategies to tighten diagnostic criteria and lower CIDI prevalence of agoraphobia for Non-Latino Blacks and Latinos. We selected questions measuring agoraphobia that were not included in our initial diagnostic algorithms (e.g., inability to leave home without a family member or friend) and entered each of these symptoms into a regression model with CIDI diagnosis and Non-Latino Black race (vs. other race/ethnicity) to predict the K-SADS agoraphobia diagnosis. Of the symptoms tested, only one significantly interacted with Non-Latino Black race to improve K-SADS prediction: “Was there ever a time in your life when you felt badly about yourself or were upset, worried, or disappointed with yourself because of your fear (or avoidance)?” This item was added as a requirement to the original CIDI diagnosis for Non-Latino Black and Latino youths.
With the addition of this symptom to the diagnostic algorithm for agoraphobia, CIDI prevalence estimates improved for Non-Latino Black (CIDI prevalence became 4.0% vs. K-SADS 2.6%; χ22 = 3.6, p = 0.06) and Latino adolescents (CIDI 2.1% vs. K-SADS 1.8%; χ22 = 0.1, p = 0.71). SP also improved. However, SN decreased from 100% to 47.5% for Latinos and from 93.8% to 69.8% for Non-Latino Blacks, leading to a decline in κ for Latinos (.56 to .42) and in AUC (from .99 to .73 for Latinos and from .95 to .84 for Non-Latino Blacks).
CIDI/K-SADS diagnostic concordance for panic disorder
For diagnoses of panic disorder, SN was significantly lower for Non-Latino Black (SN = 31.5%) than Non-Latino White and Latino youths (SN = 81.4%, 100.0%, respectively), indicating that the CIDI failed to identify a large proportion of Non-Latino Black adolescents with the disorder (κ = .34, AUC = .65). Despite differences in SN, CIDI and K-SADS prevalence estimates were similar (CIDI 1.7%, K-SADS 2.2%; χ22 = 0.5, p = 0.49). The small number of Non-Latino Black adolescents in the clinical reappraisal sample with a K-SADS diagnosis of panic disorder prevented us from investigating CIDI modifications that could improve SN.
CIDI/K-SADS diagnostic concordance for PTSD
PTSD SN was significantly lower for Non-Latino Black youths (SN = 32.9%) than others (SN = 56.9% for Non-Latino White, 100.0% for Latino youths) resulting in low κ (.36) and AUC (.65). The CIDI PTSD prevalence estimate was slightly lower than the K-SADS estimate (3.4% vs. 4.6%), but this difference was not statistically significant (χ22 = 1.7, p = 0.19). As with panic disorder, the small number of Non-Latino Black adolescents with PTSD in the clinical reappraisal sample prevented investigating CIDI modifications that could improve CIDI diagnostic algorithms.
CIDI/K-SADS diagnostic concordance for ADHD
The CIDI significantly over-estimated ADHD prevalence among Non-Latino Whites (CIDI 7.2% vs. K-SADS 6.0%; χ22 = 5.8, p = .016) and under-estimated prevalence among Latinos (CIDI 2.4% vs. K-SADS 7.4%; χ22 = 11.5, p = .001). For Latinos, SN was consequently quite low (SN = 32.4%) and, although somewhat higher, SN was only moderate for Non-Latino Blacks (SN = 53.8%) and Non-Latino Whites (SN = 64.7%). Because of the low SN, we attempted to loosen ADHD diagnostic criteria to improve CIDI identification of the disorder among minority youths. In a previous study (Green et al., 2010), we determined that the best CIDI estimate of K-SADS ADHD in the total NCS-A sample was based on parent report only. Further, parents over-estimated K-SADS ADHD, and we therefore tightened diagnostic criteria, requiring 10 or more ADHD symptoms (in any combination) and at least “some” impairment in functioning. To improve disorder identification for racial/ethnic minorities, we revisited this diagnostic algorithm and examined several modifications to the criteria. We found that the best modification required either meeting the criteria for ADHD described above, or for parents to endorse (a) six or more AD or HD symptoms, and (b) indicate that symptoms, at their worst, interfered with home life, friendships, or school/work “a lot” or “extremely.” With these modifications, the CIDI was able to identify Latino adolescents with K-SADS ADHD diagnoses perfectly (κ increased from .47 to 1.0; AUC increased from .66 to 1.0). Estimated prevalence for Non-Latino Black adolescents also became closer to the K-SADS value (CIDI 21.5% vs. K-SADS 20.4%; χ22 = 0.3, p = 0.59); however, SN remained the same and SP dropped (from SP = 92.6% to 86.8%), decreasing the overall κ (from .50 to .40) and AUC (from 0.73 to 0.70).
We found that the sensitivity of the CIDI varies by race/ethnicity for four diagnoses out of the ten considered here (ADHD, agoraphobia, panic disorder, and PTSD) and that specificity of the CIDI varies by race/ethnicity for one diagnosis (agoraphobia). Although it is not clear why these four rather than the other six disorders were particularly discrepant in their concordance with clinical diagnoses, it is noteworthy that all four have a component of physiological hyperarousal and reactivity, which previous research suggests may be particularly sensitive to the cultural context of racial/ethnic minority youth (Pina and Silverman, 2004; Varela et al., 2007).
In the case of PTSD, which has previously been identified as problematic for the CIDI to diagnose accurately in racial/ethnic minority groups (Alegria et al., 2009), it has been suggested that fully-structured instruments like the CIDI are differentially biased for minorities because they are less able than semi-structured clinical interviews to interpret the cultural context of trauma and trauma-related symptoms (Alarcon, 1995). However, we were unable to investigate this possibility in the NCS-A clinical reappraisal study because the number of minority youths with PTSD was too small to allow modifications of diagnostic criteria to be evaluated with adequate precision. The same was true for panic disorder.
The situation was different for agoraphobia and ADHD, where we were able to make modifications to improve diagnostic criteria. In the case of agoraphobia, we found that tightening the diagnostic algorithm for Latino and Non-Latino Black adolescents reduced the inflated CIDI prevalence estimates in these subsamples. In particular, we added the requirement that racial/ethnic minority youth indicate feeling badly about or disappointed in themselves because of their fear or avoidance. We speculate that this item may tap into perceptions of the cultural acceptability of symptoms. However, this change in diagnostic criteria also substantially decreased CIDI sensitivity to detect agoraphobia among racial/ethnic minority youths, leading to a decrease in individual-level concordance between diagnoses based on the CIDI and clinical diagnoses. The end result was that we did not implement any changes in the CIDI diagnosis of agoraphobia. A general absence of data on the accuracy of assessments of agoraphobia for racial/ethnic minority youths (Lewis-Fernandez et al., 2010) suggests that this is an important area for future research.
Modifications to the diagnostic algorithm for ADHD were more successful. In an earlier paper, we modified CIDI diagnostic criteria in the total sample to account for parent over-estimated of ADHD symptoms (Green et al., 2010). Here, the finding that diagnostic criteria needed to be loosened to improve concordance for Latino and Non-Latino Black parents suggests that parents of racial/ethnic minority youth are less likely to over-endorse ADHD symptoms. This finding is consistent with prior research indicating that, given comparable levels of hyperactivity, parents of racial/ethnic minority youth less often endorse symptoms than parents of Non-Latino White children (Hillemeier et al., 2007). By modifying CIDI criteria to allow more flexibility in parent symptom report, we were able to perfectly identify Latino youths with ADHD and improve the accuracy of prevalence estimates for Non-Latino Black adolescents, although these benefits occurred at the expense of a slight decrease in the specificity of diagnoses for Non-Latino Black adolescents.
Several limitations in the design of the NCS-A and of the clinical reappraisal study may have influenced our results. First, the NCS-A sample excluded school dropouts, the homeless, and non-English speakers; all of which are groups where racial/ethnic minority youth are disproportionately represented. Second, there were high rates of individual non-response and school non-response, although analysis of effects of nonresponse in the NCS-A found little evidence of bias (Kessler et al., 2009a). No data were collected on the race/ethnicity of non-respondents, so we do not know whether non-response rates differ across race/ethnic groups. Third, the K-SADS was administered by telephone (in contrast to the face-to-face CIDI administration). There is strong evidence that telephone interviews are a valid method for clinical assessment (Aneshensel et al., 1982; Rohde et al., 1997; Sobin et al., 1993) and, in the case of the NCS-A, it provided the only feasible method for this type of large-scale data collection. However, the comparison of in-person CIDI with phone K-SADS interviews likely make concordance estimates more conservative. Fourth, the design of the clinical reappraisal study, which provided K-SADS clinical interviewers with information about responses to diagnostic stem questions in the CIDI interview, may have influenced racial/ethnic differences. Fifth, all the analyses reported here were based on the untested assumption that diagnoses based on the K-SADS are equally valid for minority and nonminority youth. Sixth, the clinical reappraisal study was not specifically designed to study CIDI validity by race/ethnicity. As a result, the number of racial/ethnic minority youths in the reappraisal study sample was smaller than we would have desired, limiting statistical power to study modifications to diagnostic criteria for the least common disorders.
These findings underscore the importance of testing measurement validity by race and ethnicity. They suggest that, although CIDI diagnostic algorithms appear to function similarly across racial/ethnic groups for some disorders, there are four for which CIDI classifications have lower validity for racial/ethnic minority youth. In these cases, applying a universal framework to assessment may mask racial/ethnic differences, resulting in misleading prevalence estimates and disorder misclassification (Alegria and McGuire, 2003). In some cases we were able to adjust disorder-specific CIDI diagnostic algorithms across racial/ethnic groups to improve estimated prevalence for racial/ethnic minorities, but these benefits were offset by a diminished ability to classify individuals with disorders. In deciding whether to use these modified diagnostic algorithms, researchers should be guided by the specific purposes of diagnostic assessment and, in particular, whether they are emphasizing prevalence estimation, or individual classification.
We recommend that future studies of diagnostic validity similarly attend to the potential for differential validity across racial/ethnic groups. Results have implications for interpreting subgroup comparisons and, further, may suggest qualitative distinctions between groups in the phenomenology of disorders (Alegria et al., 2009).
ACKNOWLEDGEMENTS
This study was supported by grants from the National Institute of Health National Center on Minority Health and Health Disparities (NIH; RC1MD004588, Alegria) and the NIH National Institute of Mental Health (K01-MH085710, Green). The National Comorbidity Survey Replication Adolescent Supplement (NCS-A) is supported by the National Institute of Mental Health (NIMH; U01-MH60220 and R01-MH66627) with supplemental support from the National Institute on Drug Abuse (NIDA), the Substance Abuse and Mental Health Services Administration (SAMHSA), the Robert Wood Johnson Foundation (RWJF; Grant 044780), and the John W. Alden Trust. The views and opinions expressed in this report are those of the authors and should not be construed to represent the views of any of the sponsoring organizations, agencies, or U.S. Government. A complete list of NCS-A publications can be found at http://www.hcp.med.harvard.edu/ncs. Send correspondence to ncs@hcp.med.harvard.edu. The NCS-A is carried out in conjunction with the World Health Organization World Mental Health (WMH) Survey Initiative. We thank the staff of the WMH Data Collection and Data Analysis Coordination Centres for assistance with instrumentation, fieldwork, and consultation on data analysis. The WMH Data Coordination Centres have received support from NIMH (R01-MH070884, R13-MH066849, R01-MH069864, R01-MH077883), NIDA (R01-DA016558), the Fogarty International Center of the National Institutes of Health (FIRCA R03-TW006481), the John D. and Catherine T. MacArthur Foundation, the Pfizer Foundation, and the Pan American Health Organization. The WMH Data Coordination Centres have also received unrestricted educational grants from Astra Zeneca, BristolMyersSquibb, Eli Lilly and Company, GlaxoSmithKline, Ortho-McNeil, Pfizer, Sanofi-Aventis, and Wyeth. A complete list of WMH publications can be found at http://www.hcp.med.harvard.edu/wmh/.
Footnotes
Declaration of interests
Dr. Kessler has been a consultant for AstraZeneca, Analysis Group, Bristol-Myers Squibb, Cerner-Galt Associates, Eli Lilly & Company, GlaxoSmithKline Inc., HealthCore Inc., Health Dialog, Integrated Benefits Institute, John Snow Inc., Kaiser Permanente, Matria Inc., Mensante, Merck & Co, Inc., Ortho-McNeil Janssen Scientific Affairs, Pfizer Inc., Primary Care Network, Research Triangle Institute, Sanofi-Aventis Groupe, Shire US Inc., SRA International, Inc., Takeda Global Research & Development, Transcept Pharmaceuticals Inc., and Wyeth-Ayerst; has served on advisory boards for Appliance Computing II, Eli Lilly & Company, Mindsite, Ortho-McNeil Janssen Scientific Affairs, Plus One Health Management and Wyeth-Ayerst; and has had research support for his epidemiological studies from Analysis Group Inc., Bristol-Myers Squibb, Eli Lilly & Company, EPI-Q, GlaxoSmithKline, Johnson & Johnson Pharmaceuticals, Ortho-McNeil Janssen Scientific Affairs., Pfizer Inc., Sanofi-Aventis Groupe, and Shire US, Inc.
  • Alarcon RD. Culture and psychiatric diagnosis. Impact on DSM-IV and ICD-10. Psychiatric Clinics of North America. 1995;18:449–465. [PubMed]
  • Alegria M, Carson NJ, Goncalves M, Keefe K. Disparities in treatment for substance use disorders and co-occurring disorders for ethnic/racial minority youth. Journal of the American Academy of Child and Adolescent Psychiatry. 2011;50:22–31. [PMC free article] [PubMed]
  • Alegria M, McGuire T. Rethinking a universal framework in the psychiatric symptom-disorder relationship. Journal of Health and Social Behavior. 2003;44:257–274. [PubMed]
  • Alegria M, Shrout PE, Torres M, Lewis-Fernandez R, Abelson JM, Powell M, Interian A, Lin J, Laderman M, Canino G. Lessons learned from the clinical reappraisal study of the Composite International Diagnostic Interview with Latinos. International Journal of Methods in Psychiatric Research. 2009;18:84–95. [PMC free article] [PubMed]
  • Anderson ER, Mayes LC. Race/ethnicity and internalizing disorders in youth: a review. Clinical Psychology Review. 2010;30:338–348. [PubMed]
  • Aneshensel CS, Frerichs RR, Clark VA, Yokopenic PA. Measuring depression in the community: a comparison of telephone and personal interviews. Public Opinion Quarterly. 1982;46:110–121. [PubMed]
  • Angold A, Erkanli A, Farmer EM, Fairbank JA, Burns BJ, Keeler G, Costello EJ. Psychiatric disorder, impairment, and service use in rural African American and white youth. Archives of General Psychiatry. 2002;59:893–901. [PubMed]
  • Braaten EB, Biederman J, DiMauro A, Mick E, Monuteaux MC, Muehl K, Faraone SV. Methodological complexities in the diagnosis of major depression in youth: an 21 analysis of mother and youth self-reports. Journal of Child and Adolescent Psychopharmacology. 2001;11:395–407. [PubMed]
  • Breslau J, Javaras KN, Blacker D, Murphy JM, Normand SL. Differential item functioning between ethnic groups in the epidemiological assessment of depression. Journal of Nervous and Mental Disease. 2008;196:297–306. [PMC free article] [PubMed]
  • Cohen P, Cohen J, Kasen S, Velez CN, Hartmark C, Johnson J, Rojas M, Brook J, Streuning EL. An epidemiological study of disorders in late childhood and adolescence--I. Age- and gender-specific prevalence. Journal of Child Psychology and Psychiatry. 1993;34:851–867. [PubMed]
  • Costello EJ, Mustillo S, Erkanli A, Keeler G, Angold A. Prevalence and development of psychiatric disorders in childhood and adolescence. Archives of General Psychiatry. 2003;60:837–844. [PubMed]
  • Cuffe SP, Waller JL, Cuccaro ML, Pumariega AJ, Garrison CZ. Race and gender differences in the treatment of psychiatric disorders in young adolescents. Journal of the American Academy of Child and Adolescent Psychiatry. 1995;34:1536–1543. [PubMed]
  • Eisen SV, Gerena M, Ranganathan G, Esch D, Idiculla T. Reliability and validity of the BASIS-24 Mental Health Survey for Whites, African-Americans, and Latinos. Journal of Behavioral Health Services and Research. 2006;33:304–323. [PubMed]
  • Green JG, Avenevoli S, Finkelman M, Gruber MJ, Kessler RC, Merikangas KR, Sampson NA, Zaslavsky AM. Attention deficit hyperactivity disorder: concordance of the adolescent version of the Composite International Diagnostic Interview Version 3.0 (CIDI) with the K-SADS in the US National Comorbidity Survey Replication Adolescent (NCS-A) supplement. International Journal of Methods in Psychiatric Research. 2010;19:34–49. [PMC free article] [PubMed]
  • Grills AE, Ollendick TH. Issues in parent-child agreement: the case of structured diagnostic interviews. Clinical Child and Family Psychology Review. 2002;5:57–83. [PubMed]
  • Hillemeier MM, Foster EM, Heinrichs B, Heier B. on behalf of the Conduct Problems Prevention Research Group. Racial differences in parental reports of attention-deficit/hyperactivity disorder behaviors. Journal of Developmental and Behavioral Pediatrics. 2007;28:353–361. [PMC free article] [PubMed]
  • Johnson MR, Hartzema AG, Mills TL, De Leon JM, Yang M, Frueh C, Santos A. Ethnic differences in the reliability and validity of a Panic Disorder Screen. Ethnicity and Health. 2007;12:283–296. [PubMed]
  • Johnston C, Murray C. Incremental validity in the psychological assessment of children and adolescents. Psychological Assessment. 2003;15:496–507. [PubMed]
  • Kataoka SH, Zhang L, Wells KB. Unmet need for mental health care among U.S. children: variation by ethnicity and insurance status. American Journal of Psychiatry. 2002;159:1548–1555. [PubMed]
  • Kessler RC, Avenevoli S, Costello EJ, Green JG, Gruber MJ, Heeringa S, Merikangas KR, Pennell BE, Sampson NA, Zaslavsky AM. Design and field procedures in the US National Comorbidity Survey Replication Adolescent Supplement (NCS-A) International Journal of Methods in Psychiatric Research. 2009a;18:69–83. [PMC free article] [PubMed]
  • Kessler RC, Avenevoli S, Costello EJ, Green JG, Gruber MJ, Heeringa S, Merikangas KR, Pennell BE, Sampson NA, Zaslavsky AM. National comorbidity survey replication adolescent supplement (NCS-A): II. Overview and design. Journal of the American Academy of Child and Adolescent Psychiatry. 2009b;48:380–385. [PMC free article] [PubMed]
  • Kessler RC, Avenevoli S, Green J, Gruber MJ, Guyer M, He Y, Jin R, Kaufman J, Sampson NA, Zaslavsky AM. National comorbidity survey replication adolescent supplement (NCS-A): III. Concordance of DSM-IV/CIDI diagnoses with clinical reassessments. Journal of the American Academy of Child and Adolescent Psychiatry. 2009c;48:386–399. [PMC free article] [PubMed]
  • Kessler RC, Avenevoli S, McLaughlin KA, Green JG, Lakoma MD, Petukhova M, Pine DS, Sampson NA, Zaslavsky AM, Merikangas KR. (under review) Lifetime comorbidity of DSM-IV disorders in the NCS-R Adolescent Supplement (NCS-A) [PMC free article] [PubMed]
  • Kessler RC, Merikangas KR. The National Comorbidity Survey Replication (NCS-R): background and aims. International Journal of Methods in Psychiatric Research. 2004;13:60–68. [PubMed]
  • Kessler RC, Ustun TB. The World Mental Health (WMH) Survey Initiative Version of the World Health Organization (WHO) Composite International Diagnostic Interview (CIDI) International Journal of Methods in Psychiatric Research. 2004;13:93–121. [PubMed]
  • Kessler RC, Wittchen H-U, Abelson JM, McGonagle KA, Schwarz N, Kendler KS, Knäuper B, Zhao S. Methodological studies of the Composite International Diagnostic Interview (CIDI) in the US National Comorbidity Survey. International Journal of Methods in Psychiatric Research. 1998;7:33–55.
  • Kim-Cohen J, Caspi A, Moffitt TE, Harrington H, Milne BJ, Poulton R. Prior juvenile diagnoses in adults with mental disorder: developmental follow-back of a prospective-longitudinal cohort. Archives of General Psychiatry. 2003;60:709–717. [PubMed]
  • Komiti AA, Jackson HJ, Judd FK, Cockram AM, Kyrios M, Yeatman R, Murray G, Hordern C, Wainwright K, Allen N, Singh B. A comparison of the Composite International Diagnostic Interview (CIDI-Auto) with clinical assessment in diagnosing mood and anxiety disorders. Australian and New Zealand Journal of Psychiatry. 2001;35:224–230. [PubMed]
  • McLaughlin KA, Hilt LM, Nolen-Hoeksema S. Racial/ethnic differences in internalizing and externalizing symptoms in adolescents. Journal of Abnormal Child Psychology. 2007;35:801–816. [PMC free article] [PubMed]
  • Lewis-Fernández R, Hinton DE, Laria AJ, Patterson EH, Hofmann SG, Craske MG, Stein DJ, Asnaani A, Liao B. Culture and the anxiety disorders: recommendations for DSM-V. Depression and Anxiety. 2010;27:212–229. [PubMed]
  • Merikangas K, Avenevoli S, Costello J, Koretz D, Kessler RC. National comorbidity survey replication adolescent supplement (NCS-A): I. Background and measures. Journal of the American Academy of Child and Adolescent Psychiatry. 2009;48:367–369. [PMC free article] [PubMed]
  • Minsky S, Petti T, Gara M, Vega W, Lu W, Kiely G. Ethnicity and clinical psychiatric diagnosis in childhood. Administration and Policy in Mental Health. 2006;33:558–567. [PubMed]
  • Moffitt TE, Caspi A, Taylor A, Kokaua J, Milne BJ, Polanczyk G, Poulton R. How common are common mental disorders? Evidence that lifetime prevalence rates are doubled by prospective versus retrospective ascertainment. Psychological Medicine. 2010;40:899–909. [PMC free article] [PubMed]
  • Pina AA, Silverman WK. Clinical phenomenology, somatic symptoms, and distress in Hispanic/Latino and European American youths with anxiety disorders. Journal of Clinical Child and Adolescent Psychology. 2004;33:227–236. [PubMed]
  • Roberts RE, Roberts CR, Chen YR. Ethnocultural differences in prevalence of adolescent depression. American Journal of Community Psychology. 1997;25:95–110. [PubMed]
  • Rohde P, Lewinsohn PM, Seeley JR. Comparability of telephone and face-to-face interviews in assessing axis I and II disorders. American Journal of Psychiatry. 1997;154:1593–1598. [PubMed]
  • Sobin C, Weissman MM, Goldstein RB. Diagnostic interviewing for family studies: comparing telephone and face-to-face methods for the diagnosis of lifetime psychiatric disorders. Psychiatric Genetics. 1993;3:227–233.
  • Varela RE, Weems CF, Berman SL, Hensley L, de Bernal MCR. Internalizing symptoms in Latinos: the role of anxiety sensitivity. Journal of Youth and Adolescence. 2007;36:429–440.