|Home | About | Journals | Submit | Contact Us | Français|
Several screening tools for attention-deficit/hyperactivity disorder (ADHD) have been validated in non–substance-abusing populations, but limited data are available regarding their utility in adults with current substance use disorders. The aim of this study was to determine the sensitivity, specificity, and positive and negative predictive values of 3 commonly used ADHD screening instruments in cocaine-dependent individuals.
Adults seeking treatment for cocaine dependence (N = 102) were administered 3 self-report instruments between May 2009 and April 2011: the Conners Adult ADHD Rating Scale (CAARS), the Wender Utah Rating Scale (WURS), and the Adult ADHD Self-Report Scale-Version 1.1 (ASRS-V1.1). They then met with master’s-level clinicians who administered the Conners Adult ADHD Diagnostic Interview for DSM-IV (CAADID). With the CAADID serving as the gold standard, the validity of the screening instruments was determined, both singly and in combination.
Twenty-five (25%) of the 102 patients met DSM-IV criteria for ADHD or ADHD not otherwise specified (NOS) based on the CAADID. Kappa scores determining agreement between the screening tools and the CAADID (with ADHD NOS labeled as ADHD or labeled as not ADHD) ranged from 0.37 to 0.69. Sensitivity scores for the broadest range of ADHD cases were 80.0%, 87.5%, and 60.9% for the CAARS, WURS, and ASRS-V1.1, respectively. Positive predictive value was highest for the CAARS, at 74.1%, and negative predictive value was highest for the WURS, at 95.1%. The highest sensitivity (96.0%) was found with coadministration of the WURS and CAARS.
While all of the screening instruments were found to have adequate sensitivity and specificity, the CAARS outperformed the other instruments in regard to agreement with the CAADID and positive predictive values. However, the WURS, with the highest sensitivity in regard to the broadest range of ADHD cases, may be the single best instrument for preliminary screening purposes. Further, because the ASRS-V1.1 is the simplest and shortest instrument to administer, it may have advantages when a large number of patients need to be screened.
Attention-deficit/hyperactivity disorder (ADHD) is the most common childhood-onset neuropsychiatric disorder,1 with a prevalence in the US adult population of approximately 5%.2 It is associated with substantial morbidity, including high divorce rates, legal difficulties, and driving violations,3–5 as well as with high rates of psychiatric comorbidity.3,5–7 Of particular concern is the high prevalence of substance use disorders (SUDs) among adults with ADHD, estimated to be at least twice that of adults who do not have ADHD.2 Individuals with ADHD typically have a more severe SUD course8,9 and poorer treatment outcomes10–12 than do those without ADHD.
Although ADHD is overrepresented in individuals seeking SUD treatment,13,14 ADHD frequently goes unrecognized,15 primarily because it is not integrated into routine diagnostic protocols.16 Screening individuals with SUD for ADHD may be an efficient way to address this problem. Three commonly used instruments are the Wender Utah Rating Scale (WURS),17 which screens for ADHD using the Utah criteria; the Conners Adult ADHD Rating Scale (CAARS), in the self-report short version18; and the Adult ADHD Self-Report Scale-Version 1.1 (ASRS-V1.1).19 Several studies have shown the WURS to be internally consistent and a reliable screen for ADHD in general and treatment populations.17,20 Similarly, the CAARS and ASRS-V1.1 have been used clinically to screen for adult ADHD4,21–24 with good results. The ASRS-V1.1 has additionally been found to have good sensitivity (87.5%) in a small Spanish population of individuals engaged in SUD treatment25 and has been used to estimate prevalence in a residential SUD population.26 The Attention Deficit Scales for Adults (ADSA) has also been evaluated in SUD individuals and was found to have good sensitivity (0.71) and specificity (0.82) when a liberal cutoff was used (ie, a total score of 161).27
More information is needed, however, regarding the validity of these instruments in screening SUD individuals for ADHD. The ASRS-V1.1 has not been investigated as a screening instrument in SUD populations in the United States, nor have the WURS and CAARS. Given the unique challenges presented by those with SUD, including overlapping symptoms, diagnostic misattribution, and problems with recall,16,22,28 it is important to ascertain whether the validity of these instruments extends to the SUD population. The ADSA, in turn, is substantially longer, at 54 items, than are the above-mentioned instruments, potentially reducing its utility as a screening tool. Further, the sensitivity and specificity of the ADSA in SUD individuals were determined by reference to diagnoses made during a clinical interview, the reliability of which is unclear.27 Clinical interviews, as opposed to validated and standardized diagnostic interviews, may lead to inappropriate diagnosis given the possibility that substance-related symptoms might be incorrectly attributed to ADHD,22,28 or vice versa.
The purpose of this study was therefore to assess the clinical utility of 3 commonly administered instruments—the WURS, the CAARS, and the ASRS-V1.1—in a population of outpatient individuals seeking treatment for cocaine dependence. The Conners Adult ADHD Diagnostic Interview for DSM-IV (CAADID), a semistructured interview validated in clinical and research settings,29 served as the gold standard by which the validity of the instruments was tested. We predicted that all 3 instruments would have adequate sensitivity, specificity, and positive and negative predictive values.
The study was approved by the Institutional Review Board of the New York State Psychiatric Institute, and data were collected between May 2009 and April 2011. Participants for this study consisted of 102 individuals seeking out-patient treatment for cocaine dependence recruited through advertisements for treatment research studies at Columbia University/New York State Psychiatric Institute (NYSPI). Several studies concurrently recruited for cocaine-dependent patients at our research program; one targeted patients with ADHD and another focused on patients with major depressive disorder, and other studies enrolled cocaine-dependent individuals without these comorbidities. After providing informed consent, subjects were administered screening instruments followed by structured diagnostic interviews.
After completing screening instruments, participants were interviewed using the Structured Clinical Interview for DSM-IV (SCID-IV-TR)30 and the CAADID.29 The SCID-IV-TR is a well-established structured clinical interview that is often used in clinical treatment trials to test for a wide range of DSM-IV diagnoses. While it has numerous modules covering major diagnostic categories (eg, psychosis, affective illness, anxiety disorders, SUDs), it does not contain an ADHD module that has been validated to diagnose adult ADHD. Therefore, a widely used and validated semistructured interview, the CAADID, was also administered to all cocaine-dependent patients. The CAADID provides examples of adult symptoms to facilitate the interview and assesses age at onset, pervasiveness, and impairment in a structured manner. The CAADID also ascertains ADHD type: combined type, inattentive type, or hyperactive-impulsive type. Cases that approximated, but did not fully meet, DSM-IV ADHD criteria (eg, symptom onset in childhood, but not before 7 years of age; at least 4 symptoms, but fewer than 6) were labeled as ADHD not otherwise specified (NOS).
All diagnoses were determined by clinicians who had either a PhD or an MA degree in clinical psychology. Although clinicians had access to all screening information, they did not rely on this information in lieu of appropriately administering the CAADID. To assess fidelity, each interviewer completed written assessments that were used in supervision with a senior licensed clinical psychologist (A.M.). Further, all clinicians who carried out the diagnostic assessment for ADHD were part of a research team trained and overseen by the senior investigator (F.R.L.), who has extensive experience in diagnosing and treating individuals with ADHD. If there were questions regarding a diagnostic assessment, they were resolved at a team meeting in which the senior investigator and other psychiatrists experienced in diagnosing ADHD were present. If questions remained, these experienced psychiatrists would meet with the patient to reach a final decision. Moreover, all patients were administered the SCID-IV-TR to ensure that apparent ADHD symptoms were not better explained by another disorder. At least 10% of the SCID-IV-TR sessions and all of the CAADID interviews were audiotaped and rated by a senior clinician (A.M.) to determine interrater reliability and ensure that there was minimal diagnostic drift.
The WURS17 is a retrospective self-report instrument for adults originally consisting of 61 questions but shortened to 25 questions by its developers (Ward et al) to be administered more easily for screening purposes. Each item is rated 0 (not at all) to 4 (very much). It focuses on childhood symptoms of ADHD between ages 8 and 10 years using the Utah criteria for ADHD (inattention, hyperactivity, and impulsivity together with emotional dysregulation and conduct problems). A total score of ≥ 36 indicates possible ADHD.
The CAARS18 incorporates the 18 items of the ADHD diagnostic criteria to screen for ADHD. It is available in various versions; the self-report scale, in the short version, was used here. Each item is rated on a scale of 0 (not at all) to 3 (severe). For the short version of the CAARS, a score ≥ 23 indicates possible adult ADHD.
The shortest screen, the ASRS-V1.1, is an official instrument of the World Health Organization and has been used in large epidemiologic studies to identify possible adult ADHD.19 It consists of 6 items, each of which can be scaled from 0 to 4, and if an individual endorses 4 out of 6 items with adequate severity, then he or she screens positive for ADHD.
WURS, CAARS, and ASRS-V1.1 scores were dichotomized into “ADHD” or “no ADHD” depending on whether they exceeded the diagnostic cutoff for each measure. Similarly, CAADID assessments were dichotomized into “ADHD” or “no ADHD” based on the categorization of response indicators, with the combined, inattentive, and hyperactive categories subsumed into the “ADHD” group and with no diagnosis and ADHD NOS (for the first analysis) categorized as “no ADHD.” Using the CAADID-derived diagnosis as the reference, sensitivity (percentage of “true positives”), specificity (percentage of “true negatives”), positive predictive value (percentage of participants diagnosed with ADHD screened as positive), and negative predictive value (percentage of participants not diagnosed with ADHD screened as negative) were computed for the screening tools singly and in combination, arranged as column and row percentages of 2 × 2 frequency tables, and plotted to demonstrate the differences among the screening tools and their combinations. The same computations, tables, and plots were then carried out with ADHD NOS labeled as ADHD. Cohen κ was computed to determine the agreement between each of the screening tools and the CAADID. Kappa was also computed to determine the interrater reliability of the CAADID.
From May 2009 through April 2011, 102 participants reporting that cocaine was the primary drug of abuse signed a screening consent form and began the screening process to determine their eligibility for various outpatient treatment studies. During 1 to 2 outpatient visits, participants completed both the screening instruments for ADHD and the structured interviews (SCID-IV-TR and CAADID). Of these 102 participants meeting DSM-IV-TR criteria for cocaine dependence, 25 (25%) were found to have ADHD (15%) or ADHD NOS (10%) by the CAADID. Table 1 summarizes the sociodemographic information of the sample, as well as morbidity characteristics and baseline levels of cocaine use.
For each screening test, the sensitivity, specificity, positive predictive value, and negative predictive value were determined and plotted (Figure 1). In this analysis, ADHD NOS was labeled as not ADHD. Agreement with the CAADID was assessed for each instrument; κ scores closer to 1 indicate greater agreement. Table 2 summarizes the validity characteristics of the screening instruments in various combinations. Although all instruments had comparable characteristics, the CAARS outperformed the others in regard to all characteristics, including sensitivity. The most sensitive conjunctions (100%) arose when the CAARS was administered along with both of the other instruments, with a suggestive score on any single scale indicating the diagnosis; when the WURS was administered with the ASRS-V1.1; and when the WURS was administered with the CAARS. Interrater reliability for the CAADID was excellent, with κ = 0.96.
Analyses were repeated with the group diagnosed as having ADHD NOS by CAADID labeled as ADHD. Sensitivity, specificity, positive predictive value, negative predictive value, and agreement with the CAADID were determined and plotted independently for each screening instrument (Figure 2) and in various combinations (Table 3). The CAARS emerged with the highest κ scores and positive predictive value, but the WURS outperformed the other instruments in regard to sensitivity (87.5%). The most sensitive conjunctions arose (96.0%) when all instruments were administered together, with a suggestive score on any single scale indicating the diagnosis, and when the WURS was administered alongside the CAARS. Interrater reliability for the CAADID was excellent with κ = 0.98.
Although all screening instruments performed adequately, the CAARS outperformed the others in regard to most characteristics, including level of agreement with the CAADID, both when ADHD NOS was labeled as ADHD and when it was not. The sensitivity of the instruments was substantially strengthened when they were administered together, while the WURS demonstrated the highest sensitivity on its own for the broadest range of ADHD cases. Because high sensitivity is the most important feature of a preliminary screen, a case can be made to include the WURS in the first clinical or research encounter with SUD individuals. However, the ASRS-V1.1 is the easiest tool to administer and exhibited adequate sensitivity; it may therefore be preferred when a large number of individuals need to be screened quickly.
Before discussing these points further, it is worthwhile to consider various aspects of adult ADHD that uniquely affect the diagnostic process. ADHD is thought to persist into adulthood in about 65% of cases,31 and the disorder is most likely to first come to clinical attention in late adolescence or early adulthood, more than a decade after it emerged.32 The accurate recall of disorder onset—a crucial diagnostic criterion—might consequently be compromised. Further, ADHD is highly comorbid with other psychiatric conditions, including anxiety, affective, and personality disorders, with more than 90% of ADHD cases diagnosed with at least 1 other psychiatric condition.33 This presents the challenge of correctly attributing symptomatology to ADHD when other comorbid disorders may or may not better account for the impairment.
The acute and chronic effects of substances in actively using individuals constitute a related challenge. It may be difficult to delineate between substance-related phenomena and ADHD symptoms given the substantial overlap in clinical phenomena (eg, restlessness, emotional dysregulation, impulsivity) and because of problems with obtaining an accurate chronology of symptoms.16,22,28 Individuals with ADHD also have a more severe SUD course than do those without the disorder.8,9 For these reasons of inaccurate recall, incorrect diagnostic attribution, and increased morbidity and comorbidity, ADHD in adults with SUDs may be a difficult diagnosis to appropriately determine. A standardized set of semistructured diagnostic interviews validated for a range of diagnoses and overseen by experienced clinicians, as used here, is therefore optimal for diagnosing persistent ADHD in individuals with SUDs, although difficult to implement routinely in clinical practice.
A short, easily administered screening instrument may therefore be helpful in the preliminary diagnostic process. ADHD has been widely reported to be overrepresented in individuals seeking SUD treatment, with rates ranging from 10%–25%,13,14,22,34 consistent with the rate of ADHD in our sample (25% with ADHD NOS labeled as ADHD, and 14% when it was not). Although an argument can be made for rigorously evaluating all individuals with SUDs for ADHD16,28 given such high rates of co-occurrence, a more effective use of resources, particularly in clinical settings, would be to administer a reliable screening instrument followed by a standardized diagnostic interview for all probable cases.
Every instrument tested in this study demonstrated adequate sensitivity, specificity, and positive and negative predictive values, with the CAARS outperforming the rest overall, particularly when ADHD NOS was labeled as not ADHD, and exhibiting the greatest degree of agreement with the CAADID. Of the 3 instruments, the CAARS adheres to DSM-IV criteria in the most comprehensive manner, thus explaining its superior agreement with the CAADID. The ASRS-V1.1, also predicated on the DSM-IV, is much shorter at only 6 items, while the WURS draws on the Utah conceptualization of ADHD.
The sensitivity and specificity of all instruments changed when ADHD NOS was labeled as ADHD; most notably, the sensitivity of the CAARS dropped below that of the WURS. Importantly, individuals were designated as having ADHD NOS because they did not meet the very diagnostic criteria that are being most dramatically revised by the DSM-5 ADHD and Disruptive Behavior Disorders Work Group.35 Of ADHD NOS cases, 45% (n = 5) were designated as such because they had an age at onset greater than 7 years but less than 12 years; the other cases (n = 6) had fewer than 6 but more than 4 symptoms. Given recent research regarding persistent ADHD, the proposed changes to ADHD in the DSM-5 include changing the age at symptom onset to before 12 years old and decreasing the required number of symptoms to more than 4 in older adolescents and adults.31,36–38 The analysis incorporating ADHD NOS into the diagnosis is therefore more congruent with changing definitions of ADHD than is the analysis excluding it.
The purpose of the screening instrument is to provide guidance regarding who should subsequently receive a diagnostic interview, a procedure necessary for properly diagnosing ADHD. The most important aims of the instrument are thus to detect ADHD in the maximum number of affected individuals and to miss the disorder in a minimum number. As such, sensitivity is the most important factor to consider in evaluating clinical utility in this setting.39 In the analysis including ADHD NOS, the sensitivity of the instruments ranged from 60.9% to 87.5%, with the WURS demonstrating superior performance. It may be that the same feature of the WURS—its grounding in a non– DSM-IV–based conceptualization of ADHD—that compromised its agreement with the CAADID could also explain its superiority in capturing cases of ADHD more consistent with proposed DSM-5 criteria.
The WURS, however, is lengthy, at 25 items, and clinicians or researchers may decide that a shorter instrument, such as the 6-item ASRS-V1.1, even though demonstrating lower, marginally adequate sensitivity (60.9%), represents a better tool when quickly screening a large number of individuals. The ASRS-V1.1 was found to have high sensitivity (87.5%) in a previous study with Spanish substance users that also used the CAADID-derived diagnosis as reference,25 a value substantially higher than the sensitivity determined in a general population (68.7%) for reasons that are unclear.24 It is important to note, however, that our analysis indicates that the ASRS-V1.1 may miss 39.1% of individuals with ADHD, a substantial shortcoming that might not be justified by its short length and ease of administration. The WURS, by contrast, missed 12.5% of cases, and the CAARS missed 20%. Further, sensitivity improved significantly (96%) when the WURS was administered along with the CAARS, with a suggestive score on either scale indicating the diagnosis.
These findings should be interpreted in the context of several limitations. First, the study population was composed entirely of cocaine-dependent individuals; these results may therefore not be representative of individuals seeking treatment for nonstimulant substances such as cannabis or opioids. Future studies can aim to clarify the performance of these instruments in a wider range of substance users. Second, because the instruments were administered to all screening participants in the same order, one cannot exclude an order effect. Third, it is not possible to determine the extent to which psychiatric comorbidity, ADHD severity or subtype, and amount of cocaine use affect the validity of the instruments. A larger sample size may have allowed for making comparisons between subgroups. Finally, psychiatric diagnoses were based entirely on self-report from participants, and collateral information, academic/work records, and medical records were not available. Although the CAADID has been validated, previous research indicates that failing to obtain these other sources of history leads to lower sensitivity.2,40
Despite these limitations, this analysis of a well- characterized outpatient sample demonstrates that the WURS, ASRS-V1.1, and CAARS may be reliably administered as screening instruments to individuals seeking SUD treatment. The superior sensitivity of the WURS for the broadest range of ADHD cases indicates that it may be the single best instrument to determine who should proceed to a diagnostic interview for ADHD. As the shortest instrument, the ASRS-V1.1 has certain advantages, but it exhibited the lowest, albeit marginally adequate, sensitivity in all analyses. Clinicians treating substance users can therefore be confident that incorporating any or all of these instruments into the preliminary diagnostic process will increase their capacity to detect ADHD and, with proper interview-guided diagnosis and ADHD management, improve quality of life in affected individuals, including the course of their SUD treatment.
Funding/support: This study was supported by National Institute on Drug Abuse grants DA031771 (to Dr Dakwar), DA007294 and DA029647 (to Dr Levin), and DA023651 (to Dr Grabowski) and by the New York State Psychiatric Institute (Drs Dakwar, Mariani, and Levin).
Author contributions: All authors listed have contributed to the writing, analysis, editing, and/or proofreading of the manuscript. Dr Dakwar had full access to the data and assumes full responsibility for the integrity of the analysis.
Potential conflicts of interest: Dr Levin is a consultant to GW Pharmaceuticals and has received medication from US World Meds for a National Institutes of Health–funded study. Drs Dakwar, Pavlicova, Mariani, and Grabowski; Ms Mahony; and Messrs Glass and Brooks report no potential conflict of interest.