PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptNIH Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
J Psychiatr Res. Author manuscript; available in PMC Mar 1, 2013.
Published in final edited form as:
PMCID: PMC3289153
NIHMSID: NIHMS343201
Alcohol and drug dependence symptom items as brief screeners for substance use disorders: results from the Clinical Trials Network
Li-Tzy Wu,a* Dan G. Blazer,a George E. Woody,b Bruce Burchett,a Chongming Yang,c Jeng-Jong Pan,d and Walter Linge
aDepartment of Psychiatry and Behavioral Sciences, School of Medicine, Duke University Medical Center, Durham, NC, USA
bDepartment of Psychiatry, School of Medicine, University of Pennsylvania and Treatment Research Institute, Philadelphia, PA, USA
cSocial Science Research Institute, Duke University, Durham, NC, USA
dNational Cancer Institute, National Institutes of Health, Bethesda, MD, USA
eDavid Geffen School of Medicine, NPI/Integrated Substance Abuse Programs, University of California, Los Angeles, CA, USA
*Corresponding author: Li-Tzy Wu, Sc.D., Department of Psychiatry and Behavioral Sciences, School of Medicine, Duke University Medical Center, Box 3419, Durham, NC 27710, USA; tel.: 919-668-6067; litzy.wu/at/duke.edu
Aim
To address an urgent need for screening of substance use problems in medical settings, we examined substance-specific dependence criteria as potential brief screeners for the detection of patients with a substance use disorder (SUD).
Methods
The sample included 920 opioid-dependent adults who were recruited from outpatient treatment settings at 11 programs in 10 U.S. cities and who completed intake assessments of SUDs for a multisite study of the National Drug Abuse Treatment Clinical Trials Network (CTN003). Data were analyzed by factor analysis, item response theory (IRT), sensitivity, and specificity procedures.
Results
Across all substances (alcohol, amphetamines, cannabis, cocaine, sedatives), withdrawal was among the least prevalent symptoms, while taking large amounts and inability to cut down were among the most prevalent symptoms. Items closely related to the latent trait of a SUD showed good-to-high values of area under the receiver operating characteristic curve in identifying cases of a SUD; IRT-defined severe and less discriminative items exhibited low sensitivity in identifying cases of a SUD (withdrawal for all substances; time using for alcohol and sedatives; giving up activities for sedatives).
Conclusions
Study results suggest that withdrawal and time using are much less reliable indicators for a SUD than taking larger amounts than intended and inability to cut down and should be studied further for consideration in developing a simplified tool for screening patients for SUDs in medical settings. These findings have implications for the use of common health indicators in electronic health records systems to improve patient care.
Keywords: clinical trials network, item response theory, receiver operating characteristic curve, brief screening, sensitivity, specificity, substance use disorders
Effective screening and intervention for substance use disorders (SUDs) in health care settings constitute a national priority. The new National Drug Control Strategy, as well as the Patient Protection and Affordable Care Act (PPACA) of 2010, emphasize prevention and integration of SUD interventions and treatments into the mainstream health care system (Substance Abuse and Mental Health Services Administration [SAMHSA], 2010a; Office of National Drug Control Policy, 2010). Consequently, access to SUD treatments and the quality of care will likely be enhanced. However, primary care physicians are very busy and have many areas to assess in a relatively short time; thus, the potential benefits of integrating SUD prevention and treatment into primary care will not be realized unless procedures to screen for SUDs are identified that can easily fit into busy work schedules.
National data have shown that the vast majority of individuals with a SUD do not use any treatment in a 12-month period; financial (cost, insurance coverage) and psychological (stigma and denial) barriers are pervasive (SAMHSA, 2010b; Wu, 2010; Wu et al., 2007; Wu & Ringwalt, 2004; Wu et al., 2011), suggesting that routine screening for SUDs can be useful in promoting treatment use and reducing SUD-related problems (Madras et al., 2009; U.S. Preventive Services Task Force, 2004). The U.S. Preventive Services Task Force has recommended screening and intervention for alcohol use problems in primary care settings (U.S. Preventive Services Task Force, 2004). However, empirical data are limited to guide similar efforts for drug use problems, despite the fact that drug use is also a leading risk factor for the global burden of disability, increases health care costs, and can cause multiple medical illnesses and premature deaths (Brick, 2004; Tiet et al., 2008; World Health Organization, 2009). Lack of simplified and effective screening tools targeting drug use problems is clearly a hindrance to incorporating treatment for SUDs into routine care (Saitz et al., 2010; Tiet et al., 2008).
For practical and clinical considerations, a stepped strategy for screening, combining an initial “substance use” screen and successive “substance-related problem” screeners to detect SUDs for intervention, is recommended (Whitlock et al., 2004). For example, national survey data estimate that 9% of American adults aged 18 years or older have an alcohol or drug use disorder within a 12-month period (SAMHSA, 2011). In busy health care settings, it would be burdensome to screen all individuals for SUDs; instead, a more efficient practice would be to target the detection of SUDs at individuals who screen positive for substance use. This approach would facilitate timely intervention or treatment referral, especially among individuals in emergency room or psychiatric treatment settings where SUDs are comparatively prevalent but individuals are under-diagnosed or under-treated because SUDs have not been systematically assessed (Rockett et al., 2003; Tiet et al., 2008; Vinson et al., 2007).
To date, researchers have investigated a single-question screener for substance use in primary care (Smith et al., 2009; Smith et al., 2010), as well as a two-item screener for SUDs to facilitate detection of SUDs and treatment (Brown et al., 2001). The single-question screener identifies substance use (“How many times in the past year have you used an illegal drug or used a prescription medication for nonmedical reasons?”); a response of at least one time to the question is considered positive for drug use (Smith et al., 2010). This single screening question was found to have a high level of accuracy in identifying drug use in primary care patients, supporting its use as a brief screen (self-reported current drug use: sensitivity=92.9; specificity=94.1). The two-item screener includes questions similar to the Diagnostic and Statistical Manual of Mental Disorders-IV (DSM-IV) dependence criteria to identify individuals with a SUD (“In the last year, have you ever drunk or used drugs more than you meant to?” and “Have you felt you wanted or needed to cut down on your drinking or drug use in the last year?”). Brown et al. (2001) found that having at least one positive response to the two-item screener detected a current SUD with nearly 80% sensitivity and specificity in primary care settings, suggesting the feasibility of integrating the two dependence-related questions into a clinical interview. However, the two-item screen has not been continuously tested to further establish its utility. While the Drug Abuse Screening Test (DAST: 10-, 20-, 28-item) has been used for assessing or screening severity (low, medium, high) of drug use problems (Skinner, 1982; Yudko et al., 2007), its length has impeded widespread adoption in clinical practice. To be adopted for routine use, a short and simplified tool that is sensitive to drug use problems and useful for clinical decision-making, such as identifying those with a clinical diagnosis, is needed. Substance dependence symptoms are robust manifestations of compulsive drug use and less prone to measurement biases than abuse symptoms (Babor and Caetano, 2008; Koob and Le Moal, 2006); thus, they are good candidate screeners for detecting individuals with a high probability of having a SUD, as shown by Brown et al. (2001).
Implementation of the National Drug Control Strategy and current health care reform necessitates the expansion of screening for drug use problems to promote treatment of SUDs. To address this need, we examined substance dependence criteria as brief screeners for SUDs among 920 opioid-dependent adults. The data are drawn from an 11-site, randomized treatment trial for opioid dependence within the National Drug Abuse Treatment Clinical Trials Network (CTN). Data from this study include a geographically diverse sample of drug users with sufficient prevalences of substance-related problems that were assessed by the same diagnostic instrument so as to provide useful data about the relative values of screening items for alcohol and multiple drug use disorders commonly seen in clinical settings. In this study, we apply two-parameter item response theory (IRT), sensitivity, and specificity analyses to examine substance dependence criteria as potential screeners for SUDs. This IRT approach assumes that a latent severity trait underlies response patterns of SUDs measured by DSM-IV criteria, and recognizes item-level (criterion) variability in the discrimination and severity of the SUD severity continuum (Saha et al., 2006; Wu et al., 2009a; Wu et al., 2009b; Wu et al., 2009c). IRT analysis determines the unidimensional latent trait underlying criterion symptoms for each disorder (a one-factor syndrome), but also provides vital item-level discrimination and severity estimates for substance-specific criteria (Wu et al., 2009a; Wu et al., 2009b). IRT findings consistently show that dependence symptoms measure various ranges of a “similar” condition underlying SUD problems (Saha et al., 2006; Wu et al., 2009a; Wu et al., 2009b), indicating the presence of poor items and feasibility to identify “core” dependence items for use as a simplified screen for detecting individuals at risk for SUDs. Assuming that an IRT approach to SUDs is feasible, poorly performing items from IRT analysis (low discrimination, high severity) will have low sensitivity/specificity values in identifying individuals with a disorder; reliable items across multiple substances will be good indicators for problematic use that are likely to meet criteria for a SUD.
Specifically, here we 1) examine prevalences of substance dependence and substance-specific symptoms across multiple substances, 2) determine the unidimensionality of dependence criteria and item-level discrimination and severity estimates, and 3) calculate sensitivity, specificity, and area under the receiver operating characteristic curve (ROC-AUC) for each dependence criterion in opioid-dependent adults. ROC-AUC considers the trade-off between sensitivity and specificity values and indicates the level of accuracy in classifying cases and non-cases of a SUD according to the DSM-IV checklist (1.0=perfect classification). We hypothesize that items showing low discrimination and high severity estimates (low relevancy to the underlying trait of a disorder) will have comparatively low sensitivity/specificity (ROC-AUC) values in identifying cases of a SUD.
2.1. Data source
The National Drug Abuse Treatment Clinical Trials Network (CTN) includes 13 nodes (research centers) allied with substance abuse treatment providers in 39 states across the United States, the District of Columbia, and Puerto Rico (Tai et al., 2011). For this study, analyses were performed on data from a large, multisite CTN study that evaluated two buprenorphine/naloxone taper schedules for opioid detoxification in 11 outpatient, community-based treatment programs (Ling et al., 2009). Eligible participants were at least 15 years of age and seeking treatment for opioid dependence at one of the participating treatment programs in 10 U.S. cities (Colorado, Connecticut, New York, North Carolina, Oregon, Virginia, and Washington). Recruitment methods included word of mouth, radio announcements, newspaper advertisements, and referrals from local opioid treatment and outreach programs, alcohol and drug abuse clinics, primary care providers, local mental health centers, crisis clinics, and hospital emergency rooms. The sample included 920 opioid-dependent adults from the intake assessment. Opioid-dependent adults who did not meet eligibility criteria were also included in the analysis, which both enabled us to ascertain whether substance use status differed by randomization status and provided a larger sample size for the analysis. The Duke University Institutional Review Board approved use of these data for this study.
2.2. Study variables
Demographics included age, sex, race/ethnicity, education, and employment status, which were assessed by the CTN common demographic form. Past-year substance use and disorders (abuse or dependence on alcohol, cannabis, cocaine, inhalants, amphetamines, sedatives, hallucinogens) were assessed by the DSM-IV Checklist (Hudziak et al., 1993). At intake, all participants were administered the DSM-IV Checklist by CTN-affiliated trained interviewers (research staff who completed training for administering the DSM-IV Checklist) to assess substance use in the past year. Participants who responded affirmatively to the initial substance use question (“Have you ever used [NAME OF THE SUBSTANCE] in the past 12 months?”) then were assessed for seven substance-specific dependence criteria. Endorsement of at least three of the seven DSM-IV dependence criteria resulted in a dependence diagnosis (tolerance, withdrawal, substance often taken in large amounts/for longer periods of time, persistent desire or unsuccessful attempt to cut down, a great deal of time spent in activities necessary to get the substance, important activities given up, continued substance use despite knowledge of having recurrent physical or psychological problems). Following DSM-IV logic, individuals who did not meet the criteria for a SUD were then assessed for abuse.
The analysis focused on DSM-IV dependence criteria because dependence symptoms reflect physical and/or psychological dependence and a pattern of compulsive use, which are directly related to the extent of substance use, while abuse symptoms (role interference, hazardous use, legal problems, relationship problems) concern social or legal consequences of drug use behaviors, which are more likely to be affected and biased by a user’s age and environmental factors (Babor and Caetano, 2008). DSM-IV and the developing DSM-5 comprise the most widely used criteria for health statistics in the United States (American Psychiatric Association [APA], 2000); questions of dependence criteria, thus, are ideal candidates for brief screeners as clinicians already use the criteria and would not need to learn a new set of questions and rules.
2.3. Data analyses
Distributions of study variables and potential differences in sociodemographic and substance use status by the randomization status were examined by descriptive analyses. Prevalence rates of substance dependence in the total sample and among substance users (conditional rates of substance use disorders) then were calculated.
Among substance users, discrete factor analysis for categorical data was conducted using Mplus (v. 6; Muthén and Muthén, 2010) to describe factor loadings of dependence symptoms. The Tucker-Lewis index (TLI), comparative fit index (CFI), standardized root mean square residual (SRMR), and the root mean square error of approximation (RMSEA) were used to assess the model fit for a one-factor model, as substance dependence is generally considered a single factor (Edwards and Gross, 1976; Wu et al., 2009a; Wu et al., 2009b). A higher value of TLI and CFI (>0.94) and a lower value of SRMR (<0.8) and RMSEA (<0.07) indicate an excellent fit to the data (Browne and Cudeck, 1993; Hu and Bentler, 1999). The one-factor model for substance dependence also was assessed by the scree test (Cattell, 1966) and the ratio of the first to the second eigenvalues.
Two-parameter IRT analysis then was conducted to determine the unidimensionality (one factor) of each dependence, as well as item-level discrimination and severity (Edwards and Gross, 1976; Wu et al., 2009a). Seven substance-specific dependence criteria (dichotomous variables) among past-year users of the corresponding substance were analyzed; hallucinogen use (6.3% of the sample, n=58) and inhalant use (1.2%, n=11) were not examined. The two-parameter IRT model assumes that item responses are a function of discrimination and severity of the items included. Item severity (threshold) is defined by the location on a latent continuum at which an item has a 50% likelihood of being present; item discrimination measures the precision with which an item differentiates between individuals with levels of the latent trait above or below its severity level.
Lastly, we calculated sensitivity (the proportion of individuals with a substance-specific SUD according to the DSM-IV Checklist who had a positive response to the item), specificity (the proportion of individuals without a substance-specific SUD according to the DSM-IV Checklist who had a negative response to the item), and the ROC-AUC for each item. A high ROC-AUC value indicates a high level of accuracy in classifying cases versus non-cases of a SUD according to the DSM-IV Checklist (1.0=perfect classification; <0.7=poor discrimination) (Hanley and McNeil, 1982).
3.1. Selected characteristics of opioid-dependent adults
Of the 920 opioid-dependent adults, 55.9% were randomized patients (Table 1). The randomized group was slightly younger (mean age: 35.5 vs. 36.9 years, p=0.04) and included more employed individuals (55.0% vs. 41.5%, p<0.001) than the non-randomized group (opioid-dependent adults who either did not meet the full inclusion/exclusion criteria or who met the full criteria but did not continue for personal reasons). The groups did not differ in sex, race/ethnicity, years of education, marital status, and past-year alcohol or drug use. Overall, 67.1% were men, 31.2% were nonwhite (13.0%, African-American; 10.0%, Hispanic; 8.2%, other), 46.1% received some post-secondary education, and 45.7% were never married.
Table 1
Table 1
Demographic characteristics of opioid-dependent adults by randomization status (N = 920)
3.2. Past-year substance use and dependence among opioid-dependent adults
Past-year use of alcohol (73.0%), cocaine (65.8%), cannabis (54.0%), and sedatives (45.5%) was more prevalent than use of amphetamines (17.0%), hallucinogens (6.3%), or inhalants (1.2%). Overall, 22.7% met criteria for cocaine dependence, and lower proportions met criteria for dependence on alcohol (8.5%), cannabis (6.0%), amphetamines (3.9%), sedatives (3.5%), hallucinogens (0.2%), and inhalants (0%). Among the subset that used the substance in the past year (Table 2), conditional rates of substance-specific dependence were higher among cocaine (34.6%) and amphetamine (23.1%) users than among alcohol (11.6%), cannabis (11.1%), sedative (7.6%), or hallucinogen (3.5%) users.
Table 2
Table 2
Past-year prevalence of substance dependence symptoms and dependence among past-year users of the corresponding substance
3.3. Exploratory factor analysis of dependence symptom items
Across five substances (Fig. 1), withdrawal (3.4–12.8%) and time using (2.6–22.2%) were among the least prevalent symptoms among substance users; taking large amounts (10.5–45.8%) and inability to cut down (14.8–37.6%) were comparatively prevalent. Hallucinogens and inhalants were dropped from further analysis because of low prevalence. CFI, TLI, RMSEA, and the ratio of the first to the second eigenvalues from factor analyses (Table 3) indicated one dependence factor for each substance examined (factor loadings: alcohol, 0.76–0.95; amphetamines, 0.76–0.93; cannabis, 0.54– 0.96; cocaine, 0.75–0.93; sedatives, 0.75–0.93). Withdrawal had the lowest value of factor loadings (0.54) for cannabis dependence.
Fig. 1
Fig. 1
Past-year prevalence (%) of substance-specific dependence symptoms among substance users in a sample of treatment-seeking, opioid-dependent adults (N=920).
Table 3
Table 3
Exploratory factor analysis (EFA) of dependence symptoms among past-year users of the corresponding substance
3.4. IRT analysis and ROC-AUC of dependence symptom items
As shown in item characteristic curves (ICCs) that considered item-level discrimination and severity estimates simultaneously (Fig. 2a–2e), withdrawal (particularly for cannabis and cocaine dependence) and time using were among the most severe symptoms of the IRT-defined latent trait problems (shifted to right side); taking larger amounts was among the least severe indicators and was accompanied by a high level of discrimination estimates for all substances (steep lines).
Fig. 2
Fig. 2
Fig. 2
Fig. 2
Fig. 2a. An item response analysis of alcohol dependence symptoms among 672 alcohol users in a sample of treatment-seeking, opioid-dependent adults.
To explore whether the IRT item-level estimates suggested similar information as sensitivity/specificity analyses (Table 4), ROC-AUC was calculated (Fig. 3). ROC-AUC considers sensitivity and specificity together; a value of 1.0 (100%) indicates that the item accurately classifies cases of a SUD defined by the DSM-IV Checklist. Consistent with IRT results, items that shifted to the severe ends on ICCs (Fig. 2a–2e) showed low ROC-AUC values (≤0.70): withdrawal for alcohol, cannabis, cocaine, and sedatives; time using for alcohol and sedatives; and giving up activities for sedatives (Fig. 3). Taking larger amounts (high discrimination, low-to-medium severity on ICCs) demonstrated good-to-excellent levels of classification (ROC-AUC >0.80), especially for alcohol (ROC-AUC =0.91) and cannabis (ROC-AUC=0.90); tolerance (high discrimination, medium severity on ICCs) showed an excellent classification (ROC-AUC=0.93) for sedatives.
Table 4
Table 4
Item response theory (IRT) analysis of dependence symptoms and area under the receiver operating characteristic curve (ROC-AUC) among past-year users of the corresponding substance
Fig. 3
Fig. 3
Area under the receiver operating characteristic curve (ROC-AUC) for classifying cases versus non-cases of a substance use disorder, by substance-specific dependence symptoms. Lines extending from bars indicate 95% confidence intervals of the estimates. (more ...)
3.5. Sensitivity and specificity of dependence symptom items
To help evaluate ROC-AUC estimates, sensitivity and specificity estimates are presented in Fig. 4a and 4b. All items exhibited good-to-excellent levels of specificity (0.79–1.0), indicating that individuals who did not endorse the item were unlikely to have a SUD. However, Fig. 4a shows that three items had low-to-poor sensitivity (withdrawal and time using for all five substances; giving up activities for cannabis and sedatives), indicating many individuals with a SUD did not endorse the items.
Fig. 4
Fig. 4
Fig. 4a. Sensitivity of classifying cases versus non-cases of a substance use disorder, by substance-specific dependence symptoms. Lines extending from bars indicate 95% confidence intervals of the estimates.
4.1. Main findings
To address the urgent need for brief screening tools in keeping with the national priority to improve screening and treatment for SUDs (SAMHSA, 2010a; Office of National Drug Control Policy, 2010), we evaluated item-level psychometric (IRT) and classification (ROC-AUC) information of substance-specific dependence questions for five SUDs in a large sample of treatment-seeking, opioid-dependent adults. Across five substances, withdrawal was among the least prevalent symptoms, while taking large amounts and inability to cut down were among the most prevalent symptoms. IRT analysis provided support for the concept that items closely related to the underlying latent trait of a SUD showed good-to-high ROC-AUC values in identifying cases of SUDs (taking large amounts, inability to cut down, tolerance, medical/psychological problems), and severe and less discriminative items exhibited low sensitivity and a fair ROC-AUC in identifying a SUD (withdrawal for all substances; time using for alcohol and sedatives; giving up activities for sedatives). These results demonstrate the feasibility of applying IRT and ROC-AUC procedures to select dependence symptom items to develop an efficient, reasonably sensitive, and simplified tool to screen for SUDs in medical settings. Findings also add to the evidence for single-item screeners for “alcohol or drug use” that have shown good sensitivity and specificity in primary care settings (Smith et al., 2009; Smith et al., 2010). Therefore, expanding prior research on the two-item screener for SUDs (Brown et al., 2001), these combined results from IRT and ROC-AUC analyses suggest the value of using taking large amounts and inability to cut down as part of a simplified screener to facilitate detection of problematic substance users with a high probability for having a SUD (National Institutes of Health, 2011).
4.2. IRT and ROC-AUC indicate similar item-level results across substances
Due to competing priorities and time constraints in medical settings, brevity is crucial for incorporating a screen into routine clinical practice; the need for a brief tool for identifying SUDs has been emphasized (Bush et al., 1998; Tiet et al., 2008; Vinson et al., 2007). For instance, Brown et al. (2001) compared five screening questions for alcohol and/or drug use disorders (blackouts, use alcohol/drugs more than intended, use for feelings, need to cut down, regret). Use more than intended and need to cut down showed higher sensitivity than the others; both were then combined into a two-question screener for SUDs. When one positive response was taken as a positive result, the two-question screener yielded sensitivity and specificity of 79% and 78%, respectively, for identifying a SUD thus suggesting its utility in clinical practice (Brown et al., 2001). The two-question screener, however, does not distinguish between alcohol and drug use, and its generalizability to the range of drug use disorders is unclear (Brown et al., 2001).
To address this gap, we examined substance-specific screening questions and observed somewhat consistent findings across SUDs. IRT analysis revealed that severe items—such as withdrawal and time using, and to a lesser extent, giving up activities—had low sensitivity in identifying a SUD, suggesting that use of these items as brief screeners may disproportionately miss cases of SUD (false negative). For example, sensitivity for withdrawal ranged from 0.22–0.53, indicating that about 47–78% of individuals who met criteria for a SUD in this sample did not endorse withdrawal, which can be explained partly by its low prevalence (3.4–12.8%); this low rate indicated severe (more rare) items in IRT analysis. On the other hand, taking large amounts, inability to cut down, tolerance, and medical/psychological problems showed good-to-high discrimination values and clustered at the low-to-medium end of the IRT-defined SUD problems. These comparatively prevalent items demonstrated good-to-high ROC-AUC values in specifying cases of a SUD. For instance, sensitivity for taking larger amounts was 0.91 for alcohol, indicating that 91% of alcohol users with alcohol abuse or dependence responded affirmatively to this item. Given the consistency across SUDs, these items can be studied further for use as brief screening items for SUDs, as a positive response to one identified a high proportion of individuals with a SUD in this sample. Thus, IRT and ROC-AUC complement one another by demonstrating that poorly performing items are less reliable indicators for a SUD.
4.3. Implications for SUD screening and use of electronic health records (EHR)
These results have timely implications for the development of EHR systems and collection of “common data elements” or health indicators for SUD screening and intervention to improve patient care (Brown et al. 2001; Ghitza et al., 2011a; Ghitza et al., 2011b). Current practices generally rely on a two-step strategy (Whitlock et al., 2004). Individuals are assessed initially by a single-item screener for either alcohol or drug use status; the subset of users then is screened for SUD-related problems (e.g., CAGE, Alcohol Use Disorders Identification Test [AUDIT], DAST). However, in addition to screening for substance use and SUDs, a long list of health conditions has been recommended for screening and intervention by the U.S. Preventive Services Task Force. These competing priorities, plus the complexity of patients’ conditions, constitute barriers to screening completely for various conditions as each screener takes time to complete. Therefore, brevity and accuracy in detecting SUDs are critical considerations for adoption, as seen in the Health Information Technology for Economic & Clinical Health Act and the PPACA, which encourage the development and meaningful use of EHR to improve patient care (i.e., collections of standardized, patient-reported health indicators or common data elements) (Ghitza et al., 2011b; National Institutes of Health, 2011). Common data elements should include validated or standardized items, with brevity as a primary consideration (e.g., use of a minimal number of screening questions, combining screening items for alcohol and drugs) (Ghitza et al., 2011b).
Presently, single-item screeners for “substance use” have been identified (Smith et al., 2009; Smith et al., 2010), but simplified screeners for SUDs are lacking. Our results suggest that several dependence criteria are candidate common data elements that can be used to that end as they not only meet the principles of brevity and standardization but also distinguish between cases and non-cases of a SUD (abuse or dependence). For example, regardless of substance, a positive response to a dependence criterion (e.g., taking large amounts, inability to cut down, tolerance, medical/psychological problems) detected a high proportion of substance users with a SUD (abuse or dependence). The consistency in results and high levels of specificity support use for assessing SUDs. The next step would be to identify a brief set of dependence items (with or without combining alcohol and drugs) that can be used to screen for that diagnosis in health care settings, particularly emergency rooms and mental health settings where SUDs are comparatively prevalent but under-detected and undertreated (Rockett et al., 2003; Tiet et al., 2008; Vinson et al., 2007).
Lastly, cocaine dependence was the most prevalent comorbid SUD in this sample. Co-abuse of opioids and cocaine can increase drug-related adverse effects and poses a unique treatment challenge (Leri et al., 2003; Mello and Negus, 2007). These data suggest that taking large amounts and inability to cut down are useful items for detecting patients with cocaine use disorders to facilitate treatment management.
4.4. Limitations and strengths
These results are based on treatment-seeking, opioid-dependent adults, and the sample is not necessarily representative of all adults with a SUD. Treatment-seeking individuals may have different characteristics than persons who are being screened in primary care settings but who do not present seeking care for SUDs. While our results are consistent across substance classes, replication among substance users in different settings, including non-opioid dependent individuals, is needed. Another limitation is reliance on participants’ self-reports, which are subject to reporting bias; however, biomarkers are not available for SUD screening and diagnosis. Because assessments of SUDs rely on self-reports, IRT analysis can provide independent, psychometric evaluation for self-reported measures (Wu et al., 2009a; Wu et al., 2010). Lastly, the small sample size of hallucinogen (n=58) or inhalant (n=11) users precluded detailed analysis and interpretation of results.
Strengths include the rigorous conduct of the original trial (standardized assessments, comprehensive research staff training, protocol monitoring, regulatory control); geographic diversity (from 11 cities across the nation); and inclusion of all opioid-dependent participants from study intake in the analysis, thus making the sample more heterogeneous than that found at a single site. All participants were assessed for substance-specific SUDs using the same instrument, allowing for comparisons of all dependence criteria across various substances. Such comparisons provide vital information regarding whether a similar set of questions can apply to alcohol and various drugs, as prior research has focused on two dependence-related questions and combined alcohol and drug use into a single question (Brown et al., 2001). Another strength is the assessment of the withdrawal criterion for cannabis, hallucinogen, and inhalant use disorders, permitting comparison of this criterion with others. Withdrawal is not considered present for these SUDs in the DSM-IV (APA, 2000), but data are needed to evaluate its inclusion in DSM-5 with regards to prevalence and screening efforts.
4.5. Conclusions
While IRT analysis has played a key role in the psychometric evaluation of health-related item banks for the Patient-Reported Outcomes Measurement Information System (PROMIS), it generally has not been used in research on screeners for SUDs to support the underlying construct of patient-reported measures. This paper demonstrates the unique value of IRT analysis in determining the underlying trait of a set of measures and elucidating item-level psychometric information to help interpret results of sensitivity and specificity. As a sound measure of patient-reported items is fundamental to assessing health conditions like SUD (Wu et al., 2009a), researchers should consider incorporating IRT approaches into their analysis plans to support the psychometric quality of patient-reported items. Finally, across five substances, two DSM-IV dependence items (taking large amounts, inability to cut down) are good candidates for harmonized common data elements in SUD screening in this large treatment-seeking sample, while withdrawal and time-spent in using the substance are poor candidates as they miss many cases of SUDs.
Acknowledgments
The authors thank study participants, CTN staff and investigators who contributed to the original studies, and Amanda McMillan, MA, MPH, for her editorial assistance.
Role of the funding source: This work was made possible by research grants from the U.S. National Institute on Drug Abuse of the National Institutes of Health: R21DA027503, R33DA027503, R01DA019623, and R01DA019901 (PI: Li-Tzy Wu); U10 DA013045 (PI: Walter Ling); HSN271200522071C (PI: Dan G. Blazer); K05DA017009 and U10DA013043 (PI: George E. Woody). The sponsoring agency had no further role in the study design and analysis, the writing of the report, or the decision to submit the paper for publication. The opinions expressed in this paper are solely those of the authors.
Walter Ling has received education grants from Reckitt/Benckiser and research support from Reckitt/Benckiser and Hythiam Inc; he has also served as an occasional consultant to Reckitt/Benckiser, Titan Pharmaceuticals, US World Med, Alkermes, and DemeRx. George E. Woody is a member of the RADARS (Researched Abuse, Diversion, and Addiction-Related Surveillance) postmarketing study scientific advisory group assessing abuse of prescription medications. Denver Health administers RADARS, and pharmaceutical companies support its work.
Footnotes
This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Contributors: L-T Wu developed research aims and questions, designed the analysis plan, and wrote the drafts of the paper. L-T Wu, B Burchett, C Yang, and J-J Pan contributed to data analysis. W Ling was the principal investigator of the original trial. DG Blazer and GE Woody were principal investigators of the National Drug Abuse Treatment Clinical Trials Network. All authors contributed to interpretation of the findings and critical revision of the final manuscript.
Disclosures: The other authors have no conflicts of interest to disclose.
  • American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders, 4th ed., text revision. Washington, DC: American Psychiatric Association; 2000.
  • Babor TF, Caetano R. The trouble with alcohol abuse: what are we trying to measure, diagnose, count and prevent? Addiction. 2008;103:1057–1059. [PubMed]
  • Brick J. Handbook of the Medical Consequences of Alcohol and Drug Abuse. New York: The Haworth Press; 2004.
  • Brown RL, Leonard T, Saunders LA, Papasouliotis O. A two-item conjoint screen for alcohol and other drug problems. Journal of the American Board of Family Practice. 2001;14:95–106. [PubMed]
  • Browne MW, Cudeck R. Alternative ways of assessing model fit. In: Bollen KA, Long JS, editors. Testing Structural Equation Models. Newbury Park, CA: Sage Publications; 1993. pp. 136–162.
  • Bush K, Kivlahan DR, McDonell MB, Fihn SD, Bradley KA. The AUDIT alcohol consumption questions (AUDIT-C): an effective brief screening test for problem drinking. Ambulatory Care Quality Improvement Project (ACQUIP). Alcohol Use Disorders Identification Test. Archives of Internal Medicine. 1998;158:1789–1795. [PubMed]
  • Cattell RB. The scree test for the number of factors. Multivariate Behavioral Research. 1966;1:245–276.
  • Edwards G, Gross MM. Alcohol dependence: provisional description of a clinical syndrome. British Medical Journal. 1976;1:1058–1061. [PMC free article] [PubMed]
  • Ghitza UE, Sparenborg S, Tai B. Improving drug abuse treatment delivery through adoption of harmonized electronic health record systems. Substance Abuse and Rehabilitation. 2011a;2:125–131. [PMC free article] [PubMed]
  • Ghitza U, Lindblad R, Gore-Langton RE, Sparenborg S, Tai B. Implication of adopting standardized common data elements in health IT systems of drug abuse treatment providers. Poster presented at the College on Problems of Drug Dependence (CPDD) annual meeting; June 18–23 2011b; Hollywood, FL.
  • Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic(ROC) curve. Radiology. 1982;143:29–36. [PubMed]
  • Hu L, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Structural Equation Modeling. 1999;6:1–55.
  • Hudziak JJ, Helzer JE, Wetzel MW, Kessel KB, McGee B, Janca A, et al. The use of the DSM-III-R checklist for initial diagnostic assessment. Comprehensive Psychiatry. 1993;34:375–383. [PubMed]
  • Koob GF, Le Moal M. Neurobiology of Addiction. London: Academic Press (Elsevier); 2006.
  • Leri F, Bruneau J, Stewart J. Understanding polydrug use: review of heroin and cocaine co-use. Addiction. 2003;98:7–22. [PubMed]
  • Ling W, Hillhouse M, Domier C, Doraimani G, Hunter J, Thomas C, et al. Buprenorphine tapering schedule and illicit opioid use. Addiction. 2009;104:256–265. [PMC free article] [PubMed]
  • Madras BK, Compton WM, Avula D, Stegbauer T, Stein JB, Clark HW. Screening, brief interventions, referral to treatment (SBIRT) for illicit drug and alcohol use at multiple healthcare sites: comparison at intake and 6 months later. Drug and Alcohol Dependence. 2009;99(1–3):280–295. [PMC free article] [PubMed]
  • Mello NK, Negus SS. Effects of d-amphetamine and buprenorphine combinations on speedball(cocaine+heroin) self-administration by rhesus monkeys. Neuropsychopharmacology. 2007;32:1985–1994. [PubMed]
  • Muthén LK, Muthén BO. Mplus: Statistical Analysis with Latent Variables: User's Guide. Los Angeles: Muthén and Muthén; 2010.
  • National Institutes of Health, Society for Behavioral Medicine. Identifying Core Behavioral and Psychosocial Data Elements for the Electronic Health Record: Executive Summary. Bethesda, MD: National Institutes of Health; 2011.
  • Office of National Drug Control Policy. Washington, DC: Office of National Drug Control Policy, Executive Office of the President; 2010. National Drug Control Strategy 2011.
  • Rockett IR, Putnam SL, Jia H, Smith GS. Assessing substance abuse treatment need: a statewide hospital emergency department study. Annals of Emergency Medicine. 2003;41:802–813. [PubMed]
  • Saha TD, Chou SP, Grant BF. Toward an alcohol use disorder continuum using item response theory: results from the National Epidemiologic Survey on Alcohol and Related Conditions. Psychological Medicine. 2006;36:931–941. [PubMed]
  • Saitz R, Alford DP, Bernstein J, Cheng DM, Samet J, Palfai T. Screening and brief intervention for unhealthy drug use in primary care settings: randomized clinical trials are needed. Journal of Addiction Medicine. 2010;4:123–130. [PMC free article] [PubMed]
  • Skinner HA. The drug abuse screening test. Addictive Behaviors. 1982;7:363–371. [PubMed]
  • Smith PC, Schmidt SM, Allensworth-Davies D, Saitz R. A single-question screening test for drug use in primary care. Archives of Internal Medicine. 2010;170:1155–1160. [PMC free article] [PubMed]
  • Smith PC, Schmidt SM, Allensworth-Davies D, Saitz R. Primary care validation of a single-question alcohol screening test. Journal of General Internal Medicine. 2009;24:783–788. [PMC free article] [PubMed]
  • Substance Abuse and Mental Health Services Administration. Results from the 2010 National Survey on Drug Use and Health: National Findings. Rockville, MD: Substance Abuse and Mental Health Services Administration, Office of Applied Studies; 2011.
  • Substance Abuse and Mental Health Services Administration. What you need to know about health reform? Substance Abuse and Mental Health Services Administration News. 2010a;18(5)
  • Substance Abuse and Mental Health Services Administration. Results from the 2009 National Survey on Drug Use and Health: National Findings. Rockville, MD: Substance Abuse and Mental Health Services Administration, Office of Applied Studies; 2010b.
  • Tai B, Sparenborg S, Liu D, Straus M. The National Drug Abuse Treatment Clinical Trials Network: forging a partnership between research knowledge and community practice. Substance Abuse and Rehabilitation. 2011;2:21–28. [PMC free article] [PubMed]
  • Tiet QQ, Finney JW, Moos RH. Screening psychiatric patients for illicit drug use disorders and problems. Clinical Psychology Review. 2008;28:578–591. [PubMed]
  • U.S. Preventive Services Task Force. Screening and behavioral counseling interventions in primary care to reduce alcohol misuse: Recommendation statement. Annals of Internal Medicine. 2004;140:554–556. [PubMed]
  • Vinson DC, Kruse RL, Seale JP. Simplifying alcohol assessment: two questions to identify alcohol use disorders. Alcoholism, Clinical and Experimental Research. 2007;31:1392–1398. [PubMed]
  • Whitlock EP, Green CA, Polen MR, Berg A, Klein J, Siu A, et al. Behavioral Counseling Interventions in Primary Care to Reduce Risky/Harmful Alcohol Use. Rockville, MD: Agency for Healthcare Research and Quality; 2004.
  • World Health Organization. Switzerland: World Health Organization; 2009. Global Health Risks: Mortality and Burden of Disease Attributable to Selected Major Risks.
  • Wu LT, Ringwalt CL. Alcohol dependence and use of treatment services among women in the community. American Journal of Psychiatry. 2004;161:1790–1797. [PMC free article] [PubMed]
  • Wu LT, Pilowsky DJ, Schlenger WE, Hasin D. Alcohol use disorders and the use of treatment services among college-age young adults. Psychiatric Services. 2007;58:192–200. [PMC free article] [PubMed]
  • Wu LT, Pan JJ, Blazer DG, Tai B, Stitzer ML, Brooner RK, et al. An item response theory modeling of alcohol and marijuana dependences: a National Drug Abuse Treatment Clinical Trials Network study. Journal of Studies on Alcohol and Drugs. 2009a;70:414–425. [PubMed]
  • Wu LT, Pan JJ, Blazer DG, Tai B, Brooner RK, Stitzer ML, et al. The construct and measurement equivalence of cocaine and opioid dependences: a National Drug Abuse Treatment Clinical Trials Network (CTN) study. Drug and Alcohol Dependence. 2009b;103:114–123. [PMC free article] [PubMed]
  • Wu LT, Ringwalt CL, Yang C, Reeve BB, Pan JJ, Blazer DG. Construct and differential item functioning in the assessment of prescription opioid use disorders among American adolescents. Journal of the American Academy of Child and Adolescent Psychiatry. 2009c;48:563–572. [PMC free article] [PubMed]
  • Wu LT. Substance abuse and rehabilitation: responding to the global burden of diseases attributable to substance abuse. Substance Abuse and Rehabilitation. 2010;2010:5–11. [PMC free article] [PubMed]
  • Wu LT, Pan JJ, Blazer DG, Tai B, Stitzer ML, Woody GE. Using a latent variable approach to inform gender and racial/ethnic differences in cocaine dependence: a National Drug Abuse Treatment Clinical Trials Network study. Journal of Substance Abuse Treatment. 2010;38 Suppl 1:S70–S79. [PMC free article] [PubMed]
  • Wu LT, Blazer DG, Li TK, Woody GE. Treatment use and barriers among adolescents with prescription opioid use disorders. Addictive Behaviors. 2011;36:1233–1239. [PMC free article] [PubMed]
  • Yudko E, Lozhkina O, Fouts A. A comprehensive review of the psychometric properties of the Drug Abuse Screening Test. Journal of Substance Abuse Treatment. 2007;32:189–198. [PubMed]