Of the instruments we discuss here, the ASI and CIDI make up the common assessment battery of the National Institute on Drug Abuse (NIDA) Clinical Trials Network (CTN), which conducts studies to evaluate evidence-based treatment interventions in widely diverse community-based treatment settings and patient populations nationwide. Prior to adopting these measures, a CTN workgroup evaluated many measures for reliability, validity, efficiency, and suitability for widespread use in nonresearch settings.
RELIABILITY AND VALIDITYAn instrument’s reliability and validity are critical to its value. All the instruments discussed in this article are highly reliable and valid, but the extent of their reliability or validity may differ in particular situations.
Reliability
The question of reliability is: Will users of the instrument consistently reach the same diagnostic conclusions? A straightforward and rigorous way to answer this question is the test-retest method. Two or more clinicians use the instrument to conduct independent assessments of the same patient, and the degree of correlation among their findings is calculated. The standard statistical measure of the degree of the clinicians’ agreement, the kappa coefficient, equals 1 if agreement is complete and less than zero when agreement is no greater than chance might produce (
Cohen, 1960). Generally, a test-retest kappa score of 0.75+ indicates excellent reliability; 0.60 to 0.74 is good; 0.40 to 0.59 is fair; and less than 0.40 is poor (
Fleiss, 1981).
Validity
The question of validity is: Does the instrument truly and unambiguously assess the condition it is designed to evaluate? This question has more dimensions than the estimation of reliability; accordingly, validity is estimated with a number of methods.
The widely used SCID was the first standardized psychiatric interview based on the DSM and has been updated to correspond to the most current DSM criteria. The AUDADIS, PRISM, and SSADDA were specifically designed for substance abuse research, but are adaptable for clinical purposes, too.
Brief descriptions of these instruments follow. For a summary comparison of their properties, see .
| TABLE 1Characteristics and Selected Assessment Categories of Six Structured Assessment Instruments |
The Addiction Severity Index
The ASI (
McLellan et al., 1980,
1992) screens for problems and impairments that commonly accompany drug abuse and dependence. These include, among others, interpersonal difficulties with family, friends, and co-workers; medical conditions such as hepatitis B and C, HIV/AIDS, sexually transmitted diseases, alcoholic liver disease, acute myocardial infarction, pneumonia, and metabolic and endocrine complications (
Kresina et al., 2004;
Mertens et al., 2003); and legal troubles. The ASI provides information that clinicians can use to address these problems with appropriate interventions or referrals.
The semi-structured ASI evaluates patients’ functioning and lifetime experiences in seven domains: (1) medical conditions, (2) employment/support, (3) use of alcohol and drugs, (4) legal issues, (5) family history, (6) family/social relationships, and (7) psychiatric disorders. The administrator asks the patient to rate his or her level of distress in each domain during the past 30 days from 0 (none) to 4 (extreme) and independently rates the patient’s need for treatment in each domain from 0 (none necessary) to 9 (treatment needed to intervene in a life-threatening situation). Finally, the administrator calculates a composite score from a subset of the distress and treatment need responses. This score becomes the basis for treatment planning. Altogether, the ASI takes approximately 45 to 60 minutes to administer, plus 10 to 20 minutes for post-interview scoring.
THE DSM AND STANDARDIZED ASSESSMENTThe year 1980 saw the publication of an epochal document in psychiatry:
Diagnostic and Statistical Manual of Mental Disorders, 3rd Edition (DSM-III;
American Psychiatric Association, 1980). The DSM-III provided clinicians and researchers with standardized definitions and diagnostic criteria for more than 200 psychiatric disorders, including substance abuse and dependence disorders. Prior to this publication, clinicians and researchers commonly used the same diagnostic terms to mean different things, and clinicians often disagreed on whether patients had specific disorders (
Spitzer, Endicott, and Robins, 1975;
Spitzer and Fleiss, 1974). Substance abuse professionals engaged in semantic debates over the definition of addiction—even over the very existence of such a condition.
Following the publication of the DSM-III, diagnostic criteria were included in the mental disorders section of the
International Classification of Diseases, 10th Edition (ICD-10;
World Health Organization, 1993). The ICD-10 is widely used outside the United States to define psychiatric diagnoses.
Substance Use Disorders in DSM-IV
The current edition of the DSM, DSM-IV-TR, sets diagnostic criteria for two types of substance use disorder: dependence and abuse. Some patients seeking treatment report too few symptoms to meet the criteria for either diagnosis. In these cases, the specific symptoms, symptom clusters, and the severity of associated problems can inform effective strategies for intervention and management.
The ICD-10 criteria for substance dependence are similar to those of the DSM-IV. The ICD-10 counterpart to abuse is called “harmful use” and is less specific.
Substance Dependence
Drug or alcohol dependence is diagnosed by documenting that a patient has experienced at least three of seven criteria for a particular substance within a 12-month period. The criteria are:
- Tolerance
- Withdrawal
- Substance often taken in larger amounts or over longer period than intended
- Persistent desire or unsuccessful efforts to cut down or control use
- Great deal of time spent in activities necessary to obtain, use, or recover from the substance
- Important social, occupational, or recreational activities given up or reduced
- Continued use despite knowledge of having a persistent or recurrent physical or psychological problem likely to have been caused or exacerbated by the substance
Although the DSM-IV provides no standards for dependence severity, clinicians may specify “with physiological dependence” or “with withdrawal” to indicate the presence of tolerance (i.e., the need for higher doses to achieve intoxication or other desired effects). Withdrawal, in particular, predicts medical problems and poor outcome (
Hasin et al., 2000;
Schuckit et al., 2003). Alternatively, a symptom or criteria count can function as a measure of dependence severity (
Hasin et al., 2006b).
The DSM-IV lists substance-specific intoxication and withdrawal symptoms for most of the common classes of drugs. Two exceptions are hallucinogens and cannabis, neither of which had a known withdrawal syndrome at the time of the document’s publication. Planners for the DSM-V are considering the addition of a withdrawal syndrome for cannabis. In anticipation of such a potential change, the CIDI and AUDADIS interviews contain items related to possible marijuana withdrawal.
Test-retest studies have repeatedly shown good to excellent reliability for the diagnosis of substance dependence with the DSM-IV (
Bucholz et al., 1995;
Chatterji et al., 1997;
Easton et al., 1997;
Grant et al., 1995,
2003;
Hasin et al., 1996,
1997a;
Horton, Compton, and Cottler, 2000;
Williams et al., 1992). The DSM-IV substance dependence diagnosis also shows good validity in two forms of multi-method comparisons. One compares ICD-10, DSM-IV, and DSM-III-R diagnoses obtained from a single diagnostic interview (
Grant, 1993;
Hasin et al., 1997b;
Schuckit et al., 1994). The other compares diagnoses from a single system (such as DSM-IV) produced by different diagnostic interviews (
Cottler et al., 1997;
Pull et al., 1997).
Substance Abuse
Patients who do not meet the criteria for substance dependence may be diagnosed with substance abuse if they report experiencing one or more of four abuse symptoms repeatedly over a 12-month period. The symtoms are:
- Failure to fulfill major obligations at work, school, or home
- Recurrent use in situations in which it is physically hazardous
- Recurrent substance-related legal problems
- Continued use despite persistent social or interpersonal problems
Many clinicians have questioned the separation of substance dependence and substance abuse. Studies have shown that the DSM criteria for abuse are less valid than those for dependence. However, these studies diagnosed substance abuse hierarchically, meaning that an abuse diagnosis was considered to be redundant if dependence was present. Although DSM-IV stipulates this procedure, not everyone with dependence also meets the criteria for abuse (
Hasin and Grant, 2004). Women and minorities appear especially likely to experience dependence without abuse (
Hasin et al., 2005;
Hasin and Grant, 2004). Studies that assessed abuse regardless of whether dependence was present showed better reliability for the criteria for abuse (
Bucholz et al., 1995;
Canino et al., 1999;
Cottler et al., 1997;
Pull et al., 1997). In summary, the DSM-IV hierarchical status of abuse is problematic, but the criteria yield reliable diagnoses.
DSM-IV and Substance Use Comorbidity
Extensive comorbidity between substance use disorders and other psychiatric disorders has been reported consistently in patients (
Nunes, Hasin, and Blanco, 2004) as well as in the general population (
Grant et al., 2004a,
2004b;
Regier et al., 1990). Such comorbidity can be serious. For example, studies with acceptable response rates (70 percent or more) and reliable diagnostic assessments have consistently found an adverse effect of major depression on the outcome of substance use disorders (
Hasin, Nunes, and Meydan, 2004). Further, among patients with histories of substance dependence and major depression, the occurrence of a major depressive episode during periods of sustained abstinence predicts a higher number of suicide attempts (
Aharonovich et al., 2002).
To be accurate, assessments must address the fact that substance intoxication and withdrawal can mimic symptoms of depression, psychosis, or other independent psychiatric disorders. Accordingly, the DSM-IV distinguishes among “expected effects” of substance intoxication or withdrawal, “primary disorders,” and “substance-induced disorders.” A primary disorder is diagnosed if “the symptoms are not due to the direct physiological effects of a substance” (
American Psychiatric Association, 2000). Psychiatric disorders that co-occur with substance intoxication or withdrawal can be considered primary if (1) symptoms substantially exceed the expected effects of the substance in the amount that was used; (2) there is a personal history of psychiatric symptoms during periods of extended abstinence; (3) the onset of psychiatric symptoms clearly preceded the onset of substance use; and (4) symptoms persisted for at least a month after the cessation of intoxication or withdrawal. Symptoms that are not considered primary fall into the category either of expected effects of a substance or of a substance-induced disorder that exceeds intoxication or withdrawal effects and deserves independent clinical attention. Instrument developers have incorporated this information into some tools, in particular the SSADDA and PRISM.
The ASI’s psychometric properties have been tested extensively (
Alterman et al., 1994;
Hodgins and el-Guebaly, 1992;
Joyner, Wright, and Devine, 1996;
Kosten, Rounsaville, and Kleber, 1983;
McLellan et al., 1985;
Rogalski, 1987). Several studies have demonstrated good to excellent reliability and validity for the instrument (
Butler et al., 2001;
Hendriks et al., 1989;
Leonhard et al., 2000;
Weisner, McLellan, and Hunkeler, 2000). A 2004 summary of studies in multiple patient groups (Mäkelä) found that the reliability of composite scores varied from high (
Daeppen et al., 1996;
McLellan et al., 1985;
Peters et al., 2000) to low (
Drake, McHugo, and Biesanz, 1995;
Zanis et al., 1994;
Zanis, McLellan, and Corse, 1997). Three of the seven ASI domains (medical conditions, use of alcohol, and psychiatric disorders) have high internal consistency across studies, while the other four are more variable. Correlations between domains are usually low, except those between the drug and legal measures and those between the psychiatric and social impairment measures. The lack of across-the-board correlations is consistent with the ASI’s perspective, which is that impairment in some domains does not necessarily entail impairment in others.
The ASI, by itself, may not be a highly reliable screen for special populations, such as the homeless or dually diagnosed. For the latter groups, the ASI should be supplemented with instruments that assess comorbidity in greater depth, such as the PRISM or the SSADDA.
Many community programs include the ASI in their initial assessment battery, but informal reports suggest that some look upon it as merely required paperwork and use its information minimally, if at all, in treatment. To remedy this situation, the NIDA/Substance Abuse and Mental Health Services Administration (SAMHSA) Blending Initiative has produced a curriculum on transforming ASI data into clinically useful information (see
www.nida.nih.gov/Blending/ASI.html).
Alcohol Tolerance Item From the World Mental Health Composite International Diagnostic Interview (WMH-CIDI)
| ALCOHOL DEPENDENCE | | | | |
|---|
| Did you ever need to drink a larger amount of alcohol to get an effect, or did you ever find that you could no longer get a “buzz” or a high on the amount you used to drink? | YES | NO | DK | RF |
| (1) | (5) | (8) | (9) |
The Composite International Diagnostic Interview
The CIDI, originally developed by the World Health Organization, assesses 22 DSM-IV diagnoses, including mood, anxiety, and substance use disorders (see “Alcohol Tolerance Item From the World Mental Health Composite International Diagnostic Interview (WMH-CIDI)”). For each substance use disorder, the CIDI elicits other information useful for treatment planning, such as the patterns and course of alcohol and drug use. The fully structured instrument takes approximately 120 minutes to administer in its entirety (
Kessler and Ustün, 2004).
Various versions and adaptations of the original CIDI have been developed. The University of Michigan version, the UM-CIDI, has been used in a large international epidemiological survey (
Wittchen and Kessler, 1994), but appears to produce lower prevalence estimates than other diagnostic instruments (
Wittchen et al., 1998). To address this problem and others related to earlier versions of the CIDI, the World Mental Health Survey Initiative Version, the WMH-CIDI, was developed (
Kessler and Ustün, 2004). A complete description of WMH-CIDI modifications is reported elsewhere (
Kessler and Ustün, 2004). The WMH-CIDI is available in paper and computerized forms for download or computer-assisted administration at
www.hcp.med.harvard.edu/wmhcidi/instruments.php.
Programs or projects may use the CIDI substance use sections alone or combine them with other sections to achieve the desired range of assessment. To meet the particular needs of the substance abuse field, researchers have developed the CIDI Substance Abuse Module (CIDI-SAM), an expanded version of the original CIDI substance use section that elicits detailed information on such areas as the onset and history of substance abuse, withdrawal symptoms, common comorbidities, social consequences, and treatment history (
Cottler, Robins, and Helzer, 1989;
Horton, Compton, and Cottler, 2000;
epi.wustl.edu/epi/assessments/sam.htm).
Test-retest studies of the original CIDI and the CIDI-SAM paper versions have demonstrated good to excellent reliability for DSM-IV diagnoses of any substance use disorder or substance dependence and fair to good reliability for abuse (
Rubio-Stipec, Peters, and Andrews, 1999;
Wittchen et al., 1998). The reliability of the CIDI, version 3.0, was tested in the WHO World Mental Health Surveys by comparing CIDI-derived diagnoses to those derived with the SCID (
Haro et al., 2006). Concordance for alcohol dependence (with or without abuse) was excellent; concordance for drug dependence (with or without abuse) was fair; and concordance for alcohol abuse and drug abuse was good.
NIDA’s CTN adopted the CIDI after comparing five commonly used substance use disorder diagnostic instruments on 26 criteria, including psychometric properties, diagnostic time frames, time to administer, and training and financial considerations (
Forman et al., 2004). The CTN workgroup ultimately determined that only the CIDI met three crucial CTN requirements: it can be administered by trained research technicians with no prior clinical experience; it provides for DSM-IV, as well as
International Classification of Diseases, 10th Edition (ICD-10;
World Health Organization, 1993), substance use disorder diagnoses; and it provides for past-year and lifetime diagnoses. At this point, it is too soon to know whether CTN-related community-based programs will adopt the CIDI for clinical use.
Alcohol Use Screening From the Structured Clinical Interview for DSM (SCID)
| ALCOHOL USE SCREENING |
| What are your drinking habits like? |
How much do you drink? |
Has there ever been a time in your life when you had five or more drinks on one occasion? |
| When in your life were you drinking the most? |
How long did that period last? |
| During that time… |
How often were you drinking? |
What were you drinking? How much? |
| During that time… |
Did your drinking cause problems for you? |
Did anyone object to your drinking? |
The Structured Clinical Interview for DSM-IV
The SCID is available in different versions for researchers and clinicians. Additionally, the research version is available in formats for patients, nonpatients, and patients with psychotic disorders. The Structured Clinical Interview for DSM-IV-TR Axis I Disorders, Research Version, Patient Edition (
First et al., 2002), provides lifetime and current diagnostic assessments for many DSM-IV disorders, including substance use disorders. The separate SCID for Axis II disorders provides the basis for diagnosing personality disorders (
First et al., 1997).
The semi-structured SCID is designed for administration by interviewers with clinical expertise, but research assistants having extensive experience with a population under study have sometimes learned to administer it successfully. After an open-ended overview and brief general screening, the interviewer takes the patient through the questions on the form, following up as needed (based on clinical judgment) to clarify responses. The alcohol and drug modules contain open-ended screening questions as well (see “Alcohol Use Screening From the Structured Clinical Interview for DSM (SCID)”). Administration can take up to several hours, depending on the complexity of the patient’s substance and psychiatric history. The instrument is modular, so clinicians can make use of only those sections that pertain to assessment aims. It contains a minimal number of nondiagnostic items to keep administration time as brief as possible.
In tests among substance-abusing populations, the SCID has demonstrated excellent reliability for diagnosing DSM-III-R substance dependence (
American Psychiatric Association, 1987;
Ross et al., 1995). A small test-retest study of 52 patients with DSM-IV diagnoses showed excellent reliability for substance use disorders (
Zanarini et al., 2000). The SCID Web site (
www.scid4.org/index.html) provides information on the different versions, psychometric properties, ways to obtain copies of the interview and training materials, and procedures for arranging on-site training. A user’s guide provides basic training in the use of the SCID. In addition, an 11-hour videotape training program is available with examples of interviews with actual patients. The instrument’s developers recommend at least 20 hours of training on the full SCID for most clinicians. A Spanish-language version of the SCID (research version), in which only the questions have been translated, and a computer-assisted SCID (for Axis I disorders, clinician version), developed by an outside source, can be obtained through the SCID Web site.
The Alcohol Use Disorder and Associated Disabilities Interview Schedule
The AUDADIS (
Grant et al., 1995,
2003) provides for current (last 12 months) and lifetime DSM-IV diagnoses of major mood, anxiety, personality, and substance use disorders. Originally developed by the National Institute on Alcohol Abuse and Alcoholism (NIAAA) for use in population-based epidemiological surveys, the fully structured AUDADIS functions as an economical tool that lay staff in treatment programs can administer for intake screening. Clinicians can use the detailed descriptive data obtained by the AUDADIS to structure treatment based on a patient’s specific substance-related behaviors. In addition to alcohol, tobacco, and other drug use, modules address treatment and family history. Numerous queries address the frequency and quantity of use of each type of alcohol (e.g., beer, wine, liquor) and each illicit drug during three time periods—that of heaviest use, the past 12 months, and the interviewee’s lifetime (see “Sample Item From the Alcohol Use Disorder and Associated Disabilities Interview Schedule (AUDADIS)”).
The AUDADIS showed high reliability in a test-retest study in clinical settings where comorbidity was expected to be high (
Hasin et al., 1996). Its test-retest reliabilities for alcohol and drug consumption, abuse, and dependence, as well as those for other modules, were good to excellent (
Grant et al., 1995,
2003). The AUDADIS interview can be downloaded (niaaa.census.gov/questionaire.html). The instrument’s developers recommend using the computer-assisted version.
The Psychiatric Research Interview for Substance and Mental Disorders
The PRISM (
Hasin et al., 1996,
2006a;
Hasin, Trautman, and Endicott, 1998) is a semi-structured diagnostic interview designed expressly for assessing comorbid psychiatric disorders in individuals who abuse substances. The instrument’s strength is in differentiating independent psychiatric disorders, such as depression, from the effects of intoxication and withdrawal. Along with abuse and dependence diagnoses for specific substance categories, clinicians and researchers can use the PRISM to make current and lifetime DSM-IV diagnoses of Axis I and Axis II disorders that commonly occur with substance abuse, such as mood, anxiety, and psychotic disorders.
The PRISM sections on substance use disorders are placed at the beginning of the interview and provide a background for the overall clinical picture. Periods of chronic intoxication (defined as “at least 4 days a week for a month”) or binge use (defined as “most of the day for 3 or more days”) and extended periods of abstinence are identified and charted on a timeline. The timeline is the only part of the PRISM that is conducted in an unstructured format, and timeline information is not coded for data entry. The purpose of the timeline is to assist in differentiating primary versus substance-induced symptoms in later diagnostic sections.
PRISM developers incorporated two features into the instrument to avoid the lengthy administration time associated with many standardized interviews. First, diagnostic sections are modular, so the instrument can be tailored to fit specific treatment or research needs. Second, consumption questions in the substance use module do not seek detailed information about patterns, but simply ask how often the interviewee has used the substance “in the last 12 months” or “ever” and whether the individual has ever experienced a period of chronic intoxication or binge use. If the response to any of these broad questions is “yes,” the interviewer moves on to the abuse and dependence diagnostic module.
A recent test-retest study of 285 heavy substance users showed good to excellent reliability for most dependence diagnoses, including alcohol, cocaine, heroin, cannabis, and sedative dependence (
Hasin et al., 2006a). An independently conducted validity study of a Spanish-language version of the PRISM with the Longitudinal, Expert, All-Data Diagnosis (LEAD) procedure (
Spitzer, 1983) as the “gold standard” and the SCID found that the concordance of the three assessments in substance dependence was good to excellent. However, PRISM/LEAD concordance was significantly better than SCID/LEAD concordance on current cannabis and cocaine dependence, as well as past alcohol abuse and dependence (
Torrens et al., 2004). The English version of the PRISM can be downloaded, together with training information (
www.columbia.edu/~dsh2/prism). A computer-assisted version, which will include questions on marijuana withdrawal and modules for nicotine-related disorders, pathological gambling, and attention deficit hyperactivity disorder, will be available in 2008.
Sample Item From the Alcohol Use Disorder and Associated Disabilities Interview Schedule (AUDADIS)
Now I’d like to ask you about drinking beer.
5a. During the last 12 months, did you drink any beer, light beer or malt liquor? Do not count nonalcoholic beers. Statement D | 1 — Yes 2 — No - SKIP to Statement E, page 11 |
5b. (SHOW FLASHCARD 12) During the last 12 months, about how often did you drink any beer or malt liquor? | 1 __ Everyday 2 __ Nearly every day 3 __ 3 to 4 times a week 4 __ 2 times a week 5 __ Once a week 6 __ 2 to 3 times a month 7 __ Once a month 8 __ 7 to 11 times in the last year 9 __ 3 to 6 times in the last year 10 __ 1 or 2 times in the last year |
The Semi-Structured Assessment for Drug Dependence and Alcoholism
The SSADDA (
Pierucci-Lagha et al., 2005) was developed for use in studies of genetic influences on cocaine and opioid dependence. Derived from the Semi-Structured Assessment for the Genetics of Alcoholism, the SSADDA provides extensive coverage of the physical, psychological, social, and psychiatric manifestations of cocaine and opioid abuse and dependence in addition to a number of related Axis I and Axis II disorders. A standout feature of the SSADDA is its inclusion of questions about the onset and recency of individual alcohol and drug symptoms, permitting a temporal assessment of symptom clusters. Information about the timing of symptoms is particularly helpful in distinguishing comorbid disorders from intoxication or withdrawal effects.
The reliability of individual dependence criteria in the SSADDA has been tested to determine the extent to which independent interviewers arrive at the same diagnostic conclusions. Overall, the inter-rater reliability estimates were excellent for individual DSM-IV criteria for nicotine and opioid dependence; good for alcohol and cocaine dependence; and fair for dependence on cannabis, sedatives, and stimulants (
Pierucci-Lagha et al, 2007). A computer-assisted version of the SSADDA is available free. Further information can be obtained by contacting Dr. Amira Pierucci-Lagha, Alcohol Research Center, Department of Psychiatry, University of Connecticut School of Medicine.