PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
Drug Alcohol Depend. Author manuscript; available in PMC 2010 August 1.
Published in final edited form as:
PMCID: PMC2721976
NIHMSID: NIHMS110876

The construct and measurement equivalence of cocaine and opioid dependences: A National Drug Abuse Treatment Clinical Trials Network (CTN) study

Abstract

Introduction

Although DSM-IV criteria are widely used in making diagnoses of substance use disorders, gaps exist regarding diagnosis classification, use of dependence criteria, and effects of measurement bias on diagnosis assessment. We examined the construct and measurement equivalence of diagnostic criteria for cocaine and opioid dependences, including whether each criterion maps onto the dependence construct, how well each criterion performs, how much information each contributes to a diagnosis, and whether symptom-endorsing is equivalent between demographic groups.

Methods

Item response theory (IRT) and multiple indicators–multiple causes (MIMIC) modeling were performed on a sample of stimulant-using methadone maintenance patients enrolled in a multisite study of the National Drug Abuse Treatment Clinical Trials Network (CTN) (N=383). Participants were recruited from six community-based methadone maintenance treatment programs associated with the CTN and major U.S. providers. Cocaine and opioid dependences were assessed by DSM-IV Checklist.

Results

IRT modeling showed that symptoms of cocaine and opioid dependences, respectively, were arrayed along a continuum of severity. All symptoms had moderate to high discrimination in distinguishing drug users between severity levels. “Withdrawal” identified the most severe symptom of the cocaine dependence continuum. MIMIC modeling revealed some support for measurement equivalence.

Conclusions

Study results suggest that self-reported symptoms of cocaine and opioid dependences and their underlying constructs can be measured appropriately among treatment-seeking polysubstance users.

Keywords: Clinical trials network, Cocaine dependence, DSM-IV Checklist, Item response theory, multiple indicators–multiple causes, Opioid dependence

1. Introduction

Despite the fact that the Diagnostic and Statistical Manual of Mental Disorders-IV (DSM-IV; American Psychiatric Association [APA], 1994) has been in use approximately 14 years, several gaps in our knowledge remain on whether the classification of a diagnosis should be categorical or dimensional, whether different dependence criteria should be used for different drugs, and how measurement bias influences assessment of diagnoses (Edwards, 2007; Hughes, 2006; Saunders and Schuckit, 2006). As recently noted by Edwards (2008), the dependence syndrome was a provisional idea in 1976 and in some ways it remains so today. At present, diagnoses of substance use disorders rely on interview data. In most addiction treatment studies, investigators depend not only upon self-report diagnostic instruments for the assessment of some inclusion and exclusion criteria, but often use these data to examine differences in treatment outcomes across diagnostic categories or key demographic groups (Carroll, 1997). The quality of diagnostic measures in turn affects the validity of interpretations of study findings and evaluation of treatment interventions and programs. This issue is centrally relevant to some of the multisite studies conducted by the Clinical Trials Network (CTN) that seek to inform practice in real-world clinical settings. It is therefore important to evaluate whether the diagnostic instruments used in these studies actually measure the intended construct and whether symptom-endorsing is equivalent for drug users with diverse demographic characteristics.

To help ensure that an instrument is a sound measure that enables unbiased diagnoses for diverse subgroups, it is essential to show measurement equivalence of the instrument (McHorney and Fleishman, 2006). Measurement nonequivalence occurs when persons with equivalent levels of the dependence factor respond differently to an instrument as a function of group membership (Chen and Anthony, 2003; Grant et al., 2007). This in turn may lead to inaccurate comparisons across groups involving diagnosing participants, as well as assessing prevalence rates, risk factors, and treatment outcomes of the measured conditions (Chen and Anthony, 2003; McHorney and Fleishman, 2006). DSM-IV Checklist has been used increasingly by investigators to assess the status of substance dependence of study participants in addiction treatment studies (Peirce et al., 2006; Petry et al., 2005; Rawson et al., 2004). Despite its increasing ubiquity, little is presently known about the psychometric properties of DSM-IV Checklist for substance use disorders (Forman et al., 2004).

In this study, we apply IRT (Embretson and Reise, 2000) and MIMIC modeling (Chen and Anthony, 2003; Wu et al., in press) to examine the psychometric properties of DSM-IV criteria for cocaine and opioid dependences assessed by DSM-IV Checklist (Wu et al., 2008). The data source for the study is a multisite randomized CTN trial examining the effects of lower-cost incentives on stimulant users in six community methadone maintenance treatment settings across the country (Peirce et al., 2006), and all of which utilized DSM-IV Checklist to make diagnoses of stimulant and opioid dependences. As noted in a recent study by Saha and colleagues (2006), IRT methods can be very useful in evaluating how individual diagnostic criteria maps onto the dependence construct, how well each item performs, how much information each criteria contributes to a diagnosis, and whether symptom-endorsing is equivalent between demographic groups. This level of information can help identify poor versus good criteria items for a given diagnosis, thus providing highly relevant information for evaluating specific criteria that define underlying constructs of disorder.

While Saha et al. (2006) relied on IRT methods for assessing item-level bias between demographic groups, this present study extends that work by utilizing the MIMIC model plus IRT methods. The MIMIC model incorporates the measurement model (i.e., the underlying construct of the dependence syndrome) with structural regression equations that permit us to detect item-response bias (i.e., differential item functioning) of DSM-IV criteria for multiple background variables within a regression framework. Its improves comprehension of the impact of any item-response bias detected on the estimated dependence factor, while adjusting for the effects of multiple background variables (e.g., sex and race/ethnicity) on the estimated dependence factor (Chen and Anthony, 2003; Grant et al., 2007). This latter feature represents a unique advantage of the MMIC model over IRT methods.

Previous studies using factor analysis have found that DSM-III-R and DSM-IV criteria of cocaine or opioid dependence represent one factor, but these studies generally do not investigate measurement equivalence of their diagnostic instruments and item-level psychometric performance (Bryant et al., 1991; Feingold and Rounsaville, 1995; Morgenstern et al., 1994). One notable exception was a study by Langenbucher and colleagues (2004) that reported findings from one of the first published studies applying IRT modeling to evaluate DSM-IV criteria for substance use disorders. They examined abuse and dependence criteria for alcohol, marijuana, and cocaine use in a sample of 372 adults enrolled at addiction treatment programs. After removing two criteria (“legal problems” and “tolerance”) that demonstrated a poor fit of the 11 criteria to a unidimensional solution, they found that the remaining 9 criteria for abuse of and dependence on alcohol, marijuana, and cocaine, respectively, reflected one continuum of severity.

More recently, Gillespie and colleagues (2007) conducted IRT modeling of DSM-IV criteria for drug use disorders (marijuana, cocaine, opioids, hallucinogens, sedatives, and stimulants, respectively) in a sample of all male participants from the Virginia Twin Registry. They found that abuse and dependence criteria for each drug class were explained by a single underlying continuum of risk. Additionally, the study reported large differences in estimates of individual item severity and discrimination across drug classes. In particular, cocaine users were found to endorse most items at much lower levels of latent liability than users of other drugs. The investigators concluded that cocaine use was the most disabling drug and had more harmful effects compared to the other drugs. Similarly, Lynskey and Agrawal (2007) conducted IRT analysis to examine abuse and dependence criteria for drug use disorders in the National Epidemiologic Survey on Alcohol and Related Conditions (NESARC). They found that abuse and dependence criteria formed a single latent construct of risk and that there were some similarities and variations in item performance in regards to item severity and discrimination. Specifically, the “inability to cut down on use” and “legal problems” exhibited poorer discrimination for most drugs, whereas “legal problems” and “withdrawal” appeared to index more severity levels of the abuse/dependence continuum for several drugs. Together, these studies suggest that DSM-IV’s hierarchical distinction between abuse and dependence may not be supported (i.e., one factor), and that the contribution of each item to the underlying construct of a diagnosis differs depending on the drugs used and the study sample as shown by variations in item performance across drugs and studies (e.g., Gillespie et al., 2007; Lynskey and Agrawal, 2007; Langenbucher et al., 2004).

While these IRT studies of drug use disorders have yielded new and important information, they have some limitations. Perhaps most importantly, the prior studies examined lifetime symptoms that occurred sporadically over the course of participants’ lives (e.g., Gillespie et al., 2007; Langenbucher et al., 2004; Lynskey and Agrawal, 2007). It is unclear whether and to what extent results from this method of indexing symptoms over a lifetime apply to a current (past year) DSM-IV diagnosis of drug use disorder that requires the occurrence and clustering of a specific number of symptoms over the past 12-month (APA, 2000). Further, criterion information and measurement equivalence of the diagnostic instruments have not been fully reported in prior studies (e.g., Gillespie et al., 2007; Langenbucher et al., 2004; Lynskey and Agrawal, 2007). The lack of such empirical data make it difficult to evaluate the item-level measurement precision, the extent of information that each item contributes to the underlying construct, and whether the diagnostic instruments used to measure drug use disorders represent a fair measure for drug users with diverse sociodemographic characteristics. The later data are particularly important to research of constructs of self-reported drug use disorders because it constitutes the prerequisite for making unbiased estimates.

The present study constitutes the first known IRT modeling of DSM-IV criteria for current (past year) cocaine and opioid dependence disorders. It extends prior work by addressing both item-level psychometric performance and measurement equivalence by age, sex, race/ethnicity, and educational level (Grant et al., 2007). The study also explores whether the presence of a comorbid opioid dependence biases cocaine users’ endorsement of cocaine dependence symptoms, and whether the presence of a comorbid cocaine dependence biases opioid users’ endorsement of opioid dependence symptoms. This question heretofore has not been addressed, but is highly relevant to a valid assessment of DSM-IV diagnoses, particularly in the context of addiction treatment settings such as methadone treatment programs where drug abusers are likely to have comobid substance dependences and a diagnosis of dependence is often required for enrolling in a treatment program (e.g., Brooner et al., 1997; Disney et al., 2005; Wu et al., 2008). It is hypothesized that symptoms of cocaine and opioid dependences will form a continuum of severity and that the pattern of individual item performance will differ across cocaine and opioids because of their differences in pharmacological effects (APA, 2000; Gillespie et al., 2007). Specifically, symptoms of physical dependence on opioids (i.e., tolerance and withdrawal) will be endorsed at the low severity level on the dependence continuum, while symptoms of physical dependence on cocaine will be endorsed at the high severity levels on the continuum. It is also expected that there will be no significant item-response bias (i.e., no differential item functioning) across background variables after the level of the dependence continuum and background variables are adjusted statistically. Specifically, the following questions are evaluated: (1) where along the continuum does each criterion measure the dependence liability (item severity); (2) what criterion symptoms distinguish participants who are higher on the continuum of severity from those who are lower on the continuum (item discrimination); and (3) is the probability of endorsing symptoms of dependence at the equivalent severity level similar across groups defined by sex, age, race/ethnicity, educational level, and comorbid drug dependence (measurement equivalence)? The present study provides an excellent opportunity to investigate these questions in a geographically diverse sample of subjects from multiple treatment programs across the country (CTN) using an identical set of assessments measures.

2. Method

2.1. Data source

Statistical analyses were performed on data from public-use files of a multisite study of the National Drug Abuse Treatment CTN, which evaluated stimulant use outcomes of an abstinence-based contingency management intervention as an addition to usual care in community-based outpatient methadone maintenance treatment programs (Peirce et al., 2006). Participants were recruited from six community-based methadone maintenance treatment programs associated with the CTN and major providers in their regions. All six programs were located in urban areas in the northeastern, eastern, or southwestern United States, and had an average static patient census of 490. All programs reported substantial problems with stimulant abuse in their patient populations (Peirce et al., 2006).

Potential participants were referred by counselors or responded to information available at the clinic. Eligible participants included opioid-dependent patients who: 1) had been in the treatment program for at least 30-days and less than 3-years; 2) submitted a clinic urine sample positive for cocaine, amphetamine, or methamphetamine within 2 weeks of study entry, as verified from clinic records; 3) reported that they were not in recovery from a gambling problem; and 4) demonstrated a clear understanding of study procedures by passing a simple informed consent quiz with a score of 80% or better (Peirce et al., 2006). Participants were enrolled into the study between April 30, 2001 and February 28, 2003. Prior to randomization, participants completed the intake assessment, which obtained information on demographics, psychosocial problems, and drug use and diagnoses. Participants then provided their first study urine sample, which was tested on site; results were used to stratify participants before randomization. Participants were randomly assigned to one of two study conditions: usual care or usual care plus abstinence-based incentives for 12 weeks.

2.2. Assessments

Social and demographic variables (age, sex, race/ethnicity, and educational level) were collected at time of intake. Substance-specific substance use disorders were assessed by DSM-IV Checklist (Wu et al., 2008). Participants were asked about past year use of five types of substances: cocaine, amphetamines, opioids, alcohol, and marijuana. If participants reported using a substance in the past year, they were then asked about criterion symptoms of dependence resulting from that substance use, as specified by DSM-IV criteria (APA, 2000). Due to resource and time constraints and because a diagnosis of dependence excludes a diagnosis of abuse in DSM-IV (APA, 2000), participants who met criteria for Dependence (three or more dependence criteria) for a given substance were not administered the Abuse questions for that particular substance; participants who did not satisfy criteria for Dependence were assessed for possible Abuse (i.e., one or more positive abuse criteria). Seven DSM-IV Dependence disorder criteria were assessed: 1) tolerance; 2) withdrawal; 3) substance often taken in large amounts or for longer periods of time; 4) persistent desire or unsuccessful attempt to cut down or control use; 5) a great deal of time spent in activities necessary to get the substance; 6) important activities given up; and 7) continued substance use despite knowledge of having persistent or recurrent physical or psychological problems.

2.3. Study sample

Statistical analyses were based on 383 participants aged ≥18 years reporting past year use of either cocaine or amphetamines from DSM-IV Checklist. The mean age of participants was 41.96 years (SD=8.55). Approximately three quarters were members of minority groups: African Americans (49%), Hispanics (18%), and others (6%). More than half (55%) were male, and 35% had <12 years of education. Findings from the original trial that report response to the treatment intervention are presented in detail elsewhere (Peirce et al., 2006).

Because the sample sizes of past year use of alcohol, marijuana, and amphetamines were too small to generate stable estimates (N < 200 for each substance), this investigation focused on past year users of cocaine (N = 366) and opioids (N = 354). Consistent with prior studies of IRT modeling (e.g., Gillespie et al., 2007; Langenbucher et al., 2004; Lynskey and Agrawal, 2007), the groups of users of cocaine or opioids were not mutually exclusive. Among cocaine users, 81% met criteria for past year cocaine dependence; among opioid users, 86% met criteria for past year opioid dependence.

2.4. Data analyses

Before performing IRT modeling, we used factor analysis for binary data using a weighted least squares estimation procedure (Muthén and Muthén, 2007) to examine the assumption of unidimensionality in the IRT (Hambleton et al., 1991). Unidimensionality is established by demonstrating that a one-factor model provides the most parsimonious fit to the data. We assessed the number of factors to be retained with the scree test (Cattell, 1966), the ratio of the first eigenvalue to the second eigenvalue, and variance explained by the first eigenvector.

Two-parameter normal ogive IRT modeling (Embretson and Reise, 2000) was conducted using BILOG-MG3 (Zimowski et al., 2005). The two-parameter model allowed us to examine the relationship between participants’ item performance and traits underlying item performance (i.e., the underlying severity of the dependence factor), which is described by a monotonically increasing S-shaped item characteristic curve (ICC). An ICC is characterized by item severity and discrimination parameters. An item severity parameter indicates the position of the ICC in relation to the latent continuum, typically ranging from −3 to +3 (Embretson and Reise, 2000). It describes the severity level best measured by a specific item, and reflects the point on the latent continuum where there is a 50% chance of the criterion symptom being present. An item discrimination parameter measures the precision with which the item distinguishes between participants with levels of the latent trait above versus those with levels below the item’s severity (Langenbucher et al., 2004). Criterion symptoms with steeper slopes (high discrimination parameter) are more useful for discriminating between drug users above or below given levels on the continuum than are criterion symptoms with less steep slopes. Criterion information curves (CICs; Baker, 2001) were also examined, and they describe where on the continuum each criterion conveys the greatest amount of information in terms of measurement precision at that latent level. The greater the item information for a given level, the smaller the error involved in estimating the severity level by that item.

We then determined whether criterion symptoms functioned differently across sex (male vs. female), age group (< 40 years vs. ≥40 years), race/ethnicity (whites vs. nonwhites), educational level (< high school vs. ≥high school), and comorbid drug dependence status (opioid dependence among cocaine users; cocaine dependence among opioid users) by examining the differential criterion functioning (DCF) of severity parameters. The selection of these variables was based on their associations with cocaine or opioid dependence and on the consideration that addiction treatment studies typically examine these demographic variables or diagnostic categories as predictors of treatment outcomes (Anthony et al., 1994; Blanco et al., 2008; Chen and Kandel, 2002; Disney et al., 2005; Carroll, 1997; Substance Abuse and Mental Health Services Administration [SAMHSA], 2008). The presence of DCF indicates that the probability of endorsing a particular symptom is not equivalent across groups (i.e., differential severity level). It helps elucidate whether the differences in self-reported drug dependence across groups are biased by background characteristics. MIMIC modeling (Muthén and Muthén, 2007) was conducted to enhance our investigation of DCF. This model has the unique advantage of incorporating the measurement model and the regression model within a single conceptual framework, thus permitting us simultaneously to examine the DCF (i.e., significant direct effects) and to include statistical adjustment for variations in background covariates on the measured dependence factor. This feature of adjusting for multiple covariates is not feasible with IRT modeling. Tucker-Lewis Index (TLI), Comparative Fit Index (CFI), and Root Mean Square Error of Approximation (RMSEA) were all used to assess the fit of the MIMIC model. Values of TLI and CFI ≥ 0.95 (1 = perfect fit) and values of RMSEA ≤ 0.06 (the lower value, the better fit) indicate an excellent fit to the data (Browne and Cudeck, 1993; Hu and Bentler, 1999).

3. Results

3.1. Unidimensionality of the dependence construct

The scree test showed that a one-factor model fitted criteria for either cocaine or opioid dependence. The first eigenvalue and the ratio of the first-to-second eigenvalues were high (4.7/0.8 = 5.9) for cocaine dependence symptoms, and the first eigenvector explained 67% of the variance. Factor loadings were greater than 0.60 for each cocaine dependence item (Table 2). For opioid dependence criteria, the first eigenvalue and the ratio of the first-to-second eigenvalues were also high (4.9/0.8 = 5.9), and the first eigenvector explained 71% of the variance. Both sets of criteria also exhibited high levels of internal consistency (Cronbach alpha coefficient > 0.8).

Table 2
Factor loadings and IRT discrimination and severity estimates of past year DSM-IV cocaine and opioid dependences

3.2. ICCs: symptom discrimination and severity

As shown in Table 2, all criterion items of cocaine dependence exhibited positive discrimination parameters in distinguishing between drug users along the dependence continuum (0.86–2.44). “Time spent using cocaine” had greater discrimination than other items (the steepest line in Figure 1). Of all severity parameters, “withdrawal” represented the most severe symptom on the cocaine dependence continuum that was endorsed at higher severity levels (shifted to the right end of the ICCs), while “unable to cut down” reflected the less severe symptom and was endorsed at relatively low severity levels.

Figure 1
Item characteristic curves (ICCs) for cocaine dependence criteria

Each criterion of opioid dependence showed a good to high level of discrimination (1.19–2.18). “Giving up activities” and “taking larger amounts” had greater discrimination than the others. Severity was comparatively greater for “giving up activities,” “continued use despite having problems,” and “taking larger amounts” (Figure 2). In contrast, “tolerance” and “withdrawal” were endorsed at relatively low severity levels. As a precaution, we also examined severity and discrimination parameters using the IRT command language program (Hanson, 2002) (data not shown) and MPlus (data not shown). The patterns of results from both programs were consistent with estimates reported here from BILOG-MG3.

Figure 2
Item characteristic curves (ICCs) for opioid dependence criteria

3.3. CICs: amounts of information from each criterion

To identify criteria that provide the most information across the continuum of severity in terms of the degree of precision that each item measures at a specific level, BILOG-MG3 was used to generate CICs. As shown in Figure 3, “time spent using cocaine” provided the most information value for the cocaine dependence continuum. Specifically, severity was estimated with greatest precision at the severity level corresponding to the severity estimates of “time spent using cocaine,” and the amount of the information decreased as the severity level departed from severity estimates of that criterion. In contrast, “giving up activities” and “taking larger amounts” provided more information across the opioid dependence continuum than did other criteria (Figure 4).

Figure 3
Criterion information curves (CICs) for cocaine dependence criteria
Figure 4
Criterion information curves (CICs) for opioid dependence criteria

3.4. DCF: item equivalence across groups and the impact of item bias on the dependence factor

3.4.1. Cocaine dependence

Results of DCF from MIMIC modeling are summarized in Table 3. At the equivalent cocaine dependence level, whites were less likely than nonwhites to endorse symptoms of tolerance but were more likely to report giving up activities consequent to cocaine use. At the equivalent cocaine dependence level, cocaine users with a comorbid opioid dependence were more likely than those without such dependence to endorse symptoms of cocaine withdrawal, but less likely to endorse continued cocaine use despite physical or mental problems. The impact of the DCF analysis was evaluated by comparing the above-mentioned MIMIC model with one that excluded DCF. Very few changes were observed on the estimated association (i.e., regression coefficient) between race/ethnicity and the cocaine dependence factor (regression coefficient: 0.032 vs. 0.034 after adjusting for DCF), as well as between opioid dependence and the cocaine dependence factor (0.410 vs. 0.428 after adjusting for DCF). This observation indicates that the effect of DCF associated with race/ethnicity and with opioid dependence was minimal.

Table 3
Significant direct effects or differential criterion functioning (DCF) of DSM-IV cocaine and opioid dependence symptoms from MIMIC modeling

3.4.2. Opioid dependence

At the equivalent opioid dependence level, subjects with more education were less likely than subjects without a high school diploma to report spending a great deal of time using opioids or recovering from their effects. At the equivalent opioid dependence level, opioid users with versus without a comorbid cocaine dependence were less likely to endorse symptoms of opioid withdrawal, but more likely to endorse continued cocaine use despite physical or mental problems. A comparison of the MIMIC model without including DCF versus the one adjusting for it revealed that the estimated association between education and the opioid dependence factor increased (regression coefficient: 0.037 vs. 0.077 after adjusting for DCF). There were little changes in the estimated association between the cocaine dependence variable and the opioid dependence factor (0.533 vs. 0.528 after adjusting for DCF). The results of DCF from MIMIC modeling of cocaine and opioid dependences were found to be in line with the findings from IRT modeling of DCF (data not shown).

4. Discussion

Cocaine and opioids are two of the most common “primary” drugs of use reported by people entering substance abuse treatment programs across the country (SAMHSA, 2008). The present study contributes new and important information on the assessment and diagnosis of these two drug classes in addiction treatment settings. We found that dependence on either cocaine or opioids, as assessed by DSM-IV Checklist, was arrayed along a continuum of severity. Each individual criterion also exhibited moderate to high discrimination in distinguishing between drug users on the dependence continuum. Further, MIMIC modeling revealed some support for measurement equivalence. These findings support the clinical utility of DSM-IV Checklist in assessing dependence on cocaine and opioids among stimulant-using methadone maintenance outpatients.

4.1. Unidimensionality of the dependence construct

The dependence syndrome concept was introduced by Edwards and Gross (1976), who emphasized the coherence and unidimensionality of a set of behavioral, cognitive, and physiological components constituting a clinical syndrome for people with alcohol use disorder. This concept was the basis for diagnostic criteria for alcohol and drug dependences in DSM-III-R (APA, 1987; Bryant et al., 1991; Kosten et al., 1987), DSM-IV (APA, 1994), and ICD–10 (World Health Organization, 1992). The present findings provide impressive evidence of a single dimensional construct for the cocaine or opioid dependence syndromes and replicate prior studies that generally reported on less geographically diverse samples and treatment settings (Bryant et al., 1991; Feingold and Rounsaville, 1995; Morgenstern et al., 1994). Additionally, our findings extend earlier work by addressing an understudied area that is a prerequisite for understanding whether a diagnostic instrument is capable of making unbiased diagnoses across diverse groups of drug users. The overall pattern of these results shows that a clinical dependence syndrome for cocaine and opioids can be assessed properly in this geographically diverse sample of polysubstance users.

4.2. Item discrimination

As noted previously, only a few studies have applied IRT methods to examine the construct of drug use disorders, and the present study presents the first IRT analysis of current dependence on cocaine and opioids. Due to differences in research designs, a direct comparison of these IRT results with those of other studies is limited. Previous IRT studies of cocaine or opioid dependence analyzed lifetime symptoms of both abuse and dependence (Gillespie et al., 2007; Langenbucher et al., 2004; Lynskey and Agrawal, 2007). In comparison, we examined the dependence criteria for a current dependence disorder as specified by DSM-IV (APA, 2000). Two of the prior studies used general population samples (Gillespie et al., 2007; Lynskey and Agrawal, 2007), and only Langenbucher et al. (2004) examined drug users in a treatment setting. The latter study included nine out of the 11 lifetime symptoms of abuse and dependence in IRT modeling, and opioid use disorders were not examined. Further, none of the earlier studies examined criterion information curves (CICs) that evaluate where on the continuum each item conveys the greatest amount of information contributing to the measured construct, as well as its respective measurement precision at a specific latent level. This present study utilizes the findings from CIC’s plus all other IRT parameters to help interpret study findings.

We found that symptoms indicative of salience or compulsive drug-seeking behaviors provide good discrimination between drug users who were lower versus higher on the dependence continuum. By using the combined information from both discrimination and severity, CICs showed that “time spent using cocaine” had a high discriminative power and provided a substantially large amount of information about the cocaine dependence continuum, which may be related in part to it’s high factor loading on the dependence factor (i.e., 0.90). This finding suggests that cocaine users who exhibit “increasing amounts of time engaging in cocaine use” are at risk for progressing to more severe levels on the dependence continuum. Should this finding be confirmed in other clinical samples and settings, this criterion could expedite screening in the clinical setting in order to efficiently identify cocaine users for more intense interventions. CICs also indicated that compulsive drug-seeking behaviors – “giving up activities” and “taking large amounts” – distinguish less severe opioid users from more severe users on the IRT-defined latent trait and provided comparatively more information to the dependence continuum than others. Additional study seems warranted to better elucidate their potential for identifying opioid-dependent patients who may be at risk for escalating to more severe levels on the dependence continuum, and who may need additional medical monitoring or interventions.

4.3. Item severity

IRT modeling also identified important differences in item severity across criteria for cocaine and opioid dependences. Such findings accord with the well-documented pharmacological effects of these substances (APA, 2000). For example, marked and easily measured physiological signs of withdrawal are common among opioid users, and patients with a variety of health problems often develop tolerance to prescribed opioids and experience some withdrawal symptoms without having opioid dependence disorder (APA, 2000). We found that opioid users manifested a high rate of “tolerance” and “withdrawal,” and both criteria measured the lowest severity levels on the dependence continuum. The other study of male drug users that examined lifetime symptoms of both opioid abuse and dependence in an IRT model also reported similar findings (Gillespie et al., 1999). In contrast to opioids, cocaine produces less clearly identifiable physical signs of withdrawal symptoms (APA, 2000; Koob and Le Moal, 2006), and withdrawal typically develops after cessation of prolonged heavy use or a period of binge use (Repetto and Gold, 2005). The finding that cocaine withdrawal symptoms indexed the most severe level on the continuum is also consistent with an earlier study by Langenbucher et al (2004) who applied IRT to examine lifetime symptoms of both cocaine abuse and dependence.

The overall pattern of results from our IRT modeling has implications for DSM-V. DSM-IV distinguishes between drug dependence with (the presence of tolerance or withdrawal) versus without a physiological component (the presence of other cognitive-behavioral symptoms only); the physiological subtype is widely regarded as a severity indicator of dependence (APA, 2000). Previous research has found that cocaine users reporting a physiological component, especially withdrawal symptoms, exhibit a more severe profile of drug use as compared with those without (Disney et al., 2005; Schuckit et al., 1999). In the present study, “withdrawal” indexed the most severe level on the cocaine dependence continuum. In contrast, “tolerance” and “withdrawal” was related to the least severe form on the opioid dependence continuum. The reasons for this discrepant finding are unclear. While it is possible that the physiological component of opioid dependence is a less robust indicator of severity of the dependence syndrome than it is with cocaine, it is important to remember that the entire sample was receiving an opioid agonist (methadone) for at least 3 months. Methadone treatment may have attenuated the associations between withdrawal symptoms and severity of the opioid dependence continuum that might otherwise have been seen in opioid-dependent samples with withdrawal symptoms who were not receiving any agonist medication. Within the context of opioid dependence in the present sample of people taking methadone, the physiological component may not be as good an indicator of severity of the dependence syndrome as it is for cocaine. These findings, nonetheless, highlight item-level variations that each criterion contributes to the dependence construct in terms of discrimination and severity (Gillespie et al., 1999; Langenbucher et al., 2004). If future versions of the DSM incorporate a dimensional approach to classifying substance use diagnoses, using a simple summary of the number of positive items for each disorder in order to obtain disorder-specific severity scores does not seem well-justified (Helzer et al., 2006).

4.4. Measurement equivalence

It is worthy noting that the majority of DCF analyses of cocaine dependence items showed no significant item-response bias by participants’ gender, age group, and educational level. For opioid dependence items, there was no significant item-response bias by participants’ gender, age group, and race/ethnicity. Even for race/ethnicity (cocaine dependence items) and educational level (opioid dependence items), the large majority of comparisons didn’t show differences. Study results, however, suggest that white cocaine users are differentially less likely to endorse “tolerance,” but are more likely to endorse “giving up important activities” as compared to nonwhite cocaine users. In addition, the more educated opioid users are differentially less likely to endorse “time spent using” as compared to the less educated. These results suggest the presence of different severity levels of these items across groups. Such findings have implications for epidemiological research on cocaine and opioid dependences because the presence of DCF may bias one group to have a higher or lower risk for dependence than another if DCF is not removed or controlled statistically. They hence point to the need to statistically adjust for race/ethnicity in the analysis of cocaine dependence and to adjust for educational level in the research of opioid dependence in order to minimize the potentially confounding effects from differential self-report bias. Further studies of larger patient samples also are needed to better characterize the extent of response bias across different racial/ethnic groups. Cognitive interviews would help elucidate how attributes related to race/ethnicity and education (e.g., beliefs, attitudes, and literacy) influence participants’ interpretations and comprehension of interview questions (Johnson et al., 2006; Warnecke et al., 1997).

Finally, we examined if the presence of a comorbid opioid dependence affects cocaine users’ endorsement of cocaine dependence symptoms, and if the presence of a comorbid cocaine dependence affects opioid users’ endorsement of opioid dependence symptoms. The presence of DCF was determined by comparing item severity parameters by the comorbid status. We found that cocaine withdrawal symptoms were more likely to be endorsed by cocaine users with a comorbid opioid dependence, and that opioid withdrawal symptoms were less likely to be endorsed by opioid users with a comorbid cocaine dependence. The reason for this differential reporting of withdrawal symptoms is not clear since this is the first known psychometric study on this particular issue and all participants had been taking methadone for at least 3-months. Previous studies have suggested that cocaine and opioids are often co-used to enhance their “subjective” reinforcing effects of one or both drugs or to reduce the amount of opioids taken while maintaining “high” effects from the cocaine, and that cocaine is also frequently used by opioid-dependent patients to attenuate the intensity of opioid withdrawal symptoms (Leri et al., 2003). All of these potential drug interactions might modulate or otherwise alter the perception of opioid withdrawal symptoms, and could lead people to misattribute perceived withdrawal symptoms from one drug (e.g., opioid) to another drug (e.g., cocaine). This finding on differential item functioning is potentially very important to the assessment and treatment of cocaine and opioid dependences for several reasons. Dependence on both drugs is common among addiction patients (Disney et al., 2005). Withdrawal is also utilized as a specifier for determining dependence subtypes in DSM-IV, and it plays an important role in the assessment, treatment planning, and delivery of clinical services (APA, 2000; Kampman et al., 2001). Future research is necessary to determine if differential endorsement of cocaine and opioid withdrawal symptoms can be replicated in other samples, including samples not receiving methadone or a comparably effective opioid agonist medication.

4.5. Study limitations and strengths

These findings should be interpreted within the context of the following limitations. First, they are based on opioid users in methadone maintenance treatment settings who also reported recent stimulant use. This sample may encompass a severe group of polysubstance users, as the results reflect a narrow spectrum of dependence syndromes (i.e., low values of almost all severity parameters). Nevertheless, opioids and cocaine tend to be co-used by drug users in many treatment settings (Condelli et al., 1991; Brooner et al., 1997; Leri et al., 2003), and both substances together constitute the “primary” drugs of use reported by the majority of people seeking drug abuse treatment across the country (SAMHSA, 2008). Thus, these findings appear generalizable to a large and clinically important subgroup of treatment-seeking patients. Another limitation universal to this type of research is reliance on participants’ self-reported information, which is subject to recall errors. The fact, however, that our findings are generally consistent with earlier studies in different samples provides some assurance of the validity of the self-reported data (Babor et al., 2000). In addition, due to resource constraints and the fact that a diagnosis of Dependence excludes a diagnosis of Abuse in the DSM-IV (APA, 2000), symptom of Abuse were not assessed in participants who met criteria for Dependence for a given class of drugs. Finally, because cocaine dependence item and opioid dependence items were calibrated in two separated IRT models, item characteristic curves were not directly tested across the two drug classes.

The present study also has several noteworthy strengths. It extends prior studies of the construct of cocaine and opioid dependences by employing sophisticated IRT and MIMIC methods to shed new light on measurement equivalence and item-level psychometric performance of DSM-IV criteria for current cocaine and opioid dependences. Generalizability to clinical patients is somewhat improved due to the fact that study participants comprise a geographically diverse sample of stimulant users recruited from six methadone maintenance outpatient programs located in the northeastern, eastern, or southwestern United States. These findings also lend support for the use of DSM-IV Checklist in clinical settings given that it is relatively brief, easy to administer and score, generates current diagnoses for guiding clinical interventions (Forman et al., 2004).

4.6. Conclusions and implications

Substance use diagnoses are determined almost exclusively by patients’ self-reports, and the quality of self-report measures in turn impacts the validity of research and clinical findings. The present study evaluated the quality of a diagnostic assessment for current drug dependence within the context of multiple community-based addiction treatment settings, which are highly relevant to the assessment of the most severe subset of drug users for determining their eligibility for receiving treatment or enrolling in a study. Study results indicate that dependence on cocaine or opioids represents a unidimensional continuum of risk. “Withdrawal” measures the most severe symptom of the cocaine dependence construct and is less likely to be exhibited by cocaine users. On the other hand, “withdrawal” and “tolerance” appear to indicate lower severity levels of the opioid dependence construct. Overall, the study suggests that self-reported clinical symptoms of cocaine and opioid dependences and their underlying constructs can be measured appropriately among treatment-seeking polysubstance users. Considering that the use of an accurate and efficient measure of patient-reported symptoms is essential to clinical research in order to properly evaluate patients’ perceived changes in symptoms and functioning, the quality of patient-reported outcome measures in addiction treatment trials deserves greater research attentions than it has received thus far (e.g., Babor et al., 2000).

Table 1
Selected characteristics of outpatient stimulant users in methadone treatment programs (N = 383)

Acknowledgments

This work was supported by a contract from the U.S. National Institute on Drug Abuse of the National Institutes of Health to the Duke University Medical Center (HHSN271200522071C; Principal Investigator: Blazer). The opinions expressed in this paper are solely those of the authors and do not necessarily reflect those of the sponsoring agency. Drs. Wu, Pan, Blazer, and Tai contributed to research questions. Drs. Wu and Pan contributed to the data analysis. Dr. Brooner designed the DSM-IV Checklist. Dr. Wu wrote the first draft, and all authors contributed to the interpretations and revisions of the paper. The authors wish to thank the participants, staff, investigators, and others who made the original studies and this work possible. We also thank Jonathan McCall for editorial assistance. This study, which used existing de-identified, public-use data files, was declared exempt from review by the Duke University Institutional Review Board.

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

Conflicts of Interest: Dr. Patkar has received grant support from Pfizer, Forest Laboratories, Cephalon and Titan Pharmaceuticals and is on the speakers bureaus of Cephalon, and Reckitt-Benckiser. Dr. Stitzer has received consulting fees from Pfizer and Aradigm Pharmaceuticals and research support from Pfizer.

References

  • American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders. 4. American Psychiatric Publishing; Washington, DC: 1994.
  • American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders. 4. American Psychiatric Publishing; Washington, DC: 2000. Text Revision.
  • American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders. 3. American Psychiatric Publishing, Inc; Washington, DC: 1987. text rev.
  • Anthony JC, Warner LA, Kessler RC. Comparative epidemiology of dependence on tobacco, alcohol, controlled substances, and inhalants: basic findings from the National Comorbidity Survey. Exp Clin Psychopharmacol. 1994;2:244–268.
  • Babor T, Steinberg K, Anton R, Del Boca F. Talk is cheap: measuring drinking outcomes in clinical trials. J Stud Alcohol. 2000;61:55–63. [PubMed]
  • Baker F. The basics of item response theory. College Park, MD: ERIC Clearinghouse on Assessment and Evaluation, University of Maryland, College Park, MD; 2001.
  • Blanco C, Harford TC, Nunes E, Grant B, Hasin D. The latent structure of marijuana and cocaine use disorders: results from the National Longitudinal Alcohol Epidemiologic Survey (NLAES) Drug Alcohol Depend. 2007;91:91–96. [PMC free article] [PubMed]
  • Brooner RK, King VL, Kidorf MS, Schmidt CW, Bigelow GE. Psychiatric and substance use comorbidity in treatment-seeking opioid abusers. Arch Gen Psychiatry. 1997;54:71–80. [PubMed]
  • Browne MW, Cudeck R. Alternative ways of assessing model fit. In: Bollen KA, Long JS, editors. Testing Structural Equation Models. Sage Publications; Newbury Park, CA: 1993. pp. 136–162.
  • Bryant KJ, Rounsaville BJ, Babor TF. Coherence of the dependence syndrome in cocaine users. Br J Addict. 1991;86:1299–1310. [PubMed]
  • Carroll KM. New methods of treatment efficacy research: bridging clinical research and clinical practice. Alcohol Health Res World. 1997;21:352–359. [PubMed]
  • Cattell RB. The scree test for the number of factors. Multivariate Behavioral Research. 1996;1:245–276.
  • Chen CY, Anthony JC. Possible age-associated bias in reporting of clinical features of drug dependence: epidemiological evidence on adolescent-onset marijuana use. Addiction. 2003;98:71–82. [PubMed]
  • Condelli WS, Fairbank JA, Dennis ML, Rachal JV. Cocaine use by clients in methadone programs: significance, scope, and behavioral interventions. J Subst Abuse Treat. 1991;8:203–212. [PubMed]
  • Disney ER, Kidorf M, King VL, Neufeld K, Kolodner K, Brooner RK. Prevalence and correlates of cocaine physical dependence subtypes using the SM-IV in outpatients receiving opioid agonist medication. Drug Alcohol Depend. 2005;79:23–32. [PubMed]
  • Edwards G. Substance dependence and substance-related problems: an agenda-setting debate. A further commentary on Li et al (2007) Addiction. 2008;103:179–180. [PubMed]
  • Edwards G, Gross MM. Alcohol dependence: provisional description of a clinical syndrome. Br Med J. 1976;1:1058–1061. [PMC free article] [PubMed]
  • Embretson SE, Reise SP. Item Response Theory for Psychologists. Lawrence Erlbaum; Mahwah, NJ: 2000.
  • Feingold A, Rounsaville B. Construct validity of the dependence syndrome as measured by DSM-IV for different psychoactive substances. Addiction. 1995;90:1661–1669. [PubMed]
  • Forman RF, Svikis D, Montoya ID, Blaine J. Selection of a substance use disorder diagnostic instrument by the National Drug Abuse Treatment Clinical Trials Network. J Subst Abuse Treat. 2004;27:1–8. [PMC free article] [PubMed]
  • Gillespie NA, Neale MC, Prescott CA, Aggen SH, Kendler KS. Factor and item-response analysis DSM-IV criteria for abuse of and dependence on cannabis, cocaine, hallucinogens, sedatives, stimulants and opioids. Addiction. 2007;102:920–930. [PubMed]
  • Grant BF, Harford TC, Muthén BO, Yi HY, Hasin DS, Stinson FS. DSM-IV alcohol dependence and abuse: further evidence of validity in the general population. Drug Alcohol Depend. 2007;86:154–166. [PubMed]
  • Hambleton RK, Swaminathan H, Rogers HJ. Fundamentals of Item Response Theory. Sage Publications, Inc; Newbury Park, CA.: 1991.
  • Hanson BA. IRT Command Language (ICL) 2002. [Accessed on August 8, 2007]. at http://www.b-a-h.com/software/irt/icl/
  • Helzer JE, Bucholz KK, Gossop M. A dimensional option for the diagnosis of substance dependence in DSM-V. Int J Methods Psychiatr Res. 2007;16 Suppl 1:S24–S33. [PubMed]
  • Hu L, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Struct Equation Model. 1999;6:1–55.
  • Hughes JR. Should criteria for drug dependence differ across drugs? Addiction. 2006;101 Suppl 1:134–141. [PubMed]
  • Johnson TP, Cho YI, Holbrook AL, O’Rourke D, Warnecke RB, Chavez N. Cultural variability in the effects of question design features on respondentcomprehension of health surveys. Ann Epidemiol. 2006;16:661–668. [PubMed]
  • Kampman KM, Volpicelli JR, Mulvaney F, Alterman AI, Cornish J, Gariti P, Cnaan A, Poole S, Muller E, Acosta T, Luce D, O’Brien C. Effectiveness of propranolol for cocaine dependence treatment may depend on cocaine withdrawal symptom severity. Drug Alcohol Depend. 2001;63(1):69–78. [PubMed]
  • Koob GF, Le Moal M. Neurobiology of Addiction. Academic Press (Elsevier); London: 2006.
  • Kosten TR, Rounsaville BJ, Babor TF, Spitzer RL, Williams JB. Substance-use disorders in DSM-III-R. Evidence for the dependence syndrome across different psychoactive substances. Br J Psychiatry. 1987;151:834–843. [PubMed]
  • Langenbucher JW, Labouvie E, Martin CS, Sanjuan PM, Bavly L, Kirisci L, Chung T. An application of item response theory analysis to alcohol, cannabis, and cocaine criteria in DSM-IV. J Abnorm Psychol. 2004;113:72–80. [PubMed]
  • Leri F, Bruneau J, Stewart J. Understanding polydrug use: review of heroin and cocaine co-use. Addiction. 2003;98:7–22. [PubMed]
  • Lynskey MT, Agrawal A. Psychometric properties of DSM assessments of illicit drug abuse and dependence: results from the National Epidemiologic Survey on Alcohol and Related Conditions (NESARC) Psychol Med. 2007;37:1345–1355. [PubMed]
  • McHorney CA, Fleishman JA. Assessing and understanding measurement equivalence in health outcome measures. Issues for further quantitative and qualitative inquiry. Med Care. 2006;44:S205–S210. [PubMed]
  • Morgenstern J, Langenbucher J, Labouvie EW. The generalizability of the dependence syndrome across substances: an examination of some properties of the proposed DSM-IV dependence criteria. Addiction. 1994;89:1105–1113. [PubMed]
  • Muthén BO, Muthén LK. Mplus: Statistical Analysis with Latent Variables (Version 4.2.1) Muthén and Muthén Inc; Los Angeles, CA: 2007.
  • Peirce JM, Petry NM, Stitzer ML, et al. Effects of lower-cost incentives on stimulant abstinence in methadone maintenance treatment: a national drug abuse treatment clinical trials network study. Arch Gen Psychiatry. 2006;63:201–208. [PubMed]
  • Petry NM, Peirce JM, Stitzer ML, Blaine J, Roll JM, Cohen A, Obert J, Killeen T, Saladin ME, Cowell M, Kirby KC, Sterling R, Royer-Malvestuto C, Hamilton J, Booth RE, Macdonald M, Liebert M, Rader L, Burns R, DiMaria J, Copersino M, Stabile PQ, Kolodner K, Li R. Effect of prize-based incentives on outcomes in stimulant abusers in outpatient psychosocial treatment programs: a National Drug Abuse Treatment Clinical Trials Network study. Arch Gen Psychiatry. 2005;62:1148–1156. [PubMed]
  • Rawson RA, Marinelli-Casey P, Anglin MD, Dickow A, Frazier Y, Gallagher C, Galloway GP, Herrell J, Huber A, McCann MJ, Obert J, Pennell S, Reiber C, Vandersloot D, Zweben J. Methamphetamine Treatment Project Corporate Authors. A multi-site comparison of psychosocial approaches for the treatment of methamphetamine dependence. Addiction. 2004;99:708–717. [PubMed]
  • Repetto M, Gold MS. Cocaine and crack: neurobiology. In: Lowinson JH, Ruiz P, Millman RB, Langrod JG, editors. Substance Abuse: A Comprehensive Textbook. 4. Lippincott Williams & Wilkins; Philadelphia: 2005.
  • Saha TD, Chou SP, Grant BF. Toward an alcohol use disorder continuum using item response theory: results from the National Epidemiologic Survey on Alcohol and Related Conditions. Psychol Med. 2006;36:931–941. [PubMed]
  • Saunders JB, Schuckit MA. The development of a research agenda for substance use disorders diagnosis in the Diagnostic and Statistical Manual of Mental Disorders, fifth edition (DSM-V) Addiction. 2006;101(Suppl 1):1–5. [PubMed]
  • Schuckit MA, Daeppen JB, Danko GP, Tripp ML, Smith TL, Li TK, Hesselbrock VM, Bucholz KK. Clinical implications for four drugs of the DSM–IV distinction between substance dependence with and without a physiological component. Am J Psychiatry. 1999;156:41–49. [PubMed]
  • Substance Abuse and Mental Health Services Administration, Office of Applied Studies. National Admissions to Substance Abuse Treatment Services. U.S. Department of Health and Human Services; Rockville, MD: 2008. Treatment Episode Data Set (TEDS). Highlights - 2006. DASIS Series: S-40, DHHS Publication No. (SMA) 08-4313.
  • Warnecke RB, Johnson TP, Chávez N, Sudman S, O’Rourke DP, Lacey L, Horm J. Improving question wording in surveys of culturally diverse populations. Ann Epidemiol. 1997;7:334–342. [PubMed]
  • World Health Organization. International Classification of Diseases, 10th rev (ICD-10) World Health Organization; Geneva: 1992. [(accessed July 22, 2008)]. Available at: http://www.who.int/classifications/apps/icd/icd10online/
  • Wu LT, Blazer DG, Stitzer ML, Patkar AA, Blaine JD. Infrequent illicit methadone use among stimulant-using patients in methadone maintenance treatment programs: a national drug abuse treatment clinical trials network study. Am J Addict. 2008;17:304–311. [PMC free article] [PubMed]
  • Wu LT, Pan JJ, Blazer DG, Tai B, Stitzer ML, Brooner RK, Woody GE, Patkar AA, Blaine JD. An item response theory analysis of alcohol and marijuana dependences: a National Drug Abuse Treatment Clinical Trials Network (CTN) study. J Stud Alcohol Drugs. 2009;70(3):414–25. [PubMed]
  • Zimowski M, Muraki E, Mislevy R, Bock D. The BILOG-MG3 statistical program. Scientific Software International, Inc; Lincolnwood, IL: 2005.