This study examined the inter-rater and test-retest reliability of the DSM-IV criteria for dependence on nicotine, alcohol, cocaine, opioid, cannabis, stimulants and sedatives. Using Cohen’s κ, we found fair-to-excellent reliability for the criteria for nicotine, alcohol, cocaine, opioid, and cannabis dependence, while stimulant and sedative dependence criteria were less reliable. Of the 49 criteria evaluated in the inter-rater sub-study, 12 (24.5%) showed poor reliability (i.e., κ̣ < 0.40). In the test-retest sub-study, 8 criteria (16.3%) showed poor reliability. In both sub-studies, the vast majority of unreliable criteria were for stimulants and sedative dependence, however, the smaller number of users of these substances, resulting in lower stability of κ, may have contributed to the lower estimates of reliability for these criteria. These findings are similar to those obtained in the within- and cross-center reliability studies of the SSAGA, in which there were 13% and 19%, respectively, of criteria that were found to be unreliable (Bucholz et al., 1995
With respect to individual substances, all but one opioid dependence criterion (i.e., “using larger amounts or over a longer period than intended”) yielded a value of κ that was good or better (i.e., κ ≥ 0.60). Opioid dependence criteria evaluated using the SSAGA also showed high reliability (Bucholz et al., 1995
). Similarly high estimates of reliability were obtained for nicotine dependence criteria using both the SSADDA and the SSAGA. Values of κ for tolerance were either fair or lower, except for opioids and sedatives. In contrast, values of κ for withdrawal were good to excellent except for cannabis, stimulants, and sedatives. Although tolerance and withdrawal are both manifestations of neuroadaptation, it is likely that they have different underlying mechanisms (Cicero, 1980
; Langenbucher et al., 1997
). In addition, withdrawal is based on more dramatic and perhaps more reliably reported symptoms than is tolerance, a characteristic that could be important when establishing a history of the phenomena retrospectively. Finally, the problem with tolerance might relate to the definition used; for example, a 50% increase in the amount of the substance needed for an effect might be a relatively low threshold (Schuckit et al., 1999
The reliability estimates for individual SSADDA alcohol criteria were generally good, though somewhat lower than those obtained using the SSAGA (Bucholz et al., 1995
). Although reliability estimates were good for most cocaine dependence criteria when elicited using the SSADDA, values of κ for these criteria were also somewhat lower than those reported using the SSAGA (Bucholz et al., 1995
In the present study, reliability for most cannabis dependence criteria were fair, similar to that observed in other studies (Compton et al., 1996
; Cottler et al., 1997
; Miele et al., 2000
; Bucholz et al., 1995
). These findings complement the growing body of literature supporting the existence of the cannabis dependence syndrome and cannabis withdrawal, as well as the validity of the DSM IV dependence criteria as they apply to cannabis dependence (Budney, 2006
; Budney and Hughes, 2006
). The findings of fair reliability for cannabis withdrawal criteria, despite the heterogeneity of the sample, complement recent findings showing that cannabis withdrawal is both clinically significant and readily identified even in non-treatment seeking populations (Copersino et al., 2006
). Reliability estimates for stimulants and sedatives were less than satisfactory, similar to those reported for the SSAGA (Bucholz et al., 1995
), perhaps due in both studies to the relatively low frequency of endorsement for criteria for these disorders.
Substance-dependent subjects in the present study sample predominanly included individuals with opioid and/or cocaine dependence (cf. Gelernter et al., 2005
), while the SSAGA substance-dependent study sample predominantly included individuals with alcohol dependence. We also evaluated reliability in a more heterogeneous sample than that studied by Bucholz et al. (1995)
. Specifically, subjects in our sample had a mean of 3.1 substance use disorder diagnoses, with 34.4% having four or more such diagnoses (Pierucci-Lagha et al., 2005
). Although comparable numbers are not reported by Bucholz et al. (1994
), it is evident from the data presented that their reliability study sample was predominantly comprised of individuals with alcohol use disorders and with fewer comorbid substance use disorders than in the subjects we studied. In the context of multiple substance use disorders, subjects are more likely to misattribute their symptoms to one substance rather than another, leading to reduced criterion reliability for some disorders.
It should be noted also that estimates o
κ are not directly comparable across the SSADDA and SSAGA studies, as the populations evaluated and the prevalence of the disorders differed substantially between them. Although the κ statistic shows the extent to which agreement exceeds that predicted by chance alone, it is not possible directly to interpret differences in κ across different conditions in different studies (Stemler, 2004
). Administration of the SSAGA and SSADDA to the same subjects under controlled conditions would be required for such a direct comparison. Interpretation of reliability findings should, then, be done in the context of a particular study sample.
The results of the impact study conducted using the SSADDA suggest that the diagnosis of dependence on different substances did not generally depend on a single criterion. Specifically, none of the seven DSM-IV criteria for substance dependence individually influenced diagnostic reliability; rather, it was necessary for at least three criteria on average to change in order to affect the reliability of a substance dependence diagnosis. These findings are similar to those of Bucholz et al. (1995)
In summary, these findings showing the reliability of the individual DSM-IV criteria for dependence on different substances provides additional support for the reliability of the SSADDA in the evaluation of substance dependence. Based on its reliability in the assessment of individual criteria and diagnoses (Pierucci-Lagha et al., 2005
), its poly-diagnostic nature, its suitability for administration by lay interviewers, and its computer-assisted format, which includes skip patterns and internal consistency checking, the SSADDA is a useful diagnostic instrument for a variety of applications, including genetic and family studies of substance dependence.