Psychometric Properties of the Depression Anxiety and Stress Scale-21 in Older Primary Care Patients

doi:10.1016/j.jad.2008.01.023

J Affect Disord. Author manuscript; available in PMC 2009 October 1.

Published in final edited form as:

J Affect Disord. 2008 October; 110(3): 248–259.

Published online 2008 March 4. doi: 10.1016/j.jad.2008.01.023

PMCID: PMC2709995

NIHMSID: NIHMS70013

Psychometric Properties of the Depression Anxiety and Stress Scale-21 in Older Primary Care Patients

Andrew T. Gloster,¹ Howard M. Rhoades,² Diane Novy,³ Jens Klotsche,¹ Ashley Senior,^4,⁵ Mark Kunik,^4,^5,⁶ Nancy Wilson,^5,⁷ and Melinda A. Stanley^4,⁵

¹Institute of Clinical Psychology and Psychotherapy, Technical University of Dresden, Germany

²Department of Psychiatry and Behavioral Sciences, University of Texas Health Science Center at Houston

³Department of Anesthesiology and Pain Medicine, University of Texas M. D. Anderson Cancer Center

⁴Department of Psychiatry and Behavioral Sciences, Baylor College of Medicine

⁵Houston Center for Quality of Care and Utilization Studies, and Michael E. DeBakey Veterans Affairs Medical Center, Houston

⁶Department of Medicine, Baylor College of Medicine

⁷Department of Medicine-Geriatrics, Baylor College of Medicine

Please direct correspondence to: Andrew T. Gloster, Ph.D., Technical University of Dresden, Institute of Clinical Psychology and Psychotherapy, Chemnitzer Str. 46, D-01187, Dresden, Germany, Email: gloster/at/psychologie.tu-dresden.de, Phone: ++49-351-463-36895, Fax: ++49-351-463-36984

The publisher's final edited version of this article is available at J Affect Disord

See other articles in PMC that cite the published article.

Publisher's Disclaimer

Abstract

The Depression Anxiety Stress Scale (DASS) was designed to efficiently measure the core symptoms of anxiety and depression and has demonstrated positive psychometric properties in adult samples of anxiety and depression patients and student samples. Despite these findings, the psychometric properties of the DASS remain untested in older adults, for whom the identification of efficient measures of these constructs is especially important.

To determine the psychometric properties of the DASS 21-item version in older adults, we analyzed data from 222 medical patients seeking treatment to manage worry. Consistent with younger samples, a three-factor structure best fit the data. Results also indicated good internal consistency, excellent convergent validity, and good discriminative validity, especially for the depression scale. Receiver operating curve analyses indicated that the DASS-21 predicted the diagnostic presence of generalized anxiety disorder and depression as well as other commonly used measures.

These data suggest that the DASS may be used with older adults in lieu of multiple scales designed to measure similar constructs, thereby reducing participant burden and facilitating assessment in settings with limited assessment resources.

Keywords: Depression Anxiety Stress Scale, Older Adults, GAD, Anxiety, Assessment

Psychometric Properties of the Depression Anxiety and Stress Scale-21 in Older Primary Care Patients

Lovibond and Lovibond (1995a) developed a single measure to assess the core symptoms of depression and anxiety while maximizing discriminant validity between these constructs. Using an empirically driven iterative process, they identified a third factor, which they labeled stress. Their research resulted in the Depression Anxiety Stress Scale (DASS), which consists of 42 items comprising three scales of 14 items. Items refer to the past week; and scores range from 0, “Did not apply to me at all,” to 4, “Applied to me very much, or most of the time.” The Depression scale measures hopelessness, low self-esteem, and low positive affect. The Anxiety scale assesses autonomic arousal, physiological hyperarousal, and the subjective feeling of fear. The Stress scale items measure tension, agitation, and negative affect. The scales are considered to approximate facets of diagnostic categories, as follows: Depression scale for mood disorders, Anxiety scale for panic disorder, and Stress scale for generalized anxiety disorder (GAD; Brown, Chorpita, Korotitsch, & Barlow, 1997). With the exception of GAD and obsessive-compulsive disorder, the anxiety scale also corresponds reasonably closely to the symptomatology of other anxiety disorders. Subsequent research established a 21-item version of the DASS (DASS-21) with seven items per scale (Antony, Bieling, Cox, Enns, & Swinson, R. P., 1998).

Numerous studies have found favorable psychometric properties of the DASS in adults with anxiety and/or mood disorders (Antony et al., 1998; Brown et al., 1997, Clara, Cox, & Enns, 2001), Spanish-speaking patients (Daza et al., 2002), and community-dwelling adults (Crawford & Henry, 2003). All studies have demonstrated excellent internal consistency of the DASS scales in both the 42- and 21-item (DASS-21) versions: Depression (range=.91 to .97); Anxiety (range=.81 to .92); and Stress (range=.88 to .95). A three-factor solution reflecting the three scales has been found consistently across samples and factor-analytic techniques with only minor variations. Inter-scale correlations range as follows: Depression – Anxiety (.45 – .71; .50 or below in all English-speaking samples (Antony et al., 1998; Brown, et al., 1997; Clara et al., 2001), Anxiety – Stress (.65 – .73), and Depression – Stress (.57 – .79).

Older Adults

Despite encouraging psychometric data with the DASS in younger adults, the measure remains untested in older adults. Given the high prevalence of anxiety, depression, and comorbid anxiety-depression in older adults and the need for briefer instruments that efficiently evaluate these symptoms in older patients, the DASS may be a particularly useful measure in this population.

GAD is the most prevalent of the pervasive anxiety disorders in later life in both community and primary care samples (Blazer, George & Hughes, 1991; Beekman et al., 1998; Wittchen, Schuster & Lieb, 2001; Tolin, Robison, Gaztambide & Blank, 2005) but probably one of the most difficult to diagnose. Depression is also highly prevalent in older adults and frequently co-occurs with GAD (Beekman, et al., 2000). Differentiating GAD and depression is particularly difficult in older adults given that the relationship between depression and anxiety is further obfuscated by loss of function and increased somatic presentation (Lenze et al., 2001; Lenze et al., 2005). Some researchers refer to this state as “depletion” as opposed to pure “depression” (Schoevers, van Tilburg, Beekman & Deeg, 2005). A better understanding of these issues is clearly needed in the assessment of older adults. With respect to the DASS, the Anxiety scale contains some somatic items that could be experienced by older adults for reasons unrelated to emotion (e.g., breathing difficulty in the absence of exertion). In contrast, the DASS-D does not contain somatic items, thus limiting the likelihood of artificial score inflation observed in other measures of depression (Taylor, Lovibond, Nicholas, Cayley, & Wilson, 2005). Given these considerations of developmental changes, psychometric results from younger adults cannot automatically be generalized to the measurement of anxiety and depression in later life (Beck & Stanley, 2001).

For this population, a measure that efficiently assesses and differentiates anxiety and depression would be most useful, given the need to reduce patient burden. Furthermore, older adults usually present to primary care settings, where few resources are available to support intensive, clinician-administered differential diagnostic procedures. As such, the DASS may be particularly useful for older adults, although its psychometric properties need to be examined in this population.

Purpose and Hypotheses

This study examined the psychometric properties of the DASS-21 in a population of older adults seeking help for worry in a primary care setting. Investigation of the psychometric properties included factor structure, internal consistency, and convergent and discriminative validity. We hypothesized that: a) a three-factor solution would better fit the data than other solutions; b) the DASS would demonstrate good internal consistency across all scales; c) the DASS would demonstrate good convergent validity in this sample; d) the DASS scales would differentiate different diagnostic groups; and e) the DASS scales would predict the presence vs. absence of GAD and mood disorders as well as other symptom-specific scales.

Method

Participants

Participants were 222 primary care patients 60 years of age or older referred for evaluation in the context of an ongoing randomized clinical trial of cognitive behavioral therapy for late-life worry and GAD in Houston, Texas. Potential participants were identified via primary care physician referrals, advertisements/educational brochures in primary care clinics, and/or letters sent to random samples of clinic patients 60 years or older. Of 968 primary care patients referred to the study, 858 were contacted; and 381 signed consent to participate after an in-person meeting¹. Of those participants who signed consent, 68 individuals voluntarily withdrew from the study before the in-person diagnostic session; 58 withdrew after the in-person diagnostic, but before the follow-up telephone assessment; and 33 patients were excluded due to low MMSE (n=23), substance abuse (n=9), and bipolar disorder (n=1).

Patients were screened for anxiety using two GAD screening questions from the Patient Questionnaire portion of the Primary Care Evaluation of Mental Disorders (Spitzer et al., 1994) (“During the past month, have you often been bothered by “nerves” or feeling anxious or on edge?” and “During the past month, have you often been bothered by worrying about a lot of different things?”). Those who screened positive were further evaluated with the Mini Mental State Exam (MMSE; Folstein, Folstein, & McHugh, 1975) and the Structured Clinical Interview for Diagnosis (SCID; Spitzer, Williams, Gibbon, & First, 1992). The SCID was used to diagnose DSM-IV-TR (American Psychiatric Association, 2000) Axis I disorders. Diagnoses were assigned when overall severity was rated 4 or higher on a 0-to-8 scale. The SCID has been used in numerous studies of older adults (e.g., Stanley et al., 2003), and comparable psychometric properties for younger adults have been established (Segal, Hersen, Van Hasselt, Kabacoff, & Roth, 1993).

SCID interviews were conducted by psychology staff, advanced graduate students, predoctoral psychology interns, or postdoctoral fellows, all of whom received extensive training with the instrument before engaging in study-related assessments. Twenty-five percent of diagnostic interviews were reviewed by a second clinician or project investigator to estimate reliability of diagnostic categories. Diagnostic reliability estimates (kappa coefficient) were good: GAD=.64; depression (including MDD, dysthmia, and not otherwise specified [NOS])=.75; and anxiety diagnoses other than GAD=.73).

Individuals were excluded from the study because of (a) cognitive impairment (defined as MMSE < 24; n =23), (b) current psychosis or bipolar disorder (n=1), or (c) active substance abuse within the past month (n=9). Participants diagnosed with either a principal or co-principal GAD diagnosis were eligible for inclusion in the treatment outcome study. In the presence of comorbid diagnoses principal diagnosis referred to the diagnosis with the highest severity level. The sample used in this study consisted of both patients who were (n=134) and were not (n=88) ultimately included in the ongoing treatment study. All data were collected before treatment began.

Procedure

Approximately 3 weeks following diagnostic assessment (M=23.19 days, SD=15.29), the self-report questionnaires were administered as part of a larger assessment battery by an independent evaluator who had no other contact with study participants. All assessments were administered over the telephone. Participants were provided with blank copies of the self-report measures with which to follow along during the telephone assessment and received a $20 gift card upon completion. Data from an overlapping sample suggest comparable psychometric properties for telephone-administered and in-person instruments (Senior, et al., 2007).

Measures

To examine convergent validity of the DASS, we used self-report measures of constructs believed to parallel the three scales: anxiety (BAI), depression (BDI-II), and worry (Penn State Worry Questionnaire [PSWQ]). The Positive and Negative Affect Schedule (PANAS) and the Quality of Life Inventory (QOLI) provided a test of convergent validity for general distress, as captured by the DASS-21 total score. Discriminative validity was examined using diagnoses obtained via standardized interviews.

Anxiety

The BAI (Beck & Steer, 1990) is a 21-item, self-report measure of anxiety severity experienced over the previous week. Respondents report their symptoms based on a 4-point Likert-type scale. Factor analyses fairly consistently suggest the existence of somatic and subjective/cognitive factors in older adults (Kabacoff, Segal, Hersen & Van Hassellt, 1997; Morin et al., 1999; Wetherell & Arean, 1997). Internal consistency is adequate in community-dwelling older adults (Morin et al., 1999; alpha=.89), older psychiatric patients (Kabacoff et al., 1997; alpha=.90), and older medical patients (Steer, Willman, Kay & Beck, 1994; Wetherell & Arean, 1997; alpha=.86 to .92). The BAI demonstrated good convergent validity in older adults with GAD, but, consistent with most measures of anxiety and depression, had poor discriminate validity (Wetherell & Gatz, 2005).

Depression

The BDI-II (Beck, Steer & Brown, 1996), a widely used self-report scale to assess depressive symptoms, consists of 21 items, each of which is rated on a 4-point scale. One week test-retest reliability for the BDI-II is 0.93, and internal consistency was 0.92 among younger adult outpatients (Beck et al., 1996). Although considerable evidence exists for reliability and validity of the BDI-II in younger adults and for the original BDI in older adults, only one study specifically examined the psychometric properties of the BDI-II (Steer, Rissmiller, & Beck, 2000). In a sample of depressed geriatric inpatients, Steer et al. reported internal consistency of α=.89 and a two-factor structure consisting of cognitive and noncognitive items. Despite the paucity of psychometric research on the BDI-II in older adults, it has been widely used for research in this population and demonstrated its clinical utility with respect to its sensitivity to treatment change (Stanley et al., 2003).

Worry

The PSWQ (Meyer, Miller, Metzger & Borkovec, 1990) is a 16-item, self-report scale that assesses tendency to worry independent of worry content. Each item is rated on a 1-to-5-point Likert-type scale. The internal consistency of the PSWQ has been excellent in populations of college students (α=.90; Fresco, Heimberg, Mennin, & Turk, 2002; α=.93; Meyer, et al., 1990), and patients with anxiety disorders (α=.93; Brown, Antony, & Barlow, 1992), and good in populations of older adults (α=.83 to .85) (Stanley et al., 2001; Hopko et al., 2003). Among older adults, the PSWQ demonstrated adequate convergent validity in patients with and without GAD (Beck, Stanley, & Zebb, 1995; Stanley, Novy, Bourland, Beck & Averill, 2001) and unlike with younger adults, correlated weakly with the BDI (Hopko et al., 2003).

Affect

The PANAS (Watson, Clark, & Tellegen, 1998), a 20-item self-report instrument, was designed to measure two separate dimensions of mood using items rated on a 1- to-5-point Likert-type scale. Positive affect is defined as the degree to which an individual is enthusiastic, active, and alert. In contrast, negative affect measures the degree to which an individual experiences subjective distress and aversive feelings. When administered with instructions to recall the previous week, as in the current study, internal consistency (α) was .87 for both the positive affect scale (PA) and negative affect scale (NA) (Watson, et al., 1988). Among depressed and anxious patients, the PANAS demonstrated good discriminative (Waikar & Craske, 1997) and predictive validity (Watson & Walker, 1996). Among older adults, support has been found for the original two-factor structure (Mackinnon et al. 1999; Kercher, 1992), a revised two-factor structure (Shapiro, Roberts, & Beck, 1999), and a three-factor structure dividing negative affect into two factors (Beck, Novy, Diefenbach, Stanley, Averill & Swann, 2003).

Functional Status

The QOLI (Frisch, 1994, 1999) is a 16-item, empirically derived, self-report measure that provides a global measure of life satisfaction based on the average of satisfaction ratings across a range of life functions. Each rating is weighted according to importance of that life area assigned by the participant. Only dimensions of life function with nonzero importance ratings are included in the total score. The scale probes satisfaction with a wide array of life functions, only one of which involves work. As such, the QOLI is particularly relevant to assess life satisfaction in older adults who may be retired. Internal consistency (α) was .86 in an inpatient population and .83 in a counseling center population (Frisch, Cornell, & Villanueva, 1992). Convergent and divergent validity was demonstrated in clinical and nonclinical populations (Frisch et al., 1992). Discriminant validity of the QOLI was also demonstrated in a sample of older adults (Bourland et al., 2000).

Data Analysis

Using the total sample (n=222), we used confirmatory factor analysis (CFA) to test whether DASS factor structures reported in previous studies adequately fit data collected from older adults. A small Monte Carlo study was conducted to evaluate the power and precision of the derived CFA parameters. Parameter and standard error biases did not exceed 2.5% for any parameter and coverage rates were above 93% in all stimulated scenarios, thus indicating that the sample size of n=222 was sufficient to obtain power of 0.8 (Muthen & Muthen, 2002).

Internal consistency was calculated applying the approach for uncorrelated errors in which the reliability coefficient is calculated using the factor loadings and error variances estimated in the CFA (Raykov, 1989). This method results in reliable estimates of internal consistency when the tau-equivalence assumption (i.e., all items of a factor load equally) is violated as in this sample (Raykov, 2004).

To examine convergent validity, patterns of correlations were examined between the DASS and instruments known to measure related constructs, again using the total sample (n=222). Differences between correlations within a sample (i.e., correlated correlations) were tested using the Fisher z transformation (Meng et al., 1992).

To test the discriminative validity of the DASS scales with respect to differentiation between diagnostic categories, a subsample of patients (n=200) was assigned to four mutually exclusive diagnostic groups: GAD (n=73), mood disorder (MDD, dysthymia, or depressive disorder NOS) (n=33), comorbid GAD and mood disorder (n=61), and no diagnosis (n=33). The other 22 individuals were not diagnosed with GAD or mood disorder but did have another diagnosis. Consistent with previous research (Brown et al., 1997) comorbid diagnoses other than GAD and mood disorders were not accounted for to promote the external validity of the findings. Group differences across DASS scales were tested within the CFA framework. Binary diagnostic group variables were regressed onto the DASS factors such that the β slope quantified the difference in the strength of association between the latent factor and diagnostic groups (Brown, 2006). A priori hypotheses for each scale were as follows. With respect to the DASS-D, we predicted that patients with a mood disorder would score higher than those without. Because the DASS-A assesses imminent fear and autonomic arousal, patients with panic disorder often score highest on this scale (Brown et al., 1997), Given that the present sample of older adults did not contain enough participants with panic disorder to constitute a separate diagnostic category, we predicted that all three psychiatric diagnostic categories would score significantly higher than the nonpsychiatric group. Finally, because the DASS-S assesses tension, we also predicted that the GAD and comorbid GAD/mood diagnostic groups would be significantly different from the mood and no diagnosis groups.

We also examined the diagnostic utility with respect to statistical prediction of diagnostic categories. Specifically, two sets of receiver operating curve (ROC) analyses were used to establish the ability of the DASS to identify patients with a diagnosis of GAD and mood disorders (MDD, dysthymia). ROC analyses allowed comparisons of the psychometric utility between scales and other measures, as well as the establishment of cut-off scores for identifying diagnostic groups. Cuf-off scores were estimated using the Youden-Index (i.e., maximum of equally weighted sensitivity and specificity). Differences between two or more ROC curves were conducted for correlated diagnostic tests within the same sample using STATA 9.2 (De Long et al., 1988; Stata Corp., 2006). For the first set of ROC analyses, patients were grouped into those with a diagnosis of GAD (n=134) and those without and those with a mood- disorder diagnosis (either MDD or dysthymia) (n=94) and those without.

Results

Sample Characteristics and Descriptive Statistics of Measures

Sample characteristics, means, and standard deviations for all measures used in this study are depicted in Table 1. Consistent with convention, all DASS-21 raw scores were doubled to facilitate comparison with previous research and norms established using the DASS-42. Principal diagnoses were GAD (n=134, 60.4 %), MDD (n=24, 10.8 %), social phobia (n=5, 2.3 %), anxiety NOS (n=5, 2.3 %), adjustment disorder (n=4, 1.8 %), pain disorder (n=4, 1.8 %), panic/agoraphobia (n=3, 1.4 %), dysthymia (n=2, 0.9 %), post-traumatic stress disorder (n=2, 0.9 %), depression NOS (n=3, 1.4 %), somatization disorder (n=1, 0.5 %), specific phobia (n=2, 0.9 %), and no diagnosis (n=33, 14.9 %).

Table 1

Sample Characteristics

Factor Structure

Five structural models were tested: (a) a one-factor model in which all items were fixed to load on a single factor labeled negative affect; (b) a two-factor model collapsing the anxiety and stress scales consistent with the 1997 investigation by Brown et al. of the DASS; (c) a two-factor model collapsing anxiety and depression consistent with claims that GAD and depression are not independent constructs in older adults (Schoevers, Beekman, Deeg, Jonker & van Tilburg, 2003; Schoevers et al., 2005) and assuming that GAD symptomatology is best captured by the anxiety scale; (d) a two-factor model collapsing depression and stress also consistent with claims that GAD and depression are not independent constructs but that GAD symptomatology is best captured by the stress scale, as suggested in the literature (Brown et al., 1997); and (e) a three-factor model consistent with structures reported in Lovibond and Lovibond’s (1995b) original scale and in the 1998 examination of the DASS-21 by Antony et al.

CFA was performed using maximum-likelihood estimation with standard errors robust to non-normality with Mplus version 5 (Muthén & Muthén, Los Angeles, CA). Fit indices for the separate models are listed in Table 2. Due to limitations of chi-square tests, the Bayesian information criterion (BIC) was used to compare whether successive models resulted in improved overall fit (Hu & Bentler, 1998). Smaller BIC values indicate a better model fit, although the absolute value is not interpretable. The three factor model (Model E) resulted in the best fit among the tested models and yielded an acceptable descriptive fit based on Hu and Bentler’s recommended cut-offs (SRMR ≤ 0.08; RMSEA ≤ 0.06; CFI/ TLI ≥ 0.9).

Table 2

Fit Indices for Tested Models

Internal Consistency

Internal consistency was calculated using the entire sample. Reliability for the DASS-21 total score was ρ=.94. Consistent with psychometric theory, scale reliabilities were slightly lower for DASS-D (ρ=.87) and DASS-S (ρ=.89) but less than expected for DASS-A (ρ=.69). Cronbach’s alpha was also calculated for each of the other measures used in this study and ranged between .86 and .90.

Convergent Validity

Pearson correlation coefficients were calculated between the DASS and other measures used in this study (See Table 3²). As expected, significant positive correlations emerged for DASS-21 total scores with measures assumed to represent similar constructs (BAI, BDI, PSWQ, and PANAS-N) and significant negative correlations with measures assumed to represent dissimilar constructs (PANAS-P and QOLI). At the scale level, significant correlations were observed in the predicted direction with all other instruments for the DASS-D, DASS-S, and DASS-A (p < .05 for all), although the DASS-S did not correlate as strongly with the PSWQ as expected.

Table 3

Correlations Between DASS-21 and Other Measures

DASS scales correlated highest with measures of like constructs and lowest with measures of unlike constructs. For instance, the DASS-D was more positively correlated with the BDI-II (r=.76) than with any other measure: BAI (r=.51, z=5.73, p <.001); PSWQ (r=.52, z=5.25, p <.001); PANAS-PA (r=−.49, z=12.85, p <.001); PANAS-NA (r=.60, z=4.07 , p <.05); and QOLI (r=−.58, z=13.68, p <.001). Likewise, the DASS-A was more positively correlated with the BAI (r=.74) than other measures: BDI-II (r=.47, z=5.73, p <.001); PSWQ (r=.34, z=7.22, p <.001); PANAS-PA (r=−.19, z=10.27, p <.001); PANAS-NA (r=.57, z=4.20 , p <.001); and QOLI (r=−.19, z=10.18, p <.001). The DASS-S was more positively correlated with the PANAS-NA (r=.74), than other measures: BDI-II (r=.62, z=2.94, p <.005); BAI (r=.59, z=2.82, p <.05); PSWQ (r=.57, z=3.07, p <.05); PANAS-PA (r=−.24, z=12.34, p <.001); and QOLI (r=−.33, z=13.26, p <.001).

Discriminative Validity

Participants with a mood disorder scored significantly higher on the DASS-D relative to other diagnostic groups (see Table 4). Both depressive groups scored significantly higher than the GAD-only group (Mood Disorder Alone: β=.69, p<.001; GAD/Mood: β=.54, p<.05) and no diagnosis group (Mood Alone: β=1.52, p<.001; GAD/Mood: β=1.37, p<.001). Likewise, the group without a psychiatric diagnosis scored significantly lower on the DASS-D than all other groups. The two depressive groups did not significantly differ from one another.

Table 4

Comparisons of Diagnostic Groups on the DASS

Our prediction that all three diagnostic categories would score significantly higher on the DASS-A than the nonpsychiatric group was supported: GAD (β=.51, p<.001); Comorbid GAD/mood disorder (β=.76, p<.001); Mood Disorder Alone (β=.48, p<.05). The differences between the three diagnostic groups were not significant.

With respect to the DASS-S, our hypothesis (that the GAD and comorbid GAD/mood diagnostic groups would be significantly different from the mood and no diagnosis groups) was once again partially supported. Participants without a psychiatric diagnosis scored significantly lower on the DASS-S than all other groups (GAD: β=−1.23, p<.001; comorbid GAD/mood: β=− 1.51, p<.001; Mood: β=−1.14, p<.001). Further, the Mood alone group scored significantly lower than the GAD/mood group (β=−.36, p<.05).

Diagnostic Utility

Given that results from the CFAs suggested that the DASS does not represent a unitary construct, ROC analyses were conducted for each diagnostic group using the DASS scales. The first ROC analysis evaluated the ability of the DASS to identify GAD. The area under the curve (AUC), a statistic that varies between .5 (chance classification) and 1.0 (perfect diagnostic accuracy), for the three scales, was: (a) DASS-D, A_z=.63, p < .001; (b) DASS-A, A_z=.60, p < .05; and (c) DASS-S, A_z=.70, p < .001. Pairwise comparisons of ROCs indicated that the AUC for DASS-S was greater than that for DASS-D, X²(1)=9.9, p < .01 and DASS-A, X²(1)=5.6, p < .05. To provide a comparison, these analyses were repeated for scales associated with GAD: PSWQ (A_z=.71, p < .001, and PANAS-NA (A_z=.67, p < .001). The AUC for the DASS-S, PSWQ, and PANAS-NA was not significantly different from each other (p >.05 for all), suggesting that the DASS-S predicts the diagnostic presence of GAD as well as the PSWQ or PANAS-NA. The PSWQ did obtain higher AUC values in an earlier analysis with a subset of the sample included here, however (Webb, et al., in press). Diagnostic- accuracy calculations with respect to GAD are listed for the DASS-S in Table 5 ≥. Based on these calculations, a raw DASS-S scale score ≥ 14 represents the optimal cutoff score to best balance specificity and sensitivity when attempting to classify older adults in a primary care setting into diagnostic categories of GAD vs. non-GAD.

Table 5

Diagnostic Accuracy Values for the DASS-S When Predicting GAD

The second set of ROC analyses evaluated the ability of the DASS to identify mood disorders (MDD and dysthymia). The AUC for the three scales was (a) DASS-D, A_z=.77, p < .001; (b) DASS-A, A_z=.60, p < .05; and (c) DASS-S, A_z = .62, p < .05. The AUC was A_z=.76, p < .001 for the BDI-II. Pairwise comparisons of ROCs suggested greater AUC for the DASS-D than the DASS-A, X²(1)=12.6, p <.001, and DASS-S, X²(1)=15.0, p <.001, but not for the BDI-II, p. >05. This suggests that the DASS-D and BDI-II are equally able to predict the diagnostic presence of depression. Diagnostic-accuracy calculations for the DASS-D are displayed in Table 6. Based on these calculations, a raw DASS-D score ≥ 6 represents the optimal cutoff score derived to maximize sensitivity and specificity when attempting to classify older adults in a primary care setting into diagnostic categories of mood vs. non-mood categories. Higher cut scores on the DASS-D may be needed, however, depending on the purpose of the assessment and consequences of false-positives vs. false-negatives.

Table 6

Diagnostic Accuracy Values for the DASS-D When Predicting Mood Disorders

Discussion

This study is the first to investigate the psychometric properties of the DASS-21 in older adults. Analyses examined the factor structure, internal consistency, convergent and discriminative validity, and diagnostic utility of the DASS-21. CFA of the DASS-21 indicated that a-three factor solution best fit the data. This is consistent with findings from populations of adult anxiety-disordered patients (Antony et al., 1998), adult Spanish patients (Daza et al., 2002), and adult mood-disordered patients (Clara et al., 2001). Consistent with research on the DASS-42 (Brown et al, 1997), the three-factor solution of the DASS-21 was superior to a two-factor solution that collapsed the Anxiety and Stress scales. The three-factor solution also produced a better fit than the two-factor solutions that collapsed the Anxiety and Depression or Stress and Depression scales. This result is in contrast to suggestions that GAD and depression are not independent constructs in older adults (Schoevers et al., 2003, Schoevers, et al. 2005). This suggests that a constructural distinction between depression and anxiety/stress is necessary in this sample of older adults with predominately GAD and depression. Thus, despite significant zero-order inter-scale correlations, the present results suggest the presence of three distinct factors in older adult primary care patients.

Consistent with examinations in younger adults, the DASS-21 total score demonstrated excellent internal consistency. At the scale level, the DASS-D and DASS-S had good internal consistency; whereas the DASS-A had poor internal consistency. Although a previous investigation of the DASS-21 also reported the lowest scale internal consistency for the DASS-A (α=.87) (Antony, et al. 1998), the lower-than-expected value in this population may represent chance or a systematic difference between younger and older adults with respect to these items. Older patients present with more somatic complaints which may result in less variability in their responses than younger patients. Indeed, post-hoc exploratory factor analyses of the DASS-21 in this sample revealed that the 7 items of the DASS-A loaded onto two separate factors. Unfortunately the loadings were inconsistent across subsequent sample divisions and rotations. Regardless, caution in the interpretation of the DASS-A – and other measures of anxiety that utilize items with somatic content – is necessary in older patients, unless such effects can be ruled out. Future studies are clearly needed to examine this hypothesis.

Results strongly supported the convergent validity of the DASS-21 in older adults. The pattern of correlations between the DASS-21 total score and scale scores with associated measures was consistent with a priori predictions. Correlations between DASS scales and measures of similar constructs (i.e., BDI, BAI, and PSWQ/PANAS-NA) were consistently significantly related and nearly unanimously superior to comparison measures.

Results also provide qualified evidence for the discriminative validity of the DASS-21 in older adults. The three DASS-21 scales were tested for their ability to detect group mean differences between participants diagnosed with primary GAD, mood disorders, comorbid GAD/mood disorders, and no diagnosis. Participants diagnosed with GAD, mood, and comorbid GAD/mood scored significantly higher on each of the scales than participants without a DSM-IV diagnosis. Further, scores on the DASS-D were higher for both the pure and comorbid depression groups than the pure GAD group. Likewise, scores on the DASS-S were higher in the comorbid GAD/mood group than in the pure mood group. In contrast, no significant differences emerged across the three diagnostic groups on the DASS-A. It should be noted, however, that the patterns of means were consistent with all a priori hypotheses and may reflect Type II error. In summary, it appears as if all three DASS-21 scales are able to differentiate pathological and nonpathological samples. Moreover, the DASS-D appears to be especially adept at differentiating patients with and without depression even in the presence of comorbidity. The DASS-S, in contrast, seemed to differentiate the comorbid group from the mood only group. The performance of the DASS-A should be interpreted with caution due to characteristics of the current sample.

Using ROC analyses, we examined the diagnostic utility of the DASS-21 scales with respect to the prediction of GAD and mood disorders. These analyses indicated that the DASS-S differentiates positive versus negative GAD status equally as well as the PSWQ and PANAS-NA. Analyses also indicated that the DASS-D scale better differentiates positive versus negative mood status than the other two scales and equally as well as the BDI-II. Overall the AUC was relatively low for both scales, and especially so for the DASS-S. In comparison, the 14-item Hospital Anxiety and Depression Scale (HADS) demonstrated a larger AUC (.80) when predicting GAD in geriatric primary care patients (Wetherell, Birchler, Ramsdell, & Unüzer, 2007). Nevertheless, these results are noteworthy in that the 7 item scales of the DASS-21 predict the diagnostic status of GAD and mood disorders equally as well as two separate measures (e.g., PSWQ=16 items and BDI-II=21 items) and therefore have the potential to reduce patient fatigue. Although short forms of the BDI and BAI exist (e.g., Mori et al., 2003; Scheinthal et al., 2001), most researchers use the full scales. The relative advantages of using the DASS-21 or the HADS are related to the additional information yielded by the three integrated scales of the DASS. If further research using different samples replicates these findings and extends the results to the DASS-A scale, the DASS-21 is one candidate for an economical screening questionnaire in older adults.

Several limitations must be considered when interpreting these results. First, all instruments examined in this study are self-report measures and are subject to similar sources of method error. Future studies using a multi-trait, multi-method approach might reveal different sources of variance and suggest whether the observed weaknesses in group-level discriminative validity result from construct definition, rater source, or both. A second limitation surrounds the relatively homogeneous diagnostic constellation of this convenience sample. Whereas the sample reflects both the goals of the overarching treatment study and the prevalence of mental disorders in older adults, a greater sampling of other anxiety diagnoses is needed to fully explore the psychometric properties in older adults and allow comparison with younger adults. Likewise, participants were self-referred and screened positive to the two PRIME-MD questions. It is possible that some of the results would have been stronger in the larger population of standard clinical care patients who did not answer positive to screening questions Further, the recommended cut-off scores are dependent on diagnostic aims and may need to be lowered in clinical care where a false-negative has worse consequences than a false-positive. Finally, the cross-sectional nature of these data does not allow interpretation of psychometric stability over time. Longitudinal designs across the life-span and following therapy are encouraged in future studies.

In conclusion, the DASS-21 demonstrated positive psychometric properties in a population of older primary care patients. Results indicate that the DASS-21 has overall good-to-excellent internal consistency, a three-factor structure consistent with younger samples, very good convergent validity, and acceptable discriminative validity – especially with respect the depression scale. The difficulties surrounding group- level discriminative validity are common to all measures of anxiety and depression and, based on inter-scale correlations, the DASS-21 may be better than most. These results indicate that the DASS-21 should be further examined and potentially used as a routine screening device in older adults in primary care settings.

Acknowledgements

This research was supported by Grant 53932 from the National Institute of Mental Health to the last author. The NIMH had no further role in study design, data collection, data analysis, interpretation, writing of the manuscript, or in the decision to submit the paper for publication. None of the authors have conflicts of interest. Portions of this work were presented at the 2006 convention for the Association for Behavioral and Cognitive Therapies, November, Chicago. The authors wish to express their appreciation to Paula Wagner, Jessica Calleo, Antje Dietel, and Sonora Hudson for their help in preparing this manuscript and to the reviewers for their helpful comments.

Footnotes

This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

¹Information on demographics and reasons for refusal was not collected from patients who refused to participate

²A full correlation matrix of all measures used in this study is available upon request.

References

American Psychiatric Association. Diagnostic and statistical manual of mental disorders. 4th ed. Washington, D.C: Author; 2000.
Antony M, Bieling PJ, Cox BJ, Enns MW, Swinson RP. Psychometric properties of the 42-item and 21-item versions of the Depression Anxiety Stress Scales in clinical groups and community a sample. Psychological Assessment. 1998;10:176–181.
Beck JG, Stanley MA. Assessment of anxiety disorder in older adults: Current concerns, future prospects. In: Antony MM, Orsillo SM, Roemer L, editors. Practitioner’s guide to empirically based measures of anxiety. Dordrecht: Kluwer Academic Publishers; 2001. pp. 43–48.
Beck AT, Steer RA. Beck Anxiety Inventory: Manual. San Antonio, TX: The Psychological Corporation; 1990.
Beck AT, Steer RA, Brown GK. Beck Depression Inventory Manual. 2nd Edition. San Antonio, TX: Psychological Corporation; 1996.
Beck JG, Novy DM, Diefenbach GJ, Stanley MA, Averill PM, Swann AC. Differentiating anxiety and depression in older adults with generalized anxiety disorder. Psychological Assessment. 2003;15:184–192. [PubMed]
Beck JG, Stanley MA, Zebb BJ. Psychometric properties of the Penn State Worry Questionnaire in older adults. Journal of Clinical Geropsychology. 1995;1:33–42.
Beekman AT, Bremmer MA, Deeg DJH, van Baldom AJLM, Smit JH, de Beurs E, van Dyck R, van Tilburg WV. Anxiety disorders in later life: A report from the Longitudinal Aging Study Amsterdam. International Journal of Geriatric Psychiatry. 1998;13:717–726. [PubMed]
Beekman ATF, de Beurs E, Balkom AJLM, Deeg DJH, van Dyck R, van Tilburgh W. Anxiety and depression in later life: Co-occurrence and communality of risk factors. American Journal of psychiatry. 2000;157:89–95. [PubMed]
Blazer D, George LK, Hughes D. The epidemiology of anxiety disorders: An age comparison. In: Salzman C, Lebowitz BD, editors. Anxiety in the elderly: Treatment and research. New York: Springer; 1991. pp. 17–30.
Bourland SL, Stanley MA, Snyder AG, Novy DM, Beck JG, Averill PM, Swann AC. Quality of life in older adults with generalized anxiety disorder. Aging & Mental Health. 2000;4:315–323.
Brown TA. Confirmatory Factor Analysis for Applied Research. New York: Guilford; 2006.
Brown TA, Antony MM, Barlow DH. Psychometric properties of the Penn State Worry Questionnaire in a clinical disorders sample. Behavior and Research Therapy. 1992;30:33–37. [PubMed]
Brown TA, Chorpita BF, Korotitsch W, Barlow DH. Psychometric properties of the Depression Anxiety Stress Scales (DASS) in clinical samples. Behaviour Research and Therapy. 1997;35:79–89. [PubMed]
Clara IP, Cox BJ, Enns MW. Confirmatory factor analysis of the Depression Anxiety Stress Scales in depressed and anxious patients. Journal of Psychopathology and Behavioral Assessment. 2001;23:61–67.
Crawford JR, Henry JD. The Depression Anxiety Stress Scales (DASS): Normative data and latent structure in a large non-clinical sample. British Journal of Clinical Psychology. 2003;42:11–131. [PubMed]
Daza P, Novy DM, Stanley MA, Averill P. The Depression Anxiety Stress Scale-21: Spanish translation and validation with a Hispanic sample. Journal of Psychopathology and Behavioral Assessment. 2002;24:195–205.
Delong ER, Delong DM, Clarkepearson DI. Comparing the areas under 2 or more correlated receiver operating characteristic curves: A nonparametric approach. Biometrics. 1988;44(3):837–845. [PubMed]
Folstein MF, Folstein SE, McHugh PR. “Mini-mental state.” A practical method for grading the cognitive state of patients for the clinician. Journal of Psychiatric Research. 1975;12:189–198. [PubMed]
Fresco DM, Heimberg RG, Mennin DS, Turk CL. Confirmatory factor analysis of the Penn State Worry Questionnaire. Behaviour Research and Therapy. 2002;40:313–323. [PubMed]
Frisch MB, Cornell J, Villanueva M. Clinical validation of the Quality of Life Inventory: A measure of life satisfaction for use in treatment planning and outcome assessment. Psychological Assessment. 1992;4:92–101.
Frisch MB. Manual and treatment guide for the Quality of Life Inventory. Minneapolis: National Computer Systems, Inc; 1994.
Frisch MB. Quality of life assessment/intervention and the Quality of Life Inventory. In: Maruish ME, editor. The use of psychological testing for treatment planning and outcomes assessment. 2nd ed. Mahwah N.J: Lawrence Erlbaum; 1999.
Hopko DR, Stanley MA, Reas DL, Wetherell JL, Beck JG, Novy DM, Averill PM. Assessing worry in older adults: Confirmatory factor analysis of the Penn State Worry Questionnaire and psychometric properties of an abbreviated model. Psychological Assessment. 2003;15:173–183. [PubMed]
Hu LT, Bentler PM. Fit indices in covariance structure modeling: Sensitivity to underparameterized model misspecification. Psychological Methods. 1998;3(4):424–453.
Kabacoff RI, Segal DL, Hersen M, Van Hassellt VB. Psychometric properties and diagnostic utility of the Beck Anxiety Inventory and the State-Trait Anxiety Inventory with older adult psychiatric outpatients. Journal of Anxiety Disorders. 1997;11:33–47. [PubMed]
Kercher K. Assessing subjective well-being in the old-old: The PANAS as a measure of orthogonal dimensions of positive and negative affect. Research on Aging. 1992;42:131–168.
Kessler RC. The prevalence of psychiatric comorbidity. In: Wetzler S, Sanderson WC, editors. Treatment strategies for patients with psychiatric comorbidity. New York: Wiley; 1997.
Lenze EJ, Karp JF, Mulsant BH, Blank S, Shear MK, Houck PR, et al. Somatic symptoms in late-life anxiety: treatment issues. Journal of GeriatricPsychiatry and Neurology. 2005;18:89–96. [PubMed]
Lenze EJ, Rogers JC, Martire LM, Mulsant BH, Rollman BL, Dew MA, et al. The association of late-life depression and anxiety with physical disability: A review of the literature and prospectus for future research. American Journal of Geriatric Psychiatry. 2001;9:113–135. [PubMed]
Lovibond SH, Lovibond PF. Manual for the Depression Anxiety Stress Scales. Sydney: Psychology Foundation; 1995a.
Lovibond PF, Lovibond SH. The structure of negative emotional states: Comparison of the Depression Anxiety Stress Scales (DASS) with the Beck Depression and Anxiety Inventories. Behaviour Research and Therapy. 1995b;33:335–343. [PubMed]
Mackinnon A, Jorm AF, Christensen H, Korten AE, Jacomb PA, Rodgers B. A short form of the Positive and Negative Affect Schedule: Evaluation of factorial validity and invariance across demographic variables in a community sample. Personality and Individual Differences. 1999;27(3):405–416.
Meng XL, Rosenthal R, Rubin DB. Comparing Correlated Correlation-Coefficients. Psychological Bulletin. 1992;111(1):172–175.
Meyer TJ, Miller ML, Metzger RL, Borkovec TD. Development and validation of the Penn State Worry Questionnaire. Behaviour Research and Therapy. 1990;28:487–495. [PubMed]
Mori DL, Lambert JF, Niles BL, Orlander JD, Grace M, LoCastro JS. The BAI-PC as a screen for anxiety, depression, and PTSD in primary care. Journal of Clinical Psychology in Medical Settings. 2003;10:187–192.
Morin CM, Landreville P, Colecchi C, McDonald K, Stone J, Ling W. The Beck Anxiety Inventory: Psychometric properties with older adults. Journal of Clinical Geropsychology. 1999;5:19–29.
Muthen LK, Muthen BO. How to use a Monte Carlo study to decide on sample size and determine power. Structural Equation Modeling. 2002;9(4):599–620.
Raykov T. Behavioral scale reliability and measurement invariance evaluation using latent variable modeling. Behavior Therapy. 2004;35(2):299–331.
Scheinthal SM, Steer RA, Giffin L, Beck AT. Evaluating geriatric medical outpatients with the Beck Depression Inventory-FastScreen for medical patients. Aging & Mental Health. 2001;5:143–148. [PubMed]
Schoevers RA, Beekman ATF, Deeg DJH, Jonker C, van Tilburg W. Comorbidity and risk-patterns of depression, generalized anxiety disorder and mixed anxiety-depression in later life: Results from the AMSTEL study. International Journal of Geriatric Psychiatry. 2003;18:994–1001. [PubMed]
Schoevers RA, van Tilburg W, Beekman ATF, Deeg DJH. Depression and generalized anxiety disorder: Co-occurrence and longitudinal patterns in elderly patients. American Journal of Geriatric Psychiatry. 2005;13:31–39. [PubMed]
Segal DL, Hersen M, Van Hasselt VB, Kabacoff RI, Roth L. Reliability of diagnosis in older psychiatric patients using the Structured Clinical Interview for DSM-III-R. Journal of Psychopathology and Behavioral Assessment. 1993;15:347–356.
Senior AC, Kunik MEK, Rhoades HM, Novy DM, Wilson NL, Stanley MA. Utility of telephone assessments in an older adult population. Psychology and Aging. 2007;22:392–397. [PubMed]
Shapiro AM, Roberts JE, Beck JG. Differentiating symptoms of anxiety and depression in older adults: Distinct cognitive and affective profiles? Cognitive Therapy and Research. 1999;23:53–74.
Spitzer RL, Williams JB, Gibbon M, First MB. The Structured Clinical Interview for DSM-III-R (SCID). I. History, rationale, and description. Archives of General Psychiatry. 1992;49:624–629. [PubMed]
Spitzer RL, Williams JB, Kroenke K, Linser M, de Gruy FV, Hahn SR, et al. Utility of a new procedure for diagnosing mental disorders in primary care. The PRIME-MD 1000 study. Journal of the American Medical Association. 1994;272:1749–1756. [PubMed]
Stanley MA, Beck JG, Novy DM, Averill PM, Swann AC, Diefenbach GJ, Hopko DR. Cognitive-behavioral treatment of late-life generalized anxiety disorder. Journal of Counseling and Clinical Psychology. 2003;71:309–319. [PubMed]
Stanley MA, Novy DM, Bourland SL, Beck JG, Averill PM. Assessing older adults with generalized anxiety: A replication and extension. Behaviour Research and Therapy. 2001;39:221–235. [PubMed]
StataCorp. Stata Statistical Software. College Station, TX: Stata Corporation; 2006. Release 9.2.
Steer RA, Ball R, Ranieri WF, Beck AT. Further evidence for the construct validity of the Beck Depression Inventory-II with psychiatric outpatients. Psychological Reports. 1997;80:443–446. [PubMed]
Steer RA, Rissmiller DJ, Beck AT. Use of the Beck Depression Inventory-II with depressed geriatric inpatients. Behaviour Research and Therapy. 2000;38:311–318. [PubMed]
Steer RA, Willman M, Kay PAJ, Beck AT. Differentiating elderly medical and psychiatric outpatients with the Beck Anxiety Inventory. Assessment. 1994;4:345–351.
Taylor R, Lovibond PF, Nicholas MK, Cayley C, Wilson PH. The utility of somatic items in the assessment of depression in patients with chronic pain: A comparison of the Zung Self-Rating Depression Scale and the Depression Anxiety Stress Scales in chronic pain and clinical and community samples. The Clinical Journal of Pain. 2005;25:91–100. [PubMed]
Tolin DF, Robison JT, Gaztambide S, Blank K. Anxiety disorders in older Puerto Rican primary care patients. American Journal of Geriatric Psychiatry. 2005;13:150–156. [PubMed]
Waikar SV, Craske MG. Cognitive correlates of anxious and depressive symptomatology: An examination of the helplessness/hopelessness model. Journal of Anxiety Disorders. 1997;11:1–16. [PubMed]
Watson D, Clark LA, Tellegen A. Development and validation of brief measures of positive and negative affect: The PANAS scales. Journal of Personality and Social Psychology. 1988;5:1063–1070. [PubMed]
Watson D, Walker LM. The long-term stability and predictive validity of trait measures of affect. Journal of Personality and Social Psychology. 1996;70:567–577. [PubMed]
Webb SA, Diefenbach G, Wagener P, Novy DM, Kunik M, Rhoades HM, Stanley MA. Comparison of self-report measures for identifying late-life generalized anxiety in primary care. Journal of Geriatric Psychiatry and neurology. (in press)
Wetherell JL, Gatz M. The Beck Anxiety Inventory in older adults with generalized anxiety disorder. Journal of Psychopathology and Behavioral Assessment. 2005;27:17–24.
Wetherell JL, Arean PA. Psychometric evaluation of the Beck Anxiety Inventory with older medical patients. Psychological Assessment. 1997;9:135–144.
Wetherell JL, Birchler GD, Ramsdell J, Unüzer J. Screening for generalized anxiety disorder in geriatric primary care patients. International Journal of Geriatric Psychiatry. 2007;22:115–123. [PubMed]
Wittchen HU, Schuster P, Lieb R. Comorbidity and mixed anxiety-depressive disorder: Clinical curiosity or pathophysiological need? Human Psychopharmacology. 2001;16:S21–S30. [PubMed]