The present study examined the validity and diagnostic efficiency of the DTS (Davidson, Book, et al., 1997
) in a sample of veterans who have served since September 11th, 2001. Posttraumatic stress disorder is a relatively prevalent condition among these veterans, and there is a strong need for valid assessment tools. Brief self-report questionnaires, such as the DTS, can provide a considerable aid to the clinician and researcher in identifying those who are experiencing symptoms of PTSD. However, an invalid or misused diagnostic aid can cause more harm than good, considering the major impact that a diagnosis or positive screening can have in a patient’s treatment and subsequently his or her quality of life (Streiner, 2003
). Thus, it is critical that self-report measures of PTSD are valid in order to provide accurate aid to assessment, conduct quality research, and offer the best care to patients.
Results of the current study provide support for the DTS as a valid self-report measure of PTSD symptoms for veterans serving since 9/11, particularly in comparison to individuals without a psychiatric diagnosis. Concurrent validity was supported, as veterans with PTSD scored higher on the DTS than those with other Axis I diagnoses or no diagnosis. Convergent and divergent validity were supported, as the DTS had a stronger relationship with anxiety-related symptom scales compared to those tapping other psychopathology. The DTS demonstrated good internal consistency (alpha = 0.97), matching or exceeding alpha reported for other PTSD symptom questionnaires (Norris & Hamblen, 2004
). However, item 7, regarding the respondent’s memory of the traumatic event, was endorsed by less than half of the participants with PTSD, and demonstrated relatively low inter-correlations with other DTS items. This finding is consistent with prior research indicating that loss of memory for the event is not a reliable predictor of PTSD (Davidson, Book, et al., 1997
; Foa, Riggs, & Gershuny, 1995
), and suggests that this item would benefit from rewording or perhaps closer scrutiny as a core symptom of PTSD.
Concerning diagnostic efficiency, potency of the DTS was dependent on the comparison group used for analyses: whereas diagnostic efficiency was excellent when discriminating between veterans with PTSD and veterans with no diagnosis (AUC = 0.95), efficiency was attenuated, although still acceptable, when discriminating between PTSD and other Axis I diagnoses (AUC = 0.83). When attempting to discriminate between veterans with PTSD and those with no Axis I diagnosis, the diagnostic efficiency of the DTS in the current study (AUC = 0.95) was superior to that reported by Davidson, Book, et al. (1997
; AUC = 0.88). Whereas Davidson and colleagues reported that a DTS score of 40 provided the best efficiency (0.83), the current study found that a lower cut-point of 32 was more efficient in this sample of post-9/11 veterans (efficiency = 0.94).
Although the efficiency statistic is a common indicator of a measure’s performance, the cut-point employed by a researcher or clinician should also be informed by the intended application. One should note that efficiency reflects the overall hit rate and thus places equal value on obtaining false positives and false negatives. In practice, other utility functions that place more or less emphasis on obtaining false positives may be preferred. For example, if the DTS was used as a PTSD screening tool to identify individuals who may benefit from more time and resource-intensive assessment, a clinician would likely want to use a lower cut-point to identify as many individuals with PTSD as possible (i.e., maximize sensitivity) while accepting the risk of increasing false positives. In another circumstance, a researcher with a limited budget may want to use a higher cut-point to reduce the number of false positives enrolled in the study (i.e., maximize specificity).
In addition to testing the ability of the DTS to discriminate between PTSD and healthy veterans, this study also examined its capability to discern PTSD from other Axis I disorders. This scenario is likely of more interest to clinicians and researchers in mental health clinics and psychiatric research, who may use PTSD symptom questionnaires to aid differential diagnosis. The DTS performed adequately in this circumstance (AUC = 0.83), albeit not as well as when discriminating PTSD from healthy veterans. A cut-point of 35 provided the optimal balance of identifying as many veterans with PTSD as possible, while maximizing efficiency (0.77): 37 of 39 veterans with PTSD were correctly classified in this study, although specificity was poor, with 14 of 32 vets with other Axis I disorders mistakenly identified as having PTSD. If a clinician or researcher wishes to minimize false positives (while maximizing efficiency), a cut-point of 75 was best: in the current study, only three false positives (of 32, or 9%) were returned.
Results from our study examined the range of values for positive and predictive power that correspond to PTSD prevalence rates in several scenarios (post-9/11 veterans receiving care, post-9/11 veterans with any mental health diagnosis, 90% of treatment-seeking veterans in a specialty PTSD clinic), and these data illustrate that PPP and NPP vary depending on the prevalence of the disorder in the clinic’s population. When prevalence of a condition is low (e.g., primary care), a test is best used to rule out a condition but not to rule it in. For example, when applying the cutting score that maximized hit-rates between those with PTSD and without a psychiatric disorder in a primary care clinic setting, a positive test result would be correct 61% of the time. Thus, 39% of the time positive test results would wrong. Similarly, when the prevalence of a condition is high such as a specialty PTSD clinic, a test is best used to rule in a condition but not to rule it out (Streiner, 2003
Concerning the optimal scoring method for testing the diagnostic efficiency of the DTS, results of this study support the conventional cut-point method over a DSM-IV-based, symptom cluster method. Although the authors of the DTS utilized the cut-point method in an early validation study (Davidson, Book, et al., 1997
), other PTSD symptom questionnaires have been scored using variations of both methods (e.g., Foa et al., 1997
; Ruggiero, Del Ben, Scotti, & Rabalais, 2003
). Results of the current study indicated that although the symptom cluster method was effective at correctly classifying veterans with PTSD from those with no Axis I disorders, the cut-point method was more efficient. These findings are consistent with a recent review that found little benefit in utilizing more complex scoring methods instead of the conventional cut-point strategy (Brewin, 2005
An important issue concerning PTSD symptom questionnaires that needs further research is the potential impact of anchoring responses to one particular traumatic event (Norris & Hamblen, 2004
). Clarifying the nature of the disturbing event would appear especially important, given evidence that persons with other psychiatric disturbances, such as major depression, can report symptoms consistent with PTSD even though they have not been exposed to a Criterion A traumatic event (Bodkin, Pope, Detke, & Hudson, 2007
). The DTS attempts to capture the index trauma by requiring respondents to write a brief description of the Criterion A traumatic event “that is most disturbing to you” (Davidson, 1996
). For this reason, the DTS ostensibly has an advantage over other PTSD measures that do not document the index trauma, such as the PCL-C (Blanchard, Jones-Alexander, Buckley, & Forneris, 1996
) and the PC-PTSD (Prins et al., 2003
) when used as a PTSD screening tool. However, in this study, 20% of the participants recorded a disturbing event on the DTS that could not be clearly identified as a discrete DSM-IV PTSD Criterion A1 “trauma.” Certainly, many of the participants had a bona fide trauma in mind, as evinced by the 16 of the 47 excluded participants who met SCID I/P diagnosis for PTSD. Perhaps some participants refrained from providing detailed trauma descriptions due to fearful avoidance of reminders, or other motivational factors such as guilt, shame, or fatigue. For whatever reasons, these cases demonstrate the limitation of using an open-ended response format to record traumatic stressors on self-report questionnaires. Although rarely done in practice and not generally reported in previous studies using self-report PTSD instruments, it would seem important to assess whether the event meets criteria for a “traumatic event” prior to the respondent completing the symptom inventory. In this regard, itmay be helpful to anchor the DTS symptom ratings to the outcome of a trauma rating scale, such as the Traumatic Life Events Questionnaire (Kubany et al., 2000
; cf. PDS, Foa et al., 1997
It is notable that this sample entirely consisted of veterans who have served since September 9th, 2001. Although this specificity will be useful to those working with veterans who have served in the post-9/11 era, the extent to which these results generalize to other populations is unclear. In addition, although the participant group employed for this study was diverse, the sample was not large enough to fully examine ethnic or racial differences in diagnostic efficiency for the DTS. Further research is needed to determine generalizability of these findings across demographic groups (e.g., race, gender, and first language), the range of symptom severity and comorbidity, and a variety of settings (i.e., primary care).
Our results suggest that use of the DTS as a diagnostic screening in settings where rates of other Axis I disorders are high (e.g., a mental health clinic), particularly at the level of usual recommended screening cutoff scores, substantial diagnostic errors could result. Several symptoms associated with PTSD are also common in other Axis I mental disorders (e.g., concentration problems in Major Depressive Disorder). Thus, it is not surprising that those with an Axis I diagnosis other than PTSD had elevated DTS scores relative to those with no diagnosis, and subsequently were far more difficult to distinguish from those with PTSD. This result is also consistent with other research suggesting that PTSD symptom instruments overlap substantially with other psychological distress symptoms (Lauterbach, Vrana, King, & King, 1997
). Results are also consistent with a taxometric approach to psychopathology which acknowledges the nonspecific symptoms of distress reported by patients with PTSD, depressive disorders, and Generalized Anxiety Disorder (e.g., Bodkin et al., 2007
; O’Donnell, Creamer, & Pattison, 2004
; Watson, Gamez, & Simms, 2005
). In addition, these findings provide practical support for the view that only symptoms with diagnostic specificity be retained in future revisions of the DSM (McHugh & Treisman, 2007
; Spitzer, First, & Wakefield, 2007
Unfortunately, no reports are available concerning the ability of the PTSD Checklist (PCL) or the Posttraumatic Diagnostic Scale (Foa et al., 1997
) to discriminate between PTSD and other Axis I disorders. Our results suggest that this would be an important avenue for future studies evaluating self-report instruments for PTSD. In addition, we urge clinicians and researchers who utilize a measure’s diagnostic efficiency statistics as part of their diagnostic assessment to consider whether the comparison group used in the validation study is an appropriate match to the population they serve. Furthermore, we reiterate the consensus of the authors of the DTS that it should not be used alone to make a diagnosis of PTSD. Instead, the DTS is foremost a measure of symptom severity, and secondly an ally in comprehensive clinical assessment.