Patients and procedure
We analyzed cross-sectionally baseline data collected for a randomized controlled trial concerning an activating social work intervention for distressed patients in general practice [9
]. Patients were included in the study if they were aged 18–60, had a 'nervous breakdown' according to the GP, were employed, and had been on sick leave for no longer than three months. Dutch GPs use the diagnostic label 'nervous breakdown' (or 'overstressed', in Dutch 'overspanning') for a syndrome that is associated with too much stress to the extent that the patient cannot cope anymore [10
]. This condition is characterized by psychological distress, failure to cope, and social dysfunctioning (i.e. sick leave in employed people) [12
]. The GPs were instructed not to include patients with obvious depressive and anxiety disorders (i.e. patients in whom they had clinically diagnosed such disorders). Adequate command of the Dutch language, and no current psychological treatment were additional inclusion criteria. Between August 2001 and July 2003, 70 GPs in the city of Almere, the Netherlands, assessed and referred 370 patients to the study centre. The patients were contacted by telephone, information about the study was given and inclusion criteria were checked. Thirty patients were unwilling to participate, and a further 33 patients did not meet the inclusion criteria. With the remaining patients an appointment was made for a baseline interview in their homes and they were sent written information and baseline questionnaires by post. The home visit took place on average 5.7 days after the initial telephone contact. After the patients had been fully informed about the study, written informed consent was obtained. The participating patients (n = 307) were then interviewed, and subsequently their questionnaires were checked for missing values by the interviewer. Unfortunately, the date at which the questionnaires were completed was not recorded. Therefore, the exact time interval between the questionnaires and the interviews is unknown, but we estimate it to be on average between 2 and 5 days. Because 12 patients failed to complete the questionnaires, the sample for the present study comprised 295 patients. Prior to the start of the study, approval was obtained from the ethical committee of the Netherlands Institute of Mental Health and Addiction.
The interview comprised the mood and anxiety disorder sections of the Composite International Diagnostic Interview (CIDI), a standardized diagnostic interview, developed to be applied by trained lay interviewers, resulting in psychiatric diagnoses according to DSM-IV and ICD-10 criteria [13
]. The specific phobia section was omitted because these problems are (if not accompanied by other mental disorders) associated with relatively little disability and impairment [14
] and, therefore, isolated (non-comorbid) specific phobia appears to be of relatively little importance in patients with a 'nervous breakdown'. The obsessive-compulsive disorder section was omitted because of the low prevalence of this disorder [14
]. Accordingly, the present study used the following current DSM-IV [17
] diagnoses: major depressive disorder, dysthymia, bipolar disorder, generalized anxiety disorder, panic disorder with and without agoraphobia, agoraphobia without panic disorder, and social phobia. The CIDI was administered by five interviewers who received a training course at the Dutch WHO-CIDI Training and Reference Centre at Amsterdam, after which they were certified to deliver the fully structured CIDI interview. They used a computer-assisted version in which the questions were presented according to diagnostic algorithms and responses were entered directly into the computer. None of the interviewers had any particular expertise in psychology or psychiatry. The interviewers remained ignorant of the CIDI-diagnoses as such.
The questionnaires encompassed the Four-Dimensional Symptom Questionnaire (4DSQ) and the Hospital Anxiety and Depression Scale (HADS). The 4DSQ is a 50-item self-rating questionnaire measuring 'distress', 'depression', 'anxiety' and 'somatization' [18
]. The 4DSQ assesses psychological and psychosomatic symptoms experienced during the past seven days. The distress scale (16 items, score range 0–32) measures symptoms of general psychological distress, which is conceptualized as the most general, most basic expression of human psychological suffering [18
]. The depression scale (6 items, score range 0–12) measures severe anhedonia and depressive cognitions (including suicidal ideation), symptoms considered to be characteristic of depressive disorder [20
]. The anxiety scale (12 items, score range 0–24) measures irrational fears, panic and avoidance, characteristic features of most anxiety disorders [17
]. The somatization scale (16 items, score range 0–32) measures a range of 'psychosomatic' symptoms, characteristic of bodily distress and somatoform disorders [22
]. For all 4DSQ scales, higher scores represent higher symptom levels. In this study, we examined the 4DSQ distress scale for its ability to detect any depressive or anxiety disorder, and the 4DSQ depression and anxiety scales for their abilities to detect depressive and anxiety disorders. The 4DSQ depression and anxiety scales are supposed to detect depressive and anxiety disorders severe enough to consider specific treatment. For the 4DSQ scales two cut-off points are recommended, dividing the scales into low, moderate and high scores (Table ). High scores indicate a relatively high probability of caseness, prompting an immediate clinical diagnosis. On the other hand, low scores indicate the probable absence of a clinically relevant disorder. Moderate scores indicate a relatively low probability of caseness, warranting follow-up and reassessment after a few weeks [7
Recommended cut-off points for the 4DSQ and HADS depression and anxiety scales*
The HADS is a 14-item self-rating questionnaire measuring 'depression' and 'anxiety' [23
]. Like the 4DSQ, the HADS uses seven days as reference period. The depression scale (7 items, score range 0–21) measures mostly anhedonia, a phenomenon considered to be the central characteristic of major depressive disorder [21
]. The anxiety scale (7 items, score range 0–21) measures mostly symptoms of generalized anxiety disorder [27
]. For both HADS scales higher scores represent higher symptom levels. The depression and anxiety scales are intended to detect depressive and anxiety disorders in general medical settings. The depression scale is specifically intended to select those depressed patients which may be helped by the prescription of an antidepressant drug [21
]. Like the 4DSQ, the HADS scales employ two cut-off points, one for the detection of 'possible' and one for the detection of 'probable' depressive or anxiety disorder (Table ). Because the HADS total score is sometimes recommended as a measure of general psychological distress [28
], we examined the total score for its ability to detect any depressive or anxiety disorder.
There is a noteworthy difference in item content between the 4DSQ and the HADS. Unlike the 4DSQ, the HADS contains six positively worded items, of which five belong to the depression scale (e.g. 'I can laugh and see the funny side of things'; 'I feel cheerful'). These five depression items are assumed to measure anhedonia (i.e. loss of the ability to enjoy ordinary things in life) if the scores are reversed. In contrast, the 4DSQ depression scale heavily rests on depressive cognitions, including suicidal ideation (e.g. 'did you feel that everything is meaningless?'; 'did you ever think "If only I was dead"?'). At first glance, the 4DSQ depression scale seems to tap more severe depressive symptoms than the HADS depression scale. Regarding anxiety, unlike the HADS anxiety scale, the 4DSQ anxiety scale contains items on phobic fears and avoidance behaviour (e.g. 'were you afraid to travel on busses, trains or trams?'; 'were you afraid of becoming embarrassed when with other people?'; 'did you have to avoid certain places because they frightened you?'). In contrast, the HADS anxiety scale contains a few items of which the equivalents are included in the 4DSQ distress scale (e.g. 'feeling tense', 'worrying', 'feeling restless'). Again, at first glance, the 4DSQ anxiety scale seems to tap more severe symptoms than the HADS anxiety scale.
The interviewers collected the questionnaires after the CIDI-interview and checked the questionnaires for missing values. They did not have the knowledge, means or time to calculate scale scores. The questionnaire scores were later entered into a database by research assistants who were ignorant of the CIDI diagnoses.
First, we calculated Cronbach's α values as a measure of internal consistency and an estimate of reliability, and mean 4DSQ and HADS scores for the various diagnostic categories.
Second, we performed Receiver Operating Characteristic (ROC) analyses and estimated area under the curve (AUC) values as a measure of diagnostic accuracy. The ROC curve is a graphical representation of the sensitivity and 1-specificity values of all possible cut-off values of a continuous diagnostic variable [30
]. The AUC value represents the probability that a randomly chosen case has a higher score than a randomly chosen non-case [31
]. Differences between AUC values were tested using the method outlines by Hanley and McNeil [32
Third, we calculated sensitivity, specificity, positive predictive values (PPV) and negative predictive values (NPV) for a range of relevant cut-off points for each scale. All analyses were performed with SPSS 14.0.