Reliability and validity of the Thai version of the Calgary Depression Scale for Schizophrenia

doi:10.2147/NDT.S40292

Neuropsychiatr Dis Treat. 2013; 9: 113–118.

Published online 2013 January 16. doi: 10.2147/NDT.S40292

PMCID: PMC3552548

Reliability and validity of the Thai version of the Calgary Depression Scale for Schizophrenia

Sirijit Suttajit,¹ Manit Srisurapanont,¹ Sutrak Pilakanta,¹ Chawanun Charnsil,¹ and Siritree Suttajit²

¹Department of Psychiatry, Faculty of Medicine

²Faculty of Pharmacy, Chiang Mai University, Chiang Mai, Thailand

Correspondence: Sirijit Suttajit Department of Psychiatry, Faculty of Medicine, Chiang Mai University, Chiang Mai 50200,Thailand Tel +66 53 945422 Fax +66 53 945426 Email sirijits/at/gmail.com

This is an Open Access article which permits unrestricted noncommercial use, provided the original work is properly cited.

Abstract

Aim

The purpose of this study was to assess the reliability and validity of the Thai version of the Calgary Depression Scale for Schizophrenia (CDSS) for the evaluation of depression in patients with schizophrenia.

Methods:

Sixty patients with schizophrenia according to Diagnostic and Statistical Manual of Mental Disorders: Fourth Edition; Text Revision (DSM-IV-TR) criteria were recruited to the study The Thai version of the CDSS, the Montgomery-Åsberg Depression Rating Scale (MADRS), the Hamilton Depression Rating Scale, 17-item version (HDRS-17), and the Positive and Negative Syndrome Scale (PANSS) were administered. A major depressive episode diagnosed by a psychiatrist according to the DSM-IV-TR was used as a gold standard.

Results:

The internal consistency of the Thai version of the CDSS was very good (Cronbach’s alpha = 0.869). The inter-rater reliability was found to be in substantial agreement with the intra-class correlation coefficient of 0.979. The test-retest reliability over a period of 3 days was high, with an intra-class correlation coefficient of 0.861. The Thai version of the CDSS showed significant correlations with the MADRS (r = 0.887), the HDRS-17 (r = 0.865), and the depression item of the Positive and Negative Syndrome Scale (PANSS-G6) (r = 0.833). The areas under the receiver operating characteristic curve of the CDSS, MADRS, HDRS-17, and PANSS-G6 against the DSM-IV-TR criteria for major depressive episode were 0.993, 0.954, 0.966, and 0.933, respectively. The optimal cut-off score to discriminate between depressed and non-depressed patients was 6/7, with a sensitivity of 92.31% and specificity of 97.87%.

Conclusion:

The Thai version of the CDSS is a reliable and valid measure for the evaluation of depression in Thai patients with schizophrenia.

Keywords: CDSS, Montgomery-Åsberg Depression Rating Scale, Positive and Negative Syndrome Scale, the Hamilton Depression Rating Scale, 17-item version

Introduction

Depression is common in the course of schizophrenia. The reported prevalence rate of depression in schizophrenia ranges widely from 10% to 83% depending on the clinical setting, assessment, and duration of study.1,2 Although it leads to impairment, decreased functioning, re-hospitalization, and suicide, depression is frequently under-recognized and untreated.3–5

Depression in schizophrenia can be categorized into three subtypes: (1) depressive symptoms secondary to organic factors (caused by medications, substance abuse, or underlying medical problems); (2) nonorganic depressive symptoms occurring with an acute psychotic episode; and (3) nonorganic depressive symptoms occurring without acute psychotic symptoms (such as prodromal, post-psychotic, and negative symptoms).6 There is difficulty in distinguishing between depression and negative symptoms or extrapyramidal side-effects. Therefore, assessment of depression in schizophrenia is a challenge.

The Calgary Depression Scale for Schizophrenia (CDSS) is a clinician-rated scale specifically developed for the assessment of depression in schizophrenia. It initially consisted of eleven items, but was reduced to nine items after validity studies.7,8 The nine assessment items are (1) depression, (2) hopelessness, (3) self-depreciation, (4) guilty ideas of reference, (5) pathological guilt, (6) morning depression, (7) early wakening, (8) suicide, and (9) observed depression. The CDSS is derived from the Hamilton Depression Rating Scale (HDRS) and the Present State Examination. A study of the English version of the CDSS in 150 patients with schizophrenia showed a high level of inter-rater reliability (intra-class correlation coefficient [ICC] = 0.89) and significant correlations with HDRS as well as the Brief Psychiatric Rating Scale. The internal reliability was good (Cronbach’s alpha = 0.79).10 Furthermore, the CDSS appeared to measure depression, rather than negative or extrapyramidal side-effects.10

Although there are a number of instruments that can be used for the assessment of depression in schizophrenia, including the Montgomery-Åsberg Depression Rating Scale (MADRS), the HDRS, and the depression item of the Positive and Negative Syndrome Scale (PANSS-G6), a systematic review found that the CDSS is superior than other depressive instruments in terms of reliability and validity in patients with schizophrenia.11 It was found to be the best depressive instrument to differentiate depressive symptoms from other symptoms of schizophrenia (divergent validity) and was the least likely to misdiagnose or overlook cases of depression (predictive validity).11

The CDSS is currently available in 35 languages. It has been used in research and in clinical settings with patients with schizophrenia. The purpose of this study was to assess the reliability and validity of the Thai version of the CDSS for the evaluation of depression in Thai patients with schizophrenia.

Methods

The translation of the Thai version of the CDSS

The Thai version of the CDSS was translated from the original CDSS by a bilingual psychiatrist. Each of the nine items were carefully checked and adjusted to retain the purpose of the original version and ensure they communicated effectively and were suitable for the Thai culture. The final version was then translated back into English by another experienced bilingual psychiatrist. The re-translated English version was then examined by the author of the original CDSS, Professor Donald Addington, to determine whether the meaning was congruent with the original version. Consensus was achieved by three experienced Thai psychiatrists and a psychiatric nurse. Pilot testing was conducted in 15 patients with schizophrenia.

Subjects and procedures

This study was conducted with 60 participants at an out-and in-patient clinic of a university hospital in Chiang Mai, Thailand. Inclusion criteria were: Thai-speaking, aged 18 years or over, and diagnosis of schizophrenia according to the Diagnostic and Statistical Manual of Mental Disorders: Fourth Edition; Text Revision (DSM-IV-TR).12 Those with neurological disorders and mental retardation were excluded from the study. The study was approved by the Ethics Committee of the Faculty of Medicine, Chiang Mai University, Thailand. After the details of the study were fully explained, written informed consent was obtained from all patients before the study.

A major depressive episode diagnosed by a psychiatrist according to DSM-IV-TR criteria was used as a gold standard. Each patient was assessed with the following scales: the Thai version of the CDSS, the MADRS,13 the Hamilton Depression Rating Scale, 17-item version (HDRS-17),14 and the Positive and Negative Syndrome Scale (PANSS).15

The joint inter-rater reliability test was performed in ten patients by two raters in a room, with each rater taking turns either interviewing or observing. The test-retest validity of the Thai version of the CDSS was conducted by one rater making two ratings at intervals of 3 days or more.

Statistical analysis

SPSS software (v 16.0; IBM, Armonk, NY, USA) was used for analysis. Internal consistency was tested with Cronbach’s alpha coefficient. Inter-rater and test-retest reliability were analyzed by determining the ICC. Construct validity was investigated using the Pearson product-moment correlation coefficient between the Thai version of the CDSS scores and the scores obtained with the MADRS, the HDRS-17, the PANSS-G6, the positive and negative subscales of the PANSS, the general psychopathology subscale of the PANSS, and the total PANSS score. To examine the diagnostic validity of the four depressive scales (the Thai version of the CDSS, the MADRS, the HDRS-17, and the PANSS-G6), the areas under the receiver operating characteristic (ROC) curve were obtained.

Results

Sample characteristics

Of the 60 participants, 33 (55%) were female, ten (16.7%) were married, and 25 (41.6%) had completed at least secondary education. The mean age of participants was 40.4 ±13.6 years old. The prevalence of a major depressive episode according to DSM-IV-TR criteria was 21.7%. Five patients (8.3%) were taking antidepressants. The mean scores ± standard deviation were: 3.88 + 4.05 (minimum = 0, maximum = 22) for the Thai version of the CDSS, 6.87 ± 7.73 for the MADRS, and 7.07 ± 5.93 for the HDRS-17.

For the PANS S scores, the mean scores were 45.12 ± 12.74 for the PANSS total, 10.92 ± 4.06 for the PANSS positive subscale, 11.32 ± 4.06 for the PANSS negative subscale, the mean score for PANSS-G6 was 2.07±1.27 and 22.88 ± 6.05 for the PANSS general psychopathology subscale.

Reliability

Internal consistency

The internal consistency of the Thai version of the CDSS was very good (Cronbach’s alpha = 0.869). All of the items appeared to correlate well with the scale overall, although the item-total correlation was low for early wakening (C7). Moreover, all items except early wakening (C7) appeared to be essential components of the scale (Table 1).

Table 1

The internal reliability analysis and the inter-rater reliability of the Thai version of the Calgary Depression Scale for Schizophrenia

Inter-rater reliability

Inter-rater reliability was tested on a sample of ten patients with schizophrenia. The overall inter-rater reliability was found to be in substantial agreement with the ICC of 0.979 (P < 0.001). The ICC for inter-rater reliability for each item was also high, ranging from 0.800 to 1.000 (Table 1). The test-retest reliability showed almost perfect agreement, with an ICC of 0.861 (P < 0.001) for the total score of the Thai version of the CDSS.

Validity

The Thai version of the CDSS showed significant correlations with the MADRS (r = 0.887, P < 0.01), the HDRS (r = 0.865, P < 0.01), and the PANSS-G6 (r = 0.833, P < 0.01). There were also significant correlations between the Thai version of the CDSS and the PANSS scores (positive, negative, general psychopathology, and total score) (Table 2). The areas under the ROC curve of the CDSS, MADRS, HDRS-17, and PANSS-G6 against the DSM-IV-TR criteria for major depressive episode were 0.993, 0.954, 0.966, and 0.933, respectively (Figure 1). The optimal cut-off scores to discriminate between depressed and non-depressed patients was 6/7, with a sensitivity of 92.31% and specificity of 97.87% (Table 3).

Table 2

Correlations between depression rating scales and the Positive and Negative Syndrome Scale (PANSS) (Pearson product-moment correlation)

Figure 1

The receiver operating characteristic (ROC) curve of the depression rating scales and the Diagnostic and Statistical Manual of Mental Disorders: Fourth Edition; Text Revision12 (DSM-IV-TR) for major depressive episode diagnosis.

Table 3

Performance of the Thai version of the Calgary Depression Scale for Schizophrenia in screening for major depressive episodes in patients with schizophrenia

Discussion

This study demonstrates that the Thai version of the CDSS is a reliable and valid measure for the evaluation of depression in Thai patients with schizophrenia. The internal consistency of the Thai version of the CDSS, which indicates the agreement among the individual components of the measure, was very good (Cronbach’s alpha = 0.869). This result supports the high internal consistency of the CDSS, which was also found in other versions, including the Japanese version (Cronbach’s alpha = 0.82), the Greek version (Cronbach’s alpha = 0.87), and the French version (Cronbach’s alpha = 0.82).16–18 All the items appeared to correlate well with the scale overall, although the item-total correlation was low in early wakening (C7). This finding was similar to that reported in the French version.16 The small item-total correlation in C7 might be due to use of benzodiazepines, which were highly prescribed in this population, as well as the low severity of depressive symptoms and psychotic symptoms found in the study participants.

The overall inter-rater reliability of the Thai version of the CDSS was found to be in substantial agreement (ICC = 0.979) and the ICC for each item was also high. These findings are similar to those for other versions of the CDSS.16,18,19 The test-retest was done at intervals of 3 days or more. Although depression in patients with schizophrenia might change over time, the test-retest reliability for the total score of the Thai version of the CDSS in this study was high, with an ICC of 0.861.

The mean total score of Thai version of the CDSS in this study significantly correlates with those of the MADRS, the HDRS-17, and the PANSS-G6. These results are similar to those of Addington et al, who reported significant relationships between the CDSS, HDRS, and Beck Depression Inventory7 Our study also found significant correlations between the Thai version of the CDSS and the PANSS score (positive, negative, general psychopathology, and total score). The relationship between the CDSS and PANSS scores identified in previous studies has been controversial. Lançon et al found significant correlations between the CDSS, the PANSS positive subscale, the PANSS general psychopathology subscale, and PANSS total score,20 while Addington et al reported a significant correlation between the CDSS and the PANSS negative subscale.7 Additionally, some items of the CDSS, including guilty ideas of reference (C4) and early wakening (C7) were found to be significantly correlated with the PANSS positive subscale when the patients were in remission.21 These correlations might represent the contamination of the CDSS by the psychotic symptoms in patients with schizophrenia. Nevertheless, there was no significant correlation between the CDSS and the PANSS positive and negative subscale, and the Scale for the Assessment of Negative Symptoms (SANS) score on admission.21

The DSM-IV-TR criteria for major depressive episode were utilized as a gold standard following the original study of Addington et al, which used DSM-III-R criteria.10 It was found that the area under the ROC curve of the CDSS against the DSM-IV-TR criteria for major depressive episode was higher than that of the other depressive measures. This indicates that the Thai version of the CDSS is more effective than the MADRS, HDRS-17, and PANSS-G6 in evaluating depressive symptoms in patients with schizophrenia. The cut-off point of 6/7 for detecting major depressive episodes provided a high sensitivity (92.31%) and specificity (97.87%) as well as a high positive predictive value (92.31%) and negative predictive value (97.87%).

Some limitations should be taken into account when interpreting the findings of this study. First, the depression and psychotic symptoms of the patients with schizophrenia in this study were moderately low. This problem has been reported in studies looking at other versions of the CDSS and may limit the use of the scale in patients with severe depressive and psychotic symptoms.16 However, it may have an advantage in detecting borderline cases of depression. Second, the particular sample characteristics from a university hospital in Thailand might reduce generalization to other patients with schizophrenia in different settings. Third, some of the patients were being treated with antidepressants and benzodiazepines during the evaluation, which may have interfered with some items in the CDSS. As such, further study of the reliability and validity with a wide range of depressive and psychotic symptoms in different settings, such as in a psychiatric hospital, should be conducted to confirm the results of this study.

Conclusion

The Thai version of the CDSS is a reliable and valid measure for the evaluation of depression in Thai patients with schizophrenia with the optimal cut-off score of seven or greater. The scale is recommended for the assessment of depression in patients with schizophrenia in research and clinical practice. Further study on the reliability and validity of the version with a wide range of depressive and psychotic symptoms in different settings should be conducted.

Acknowledgment

This study was supported by a grant from the Faculty of Medicine, Chiang Mai University.

Footnotes

Disclosure

The authors declare no conflicts of interest in this work.

References

1. Buckley PF, Miller BJ, Lehrer DS, Castle DJ. Psychiatric comorbidities and schizophrenia. Schizophr Bull. 2009;35(2):383–402. [PMC free article] [PubMed]

2. Martin RL, Cloninger CR, Guze SB, Clayton PJ. Frequency and differential diagnosis of depressive syndromes in schizophrenia. J Clin Psychiatry. 1985;46((11 Pt 2)):9–13. [PubMed]

3. Addington DD, Azorin JM, Falloon IR, Gerlach J, Hirsch SR, Siris SG. Clinical issues related to depression in schizophrenia: an international survey of psychiatrists. Acta Psychiatr Scand. 2002;105(3):189–195. [PubMed]

4. Johnson DA. Studies of depressive symptoms in schizophrenia. Br J Psychiatry. 1981;139:89–101. [PubMed]

5. Kasckow J, Montross L, Golshan S, et al. Suicidality in middle aged and older patients with schizophrenia and depressive symptoms: relationship to functioning and Quality of Life. Int J Geriatr Psychiatry. 2007;22(12):1223–1228. [PMC free article] [PubMed]

6. Bartels SJ, Drake RE. Depressive symptoms in schizophrenia: comprehensive differential diagnosis. Compr Psychiatry. 1988;29(5):467–483. [PubMed]

7. Addington D, Addington J, Maticka-Tyndale E, Joyce J. Reliability and validity of a depression rating scale for schizophrenics. Schizophr Res. 1992;6(3):201–208. [PubMed]

8. Addington D, Addington J, Schissel B. A depression rating scale for schizophrenics. Schizophr Res. 1990;3(4):247–251. [PubMed]

9. Addington D, Addington J, Maticka-Tyndale E. Assessing depression in schizophrenia: the Calgary Depression Scale. Br J Psychiatry Suppl. 1993. pp. 39–44. [PubMed]

10. Addington D, Addington J, Maticka-Tyndale E. Specificity of the Calgary Depression Scale for schizophrenics. Schizophr Res. 1994;11(3):239–244. [PubMed]

11. Lako IM, Bruggeman R, Knegtering H, et al. A systematic review of instruments to measure depressive symptoms in patients with schizophrenia. J Affect Disord. 2012;140(1):38–47. [PubMed]

12. American Psychiatric Association (APA) Diagnostic and Statistical Manual of Mental Disorders: Fourth Edition; Text Revision. Washington DC: APA; 2000.

13. Satthapisit S, Posayaanuwat N, Sasaluksananont C, Kaewpornsawan T, Singhakun S. The comparison of Montgomery and Asberg Depression Rating Scale (MADRS Thai) to diagnostic and statistical manual of mental disorders (DSM) and to Hamilton Rating Scale for Depression (HRSD): validity and reliability. J Med Assoc Thai. 2007;90(3):524–531. [PubMed]

14. Lotarkul M, Sookanich P, Sukying C. Thai Hamilton Rating Scale for Depression development. J Psychiatr Assoc Thai. 41((4)):235–246. (Thai).

15. Kay SR, Fiszbein A, Opler LA. The positive and negative syndrome scale (PANSS) for schizophrenia. Schizophr Bull. 1987;13(2):261–276. [PubMed]

16. Bernard D, Lançon C, Auquier P, Reine G, Addington D. Calgary Depression Scale for Schizophrenia: a study of the validity of a French-language version in a population of schizophrenic patients. Acta Psychiatr Scand. 1998;97(1):36–41. [PubMed]

17. Kaneda Y, Fujii A, Ohmori T. Psychometric properties of the Japanese version of the Calgary Depression Scale for Schizophrenics. J Nerv Ment Dis. Apr 2000;188((4)):237–239. [PubMed]

18. Kontaxakis VP, Havaki-Kontaxaki BJ, Margariti MM, et al. The Greek version of the Calgary depression scale for schizophrenia. Psychiatry Res. 2000;94(2):163–171. [PubMed]

19. Xiao W, Liu H, Zhang H, et al. Reliability and validity of the Chinese version of the Calgary Depression Scale for Schizophrenia. Aust NZJ Psychiatry. 2009;43(6):548–553. [PubMed]

20. Lançon C, Auquier P, Reine G, Bernard D, Toumi M. Study of the concurrent validity of the Calgary Depression Scale for Schizophrenics (CDSS) J Affect Disord. 2000;58(2):107–115. [PubMed]

21. Addington D, Addington J, Atkinson M. A psychometric comparison of the Calgary Depression Scale for Schizophrenia and the Hamilton Depression Rating Scale. Schizophr Res. 1996;19:2–3. 205–212. [PubMed]

Articles from Neuropsychiatric Disease and Treatment are provided here courtesy of
Dove Press