This study represents the first attempt to synthesize the validity and added value of the CAM-ICU and ICDSC in ICU patients. Our results showed that the overall accuracy of the CAM-ICU is excellent, with pooled values for sensitivity and specificity of 80% and 95.9%, respectively. In addition, the pooled values for the sensitivity and specificity of the ICDSC were 74% and 81.9%, respectively. Thus, the currently available data support the use of the CAM-ICU or of the ICDSC as screening tools for delirium in critically ill patients. In addition, because of its high specificity, the CAM-ICU is an excellent diagnostic tool to delirium. This is relevant because a validated tool should be used routinely for monitoring critically ill patients and when delirium is present an algorithm to investigate its cause and a therapeutic strategy should be performed.
After the first validation study [19
], the CAM-ICU was translated into and validated in many languages [6
]. Although studies published in non-English languages have been excluded from this systematic review and meta-analysis, some have shown similar accuracy to the CAM-ICU. Tobar et al.
evaluated 29 ventilated patients in the ICU and showed a sensitivity and specificity of 80% and 96%, respectively [6
]. Additionally, Toro et al.
evaluated 129 patients and observed a sensitivity of 79.4% and a specificity of 97.9% for the CAM-ICU [8
]. These same authors performed a subgroup analysis with the ventilated patients (n
= 29), and the results suggested better sensitivity (92.9% versus 79.4%) and worse specificity (86.7% versus 97.9%) in this subgroup of patients. Both studies were published in the Spanish language. Chuang et al.
validated a Chinese version of the CAM-ICU and again reported high sensitivity (96%) when it was performed by a physician [12
The present meta-analysis has shown that the pooled sensitivity of the CAM-ICU was 80%, which demonstrates that this tool has good performance for screening patients with delirium in ICU. However, it is also evident that no other validation study has found as high a sensitivity as was observed in the initial studies by Ely et al.
]. In addition, there was an even lower sensitivity when the CAM-ICU was used in daily practice, that is, outside of a methodology for validation [14
Although there is no clear explanation for this loss of sensitivity in the most recent studies, it is possible that the evaluation in cohorts of patients that were less sedated, which is a current trend [23
], contributes to decreases in the accuracy of the CAM-ICU. In this systematic review, a higher sensitivity of the CAM-ICU was observed in two studies in subgroups of patients with RASS < 0 [4
]. Also, a higher sensitivity seems to be present in sedated patients and it is suggested by the differences in accuracy of the CAM-ICU between ventilated and non-ventilated patients. Although no studies compared the accuracy in these subgroups of patients, the study by Toro et al.
] (not included in this systematic review) is consistent with Ely's study [19
] and indicates excellent sensitivity in the subgroup of patients undergoing mechanical ventilation. Again, perhaps the sedation effects can contribute to these findings. It is reasonable to hypothesize that feature 2 (inattention) or feature 3 (disorganized thinking) of the CAM-ICU is less likely to be detected when patients are less sedated. Recently, Vasilevskis et al.
suggested a more intense approach to the detection of inattention when the CAM-ICU is used in daily practice [24
]. In addition, feature 1 (an acute onset of mental status changes) may be most frequently considered to be positive in patients with sedation and thus increases the sensitivity of the tool. Of course, more studies are necessary to explain and prove this hypothesis.
The four features of the CAM-ICU - 1) acute onset of mental status changes or fluctuating course; 2) inattention; 3) disorganized thinking; and 4) altered level of consciousness - have objective definitions. This characteristic likely justifies the high inter-rater reliability reported in several studies [4
Moreover, the specificity of the CAM-ICU is high. The pooled value for specificity was 96%, suggesting that when the CAM-ICU is positive, it is not necessary to confirm the diagnosis of delirium by the DSM-IV criteria, improving its feasibility in the ICU. In other words, the CAM-ICU is not only adequate for screening but also a good confirmatory diagnostic tool for delirium in critically ill patients.
Recently, Guenther et al.
published a study of the accuracy of the CAM-ICU Flowsheet, comparing it with the DSM-IV criteria [25
]. Interestingly, they found a sensitivity of 88% to 92% and an excellent specificity of 100%. Clearly, the CAM-ICU and the CAM-ICU Flowsheet are very similar tools. However, our previous study, despite an excellent correlation (kappa: 0.96) between these tools [9
], showed that they were not identical, so we decided not to add the Guenther's study in this meta-analysis.
A Canadian group developed and validated the ICDSC [20
] motivated by the same challenge: diagnosing delirium in critically ill and mechanically ventilated patients.
The ICDSC checklist is an eight-item screening tool (one point for each item) that is based on DSM criteria and applied to data that can be collected through medical records or to information obtained from the multidisciplinary team.
Bergeron et al.
developed and validated the ICDSC in a mixed ICU [20
]. All of the information used to complete the scale was collected from the patient, the primary nurses' evaluation and the chart in the previous 24 hours. With a cutoff score of four points, they showed a sensitivity of 99% and a specificity of 64%. Similar results were found by our group [9
]. However, we observed that the sensitivity of the ICDSC was not consistently high in all studies, and that the pooled value for sensitivity in this meta-analysis was 74%. These results suggest that this tool does not appear to be as accurate as the CAM-ICU for screening purposes. George et al.
, using a different threshold for positivity (3 rather than 4), showed a higher sensitivity (from 75% to 90%) and, consequently, improved screening characteristics of this tool [21
]. However, these changes in the cutoff decreased the specificity of the ICDSC, which was already lower than that observed for the CAM-ICU. The pooled value for specificity of the ICDSC in this meta-analysis was 82%.
Additionally, a recent study by Tomasi et al.
suggested that the CAM-ICU is a better predictor of outcomes than the ICDSC, which is likely related to the high rate of false positives with the ICDSC [26
]. At least two characteristics of the ICDSC might explain its lower sensitivity and specificity. First, the information is collected from the previous 24 hours. Delirium is characterized by its fluctuation, with the possibility of resolution over a long period of evaluation. Additionally, the evaluation of inattention ("easily distracted by external stimuli" [20
]), for example, may hinder an effective response by the evaluator.
Despite the limitations described above, the inter-rater reliability of the ICDSC appears to be good. George et al.
] reported an inter-rater agreement of 0.947 (95% confidence interval, 0.870 to 0.979), and in the study by Bergeron et al.
], the calculated alpha value was between 0.71 and 0.79.
Interestingly, both tools have worse sensitivity when patients with hypoactive delirium are tested. This issue is relevant because this subtype of delirium is the most prevalent [27
]. A lower prevalence of delusions and perceptual disturbances in hypoactive delirium does not appear to explain these findings [28
Despite the observation that no studies compared the accuracy of both tools in ventilated versus non-ventilated patients, most studies included these two types of patients.
Both tools are important in the care of the critically ill patients, each one with features that allow its use at different times or together. The CAM-ICU, to be quite specific, seems to be the ideal tool for the diagnosis of delirium in critically ill patients. In turn, the ICDSC, by its features not dichotomous, allows the diagnosis of subsyndromal delirium, which has potential prognostic implications [29
] and can identify patients with potential therapeutic benefit [30
Our findings should be understood in the context of some limitations. First, studies published in non-English languages were excluded. Unfortunately, a substantial part of the core information was not available from these studies precluding its use in the meta-analysis. However, as described above, the accuracy of the CAM-ICU appears to be consistent with the results of some of these studies. Second, this study cannot explain the findings with different accuracies of these tools in subgroups of patients (ventilated and nonventilated, RASS < 0, subtypes of delirium), but likely, this is a limitation of the tools. Additionally, the use of the CAM-ICU in patients with non-invasive ventilation has an excellent accuracy; however, its data are limited to a single study involving a small number of observations. This reflects the need for studies to evaluate specific groups of patients.