|Home | About | Journals | Submit | Contact Us | Français|
To validate a diagnostic instrument for pediatric delirium in critically ill children, both ventilated and nonventilated, that uses standardized, developmentally appropriate measurements.
A prospective observational cohort study investigating the Pediatric Confusion Assessment Method for Intensive Care Unit (pCAM-ICU) patients in the pediatric medical, surgical, and cardiac intensive care unit of a university-based medical center.
A total of 68 pediatric critically ill patients, at least 5 years of age, were enrolled from July 1, 2008, to March 30, 2009.
Criterion validity including sensitivity and specificity and interrater reliability were determined using daily delirium assessments with the pCAM-ICU by two critical care clinicians compared with delirium diagnosis by pediatric psychiatrists using Diagnostic and Statistical Manual, 4th Edition, Text Revision criteria.
A total of 146 paired assessments were completed among 68 enrolled patients with a mean age of 12.2 yrs. Compared with the reference standard for diagnosing delirium, the pCAM-ICU demonstrated a sensitivity of 83% (95% confidence interval, 66–93%), a specificity of 99% (95% confidence interval, 95–100%), and a high interrater reliability (κ = 0.96; 95% confidence interval, 0.74–1.0).
The pCAM-ICU is a highly valid reliable instrument for the diagnosis of pediatric delirium in critically ill children chronologically and developmentally at least 5 yrs of age. Use of the pCAM-ICU may expedite diagnosis and consultation with neuropsychiatry specialists for treatment of pediatric delirium. In addition, the pCAM-ICU may provide a means for delirium monitoring in future epidemiologic and interventional studies in critically ill children. (Crit Care Med 2011; 39:150–157)
Delirium, or acute brain dysfunction, is a disturbance of consciousness characterized by an acute change or fluctuating course of mental status, inattention, and the inability to receive, process, store, or recall information (1). Delirium is an independent predictor of prolonged mechanical ventilation, increased length of hospitalization, higher 6-month mortality (2–9), and long-term cognitive impairment in adults (5, 10). The reported prevalence rates of delirium in ventilated adults are as high as 60–80% and in the range of 20–60% in less critically ill or nonventilated adults (2, 3, 7, 11–13). Despite the current understanding of delirium in adult populations, the data regarding the epidemiology, morbidity, and mortality associated with delirium in critically ill children are sparse (14–17).
The true prevalence of delirium among critically ill children remains unknown. Emerging literature from psychiatric specialists reports the prevalence of delirium to be approximately 10% among select pediatric cohorts for which psychiatric consultation was sought by the medical or pediatric intensive care unit (PICU) team (14, 17). This is likely to be an underestimation of the true prevalence, as demonstrated in early adult delirium literature (18–23), given the absence of validated bedside tools to diagnose pediatric delirium and the limited awareness of its presentation among healthcare providers. Furthermore, a significant number of critically ill patients with delirium present with the hypoactive subtype (24–26) characterized primarily by inattention and negative symptoms such as “lethargy,” which often goes unnoticed; this subtype may actually be associated with worse outcomes (26–28). The hyperactive delirium subtype, commonly called by the misnomers of intensive care unit psychosis or intensive care unit syndrome, is readily identified because of its characteristic heightened psychomotor activity (agitation, pulling out catheters) and hallucinations (ie, “positive” symptoms) but occurs less frequently (21, 22, 29). To characterize the epidemiology of this organ dysfunction on a large scale prospectively, there is an unmet need for the development of a reliable and valid tool for monitoring delirium in critically ill children that can be performed routinely and quickly in concert with consultation by subspecialty-trained psychiatrists.
In 1990, Inouye and colleagues (30) met the same need in the adult population when they published a landmark validation study of the Confusion Assessment Method, which was followed in 2001 by collaboration with Ely (31, 32) to adapt and validate an intensive care unit version of this tool (Confusion Assessment Method for Intensive Care Unit [CAM-ICU]) for use in critically ill adults on and off mechanical ventilation. To date, both tools are the most widely used methods of delirium monitoring in adults around the world (33), each translated into approximately 20 different languages (www.icudelirium.org). Until such a tool is available for the pediatric specialty, the field of pediatric delirium will remain limited by the absence of large-scale routine delirium monitoring, a marginal understanding of the epidemiology and risk factors for both pediatric delirium and the potential of related long-term cognitive complications, and creation of successful interventions for critically ill children with acute brain dysfunction.
In this report, we describe the interdisciplinary development of a pediatric delirium monitoring instrument called the pediatric CAM-ICU (pCAM-ICU) as well as the prospective testing of its validity (sensitivity and specificity) and interrater reliability.
The study population consisted of pediatric patients aged ≥5 yrs, ventilated or nonventilated, regardless of admitting diagnosis, hospitalized at the Monroe Carell Jr Children's Hospital at Vanderbilt PICU. The Monroe Carell Jr Children's Hospital at Vanderbilt is a pediatric academic medical center with 36 PICU beds, which includes medical, surgical, and cardiac critical care. The Institutional Review Board approved this study, and informed consent was obtained from the parent or guardian in addition to patient assent when age-appropriate. The study interval was from July 1, 2008, to March 30, 2009.
The currently accepted reference standard for the diagnosis of pediatric delirium requires patient evaluation by a pediatric psychiatrist using Diagnostic Statistical Manual (DSM) criteria (1). This diagnostic requirement can be both time-consuming and constrained by the availability of specialty-trained personnel. Therefore, without a bedside pediatric delirium tool, consistent monitoring of delirium in the PICU setting remains unrealistic. This obstacle has been overcome in adult critically ill patients with instruments such as the CAM-ICU (31, 32) (Fig. 1), which uses the four cardinal clinical features of delirium for diagnosis: (Feature 1) fluctuation or an acute change in mental status, (Feature 2) inattention, (Feature 3) an altered level of consciousness, and (Feature 4) disorganized thinking. Delirium diagnosis (ie, CAM-ICU-positive) is present when a patient demonstrates Features 1 and 2 plus either Feature 3 or 4 (Fig. 1). Feature 2 (inattention) is the pivotal feature for diagnosing delirium (34).
Pediatric patients cannot be assessed using the adult CAM-ICU in its current form because it assumes a mature level of cognition and therefore cannot differentiate between responses of normal developmental differences among children and deviations of cognition. The pCAM-ICU therefore used the general structure of the previously validated Confusion Assessment Method (19) and CAM-ICU (31, 32) in addition to required adaptations for age-appropriate cognitive assessment. These adaptations were generated using a comprehensive literature review and consultation with numerous child development and delirium experts, including pediatricians, pediatric psychiatrists, psychologists, and neurologists. This resulted in a refined tool that could be tested in both verbal and nonverbal children with the cognition expected of a developmentally appropriate 5-yr-old child (Fig. 2). Feature 1 (acute change or fluctuation in mental status) was not altered from the original CAM-ICU. Particular consideration was placed on the attention screening examination (ASE), which is designed to assess for inattention (feature 2) among verbal and nonverbal intensive care unit patients (35) and used as a central feature of the CAM-ICU (31, 32). Inattention can be assessed using one of two validated attention screening examinations, the ASE auditory (vigilance A letter test) or the ASE visual. The ASE auditory test (35, 36) consists of asking patients to squeeze the assessors hands when they hear the letter “A” while not squeezing if another letter is called out. For the pCAM-ICU, the ASE letter sequence was modified based on our test population outcomes retaining the five As and five non-As previously validated. The ASE visual (35) requires picture recall and was therefore modified for pediatric use from the adult version using elementary, bold-colored pictures, which were easily identified by children. Feature 3 (altered level of consciousness) was not modified and was assessed using the Richmond Agitation Sedation Scale (RASS) (37, 38). Feature 4 (disorganized thinking) underwent significant modification to include questions that were developmentally appropriate for a child as young as 5 yrs. The commands (ie, “hold up this many fingers”) for feature 4 were retained as in the original CAM-ICU.
Daily screening of the computerized census for the PICU was conducted Monday through Friday by the research team. Enrolled patients underwent paired, blinded evaluations <3 hrs apart, once daily Monday through Saturday, from PICU admission to PICU discharge. The blinded paired assessments included patients' evaluation with the pCAM-ICU either before or after full diagnostic psychiatric evaluation for delirium. This remained an observational cohort study to validate a pediatric delirium diagnostic tool. The research team provided a review of delirium to the PICU clinicians and nursing staff. Patient management was the responsibility of the medical team.
The pCAM-ICU team consisted of two pediatric intensivists (H.A.B.S., J.B.), one pediatric critical care nurse practitioner (M.K.T.), and three pediatric registered nurses (K.W., T.B., P.B.), who used the pCAM-ICU for diagnosis of delirium. Patient assessments using the pCAM-ICU followed a two-step approach (Fig. 3) based on Plum and Posner's (39) definition of consciousness, which is comprised of two key components: arousal and content. Patient arousal was assessed first using the RASS (Fig. 4). The RASS (61), a valid and highly reliable sedation scale, describes three categories of an abnormal arousal state including: +1 through + 4 for agitation, − 1 through −3 for decreased arousal with retained responsiveness to verbal stimulation, and −4 or −5 for patients unable to respond to any verbal stimulation and thus qualified as comatose. Because response to verbal communication is required for the diagnosis of delirium, patients in a coma (unresponsive to verbal stimulation) were graded “unable to assess” and were not assigned a diagnosis of delirium present or absent. All patients with an arousal level of RASS ≥−3 were assessed for delirium using the pCAM-ICU.
Patient content of consciousness was then assessed using the pCAM-ICU and scored as described subsequently. Feature 1 (acute on-set or fluctuation of mental status changes) used baseline information obtained on enrollment as well as daily assessments using the Glasgow Coma Scale (40), sedation scales such as the RASS (37, 38), and/or perceptions of the bedside nurse or family members. Any current altered mental state or fluctuation during the prior 24-hr period in the patient's mental status indicated a positive feature 1. Feature 2 (inattention) was assessed using the ASE auditory and visual components. Neither test required verbal responses and thus was ideally suited for mechanically ventilated patients. There were two sets of pictures (sets A and B) that were alternated daily. Feature 2 does not require the performance of both the ASE auditory and visual in clinical practice. If the individual ASE score was <8, feature 2 was positive (ie, presence of inattention). Assessment of feature 3 (level of consciousness) was completed using clinical evaluation (eg, general assessment or RASS). Any level of consciousness other than “alert and calm” indicated a positive feature 3. Both the original Confusion Assessment Method (30) and the pCAM-ICU demonstrate intentional overlap between feature 1 and feature 3, because a patient can have fluctuation in mental status over time (feature 1-positive) yet be alert and calm during the evaluation (ie, feature 3-negative). Finally, assessment of feature 4 (disorganized thinking), a simple command plus four questions (alternate sets of questions for multiple days), was positive if a combined score between the questions and command was less than out of 5. The patient was diagnosed with delirium (ie, pCAM-ICU-positive) if he or she manifested both features 1 and 2 plus either feature 3 or 4 (Fig. 1).
The Psychiatry Reference Standard evaluators included two pediatric delirium experts (D.C.F. and K.M.) from the Department of Psychiatry. Delirium assessments using DSM criteria were standardized during a pretesting period (1). The Psychiatry Reference Standard assessments for delirium were performed using DSM criteria for delirium and any and all means of patient evaluation and testing as well as data-gathering from those who understood the patient's behavior and thinking the best that day (eg, the patient's nurses and surrogates), thus maximizing the psychiatrists' ability to arrive at the truth related to cognitive functioning. Psychiatry Reference Standard assessments were completed independent of the pCAM-ICU assessments. Patients' level of cognition was assessed by the delirium experts, and if cognitive development was deemed significantly less than expected for 5 yrs of age, he or she was withdrawn from the study. The psychiatry reference standard assessments for delirium were categorized as presence or absence of delirium. Additionally, when patients did not fulfill diagnostic criteria for delirium, the delirium experts noted whether subsyndromal delirium (SSD) was present using an a priori agreed-on definition. Subsyndromal delirium is a condition in which children possess more than one key DSM feature of delirium without demonstrating full delirium criteria (41, 42).
Criterion validity was determined by the ability of the pCAM-ICU to classify patients into normal and delirious categories compared with the reference rater's evaluation using DSM criteria. We assessed overall validity in addition to validity for subgroups based on age (5–12 yrs and >12 yrs) and those patients ventilated and nonventilated. The 2 × 2 tables were created using data from the comparison of reference standard ratings and pCAM-ICU ratings including the number of true-positives, false-positives (FP), true-negatives, and false-negatives (FN). Sensitivity = true-positive/(true-positive + FN); specificity = true-negative/(FP + true-negative); accuracy = (true-positive + true-negative)/(true-positive + FP + FN + true-negative), and likelihood ratio = sensitivity/(1 − specificity). Statistical significance was defined as p ≤ .05. Statistical analyses including 95% confidence intervals (CIs) for the previous test characteristics were performed using SAS software program version 8 (SAS Institute, Cary, NC).
Two members of the medical team were present for each patient assessment with the pCAM-ICU. One member of the medical team used the pCAM-ICU for patient assessment and rated patient responses, whereas the second member observed and independently rated patient responses. Both the rater and observer scored each feature of the pCAM-ICU independently as positive or negative on separate evaluation sheets. We chose this method of interrater reliability rather than test–retest to minimize stress in children and families dealing with critical illness and to avoid learning on the part of the patient. The κ coefficient was used to assess interrater reliability between the rater and observer regardless of clinical role. During pilot testing, the pCAM-ICU took on average <2 mins to complete and at most 4 mins.
Sample size for this study was computed to provide a sufficient number of patients to provide the lower CI of 70%. Assuming a prevalence of 15% for pediatric delirium, and point estimate for sensitivity of 90%, the sample size of 60 patients was required for the analysis. We expected 20% of patients to be excluded as a result of withdrawal or incomplete data. With 60 patients, the lower CI for specificity was estimated to be 82%.
Patient characteristics, including gender, ethnicity, race, admission diagnosis, and age, were summarized using proportions for categorical variables with mean and standard deviations for continuous variables. The criterion validity of the pCAM-ICU was measured through examining sensitivity and specificity, positive predictive value, and negative predictive value as described previously in validity testing. Because delirium was assessed repeatedly over the PICU stay, to estimate se of proportions of events in repeated measures, we used generalized estimating equations with logit-link function and Huber-White Sandwich estimators (43). The reliability of the pCAM-ICU was assessed through evaluating κ coefficients. For κ, CIs were constructed using nonparametric bootstrap with 1000 bootstrap samples to account for correlation among repeated measures (44). A similar approach through bootstrap methods was used to compute validity measures in subgroup specific analysis because generalized estimating equations could not be applied as a result of the small sample size.
There were a total of 437 patients admitted to the PICU during the study period. We screened a total of 305 patients who were at least 5 yrs of age and subsequently consented 93 patients. Figure 5 illustrates participant flow. Of the 93 consented patients, 22 patients did not complete the study protocol as a result of the following: transfer in <24 hrs of PICU admission and before paired assessment completion (13), remained comatose (7), withdrawn by family (1), or had developmental delay (1). Therefore, 71 patients underwent daily paired delirium assessments. At the time of analysis, three patients had a single paired assessment, which occurred >3 hrs apart; based on our protocol, these were subsequently excluded. Final statistical analysis was conducted on 68 children who were evaluated for delirium by the pCAM-ICU and the reference raters using DSM criteria. Patient characteristics for the 68 patients with paired assessments are presented in Table 1. The mean (SD) age of the study population was 12.2 (3.4) years. The admission diagnoses for enrolled patients demonstrate a broad range of medical, cardiac, and surgical intensive care unit illnesses. The most common admission diagnoses were surgical intervention for congenital heart disease (18%) and respiratory insufficiency in the setting of status asthmaticus (12%). The mean (SD) Pediatric Risk of Mortality (PRISM) score was 8.6 (7.0). Six percent of the study assessments were conducted while patients were on mechanical ventilation, indicating a moderate overall level of severity of illness.
A total of 146 blinded, paired assessments between reference standard delirium experts and the pCAM-ICU raters were completed. The reference standard DSM ratings diagnosed delirium in 18 of the 146 (12.3%) patient assessments, or nine of 68 (13.2%) patients, whereas the pCAM-ICU diagnosed delirium in 16 of 146 (11%) patient assessments, or eight of 68 (11.8%) patients. Of the nine patients with delirium during the study period, the mean age was 13.8 (1.9) yrs and 77.8% were male.
The sensitivity of the pCAM-ICU was 83% (95% CI, 66–93%) and the specificity was 99% (95% CI, 95–100%) (Table 2). This resulted in a positive predictive value of 93% (95% CI, 63–99%) and a negative predictive value of 98% (95% CI, 93–99%) for the pCAM-ICU. The likelihood ratio for the pCAM-ICU in the diagnosis of delirium was 105 (95% CI, 14– 748). The pCAM-ICU was completed with very high interrater reliability of κ = 0.96 (95% CI, 0.74–1.0). We specifically assessed the validity of feature 2 given that we had modified the original ASE Auditory test for the pCAM-ICU. Of the 146 assessments, inattention was diagnosed by the pCAM-ICU during 16 vs. 18 assessments positive for inattention by reference raters. The sensitivity of the pCAM-ICU feature 2 for inattention was 83% (95% CI, 44–100%) and the specificity was 99% (95% CI, 98–100%).
To evaluate the performance of the pCAM-ICU in patient groups that may pose particular challenges in delirium assessment, subgroup analyses were completed based on age and requirement of mechanical ventilation (Table 2). In patients ≤12 yrs of age, the sensitivity and specificity of the pCAM-ICU were both 100% using 55 paired assessments among 34 patients. In patients > 12 yrs of age, the sensitivity and specificity were 80% (95% CI, 59–91%) and 99% (95% CI, 91–100%). For patients' assessments of delirium while on mechanical ventilation, the pCAM-ICU had a sensitivity and specificity of 75% (95% CI, 66–100%) and 92% (95% CI, 67–100%) using 17 paired assessments.
There were four discordant pCAM-ICU ratings, of which three were FN and one was FP. Each FN case involved paired assessments that were >2 hrs apart with intervening clinical events that overtly altered the patients' status. Acute hypoxia and hypotension were among the clinical events occurring between these paired assessments. The FP case involved a patient with SSD diagnosed by DSM rating although met full criteria for delirium diagnosis by the pCAM-ICU. The reference raters determined that eight patient assessments (5.5%) met criteria for SSD. Of the eight subsyndromal DSM assessments, seven (87.5%) had at least two positive pCAM-ICU features. Figure 6 demonstrates the median number of pCAM-ICU features positive, as measured by the medical team, in patients rated by the DSM raters as no delirium, SSD, and full delirium. Patients with no delirium by the reference rater had a median of zero pCAM-ICU features positive; this increased to two in those with SSD and three in patients with full diagnostic criteria of delirium.
The major advance of this investigation is providing the pediatric medical community with a valid and reliable tool for delirium diagnosis on which to build a future of quality improvement projects, cohort studies, and interventional trials. Using the pCAM-ICU, nonpsychiatrictrained nurses and physicians demonstrated a high sensitivity of 83%, specificity of 99%, and an excellent interrater reliability (κ = 0.96) in the diagnosis of delirium among critically ill children. The Psychiatry Reference Standard raters determined that pediatric delirium occurred in 13.2% of patients assessed, similar to previously published retrospective “consultant-based” reports (14, 17, 45–47). Prompt pediatric delirium diagnosis and routine monitoring can be provided by the interdisciplinary PICU team using the pCAM-ICU in conjunction with the important role of subspecialty psychiatry consultation in the care of critically ill children with delirium. This collaboration is invaluable for definitive diagnostic and treatment considerations. Although neither this report nor those previously published studies suffice as generalized cohort studies, the pCAM-ICU (or subsequently published tools) will allow the medical profession to define the epidemiology of acute brain dysfunction and its sequelae in pediatric critical care by incorporating routine, prospective monitoring for delirium by PICU personnel.
A major risk factor for the development of delirium in patients on mechanical ventilation is the administration of psychoactive agents such as benzodiazepines (13, 48, 49). A recent prospective cohort study by Khemani and colleagues (50) revealed that one of three children with critical illness require mechanical ventilation during their PICU course). Greater than 90% of infants and children supported on mechanical ventilation receive psychoactive medications (51, 52). In the effort to optimize patient comfort while on mechanical ventilation, physicians and nurses may deliver sedatives in quantities disproportionate to patient needs (53–56). These medications are the most significant and modifiable iatrogenic risk factor for delirium in adults (13, 48, 49). Evidence is mounting that the selection of specific pharmacologic agents for pain and sedation management during critical illness may contribute to the development of acute brain dysfunction in addition to long-term comorbidities in children. Indeed, Colville and colleagues describe that one of three children admitted to the PICU reported delusional memories of their experience during critical illness, which were significantly associated with longer duration of benzodiazepine administration during hospitalization and higher posttraumatic stress scores after discharge (57). Conversely, adequate management of pain and anxiety can reduce the incidence of posttraumatic stress disorder (58). Thus, the prevention of pediatric delirium includes not only the appropriate treatment of pain and suffering of critically ill patients, but also the responsibility to monitor ongoing pharmacologic needs and consider nontraditional drug classes. Clearly, the use of a pediatric delirium diagnostic tool such as the pCAM-ICU in concert with current monitoring guidelines for pain and sedation will add to our ability to confirm the presence of delirium and study its risk factors and relationship to the common use of sedation and analgesia in the PICU.
Several limitations of this investigation warrant discussion. Our pediatric delirium prevalence of 13.2% and that of previous reports is an underestimation of the true incidence of pediatric delirium as a result of several factors. Despite our diligent attempts, consent was not obtained on several study participants until the second or third day of PICU admission largely as a result of parental anxiety and needed time for evaluation of the consent form. In addition, the consent process and therefore the potential for missing cases of pediatric delirium were affected by the constraints on research personnel availability on weekends and holidays. This disturbance in consecutive and timely enrollment almost certainly led to missed episodes of delirium among enrolled patients and those ultimately not consented. In addition, patients were assessed once daily for delirium. We know from our adult cohorts that adding an additional delirium assessment generally increases the relative delirium rates by approximately 15% (E.W.E., unpublished observations). Our study design, which allowed for up to 3 hrs between reference rater and pCAM-ICU assessments as a result of research personnel availability, also contributed to the discrepancies between pCAM-ICU and DSM ratings (FN). Clearly changes in the patients' clinical condition, administration of drugs, and institution of other therapies (oxygen or vasopressor administration) between assessments hastened a patient's transition into or out of a delirious state. The other discrepancy (FP) occurred in a patient diagnosed with SSD by the reference rater. The diagnosis of SSD does not accurately describe the continuum of the delirium disease process, because patients may be transitioning to full-blown delirium or beginning resolution. The pCAM-ICU assesses a patient's current clinical state regardless of where they are on the continuum of having delirium vs. the ability of psychiatric evaluation to observe finer signs or symptoms, which together may be enough for the diagnosis of SSD. Despite these shortcomings in study design, the pCAM-ICU still demonstrated high sensitivity and specificity. A limitation of our study population included having a smaller percentage of ventilated children with the highest percentage of ventilation occurring in children <5 yrs of age usually presenting for congenital heart surgery. Finally, our results are limited by the lower limit of the CI on sensitivity being 66%, slightly lower than our goal of 70%. The point estimates for sensitivity and specificity, however, were very strong at 83% and 99% with logical explanations for discordant ratings. Like all clinical tools, the pCAM-ICU may benefit from future testing that would strengthen these already highly valid results as well as address the remaining need for an instrument to use in children <5 yrs old.
We demonstrated that the pCAM-ICU can be reliably used by nonpsychiatrically trained clinicians to diagnose delirium in children ≥5 yrs of age with high sensitivity and specificity. In this and other studies, approximately one in ten critically ill children are at significant risk for the development of acute brain dysfunction. Consequently, the lack of awareness, ability to monitor, and appropriately intervene for delirium perhaps place these patients at jeopardy for poorer outcomes. This instrument provides the first means by which to rapidly (average 2 mins) and easily diagnose pediatric delirium in critically ill children at the bedside. Incorporation of the pCAM-ICU into clinical practice may further our understanding of the clinical presentation, risk factors, and possible interventions for critically ill children with delirium.
Dr. Pandharipandge is supported by a Veterans Affairs Career Development Award, the Clinical Science Research and Development (CSRD) program; and received honoraria/speaking fees and grants from Hospira. Dr. Ely is supported by the National Institutes of Health (AG027472) and the Veterans Affairs Tennessee Valley Geriatric Research, Education and Clinical Center (GRECC), Nashville,TN; consulted for Eli Lilly and consulted for and received grants from Hospira and Pfizer.
The remaining authors have not disclosed any potential conflicts of interest.