|Home | About | Journals | Submit | Contact Us | Français|
Determine levels of agreement among intensive care unit patients and their family members, nurses, and physicians (proxies) regarding patients’ symptoms and compare levels of mean intensity (i.e., the magnitude of a symptom sensation) and distress (i.e., the degree of emotionality that a symptom engenders) of symptoms among patients and proxy reporters.
Prospective study of proxy reporters of symptoms in seriously ill patients.
Two intensive care units in a tertiary medical center in the Western United States.
Two hundred and forty-five intensive care unit patients, 243 family members, 103 nurses, and 92 physicians.
On the basis of the magnitude of intraclass correlation coefficients, where coefficients from .35 to .78 are considered to be appropriately robust, correlation coefficients between patients’ and family members’ ratings met this criterion (≥.35) for intensity in six of ten symptoms. No intensity ratings between patients and nurses had intraclass correlation coefficients >.32. Three symptoms had intensity correlation coefficients of ≥.36 between patients’ and physicians’ ratings. Correlation coefficients between patients and family members were >.40 for five symptom-distress ratings. No symptoms had distress correlation coefficients of ≥.28 between patients’ and nurses’ ratings. Two symptoms had symptom-distress correlation coefficients between patients’ and physicians’ ratings at >.39. Family members, nurses, and physicians reported higher symptom-intensity scores than patients did for 80%, 60%, and 60% of the symptoms, respectively. Family members, nurses, and physicians reported higher symptom-distress scores than patients did for 90%, 70%, and 80% of the symptoms, respectively.
Patient–family intraclass correlation coefficients were sufficiently close for us to consider using family members to help assess intensive care unit patients’ symptoms. Relatively low intraclass correlation coefficients between intensive care unit clinicians’ and patients’ symptom ratings indicate that some proxy raters overestimate whereas others underestimate patients’ symptoms. Proxy overestimation of patients’ symptom scores warrants further study because this may influence decisions about treating patients’ symptoms.
Intensive care unit (ICU) patients often experience substantial pain and/or discomfort, whether associated with recent surgery (1) or trauma (2), from procedures performed while in the ICU (3–5), or associated with a variety of medical conditions (1). In addition, a significant percentage of these patients experience many other distressing symptoms, including dyspnea (6, 7), anxiety/fear (7–9), difficulty in sleeping (8, 10, 11), hallucinations (12), and thirst (11, 13). Although previous research noted that ICU patients who are able to communicate may receive medications or other therapies to relieve these symptoms, even those who can express their concerns experience persistent unrelieved symptoms (14, 15). What is not known is whether patients who are unable to communicate have the same experiences (16). For the patient who cannot communicate, critical care providers use a variety of other means to assess patient discomfort, pain (16–19), dyspnea (20), and other symptoms, including assessment of physiologic signs. However, no consistent method has been used to assess these symptoms and appropriately treat them. In fact, in many cases the provider or family member may recommend treatment based on the presumed discomfort or anticipated pain associated with a procedure, without having the ability to confirm the findings with the patient.
Information about the assessment of patients’ symptoms by proxies is available from studies performed outside the ICU. Indeed, the use of proxies as substitutes for patients’ reports of symptoms often occurs in palliative care and other settings (17–20). In the non-ICU setting, family members are identified as an important source of proxy information when the patient is seriously ill and unable to communicate symptoms (17). Family members are able to report their own perceptions and observations as well as what the patient may have communicated to them previously (19, 21). The accuracy and thoroughness of nurses’ (22) and physicians’ (23) assessments of the symptoms of palliative care patients are variable, with nurses’ ratings appearing to more closely approximate palliative care patients’ ratings (17, 18, 24). Agreement between patients’ and proxies’ ratings of their symptoms may not depend on the type of proxy (i.e., family member, nurse, or physician) (19, 25). These findings suggest that proxy reports may be useful with ICU patients who have impaired communication or cognitive problems, or when self-report may be too burdensome for patients. ICU nurses and physicians are another potential source of proxy information. However, the accuracy of proxy estimates of multiple symptoms has not been evaluated in seriously ill ICU patients.
Only one study evaluated the effectiveness of proxy (nurse) reporters of ICU patients’ symptoms (26). In this study, pain was assessed by both ICU patients who could communicate and their nurses. A moderate-to-strong correlation (r2 = .45; p < .05) was found between the two groups of reporters at the first evaluation time. However, the sample size (n = 31) was small, and pain was the only symptom assessed. In another study, Desbiens and colleagues (27) found that family members of hospitalized patients correctly assessed the patient’s level of pain 53% of the time. However, their study included patients outside ICUs and only assessed one symptom.
This study aimed to address both symptom intensity and distress since they represent two different constructs. That is, intensity is related to the magnitude of the symptom sensation, and distress is related to the degree of emotionality that the symptom engenders (28). Since the magnitudes of intensity and distress are frequently different (28, 29), these differences could warrant different treatment interventions, making it important to assess both. The purpose of this study was, first, to determine levels of agreement among proxy reporters (i.e., patients, family members, nurses, physicians) regarding ICU patients’ symptom characteristics (i.e., intensity, distress) for each of the ten symptoms. The symptoms were pain, tiredness, shortness of breath (SOB), restlessness, anxiety, sadness, hunger, fear, thirst, and confusion. The second study purpose was to compare the mean intensity and distress ratings of the ten symptoms among reporter groups to determine whether patients’ ratings were higher, lower, or similar to those of proxy reporters.
This study was conducted in two ICUs in a tertiary medical center in Western United States. Patients aged ≥18 yrs who met the following criteria were considered to be seriously ill and, thus, were eligible for this study: if they had a first 24-hr Acute Physiology, Age, Chronic Health Evaluation II score ≥20 (30); if they were in the ICU for ≥3 days; and had one or more of the following diagnoses: acute cardiac and/or respiratory failure; chronic liver failure with cirrhosis; multiple organ system dysfunction and sepsis; or any system failure associated with a diagnosis of a malignancy (31, 32).
A ten-item symptom checklist was developed based on findings from previously published studies and an extensive review of the literature on symptom-assessment instruments. Both physical and psychological symptoms were included in the checklist. The symptom list was limited to ten items (i.e., pain, tiredness, SOB, restlessness, anxiety, sadness, hunger, fear, thirst, and confusion) to reduce reporter burden, and provide a valid and easily administered assessment at the bedside. Reporters were asked to rate whether the symptom was present in the patient and, if present, its intensity (1 = mild; 2 = moderate; 3 = severe) and distress (1 = not very distressing; 2 = moderately distressing; 3=very distressing). Validity and reliability of this instrument have been reported previously (33).
Patient demographics and clinical characteristics were collected; these included age, gender, ethnicity, diagnosis, and first 24 hrs in ICU Acute Physiology, Age, Chronic Health Evaluation II score. Calculation of patient’s Acute Physiology, Age, Chronic Health Evaluation II scores was done using recommended methods (34), and interrater reliability of chart abstractions was assured, as previously described (33). Data from family member, nurse, and physician were collected; these included age, gender, ethnicity, years in clinical practice, and years in critical care practice.
The subject’s approval was obtained from the Institutional Review Board at the tertiary medical center where this study was performed. Patients were screened consecutively by the research team, which assessed their eligibility to participate. If the patient met the eligibility criteria (including being calm and cooperative) (35) and was willing to participate, and a family member, nurse, and physician agreed to participate, the patient signed the consent form. However, if the patient was unable to sign the consent because of physical impairment, a family member was asked to provide written witnessed consent for the patient. In some cases, delegated family members gave surrogate consent for enrollment when it was determined that patients could not communicate. The patient’s family member who was identified as his/her closest surrogate as well as the nurse and physician providing care on the study day were approached to participate in the study and completed informed consent documents. The person who was primarily responsible for making medical decisions on the study day was deemed “the patient’s physician.” This person was sometimes a nurse practitioner.
Research nurses interviewed the patients, their family member, nurse, and physician to assist them in completing the symptom surveys. When the patient was not able to participate, the surveys were limited to the other three reporters. Each survey was completed independently, and reporters were not aware of the others’ answers. As timing of assessments could influence concordance between symptom ratings and discrepancies do occur when assessments are not closely matched in time (22, 23), all surveys were completed within 2–8 hrs of each other (mean = 3.34 + 1.34 hrs). This time range was referred to as the “reporters’ window of time.” Proxy reporters were asked to base their assessments on the most recent time period that they spent with the patient that day. We included the reporters’ window of time as a variable in our statistical models but found that time produced little change in estimates of interest.
Agreement levels among reporter groups were assessed through use of intraclass correlation coefficients (ICCs), a standard measure of interrater reliability (i.e., agreement) of quantitative ratings (36). Within-patient-set ICCs were estimated using moment-based methods (37) as implemented by the Generalized Estimating Equation (GEE) approach and routines in the SPSS package. The GEE method takes into account both clustering of assessments within the patient set as well as missing values. It also allows for the estimates of ICCs to vary over different pairs of reporters (38).
The estimated correlation coefficients derived from GEE were categorized according to the distribution of correlation coefficients recommended by Hemphill (39) for behavioral/psychological assessments (vs. medical tests). In this method, correlation coefficients of .35 to .78 are considered to be in the upper third of correlation distributions. In addition, in situations where high ICCs would be expected such as in family research, Hox (40) recommends considering an ICC of .30 to represent a large effect. For these reasons, we took ICCs of .35 to .78 to be reasonably robust in magnitude.
Differences among reporter groups in intensity and distress levels were assessed using mixed-effects ordinal logistic regression models with random intercepts through use of a STATA-affiliated program package called generalized linear latent and mixed models (41). The responses in these models were the ordinal intensity ratings that could range from 0 = no, 1 = mild, 2 = moderate, to 3 = severe and the distress ratings that could range from 0 = no distress, 1= not very distressing, 2 = moderately distressing, 3 = very distressing. The models included a random intercept to accommodate the multiple ratings for each patient and indicators for reporter group. The null hypothesis (i.e., the rating levels were the same across the four reporter groups) was tested using a Wald chi-square with three degrees of freedom. The ordinal models assessed within-patient differences in rating levels among family members, nurses, physicians, and patients using odds ratios. GEE and mixed-effects ordinal analyses included the drug window variable, and the adjusted estimates differed little from the unadjusted ones.
A total of 1,273 patients were screened, and 245 were enrolled. Of the 245 patients, 171 patients were able to provide symptom reports on one or more of seven study days over a 14-day study period. (Please refer to previous patient symptom report and previously published flow diagram of study sample) (33). This report is limited to the first study day defined as the day that family members consented to participate in the study. This was also the day when most data were collected from all reporters. On that day, 89 patients (36%) were able to provide some responses about symptom intensity and distress. Symptoms for the remaining 156 patients were reported by their family members, nurses, and physicians. Table 1 provides information about the patients’ age, gender, ethnicity, and severity of illness. It also provides demographic information for participating family members and clinicians as well as the clinicians’ professional and critical care practice history.
Table 2 presents the predicted ICCs between patients’ ratings of symptom intensity and the ratings of the other reporters. The ICCs between patients’ and family members’ ratings were in the upper-third distribution (i.e., from .35 to .78) for six of ten symptoms (i.e., pain, SOB, restlessness, anxiety, sadness, and fear). No symptom intensity ICCs between patients’ and nurses’ ratings were in the upper-third distribution. Three symptom intensity ICCs between patients’ and physicians’ ratings were in the upper-third distribution (i.e., pain, tiredness, SOB).
Table 3 presents the predicted ICCs between patients’ ratings of symptom distress and the ratings of each of the other reporters. The correlation coefficients between patients’ and family members’ ratings were in the upper-third distribution for five of ten distress scores (i.e., pain, SOB, restlessness, fear, thirst). No symptom distress correlation coefficients between patients’ and nurses’ ratings were in the upper-third distribution. Two symptom distress correlation coefficients between patients’ and physicians’ ratings were in the upper-third distribution (i.e., anxiety, thirst).
Table 4 presents mean symptom intensity scores for all reporters. The full range of scores (i.e., 0–3) was used for this analysis. Statistically significant differences in intensity ratings were found between patients and the other reporters for pain, sadness, fear, confusion, and restlessness (p = .001–.03) The ordinal logistic regression analyses showed that the odds of higher pain intensity rating were two times greater among family members, nurses, and physicians than among patients. Similarly, the odds of higher intensity of sadness were two to three times greater among family members, nurses, and physicians than among patients. The odds of higher intensity of fear were four times greater among family members, nurses, and physicians than among patients, and the odds of higher confusion intensity were five to 11 times greater among family members, nurses, and physicians than among patients (Table 5). However, all differences were <1 point on the symptom intensity scale. No statistically significant differences were found between patients’ and the other reporters’ ratings of the intensity of tired-ness, thirst, anxiety, hunger, and SOB.
Table 6 presents mean symptom distress scores for all reporters. The full range of scores (i.e., 0–3) was used for this analysis. Statistically significant differences in distress ratings were found between patients and the other reporters for pain, fear, confusion, tiredness, restlessness, and sadness (p < .05 to p < .001). The ordinal logistic regression analyses showed that the odds of higher distress from pain were two times greater among family members, nurses, and physicians than among patients. Similarly, the odds of higher distress for fear were three to five times greater among family members, nurses, and physicians than among patients, whereas the odds of higher distress for confusion intensity were 5–9 times greater among family members, nurses, and physicians than among patients. Finally, the odds of higher distress for sadness were three times greater among family members and physicians than among patients. However, all differences were <1 point on the symptom distress scale. No statistically significant differences were found between patients’ and the others’ ratings of distress for thirst, anxiety, hunger, and SOB.
This study is the first to report “real-time” and comprehensive evaluations of patients’ and proxy reporters’ ratings of common, severe, and distressing symptoms in ICU patients. Although, in general, differences between patients’ and proxies’ ratings were small, family members’ ratings were closer in agreement with patients’ ratings than clinicians’ ratings were.
The first aim of this study was to determine ICCs, a measure of agreement, among reporters regarding ICU patients’ ratings of the intensity and distress of ten common symptoms. ICCs, as a measure of agreement, indicate how consistently pairs of raters rank order symptoms (18). ICCs between patients and family members were more robust in magnitude for the intensity of several symptoms than ICCs between patients and their nurses and physicians. Patient-family member ICCs at an acceptable level of magnitude (39, 40) ranged from .35 for SOB to .48 for sadness. ICCs for intensity ratings for fear, anxiety, pain, and restlessness fell within that range.
In terms of symptom distress, family members again were in higher agreement with patients’ ratings for more symptoms than were either nurses or physicians. Patient-family ICCs for distress at an acceptable level of magnitude ranged from .40 for SOB to .47 for restlessness. ICCs for distress ratings for fear, pain, and thirst fell within that range. These robust ICCs between patients and family members may be due to closer and more stable relationships between patients and family members (29) than between patients and ICU clinicians. Indeed, Hinton (42) found that symptom ratings between family members and 71 terminally ill cancer patients were moderately (κ >.50) to almost perfectly (κ >.85) matched. In this present study, 50% of the family members were spouses, and 68% of patients and family members lived together. This suggests close and long-term relationships with each other. In addition, compared to clinicians family members may have had prior opportunities to observe the patient’s pain or other symptoms at home and/or over a longer period of time (19) and make judgments about the patient’s symptoms.
For all symptoms, levels of agreement on symptom intensity and distress between patients and nurses were low. In addition, levels of agreement on symptom intensity between patients and physicians were low except for tiredness (0.52), pain (0.40), and SOB (0.36). Levels of agreement on symptom distress between patients and physicians were low except for thirst (0.49) and anxiety (0.40). These findings are consistent with previous reports of low concordance between nursing (22) and medical (23) documentation of symptoms and advanced cancer patients’ reports of their symptoms. These low levels of agreement between clinicians’ and patients’ ratings suggest that some clinicians overestimate, whereas other clinicians underestimate patients’ symptoms (18). These lower ICCs between the patients and the patients’ nurses and physicians than between patients and family members may be due to the fact that the ICU clinicians are focused on many responsibilities and tasks, leaving them less time to focus on patients’ symptoms. Many times clinicians are focused on keeping patients alive, leaving them less time to assess patients’ multiple symptoms. To the disadvantage of clinicians, there is a general lack of protocols that could be used to routinely and quickly assess common symptoms. In addition, the emphasis on evaluation of signs vs. symptoms in ICU patients may contribute to the discordant ratings. Indeed, when ICU nurses (n = 22) were asked about the symptoms they observed in their patients, they often reported signs (e.g., fever and hypotension) rather than symptoms (43). Assessing multiple symptoms in critically ill patients would be a general broadening of ICU clinicians’ scope of patient care. In the future, ICU clinicians could develop the type of expertise in symptom assessment commonly found in oncology and palliative care colleagues.
The second aim of this study was to compare mean intensity and distress scores of the ten symptoms among reporters. Compared to ICCs, which should be high, mean differences in ratings should be low between patients and proxy reporters (18). It is interesting that no significant differences in scores between patients and proxies were found for tiredness, SOB, anxiety, hunger, and thirst (intensity) and SOB, anxiety, hunger, and thirst (distress). In studies of surrogate ratings of oncology patients’ symptoms, scores increased in proportion to the degree of symptom intensity and distress. That is, the higher the patients’ symptom rating, the more similar were the scores of the proxy raters (22, 23, 44). Our findings are consistent with these reports. As noted in Table 4 (intensity) and Table 6 (distress), there are no statistically significant differences between patients and proxy reporters for the symptoms that had the highest patient scores. Specifically, the highest intensity scores reported by patients in this study were associated with tiredness, SOB, anxiety, and thirst, and three of the highest distress scores reported by patients were SOB, anxiety, and thirst.
Family members, nurses, and physicians rated symptom intensity and distress scores higher than patients did for five symptoms (pain, sadness, fear, hunger, confusion). (Tables 4 and and6).6). This is similar to the scores for family members of cancer patients who tended to overestimate the intensity and distress of patients’ pain, (45, 46). Similarly, in a study of 2,645 seriously ill hospitalized patients (27) family members overestimated the patients’ pain 16.8% of the time and underestimated it 9.7% of the time. Conversely, other reports have described clinicians’ underestimation of palliative-care patients’ symptoms (18). Research clinicians and nurses rated symptoms of patients with human immunodeficiency lower than patients did (47); and physicians’ and/or nurses’ ratings of pain were lower than those of cancer patients (18). Nurses caring for patients in emergency departments (48, 49) and surgical units (26) also underrated patients’ pain. Taken as a whole, clinicians both under- and overrate patients’ symptoms, indicating a need for more clinical and research attention to “best” ways to improve concordance of symptom reports, to the benefit of patients.
It is not entirely clear why both family members and clinicians overestimated the intensity and distress of five symptoms (pain, sadness, hunger, fear, confusion). It may be that these patient’s proxies project themselves into the patient’s situation and imagine what it would be like for them, rather than the patient, to have the symptoms (18, 19). The alternative approach to estimating patients’ symptoms could be for proxies to take a more neutral or “imagine patient” perspective in their assessments. Lubuchuk and colleagues (50) found that 98 family care-givers’ judgments about cancer patients’ symptoms corresponded more closely to their loved ones’ symptom reports when caregivers remained neutral or “placed themselves in the patient’s shoes” (p. 2379). These findings were replicated in another study of breast or prostate cancer patients and their family member care-givers (51). In short, less discrepancy may occur between patients and proxy reporters if proxy reporters are prompted to take a neutral approach or to empathize with the patient (i.e., imagine what it must be like for the patient) and avoid being absorbed in their own feelings of what it would be like if they were the patient (i.e., to feel sympathy) (52). Further research is warranted to determine whether less discrepancy may occur between patients and proxy reporters if proxy reporters used a neutral or empathic approach when doing symptom assessments.
Overestimation on the part of the proxy reporters could also be due to patient underreporting of their symptoms. Patients have many reasons to minimize their symptoms as they try to cope with and avoid burdening others (29, 53). However, this explanation may be less relevant in an ICU setting, because patients are seriously ill and may not have the cognitive capacity to intentionally underreport their symptoms for the sake of others.
The type of symptoms may also influence the closeness of patient–proxy symptom assessments such as the visibility of a physical problem vs. less visible psychological symptoms (19, 20, 22, 23, 29, 54). However, in this study, ICCs were relatively robust for both physical (e.g., SOB, pain, restlessness) and psychological (e.g., anxiety, fear) symptoms, at least between patients and family members. Again, this finding may reflect a measure of closeness between patients and family members (55).
The experience of proxies may also influence the closeness of assessments. Most of the physicians in this study were trainees with variable levels of experience in critical care or in the assessment of these symptom complexes. Nurse practitioners are also relatively new providers at this level. More experienced attending physicians might report a symptom assessment closer to that given by the patient. However, the more experienced bedside nurses did not have a high agreement with patients about the patient symptoms. These discrepancies between experience and concordance of symptom reports warrant further investigation.
Interestingly, there were difference in reporters’ ratings of the intensity and distress of restlessness and fear but no differences in the intensity and distress of anxiety. These symptoms could be considered as overlapping one another. It seems apparent that respondents were able to discriminate among these symptoms.
With regard to confusion, all proxy reporters in this study rated the severity and distress associated with confusion higher than the patients did. The reason for this finding is unclear but warrants consideration because of the increased clinical attention being given to delirium in ICU patients (56–58). Although confusion and delirium are not the same phenomenon, patients in this study were able to know whether they were or were not confused, as noted in a previous report from this study (33). In fact, both delirious and nondelirious patients were able to report whether they felt confused. The increased use of delirium scales in ICUs (59, 60) may be sensitizing clinicians to the symptom of delirium. However, if a patient is not assessed for both confusion and delirium and the patient is wrongfully assumed to be confused or delirious, opportunities for important communication between clinicians and patients about the patient’s general condition, progress, and symptoms could be missed.
Some patients (64%) were unable to report their symptoms on the study day due to the severity of their illness or impaired level of alertness. Although this “missingness” in data was addressed through the use of GEE as the method of statistical analysis, the specific symptom reports from these patients are, nevertheless, unknown. Having all reporters provide their symptom ratings closer together could have provided a more sensitive, “real-time” picture of the patients’ symptoms. However, the average 3.3-hr reporters’ window of time did not result in statistically significant differences in symptom ratings among the reporters. Finally, using 0 to 3-point symptom intensity and distress scales, rather than a 0–10 scale decreased the sensitivity of the measures and, thus, may have limited the strength of the ICCs. A 0–3 point range was selected for this study because the scale was developed from symptom assessment scales used in palliative care settings that used 0–3 ranges.
Findings from this study suggest that proxy reports, especially from family members, are often similar to patients’ self-reports of their symptoms. For family members, symptom ICCs were high, whereas differences in symptom magnitudes were low. In addition, no significant differences were found between patients’ and all proxies’ reports of the intensity of tiredness, SOB, anxiety, hunger, and thirst and reports of the distress of SOB, anxiety, hunger, and thirst. Indeed, proxy ratings were more consistent with what patients reported if the patients’ symptom intensity and/or distress reports were high. Although proxy data may not perfectly match patient symptoms, proxy symptom reports can be used to track trends over time (17). Furthermore, when the self-report capability of the patient deteriorates, others should be seriously considered as useful sources of proxy information (20, 24) about ICU patients’ symptoms. An integrated approach to symptom assessment could incorporate multiple proxy raters of patients’ symptoms, including family members who were in closer agreement with the patient in this study. Using structured symptom assessment tools may be the most appropriate way to accurately assess noncommunicative patients’ symptoms, but this untested assumption requires further study (29). As noted by Justice et al (47), “only symptoms that providers recognize and report ‘count’ in most clinical and research settings” (p. 397). Recognizing, reporting and, ultimately treating ICU patients’ symptoms should be a priority for those closest to the patient.
This study was supported by a grant from the National Institute for Nursing Research (NR008247).
The authors have not disclosed any potential conflicts of interest.
For information regarding this article: firstname.lastname@example.org