This study suggests that this quality-improvement intervention had no effect on family- and nurse-assessed outcomes. Prior studies suggest that palliative care and family communication interventions were associated with reduced ICU length of stay before death presumably because of earlier decision-making about withdrawal of life support (8
). We saw no significant change in the number of ICU days before death and no significant change in the time from ICU admission to withdrawal of mechanical ventilation. In addition, we examined nine elements of palliative care and found no consistent evidence to suggest the intervention increased these elements of palliative care.
One concern that may affect interpretation of our results was the response rate from family and nurses. The goal of the clustered trial was to assess outcomes at the hospital level and we obtained adequate samples for all outcomes at all hospitals. Furthermore, the family response rate was low in part because we sent surveys to patients' homes after death and some of these surveys may have never reached a family member. Our response rate is similar to other survey studies enrolling family after death and studies enrolling clinicians to assess end-of-life care (32
). However, potential for response bias should be considered. We previously examined patient differences between responding and nonresponding family members showing that nonresponse bias results in an overestimation of the quality of palliative care (37
). Because we have data on patient and patient-care characteristics for 97% of eligible patients, we were able to construct weights reflecting the propensity for survey response. Analyses using these weights did not alter the findings. Therefore, it seems clear that this intervention did not improve palliative care or outcomes.
Another potential explanation for lack of change in family- and nurse-assessed outcomes is that perhaps these outcomes are not sensitive to important changes. Although the QODD and the FS-ICU were developed and validated as end-of-life-care outcomes, the responsiveness of these measures is unknown. Therefore, we are unable to know for certain whether there might have been important changes that these instruments are unable to detect. However, we do have compelling data that these outcomes can differentiate quality of end-of-life care. We previously reported that family-QODD scores were 7 points higher for patients who died in the location they preferred (home or institution) as compared with patients who did not (19
). In this study we found that patients who died after a transition to comfort measures only had QODD scores 10 points higher than patients who died with full support. Our study had a power of 0.62 to detect a 7-point difference and a power of 0.87 to detect a 10-point difference. It is possible that a larger sample might be needed to definitively exclude an important but smaller improvement in quality of dying. Finally, it is also possible that these instruments measure important differences in family experience, but that these family experiences are determined by many other factors over which clinician behavior may have little influence (38
). However, there is evidence that clinician behaviors are important determinants of family experience and may be an important target for future interventions (6
The most plausible explanation for our negative results is that the intervention was ineffective. We previously published a single-center before–after study of the same intervention showing no significant improvement in family ratings of quality of dying or satisfaction with care, but an improvement in nurses' ratings of quality of dying and a significant reduction in ICU days before death (13
). This prior study used the same intervention but took place at the home institution of investigators, where the intervention was easier to implement. Implementation at other sites was more challenging and may have resulted in a lower “dose.” Although we delivered all five components to all intervention sites and demonstrated that ICU clinicians rated these components highly, ICU clinicians had many competing demands requiring longer implementation with less uptake of the intervention than planned. We also found that, although designed as an interdisciplinary intervention, it was difficult to transcend the silos of clinical disciplines (42
What are the lessons from this trial? There is growing interest in improving quality and reducing costs. Although it is difficult to accomplish both of these goals with a single intervention, enhancing palliative care within acute and critical care is one approach that has generated excitement for this potential (43
). There is evidence that earlier and more effective communication with patients and families about end-of-life care may result in higher-quality care that minimizes ineffective life-prolonging treatments, reduces costs, and improves quality of life (8
). Unfortunately, our study suggests that a quality-improvement intervention designed to educate ICU clinicians about palliative care and implemented by experts outside the institution is unlikely to have these benefits. Our findings, together with prior studies, suggest that interventions may need to be implemented from within an institution with stronger intrainstitutional support (11
), and that interventions may be more effective if they bring clinicians with palliative care expertise directly into the care of individual critically ill patients and their families (9
This study has additional important limitations. First, a randomized trial of hospital-based interventions requires randomizing hospitals, which is expensive and time-consuming. Our effective sample size is limited by the number of hospitals in the study. Second, randomizing hospitals resulted in unequal distribution of patient characteristics between the two groups. We used baseline data from each hospital and multivariate techniques to adjust for these differences. Third, implementation of this multifaceted intervention was complex and it is difficult to measure the “dose” delivered. Assessment of delivery of this intervention suggests all intervention components were implemented with high levels of clinician satisfaction, but we are limited in our ability to measure the degree of uptake. Fourth, this study was confined to one region of the United States and may not generalize to other regions. However, it is notable that the QODD scores from these institutions are comparable to studies done in ICUs in other areas (20
). Finally, given that this was a negative study, it would be informative to know if clinician attitudes were changed by the intervention. Unfortunately, we do not have data to assess this question.
In summary, our study demonstrates no effect of this multifaceted, quality-improvement intervention on family- or nurse-assessed outcomes or delivery of palliative care. Furthermore, this study suggests that efforts to improve family and nurse experiences of end-of-life care in the ICU will require an intervention with more institutional support and direct involvement in the care of individual patients and their families.