We have proposed a novel composite outcome measure for use in clinical trials of critical care interventions. We have used a database from a randomized trial of nutrition therapy to demonstrate the prevalence of persistent organ dysfunction amongst survivors of critical illness and to provide realistic estimates for sample size comparisons. We have shown that about a quarter of survivors at Day 28 will still have a need for on-going support from life sustaining technologies in an ICU (POD). These patients who survive to Day 28 and still have POD have a much higher subsequent mortality rate, prolonged hospital course, and reduced quality of life at three months compared to survivors without POD. This is consistent with a prognostic model proposed by Carson and colleagues that showed the patients who undergo prolonged mechanical ventilation (21 days) and have persistent organ dysfunction (need for vasopressor and haemodialysis) have a much greater mortality than those patients with prolonged mechanical ventilation without ongoing organ dysfunction [15
Composite endpoints are rare in the critical care medicine literature; we are aware of only a few examples. Nathens and colleagues evaluated the effect of an antioxidant supplementation strategy in critically ill trauma patients and reported the combined endpoint of ARDS and pneumonia to reflect overall pulmonary morbidity [16
]. In a randomized trial of atrial natriuretic peptide in ischemic acute renal failure in critically ill patients, Sward and colleagues reported on the combined event rate of dialysis or death before or at Day 21 [17
]. Schweickert et al.
evaluated the impact of early physical and occupational therapy on mechanically ventilated patients and chose a primary endpoint of independence with activities of daily living and ability to walk [18
]. Finally, Shuster and colleagues conducted a randomized trial of empiric fluconazole in patients at high risk of systemic candidiasis and used a composite that consisted of the following four variables: resolution of fever, absence of invasive fungal infection, no discontinuation because of toxicity, and no need for a non-study, systemic antifungal medication [19
]. In no case was a validation exercise, such as we have reported here, conducted to support the use of such composites.
When POD is combined with death to form a composite endpoint, the overall event rate in our dataset increases from 27% for mortality alone to 46% for POD or mortality with increased gains in statistical efficiencies that reduce the sample size required to show clinically important differences compared to when 28-day mortality is used as the primary endpoint. However, the benefits of using POD + death as an outcome measure are more than its impact on statistics and sample sizes. Conceptually, POD + death provide a more comprehensive evaluation of a critical care intervention, evaluating both measures of morbidity combined with mortality. The present reliance on 28-day mortality is not optimal since it would suggest that a patient alive at Day 28 is considered a 'treatment success' even if they are in renal failure and bedridden requiring ongoing mechanical ventilation. To the extent that these surviving patients with major organ dysfunction requiring support occur more frequently in one group in a clinical trial, this important difference will be missed when the focus is on mortality alone and leads to erroneous conclusions about the efficacy of the intervention. However, the validity of a composite measure depends on the extent to which each individual component is similar in its importance to individual patients [20
]. When the individual components are equally considered as adverse by patients, then it may be appropriate to combine them. However, when there is a gradient of importance across the individual endpoints, it may be invalid to combine them in the form of a composite. For example, investigators evaluated the effect of corticosteroids amongst patients with chronic obstructive pulmonary disease and chose to combine death from any cause, need for mechanical ventilation, and the unblinded administration of corticosteroids together in the form of a composite endpoint [22
]. However, from a patient's point of view, it could be argued that a short course of corticosteroids would be judged to be much less of concern than death or the need for mechanical ventilation, which questions the validity of this composite. We have shown that patients surviving at Day 28 with POD will suffer from ongoing risk of serious morbidity, increased mortality, and reduced quality of life at three months. These findings support the argument that POD is similar (but not the same) in magnitude to death and it would be appropriate to combine with death.
Because POD is associated with worse outcomes in the future, some may argue that hospital mortality or longer-term mortality assessments be considered as the primary outcome to judge the efficacy of critical care interventions [2
]. We are sympathetic to this notion; however, there are many determinants to mortality unrelated to the ICU illness and the effect of ICU therapy. For example, an 82-year-old patient was admitted to ICU with aspiration pneumonia and respiratory failure following a stroke. Initially, the patient was in shock, requiring vasopressors and had a transient elevation in the serum creatinine. He was enrolled in our REDOXS study and received 14 days of nutrient therapy (or placebo). After two weeks, the patient was liberated from mechanical ventilation and shortly thereafter was discharged from ICU where he had a prolonged stay in hospital convalescing and rehabilitating. Two months after his admission to ICU, he died from complications related to his original stroke and functional disability. This case demonstrates how an ICU therapy may be efficacious in resolving the ICU illness and facilitating a good recovery in the short term but the patient ultimately dies at a time point very remote from his ICU illness. If we use hospital or longer-term time frames to judge the efficacy of our studied ICU therapies, we increase the likelihood that these non-attributable events may occur, resulting in so much noise that it diminishes our effect to detect a signal (that is, the intervention being studied has no chance at altering them) [23
]. If we use a time frame more proximate to the ICU admission and application of ICU therapy, we may have greater sensitivity and power to detect treatment effects compared to later outcome assessments. POD + death at 28 days enables a comprehensive assessment of both morbidity and mortality at a time point that will be less likely confounded by random events unrelated to the ICU treatment compared to hospital or longer-term outcomes. Nevertheless, POD + death could be determined at a longer (3 months) or shorter (14 days) time frame depending on the nature of the study. The optimal time to evaluate the outcome determination may vary depending on a number of clinical and methodological factors; and it should be kept in mind that, since POD is time dependent, conclusions may depend on the timeframe selected.
The validity of combining individual events together to form a composite is also dependent on the consistency of the treatment effect across events [20
]. For example, we would not want to call a treatment successful if it reduced POD while increasing mortality. However, this is very unlikely since for most patients POD would be on the pathway to death and will be present at the time of death. Composite endpoints may also be misleading if the number of events in the components of greater importance is small compared to the number of events in the components of lesser importance [21
]. For example, a statement by investigators that a novel intervention reduces a composite of death, need for dialysis, and reduced serum creatinine is problematic if most of the events were related to a rise in creatinine and investigators found a large apparent treatment effect on rising creatinine but not on the need for dialysis or death. With respect to POD + death, the opposite is found. The most important event, death, occurs much more frequently (27%) than the rates of individual organ failures at 28 days (2 to 14%).
Finally, for composite endpoints to be useful, we need to demonstrate that a critical care intervention has a consistent treatment effect across all individual components before we can have confidence in the interpretation of the composite endpoints. The measurement of a treatment effect can be diluted by combining endpoints, some of which are modified by the intervention, others which are not. For example, investigators studied the effects of carvedilol in patients with left ventricular dysfunction following myocardial infarction [24
]. The primary endpoint was the combined rates of all deaths and cardiovascular hospital admission. The study showed that carvedilol was associated with a significant reduction in all-cause mortality (P
= 0.03) but when combined with hospital admission, there was no overall effect (P
= 0.30). The mortality effect disappeared when combined with an endpoint that did not change with the intervention. Thus, therapeutic interventions need to have efficacy on all the components before it will be advantageous to combine them. For example, with the mortality and POD rates observed so far in REDOXS, adding POD to a mortality RRR of 25% would only improve efficiency if the RRR of POD was at least 5%. POD and death can 'sensibly' be combined as they are aspects of the same underlying disease process, an underlying inflammatory/immunological process that results in acute organ failure and may result in death or persistent organ failure. We intend to analyze and describe the treatment effects of glutamine and antioxidants on mortality and the individual components of POD with the final results of the REDOXS study when it is completed.
Compared to other options for outcome measures, POD + death offers several other advantages. In contrast to other measures of organ dysfunction/failure which require daily data collection, POD + death requires data collection only on the day of assessment. Furthermore, determination of POD uses readily available clinical parameters that will be easily discerned either prospectively or retrospectively and does not suffer from large amounts of missing data like all other organ failure scoring systems. Compared to traditional mortality endpoints considered in this analysis, POD + death requires fewer study subjects to achieve adequate power to detect a treatment effect. Large differences in 'failure-free days' would require the fewest patients. However, the advantage of POD is its simplicity, lack of missing data, and robustness. To our knowledge, validation exercises similar to what we have performed with POD have not been performed for such endpoints. Finally, by combining death with POD, we avoid the problem of maintaining type I error control over multiple outcomes, and we eliminate the complexities involved with analyzing and interpreting non-fatal outcomes for which death is a competing risk.
The measurement of POD suffers limitations that are common to other organ failure scoring systems. Ideally, scores should be independent of clinician determined therapeutic variables, to minimize bias in the determination of the endpoint [4
]. However, this is virtually impossible for all organ failure assessment scoring systems. For example, PaO2/FiO2 ratios will vary dependent on use of positive end expiratory presure, cardiovascular scores will be influenced by use of vasopressors, platelet counts are influenced by platelet transfusions, and so on. For POD, all outcomes assessments are determined by the use of life support technologies, which are determined by clinicians. This limitation will be less important in the context of double-blinded randomized clinical trials.
Other limitations specific to POD is the lack of correlation between POD and six-month quality of life assessment among survivors. This is offset by the presence of POD at 28 days being associated with mortality at six months and may be explained by the fact that patients with a poor outcome have either died or fully recovered by six months.