Traumatic brain injury often results in a multifaceted disability, and recovery from TBI is a complex process that involves many aspects of functional and cognitive change. Therefore, measuring outcome presents many challenges in large phase III trials of patients with TBI. Targeting only 1 aspect of possible improvement may not be sufficient to determine the effectiveness of a new intervention and it is susceptible to an increased type II error. To address this issue, the TBI-CT Network Outcome Subcommittee has proposed to use a core of 9 measures that in combination will evaluate outcome in the Network’s first trial. These measures cover 2 components of important outcome: functional status and cognitive abilities.
Many clinical trials of patients with TBI still rely on the GOS33
as the primary measure of recovery and to determine treatment effect. In a recent article, Lu et al38
point out the pitfalls associated with the use of the GOS in TBI trials and, in particular, the vulnerability of this scale to misclassification. The authors illustrate how common types of misclassification result in considerable loss of statistical power and the attenuation of the true treatment effect. The same authors also suggest that the extended scale, the GOS-E, suffers from the same misclassification bias.
have suggested a dichotomous outcome over the 5 GOS categories in an attempt to maximize the sensitivity of the scale. Murray and colleagues45
described a statistical methodology, the sliding dichotomy, that may improve the statistical power of the GOS. The sliding dichotomy is a statistical model-based procedure that uses information on patient’s baseline characteristics to predict that patient’s outcome. On the basis of this procedure, a favorable outcome is defined as better than would be expected, taking account of each individual patient’s baseline prognosis. The main advantage of this procedure is that response is tailored to individual patients rather than being fixed equal for all study participants. Therefore, patients with poor prognostic characteristics who are expected to make less progress will have a lower cutoff than patients with good prognostic variables. Compared with a procedure in which the cutoff is fixed and constant across severity groups and other prognostic factors, the sliding dichotomy procedure increases the chance that even severe patients reach a favorable outcome, thus increasing the power of the study.
Contrary to this approach, the Outcome Measures Subcommittee chose a fixed cutoff because it decided that it was important to determine whether an intervention works according to very general standards; for example, does the intervention increase the proportion of patients who are able to return to their previous life activities (eg, back to family and work). This type of outcome has a wider public health as well as social importance. An intervention that improves outcome from vegetative to severe disability status in a subset of the patient population, although of great importance to the individual, will not have the same impact in a more general setting. Moreover, the interpretation of favorable response according to the sliding dichotomy is not straightforward and not generalizable whereas the use of a fixed cutoff makes it easier to translate to a different or a more general population.
Another difficulty of the sliding dichotomy approach is that the methodology is model-based and thus requires the existence of an existing data set to build the model and determine the patient-specific cutoffs. Because it is not known whether the cutoffs determined in other studies, with a specific set of covariates, also apply to a newly designed study with possibly a different set of covariates, the only way to use this procedure is to wait until the end of the study and then use the collected data to construct a model and determine the cutoffs. This approach is obviously data driven and is subject to criticism as being post hoc. On the other hand, fixed cutoffs can be declared a priori and do not depend on any specific model.
When specifying fixed cutoffs some concern may arise regarding ceiling or floor effects that may result by including very impaired or very intact patients in the sample. If a large proportion of patients are too impaired to improve or are already intact enough to score above the cutoff at baseline, the power of the study may decrease. For the COBRIT trial, the Outcome Subcommittee considered that this would not be an issue since the trial does not cover the whole spectrum of TBI but rather focuses on a sample of patients who, according to published research, have injury to the brain substantial enough to avoid ceiling effects on the selected measures.
The approach proposed by the TBI-CT Network Outcome Subcommittee uses a dichotomized version of the GOS-E. However, because the outcome is measured by 8 additional scales the risk of loss of power and attenuation of the effect size due to misclassification is greatly minimized as compared with a study that relies on the GOS-E alone.
Another concern about the global test procedure pertains to the assumption of equal effect size across all measures. Although the global test procedure can be carried out even if not all outcomes have, in fact, the same effect size, power may be reduced if some outcomes depart significantly from the common effect size. However, if that happened, it would indicate an ambiguous situation, in which a treatment would be beneficial on some outcomes and inert or even harmful on others, a situation in which careful judgment would be required. Input from independent, objective sources may also prove useful in the evaluation of ambiguous results.
A final consideration concerns the use of the multiple outcome measures in relation to regulatory agencies. Regulatory approval in the form of an investigational new drug or a new drug approval by the Food and Drug Administration (FDA) is often required before a new drug is used in a clinical trial. The FDA has very clear guidelines about the design, sample size calculation, statistical analysis, and choice of the primary outcome. For trials addressing rehabilitation and treatment of TBI, the FDA has historically recommended the use of the GOS-E as the single primary outcome. The use of a global test procedure to simultaneously test several outcomes was proposed to the FDA and granted approval in the past. The TBI-CT Network was also successful in securing approval from the FDA to use the binary global test procedure for the Network’s first trial.
Measuring outcome in trials of patients with TBI presents several challenges. We believe that our choice of a global outcome approach for the COBRIT trial addresses many of these issues and will provide a robust and comprehensive estimate of the treatment effect.