PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
Int J Obes (Lond). Author manuscript; available in PMC 2012 December 1.
Published in final edited form as:
PMCID: PMC3130885
NIHMSID: NIHMS275922

Getting Carried Away: A Note Showing Baseline Observation Carried Forward (BOCF) Results Can be Calculated from Published Complete-Cases Results

Kathryn A. Kaiser, Ph.D.,1,3 Olivia Affuso, Ph.D.,2,3 T. Mark Beasley, Ph.D.,1 and David B. Allison, Ph.D.1,3

Abstract

Objective

Randomized controlled trials (RCTs) in obesity are plagued by missing data due to participant drop-outs. Most methodologists and regulatory bodies agree that the primary analysis of such RCTs should be based on the intent-to-treat (ITT) principle, such that all randomized subjects are included in the analysis, even those who dropped out. Unfortunately, some authors do not include an ITT analysis in their published reports. Here we show that one form of ITT analysis, baseline observation carried forward (BOCF), can be performed utilizing only information available in a published complete case (CC) analysis, permitting readers, editors, meta-analysts, and regulators to easily conduct their own ITT analyses when the original authors do not report one.

Method

We mathematically derive a simple method for estimating and testing treatment effects using the BOCF to allow a more conservative comparison of treatment effects when there are drop outs in a clinical trial. We provide two examples of this method using available CC analysis data from reported obesity trials to illustrate the application for readers who wish to determine a range of treatment effects based on published summary statistics.

Conclusion

Commonly used CC analyses may lead to inflated Type I error rates and/or treatment effect estimates. The method described herein can be useful for researchers who wish to estimate a conservative range of plausible treatment effects based on limited reported data. Limitations of this method are discussed.

Keywords: baseline observation carried forward, complete cases, obesity interventions, treatment effects, randomized controlled trials, intention-to-treat

INTRODUCTION

Randomized controlled trials (RCTs) are the gold standard for providing unbiased estimates and tests of the causal effects of obesity treatment and prevention strategies. Despite increased efforts to minimize the loss of participation before the study protocol is completed, drop outs remain a near-ubiquitous phenomenon (1). There are many statistical methods available to accommodate missing data and to use all available data to estimate and test treatment effects (2;3). While the intention-to-treat (ITT) principle is commonly accepted as an appropriate foundation for primary analyses, many researchers continue to use only a complete case (CC) analysis, which may introduce significant bias in tests and estimates of treatment effects when there are dropouts (2;3). In a 2010 analysis of obesity trials, CC analysis was the most common method used (34% of papers) (4). The prevalence of CC analysis creates a problem for anyone (e.g., readers, clinicians, consumers, regulatory agencies, meta-analysts) who wishes to rely on these papers to draw confident conclusions about the efficacy of the treatment studied.

There are several approaches to ITT analysis: most authors define it as an analysis including all cases who were randomized to a treatment arm, while a modified approach is to include only those cases who were randomized and received at least one exposure to the assigned treatment (5). One method of ITT analysis which has been recommended by some authors is baseline observation carried forward (BOCF) (6-8). BOCF is similar to the LOCF (Last Observation Carried Forward) in that the imputation is applied to participants who drop out of the study before the first post-baseline measure is taken. In LOCF, participants have the last value recorded carried forward depending on whether exposure to treatment is received. In BOCF, for any subject who does not have any post-baseline outcome (e.g., weight) measurement at the endpoint under study, one simply imputes the baseline value for that variable. This means that if change in the variable (e.g., weight loss or percent weight loss) is analyzed as the outcome, for all subjects who do not have a value at the study endpoint, one simply inserts the value as zero, for “no change”. Then the complete data file can be analyzed with conventional methods.

We are not advocating BOCF as a method in general. Single imputation methods, of which BOCF is one, have serious disadvantages (6;9) and BOCF in particular is strictly valid only under the assumption that all subjects who dropped out returned exactly to their baseline value on the outcome measure. BOCF is thus likely to be biased and highly conservative in practice. Nevertheless, when no other ITT analysis is available, BOCF can be used to place a plausible (though not definitive) lower bound point estimate on treatment effects (6).

In light of the frequently absent information from ITT analyses in published obesity RCTs, we provide a method for any reader, researcher, or regulatory agent to calculate a more conservative estimate of treatment effects for a published study using commonly available data supplied within most papers. We provide the mathematical derivation and show two examples from published studies that illustrate the application.

METHOD

Notation

equation M1 : The mean weight loss in the treatment group in a complete cases analysis.

equation M2 : The mean weight loss in the control group in a complete cases analysis.

πT : The proportion of subjects in the treatment group who finished the trial (i.e., were in the complete cases analysis).

πC : The proportion of subjects in the control group who finished the trial (i.e., were in the complete cases analysis).

NT : The number of subjects randomized to the treatment group.

NC : The number of subjects randomized to the control group.

equation M3: The estimated variance of the outcome measurement (e.g., weight change) in the treatment group in a complete cases analysis.

equation M4: The estimated variance of the outcome measurement in the control group in a complete cases analysis.

Derivation

It can then be shown that the mean weight loss in a BOCF analysis will be equation M5 in the treatment group and will be equation M6 in the control group, leading to a point estimate of the unstandardized treatment effect of equation M7 in a BOCF analysis. In deriving the variance of weight loss in the BOCF analysis, we note the within-group Sums of Squares for weight loss in the treatment group is equation M8 and likewise the within-group Sums of Squares for weight loss in the control group is equation M9. From this, the within-group Sums of Squares for weight loss in the treatment group in the BOCF analysis is equation M10 and the variance is equation M11. Similarly, within-group Sums of Squares in the control group in the BOCF analysis is equation M12 with a variance of equation M13. Then, the variance of equation M14 is equation M15 and the variance of equation M16 is equation M17. Further, equation M18 will be asymptotically distributed as t with NC + NT - 2 degrees of freedom.

When sample sizes are large (e.g., NC and NT > 200), a quick and easy way to compute an approximate variance of weight loss in the BOCF analysis is equation M19 in the treatment group and correspondingly, equation M20 in the control group. A symmetric, asymptotic 95% confidence interval for equation M21 may be obtained using equation M22, with equation M23 (the estimated standard error) obtained as:

equation M24

Illustration

We searched for recently published weight loss RCTs that reported a CC analysis to illustrate this method for BOCF analysis using reported data. Example 1 shows an outcome with BOCF that is similar to the originally reported CC analysis, strengthening confidence in the reported conclusion. In Example 2, the BOCF analysis reveals how even slight differences in dropout rates between groups can affect both the treatment estimate and its statistical significance, thus casting doubt on the original conclusion. Note that in each case, the BOCF increases the degrees of freedom compared to a CC analysis and therefore lowers the critical t value.

Example 1(10)

This study (10) evaluated the effects of conjugated linoleic acid supplementing the diets of overweight and obese participants on regional-specific fat mass over a six month trial. A total of 118 persons were randomized to a control (NC=59) or treatment group (NT=59). Dropouts were similar between groups at 18 and 17 persons, respectively. Therefore, πC = (41/59) = 0.695 and πT = (42/59) = 0.712. As can be seen in Table 1, with an observed treatment difference between the treatment and control groups of 1.5 kg favoring treatment, both methods result in similar interpretations of intervention effects [CC: t (81)= 2.024, p = 0.0462; BOCF: t (116)= 2.013, p = 0.0464]. Thus, the BOCF estimate provides more confidence in the conclusion provided using the CC analysis.

Table 1
Illustration of input data and results of reported complete cases analysis versus baseline observation carried forward analysis in two example weight loss studies

Example 2(11)

This study (11) examined the effects of eight weeks on a very low carbohydrate, high saturated fat diet compared to a high carbohydrate, low saturated fat diet on 107 participants with abdominal obesity. Participants were randomized between treatment (NC = 57) and control (NT = 50) and 5 dropped out of the treatment group while 3 dropped out of the control group. Therefore, πC = (47/50) = 0.940 and πT = (52/57) = 0.912. A raw treatment effect was 1.3 kg, favoring treatment. In contrast to example 1, a difference in the estimates of the treatment effects can be seen between the CC analysis and the BOCF analysis [CC: t(97) = 2.339, p = 0.0124 and BOCF: t(105)= 1.623, p = .1076]. This suggests that in this case, the BOCF result may call into question the published conclusion drawn using the CC analysis.

DISCUSSION

Our method has assumptions and limitations which should be noted. First, it simply reproduces what an ordinary BOCF analysis of raw data would produce and therefore ‘inherits’ all the limitations of BOCF - see (9) for discussion. Second, the t-statistic and confidence interval (CI) method derived are strictly valid in finite samples if all the Gauss-Markov assumptions are met (12), which includes normality of residuals. In BOCF, normality of residuals will almost certainly not hold and because the raw data are not available, in our approach, we cannot switch to a non-parametric analysis as one could with a raw data BOCF analysis. However, even if residuals are not normally distributed, the method is asymptotically valid and considering the typical sample sizes in obesity RCTs, non-normality is unlikely to be problematic.

The method we offer can easily be performed with an ordinary spreadsheet. While the BOCF approach seems likely to be extremely conservative in most situations, it can provide a useful lower boundary when attempting to determine a range of treatment effect estimates as compared to CC analyses, which are often likely to be at the upper bound. This proposed method will often reduce treatment effect estimates, especially when more participants drop out in the treatment group compared to the control group as seen in Example 2. One author has argued for the validity of the BOCF approach specifically with obesity interventions, due to evidence indicating that many persons regain most of the weight lost after many types of treatment approaches (7). Of course, it is not definitively known whether weight returns, on average, to baseline levels during the trial period for participants who drop out in any given trial. However, this highly conservative approach does offer an estimation of the treatment effects following the principles of ITT.

Acknowledgements

The authors thank Dr. Kyle Grimes for editorial comments on an earlier version of this manuscript. We also thank the anonymous reviewers for providing helpful suggestions to improve this manuscript.

Support: Supported in part by NIH grants R01DK078826, P30DK056336, and T32HL007457. The opinions expressed are those of the authors and not necessarily the NIH or any organization with which the authors are affiliated.

Footnotes

Disclosure: Dr. Allison has received grants, honoraria, donations, and consulting fees from numerous food, beverage, pharmaceutical companies, and other commercial and nonprofit entities with interests in obesity and randomized controlled trials.

Conflict of interest: The authors declare no conflict of interest.

References

1. Elobeid MA, Padilla MA, McVie T, Thomas O, Brock DW, Musser B, et al. Missing data in randomized clinical trials for weight loss: scope of the problem, state of the field, and performance of statistical methods. PLoS One. 2009;4/8:e6624. [PMC free article] [PubMed]
2. Gadbury GL, Coffey CS, Allison DB. Modern statistical methods for handling missing repeated measurements in obesity trial data: beyond LOCF. Obes Rev. 2003;4/3:175–184. [PubMed]
3. National Research Council The Prevention and Treatment of Missing Data in Clinical Trials. The National Academies Press; Washington, D.C.: 2010.
4. Cox T, Affuso O, Kaiser K, Ingram K, Robertson H, Allison D. Reporting quality of obesity randomized controlled trials: A preliminary review; Proceedings of the International Association for the Study of Obesity; International Congress on Obesity, Stockholm, Sweden. July 2010.
5. Moher D, Hopewell S, Schulz KF, Montori V, Gotzsche PC, Devereaux PJ, et al. CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. BMJ. 2010;340:c869. [PMC free article] [PubMed]
6. Shao J, Jordan DC, Pritchett YL. Baseline observation carry forward: reasoning, properties, and practical issues. J Biopharm Stat. 2009;19/4:672–684. [PubMed]
7. Ware JH. Interpreting incomplete data in studies of diet and weight loss. N Engl J Med. 2003;348/21:2136–2137. [PubMed]
8. Kenward MG, Molenberghs G. Last observation carried forward: a crystal ball? J Biopharm Stat. 2009;19/5:872–888. [PubMed]
9. Liu-Seifert H, Zhang S, D’Souza D, Skljarevski V. A closer look at the baseline-observation-carried-forward (BOCF) Patient Prefer Adherence. 2010;4:11–16. [PMC free article] [PubMed]
10. Gaullier JM, Halse J, Hoivik HO, Hoye K, Syvertsen C, Nurminiemi M, et al. Six months supplementation with conjugated linoleic acid induces regional-specific fat mass decreases in overweight and obese. Br J Nutr. 2007;97/3:550–560. [PubMed]
11. Keogh JB, Brinkworth GD, Noakes M, Belobrajdic DP, Buckley JD, Clifton PM. Effects of weight loss from a very-low-carbohydrate diet on endothelial function and markers of cardiovascular disease risk in subjects with abdominal obesity. Am J Clin Nutr. 2008;87/3:567–576. [PubMed]
12. Berry W. Understanding Regression Assumptions. Sage Publications; Thousand Oaks, CA: 1993.