PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of jcoHomeThis ArticleSearchSubmitASCO JCO Homepage
 
J Clin Oncol. 2009 May 1; 27(13): 2245–2252.
Published online 2009 March 23. doi:  10.1200/JCO.2008.16.2586
PMCID: PMC2674005

Interpreting Trial Results in Light of Conflicting Evidence: A Bayesian Analysis of Adjuvant Chemotherapy for Non–Small-Cell Lung Cancer

Abstract

Purpose

When successive randomized trials contradict prior evidence, clinicians may be unsure how to evaluate them: Does accumulating evidence warrant changing practice? An increasingly popular solution, Bayesian statistics quantitatively evaluate new results in context. This study provides a clinically relevant example of Bayesian methods.

Methods

Three recent non–small-cell lung cancer adjuvant chemotherapy trials were evaluated in light of prior conflicting data. Results were used from International Adjuvant Lung Trial (IALT), JBR.10, and Adjuvant Navelbine International Trialist Association (ANITA). Prior evidence was sequentially updated to calculate the probability of each survival benefit level (overall and by stage) and variance. Sensitivity analysis was performed using expert opinion and uninformed estimates of survival benefit prior probability.

Results

The probability of a 4% survival benefit increased from 33% before IALT to 64% after IALT. After sequential updating with JBR.10 and ANITA, this probability was 82% (hazard ratio = 0.84; 95% CI, 0.77 to 0.91). IALT produced the largest decrease in variance (61%) and decreased the chance of survival decrement to 0%. Sensitivity analysis did not support a survival benefit after IALT. However, sequential updating substantiated a 4% survival benefit and, for stage II and III, more than 90% probability of a 6% benefit and 50% probability of a 12% benefit.

Conclusion

When evaluated in context with prior data, IALT did not support a 4% survival benefit. However, sequential updating with JBR.10 and ANITA did. A model for future assessments, this study demonstrates the unique ability of Bayesian analysis to evaluate results that contradict prior evidence.

INTRODUCTION

When pivotal trial results are published, one of the primary issues is whether the totality of accumulating scientific evidence is strong enough to support changing clinical practice. However, it is often unclear how to sequentially evaluate results from successive, and sometimes conflicting, trials. For example, before publication of the International Adjuvant Lung Trial (IALT) in 2004, no survival benefit had been found for adjuvant cisplatin-based chemotherapy in non–small-cell lung cancer (NSCLC): A 1995 meta-analysis found no statistically significant benefit,1 subsequent trials were not positive2,3 and the 2001 lung cancer consensus panel declared adjuvant chemotherapy unproven and experimental.4,5 Therefore, the positive results of the IALT were controversial.68 Subsequently, two similarly designed adjuvant chemotherapy trials published positive results: the National Cancer Institute of Canada Clinical Trials Group (JBR.10) in 20059 and the Adjuvant Navelbine International Trialist Association (ANITA) in 200610 (Table 1). However, the magnitude of the absolute 5-year overall survival (OS) benefit varied: 4.1%, 15%, and 8.6%, respectively for IALT, JBR.10, and ANITA. In light of this variability, earlier negative evidence, and the preliminary negative results for adjuvant carboplatin (2007),11 clinicians may wonder how best to evaluate the conflicting evidence: Does adjuvant chemotherapy produce an OS advantage for NSCLC for each stage? And, if so, what is the magnitude of this benefit?

Table 1.
Trial Data Used in Bayesian Sequential Updating Analysis for Cisplatin-Based Adjuvant Chemotherapy for NSCLC

The task of integrating unexpected or conflicting results with prior evidence is often left to the clinician. Although discussions in published articles may qualitatively place trial findings within the context of available evidence, the crucial question is frequently not answered: What is the probability that the new result is correct? And how much does the study contribute?1217 Unfortunately, classical methods for iterative data incorporation are limited: (1) successive qualitative evaluations by individual clinicians may lead to heterogeneous conclusions, (2) appraisal by expert panels may lack transparency and timeliness, and (3) meta-analysis techniques generally evaluate a static set of trials, do not recognize time order, cannot incorporate diverse types of evidence, and are cumbersome to update.1820 For instance, systematic reviews of adjuvant chemotherapy for NSCLC do not describe the relative contribution of each study nor the probability that the survival benefits observed in each trial are true.21,22

The scientific dilemma is how to sequentially evaluate and quantitatively integrate new trials with prior evidence to make informed decisions. Bayesian statistics is one solution that mathematically updates prior evidence with new data in a dynamic process.16,17,2325 Bayesian methods are used in biostatistics, astrophysics, and genomics to quantify the reliability of results, to sharpen the assessment of risk, and to determine the amount of information contributed by a study.2531 These features facilitate sequential evaluation of trials. Furthermore, Bayesian approaches explicitly and quantitatively describe the data synthesis process, enhancing transparency, accuracy, and reproducibility.31a,31b,31c,31d

Bayesian techniques are emerging in oncology research32 and have been used for trial design,3336 monitoring35,3743 and data evaluation,33,4447 pharmacokinetic evaluation,4850 prediction of phase III success,51 and models of cancer risk,5257 recurrence,5860 and mortality.57,61,62 Clinicians frequently use Bayesian approaches during iterative clinical assessments to update the probability of a diagnosis. For example, a physician evaluating a 60-year-old patient with a cough may consider lung cancer more likely if a history of hemoptysis is later elicited. This “clinical judgment” can be formalized by calculating the positive predictive value (PPV) of lung cancer, as is commonly done for imaging studies.28,42 The PPV calculation is derived from Bayes Theorem, the mathematical backbone of Bayesian statistics.28,42 Similarly, Bayesian analysis can determine the PPV of a trial.

In contrast to classical statistical approaches, the results of Bayesian analysis are directly interpreted as the probability that a therapy produces a survival advantage,16 the finding most relevant to clinicians. Therefore, Bayesian analysis may be helpful in evaluating whether new results should change clinical practice and the focus of future clinical and basic science investigations.

Although cardiology trials have been appraised with Bayesian tools,15,28 to our knowledge, this approach has not been applied to oncologic therapeutic trials and has not been used to evaluate successively published trials. We sequentially updated prior evidence with the results of IALT, JBR.10, and ANITA using Bayesian methods to (1) establish the magnitude of the OS benefit, (2) determine the amount of information contributed by each trial, and (3) explore the OS benefit supported for each stage. Also, we conducted an expert opinion survey and performed a sensitivity analysis on the prior probability of an OS benefit.

METHODS

Methodologic Approach

Bayes Theorem determines the probability (P) of an outcome (θ) given (|) new data (X)16,23:

equation image

Therefore, the PPV for lung cancer, P(θ|X) depends on (1) the pretest probability of lung cancer (P(θ)) given the patient's symptoms, (2) a test result (X), and (3) the strength of the test result (P(X|θ)) as determined by the sensitivity and specificity.16,23

In the current analysis, the updated probability (P(θ|X)) of a 5-year OS benefit for adjuvant chemotherapy (θ) was computed from the hazard ratio (HR) found in IALT (X) and the pre-IALT survival benefit data P(θ) (Fig 1 and Appendix, online only). The distribution of X and θ was obtained from the literature, such that P(X|θ) was the probability of the results of IALT given the existence or absence of an OS benefit. As additional trials were published, the probability of an OS benefit was updated, first with the results of JBR.10 and then ANITA.

Fig 1.
Flow diagram of method for updating the probability of a survival benefit. IALT, International Adjuvant Lung Trial; ANITA, Adjuvant Navelbine International Trialist Association.

Establishing the Probability of a Survival Benefit Before the IALT

The probability of an OS benefit for cisplatin-based adjuvant chemotherapy for NSCLC before the publication of the IALT was established through a systematic literature review. MEDLINE was searched from January 1965 through January 2004 using the terms “lung cancer,” “NSCLC,” and “chemotherapy” (adjuvant, cisplatin-based, or postoperative) and expanded by reviewing the references in these articles.

We identified the 1995 Non–Small Cell Lung Cancer Collaborative Group meta-analysis as the most comprehensive published quantitative summary of the data before the initiation of IALT. This meta-analysis did not show a statistically significant OS benefit for cisplatin-based adjuvant chemotherapy: the HR for death at 5 years was 0.87 (P = .08) and the absolute OS benefit was 5% (95% CI, 1% to 10%)(1) (Table 1).

Deriving the Prior Probability Curve

For the primary analysis (base case), uncertainty regarding the probability of an OS benefit derived from the meta-analysis was represented by a probability distribution curve constructed by converting the 5-year survival probability to the exponential distribution of the logarithm of the HR. The standard deviation (SD) was inferred from the CI.

The null hypothesis was the same as in the IALT: no OS benefit for adjuvant chemotherapy. Because there was uncertainty before IALT, and equipoise should exist to ethically initiate a trial,67 the prior probability of the null was set at 50% (half the weight of the curve lay at zero; see Appendix, online only). The base case prior probability curve represents the probability of each survival benefit level being true given the meta-analysis result (solid line, Fig 2A and Table 2).

Fig 2.
(A) Prior probability of a survival benefit before IALT. x-axis = 5-year survival benefit level. y-axis = cumulative probability of the survival benefit level. (B) Updated probability of survival benefit after IALT. (C, D) Sequentially updated probability ...
Table 2.
Probability of a Survival Benefit for Cisplatin-Based Adjuvant Chemotherapy for NSCLC Before and After Sequential Updating: Base Case And Sensitivity Analysis

Updating Prior Knowledge With the Results of the IALT, JBR.10, and ANITA

Using Bayes Theorem, the base case prior probability curve was updated with the logarithm of the HR for OS found in IALT (HR = 0.86; P < .03). This computation yielded the posterior probability of survival benefit after IALT. As subsequent trials were published, the probability of a survival benefit after IALT was similarly updated with JBR.10 (HR = 0.69; P = .04), and this new result was then updated with ANITA (HR = 0.80; P = .017; see Appendix, online only).

The Q statistic was calculated and assessed at the P < .05 level to evaluate statistical heterogeneity of the trial results for survival benefit (Appendix, online only).68 Calculations were performed in SAS version 9.1 (SAS Institute Inc, Cary, NC) and the code is available on request.

Sensitivity Analysis

Sensitivity analysis was performed to evaluate the effect of different approaches to the synthesis of prior evidence.28,6972 The analysis was repeated with two approaches described in the literature that capture a range of possible prior probability curves: (1) expert opinion curve based on a survey of thoracic oncologists, and (2) modified uninformed curve reflecting pessimism and uncertainty about survival benefit (Fig 2).

Expert opinion curve.69,70

Thoracic oncologists attending the International Novel Agents in the Treatment of Lung Cancer Conference in Cambridge, MA, were surveyed in September 2003. Opinions were elicited73 about the 5-year OS benefit of cisplatin-based adjuvant chemotherapy before and after the results of the IALT were presented at the American Society of Clinical Oncology Annual Meeting in June 2003.74 The respondents' ability to discriminate changes in opinion over time was assessed.33,35

A distribution curve for the probability of a survival advantage before the presentation of IALT was constructed by converting survival estimates to HRs (exponential distribution). The null hypothesis was no survival benefit and equipoise was assumed (Appendix, online only).

Modified uninformed curve.

A prior probability curve reflecting uncertainty about the magnitude of survival benefit and pessimism about prior data was created to accommodate negative trials reported after the meta-analysis and potential limitations of the base case.2,7 This modified uninformed curve was developed by adapting two approaches described in the literature25: (1) a noninformative prior (horizontal line), indicating complete uncertainty, and (2) a downweighted base case, reflecting pessimism.

The modified uninformed curve was symmetric around zero (equal likelihood of benefit and harm). With a variance 75 times higher than the base case, the modified uninformed curve reflected extreme uncertainty and pessimism while avoiding substantial weight placed on improbable results. The same null hypothesis was assumed.

RESULTS

Prior Probability Curves

With a mean of 0.15% and SD of 0.08%, the base case prior probability curve had a higher probability of all positive survival benefit levels than the expert opinion curve (mean = 0.10%; SD = 0.09%). The modified uninformed curve was close to a straight line, with a 15.4% probability of survival decrement and about 30% probability for each positive survival benefit level less than 7% (Fig 2A and Table 2).

Base Case Analysis

After sequentially updating with IALT, JBR.10, and ANITA, the base case curve shifted toward a higher cumulative probability of survival benefit (Figs 2A through through2D).2D). The Q statistic across all three trials (1.48; P = .85) was not statistically significant, suggesting it was appropriate to analyze the trials in combination.

For the base case, the probability of any OS benefit increased from 48% before IALT to 86% after IALT, and 89% after sequential updating with JBR.10 and ANITA (Table 2). The probability of a 5-year 4% OS benefit increased from 33% before IALT to 64% after IALT, and 82% after sequential updating. The probability of a survival decrement was less than 1% after IALT.

The HR was higher after sequential updating than when JBR.10 or ANITA were considered in isolation, but lower than for IALT (Fig 3). Uncertainty about the HR decreased after sequential updating and the 95% CI narrowed. IALT contributed the most information to the final HR estimate: the variance decreased by 61% after IALT (0.0025) and an additional 28% after JBR.10 and ANITA (0.0018; see Appendix, online only).

Fig 3.
Hazard ratio for death before and after updating with base case (gold), expert opinion (blue), and modified uninformed curves (gray).

Sensitivity Analysis

Expert opinion survey results.

With a response rate of 85% (n = 17), respondents had practiced a median of 11 years (interquartile range, 5 to 17; 85% North American, 15% European). Before IALT, 16.7% estimated an OS benefit for adjuvant chemotherapy (mean benefit = 2.9%; SD = 3%) and 11.1% offered it to patients. After the results of the IALT were presented, 72% estimated an OS benefit (mean benefit = 4.5%; SD = 0.51%).

Sequential updating sensitivity analysis.

The probability of any survival benefit was robust to the choice of the prior probability curve: after each updating, the curves move closer together (Fig 2). After sequential updating for the expert opinion (uninformed) curve, the probability of any survival benefit increased from 43% (35%) before IALT to 85% (61%) after IALT and 89% (88%). After IALT, the chance of a survival decrement remained less than 1% (Table 2).

In contrast to the probability of OS benefit, the probability of specific OS benefit levels did not consistently increase to more than 50% until the sequential updating of all three trials. For example, the probability of a 5-year 4% absolute OS benefit increased from 21% (uninformed curve, 30%) before IALT to 43% (uninformed curve, 52%) after IALT and to 77% (uninformed curve, 81%) after sequential updating for the expert opinion curve. Similar to the base case, sequential updating produced a precise HR for death, and IALT decreased the variance the most: by 65% (uninformed curve, 83%) after IALT (0.0028 [uninformed curve, 0.0020]) and an additional 29% (uninformed curve, 38%) after sequential updating (0.0020 [uninformed curve, 0.0026]) for the expert opinion curve.

Stage analysis.

Exploratory subgroup analysis using stage-specific priors from meta-analysis results suggested that even after sequential updating, the probability of any OS benefit for stage I NSCLC never exceeded 37% and the maximal OS benefit was 2% (Table 3). However, for stage II and III NSCLC, there was a more than 90% probability of a 6% survival benefit and a 50% probability of a 7% survival benefit.

Table 3.
Base Case (meta-analysis) Stage-Specific Probability of a Survival Benefit for Cisplatin-Based Adjuvant Chemotherapy for NSCLC Before and After Sequential Updating: Base Case by Stage

DISCUSSION

To our knowledge, this study is the first Bayesian analysis to quantitatively evaluate the results of sequentially published oncology therapeutics trials in context with prior conflicting evidence. Our analysis supports a survival benefit for cisplatin-based adjuvant chemotherapy for resected NSCLC, consistent with other meta-analyses.21,22,75 Uniquely, we show that although IALT contributed the most information, it was not sufficient to establish an OS benefit. The HR for death after sequential updating was confirmed by sensitivity analysis and was higher than JBR.10 or ANITA in isolation and lower than IALT in isolation. In addition, our results suggest there is a low probability of a clinically meaningful survival benefit for stage I NSCLC. Furthermore, exploratory analysis suggests the 5-year absolute OS benefit for stage II and III may be as high as, but is not likely to exceed, 7%.

The need for additional trials to substantiate the existence of a survival benefit is consistent with the controversy engendered by the publication of the IALT. The modest OS benefit found in our study is supported by the negative preliminary analysis of carboplatin-based adjuvant chemotherapy for stage IB NSCLC in the Cancer and Leukemia Group B protocol 963.76 This finding emphasizes the need to consider the positive cisplatin-based chemotherapy trials in context with prior negative results.

Our study illustrates that Bayesian methods provide unique and complimentary information to other types of analyses. In contrast to frequentist methods, which calculate a P value for the probability of the trial result given no survival benefit, Bayesian analysis determines the probability of a survival benefit given the trial result, the relevant information for clinical decision making.77,78 In addition, Bayesian methods provide the probability of each survival benefit level as opposed to the point estimate and CI produced by frequentist analysis. Furthermore, Bayesian approaches combine different types of evidence, borrowing strength from each to comprehensively evaluate a clinical question.77,78 Finally, our Bayesian analysis quantitatively integrated data over time, mirroring the evaluation process performed by clinicians as new data are published.

Our findings must be considered within the limitations of the study. Although all three trials evaluated were randomized, controlled trials and tested the same hypothesis, there were differences between the trials, and summary statistics were used. Notably, JBR.10 evaluated stages I to II, whereas IALT and ANITA assessed stages I to III. Other differences were cisplatin dose, couplet choice, and radiation use.79 Therefore, we evaluated this potential bias. First, the Q statistic was not significant (ie, the null hypothesis of homogeneity was not rejected). Second, the variance of the posterior was smaller than the variance of the prior, also indicating lack of statistical heterogeneity. Additionally, prior work suggests minimal information may be lost when summary statistics are based on large studies.80 Therefore, we believe potential bias owing to dissimilar populations is minimal. (For further discussion, see Appendix, online only).

We acknowledge that the use of a prior probability distribution in Bayesian analysis has been criticized as subjective.81 Therefore, we used three approaches to control for potential biases: (1) the base case prior probability distribution was derived from data published before the initiation of the IALT, (2) a sensitivity analysis was performed, and (3) a modified uninformed curve was constructed to reflect extreme pessimism and uncertainty. The sensitivity analysis explicitly allowed a range of interpretations of the prior evidence to be formally expressed and evaluated.28 In addition, the assumption of equipoise reduced the risk that a priori knowledge unduly influenced the final result. Finally, although the expert opinion survey was administered after IALT was presented, the full study had not been published,6 and the effect of timing appeared minimal: responses were biased against a positive benefit.

With the initial analysis undertaken at a time of great uncertainty, our results demonstrate that Bayesian analysis can highlight sources of the controversy: when considered in context with evidence available at the time of publication, the IALT did not support a 4% survival benefit, even though it did in isolation. Therefore, additional evidence was needed to strengthen the results. This need for additional data suggests that the evidence against the null hypothesis may be weaker than implied by the P value (ie, the P value can be much smaller than the posterior probability for the same hypothesis).78,82 Therefore, Bayesian methods increase the quantitative rigor of trial evaluation and may decrease the number of false results accepted by the oncology community. Potential applications include clarifying the role of breast magnetic resonance imaging83,84 and adjuvant chemotherapy for stage II colon cancer.85,86 Bayesian analysis may be particularly important when new findings may change clinical practice, the comparator arm of future trials, or the focus of basic science research. This study offers a potential model for evaluating future studies whose results contradict prior evidence.

Supplementary Material

[Data Supplement]

Acknowledgment

We thank Milton Weinstein, PhD, Ralph Nachman, MD, John Roseman, MD, and Donald Halstead for their contributions to previous versions of this manuscript.

Appendix

Methodologic Details

Given the large sample size of each trial and the fact that individual level data were not available, the hazard ratio (HR) for each trial was assumed to follow a lognormal distribution (equivalently, the logarithm of the HR follows a normal distribution), and all calculations were based on HR. The mean and the standard deviation (SD) of the HR for each trial was inferred from the reported CIs using one of the following equations, which describe the lower (LL) and upper (UL) limits of the CIs for the HR:

equation image

equation image

For ease of clinical interpretation, results are presented as HRs and as 5-year survival increments. Because the exponential distribution has a single parameter (death rate), it is possible to convert the HR to 5-year survival probabilities without requiring additional information or making further assumptions.

We weighted trials with more precise information (ie, less variability) such that they contributed more information to the analysis. To achieve this weighting, we derived the variance of the probability distribution for the HR from the SE. This approach is similar to weighting the HR inversely proportional to the SEs.

Formulating the Prior Probability Curve

The base case prior probability curve formulations follow the description in Methodologic Details above: the log(HR) of the prior was assumed to have a normal distribution with the mean equal to the mean log(HR) of the meta-analysis and the SD inferred from the CIs of the meta-analysis. Conversion from HR to 5-year survival probabilities was performed using the exponential distribution.

Given that the null hypothesis (θ) of the adjuvant chemotherapy trials (and this Bayesian analysis) was “no survival difference between the treatment and control groups” (θ = 0), modification to Bayes formula was required. A continuous prior for θ implies a probability of zero for θ = 0, the posterior probability (after updating) of θ = 0 will also be zero. For this reason, we chose a mixture prior which allocates a portion (π0) of the prior probability to the null hypothesis (θ = 0) and distributes the rest to all the possible values of HR according to a lognormal distribution, an approach extensively studied in the literature [Berger JO: Statistical Decision Theory and Bayesian Analysis (ed 2). New York, NY, Springer-Verlag, 1985]. Because equipoise is generally required to ethically initiate a trial and there was uncertainty in the literature about the survival benefit of adjuvant chemotherapy, the probability of the null hypothesis (π0) was assumed to be 50%. With 50% of the weight at 0%, the remaining probability (1−π0,) was distributed over the real line using a continuous prior, f(θ). Prior information about θ is summarized as:

equation image

Updating With Trial Results

Conversion of trial results into the logarithm of the HR.

In large samples, the distribution of the HR remains skewed, whereas the logarithm of the HR can be approximated by a normal distribution. Therefore, we assigned θ to be logarithm of the HR with a normal distribution, an approach successfully used by Faraggi and Simon (Faraggi D, Simon R: Stat Med 16:2573-2585, 1997). Therefore,

equation image

and [theta w/ hat] is used for X (the “new data”) in the analysis. Bayes theorem now appears as:

equation image

Bayesian calculation.

Using Bayes theorem, the above prior using π0 and f(θ) translate into the following posterior distribution:

equation image

In this expression, p0 is the posterior probability of the null hypothesis and is given by:

equation image

where Bayes Factor (BF) is a common Bayesian summary statistic for hypothesis testing. BF is calculated as:

equation image

Using the above formulas, BF and then p0 were calculated (Kass RE: J Am Stat Assoc 90:773-795, 1995). In addition, the entire prior and posterior probability functions (equations 3 and 6, respectively) were used to plot the probability of benefit.

We considered γ to be known and fixed. Because it is difficult to sequentially update γ in the Bayesian sequential updating framework without access to the full data, our decision was dictated the type of data available.

Sensitivity Analysis Prior Probability Curves

Expert opinion prior survey.

Survey respondents were asked: “Before the American Society of Clinical Oncology 2003 meeting, what would have been your best estimate of the effect of cisplatin-based adjuvant chemotherapy on overall 5-year survival (for patients with resected non–small-cell lung cancer in absolute percentage)?” In addition, four separate questions ascertained the probability of each specific survival benefit level: “Before the American Society of Clinical Oncology 2003 annual meeting, how would you have assessed the chances of any (a 3%, a 5%, or a negative) 5-year survival benefit for patients receiving cisplatin-based adjuvant chemotherapy (non–small-cell lung cancer)?”

Construction of the expert opinion prior probability curve.

The mean response for each specific survival benefit and for the estimated average absolute benefit was calculated. A cumulative normal distribution was fitted to these means and the distribution was obtained by minimizing the sum of squares between the fitted line and the observed means using the OPTIM function in R (http://www.cran.r-project.org).

Evaluation of Heterogeneity Between Trials

The Q statistic was calculated and evaluated at the P = .05 level as described in the text. To further assess for trial heterogeneity at each sequential update, we compared the variance of the posterior with the variance of the prior. In the setting of sequential updating, heterogeneity between trials results in a posterior variance that is larger than the prior variance. In our analysis, the posterior variance was smaller than the prior variance at each update. Therefore, we considered the level of trial homogeneity to be acceptable within the Bayesian framework.

It is plausible that future applications of Bayesian sequential analysis may include situations with strong evidence against homogeneity. Potential strategies to address this issue are discussed in the Bayesian meta-analysis literature, and different situations may require unique solutions (McMahon PM, Zaslavsky AM, Weinstein MC, et al: Med Decis Making 26:497-511, 2006; Sutton AJ, Abrams KR: Stat Methods Med Res 10:277-303, 2001; Sutton AJ: Methods for Meta-Analysis in Medical Research. Chichester, NY, J. Wiley, 2000; White IR, Higgins JP, Wood AM: Stat Med 27:711-727, 2008; Conlon EM: Funct Integr Genomics 8:43-53, 2008; Schmidt FL, Raju NS: J Appl Psychol 92:297-308, 2007; Prevost AT, Mason D, Griffin S, et al: Psychol Methods 12:434-450, 2007; Baker R, Jackson D: Health Care Manag Sci 11:121-131, 2008; Brannick MT: J Appl Psychol 86:468-480, 2001). As a potential advantage of a Bayesian meta-analysis, covariates can be investigated for sources of heterogeneity (Dixon DO, Simon R: Stat Med 11:13-22, 1992; Sutton AJ, Kendrick D, Coupland CA: Stat Med 27:651-669, 2008; Nam IS, Mengersen K, Garthwaite P: Stat Med 22:2309-2333, 2003; Warn DE, Thompson SG, Spiegelhalter DJ: Stat Med 21:1601-1623, 2002; Nixon RM, Thompson SG: Health Econ 14:1217-1229, 2005).

Contribution of the Prior

One commonly used method to measure the contribution of the prior to the final results is to calculate the ratio of the prior precision to the posterior precision (precision is defined as the reciprocal of the variance). A high value for this ratio indicates a high contribution from the prior (Gelman A: Bayesian Data Analysis (ed 2). Boca Raton, FL, Chapman & Hall/CRC, 2004). For our analysis, this ratio was 0.29 for the base case, 0.33 for the expert opinion, and 0.03 for the skeptical prior. Although there are no established thresholds against which to compare these ratios, they all point to the strong influence of the data on the posterior. This finding supplements our conclusion that our analysis was relatively robust to the choice of prior.

Footnotes

Supported in part by National Cancer Institute Grant No. R25T CA 92203 (R.A.M. and T.G.R.). T.G.R. was also supported in part by an unrestricted Health Outcomes Starter grant from the PhRMA Foundation.

Presented in part at the 28th Annual Meeting of the Society of Medical Decision Making Annual Meeting, October 15-18, 2006, Boston, MA; and at the 40th Annual Meeting of the American Society of Clinical Oncology, June 5-8, 2004, New Orleans, LA.

Authors' disclosures of potential conflicts of interest and author contributions are found at the end of this article.

AUTHORS' DISCLOSURES OF POTENTIAL CONFLICTS OF INTEREST

Although all authors completed the disclosure declaration, the following author(s) indicated a financial or other interest that is relevant to the subject matter under consideration in this article. Certain relationships marked with a “U” are those for which no compensation was received; those relationships marked with a “C” were compensated. For a detailed description of the disclosure categories, or for more information about ASCO's conflict of interest policy, please refer to the Author Disclosure Declaration and the Disclosures of Potential Conflicts of Interest section in Information for Contributors.

Employment or Leadership Position: Thomas G. Roberts Jr, Noonday Asset Management, L.P (C) Consultant or Advisory Role: Thomas J. Lynch, Astra Zeneca (C), Genentech (C), OSI (C), Roche (C), Chugai (C), Merk-Sorono (C), Millenium (C), Sanofi (C), Lilly (C), Xelixis (C) Stock Ownership: None Honoraria: None Research Funding: None Expert Testimony: None Other Remuneration: Thomas J. Lynch, Genzyme, Inc

AUTHOR CONTRIBUTIONS

Conception and design: Rebecca A. Miksad, Mithat Gönen, Thomas J. Lynch, Thomas G. Roberts Jr

Administrative support: Rebecca A. Miksad

Provision of study materials or patients: Rebecca A. Miksad, Thomas G. Roberts Jr

Collection and assembly of data: Rebecca A. Miksad, Thomas G. Roberts Jr

Data analysis and interpretation: Rebecca A. Miksad, Mithat Gönen, Thomas J. Lynch, Thomas G. Roberts Jr

Manuscript writing: Rebecca A. Miksad, Mithat Gönen, Thomas G. Roberts Jr

Final approval of manuscript: Rebecca A. Miksad, Mithat Gönen, Thomas J. Lynch, Thomas G. Roberts Jr

REFERENCES

1. Chemotherapy in non-small cell lung cancer. A meta-analysis using updated data on individual patients from 52 randomised clinical trials—Non-Small Cell Lung Cancer Collaborative Group. BMJ. 1995;311:899–909. [PMC free article] [PubMed]
2. Scagliotti GV, Fossati R, Torri V, et al. Randomized study of adjuvant chemotherapy for completely resected stage I, II, or IIIA non-small-cell Lung cancer. J Natl Cancer Inst. 2003;95:1453–1461. [PubMed]
3. Waller D, Peake MD, Stephens RJ, et al. Chemotherapy for patients with non-small cell lung cancer: The surgical setting of the Big Lung Trial. Eur J Cardiothorac Surg. 2004;26:173–182. [PubMed]
4. Ihde D, Ball D, Arriagada R, et al. Postoperative adjuvant therapy for non-small cell lung cancer: A consensus report. Lung Cancer. 1994;11(suppl 3):S15–S17. [PubMed]
5. Tonato M. Consensus conference on medical treatment of non-small cell lung cancer: Adjuvant treatment. Lung Cancer. 2002;38(suppl 3):S37–S42. [PubMed]
6. Arriagada R, Bergman B, Dunant A, et al. Cisplatin-based adjuvant chemotherapy in patients with completely resected non-small-cell lung cancer. N Engl J Med. 2004;350:351–360. [PubMed]
7. Johnson BE. Adjuvant chemotherapy for non-small-cell lung cancer: The end of the beginning. J Natl Cancer Inst. 2003;95:1422–1424. [PubMed]
8. Le Chevalier T, Lynch T. Adjuvant treatment of lung cancer: Current status and potential applications of new regimens. Lung Cancer. 2004;46(suppl 2):S33–S39. [PubMed]
9. Winton T, Livingston R, Johnson D, et al. Vinorelbine plus cisplatin vs. observation in resected non-small-cell lung cancer. N Engl J Med. 2005;352:2589–2597. [PubMed]
10. Douillard J-Y, Rosell R, Delena M, et al. ANITA: Phase III adjuvant vinorelbine (N) and cisplatin (P) versus observation (OBS) in completely resected (stage I-III) non-small cell lung cancer (NSCLC) patients (pts): Final results after 70-month median follow-up. J Clin Oncol. 2005;23(suppl):624s. abstr 7013.
11. Strauss GM, Herndon J, Maddaus MA, et al. Randomized clinical trial of adjuvant chemotherapy with paclitaxel and carboplatin following resection in Stage IB non-small cell lung cancer (NSCLC): Report of Cancer and Leukemia Group B (CALGB) Protocol 9633. J Clin Oncol. 2004;22(suppl):621s. abstr 7019.
12. Clarke M, Chalmers I. Discussion sections in reports of controlled trials published in general medical journals: Islands in search of continents? JAMA. 1998;280:280–282. [PubMed]
13. Moher D, Jones A, Lepage L. Use of the CONSORT statement and quality of reports of randomized trials: A comparative before-and-after evaluation. JAMA. 2001;285:1992–1995. [PubMed]
14. Moher D, Schulz KF, Altman D. The CONSORT statement: Revised recommendations for improving the quality of reports of parallel-group randomized trials. JAMA. 2001;285:1987–1991. [PubMed]
15. Goodman S, Langer A. Bayesian analysis and the GUSTO trial: Global Utilization of Streptokinase and Tissue Plasminogen Activator in Occluded Arteries. JAMA. 274:873. author reply 874, 1995. [PubMed]
16. Goodman SN. Toward evidence-based medical statistics: 2. The Bayes factor. Ann Intern Med. 1999;130:1005–1013. [PubMed]
17. Goodman SN. Toward evidence-based medical statistics: 1. The P value fallacy. Ann Intern Med. 1999;130:995–1004. [PubMed]
18. Demets DL. Methods for combining randomized clinical trials: Strengths and limitations. Stat Med. 1987;6:341–350. [PubMed]
19. Thompson SG, Pocock SJ. Can meta-analyses be trusted? Lancet. 1991;338:1127–1130. [PubMed]
20. Chambers I. Improving the quality and dissemination of review of clinical research. In: Lock SP, editor. The Future of Medical Journals: In Commemoration of 150 Years of the BMJ. London, United Kingdom: British Society; 1991.
21. Alam N, Darling G, Evans WK, et al. Adjuvant chemotherapy for completely resected non-small cell lung cancer: A systematic review. Crit Rev Oncol Hematol. 2006;58:146–155. [PubMed]
22. Bria E, Cuppone F, Cecere FL, et al. Adjuvant chemotherapy for non-small cell lung cancer. J Thorac Oncol. 2007;2:S7–S11. [PubMed]
23. Hornberger J. Introduction to Bayesian reasoning. Int J Technol Assess Health Care. 2001;17:9–16. [PubMed]
24. Browner WS, Newman TB. Are all significant P values created equal? The analogy between diagnostic tests and clinical research. JAMA. 1987;257:2459–2463. [PubMed]
25. Berger JO. Statistical Decision Theory and Bayesian Analysis. ed 2. New York, NY: Springer-Verlag; 1985.
26. Malakoff D. Bayes offers a ‘new’ way to make sense of numbers. Science. 1999;286:1460–1464. [PubMed]
27. Daw EW, Wijsman EM, Thompson EA. A score for Bayesian genome screening. Genet Epidemiol. 2003;24:181–190. [PubMed]
28. Brophy JM, Joseph L. Placing trials in context using Bayesian analysis: GUSTO revisited by Reverend Bayes. JAMA. 1995;273:871–875. [PubMed]
29. Otto CM. Textbook of Clinical Echocardiography. ed 2. Philadelphia, PA: W.B. Saunders; 1999.
30. Pocock SJ, Spiegelhalter DJ. Domiciliary thrombolysis by general practitioners. BMJ. 1992;305:1015. [PMC free article] [PubMed]
31. Maurer DE, Schirmer RW, Kalandros MK, et al. Sensor fusion architectures for ballistic missile defense. Johns Hopkins APL Technical Digest. 2006;27:19–31.
31a. Hung HM, O'Neill RT, Wang SJ, et al. A regulatory view on adaptive/flexible clinical trial design. Biom J. 2006;48:565–573. [PubMed]
31b. Sheingold SH. Can Bayesian methods make data and analyses more relevant to decision makers? A perspective from Medicare. Int J Technol Assess Health Care. 2001;17:114–122. [PubMed]
31c. Berry DA. Clinical trials: Is the Bayesian approach ready for print time? Yes! Stroke. 2005;36:1621–1622. [PubMed]
31d. Berry DA. The Bayesian Principle: Can we adapt? Stroke. 2005;36:1623–1624. [PubMed]
32. Beckman M. Are Bayes' days upon us? Statistical methods could change the conduct of clinical trials. J Natl Cancer Inst. 2006;98:1512–1513. [PubMed]
33. Tan SB, Chung YF, Tai BC, et al. Elicitation of prior distributions for a phase III randomized controlled trial of adjuvant therapy with surgery for hepatocellular carcinoma. Control Clin Trials. 2003;24:110–121. [PubMed]
34. Morita S, Sakamoto J. Application of an adaptive design to a randomized phase II selection trial in gastric cancer: A report of the study design. Pharm Stat. 2006;5:109–118. [PubMed]
35. Parmar MK, Spiegelhalter DJ, Freedman LS. The CHART trials: Bayesian design and monitoring in practice—CHART Steering Committee. Stat Med. 1994;13:1297–1312. [PubMed]
36. Greenhouse JB. On some applications of Bayesian methods in cancer clinical trials. Stat Med. 1992;11:37–53. [PubMed]
37. Ashby D, Tan SB. Where's the utility in Bayesian data-monitoring of clinical trials? Clin Trials. 2005;2:197–205. discussion 205-208. [PubMed]
38. Fayers PM, Ashby D, Parmar MK. Tutorial in biostatistics Bayesian data monitoring in clinical trials. Stat Med. 1997;16:1413–1430. [PubMed]
39. Freedman LS, Spiegelhalter DJ. Application of Bayesian statistics to decision making during a clinical trial. Stat Med. 1992;11:23–35. [PubMed]
40. Freedman LS, Spiegelhalter DJ, Parmar MK. The what, why and how of Bayesian clinical trials monitoring. Stat Med. 1994;13:1371–1383. discussion 1385-1389. [PubMed]
41. Grossman J, Parmar MK, Spiegelhalter DJ, et al. A unified method for monitoring and analysing controlled trials. Stat Med. 1994;13:1815–1826. [PubMed]
42. Berry DA. A case for Bayesianism in clinical trials. Stat Med. 1993;12:1377–1393. discussion 1395-1404. [PubMed]
43. Freidlin B, Simon R. Adaptive signature design: An adaptive clinical trial design for generating and prospectively testing a gene expression signature for sensitive patients. Clin Cancer Res. 2005;11:7872–7878. [PubMed]
44. Henderson IC, Berry DA, Demetri GD, et al. Improved outcomes from adding sequential paclitaxel but not from escalating doxorubicin dose in an adjuvant chemotherapy regimen for patients with node-positive primary breast cancer. J Clin Oncol. 2003;21:976–983. [PubMed]
45. Newcombe RG. Bayesian estimation of false-negative rate in a clinical trial of sentinel node biopsy. Stat Med. 2007;26:3429–3442. [PubMed]
46. Gennari A, Amadori D, De Lena M, et al. Lack of benefit of maintenance paclitaxel in first-line chemotherapy in metastatic breast cancer. J Clin Oncol. 2006;24:3912–3918. [PubMed]
47. Dixon DO, Simon R. Bayesian subset analysis in a colorectal cancer clinical trial. Stat Med. 1992;11:13–22. [PubMed]
48. Gauvin A, Pinguet F, Culine S, et al. Bayesian estimate of vinorelbine pharmacokinetic parameters in elderly patients with advanced metastatic cancer. Clin Cancer Res. 2000;6:2690–2695. [PubMed]
49. Salinger DH, McCune JS, Ren AG, et al. Real-time dose adjustment of cyclophosphamide in a preparative regimen for hematopoietic cell transplant: A Bayesian pharmacokinetic approach. Clin Cancer Res. 2006;12:4888–4898. [PubMed]
50. Wong M, Balleine RL, Blair EY, et al. Predictors of vinorelbine pharmacokinetics and pharmacodynamics in patients with cancer. J Clin Oncol. 2006;24:2448–2455. [PubMed]
51. Chen TT, Chute JP, Feigal E, et al. A model to select chemotherapy regimens for phase III trials for extensive-stage small-cell lung cancer. J Natl Cancer Inst. 2000;92:1601–1607. [PubMed]
52. Berry DA, Iversen ES, Jr, Gudbjartsson DF, et al. BRCAPRO validation, sensitivity of genetic testing of BRCA1/BRCA2, and prevalence of other breast cancer susceptibility genes. J Clin Oncol. 2002;20:2701–2712. [PubMed]
53. Iversen E, Parmigiani G, Berry DA. Validating bayesian prediction models: A case study in genetic susceptibility to breast cancer. In: Gatsonis C, Kass RE, Carlin B, et al., editors. Case Studies in Bayesian Statistics. New York, NY: Springer; 1998. pp. 321–338.
54. Locatelli I, Rosina A, Lichtenstein P, et al. A correlated frailty model with long-term survivors for estimating the heritability of breast cancer. Stat Med. 2007;26:3722–3734. [PubMed]
55. Berry DA, Parmigiani G, Sanchez J, et al. Probability of carrying a mutation of breast-ovarian cancer gene BRCA1 based on family history. J Natl Cancer Inst. 1997;89:227–238. [PubMed]
56. Humphrey LL, Helfand M, Chan BK, et al. Breast cancer screening: A summary of the evidence for the U.S. Preventive Services Task Force. Ann Intern Med. 2002;137:347–360. [PubMed]
57. Berry DA, Inoue L, Shen Y, et al. Modeling the impact of treatment and screening on U.S. breast cancer mortality: A Bayesian approach. J Natl Cancer Inst Monogr. 2006:30–36. [PubMed]
58. Pauler DK, Finkelstein DM. Predicting time to prostate cancer recurrence based on joint models for non-linear longitudinal biomarkers and event time outcomes. Stat Med. 2002;21:3897–3911. [PubMed]
59. Estey EH, Thall PF, Wang X, et al. Effect of circulating blasts at time of complete remission on subsequent relapse-free survival time in newly diagnosed AML. Blood. 2003;102:3097–3099. [PubMed]
60. Bibb J, Hromas R, Rabinowitz I. A Bayesian approach to a patient with a residual mass after treatment for non-Hodgkin's lymphoma of the thyroid. J Clin Oncol. 2005;23:8911–8913. [PubMed]
61. McMahon PM, Zaslavsky AM, Weinstein MC, et al. Estimation of mortality rates for disease simulation models using Bayesian evidence synthesis. Med Decis Making. 2006;26:497–511. [PubMed]
62. Clèries R, Ribes J, Esteban L, et al. Time trends of breast cancer mortality in Spain during the period 1977-2001 and Bayesian approach for projections during 2002-2016. Ann Oncol. 2006;17:1783–1791. [PubMed]
63. Reference deleted
64. Reference deleted
65. Reference deleted
66. Reference deleted
67. Freedman B. Equipoise and the ethics of clinical research. N Engl J Med. 1987;317:141–145. [PubMed]
68. Rosner B. Design and Analysis Techniques for Epidemiologic Studies: Fundamentals of Biostatistics. Pacific Grove, CA: Duxbury; 2000. pp. 577–676.
69. Gelman A. Bayesian Data Analysis. ed 1. London, United Kingdom: Chapman & Hall; 1995.
70. Carlin BP, Louis TA. Bayes and Empirical Bayes Methods for Data Analysis. ed 2. Boca Raton, FL: Chapman & Hall/CRC; 2000.
71. Genest C, Zidek J. Combining probability distributions: A critique and an annotated bibliography (with discussion) Stat Sci. 1986;1:114–148.
72. Tversky A. Assessing uncertainty (with discussion) J R Stat Soc B. 1974;36:148–159.
73. Savage L. Elicitation of personal probabilities and expectations. J Am Stat Assoc. 1971;66:783–801.
74. Le Chevalier T. Results of the Randomized International Adjuvant Lung Cancer Trial (IALT): Cisplatin-based chemotherapy (CT) vs no CT in 1867 patients (pts) with resected non-small cell lung cancer (NSCLC) Proc Am Soc Clin Oncol. 2003;22:2a. abstr 6.
75. Pignon JP, Tribodet H, Scagliotti GV, et al. Lung Adjuvant Cisplatin Evaluation (LACE): A pooled analysis of five randomized clinical trials including 4,584 patients. J Clin Oncol. 2006;24(suppl):366s. abstr 7008.
76. Strauss GM, Herndon JE, Maddaus MA, et al. Adjuvant chemotherapy in stage IB non-small cell lung cancer (NSCLC): Update of Cancer and Leukemia Group B (CALGB) protocol 9633. J Clin Oncol. 2006;24(suppl):365s. abstr 7007.
77. Sutton AJ, Abrams KR. Bayesian methods in meta-analysis and evidence synthesis. Stat Methods Med Res. 2001;10:277–303. [PubMed]
78. Winkler RL. Why Bayesian analysis hasn't caught on in healthcare decision making. Int J Technol Assess Health Care. 2001;17:56–66. [PubMed]
79. Felip E, Cedres S, Peralta S, et al. Adjuvant chemotherapy in non-small cell lung cancer (NSCLC) Ann Oncol. 2007;18(suppl 9):ix143–ix146. [PubMed]
80. Faraggi D, Simon R. Large sample Bayesian inference on the parameters of the proportional hazard models. Stat Med. 1997;16:2573–2585. [PubMed]
81. Howard G, Coffey CS, Cutter GR. Is Bayesian ready for use in phase III randomized trials? Beware the sound of the sirens. Stroke. 2005;36:1622–1623. [PubMed]
82. Berger JO, Selke T. Testing a point null hypothesis: The irreconcilability of P values and evidence. J Am Stat Assoc. 1987;82:112–122.
83. Lehman CD, Gatsonis C, Kuhl CK, et al. MRI evaluation of the contralateral breast in women with recently diagnosed breast cancer. N Engl J Med. 2007;356:1295–1303. [PubMed]
84. Smith RA. The evolving role of MRI in the detection and evaluation of breast cancer. N Engl J Med. 2007;356:1362–1364. [PubMed]
85. Sobrero A. Should adjuvant chemotherapy become standard treatment for patients with stage II colon cancer? For the proposal. Lancet Oncol. 2006;7:515–516. [PubMed]
86. Köhne CH. Should adjuvant chemotherapy become standard treatment for patients with stage II colon cancer? Against the proposal. Lancet Oncol. 2006;7:516–517. [PubMed]

Articles from Journal of Clinical Oncology are provided here courtesy of American Society of Clinical Oncology