|Home | About | Journals | Submit | Contact Us | Français|
To the Editor:—We appreciate O’Connor’s interest in our article. We did not exclude open-label trials because some studies have not found lack of double-blind methods to be associated with differences in estimates of treatment effects.1 Although open-label design could have a greater tendency to bias results in trials that evaluate subjective outcomes such as pain, we found no evidence that this occurred here, as there was no statistical heterogeneity among head-to-head trials. In addition, even if the open-label trial2 is excluded from the analysis, there is no difference in likelihood of pain relief between gabapentin and tricyclics (pooled relative risk 0.89, 95% confidence interval [CI] 0.62 to 1.27), and the discrepancy between the direct and indirect meta-analyses remains statistically significant (Δ=0.77, p=0.028). Placebo arms of active-controlled trials can provide useful information, but their absence does not invalidate direct comparisons of relative effectiveness between two competing therapies.3
We agree with O’Connor that publication bias is an important issue to consider. We did not include unpublished studies because results often change significantly between early presentation and final publication, and it is often difficult to adequately assess their methods and study quality.4 The unpublished trial cited by O’Connor found trends towards higher rates of pain relief among patients treated with 1200 or 2400 mg/day of gabapentin versus placebo, though only the 1200 mg arm reached statistical significance.5 Its inclusion would have attenuated the efficacy estimate from placebo-controlled trials of gabapentin, thus increasing the discrepancy between direct and indirect estimates. Effects of publication bias are difficult to predict, however, as graphical and statistical tests suggest its presence in placebo-controlled trials of tricyclics, but not gabapentin.
We thank O’Connor for pointing out an error in data abstraction from a trial by Gorson et al.6 We based our analysis on the number of patients randomized in the first crossover period rather than on all patients enrolled. Using the correct data (17/40 vs. 9/40), one finds the estimate for pain relief in placebo-controlled gabapentin trials is virtually identical to the estimate presented in our article (pooled relative risk 2.17 [95% CI 1.76 to 2.67]), and there is no effect on the indirect estimate.
We do not agree that indirect comparisons remain the best source of information on relative efficacy of gabapentin versus tricyclics. Compared to the gabapentin trials, the placebo-controlled tricyclic trials had more methodological shortcomings, more frequently used a crossover design, reported lower placebo response rates, and had findings that suggested publication bias. All of these factors violate the critical assumption underlying indirect comparisons—that treatment effects should be consistent across all studies.7 The fact that a large discrepancy between direct and indirect comparisons is present after the publication of only a handful of head-to-head trials underscores the need for more evidence, but doesn’t make problematic indirect comparisons any more valid.