Background

While there is some consensus on methods for investigating statistical and methodological heterogeneity, little attention has been paid to clinical aspects of heterogeneity. The objective of this study is to summarize and collate suggested methods for investigating clinical heterogeneity in systematic reviews.

Methods

We searched databases (Medline, EMBASE, CINAHL, Cochrane Library, and CONSORT, to December 2010) and reference lists and contacted experts to identify resources providing suggestions for investigating clinical heterogeneity between controlled clinical trials included in systematic reviews. We extracted recommendations, assessed resources for risk of bias, and collated the recommendations.

Results

One hundred and one resources were collected, including narrative reviews, methodological reviews, statistical methods papers, and textbooks. These resources generally had a low risk of bias, but there was minimal consensus among them. Resources suggested that planned investigations of clinical heterogeneity should be made explicit in the protocol of the review; clinical experts should be included on the review team; a set of clinical covariates should be chosen considering variables from the participant level, intervention level, outcome level, research setting, or others unique to the research question; covariates should have a clear scientific rationale; there should be a sufficient number of trials per covariate; and results of any such investigations should be interpreted with caution.

Conclusions

Though the consensus was minimal, there were many recommendations in the literature for investigating clinical heterogeneity in systematic reviews. Formal recommendations for investigating clinical heterogeneity in systematic reviews of controlled trials are required.

Background

In meta-regression, as the number of trials in the analyses decreases, the risk of false positives or false negatives increases. This is partly due to the assumption of normality that may not hold in small samples. Creation of a distribution from the observed trials using permutation methods to calculate P values may allow for less spurious findings. Permutation has not been empirically tested in meta-regression. The objective of this study was to perform an empirical investigation to explore the differences in results for meta-analyses on a small number of trials using standard large sample approaches verses permutation-based methods for meta-regression.

Methods

We isolated a sample of randomized controlled clinical trials (RCTs) for interventions that have a small number of trials (herbal medicine trials). Trials were then grouped by herbal species and condition and assessed for methodological quality using the Jadad scale, and data were extracted for each outcome. Finally, we performed meta-analyses on the primary outcome of each group of trials and meta-regression for methodological quality subgroups within each meta-analysis. We used large sample methods and permutation methods in our meta-regression modeling. We then compared final models and final P values between methods.

Results

We collected 110 trials across 5 intervention/outcome pairings and 5 to 10 trials per covariate. When applying large sample methods and permutation-based methods in our backwards stepwise regression the covariates in the final models were identical in all cases. The P values for the covariates in the final model were larger in 78% (7/9) of the cases for permutation and identical for 22% (2/9) of the cases.

Conclusions

We present empirical evidence that permutation-based resampling may not change final models when using backwards stepwise regression, but may increase P values in meta-regression of multiple covariates for relatively small amount of trials.

Background

The objective of this review is to determine the effectiveness of Harpagophytum procumbens preparations in the treatment of various forms of musculoskeletal pain.

Methods

Several databases and other sources were searched to identify randomized controlled trials, quasi-randomized controlled trials, and controlled clinical trials testing Harpagophytum preparations in adults suffering from pain due to osteoarthritis or low back pain.

Results

Given the clinical heterogeneity and insufficient data for statistical pooling, trials were described in a narrative way, taking into consideration methodological quality scores. Twelve trials were included with six investigating osteoarthritis (two were identical trials), four low back pain, and three mixed-pain conditions.

Conclusions

There is limited evidence for an ethanolic Harpagophytum extract containing less than <30 mg harpagoside per day in the treatment of knee and hip osteoarthritis. There is moderate evidence of effectiveness for (1) the use of a Harpagophytum powder at 60 mg harpagoside in the treatment of osteoarthritis of the spine, hip and knee; (2) the use of an aqueous Harpagophytum extract at a daily dose of 100 mg harpagoside in the treatment of acute exacerbations of chronic non-specific low back pain; and (3) the use of an aqueous extract of Harpagophytum procumbens at 60 mg harpagoside being non-inferior to 12.5 mg rofecoxib per day for chronic non-specific low-back pain (NSLBP) in the short term. Strong evidence exists for the use of an aqueous Harpagophytum extract at a daily dose equivalent of 50 mg harpagoside in the treatment of acute exacerbations of chronic NSLBP.

Background The CONSORT statement is intended to improve reporting of randomised controlled trials and focuses on minimising the risk of bias (internal validity). The applicability of a trial’s results (generalisability or external validity) is also important, particularly for pragmatic trials. A pragmatic trial (a term first used in 1967 by Schwartz and Lellouch) can be broadly defined as a randomised controlled trial whose purpose is to inform decisions about practice. This extension of the CONSORT statement is intended to improve the reporting of such trials and focuses on applicability.

Methods At two, two-day meetings held in Toronto in 2005 and 2008, we reviewed the CONSORT statement and its extensions, the literature on pragmatic trials and applicability, and our experiences in conducting pragmatic trials.

Recommendations We recommend extending eight CONSORT checklist items for reporting of pragmatic trials: the background, participants, interventions, outcomes, sample size, blinding, participant flow, and generalisability of the findings. These extensions are presented, along with illustrative examples of reporting, and an explanation of each extension. Adherence to these reporting criteria will make it easier for decision makers to judge how applicable the results of randomised controlled trials are to their own conditions. Empirical studies are needed to ascertain the usefulness and comprehensiveness of these CONSORT checklist item extensions. In the meantime we recommend that those who support, conduct, and report pragmatic trials should use this extension of the CONSORT statement to facilitate the use of trial results in decisions about health care.

Pragmatic trials are designed to inform decisions about practice, but poor reporting can reduce their usefulness. The CONSORT and Practihc groups describe modifications to the CONSORT guidelines to help readers assess the applicability of the results