Search tips
Search criteria

Results 1-2 (2)

Clipboard (0)
Year of Publication
Document Types
1.  Investigating clinical heterogeneity in systematic reviews: a methodologic review of guidance in the literature 
While there is some consensus on methods for investigating statistical and methodological heterogeneity, little attention has been paid to clinical aspects of heterogeneity. The objective of this study is to summarize and collate suggested methods for investigating clinical heterogeneity in systematic reviews.
We searched databases (Medline, EMBASE, CINAHL, Cochrane Library, and CONSORT, to December 2010) and reference lists and contacted experts to identify resources providing suggestions for investigating clinical heterogeneity between controlled clinical trials included in systematic reviews. We extracted recommendations, assessed resources for risk of bias, and collated the recommendations.
One hundred and one resources were collected, including narrative reviews, methodological reviews, statistical methods papers, and textbooks. These resources generally had a low risk of bias, but there was minimal consensus among them. Resources suggested that planned investigations of clinical heterogeneity should be made explicit in the protocol of the review; clinical experts should be included on the review team; a set of clinical covariates should be chosen considering variables from the participant level, intervention level, outcome level, research setting, or others unique to the research question; covariates should have a clear scientific rationale; there should be a sufficient number of trials per covariate; and results of any such investigations should be interpreted with caution.
Though the consensus was minimal, there were many recommendations in the literature for investigating clinical heterogeneity in systematic reviews. Formal recommendations for investigating clinical heterogeneity in systematic reviews of controlled trials are required.
PMCID: PMC3564789  PMID: 22846171
2.  An empirical study using permutation-based resampling in meta-regression 
Systematic Reviews  2012;1:18.
In meta-regression, as the number of trials in the analyses decreases, the risk of false positives or false negatives increases. This is partly due to the assumption of normality that may not hold in small samples. Creation of a distribution from the observed trials using permutation methods to calculate P values may allow for less spurious findings. Permutation has not been empirically tested in meta-regression. The objective of this study was to perform an empirical investigation to explore the differences in results for meta-analyses on a small number of trials using standard large sample approaches verses permutation-based methods for meta-regression.
We isolated a sample of randomized controlled clinical trials (RCTs) for interventions that have a small number of trials (herbal medicine trials). Trials were then grouped by herbal species and condition and assessed for methodological quality using the Jadad scale, and data were extracted for each outcome. Finally, we performed meta-analyses on the primary outcome of each group of trials and meta-regression for methodological quality subgroups within each meta-analysis. We used large sample methods and permutation methods in our meta-regression modeling. We then compared final models and final P values between methods.
We collected 110 trials across 5 intervention/outcome pairings and 5 to 10 trials per covariate. When applying large sample methods and permutation-based methods in our backwards stepwise regression the covariates in the final models were identical in all cases. The P values for the covariates in the final model were larger in 78% (7/9) of the cases for permutation and identical for 22% (2/9) of the cases.
We present empirical evidence that permutation-based resampling may not change final models when using backwards stepwise regression, but may increase P values in meta-regression of multiple covariates for relatively small amount of trials.
PMCID: PMC3351721  PMID: 22587815

Results 1-2 (2)