Diabetic peripheral neuropathy (DPN) is very common in people with diabetes. Chinese herbal medicine (CHM) therapy has been developed for DPN empirically over the years. The aim of this systematic review and meta-analysis was to assess the efficacy and safety of CHMs for patients suffering from DPN.
We performed a meta-analysis of randomized-controlled clinical trials (RCTs) evaluating the efficacy and safety of CHM on DPN. Six databases were searched up to November 2012. The primary outcome measures were the absolute values or changing of motor or sensory nerve conduction velocity (NCV), and the secondary outcome measurements were clinical symptoms improvements and adverse events. The methodological quality was assessed by Jadad scale and the twelve criteria recommended by the Cochrane Back Review Group.
One hundred and sixty-three studies claimed RCTs. Ten studies with 653 individuals were further identified based on the Jadad score ≥3. These 10 studies were all of high methodological quality with a low risk of bias. Meta-analysis showed the effects of NCV favoring CHMs when compared with western conventional medicines (WCM) (P<0.05 or P<0.01). There is a significant difference in the total efficacy rate between the two groups (P<0.001). Adverse effects were reported in all of the ten included studies, and well tolerated in all patients with DPN.
Despite of the apparently positive findings and low risk of bias, it is premature to conclude the efficacy of CHMs for the treatment of DPN because of the high clinical heterogeneity and small sample sizes of the included studies. However, CHM therapy was safe for DPN. Further standardized preparation, large sample-size and rigorously designed RCTs are required.
It is fundamental that randomised controlled trials (RCTs) are properly conducted in order to reach well-supported conclusions. However, there is emerging evidence that RCTs are subject to biases which can overestimate or underestimate the true treatment effect, due to flaws in the study design characteristics of such trials. The extent to which this holds true in oral health RCTs, which have some unique design characteristics compared to RCTs in other health fields, is unclear. As such, we aim to examine the empirical evidence quantifying the extent of bias associated with methodological and non-methodological characteristics in oral health RCTs.
Methods and analysis
We plan to perform a meta-epidemiological study, where a sample size of 60 meta-analyses (MAs) including approximately 600 RCTs will be selected. The MAs will be randomly obtained from the Oral Health Database of Systematic Reviews using a random number table; and will be considered for inclusion if they include a minimum of five RCTs, and examine a therapeutic intervention related to one of the recognised dental specialties. RCTs identified in selected MAs will be subsequently included if their study design includes a comparison between an intervention group and a placebo group or another intervention group. Data will be extracted from selected trials included in MAs based on a number of methodological and non-methodological characteristics. Moreover, the risk of bias will be assessed using the Cochrane Risk of Bias tool. Effect size estimates and measures of variability for the main outcome will be extracted from each RCT included in selected MAs, and a two-level analysis will be conducted using a meta-meta-analytic approach with a random effects model to allow for intra-MA and inter-MA heterogeneity.
Ethics and dissemination
The intended audiences of the findings will include dental clinicians, oral health researchers, policymakers and graduate students. The aforementioned will be introduced to the findings through workshops, seminars, round table discussions and targeted individual meetings. Other opportunities for knowledge transfer will be pursued such as key dental conferences. Finally, the results will be published as a scientific report in a dental peer-reviewed journal.
Oral & Maxillofacial Surgery; Oral Medicine
Motivation: Permutation tests have become a standard tool to assess the statistical significance of an event under investigation. The statistical significance, as expressed in a P-value, is calculated as the fraction of permutation values that are at least as extreme as the original statistic, which was derived from non-permuted data. This empirical method directly couples both the minimal obtainable P-value and the resolution of the P-value to the number of permutations. Thereby, it imposes upon itself the need for a very large number of permutations when small P-values are to be accurately estimated. This is computationally expensive and often infeasible.
Results: A method of computing P-values based on tail approximation is presented. The tail of the distribution of permutation values is approximated by a generalized Pareto distribution. A good fit and thus accurate P-value estimates can be obtained with a drastically reduced number of permutations when compared with the standard empirical way of computing P-values.
Availability: The Matlab code can be obtained from the corresponding author on request.
Supplementary information:Supplementary data are available at Bioinformatics online.
Large-scale statistical analyses have become hallmarks of post-genomic era biological research due to advances in high-throughput assays and the integration of large biological databases. One accompanying issue is the simultaneous estimation of p-values for a large number of hypothesis tests. In many applications, a parametric assumption in the null distribution such as normality may be unreasonable, and resampling-based p-values are the preferred procedure for establishing statistical significance. Using resampling-based procedures for multiple testing is computationally intensive and typically requires large numbers of resamples.
We present a new approach to more efficiently assign resamples (such as bootstrap samples or permutations) within a nonparametric multiple testing framework. We formulated a Bayesian-inspired approach to this problem, and devised an algorithm that adapts the assignment of resamples iteratively with negligible space and running time overhead. In two experimental studies, a breast cancer microarray dataset and a genome wide association study dataset for Parkinson's disease, we demonstrated that our differential allocation procedure is substantially more accurate compared to the traditional uniform resample allocation.
Our experiments demonstrate that using a more sophisticated allocation strategy can improve our inference for hypothesis testing without a drastic increase in the amount of computation on randomized data. Moreover, we gain more improvement in efficiency when the number of tests is large. R code for our algorithm and the shortcut method are available at .
Traditional permutation (TradPerm) tests are usually considered the gold standard for multiple testing corrections. However, they can be difficult to complete for the meta-analyses of genetic association studies based on multiple single nucleotide polymorphism loci as they depend on individual-level genotype and phenotype data to perform random shuffles, which are not easy to obtain. Most meta-analyses have therefore been performed using summary statistics from previously published studies. To carry out a permutation using only genotype counts without changing the size of the TradPerm P-value, we developed a Monte Carlo permutation (MCPerm) method. First, for each study included in the meta-analysis, we used a two-step hypergeometric distribution to generate a random number of genotypes in cases and controls. We then carried out a meta-analysis using these random genotype data. Finally, we obtained the corrected permutation P-value of the meta-analysis by repeating the entire process N times. We used five real datasets and five simulation datasets to evaluate the MCPerm method and our results showed the following: (1) MCPerm requires only the summary statistics of the genotype, without the need for individual-level data; (2) Genotype counts generated by our two-step hypergeometric distributions had the same distributions as genotype counts generated by shuffling; (3) MCPerm had almost exactly the same permutation P-values as TradPerm (r = 0.999; P<2.2e-16); (4) The calculation speed of MCPerm is much faster than that of TradPerm. In summary, MCPerm appears to be a viable alternative to TradPerm, and we have developed it as a freely available R package at CRAN: http://cran.r-project.org/web/packages/MCPerm/index.html.
Objective. To evaluate the effectiveness of Tuina-focused integrative Chinese medical therapies (TICMT) on inpatients with low back pain (LBP). Methods. 6 English and Chinese databases were searched for randomized controlled trials (RCTs) of TICMT for in-patients with LBP. The methodological quality of the included RCTs was assessed based on PEDro scale. And the meta-analyses of TICMT for LBP on pain and functional status were conducted. Results. 20 RCTs were included. The methodological quality of the included RCTs was poor. The meta-analyses' results showed that TICMT had statistically significant effects on pain and functional status, especially Tuina plus Chinese herbal medicine (standardised mean difference, SMD: 1.17; 95% CI 0.75 to 1.60 on pain; SMD: 1.31; 95% CI 0.49 to 2.14 on functional status) and Tuina plus acupuncture (SMD: 0.94; 95% CI 0.38 to 1.50 on pain; SMD: 0.53; 95% CI 0.21 to 0.85 on functional status). But Tuina plus moxibustion or hot pack did not show significant improvements on pain. And the long-term evidence of TICMT was far from sufficient. Conclusions. The preliminary evidence from current studies suggests that TICMT might be effective complementary and alternative treatments for in-patients with LBP. However, the poor methodological quality of the included RCTs means that high-quality RCTs with long follow-up are warranted.
We conducted a systematic review to evaluate the efficacy and safety of Chinese herbal medicine (CHM) for dysfunctional uterine bleeding (DUB) by performing a meta-analysis. Randomized controlled trials (RCTs) or quasi-RCTs comparing CHM vs no treatment, placebo, conventional western medicine (CWM), or general non-specific surgical treatment for DUB were identified by electronic and manual searches. Trials of CHM treatments with CWM treatments were compared with CWM treatments alone. Jadad scale and allocation concealment were used to assess the quality of included studies. Four RCTs or quasi-RCTs involving 525 patients were included. The methodological quality was poor in all trials except one trial. No serious adverse events were reported in the included studies. With the lack of trials comparing CHM with no treatment or placebo, it is impossible to accurately evaluate the efficacy of CHM. However, CHM in these studies seem to show an encouraging comparative effectiveness with CWM. More RCTs with a higher quality are required.
Chinese herbal medicine; dysfunctional uterine bleeding; meta-analysis; randomized controlled trials; systematic review
In microarray experiments with small sample sizes, it is a challenge to estimate p-values accurately and decide cutoff p-values for gene selection appropriately. Although permutation-based methods have proved to have greater sensitivity and specificity than the regular t-test, their p-values are highly discrete due to the limited number of permutations available in very small sample sizes. Furthermore, estimated permutation-based p-values for true nulls are highly correlated and not uniformly distributed between zero and one, making it difficult to use current false discovery rate (FDR)-controlling methods.
We propose a model-based information sharing method (MBIS) that, after an appropriate data transformation, utilizes information shared among genes. We use a normal distribution to model the mean differences of true nulls across two experimental conditions. The parameters of the model are then estimated using all data in hand. Based on this model, p-values, which are uniformly distributed from true nulls, are calculated. Then, since FDR-controlling methods are generally not well suited to microarray data with very small sample sizes, we select genes for a given cutoff p-value and then estimate the false discovery rate.
Simulation studies and analysis using real microarray data show that the proposed method, MBIS, is more powerful and reliable than current methods. It has wide application to a variety of situations.
To assess whether reported trial quality or trial characteristics are associated with trial outcome.
Study Design and Setting
We identified all eligible randomized controlled trials (RCTs) of arthroplasty from 1997 and 2006. Trials were classified based on whether the main trial outcome was reported to be positive (n=90) or negative (n=94). Multivariable logistic regression analyses studied the association of reporting of trial quality measures (blinding, placebo use, allocation procedure; overall quality) and trial characteristics (intervention type, number of patients/centers, funding) with positive trial outcome.
RCTs that used placebo or blinded care providers, used pharmacological interventions, had higher Jadad quality scores or sample size >100 patients were significantly more likely to report positive result in univariate analyses. Multivariable regression did not identify methodological quality of RCTs, but rather that sample size, was associated with trial outcome. Studies with >100 patients were 2.2 times more likely to report a positive result than smaller studies (p=0.04).
Lack of association of reported trial quality with positive outcome in multivariable analyses suggests that previously observed association of reported study quality with study outcome in univariate analyses may be mediated by other study characteristics, such as study sample size.
Arthroplasty RCTs with sample size of 100 patients or more are significantly more likely to report a positive rather than a negative outcome.RCT quality was not associated with study outcome (positive vs. negative) in multivariable analyses, in contrast to previous studies that found an association of quality with outcomes in univariate analyses that did not adjust for sample size or type of intervention.Previously reported associations of study quality and outcomes may have been mediated by these characteristics.Future studies examining correlates of study outcomes should control for sample size, study quality and type of intervention.
Assessment; Quality; Arthroplasty; Randomized Controlled Trial; Trial Outcome
Meta-analysis has become a key component of well-designed genetic association studies due to the boost in statistical power achieved by combining results across multiple samples of individuals and the need to validate observed associations in independent studies. Meta-analyses of genetic association studies based on multiple SNPs and traits are subject to the same multiple testing issues as single-sample studies, but it is often difficult to adjust accurately for the multiple tests. Procedures such as Bonferroni may control the type I error rate but will generally provide an overly harsh correction if SNPs or traits are correlated. Depending on study design, availability of individual-level data, and computational requirements, permutation testing may not be feasible in a meta-analysis framework. In this paper we present methods for adjusting for multiple correlated tests under several study designs commonly employed in meta-analyses of genetic association tests. Our methods are applicable to both prospective meta-analyses in which several samples of individuals are analyzed with the intent to combine results, and retrospective meta-analyses, in which results from published studies are combined, including situations in which 1) individual-level data are unavailable, and 2) different sets of SNPs are genotyped in different studies due to random missingness or two-stage design. We show through simulation that our methods accurately control the rate of type I error and achieve improved power over multiple testing adjustments that do not account for correlation between SNPs or traits.
meta-analysis; association study; multiple testing; SNPs
Chinese herbal medicine has shown promise for heroin detoxification. This review extends a prior meta-analysis of Chinese herbal medicine for heroin detoxification, with particular attention to the time course of symptoms. Both English and Chinese databases were searched for randomized trials comparing Chinese herbal medicine to either α2-adrenergic agonists or opioid agonists for heroin detoxification. The methodological quality of each study was assessed with Jadad’s scale (1–2 = low; 3–5 = high). Meta-analysis was performed with fixed- or random-effect models in RevMan software; outcome measures assessed were withdrawal-symptoms score, anxiety, and adverse effects of treatment. Twenty-one studies (2,949 participants) were included. For withdrawal-symptoms score relieving during the 10-day observation, Chinese herbal medicine was superior to α2-adrenergic agonists in relieving opioid-withdrawal symptoms during 4–10 days (except D8) and no difference was found within the first 3 days. Compared with opioid agonists, Chinese herbal medicine was inferior during the first 3 days, but the difference became non-significant during days 4–9. Chinese herbal medicine has better effect on anxiety relieving at late stage of intervention than α2-adrenergic agonists, and no difference with opioid agonists. The incidence of some adverse effects (fatigue, dizziness) was significantly lower for Chinese herbal medicine than for α2-adrenergic agonists (sufficient data for comparison with opioid agonists were not available). Findings were robust to file-drawer effects. Our meta-analysis suggests that Chinese herbal medicine is an effective and safety treatment for heroin detoxification. And more work is needed to determine the specific effects of specific forms of Chinese herbal medicine.
Chinese herbal medicine; Heroin; Detoxification; Meta-analysis
The outcome of a functional genomics pipeline is usually a partial list of genomic features, ranked by their relevance in modelling biological phenotype in terms of a classification or regression model. Due to resampling protocols or to a meta-analysis comparison, it is often the case that sets of alternative feature lists (possibly of different lengths) are obtained, instead of just one list. Here we introduce a method, based on permutations, for studying the variability between lists (“list stability”) in the case of lists of unequal length. We provide algorithms evaluating stability for lists embedded in the full feature set or just limited to the features occurring in the partial lists. The method is demonstrated by finding and comparing gene profiles on a large prostate cancer dataset, consisting of two cohorts of patients from different countries, for a total of 455 samples.
Linkage analysis in multivariate or longitudinal context presents both statistical and computational challenges. The permutation test can be used to avoid some of the statistical challenges, but it substantially adds to the computational burden. Utilizing the distributional dependencies between π̂ (defined as the proportion of alleles at a locus that are identical by descent (IBD) for a pairs of relatives, at a given locus) and the permutation test we report a new method of efficient permutation. In summary, the distribution of π̂ for a sample of relatives at locus x is estimated as a weighted mixture of π̂ drawn from a pool of ‘representative’ π̂ distributions observed at other loci. This weighting scheme is then used to sample from the distribution of the permutation tests at the representative loci to obtain an empirical P-value at locus x (which is asymptotically distributed as the permutation test at loci x). This weighted mixture approach greatly reduces the number of permutation tests required for genome-wide scanning, making it suitable for use in multivariate and other computationally intensive linkage analyses. In addition, because the distribution of π̂ is a property of the genotypic data for a given sample and is independent of the phenotypic data, the weighting scheme can be applied to any phenotype (or combination of phenotypes) collected from that sample. We demonstrate the validity of this approach through simulation.
Empirical significance level; Mixture distribution; Linkage analysis
A large number of infertile couples are choosing Chinese herbal medicine (CHM) as an adjuvant therapy to improve their success when undergoing in vitro fertilization (IVF). There is no systematic review to evaluate the impact of CHM on the IVF outcomes.
To evaluate the effectiveness of CHM with concurrent IVF versus IVF alone on the outcomes of IVF and its safety.
The protocol of this study is registered at PROSPERO. Eligible RCTs searched from 8 databases which compared a combination of CHM and IVF with IVF alone were included. Two authors independently selected studies, extracted data and assessed methodological quality. Meta-analysis of RCTs was conducted if there was non-significant heterogeneity (evaluated by I2 test) among trials. All statistical analysis was performed using RevMan 5.1 software.
Twenty trials involving 1721 women were included in the meta-analysis. Three trials were evaluated as having an unclear risk of bias. The remaining trials were evaluated as having a high risk of bias. Combination of CHM and IVF significantly increases clinical pregnancy rates (OR 2.04, 95%CI 1.67 to 2.49, p<0.00001) and ongoing pregnancy rates (OR 1.91, 95%CI 1.17 to 3.10, p = 0.009). Use of CHM after embryo transfer had no better outcome in reducing the rate of ovarian hyper stimulation syndrome (OR 0.39, 95%CI 0.14 to 1.11, p = 0.08).
This meta-analysis showed that combination of IVF and CHM used in the included trials improve IVF success, however due to the high risk of bias observed with the trials, the significant differences found with the meta-analysis are unlikely to be accurate. No conclusion could be drawn with respect to the reproductive toxicity of CHM. Further large randomized placebo controlled trials are warranted to confirm these findings before recommending women to take CHM to improve their IVF success.
Motivation: A gene set test is a differential expression analysis in which a P-value is assigned to a set of genes as a unit. Gene set tests are valuable for increasing statistical power, organizing and interpreting results and for relating expression patterns across different experiments. Existing methods are based on permutation. Methods that rely on permutation of probes unrealistically assume independence of genes, while those that rely on permutation of sample are suitable only for two-group comparisons with a good number of replicates in each group.
Results: We present ROAST, a statistically rigorous gene set test that allows for gene-wise correlation while being applicable to almost any experimental design. Instead of permutation, ROAST uses rotation, a Monte Carlo technology for multivariate regression. Since the number of rotations does not depend on sample size, ROAST gives useful results even for experiments with minimal replication. ROAST allows for any experimental design that can be expressed as a linear model, and can also incorporate array weights and correlated samples. ROAST can be tuned for situations in which only a subset of the genes in the set are actively involved in the molecular pathway. ROAST can test for uni- or bi-direction regulation. Probes can also be weighted to allow for prior importance. The power and size of the ROAST procedure is demonstrated in a simulation study, and compared to that of a representative permutation method. Finally, ROAST is used to test the degree of transcriptional conservation between human and mouse mammary stems.
Availability: ROAST is implemented as a function in the Bioconductor package limma available from www.bioconductor.org
Supplementary information: Supplementary data are available at Bioinformatics online.
In Mexico, local empirical knowledge about medicinal properties of plants is the basis for their use as home remedies. It is generally accepted by many people in Mexico and elsewhere in the world that beneficial medicinal effects can be obtained by ingesting plant products. In this review, we focus on the potential pharmacologic bases for herbal plant efficacy, but we also raise concerns about the safety of these agents, which have not been fully assessed. Although numerous randomized clinical trials of herbal medicines have been published and systematic reviews and meta-analyses of these studies are available, generalizations about the efficacy and safety of herbal medicines are clearly not possible. Recent publications have also highlighted the unintended consequences of herbal product use, including morbidity and mortality. It has been found that many phytochemicals have pharmacokinetic or pharmacodynamic interactions with drugs. The present review is limited to some herbal medicine that are native or cultivated in Mexico and that have significant use. We discuss the cultural uses, phytochemistry, pharmacological and toxicological properties of the following following plant species: Nopal (Opuntia ficus), Peppermint (Mentha piperita), Chaparral (Larrea divaricata), Dandlion (Taraxacum officinale), Mullein (Verbascum densiflorum), Chamomile (Matricaria recutita), Nettle or Stinging Nettle (Urtica dioica), Passionflower (Passiflora incarmata), Linden Flower (Tilia europea), and Aloa (Aloa vera). We conclude that our knowledge of the therapeutic benefits and risks of some herbal medicines used in Mexico is still limited and efforts to elucidate them should be intensified.
In order to reduce systematic errors (such as language bias) and increase the precision of the summary treatment effect estimate, a comprehensive identification of randomised controlled trials (RCT), irrespective of publication language, is crucial in systematic reviews and meta-analyses. We identified trials in the German general health care literature.
Eight German language general health care journals were searched for randomised controlled trials and analysed with respect to the number of published RCTs each year and the size of trials.
A total of 1618 trials were identified with a median total number of 43 patients per trial. Between 1970 and 2004 a small but constant rise in sample size from a median number of 30 to 60 patients per trial can be observed. The number of published trials was very low between 1948 and 1970, but increased between 1970 and 1986 to a maximum of 11.2 RCTs per journal and year. In the following time period a striking decline of the number of RCTs was observed. Between 1999 and 2001 only 0.8 RCTs per journal and year were published, in the next three years, the number of published trials increased to 1.7 RCTs per journal and year.
German language general health care journals no longer have a role in the dissemination of trial results. The slight rise in the number of published RCTs in the last three years can be explained by a change of publication language from German to English of three of the analysed journals.
AIM: To determine the efficacy of probiotic supplementation on intestinal transit time (ITT) and to identify factors that influence these outcomes.
METHODS: A systematic review of randomized controlled trials (RCTs) of probiotic supplementation that measured ITT in adults was conducted by searching MEDLINE and EMBASE using relevant key word combinations. Main search limits included RCTs of probiotic supplementation in healthy or constipated adults that measured ITT. Study quality was assessed using the Jadad scale. A random effects meta-analysis was performed with standardized mean difference (SMD) of ITT between probiotic and control groups as the primary outcome. Meta-regression and subgroup analyses were conducted to examine the impact of moderator variables on ITT SMD.
RESULTS: A total of 11 clinical trials with 13 treatment effects representing 464 subjects were included in this analysis. Probiotic supplementation was associated with decreased ITT in relation to controls, with an SMD of 0.40 (95%CI: 0.20-0.59, P < 0.001). Constipation (r2 = 39%, P = 0.01), higher mean age (r2 = 27%, P = 0.03), and higher percentage of female subjects (r2 = 23%, P < 0.05) were predictive of decreased ITT with probiotics in meta-regression. Subgroup analyses demonstrated statistically greater reductions in ITT with probiotics in subjects with vs without constipation and in older vs younger subjects [both SMD: 0.59 (95%CI: 0.39-0.79) vs 0.17 (95%CI: -0.08-0.42), P = 0.01]. Medium to large treatment effects were identified with Bifidobacterium Lactis (B. lactis) HN019 (SMD: 0.72, 95%CI: 0.27-1.18, P < 0.01) and B. lactis DN-173 010 (SMD: 0.54, 95%CI: 0.15-0.94, P < 0.01) while other single strains and combination products yielded small treatment effects.
CONCLUSION: Overall, short-term probiotic supplementation decreases ITT with consistently greater treatment effects identified in constipated or older adults and with certain probiotic strains.
Constipation; Gastrointestinal; Intestinal transit time; Meta-analysis; Probiotics
When sample replicates are limited in a label-free proteomics experiment, selecting differentially regulated proteins with an assignment of statistical significance remains difficult for proteins with a single-peptide hit or a small fold-change. This paper aims to address this issue. An important component of the approach employed here is to utilize the rule of Minimum number of Permuted Significant Pairings (MPSP) to reduce false positives. The MPSP rule generates permuted sample pairings from limited analytical replicates and simply requires that a differentially regulated protein can be selected only when it is found significant in designated number of permuted sample pairings. Both a power law global error model with a signal-to-noise ratio statistic (PLGEM-STN) and a constant fold-change threshold were initially used to select differentially regulated proteins. But both methods were found not stringent enough to control the false discovery rate to 5% in this study. On the other hand, the combination of the MPSP rule with either of these two methods significantly reduces false positives with little effect on the sensitivity to select differentially regulated proteins including those with a single-peptide hit or with a <2-fold change.
A new version of the False Selection Rate variable selection method of Wu, Boos, and Stefanski (2007) is developed that requires no simulation. This version allows the tuning parameter in forward selection to be estimated simply by hand calculation from a summary table of output even for situations where the number of explanatory variables is larger than the sample size. Because of the computational simplicity, the method can be used in permutation tests and inside bagging loops for improved prediction. Illustration is provided in clinical trials for linear regression, logistic regression, and Cox proportional hazards regression.
Bagging; False discovery rate; False selection rate; Forward selection; LASSO; Model error; Model selection; Regression
Systematic review and meta-analysis currently underpin much of evidence-based medicine. Such methodologies bring order to previous research, but future research planning remains relatively incoherent and inefficient.
To outline a framework for evaluation of health interventions, aimed at increasing coherence and efficiency through i) making better use of information contained within the existing evidence-base when designing future studies; and ii) maximising the information available and thus potentially reducing the need for future studies.
The framework presented insists that an up-to-date meta-analysis of existing randomised controlled trials (RCTs) should always be considered before future trials are conducted. Such a meta-analysis should inform critical design issues such as sample size determination. The contexts in which the use of individual patient data meta-analysis and mixed treatment comparisons modelling may be beneficial before further RCTs are conducted are considered. Consideration should also be given to how any newly planned RCTs would contribute to the totality of evidence through its incorporation into an updated meta-analysis. We illustrate how new RCTs can have very low power to change inferences of an existing meta-analysis, particularly when between study heterogeneity is taken into consideration.
While the collation of existing evidence as the basis for clinical practice is now routine, a more coherent and efficient approach to planning future RCTs to strengthen the evidence base needs to be developed. The framework presented is a proposal for how this situation can be improved.
A meta-analysis was performed to evaluate the use of clinical pathways for hip and knee joint replacements when compared with standard medical care. The impact of clinical pathways was evaluated assessing the major outcomes of in-hospital hip and knee joint replacement processes: postoperative complications, number of patients discharged at home, length of in-hospital stay and direct costs.
Medline, Cinahl, Embase and the Cochrane Central Register of Controlled Trials were searched. The search was performed from 1975 to 2007. Each study was assessed independently by two reviewers. The assessment of methodological quality of the included studies was based on the Jadad methodological approach and on the New Castle Ottawa Scale. Data analysis abided by the guidelines set out by The Cochrane Collaboration regarding statistical methods. Meta-analyses were performed using RevMan software, version 4.2.
Twenty-two studies met the study inclusion criteria and were included in the meta-analysis for a total sample of 6,316 patients. The aggregate overall results showed significantly fewer patients suffering postoperative complications in the clinical pathways group when compared with the standard care group. A shorter length of stay in the clinical pathway group was also observed and lower costs during hospital stay were associated with the use of the clinical pathways. No significant differences were found in the rates of discharge to home.
The results of this meta-analysis show that clinical pathways can significantly improve the quality of care even if it is not possible to conclude that the implementation of clinical pathways is a cost-effective process, because none of the included studies analysed the cost of the development and implementation of the pathways. Based on the results we assume that pathways have impact on the organisation of care if the care process is structured in a standardised way, teams critically analyse the actual organisation of the process and the multidisciplinary team is highly involved in the re-organisation. Further studies should focus on the evaluation of pathways as complex interventions to help to understand which mechanisms within the clinical pathways can really improve the quality of care. With the need for knee and hip joint replacement on the rise, the use of clinical pathways might contribute to better quality of care and cost-effectiveness.
Small-study effects refer to the fact that trials with limited sample sizes are more likely to report larger beneficial effects than large trials. However, this has never been investigated in critical care medicine. Thus, the present study aimed to examine the presence and extent of small-study effects in critical care medicine.
Critical care meta-analyses involving randomized controlled trials and reported mortality as an outcome measure were considered eligible for the study. Component trials were classified as large (≥100 patients per arm) and small (<100 patients per arm) according to their sample sizes. Ratio of odds ratio (ROR) was calculated for each meta-analysis and then RORs were combined using a meta-analytic approach. ROR<1 indicated larger beneficial effect in small trials. Small and large trials were compared in methodological qualities including sequence generating, blinding, allocation concealment, intention to treat and sample size calculation.
A total of 27 critical care meta-analyses involving 317 trials were included. Of them, five meta-analyses showed statistically significant RORs <1, and other meta-analyses did not reach a statistical significance. Overall, the pooled ROR was 0.60 (95% CI: 0.53 to 0.68); the heterogeneity was moderate with an I2 of 50.3% (chi-squared = 52.30; P = 0.002). Large trials showed significantly better reporting quality than small trials in terms of sequence generating, allocation concealment, blinding, intention to treat, sample size calculation and incomplete follow-up data.
Small trials are more likely to report larger beneficial effects than large trials in critical care medicine, which could be partly explained by the lower methodological quality in small trials. Caution should be practiced in the interpretation of meta-analyses involving small trials.
When testing large numbers of null hypotheses, one needs to assess the evidence against the global null hypothesis that none of the hypotheses is false. Such evidence typically is based on the test statistic of the largest magnitude, whose statistical significance is evaluated by permuting the sample units to simulate its null distribution. Efron (2007) has noted that correlation among the test statistics can induce substantial interstudy variation in the shapes of their histograms, which may cause misleading tail counts. Here, we show that permutation-based estimates of the overall significance level also can be misleading when the test statistics are correlated. We propose that such estimates be conditioned on a simple measure of the spread of the observed histogram, and we provide a method for obtaining conditional significance levels. We justify this conditioning using the conditionality principle described by Cox and Hinkley (1974). Application of the method to gene expression data illustrates the circumstances when conditional significance levels are needed.
Conditional p-value; Gene expression data; Genome-wide association data; Multiple testing; Overall p-value
Objective: To systematically review the effects of omega-3 poly unsaturated fatty acids (FA) enriched nutrition support on the mortality of critically illness patients. Methods: Databases of Medline, ISI, Cochrane Library, and Chinese Biomedicine Database were searched and randomized controlled trials (RCTs) were identified. We enrolled RCTs that compared fish oil enriched nutrition support and standard nutrition support. Major outcome is mortality. Methodological quality assessment was conducted based on Modified Jadad’s score scale. For control heterogeneity, we developed a method that integrated I2 test, nutritional support route subgroup analysis and clinical condition of severity. RevMan 5.0 software (The Nordic Cochrane Centre, Copenhagen, Denmark) was used for meta-analysis. Results: Twelve trials involving 1208 patients that met all the inclusion criteria. Heterogeneity existed between the trials. A random model was used, there was no significant effect on mortality RR, 0.82, 95% confidence interval (CI) (0.62, 1.09), p = 0.18. Knowing that the route of fish oil administration may affect heterogeneity, we categorized the trials into two sub-groups: parenteral administration (PN) of omega-3 and enteral administration (EN) of omega-3. Six trials administered omega-3 FA through PN. Pooled results indicated that omega-3 FA had no significant effect on mortality, RR 0.76, 95% CI (0.52, 1.10), p = 0.15. Six trials used omega-3 fatty acids enriched EN. After excluded one trial that was identified as source of heterogeneity, pooled data indicated omega-3 FA enriched EN significant reduce mortality, RR=0.69, 95% CI [0.53, 0.91] (p = 0.007). Conclusion: Omega-3 FA enriched nutrition support is safe. Due to the limited sample size of the included trials, further large-scale RCTs are needed.
omega-3 fatty acids; severe illness; parenteral nutrition; enteral nutrition; meta-analysis