We found consistent effects of traditional Chinese medicines when combined with TACE versus TACE alone. The majority of studies included in our analysis were small or of moderate size and none can provide definitive answers on treatment options, although compelling results related to bufotoxin, astragalus and products containing ginseng, astragalus and mylabris warrant further examination. Our study also highlights the utility that searching in non-English languages may have on identifying potentially useful new interventions for common diseases. While our study finds compelling results, there is also reason for caution, given the poor reporting of clinical trials in China. Only independently conducted research from high-quality research teams will strengthen the inference of effectiveness.
Strengths of our study include our extensive searches of literature in both English and in Chinese languages, and using Chinese language databases for our search. Two of us (PW, JL) understand and read Mandarin and Cantonese, along with English, thus allowing searches across several languages. We applied a broad criteria for pooling studies. We included any TCM formulation and then conducted a meta-regression analysis to determine if specific preparation yielded differing effects over the broad group, and in several cases did.
Limitations of our study include the underlying concern about the quality of the included studies. As we highlight, the majority of studies were small, with typically 30 participants per arm. Meta-analysis aims to overcome issues of power through pooling, thus increasing sample size and power. We applied an OIS on the overall event rate of partial response and found that a pooled sample size of 1,108 provided sufficient evidence of an effect. This did not apply to specific formulations. We further assessed issues of methodological rigour as two major concerns with Chinese-based clinical trials. Firstly, is that only positive trials are published in Chinese medical journals, and second, is that some trials reported as randomized are, in fact, not randomized. A recent evaluation by Wu et al. found that many studies labelled as RCTs with Chinese journals were, in fact, not randomized[71
] In our own experience, we recognize many Chinese clinical trialists have not been exposed to appropriate clinical epidemiology training. We examined publication bias through both visual inspection of the funnel plot on the primary outcome (PR) and through statistical tests, but were unable to identify publication bias. However, funnel plots cannot rule out publication bias and we remain cautious that many negative trials likely exist.
From a clinical standpoint, the results of this study are very encouraging but should be implemented with caution. The average clinician will be reassured that TCM interventions, both herbal-based and animal/insect-based, were safely combined with chemotherapy. The average clinician, however, likely will not scrutinize the results of this study using evidence-based principles and may implement our findings into practice due to the overwhelming positive response in our meta-analysis. Given this tendency, the results from this study should be carefully disseminated to the medical community with the caveat that although promising, our findings need to be confirmed via a RCT conducted in a Western academic setting.
Our study may prove useful for a number of reasons. Firstly, there is reason to further examine the evidence of several of the interventions included in our analysis. Other investigators have examined the role of herbal medicines and TCM interventions for hepatocellular cancers, lung cancers and hepatitis and found compelling evidence in humans [72
] However, perhaps a far more important finding from our analysis and approach is the role that searching for clinical trials in non-English languages may play in drug discovery. Important first line drugs, such as artemisin-based therapies for malaria, have been discovered through searching existing trials in non-English languages. [76
There have now been two studies prior to ours that examined the role of TCM interventions on survival and clinical outcomes in patients also receiving TACE. [72
] The first study, by Shu et al[72
], published in 2005, included 26 RCTs of interventions including 2079 patients. Similar to ours, they found improved survival at 12 months [RR 1.55, 95% CI, 1.39–1.72], at 24 months [RR 2.15, 95% CI, 1.75–2.64], and at 36 months [RR, 2.76, 95% CI, 1.95–3.91]. Tumor response was also significantly increased [RR 1.39, 95% CI, 1.24–1.56]. A more recent study, published in 2009 by Cho and Chen, [75
] included 30 studies including 2428 patients. As with ours, they found increased survval at 12 months [OR 1.92, 95% CI, 1.43–2.57], at 24 months [OR 3.55, 95% CI, 2.36–5.36], and at 36 months [OR 5.12, 95% CI, 2.76–9.52]. The inflated effect sizes found in the study by Cho and Chen may be related to their choice of effect size of OR rather than the more conservative RR((([77
] Given that all three reviews found compelling evidence of a role for TCM in hepatocellular cancers, it seems appropriate that further evaluations, in a non-Chinese setting, occur in order to determine if we have a possible new opportunity for drug development.
Our study builds on the findings of others about the heterogeneous quality of randomized trials from China. In our own experience in China, we have doubts that many methodological features attributed to randomized trials, were in fact conducted. A previous analysis, by Vickers et al, found that most trials conducted in China were reported as positive,[78
] a finding our analysis also supports8
. While several explanations for this phenomenon exist, a likely explanation is the slow uptake of evidence-based medicine and clinical trials methodology in academic research centres[79
] With the opening of the Chinese Cochrane Centre, we hope that clinical epidemiology will receive considerably more attention[80
In conclusion, our study provides important inferences about new potential therapeutic options for hepatocellular cancers. While these finds are compelling, there is a need for confirmation of these studies in well-conducted RCTs conducted in Western settings. Until such time, potentially useful interventions cannot be wholly recommended based on evidence alone.