|Home | About | Journals | Submit | Contact Us | Français|
The Cochrane Collaboration, an international not-for-profit organization that prepares and maintains systematic reviews of randomized trials of health care therapies, has produced reviews summarizing much of the evidence on Traditional Chinese Medicine (TCM). Our objective was to review the evidence base according to Cochrane systematic reviews.
In order to detect reviews focusing on TCM, we searched the titles and abstracts of all reviews in Issue 4, 2008 of the Cochrane Database of Systematic Reviews. For each review, we extracted data on the number of trials included and the total number of participants. We provided an indication of the strength of the review findings by assessing the reviewers' abstract conclusions statement. We supplemented our assessment of the abstract conclusions statements with a listing of the comparisons and outcomes showing statistically significant meta-analyses results.
We identified 70 Cochrane systematic reviews of TCM, primarily acupuncture (n = 26) and Chinese herbal medicine (n = 42), and 1 each of moxibustion and t'ai chi. Nineteen (19) of 26 acupuncture reviews and 22/42 herbal medicine reviews concluded that there was not enough good quality trial evidence to make any conclusion about the efficacy of the evaluated treatment, while the remaining 7 acupuncture and 20 herbal medicine reviews and each of the moxibustion and t'ai chi reviews indicated a suggestion of benefit, which was qualified by a caveat about the poor quality and quantity of studies. Most reviews included many distinct interventions, controls, outcomes, and populations, and a large number of different comparisons were made, each with a distinct forest plot.
Most Cochrane systematic reviews of TCM are inconclusive, due specifically to the poor methodology and heterogeneity of the studies reviewed. Some systematic reviews provide preliminary evidence of Chinese medicine's benefits to certain patient populations, underscoring the importance and appropriateness of further research. These preliminary findings should be considered tentative and need to be confirmed with rigorous randomized controlled trials.
Traditional Chinese Medicine (TCM) approaches include acupuncture, herbal medicine, moxibustion and t'ai chi/qigong. Acupuncture and herbal medicine are two of the most commonly used complementary medicine therapies.1,2 In a nationally representative U.S. survey conducted in 2002, almost 20% of adults had used herbal therapies in the past year,3 and about 1% had used acupuncture.4 Among Asian-Americans, the prevalence was much higher, with 75%–100% reporting using TCM, primarily herbal medicine.2 The allure of these therapies is not driven by dissatisfaction with conventional medicine, according to recent evidence,1,5,6 but rather by the belief that combining Chinese and conventional medicine provides more optimal healing than conventional medicine alone.1,7–9 Chinese medicine users also enjoy a sense of participation in their own healing,10 and feel a congruence between Chinese medicine and their personal values and philosophical orientation.5,10,11 The use of Chinese medicine therapies is more prevalent among women and those of higher socio-economic status,7,8,10,12,13 with poorer health and higher education levels being additional predictors of its use.8–12 Health concerns driving the use of TCM therapies are primarily those of chronic pain, musculoskeletal problems, and mood disturbances, including back and neck pain, joint pain and stiffness, headaches, anxiety, and depression.1,2,8–10 TCM users report improvements in both specific symptoms and overall quality of life, according to several surveys.10
Increasing use of TCM approaches in the United States contributed to the establishment in 1998 of the National Center for Complementary and Alternative Medicine within the U.S. National Institutes of Health. This led to a dramatic expansion in the number and quality of randomized controlled trials (RCTs) funded to study TCM, many of which have focused on acupuncture and traditional Chinese herbal therapies. Concurrent with these trials, systematic reviews have been conducted in order to assess the current state of the research evidence on TCM therapies, and to provide direction for future research.
The independent and not-for-profit Cochrane Collaboration is the world's leading producer of up-to-date systematic reviews of health care therapies.14 The Collaboration currently involves more than 15,000 contributors, most of whom are volunteers, from over 100 countries.15 The structure of the Collaboration is based upon Cochrane Review Groups, which perform systematic reviews on specific health care problems (e.g., schizophrenia, breast cancer); Cochrane Fields, which focus on populations, interventions, or other health care areas that are related to multiple Cochrane Review Groups (e.g., child health, complementary and alternative medicine); and Cochrane Centers, which provide support and assistance to Cochrane Review Groups and Fields in specific countries or geographical areas (e.g., the Australasian Cochrane Centre). The principal product of the Cochrane Collaboration, the Cochrane Database of Systematic Reviews (CDSR), included 3625 complete systematic reviews and 1921 review protocols, as of Issue 4, 2008. Because the Cochrane Collaboration relies on grants and donations and does not accept conflicted funding,16 Cochrane reviews are largely free from competing financial interests that have the potential to distort and exaggerate findings. The Cochrane Collaboration has overseen a remarkable growth in the evidence base for Chinese medicine therapies. Much of this work has been conducted by the Chinese Cochrane Center and the Cochrane Complementary and Alternative Medicine (CAM) Field. The Cochrane (CAM) Field is based at the University of Maryland School of Medicine, Center for Integrative Medicine. One of the functions of this Field is to develop databases of all CAM-related Cochrane reviews, classify these reviews according to the CAM treatment modality investigated (e.g., acupuncture, Chinese herbal medicine), and disseminate information about the reviews to CAM researchers, providers, and consumers worldwide.
To determine the current extent of the Cochrane coverage of TCM therapies, we identified each Cochrane review whose primary focus was Chinese herbal medicine, acupuncture, moxibustion, t'ai chi, qigong, or tui na (Chinese massage). In order to detect reviews focusing on TCM, we searched the titles and abstracts of all reviews in Issue 4, 2008 of the CDSR using the text word search strategy in Box 1. We read the title and abstract of each retrieved review in order to confirm that the review was focused on TCM, to identify reviews that have been withdrawn from publication (e.g., because the review is no longer considered current), and to classify the TCM topic of the review. We did not include reviews that stated their focus was on all CAM therapies, on general herbal interventions, or on any other therapies that were not listed above or otherwise identified as specifically Chinese. For example, in the Chinese herbal medicine category, we included only those reviews of herbal medicine used traditionally as TCM, and excluded reviews of individual herbs primarily used in the West. Although Ginkgo biloba is sometimes used in the West, it is also commonly used in Chinese medicine, either alone or in combination with other herbs, and therefore we included ginkgo reviews.
For each review, we extracted data on the number of trials included and the total number of participants. As an indicator of study quality, we also extracted data on the number of trials using adequate versus inadequate/unclear allocation concealment. We used allocation concealment as an indicator of the quality of the included trials for two reasons: first, unconcealed allocation is the study defect shown to be most strongly associated with exaggerated treatment effects,17 and second, allocation concealment is the only quality measure that is reported in a standardized way for all trials in a Cochrane review (i.e., as adequate, unclear, or inadequate).
In Cochrane reviews that include many distinct interventions, controls, outcomes, and populations, a large number of different comparisons will be made, each with a distinct forest plot. (Forest plots are used to graphically present outcomes of the comparison between a given intervention and a given control for a specific outcome.) To indicate the number of comparisons in a review, and hence its complexity, as well as to show the proportion of comparisons with positive findings, we also extracted the following data for each review: number of forest plots; number of forest plots with a meta-analysis (i.e., statistical pooling of the trial data in the forest plot); and the number of meta-analyses showing a statistically significant benefit of the intervention relative to a control. We included the number of meta-analyses, as conduct of a meta-analysis suggests sufficient homogeneity between the studies under review to allow their results to be quantitatively synthesized.
In addition to identifying, listing, and providing descriptive and analytic data about the individual TCM reviews, we also wanted to provide an indication of the nature of the review findings by an assessment of the reviewers' abstract conclusions statement. In Cochrane reviews, the authors' conclusions are statements crafted by the review authors to represent the best overall assessment of a treatment's effect. These statements are declared only after consideration not only of the findings, but of the nuanced factors contributing to them, including risk of bias, as well as heterogeneity of design, setting, dosage, duration, and outcome measurement among the studies analyzed within the reviews. The Cochrane reviewers attempt to find optimal wording to summarize this complexity for the busy reader who is seeking a brief synopsis of whether or not the treatment “works.” Editors of Cochrane Review Groups recognize the paramount importance of these statements, and methodically examine them during the peer review process to ensure that they accurately reflect the bottom-line message. Given the thoughtfulness imbued in the abstract conclusion statement, we felt this would provide an informative reflection of the review. For each review, we assigned the abstract conclusions statement into one of the following two categories: (A) the statement indicated a suggestion of benefit, which was qualified by a caveat about the poor quality and quantity of included studies; or (B) the statement indicated that the currently available data do not allow any conclusions to be drawn. As an example of an “A” category review, the “Herbal medicines for treatment of irritable bowel syndrome” review, concluded: “Some herbal medicines may improve the symptoms of irritable bowel syndrome. However, positive findings from less rigorous trials should be interpreted with caution due to inadequate methodology, small sample sizes, and lack of confirming data. Some herbal medicines deserve further examination in high-quality trials.” As an example of a “B” category review, the “Acupuncture for stroke rehabilitation” review, concluded: “Currently there is no clear evidence on the effects of acupuncture on subacute or chronic stroke. Large methodologically-sound trials are required.” Both “A” and “B” category reviews generally noted that additional methodologically sound trials were needed before definitive conclusions could be reached. Because the distinction between the “A” and “B” reviews was not always obvious, this assignment was carried out by 2 reviewers making independent assessments. In the case of disagreement between the 2, a consensus was reached by discussion; this occurred in 5/70 (7%) of cases. For the reviews classified as category A, we listed the outcomes within each review that showed statistically significant meta-analyses. Also for reviews classified as A, all review data were dually extracted by 2 authors with consensus reached by discussion in cases of disagreement; for reviews classified as B, 1 author extracted all data.
Figure 1 shows the results of our search of the CDSR. We identified 70 Cochrane systematic reviews of TCM, primarily acupuncture and Chinese herbal medicine. Table 118–43 lists the 26 reviews of acupuncture. Table 244–84 lists the 42 reviews of Chinese herbal medicine, and Table 385,86 lists the 2 reviews of other TCM therapies (i.e., 1 each of moxibustion and t'ai chi; 0 reviews of qigong or tui na were identified).
We identified 26 Cochrane reviews dedicated to assessing the efficacy of acupuncture (Table 1). Five (5) reviews focused on treatment of pain (low back pain, shoulder pain, lateral elbow pain, idiopathic headache, and neck disorders), and five on mental or addiction disorders (depression, schizophrenia, vascular dementia, cocaine addiction, and smoking cessation). Three (3) reviews focused on gynecological or pregnancy-associated conditions (primary dysmenorrhea, assisted conception, and induction of labor), three reviews focused on stroke (acute stroke, dysphagia in acute stroke, and stroke rehabilitation), and two reviews addressed nausea or vomiting (chemotherapy-induced or postoperative). There was one acupuncture review for each of the following conditions: arthritis, asthma, Bell's palsy, epilepsy, glaucoma, irritable bowel syndrome, and restless legs syndrome.
Two (2) of 26 (8%) acupuncture reviews were empty reviews; that is, the authors could not find any randomized controlled trials on the relevant topic. Of the remaining reviews, 7 (29%) suggested that there might be some benefit, which was qualified by a caveat about the poor quality and quantity of studies (category A). For 2 of the 7 reviews in category A (29%), the review authors did not conduct any meta-analyses and their conclusion statements were based on a review of the individual trials.29,33 Meta-analyses were conducted in the remaining 5 (71%). The conditions for which benefit was found for acupuncture were vomiting, nausea, back and neck pain, headache, and assisted conception.
Seventeen (17) of 24 (71%) of the nonempty reviews concluded that there was not enough good quality trial evidence to make any conclusion (as yet) about the efficacy of the evaluated treatment (category B). In 7/17 (41%) of the reviews in which no benefit from treatment could be conclusively established, significant results were found for at least one meta-analytic outcomes.21,22,27,30,31,35,37 However, the review authors concluded that either the quality of the trials was not sufficient to support a judgment of benefit,22,27,31,35,37 that the endpoints were clinically unimportant,30 or that the heterogeneity21,31,37 or small sample sizes22,31,35 precluded drawing conclusions of a benefit. Table 1 shows the number of significant meta-analyses and, for the A category reviews, the outcomes assessed in these meta-analyses.
We identified 42 Cochrane reviews dedicated specifically to assessing the efficacy of Chinese herbal medicine (Table 2). Eight (8) reviews focused on treatment of stroke (acute ischemic stroke, acute cerebral infarction), six on treatment of heart problems (angina pectoris, acute myocardial infarction, viral myocarditis, heart failure), five on treatment of mental disorders (schizophrenia, cognitive impairment and dementia, Alzheimer's disease), four on treatment of respiratory problems (severe acute respiratory syndrome, acute bronchitis, common cold), and three each upon treatment of hepatitis (asymptomatic hepatitis B carriers, chronic hepatitis B, hepatitis C virus infection) and gynecological or pregnancy problems (pre-eclampsia, ectopic pregnancy, dysmenorrhea). The remaining 13 reviews focused on a range of problems, from lung cancer to tinnitus (see titles of reviews in Table 2). Twenty reviews were focused on single herbs or herbal preparations, while the remainder concerned multiple Chinese herbs or multiple formulations of Chinese herbs.
Of the 42 Cochrane reviews on Chinese herbal medicine, 6/42 (14%) were empty. Of the remaining reviews, 20/36 (56%) of the authors' abstract conclusions supported the possible efficacy of Chinese herbal formulas for treating specific health conditions (category A), while 17/36 (47%) failed to find enough good quality trial evidence to say whether or not the treatment was possibly effective (category B). Of the reviews in category A, 5/19 (26%) did not contain any meta-analyses and the conclusions were based on a review of individual trials. The conditions for which possible benefit was found for Chinese herbal medicine were atopic eczema, primary dysmenorrhea, schizophrenia, nephritic syndrome, angina, type II diabetes mellitus, severe acute respiratory syndrome, acute pancreatitis, hepatitis B, common cold, side-effects of chemotherapy in breast cancer, irritable bowel syndrome, viral myocarditis, Alzheimer's disease, ischemic stroke, and heart failure.
Of the nonempty reviews for which no benefit from treatment could be conclusively established (i.e., category B reviews), 7/17 (41%) showed statistically significant results from a meta-analysis. In these cases, the review authors concluded that either the quality of the trials was not sufficient to support a judgment of benefit,52,64–67,70,71 the sample sizes were too small,64,66,67 or there was a risk of conflicted interest.52 Table 2 lists the Chinese herbal medicine reviews, and shows the number of significant meta-analyses, as well as the outcomes associated with these significant meta-analyses in cases where the authors' conclusions suggested benefit.
We found two Cochrane reviews on TCM that were not acupuncture or herbal medicine. One (1) review was on the use of moxibustion for cephalic version in breech presentation and one review was on t'ai chi in the treatment of rheumatoid arthritis. Both reviews identified trials, but only one review conducted meta-analyses. The author conclusion statements in these reviews were cautiously positive, and thus the reviews were placed in category A. These reviews are listed in Table 3.
TCM therapies, including acupuncture and herbal medicine, are used increasingly to complement conventional medical care. Scientific research has grown in response to this surge in popular use, with the number of projects funded to study TCM approaches seeing a dramatic rise in the last decade. The Cochrane Collaboration has summarized much of the evidence on acupuncture and Chinese herbal medicine, and its Cochrane Database of Systematic Reviews currently includes 26 systematic reviews on acupuncture and 42 systematic reviews on Chinese herbal medicine. Many of these systematic reviews are inconclusive, due specifically to the poor methodology and heterogeneity of the studies reviewed. However, several systematic reviews provide preliminary evidence of Chinese medicine's benefits to certain patient populations. For example, acupuncture may be helpful for those with postoperative nausea and vomiting, chemotherapy-induced nausea and vomiting, various types of chronic pain, and as an adjuvant treatment for in vitro fertilization. Chinese herbal medicine may also be helpful for a wide range of conditions, including huperzine A for Alzheimer's disease and sanchi for acute ischemic stroke.
We indicated which reviews showed the most promising evidence by categorizing the authors' abstract conclusions statements, and for those statements suggesting a possible benefit, we listed the comparisons/outcomes that were statistically significant. An advantage of selecting the most promising reviews based on the abstract conclusions statements are that these statements provide the most informative and concise summary of the treatments' effects. Their drawback is that they do not include quantitative estimates of effect, and also they may be somewhat subjective or inaccurate, depending on the reviewer's interpretation of the evidence. Therefore, we supplemented the categorization of the abstract conclusions with a listing of the comparisons/outcomes showing statistically significant meta-analyses results.
However, benefits suggested in the meta-analyses need to be interpreted with much caution. The data generally have been pooled from trials with a variety of methodological quality, some of which were imprecise and carried a high risk of bias. Some reviews with positive abstract conclusion statements do not include any meta-analyses, due to the heterogeneity of the trials included, rendering the summary less quantitative and more qualitative. An additional limitation is that a majority of trials included here have been carried out and published in China. The quality of these trials typically has been low,87,88 as indicated by noncompliance with CONSORT statement recommendations89 such as describing randomization, allocation concealment, sample size calculation, and participant follow-up procedures. The Chinese government recently has made substantial investments into funding Chinese medicine clinical research,90 which already has resulted in important improvements and greater compliance to international standards.91 Once a substantial number of trials of improved methodological quality are completed, future updates of currently inconclusive Cochrane reviews of Chinese medicine may reach more definitive conclusions, and true evidence-based Chinese medicine will be possible.
In summary, the current Cochrane systematic review evidence for TCM herbs is somewhat inconsistent and not yet fully convincing because of the methodological problems encountered when summarizing studies that are heterogeneous on the type of herbal formulas, use of control conditions, formula dosages, and durations of use.34 Rather than proclaiming the success of TCM approaches, our intention here is to highlight those trials that suggest possible benefit in order to encourage further rigorous study in these areas. These potential benefits need to be confirmed in large, rigorous, double-blind randomized placebo-controlled trials.
The reason we have restricted this overview to Cochrane reviews is because they are implicitly trustworthy, given the rigorous methodological standards imposed by the Cochrane Collaboration in its acceptance of articles. Indeed, Cochrane reviews require the use of explicit and transparent methods, are peer-reviewed at both the protocol and complete review stage, and are regularly updated. For these reasons, they have been found to be of comparable or better quality than reviews published in even the leading print journals.92,93 We also restricted to Cochrane reviews because they use fairly homogeneous methods for both their preparation and their peer review, as described in the Cochrane Reviewer's Handbook, and therefore across Cochrane reviews, the authors' conclusions statements for a given level of evidence would be fairly standard and consistent. If we had included non-Cochrane reviews, we might have introduced a source of heterogeneity to the conclusions presented in the tables, due not to differences in the evidence from the RCTs reviewed but rather to a difference in the location of publication. A limitation of restricting to Cochrane reviews is that the breadth of TCM topics covered by the Cochrane Database is not yet comprehensive, and there are some topics that are currently still at the protocol or registered title stage. Two notable omissions from the tables are the absence of Cochrane reviews of acupuncture for postoperative pain and acupuncture for osteoarthritis; both of these reviews are currently published only as protocols in the Cochrane Database, but both should be available as full reviews within the next year. We are currently conducting a project to identify high priority but not yet registered Cochrane titles by comparing the lists of current Cochrane titles against the contents of the Cochrane Complementary Medicine Field register of trials to identify condition/treatment pairings for which RCTs have been published but no Cochrane title has yet been registered. The results of this project may identify other omissions in the current list of Cochrane reviews. A final limitation of our restricting to Cochrane reviews is that many of these reviews are several years old and their conclusions may not reflect the most current evidence. Although Cochrane policy states that reviews should ideally be updated every 2 years, in practice this is difficult to achieve because updates can involve a substantial amount of work, often approaching the effort level required to prepare a new review (which has been estimated at 1139 person hours94), and yet updates garner little academic recognition for the review authors. To address this, there are various initiatives under way to increase the frequency of updating of Cochrane reviews, including the provision of funding support from the U.S. National Center for Complementary and Alternative Medicine (as disbursed through the Cochrane Complementary Medicine Field (http://medschool.umaryland.edu/integrative/cochrane_bursary.asp) and the UK Department of Health. Despite this limitation in updating frequency, Cochrane reviews are often still more current than reviews published in print journals, which are fixed and therefore may become obsolete shortly after publication.
For those seeking hard and fast answers to their health care questions, it may seem discouraging that the evidence for many therapies is undetermined. However, with the continued investment by federal funders such as the National Center for Complementary and Alternative Medicine at the National Institutes of Health, the Cochrane Collaboration, and other organizations in supporting rigorous evaluation of TCM therapies, the evidence base will continue to grow. Questions that cannot be answered definitively today will become resolved in the future as current Cochrane reviews are updated to include newer and better trials. Twenty (20) years ago, there were very few RCTs conducted on these modalities, and systematic review methodology was in its infancy. Now, after decades of work and innovation, true evidence-based TCM is becoming a reality.
EM, EK, and BMB were supported in part by Whole-HealthMD.com, a program of Healthways WholeHealth Networks. EM, KC, SW, and BMB were also supported in part by Grant Number R24 AT001293 from the National Center for Complementary and Alternative Medicine (NCCAM) of the U.S. National Institutes of Health. Neither funding body had any role in the conception of this article, nor in the collection, analysis, and interpretation of the studies reviewed; the writing of the manuscript; or the decision to submit the manuscript for publication.
No competing financial interests exist.