|Home | About | Journals | Submit | Contact Us | Français|
A comprehensive systematic review of economic evaluations of complementary and integrative medicine (CIM) to establish the value of these therapies to health reform efforts.
PubMed, CINAHL, AMED, PsychInfo, Web of Science and EMBASE were searched from inception through 2010. In addition, bibliographies of found articles and reviews were searched, and key researchers were contacted.
Studies of CIM were identified using criteria based on those of the Cochrane complementary and alternative medicine group. All studies of CIM reporting economic outcomes were included.
All recent (and likely most cost-relevant) full economic evaluations published 2001–2010 were subjected to several measures of quality. Detailed results of higher-quality studies are reported.
A total of 338 economic evaluations of CIM were identified, of which 204, covering a wide variety of CIM for different populations, were published 2001–2010. A total of 114 of these were full economic evaluations. And 90% of these articles covered studies of single CIM therapies and only one compared usual care to usual care plus access to multiple licensed CIM practitioners. Of the recent full evaluations, 31 (27%) met five study-quality criteria, and 22 of these also met the minimum criterion for study transferability (‘generalisability’). Of the 56 comparisons made in the higher-quality studies, 16 (29%) show a health improvement with cost savings for the CIM therapy versus usual care. Study quality of the cost-utility analyses (CUAs) of CIM was generally comparable to that seen in CUAs across all medicine according to several measures, and the quality of the cost-saving studies was slightly, but not significantly, lower than those showing cost increases (85% vs 88%, p=0.460).
This comprehensive review identified many CIM economic evaluations missed by previous reviews and emerging evidence of cost-effectiveness and possible cost savings in at least a few clinical populations. Recommendations are made for future studies.
Between 1990 and 2007, four nationally representative surveys demonstrated that a third or more of US adults routinely used complementary and alternative medicine (CAM) therapies to treat their principal medical conditions.1–4 Total expenditures for CAM therapies were estimated at US$14 billion in 1990,1 US$27 billion in 19972 and US$34 billion in 2007.4 The 2007 US National Health Inventory Survey found that out-of-pocket expenditures for CAM therapies accounted for 11% of all out-of-pocket healthcare expenditures by Americans.4 Similar use numbers are seen in other countries.5–8 However, despite the popularity of and substantial expenditures on CAM therapies, their cost-effectiveness remains ill-defined and controversial.
Economic evaluations allow costs to be included, alongside data on safety and effectiveness, in healthcare policy decisions. As healthcare costs rise, the availability of these economic evaluations becomes increasingly important to the formulation of disease management strategies which are both clinically effective and financially responsible. According to the National Center for Complementary and Alternative Medicine (NCCAM), CAM is ‘a group of diverse medical and healthcare systems, practices and products that are not generally considered part of conventional medicine’.9 In not being part of conventional medicine, individual complementary therapies and emerging models of integrative medicine (ie, coordinated access to both conventional and complementary care)—collectively termed as complementary and integrative medicine (CIM)—are often excluded in financial mechanisms commonly available for conventional medicine,2 and are rarely included in the range of options considered in the formation of healthcare policy. The availability of economic data could improve the consideration and appropriate inclusion of CIM in strategies to lower overall healthcare costs. In addition, economic outcomes are relevant to the licensure and scope of practice of practitioners, industry investment decisions (eg, the business case for integrative medicine), consumers and future research efforts (ie, through identifying decision-critical parameters for additional research10).
A number of systematic reviews of economic evaluations of CIM have been published.11–23 Five of these prior reviews attempted to capture all economic evaluations of CIM therapies across all conditions.11 19–21 23 However, it is unclear as to whether all or even the majority of economic evaluations of CIM have been identified by these reviews. The searches are dated; the search strategy in the most recent review only captured articles published through 2007.23 The databases searched were limited—for example, only one used CINAHL,21 and only two others used EMBASE,19 23 in addition to Medline and AMED. Finally, these reviews generally used limited search terms to identify CIM studies. All but one only used variations on the terms ‘complementary’ or ‘alternative’ ‘medicine’ or ‘therapy’. Unfortunately, other reviewers have found that these search terms do not capture all CIM studies,24 25 which may be a reflection of the difficulty in defining what is and is not CIM.26 The search by Maxion-Bergemann et al11 also added individual therapies as search terms, but only included homeopathy, phytotherapy, traditional Chinese medicine, anthroposophic medicine and neural therapy. No search included ‘integrative medicine’.
The goal of this paper is to identify, to the extent possible, all published economic evaluations of CIM, describe the types of CIM evaluated and the clinical conditions for which they have been evaluated, and identify the recent (and therefore, most cost-relevant) higher-quality studies and highlight their results for policy makers. We also make recommendations for future economic evaluations of CIM.
Six electronic databases were searched from their inception through December 2010: PubMed, CINAHL, AMED, PsychInfo, Web of Science and EMBASE. To be as comprehensive as possible, a combination of 11 medical subject headings (MeSH) and 39 other search terms were used (box 1). In addition, bibliographies of found articles and reviews were searched, and key researchers in various areas of CIM were contacted for their lists of known studies. Although non-English language articles were collected, they are not analysed in this review.
Search terms used for the PubMed search: (Complementary Therapies (medical subject headings (MeSH)), Dietary Supplements (MeSH), Micronutrients (MeSH), Trace Elements (MeSH), Vitamins (MeSH), acupuncture, alternative medicine, ayurvedic medicine, chiropractic, biofeedback, collaborative medicine, complementary and alternative medicine, botanical medicine, complementary medicine, diet, energy medicine, herbal medicine, herbs, homeopathy, hypnosis, integrated medicine, integrative medicine, massage, meditation, mind-body medicine, minerals, naturopathic medicine, naturopathy, nutrients, nutritional supplements, relaxation, spa therapy, traditional Chinese medicine OR vitamins) AND (Cost-Benefit Analysis (MeSH), Cost Control (MeSH), Cost Savings (MeSH), Costs and Cost Analysis (MeSH), Economics (MeSH), economics (Subheading), Insurance (MeSH), cost benefit, cost effectiveness, cost identification, cost minimisation, cost utility, economic evaluation, insurance claims, managed care OR technology assessment). Searches in the other five databases used the same text words and (where available) analogous controlled vocabulary terms. All searches were restricted to human studies.
Defining a comprehensive search strategy for CIM is challenging.24 27–29 There have been a number of efforts to develop a concise definition of CAM.26 30 This review used the one developed by the members of the Cochrane CAM Field31 and then added the terms ‘integrative’, ‘integrated’ and ‘collaborative’ medicine. The Cochrane CAM definition starts with the NCCAM definition9 and then refines it by specifically including all therapies ‘based upon the theories of a medical system outside the Western allopathic medical model’ (eg, traditional Chinese medicine and Reiki), and including others depending on the context and setting of their use. The context of use considers treatment/condition combinations and excludes those ‘currently considered to be standard treatment’, and the setting of use generally includes self-care and therapies delivered by CIM providers, but excludes therapies ‘delivered exclusively by conventionally credentialed medical personnel or exclusively within hospital settings’. Therefore, therapies such as chemotherapy regimens (eg, chronotherapy32), and therapies requiring surgical implantation (eg, neuroreflexotherapy33) or the placement of a feeding tube34 were not included even though these therapies appeared in our search. In cases where CIM therapies (eg, biofeedback or hypnosis) were included as part of a package of care (eg, with cognitive behavioural therapy), a judgement was made as to whether the CIM portion of the treatment made up half or more of the overall package of care under study. If so, the package of care was included as CIM. Because more than half of CIM users use multiple CIM therapies,35 studies of packages of therapies and coordinated care were identified as such.
Articles were categorised as full economic evaluations if they compared the costs (inputs) and consequences (economic, clinical and/or humanistic outcomes36) of two or more therapeutic alternatives applied to the same patient population (ref.37, p. 11). Otherwise, they were considered partial evaluations, for example, cost-identification or cost-comparison studies.38 Studies that estimated resource utilisation were included as economic evaluations if the utilisation data were detailed enough to allow monetary valuation.
Two reviewers (PMH and BLP) evaluated all articles for inclusion and extracted all data. Disagreements were resolved by discussion between the two review authors, or, if needed, by the other coauthors. Because the results of economic evaluations can rapidly lose relevance with time, mainly due to changes in practice patterns and cost structures, data were extracted only from the economic evaluations published 2001–2010. Extracted data were entered into an Excel template developed for a previous review.20 The type(s) of CIM evaluated and the target population were captured for all economic evaluations. Various indicators of study quality were captured for all full economic evaluations, and more detailed data and results were captured only for those full economic evaluations that met five quality criteria.
The quality of an economic evaluation can be judged along two general dimensions: (1) whether the study was a quality measure of outcomes for its target population and location—that is, whether it was internally valid; and (2) whether enough information is provided for the study's results to be transferable (‘generalisable’).39 Health outcomes are to some extent considered generalisable across settings; however, because resource availability, practice patterns and relative prices can vary greatly, economic outcomes usually are not.40 Therefore, one goal in economic evaluation is to ensure the transferability of study results—that is, to provide enough study detail so that results can be adapted (usually via modelling) to apply to other settings.39 The 35-item BMJ checklist captures components of both dimensions of quality and was applied to all full economic evaluations.41 We also chose five quality criteria by which to identify a subset of full economic evaluations to highlight as being of most interest to policy makers. These quality criteria are based on recommendations made by the US Panel on Cost Effectiveness in Health and Medicine 42 and by well-known experts in the field,37 and focus on the quality of the underlying study (the first type of quality):
Because the prices used to value resources are highly location-specific and setting-specific,39 46 we also note, for the articles meeting the above criteria, the presence of a study reporting criterion essential for the transferability of study results (usually via modelling):39 40 separate reporting of unit costs from resource use for economic evaluations alongside trials, or from model parameters (eg, transition probabilities) for economic evaluations using models.
Other data extracted for the economic evaluations which meet the five study-quality criteria are: treatment and study duration, primary clinical and economic outcome measures, the setting in which treatment took place, study design and sample size, the type (table 1) and perspective (ie, the point of view used to define costs) of the economic analysis, and incremental cost-effectiveness of the CIM alternative compared to usual care. Incremental cost-effectiveness is reported in 2011 US$ and is calculated from reported results by first converting the study currency to US$ using the Federal Reserve annual exchange rate47 for the study's currency year and then inflated to 2011 values using the medical care component of the Consumer Price Index.48
Finally, up to three additional quality measures are included for each of these studies. The Tufts CEA Registry49 quality score is recorded when it was available (note it is only available for cost-utility analyses, CUAs). A Jadad score50 with minor modifications (the two possible points for blinding were replaced with one point for the use of a blinded assessor)51 was calculated for the economic evaluations that included a randomised trial. The percentage of the applicable items from the 35-item BMJ checklist that were met by each article is also reported.41
As shown in figure 1, the database search identified 270 published economic evaluations. An additional 68 articles were added through the bibliography and expert-supplied list search for a total of 338 economic evaluations of CIM. Of these, 204 (60%) were published from 2001 through 2010 (114 full and 90 partial economic evaluations). Of the recent full economic evaluations almost all (103, 90%) examined the effect of one CIM therapy and most of the balance (10, 9%) examined the effect of two or more CIM therapies provided by the same practitioner. Only one looked at the effect of multiple CIM therapies provided by different CIM providers.52 CIM was generally evaluated as an adjunct to usual care.
As shown in table 2, the 204 economic evaluations published in the past 10 years are spread across a wide range of CIM therapies applied to a number of different study populations. The biggest concentration of full economic evaluations (19 in number) pertained to the use of NCCAM's definition of manipulative and body-based practices (eg, chiropractic, osteopathic manipulation, massage, etc) for the treatment of back pain.53–72 However, even this subgroup is fairly heterogeneous in terms of the therapy (or therapies) tested and/or the type of back pain treated. Eight of these comparisons involved chiropractic care for back pain; one for chronic,53 one for acute57and six for either type.59 60 63 64 67 68 Five evaluated spinal manipulation and manual therapy provided by physiotherapists for chronic back pain (one),65 acute back pain (two)58 69 or either (two).56 68 Four involved osteopathic manipulation; one for chronic71 and one for subacute back pain72 and two for musculoskeletal conditions including back pain.66 68 Three evaluated massage; two for chronic55 62 and one for acute back pain.57 The last two studies evaluated a musculoskeletal physician (treatment ‘with a combination of manual therapy, injections, acupuncture and other pain management techniques’) for orthopaedic referrals;54 and a Finnish folk medicine practice called ‘bone setting’ for the treatment of patients with chronic back pain.61
Table 3 shows the results of the application of the 35-item BMJ checklist to the full economic evaluations published 2001–2010.41 On average, the number of applicable items met by each article stayed fairly constant during this period. However, the application of two key items (ie, the proper use of discounting and the inclusion of sensitivity analysis) and the disclosure of funding sources improved significantly, and reporting of the study time horizon worsened significantly. As expected, the average overall and individual-item percentages were higher for the higher-quality articles (those meeting the five study-quality criteria) and for CUAs of CIM. It is not surprising that CUA's quality is higher. They generally involve more effort than other CEAs and are required or recommended by various national guidelines.42 73–75 Nevertheless, it seems as though the quality of CUAs of CIM is generally comparable to, or slightly better than, that seen in CUAs across all medicine, at least in terms of the Tufts quality score, disclosure of funding sources and the five items where comparable data are available.76 77
The number of full evaluations meeting each of the five study-quality criteria are: comparison to usual care 97 (85%), all costs from a recognised perspective 96 (84%), health outcomes from a randomised or matched-control trial 86 (75%), patient-specific data on costs and outcomes 105 (92%) and sensitivity analyses 37 (32%). Sixty-two (54%) of full evaluations met the first four of these and 31 (27%) met all five. A summary of the results of these 31 higher-quality articles (covering 28 different studies) is shown in table 4.54 60 62 68 71 78–103 Twenty-two of these articles (19 of the studies) reported resource use (trials) or model parameters (models) separate from unit prices—a minimum measure of study transferability.54 62 68 71 78 80–85 87–93 95 100 101 103 For those studies which included a randomised trial, the modified Jadad scores ranged from 2 to 4 on a scale from 0 to 4. The Tufts CEA Registry quality scores for the studies containing a CUA ranged from 4 to 6.5 on a scale from 1 to 7. The percentage of the applicable items on the BMJ checklist met by these studies ranged from 66% to 97%.
Of the 56 comparisons made in these studies, 16 (29%) are cost saving—that is, the added CIM therapy had better health outcomes and lower costs than usual care alone. Cost savings were seen for acupuncture alone (instructional visits with an acupuncturist followed by home self-care by the partner for pregnant women with breech presentations at 33 weeks in terms of reductions in both breech presentation at birth and ceasareans in the Netherlands,91 and treatment by traditional Chinese medicine-trained licensed acupuncturists in private acupuncture clinics in the UK for low-back pain in terms of quality-adjusted life-years or QALYs from the societal perspective85) and in combination with other therapies (along with manual therapy, injections and other pain management for patients referred to an orthopaedic surgeon's office in Scotland who were unlikely to need surgery in terms of both improvements in health-related quality of life and QALYs54). Cost savings were also seen for manual therapy delivered by a physiotherapist, who is also a registered manual therapist, for neck pain in terms of perceived recovery, pain, neck disability and QALYs82; for preoperative oral supplementation with arginine and ω-3 fatty acids for patients with gastrointestinal cancer undergoing surgery102; for vitamin K1 supplementation for postmenopausal women with osteopenia and osteoporosis in terms of QALYs103; for supplementation with vitamins C and E and β-carotene for cataract prevention90; for fish oil supplementation in men with a history of heart attack87; for tai chi to prevent hip fractures in nursing home residents95 and for naturopathic care offered through a worksite clinic for chronic low-back pain in terms of both reductions in absenteeism and gains in QALYs.80
Of the 28 cost-utility comparisons, one (massage for low-back pain62) was dominated— that is, had worse health outcomes and higher costs than usual care. Five (18%) are cost saving,54 80 82 85 103 5 (18%) have incremental cost-effectiveness ratios (ICERs) between US$0 and US$10 000 per quality-adjusted life-year (QALY),68 71 81 85 97 and 89% had ICERs less than US$50 000/QALY, a threshold often considered to represent the upper limit of society's value for a QALY.104 The cost-saving cost-utility studies were included in the paragraph above (ie, those that mention QALYs). The studies with cost-utility ICERs between US$0 and US$10 000 per QALY were: treatment by traditional Chinese medicine-trained licensed acupuncturists in private acupuncture clinics in the UK for low-back pain.85 hospital-based acupuncture by licensed oriental medical doctors in South Korea for 60-year-old women with first-time acute low-back pain,81 acupuncture from physicians with at least 140 h of training (A-diploma) in Germany for patients with dysmenorrhoea,97 osteopathic spinal manipulation by a general practitioner who is a registered osteopath in the UK for patients with subacute back pain,71 and an exercise programme plus spinal manipulation from a chiropractor, osteopath or physiotherapist at a private or National Health Service (NHS) site in the UK for low-back pain.68 The average percentage of applicable BMJ checklist items met by each study was slightly lower for those studies with at least one cost-saving comparison (85% vs 88%), but the difference was not statistically significant (t test=0.75, p value=0.460).
This comprehensive systematic review identified 338 economic evaluations of CIM; 204 of which were published recently (2001–2010) covering a wide range of CIM therapies for a variety of populations. Although most patients who use CIM use more than one modality35 and despite the attention given to integrative medicine (coordinated access to conventional medicine and CIM),105 this systematic review found only one study that examined the effects of use of multiple CIM practitioners.52 In general, the quality of the recent full economic evaluations has held constant and is in line with what is seen in economic evaluations in conventional medicine. Details of the 31 recent higher-quality full economic evaluations indicate potential cost-effectiveness and cost savings across a variety of CIM therapies applied to different conditions. Owing to the non-generalisable nature of economic evaluations, the cost estimates shown are specific to their study settings.40 However, 22 articles provided at least the minimum information for study transferability. Therefore, their results could be adapted via modelling to determine the economic impact of these interventions in other settings.
The strengths of this study are the comprehensive search strategy, which revealed a substantial number of published economic evaluations of CIM, the use of two reviewers and the use of multiple measures of study quality. Higher-quality studies were identified and highlighted for policy makers using a simple objective list of quality criteria, which reduced the potential for bias. The weaknesses of this study are similar to those of the other systematic reviews. The reviewers were not blinded to journals and article authors, which may have influenced results. Also, some aspects of what makes a quality economic evaluation could not be judged from what was reported. For example, ideally, pragmatic trials enrol patients typical of normal caseload in typical settings with typically trained and experienced practitioners following them under routine conditions (ref.37, p. 251). Judgements as to whether these criteria were met were not always possible from the reports, and were beyond the scope of this review. Finally, publication bias was not assessed. However, since the major goal of this study was to establish the extent of the published literature on this topic and to highlight the results of the higher-quality studies, it is not clear that publication bias is relevant here.
The number of economic evaluations of CIM found and reviewed by this study far exceeds the numbers found in previous studies.11 19–21 23 This study found a total of 338 economic evaluations of CIM published between and including 1979 and 2010; 211 of these were full economic evaluations. White and Ernst19 identified 34 economic evaluations of CAM published 1987–1999; 11 of which were full economic evaluations. Between 1999 and October 2004, Herman et al20 identified 56 economic evaluations of CAM (39 full evaluations). Between 1994 and May 2004 Hulme and Long21 identified 19 full economic evaluations of CAM, and over a similar period (1995–2007) Doran et al23 found 43 economic evaluations (15 full evaluations). Maxion-Bergemann et al11 identified 5 (1 full) economic evaluations over an unspecified search period. The large number of economic evaluations found in this study reflects the facts that: (1) all evaluations from previous reviews were included; (2) a number of studies have been published since the last search dates of prior reviews and (3) a more extensive search strategy was used. It should be noted that 20% of the articles (68 of 338) in this review were identified through bibliography searches and from expert lists. Therefore, even the application of a long list of search terms to multiple databases does not guarantee that all CIM studies will be identified. However, there is some evidence that the indexing of these articles in medical databases is improving; studies from bibliographies and expert lists made up 32% of found articles published 2000 and before, but only 12% recent articles.
There are several implications of this study for policy makers, clinicians and future researchers. First, there is a large and growing literature of quality economic evaluations in CIM. However, although indexing is improving in databases, finding these studies can require going beyond simple CIM-related search terms. Second, the results of the higher-quality studies indicate a number of highly cost-effective, and even cost saving, CIM therapies. Almost 30% of the 56 cost-effectiveness, cost-utility and cost-benefit comparisons shown in table 4 (18% of the CUA comparisons) were cost saving. Compare this to 9% of 1433 CUA comparisons found to be cost saving in a large review of economic evaluations across all medicine.106 Third, by meeting the five study-quality criteria, the studies shown in table 4 can each be considered a reasonable indicator of the health and economic impacts of the CIM therapy studied, at least in that population and setting. These studies, especially those showing cost savings, should be considered further for applicability in other settings. This requires the study to be transferable.39 Fortunately, the majority of the higher-quality studies met our measure of study transferability—resource use or model parameters, and unit costs were reported separately.
Given the substantial number of economic evaluations of CIM found in this comprehensive review, even though it can always be said that more studies are needed, what is actually needed are better-quality studies—both in terms of better study quality (to increase the validity of the results for its targeted population and setting) and better transferability (to increase the usefulness of these results to other decision makers in other settings). Therefore, the following recommendations are made.
The authors wish to acknowledge and most gratefully thank Sandy Kramer of the University of Arizona Health Sciences Library for her assistance in the development and application of the search strategy and for eliminating duplicates from the search results. We would also like to thank Robert Scholten and P Scott Lapinski of the Harvard Medical School for their assistance with the EMBASE searches.
Contributions: PMH conceived of the idea for the paper, designed the search strategy, reviewed the references found, extracted the data from each included article and is the guarantor for this study. In parallel, BLP also reviewed the references found, extracted data from included articles and worked with PMH to resolve any discrepancies between reviewers. CMW provided practical insight and an international perspective to the design of the paper and interpretation of results. DME participated in the early design of the study, including the data extraction plan, inclusion/exclusion criteria and the interpretation of results. All authors contributed to the drafting and editing of the manuscript.
Funding: The Bernard Osher Foundation supports a portion of DME's time for research in integrative medicine. The Foundation had no control or influence over the design or execution of this study, nor no input into this manuscript.
Competing interests: None.
Provenance and peer review: Not commissioned; externally peer reviewed.
Data sharing statement: The full list of found articles is available in a word document from the corresponding author.