Search tips
Search criteria 


Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
Nurs Res. Author manuscript; available in PMC 2010 November 1.
Published in final edited form as:
PMCID: PMC2880328

Methodological Issues in Trials of Complementary and Alternative Medicine Interventions

Alla Sikorskii, PhD, Assistant Professor, Gwen Wyatt, RN, PhD, Professor, David Victorson, PhD, Research Assistant Professor, Gwen Faulkner, BA, Research Assistant, and Mohammad Hossein Rahbar, PhD, Professor of Epidemiology and Biostatistics



Complementary and alternative medicine (CAM) use is widespread among cancer patients. Information on safety and efficacy of CAM therapies is needed for both patients and health care providers. Well-designed randomized clinical trials (RCTs) of CAM therapy interventions can inform both clinical research and practice.


To review important issues that affect the design of RCTs for CAM interventions.


Using the methods component of the Consolidated Standards for Reporting Trials (CONSORT) as a guiding framework, and a National Cancer Institute-funded reflexology study as an exemplar, methodological issues related to participants, intervention, objectives, outcomes, sample size, randomization, blinding, and statistical methods were reviewed.


Trials of CAM interventions designed and implemented according to appropriate methodological standards will facilitate the needed scientific rigor in CAM research. Interventions in CAM can be tested using proposed methodology, and the results of testing will inform nursing practice in providing safe and effective supportive care and improving the well-being of patients.

Keywords: complementary and alternative therapies, methodology

Patients have been using complementary therapies since before the beginning of nursing as a profession (American Nurses Association & American Holistic Nurses Association, 2007); however, it has only been since the early 1990s that health care providers have acknowledged their widespread use (Eisenberg et al., 1998). The use of complementary and alternative medicine (CAM) has become prevalent, with estimates of about 1 in 3 people in the United States (or 72 million adults) using CAM (Tindle, Davis, Phillips, & Eisenberg, 2005). Despite popular use of CAM as an adjunct to conventional medical services, most patients do not disclose the use to their health care providers (Eisenberg et al., 1998), with especially high rates of patient nondisclosure among members of ethnic minority groups (Chao, Wade, & Kronenberg, 2008). The out-of-pocket expenditures for CAM are estimated at roughly $34 billion dollars annually (MacLennan, Wilson, & Taylor, 2002). Although only a decade ago this figure was comparable to out-of-pocket expenditures for conventional U.S. medical care (Eisenberg et al., 1998), CAM therapies appear to be increasingly affordable due to skyrocketing health care costs, insurance premiums, and out-of-pocket payments (Pagan & Pauly, 2005). With such high rates of use, it is vital that reliable information is available regarding the safety and efficacy of CAM therapies. While many CAM therapies offer safe and effective care, others may pose risks that health care providers must discuss with patients (Angell & Kassirer, 1998). In particular, CAM therapies can be offered as supportive care in treatment of cancer and other chronic conditions (Ernst, Schmidt, & Baum, 2006). Well-conducted CAM research is needed to inform evidence-based clinical practice.

To date, many CAM therapy research findings have been inconsistent due to a lack of standardization in the design and scientific methodology of trials. Early studies were often descriptive in nature and included heterogeneous populations of patients. While there is debate on how best to study CAM (Buchanan et al., 2005; Efficace et al., 2006), there is general agreement that randomized clinical trials (RCTs) can provide valid and reliable evidence that informs clinical research and practice.

The purpose of this paper is to review the methodological issues arising in trials of CAM therapies. The methods component of the Consolidated Standards for Reporting Trials (CONSORT) statement serves as a guiding framework (Altman et al., 2001; Begg et al., 1996). First introduced in 1996, and revised in 2001, the CONSORT contains a set of recommendations for the reporting of RCTs. Quality reporting is crucial for the identification of trials with poor design and methodological issues. Results from such trials may be biased, and these biases may negatively affect health decision making of patients, health care providers, and health policy makers. (Altman et al., 2001). Studies in CAM that utilize a RCT design rely upon methods that are identical to other RCTs and can adhere to the same standards as RCTs of conventional medicine interventions (such as the ones listed in the CONSORT). However, some of the specific methodological issues are unique and particularly difficult to address with CAM therapies. The CONSORT includes a checklist and a diagram for documenting the flow of participants through the trial. The checklist covers all areas of the trial report: title and abstract, introduction, methods, results and discussion, and is endorsed by many top health journals, including Nursing Research. Using the checklist of the methods component of the CONSORT, the challenges in designing RCTs with CAM therapies are reviewed, examples of approaches that can address these challenges are provided, and areas where further research is needed to improve the rigor of RCTs are identified.

To illustrate the basic principles of sound CAM research methodology within an RCT as defined in the CONSORT statement, a National Cancer Institute-funded multisite study currently being conducted by the authors (Reflexology: An Intervention for Advanced Breast Cancer, R01, CA104883) was used as an exemplar. In this multisite study, the target is women with advanced breast cancer undergoing chemotherapy, who upon consent and completion of a baseline interview, are randomized into one of three study groups: reflexology, placebo, or control. Reflexology, the CAM intervention evaluated in this RCT, is defined as a method of using thumb pressure on reflex areas of the feet which correspond to the glands, organs, and parts of the body (Byers, 1996). Women in the reflexology and placebo groups are blinded to their study group assignment and receive four standardized weekly foot sessions delivered by specially trained providers (reflexologists or placebo providers). Data on the primary outcomes of quality of life (QOL), and symptoms in particular, are collected prospectively at three time points: baseline; immediately following a series of foot sessions (7 weeks after baseline); and 13 weeks after baseline. To date, 307 women have been enrolled in the study, with the target recruitment of 450.


The characteristics of interventions undergoing testing lead to specific issues and challenges investigators must address when designing and implementing RCTs of CAM therapies. The eight key points of the methods portion of the CONSORT checklist include the following: (a) participants; (b) intervention; (c) objectives; (d) outcomes; (e) sample size; (f) randomization procedure; (g) blinding; and (h) statistical methods. These points correspond to items 3-12 in the CONSORT checklist, and we cover the aspects of these items relevant to CAM intervention trials.


Choice of a target population

The methods checklist of the CONSORT begins with eligibility criteria for the participants. As with any intervention, it is important to specify the target population clearly. While the use of CAM therapies has grown in recent decades, there are certain subgroups of the population that are much more or much less likely to be interested in participating in CAM studies. Examples of subgroups of people who are more likely to be interested in participating include women (Richardson, Sanders, Palmer, Greisinger, & Singletary, 2000), people with at least some college education (Boon, Olatunde, & Zick, 2007), people with specific health problems (Astin, 1998), and youth (Cassileth & Vickers, 2005). Further, there may be subgroups of patients who initially may be interested in participating in a trial and provide consent but drop out before completing the study. Among women with advanced breast cancer, one such subgroup in the reflexology trial is Black women (Sikorskii, Wyatt, Siddiqi, & Tamkus, 2009, in press). Pilot data are critical in providing such information that can be used in selecting a target population.

The importance of pilot data is best demonstrated by a quasi-experimental study that informed the current reflexology RCT. When cancer patients were offered a choice of one or more CAM therapies, those with lower well-being and higher levels of anxiety were attracted to guided imagery (Wyatt, Sikorskii, Siddiqi, & Given, 2007). The attrition rate among those who chose this therapy was 41%, and those who dropped out reported significantly worse symptoms of depression, anxiety, and physical well-being compared to those who stayed in the guided imagery group. Such results would threaten the external validity of the findings in a RCT because of potential attrition bias, or reliance on the assumptions of missing data techniques, which are not always verifiable (Senn, 2007). In addition, with a high attrition rate, a RCT would have unnecessary cost from wasted research resources spent on participants who did not complete the study, resulting in lack of longitudinal data.

In the CONSORT framework, one of the keys to ensuring external validity of the RCT findings is specifying the set of inclusion and exclusion criteria reflecting the target population likely to complete the intervention. For example, in a pilot study conducted with a heterogeneous population of cancer patients (Wyatt et al., 2007), it was found that more women than men were interested in receiving reflexology, and women with breast cancer were more interested than women with other solid tumors. Therefore, the inclusion criteria for the reflexology study were formulated so that the participants included women with late-stage breast cancer who were undergoing chemotherapy. Exclusion criteria for patients to enter the study included regular use of CAM similar to reflexology because it would prevent effective blinding of the participants as discussed later in this paper. Once the study participants are recruited, it is crucial to assess their characteristics, because one of the difficulties in CAM research is that people who agree to participate in research studies have strong preferences for using CAM and may not agree to being randomized, leading to a threat to external validity of findings (Ziegler, 2009).


The CONSORT requires the specification of details of the interventions for each group of the trial, and how and when the interventions were administered. These details include the timing and duration of the interventions, how the placebo was disguised and who delivered it, as well as who delivered the interventions. We discuss the specifics of these points as related to CAM therapies below.

Dose of the intervention

The term “dose” is not used in the CONSORT, but timing and duration of the intervention are mentioned. Timing and duration are related to dose, particularly with CAM. It is very difficult to quantify the intensity of many CAM therapies. The definition of dose is much clearer for CAM interventions from the biologically based category such as vitamins and herbal supplements. However, a rigorous definition of dose is needed also for RCTs of CAM therapies from other major CAM categories: mind-body interventions, energy therapies, manipulative and body-based methods, and alternative medical systems. In determining the dose, three critical dimensions have to be considered: number of sessions of the intervention, the duration of each session, and the amount of time between sessions. However, for most of the therapies mentioned, dose can be specified only in terms of the number of sessions delivered by the provider (e.g., for acupuncture) or self-administered by a patient (e.g., for mindfulness meditation).

During planning of the exemplar reflexology study, review of the reports of the three other reflexology studies conducted in the US (Ernst & White, 2000; Oleson & Flocco, 1993; Stephenson, Weinrich, & Tavakoli, 2000) revealed that duration of session dose has varied from 30 to 40 minutes; 30-minute sessions were used in the two cancer-specific studies. In addition, in these three studies 1 to 8 sessions per study were conducted, so this wide variation provided less guidance on frequency of sessions. Either 1 or 3 sessions, however, were used in the two cancer studies. In pilot work with 100 cancer patients (Wyatt et al., 2007), the weekly sessions were evaluated and significant changes on key outcome variables were observed. Therefore, based on the pilot work, the existing literature, and the expert's dose recommendation (drawn from 22 years of reflexology practice as a certified reflexologist), a dose of one time per week for 4 weeks (30 minutes per session) was implemented. In determining the dose, the number of sessions and duration of the intervention have to be adequate to allow for meaningful change to occur in primary and secondary outcomes.

Distinguishing therapeutic and placebo effects

Beyond determining the dose for the intervention group of the reflexology study, it was necessary to determine comparison groups. A placebo foot protocol was developed to examine the added attention versus therapeutic effects of the CAM intervention, just as necessary for conventional medicine interventions. However, the techniques for achieving this may be therapy-specific and differ from techniques used with conventional medicine interventions. For example, the choice of placebo pill frequently used in drug trials may be complicated in trials of herbal supplements. One of the difficulties is the distinct smell of many supplements (e.g., ginger; Zick et al., 2008) that people are familiar with. If the placebo pill does not have this distinctive smell, blinding would be jeopardized, while if the placebo pill has the true smell, the placebo may be contaminated with the active ingredient.

The placebo session should include all the same phases as the experimental CAM session (i.e., preparation for session, timing of the session, and concluding the session). The placebo, however, should not involve any of the active ingredients of the experimental CAM intervention but should appear similar, if not equal. In the exemplar reflexology trial, placebo group participants receive a foot manipulation that does not target specific reflexology points of the foot and does not utilize techniques specific to reflexology. However, while avoiding active elements of reflexology, placebo participants may experience a therapeutic effect above and beyond attention through human touch and massage-like manipulation. The inclusion of a control group in the design provides statistical control over the changes in outcomes over time that occurs from the natural course of disease and its treatment. In addition, due to randomization, potential confounding factors will be distributed equally across study groups. Many of these potential confounding factors are identified and measured. For example, data on the use of conventional medicine pharmacological agents are collected from the medical records at the end of the study. The use of these agents could affect outcomes, but would be expected to be the same in the experimental and control groups. The comparison of these groups in the analysis would allow for the identification of the effect of the intervention over and above other factors that are present equally in study groups.

Treatment fidelity

Treatment fidelity is defined as adherent and competent delivery of the intervention by the interventionist as set forth in the research plan (Santacroce, Maccarelli, & Grey, 2004). While the CONSORT does not include the term “fidelity,” the requirements of careful description of the interventions and specification of who delivered them reflect treatment fidelity. In designing and implementing CAM RCTs, it is important to provide standardized training for both the treatment and placebo providers. In the reflexology study, a procedure manual for the intervention was developed to outline all study protocol steps, and all providers are trained based on the manual. Training also includes didactic information, written steps, role-playing, demonstrations, and return demonstrations based upon an established protocol criterion for the treatment. In many types of CAM interventions (e.g., massage, therapeutic touch, reflexology), the influence of social supportive factors can contribute (and add noise) to the therapeutic effect of the treatment. The providers are trained to leverage and manage interpersonal interactions that may occur during the treatment itself. Treatment and placebo providers are instructed not to initiate any conversation once the intervention begins, but rather to instruct the patient to relax and close her eyes. If the patient persists in conversation, the topics are kept general, such as the weather or current events, responding with pleasant closed-ended comments. All sessions are conducted in a similar environment, whether in the patient's home or a clinic. With multiple sites involved, quality assurance procedures are built into the design so that all intervention protocols are administered uniformly across sites.


The original CONSORT statement and its modifications call for the specification of the questions that the trial was designed to answer. The a-priori formulation of hypotheses is especially important in view of trials designed to test equivalence or non-inferiority as opposed to superiority of an intervention to a placebo or control. Study hypotheses clearly distinguish superiority designs where an intervention is hypothesized to be better than the placebo or control (or both) from equivalence designs where two or more interventions are hypothesized to differ by no more than a pre-specified margin. When study hypotheses are clearly stated, a non-significant result can not be incorrectly interpreted as evidence of equivalence. According to Altman (one of the developers of the CONSORT) and Bland's British Journal of Medicine publication title, “…absence of evidence is not evidence of absence” (Altman & Bland, 1995).

In the exemplar reflexology study, which is a superiority trial, the objective is to test the value-added effect of reflexology to conventional care alone, and the study is a superiority trial. The placebo group was included to be certain the effect is based on the active ingredient of reflexology and not the attention given during the reflexology sessions. All three groups receive conventional medical care. The following hypotheses were formulated a priori based on the following objectives: (a) reflexology will have significantly better outcomes than control; (b) reflexology will have significantly better outcomes than placebo; and (c) placebo will have significantly better outcomes than control.


Item 6 of the CONSORT checklist requires a clear definition of primary and secondary outcomes, and how and when data on these outcomes are collected. This item is important as it defines the approaches to sample size calculations and statistical analyses as described later.

Primary outcomes

Patients turn to CAM therapies for a variety of reasons, one of which is seeking to improve QOL and manage symptoms from diseases or treatments. It was reported recently that more than 80% of women diagnosed with breast cancer turn to CAM for symptom management (Boon et al., 2007). The choice of QOL as a primary outcome in a CAM therapy trial is consistent with patients' goals to reduce disease or treatment-related symptoms and side effects and improve health-related QOL. In addition, using QOL as a primary outcome of a CAM therapy trial is supported by regulatory agencies such as the Food and Drug Administration (U.S. Department of Health and Human Services (FDA), 2006) and, specifically in cancer, by the American Society of Clinical Oncology (American Society of Clinical Oncology, 2001).

In the exemplar reflexology study, the primary outcome of health-related QOL is conceptualized as a holistic multidimensional interaction of life domains: physical, psychological, social, and spiritual. Within each QOL domain, specific components are recognized. The components of QOL are symptoms, functional status, and general health perceptions, such as satisfaction with life (Wilson & Cleary, 1995). Each QOL domain (physical, psychological, spiritual, social) contains these components in association with one another. For example, higher levels of symptom severity have been shown to be associated with worse functioning (Dodd, Miaskowski, & Paul, 2001). Among cancer patients in particular, reduction in symptom severity can help maintain treatment regimen and ultimately may affect survival (Campagnaro et al., 2008). The most prevalent symptoms reported by breast cancer patients in the exemplar reflexology study include fatigue, weakness, pain, insomnia, peripheral neuropathy, poor appetite, change in taste, dry mouth, difficulty concentrating, memory problems, restlessness, and mood changes. Associations exist among multiple symptoms, and a holistic approach of treating the mind, body, and spirit is at the center of CAM.

Assessments of multiple symptoms and other components of the QOL domains can be included as part of the trial design. The construct that summarizes a patient's symptom experience is referred to in the literature as symptom burden. Symptom burden is defined as a summary of subjective expression of the disease itself or the products of disease treatment (side effects or toxicities; (Cleeland, 2007). Measures of symptom burden created by multiple symptoms include severity of symptoms and a summary of the patient's perception of the impact of these symptoms on daily living. These measures may be sufficient outcomes particularly in populations experiencing deterioration in health; for example, end-of-life populations where an improvement in QOL may not be expected (Cleeland, 2007). A reduction in symptom burden by a CAM therapy intervention may be achieved with or without an immediate impact on other QOL components, just as is the case with trials of pharmacological interventions (Sloan et al., 2007). If a therapy is proven to reduce severity of a specific symptom or symptoms, it is beneficial to patients even if other QOL components fail to improve (Jatoi, Kumar, Sloan, & Nguyen, 2003). In the reflexology study, measures of pain and fatigue are included, as well as measures of other symptoms commonly reported by women with breast cancer undergoing chemotherapy.

Repeated measure outcomes

The flow diagram of the CONSORT requires specification of the numbers of participants who completed each step. This includes longitudinal assessments that allow for determining not only if the effect of the intervention is present, but also if the effect is sustained over time. In addition, data collection at the time points within the intervention can be performed. For instance, with the reflexology study that consists of 4 sessions over a period of 5 weeks, data could be collected at each of the 4 sessions. Following each intervention session, patient reported outcome (PRO) data and provider-reported data on delivery of the intervention can be collected. Analysis of these data can provide important information as to when and how changes in outcomes occur. This information complements the CONSORT diagram that displays the numbers of patients who were allocated to and completed the intervention and placebo protocols. Longitudinal analyses of intervention sessions can shed light on mechanisms of action of CAM therapies by answering questions about when and how changes in QOL occurred and which symptoms responded to interventions under examination.

Secondary outcomes

The CONSORT statement is applicable to trials of interventions targeting a wide range of conditions, and numerous outcomes related to these conditions can be evaluated. In CAM trials where QOL, including symptoms, is the primary outcome, important secondary outcomes may be time in days or weeks needed to achieve a pre-specified level of clinically meaningful effect. Such outcomes are referred to as time-to-event. For these types of outcomes, it is possible that for some participants the time-to-event of interest may not be observed during the study period, resulting in censored data. Such data can be analyzed using survival methods (Sikorskii, Given, You, Jeon, & Given, 2009, in press). In cancer research, survival analysis deals not only with survival time, but also with times-to-events other than death (e.g., time-to-recurrence, time-to-disease progression, or time-to-treatment failure). These outcomes have been used in studies testing effects of various conventional medicine interventions (Punt et al., 2007), but RCTs of CAM interventions have not included time-to-response or time-to-other events as outcomes.

Another important secondary outcome in CAM research can be the use of health services. For example, with cancer patients, effective symptom management can be evaluated, not only by assessing the effectiveness of a CAM therapy on symptom reduction or control, but also through observing lower numbers of emergency room visits and hospitalizations due to complications or symptom exacerbations. Another significant effect of a CAM intervention may be helping patients stay on treatment longer, without dose delays or reductions of chemotherapy. These types of clinical data can be invaluable in placing greater real world meaning and context behind a person's self-reported fatigue or pain scores.

Sample Size

Sample size is a critical factor in any RCT, and CAM RCTs must hold to comparable standards. Sample size must be hypotheses-driven and, when possible, based on pilot work as well as clinical meaning of the hypothesized difference in outcomes between study groups. The balance between clinical importance of the differences between trial arms and statistical significance considerations is emphasized by explanation of item 7 in the CONSORT framework. Attrition rates for the population also must be a consideration. The exemplar reflexology study was powered to detect meaningful differences between the groups at the end of the intervention (7 weeks). Sample size calculations were based on medium effect size of 0.4 for pair-wise group comparisons listed in study hypotheses.

There were two considerations in choosing the effect size. First, such differences between groups were deemed attainable under the reflexology intervention as shown in pilot work. Second, the differences in outcomes that correspond to this effect size were deemed clinically meaningful as defined in the literature (Cella, Hahn, & Dineen, 2002). The sample size calculation, using a 5% level of significance and based on power of at least 80%, resulted in the requirement of 100 women per group, or a total of 300. However, attrition is a factor in all RCT, especially in those that target an end-of-life population where CAM therapy may be used as supportive care. A plan to oversample and recruit 450 women was put in place based on projected 33% attrition rate in the population of women with advanced breast cancer. Sample size consideration such as those mentioned here are necessary to produce results that are clinically meaningful and statistically significant.

Randomization Procedure

Creating similar groups at baseline is at the core of the causal inference from a RCT. When outcomes are compared post intervention, the difference between study groups is attributed to the intervention because all other factors that may be associated with outcomes were expected to be similar across groups. However, there are multiple factors that may influence the outcomes, and balancing them across groups may not be achieved with simple randomization. (Senn, 2007). These factors are referred to as prognostic factors, and a set of the most important ones should be identified based on the literature and pilot work. Many randomization procedures involve specific strategies to balance such factors as it is impossible to control all variable distributions across groups.

One approach to balancing the groups with respect to prognostic factors is stratification, and running separate randomization procedures within each stratum. However, as the number of prognostic factors and their levels increase, stratification becomes less feasible. To remedy this problem, a minimization procedure was proposed by Taves (1974); it is known as adaptive randomization. To randomize a patient, the procedure is used to quantify the current imbalance among groups based on all specified prognostic factors, and the patient is allocated so that the imbalance among the groups is minimized at each step. With some minimization procedures, the first patient is allocated at random, and later allocations are determined by patient characteristics as well as the distribution of prior patients in the study arms. With other minimization procedures, the allocation remains random, but the probabilities of patient being assigned to each group are adjusted at each step to increase or decrease the chance of allocating the patient to one group compared to others if needed, based on the imbalance score. The explanation of the CONSORT items (Altman et al., 2001) describes pure minimization where only the first patient is allocated at random, and suggests that minimization is an acceptable and sometimes superior alternative to randomization. Adaptive randomization as described above and implemented in the reflexology study is even closer to randomization with respect to eliminating biases, yet it provides control over the distribution of prognostic factors across groups, something that simple randomization may not do.

In the reflexology study, prognostic factors used in the minimization procedure included the recruitment site, levels of pain and fatigue, and goal of therapy to assure the creation of three comparable groups at baseline. These variables were selected since they may be predictive of the outcomes. The recruitment site variable has 14 categories (one for each participating clinic in the Midwest). The selection of this variable as a prognostic factor was driven by the possibility of differences in standard care across different oncology settings. The CONSORT specifically mentions these potential differences, and the need to consider them and report the information on recruitment sites to provide evidence of external validity of the findings. Further, large numbers of cancer patients turn to CAM for symptom management and so symptoms are important variables to consider as potential prognostic factors in CAM trials. Among multiple symptoms, pain and fatigue were selected as most prevalent for the end-of-life population, and often are associated with other symptoms (Given, Given, Azzouz, Kozachik, & Stommel, 2001). Pain and fatigue are categorized into two levels (mild or moderate versus severe) based on the established cut-points for the greatest interference with daily life (Mendoza et al., 1999; Zelman, Smith, & Miaskowski, 2005). The goal of therapy has four categories (curative, maintenance, palliative, and uncertain). Depending upon the goal of therapy, the selection of chemotherapy agents may vary. For instance, curative chemotherapy may be more aggressive than palliative and produce more severe symptoms.


As stated in the CONSORT, blinding may be done in relation to data collectors, intervention providers, and patients. The interviewers can be blinded as they collect data to avoid any comments that may result in bias. The providers of the intervention can be blinded when they are not an expert in the intervention. Finally, patients can be blinded to their group assignment.

In the reflexology study, all interviewers (telephone data collectors) are blinded to which study group women are assigned to. It was not possible to blind the expert reflexologists to the intervention they were providing. Both types of providers (reflexologists and placebo) were trained not to disclose any information regarding the study group to the participants, maintaining participant blinding during the intervention. The key component to success in blinding of patients in the reflexology study is exclusion of women who are using reflexology or similar therapies regularly, because these women would be able to tell the difference between true reflexology and placebo. When blinding is in place, the assessment of the success of blinding is included in the CONSORT. In the reflexology study, at the end of the final interview of the study, women in the reflexology and placebo groups are asked to guess their study group allocation, and were debriefed on group assignment. Probabilities of correct guesses are monitored by the study team as an additional safeguard for the integrity of blinding and the fidelity of the intervention.

Statistical Methods

The requirement of the specification of statistical methods used to produce an estimate of the effect of the intervention concludes the list of items in the methods section of the CONSORT framework. In the exemplar reflexology study, standard statistical methods are used in the initial comparison of participants assigned to the three study groups (reflexology, placebo, and control). First, prognostic factors are monitored. With minimization procedure in place, the study groups have similar distributions of these variables. Further, other measures that could reflect potential confounders are compared across study groups including medical record data such as comorbid conditions, chemotherapy protocols, and administration of medications to manage symptoms. The use of CAM therapies is compared also across study groups, because patients who turn to CAM often use more than one CAM therapy (Wyatt et al., 2007).

All of these variables have the potential to impact the outcome variables to be evaluated post-experiment. If differences between study groups are observed despite computerized minimization, the relevant variables will be included as covariates in the analyses. An important specification of analysis listed in the CONSORT is whether or not the analysis is intent to treat. Intent to treat strategy provides a conservative estimate of the effect of the intervention since women are analyzed as randomized regardless of their adherence to the protocol (i.e., number of foot sessions completed). The study groups will be compared on primary and secondary outcomes using generalized linear models that include study group, baseline value of the outcome, prognostic factors, and any variables that have found to have a different distribution across study groups at baseline or from medical records. The inclusion of baseline values of outcomes and prognostic factors will enable control for pre-intervention influences, and allow for the tests of the effect of intervention over and above the influences of these factors (Senn, 2007). These analyses methods are used widely in trials of conventional medicine interventions, and would facilitate rigorous evaluation of CAM therapies.


While the CONSORT statement was designed as a guide for trial reporting and not as a quality assessment instrument, it includes items that are crucial to securing internal and external validity of a RCT. In turn, the findings from the trial are only as good as the internal and external validity of the trial itself. By considering the items from the methods section of the CONSORT, high quality trials can be designed, and then successfully implemented. The findings from well designed and implemented trials provide a solid evidence base for the intervention tested to be translated to clinical practice. Thus the CONSORT serves as a conceptual guide for methodological considerations for trial designs and contributes to the growing evidence base of CAM interventions. For cancer patients, CAM therapies may provide much needed supportive care during and following treatment, and it is critical that exploratory, dose-ranging, and efficacy CAM trials utilize high quality methods to ensure the highest degree of standardization possible.

The exemplar reflexology study described throughout illustrates how these methods can be implemented effectively to bring transparency to design issues. It is also important to note that the methods section of the CONSORT checklist does not include data management and data quality assurance, the key items in trial implementation. Inclusion and elaboration of these considerations are important, particularly for multisite trials. As the number of multisite and multicenter trials increase, it will be beneficial to expand the CONSORT checklist to include data management and data quality assurance in order to significantly improve the quality of data reported from multicenter and multisite clinical trials. Since the exemplar reflexology intervention study is a multisite trial, steps have been taken to ensure the data, and these important points will be included in future publications from this project.

Designing and conducting CAM interventions can be challenging and requires a great deal of forethought and planning to account for the myriad of threats to internal and external validity often inherent to CAM RCTs. Although a certain degree of study design flexibility may be necessary to maintain the integrity of a CAM intervention in a clinical trial, the highlighted methodological approaches can be incorporated with little to no effect on the overall authenticity of the CAM practice under study. As nursing professionals continue to address the challenges of today's ever-changing health care environment, being equipped with an armament of effective CAM tools that are based on well-designed research can help reinforce and support the lives of patients.


This research was supported by grant R01 CA104883 from the National Cancer Institute. The contents represent original work and have not been published elsewhere. No commercial party having a direct financial interest in the results of the research supporting this article has or will confer a benefit upon the authors or upon any organization with which the authors are associated.


EDITOR'S NOTE. Materials documenting the review process for this article are posted at

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

Contributor Information

Alla Sikorskii, Department of Statistics and Probability, College of Natural Science, Michigan State University, East Lansing, Michigan.

Gwen Wyatt, College of Nursing, Michigan State University, East Lansing, Michigan.

David Victorson, Department of Medical Social Sciences, Northwestern University Feinberg School of Medicine, Chicago, Illinois.

Gwen Faulkner, Center on Outcomes, Research and Education, NorthShore University HealthSystem, Evanston, Illinois.

Mohammad Hossein Rahbar, The University of Texas School of Public Health at Houston, Director, Biostatistics/Epidemiology/Research Design (BERD) Core Center for Clinical and Translational Sciences, The University of Texas Health Science Center at Houston, Houston, Texas.


  • Altman DG, Bland JM. Absence of evidence is not evidence of absence. BMJ. 1995;311(7003):485. [PMC free article] [PubMed]
  • Altman DG, Schultz KF, Moher D, Egger M, Davidoff F, Elbourne D, et al. The revised CONSORT statement for reporting randomized trials: Explanation and elaboration. Annuals of Internal Medicine. 2001;134 [PubMed]
  • American Nurses Association & American Holistic Nurses Association. Holistic nursing: Scope and standards of practice. American Nurses Association. 2007;1
  • American Society of Clinical Oncology. Quality of life: Symptom management an essential component of cancer care. [Electronic Version] 2001. Retrieved December 31, 2008 from
  • Angell M, Kassirer JP. Alternative medicine - the risks of untested and unregulated remedies. The New England Journal of Medicine. 1998;339:839–841. [PubMed]
  • Astin JA. Why patients use alternative medicine: Results of a national study. The Journal of the American Medical Association. 1998;279:1548–1553. [PubMed]
  • Begg C, Cho M, Eastwood S, Horton R, Mher D, Olkin I, et al. Improving quality of reporting of randomized controlled trials. The CONSORT statement. Journal of the American Medical Association. 1996;276:637–639. [PubMed]
  • Boon HS, Olatunde F, Zick SM. Trends in complementary/alternative medicine use by breast cancer survivors: Comparing survey data from 1998 and 2005. BMC Women's Health. 2007;7:4–10. [PMC free article] [PubMed]
  • Buchanan DR, White JD, O'Mara AM, Kelaghan JW, Smith WB, Minasian LM. Journal of Clinical Oncology. 6682-6689 Vol. 23. 2005. Research-design issues in cancer-symptom-management trials using complementary and alternative medicine: Lessons from the National Cancer Institute Community Clinical Oncology Program experience. [PubMed]
  • Byers D. Better health with foot reflexology: The original Ingham Method. St. Petersburg: Ingham Publishing, Inc.; 1996.
  • Campagnaro E, Saliba R, Giralt S, Roden L, Medoza F, Aleman A, et al. Symptom burden after autologous stem cell transplantation for multiple myeloma. Cancer. 2008;112:1617–1624. [PubMed]
  • Cassileth BR, Vickers AJ. High prevalence of complementary and alternative medicine use among cancer patients: Implications for research and clinical care. Journal of Clinical Oncology. 2005;23:2590–2592. [PubMed]
  • Cella DF, Hahn EA, Dineen K. Meaningful change in cancer-specific quality of life scores: Differences between improvement and worsening. Quality of Life Research. 2002;11:207–221. [PubMed]
  • Chao MT, Wade C, Kronenberg F. Disclosure of complementary and alternative medicine to conventional medical providers: Variation by race/ethnicity and type of CAM. Journal of the National Medical Association. 2008;100(1341-1349) [PMC free article] [PubMed]
  • Cleeland CS. Symptom burden: Multiple symptoms and their impact as patient-reported outcomes. Journal of the National Cancer Institiute Monographs. 2007:16–21. [PubMed]
  • Dodd MJ, Miaskowski C, Paul SM. Symptom clusters and their effect on the functional status of patients with cancer. Oncology Nursing Forum. 2001;28:465–470. [PubMed]
  • Efficace F, Horneber M, Lejeune S, Van Dam F, Leering S, Rottmann M, et al. Methodological quality of patient-reported outcome research was low in complementary and alternative medicine in oncology. Journal of Clinical Epidemiology. 2006;59:1257–1265. [PubMed]
  • Eisenberg DM, Daves RB, Ettner SL, Appel S, Wilkey S, Van Rompay M, et al. Trends in alternative medicine use in the United States, 1990-1997: Results of a follow-up national study. The Journal of the American Medical Association. 1998;280:1569–1575. [PubMed]
  • Ernst E, Schmidt K, Baum M. Complementary/alternative therapies for the treatment of breast cancer. A systematic review of randomized clinical trials and a critique of current terminology. The Breast Journal. 2006;12(526-530) [PubMed]
  • Ernst E, White A. The BBC survey of complementary medicine use in the UK. Complementary Therapies in Medicine. 2000;8:32–36. [PubMed]
  • Given CW, Given B, Azzouz F, Kozachik S, Stommel M. Predictors of pain and fatigue in the year following diagnosis among elderly cancer patients. Journal of Pain and Symptom Management. 2001;21(6):456–466. [PubMed]
  • Jatoi A, Kumar S, Sloan JA, Nguyen PL. On appetite and its loss. Journal of Clinical Oncology. 2003;21:79s–81s. [PubMed]
  • MacLennan AH, Wilson DH, Taylor AW. The escalating cost and prevalence of alternative medicine. Preventive Medicine. 2002;35:166–173. [PubMed]
  • Mendoza TR, Wang XS, Cleeland CS, Morrissey M, Johnson BA, Wendt JK, et al. The rapid assessment of fatigue severity in cancer patients: Use of the Brief Fatigue Inventory. Cancer. 1999;85:1186–1196. [PubMed]
  • Oleson T, Flocco W. Randomized control study of premenstrual symptoms treated with ear, hand, and foot reflexology. Obstetrics and Gynecology. 1993;2:906–911. [PubMed]
  • Pagan JA, Pauly MV. Access to conventional medical care and the use of complementary and alternative medicine. Health Affairs. 2005;24:255–262. [PubMed]
  • Punt CJ, Buyse M, Kohne CH, Hohenberger P, Labianca R, Schmoll HJ, et al. Endpoints in adjuvant treatment trials: A systematic review of the literature in colon cancer and proposed definitions for future trials. Journal of the National Cancer Institute. 2007;99:998–1003. [PubMed]
  • Richardson MA, Sanders T, Palmer JL, Greisinger A, Singletary SE. Complementary/alternative medicine use in a comprehensive cancer center and the implications for oncology. Journal of Clinical Oncology. 2000;18:2505–2514. [PubMed]
  • Santacroce SJ, Maccarelli LM, Grey M. Intervention fidelity. Nursing Research. 2004;53(1):63–66. [PubMed]
  • Senn S. Statistical Issues in Drug Development. 2. Chichester: John Wily & Sons; 2007.
  • Sikorskii A, Given C, You M, Jeon S, Given B. Response analysis for multiple symptoms revealed differences between arms of a symptom management trial. Journal of Clinical Epidemiology 2009 in press. [PMC free article] [PubMed]
  • Sikorskii A, Wyatt G, Siddiqi A, Tamkus D. Recruitment and early retention of women with advanced breast cancer in a complementary and alternative medicine (CAM) trial. Evidence-Based Complementary and Alternative Medicine 2009 in press. [PMC free article] [PubMed]
  • Sloan JA, Berk L, Roscoe J, Fisch MJ, Shaw EG, Wyatt G, et al. Integrating patient-reported outcomes into cancer symptom management clinical trials supported by the National Cancer Institute-sponsored clinical trials networks. Journal of Clinical Oncology. 2007;25:5070–5077. [PubMed]
  • Stephenson NL, Weinrich SP, Tavakoli AS. The effects of foot reflexology on anxiety and pain in patients with breast and lung cancer. Oncology Nursing Forum. 2000;27:67–72. [PubMed]
  • Taves DR. Minimization: A new method of assigning patients to treatment and control groups. Clinical Pharmacological Therapy. 1974;15:443–453. [PubMed]
  • Tindle HA, Davis RB, Phillips RS, Eisenberg DM. Trends in use of complementary and alternative medicine by US adults: 1997-2002. Alternative Therapies in Health and Medicine. 2005;11:42–49. [PubMed]
  • U.S. Department of Health and Human Services (FDA) Guidance for Industry. Patient-reported outcome measures: Use in medical product development to support labeling claims (Draft Guidance) Washington, D.C.: 2006. [PMC free article] [PubMed]
  • Wilson IB, Cleary PD. Linking clinical variables with health-related quality of life. A conceptual model of patient outcomes. The Journal of the American Medical Association. 1995;273:59–65. [PubMed]
  • Wyatt G, Sikorskii A, Siddiqi A, Given CW. Feasibility of a reflexology and guided imagery intervention during chemotherapy: Results of a quasi-experimental study. Oncology Nursing Forum. 2007;34:635–642. [PubMed]
  • Zelman D, Smith M, Miaskowski C. Categorizing the severity of cancer pain: Further exploration of the established cutpoints. Pain. 2005;113:37–44. [PubMed]
  • Zick SM, Ruffin MT, Lee J, Normolle DP, Siden R, Alrawi S, et al. Phase II trial of encapsulating ginger as a treatment for a chemotherapy-induced nausea and vomiting. Supportive Care in Cancer, Electronic publication ahead of print 2008 [PubMed]
  • Ziegler R. Mistletoe preparation iscador: Are the methodological concerns with respect to controlled clinical trials? Evidence-Based Complementary and Alternative Medicine. 2009;6(1):19–30. [PMC free article] [PubMed]