The net effect of alternative allocation ratios on recruitment time and trial cost

doi:10.1177/1740774509103485

Clin Trials. Author manuscript; available in PMC 2010 May 4.

Published in final edited form as:

Clin Trials. 2009 April; 6(2): 126–132.

doi: 10.1177/1740774509103485

PMCID: PMC2864093

NIHMSID: NIHMS191506

The net effect of alternative allocation ratios on recruitment time and trial cost

Ralitza Vozdolska,^a Mary Sano,^b Paul Aisen,^c and Steven D Edland^a^c

^a Department of Family and Preventive Medicine, University of California San Diego, La Jolla, CA, USA

^b Mount Sinai School of Medicine, New York, NY, USA

^c Department of Neuroscience, University of California San Diego, La Jolla, CA, USA

Author for correspondence: Steven D. Edland, Departments of Neuroscience and Family and Preventive Medicine, University of California San Diego, 9500 Gilman Drive MC0949, La Jolla, CA 92093-0949, USA. Email: sedland/at/ucsd.edu

The publisher's final edited version of this article is available at Clin Trials

See other articles in PMC that cite the published article.

Abstract

Background

Increasing the proportion of subjects allocated to the experimental treatment in controlled clinical trials is often advocated as a method of increasing recruitment rates and improving the performance of trials. The presumption is that the higher likelihood of randomization to the experimental treatment will be perceived by potential study enrollees as an added benefit of participation and will increase recruitment rates and speed the completion of trials. However, studies with alternative allocation ratios require a larger sample size to maintain statistical power, which may result in a net increase in time required to complete recruitment and a net increase in total trial cost.

Purpose

To describe the potential net effect of alternative allocation ratios on recruitment time and trial cost.

Methods

Models of recruitment time and trial cost were developed and used to compare trials with 1:1 allocation to trials with alternative allocation ratios under a range of per subject costs, per day costs, and enrollment rates.

Results

In regard to time required to complete recruitment, alternative allocation ratios are net beneficial if the recruitment rate improves by more than about 4% for trials with a 1.5:1 allocation ratio and 12% for trials with a 2:1 allocation ratio. More substantial improvements in recruitment rate, 13 and 47% respectively for scenarios we considered, are required for alternative allocation to be net beneficial in terms of tangible monetary cost.

Limitations

The cost models were developed expressly for trials comparing proportions or means across treatment groups.

Conclusions

Using alternative allocation ratio designs to improve recruitment may or may not be time and cost-effective. Using alternative allocation for this purpose should only be considered for trial contexts where there is both clear evidence that the alternative design does improve recruitment rates and the attained time or cost efficiency justifies the added study subject burden implied by a larger sample size.

Introduction

Timely recruitment is critical to the successful performance of any clinical trial. Failure to achieve recruitment goals can compromise statistical power or lead to an extension of the study time period and budget overruns [1,2]. Several strategies may be used to improve recruitment rates in clinical trials. Treves and colleagues [3] consider easing inclusion/exclusion criteria to expand the number of eligible patients and thus increase the recruitment rate. This approach is problematic in some contexts, for example, when stringent inclusion criteria are required to ensure the specificity of disease diagnosis or to ensure subjects are within the stage of disease progression that is the target of active treatment. Another strategy for increasing recruitment rates is to add study sites in an effort to access and capture a greater number of potential subjects. Drawbacks of this tactic include increased cost and the increased burden of management and coordination [4].

In contrast to adjustments of inclusion criteria or study infrastructure, strategies for improving recruitment yield focus on the factors influencing potential study enrollees’ decisions to decline trial entry [5]. Among potential enrollees’ primary concerns are randomization and whether or not the trial offers active treatment in all arms. Even when experimental treatments are offered on the background of current standard therapy, there may be incentives to being randomized to the presumably promising experimental treatment arm, and study designs that randomize more participants into the experimental arm may therefore increase the acceptability of the studies to potential enrollees and increase the yield of recruitment efforts [2,5–8].

Counterbalancing the potential advantages of a higher allocation ratio is the fact that a larger sample size is required to maintain statistical power for allocation ratios other than 1:1. A larger sample size has a cost in terms of study subject burden, and may also imply more time required for recruitment if the alternative allocation ratio does not boost recruitment rates sufficiently. In regard to total financial cost, changing the allocation ratio is net beneficial only if the recruitment window is reduced sufficiently to realize reduced administrative costs equal to the per subject costs associated with the higher sample size. Such cost considerations are relevant, as there is increasing pressure to justify clinical research from an economic and cost-effectiveness perspective [9–12].

In this article, we use an economic cost-minimization analysis [13] to describe the net effect of increasing the allocation ratio on recruitment time and clinical trial cost. Models of trial time and costs are developed and applied to a range of cost and recruitment rate scenarios. We look specifically at clinical trials comparing a continuous outcome or proportion across treatment arms after a specified period of follow-up time. The models developed here do not apply to trials with a survival analytic time to an event outcome. Similarly, we assume throughout that trials are powered under the assumption of equal variance in outcome across all treatment arms.

Alternative allocation ratios have been recommended for reasons other than to improve recruitment rates. For example, trials with multiple treatments tested against a single referent treatment arm are optimally efficient in terms of total sample size when the referent arm sample size is larger than the sample size of the comparison arms [14]. Trials may allocate more subjects to the experimental treatment arm to obtain more information about adverse events, or specifically to comply with Food and Drug Administration risk assessment guidelines prior to marketing a new drug [15]. Trials that anticipate a higher dropout rate in the treatment arm due to known treatment side-effects may over-recruit the treatment arm in compensation [16]. Finally, several authors have proposed alternative allocation ratios with fewer subjects allocated to the active treatment arm when the active treatment has special risks or costs compared to the control arm [17–19]. To the extent that allocation ratio affects recruitment rates, the cost models developed below can be used to aid in assessing the cost of performing these varied trials, although, we caution that the relative cost models developed below apply only to those circumstances where powering to detect a prespecified treatment effect size is the primary consideration driving trial design.

Methodology

The net cost effect of changing the allocation ratio is largely determined by three factors: (1) the total sample size, which has to be increased under unequal allocation to maintain statistical power; (2) the time period of active recruitment, which may increase or decrease depending on the effect of the allocation ratio on recruitment rates; and (3) the relative cost of active and placebo treatments. Although there are exceptions [17–19], the difference in cost between active and placebo treatment is negligible compared to total trial costs in most applications, so that it is the cost trade-offs of the first two factors above which determine if and when modifying the allocation ratio reduces the tangible costs of a clinical trial. These will be the focus of our modeling exercise.

Total sample size under unequal allocation

As described for example by Meinert [14], changing the allocation ratio from 1:1 requires an increase in sample size to maintain statistical power to detect a specified effect size. For a given sample size under a 1:1 allocation plan (sample size N_1:1 say), the sample size required to achieve comparable power under an alternative allocation strategy a:b (sample size N_a:b say), for many analysis plans is equal to

(1)

To simplify presentation, we maintain the usual assumption that variance is the same within the experimental and control arms. Formula (1) holds for studies comparing the mean of continuous outcomes (Meinert [14], equation (9.13)), for studies comparing the mean change from baseline of continuous outcomes (Meinert [14], equation (9.15)), and for studies with a binary outcome powered using the inverse sine transformation approximation to Fisher’s exact test (Meinert [14], equation (9.9)). By Equation (1) we can see, for example, that changing the allocation ratio to 1.5:1 requires about a 4% larger sample, and changing the ratio to 2:1 requires about a 12% larger sample to maintain statistical power.

Time period of active recruitment under unequal allocation

Given a recruitment rate under equal allocation (recruitment rate RR_1:1 say), the time period of active recruitment is equal to N_1:1/RR_1:1. The comparable figure for a trial with unequal allocation given a recruitment rate under the alternative design (recruitment rate RR_a:b say) is N_a:b/RR_a:b.

The time period of active recruitment under unequal allocation can be described as a percentage of the time period of active recruitment under equal allocation as

which, after substituting for N_a:b using Equation (1), simplifying, and rearranging terms, can be expressed as

(2)

The numerator of Equation (2) is the proportional increase in sample size required under the alternative allocation design, and the denominator is the proportional change in recruitment rate attained under the alternative allocation design. Hence, for example, if the recruitment rate is not increased by changing the allocation ratio (if RR_a:b = RR_1:1), then the recruitment time increases in direct proportion with the increase in sample size. Relative recruitment time decreases as a function of RR_a:b. Recruitment time is equal under the two designs when RR_a:b = b(a/b + 1)²/4a × RR_1:1, and recruitment time is shorter for the alternative design when RR_a:b is larger than this amount.

For example, a trial with 2:1 allocation requires the same amount of time to recruit as a comparably powered trial with 1:1 allocation if the recruitment rate is increased by about 12%. Similarly, a trial with 1.5:1 allocation is time-neutral if the recruitment rate is increased by about 4%. Recruitment rates greater than these break-even points result in reductions in total trial time. For example, if 2:1 allocation increases the recruitment rate by 50%, then recruitment time is 75% that of a 1:1 design, while if 1.5:1 allocation increases the recruitment rate by 50%, recruitment time is 70% that of a 1:1 design.

Net cost under unequal allocation

Calculating the net cost effect of alternative allocation ratios requires some assumptions about the per subject costs (which affect trial cost via the increase in sample size) and the per day administrative costs (which affect trial cost via the potential effect on recruitment time). For the purpose of modeling the net cost effects of changes in the allocation ratio, we partition the cost of performing a trial into three categories:

Fixed costs independent of the trial design. These costs could include expenses related to drug development, miscellaneous costs such as the cost of filing an Investigational New Drug (IND) application with the Food and Drug Administration, and costs related to statistical analysis and reporting. None of these costs depend on the length of the recruitment window, study duration, or number of subjects recruited into the study.
Administrative costs per day. In this category we consider costs such as expenses for rent, salaries of administrative and nonclinical staff, and support of a Data Safety Monitoring Board [20]. These costs are considered fixed per day with total costs dependent on the length of the trial. The tangible monetary benefit of reducing the total time required to perform a trial is the savings realized by the reduction in expenditures for administrative staff and infrastructure. The total length of time required to perform a trial is determined by the length of longitudinal observation after recruitment, which is specified by scientific considerations and is not modifiable, plus the length of time required to recruit subjects into the trial, which is increased by increasing the number of subjects recruited and reduced by accelerating the recruitment rate.
Per subject costs. Per subject costs include costs related to medication (e.g., the cost of production, storage, and shipment), and costs of recruitment and follow-up observation at the sites. For multicenter trials the latter are typically reimbursed at a fixed rate per enrolled subject. Hence, per-subject costs increase approximately linearly with the number of subjects recruited.

Using these categories, the total trial cost is calculated as

To simplify cost calculations, we assume that the administrative costs are static, that is, we assume the per day cost is independent of sample size within the range of sample sizes considered in a given application of this formula. Breaking the trial length component into length of recruitment plus length of follow-up, we further divide the total cost into costs that do not depend on the recruitment rate and costs that may be modified by the recruitment rate (modifiable costs) as follows:

where modifiable costs (MC) are costs that may be influenced by a change in the allocation ratio:

Defining recruitment rate as the number of subjects recruited on average per day, MCs can be re-expressed as a function of recruitment rate (RR) and sample size (N) as

(3)

Using Equation (3)

Equation (3) can be used to calculate MC under alternative designs and thereby characterize the net effect of changing the allocation ratio on trial costs. Recall that modifying the allocation ratio has potentially two effects on study cost. First, changing the allocation ratio from 1:1 requires a larger sample size to maintain the same target statistical power. For cost comparisons, N_1:1 can be calculated from standard power formulas and N_a:b can be calculated using Equation (1). Second, changing the allocation ratio may affect the recruitment rate. The relationship between allocation ratio and recruitment rate varies from context to context. If prior data are available to indicate expected recruitment rates under different allocation ratios in a given trial context, these rates can be used in Equation (3) when planning a trial. Such data are generally not available, and a more likely application of Equation (3) is to estimate and compare MCs using presumed or plausible recruitment rates for the allocation ratios under consideration. We illustrate this application of Equation (3) in the following.

Sample calculation of recruitment time and modifiable cost

Figure 1 presents an example of calculating recruitment time and total modifiable cost in the context of a treatment trial sufficiently powered with a sample size of 200 active and 200 placebo treated subjects. We assume for this exercise a cost of $12,000 per subject in site reimbursement and drug delivery-related costs, administrative costs per day of $3000, and an expected baseline recruitment rate under 1:1 allocation of 15 subjects recruited per month. These numbers roughly represent anticipated costs for a placebo-controlled treatment trial of docosahexaenoic acid to slow the clinical progression of Alzheimer’s disease. We considered two comparably powered alternative designs, one with 1.5:1 allocation and a total sample size of 416 subjects, and one with 2:1 allocation and 448 total subjects. For each design, we calculated time and total MCs under a range of recruitment rates that may plausibly be realized under the alternative designs (a range from no effect of the allocation ratio on recruitment rates to a 50% increase). An improvement in recruitment rates of as high as about 50% was suggested by a recent survey of caregivers of likely study subjects [8]. The referent 1:1 allocation design has 7.2 million dollars of total modifiable costs and would take 27 months to recruit. These reference values are indicated with a horizontal line on Figure 1. Changing the allocation ratio to 1.5:1 is net time beneficial when the recruitment rate is increased by more than 4%, while changing the allocation ratio to 2:1 is net time beneficial when the recruitment rate is increased by more than 12% (Figure 1). In terms of total MCs, the 1.5:1 design is cost-beneficial if the recruitment rate is increased by more than 13%, and the 2:1 design is cost-beneficial if the recruitment rate is increased by more than 47%. The potential time and cost savings are substantial if the alternative allocation ratio does improve recruitment rates. For example, if the 1.5:1 design realizes a 50% increase in recruitment rate, total time for recruitment will be reduced by over 7 months, and total cost will be reduced by over a half million dollars.

Figure 1

Sample calculation of total time to complete recruitment (dashed lines) and total modifiable costs (solid lines) under 2:1 allocation (circle symbols) and 1.5:1 allocation (triangle symbols) as a function of the percentage increase in recruitment rate (more ...)

Relative cost under unequal allocation

The parameter values required for Equation (3) vary from context to context. To make more general conclusions about the effect of allocation ratio on cost, we derive an equation expressing the MC of a trial with unequal allocation as a percentage of the MC of a trial with equal allocation. First, define the relative MC (RelMC) as

where MC_a:b and MC_1:1 are, respectively, the modifiable costs calculated for a trial with the alternative, a:b allocation and a trial with the referent, 1:1 allocation. Substituting the formulas for MC_a:b and MC_1:1 and rearranging terms,

Substituting N_1:1 × b(a/b + 1)²/4a for N_a:b and canceling terms, and substituting c for the cost per subject/cost per day term to simplify presentation,

(4)

Interpreting Equation (4)

If the recruitment rate is unchanged by altering the allocation ratio (in which case RR_a:b = RR_1:1), then the modifiable cost under a:b allocation is simply b(a/b + 1)²/4a × 100% the modifiable cost of a trial with 1:1 allocation, for example 104% and 112% for 1.5:1 and 2:1 allocation, respectively. RelMC decreases as the RR_a:b increases, with the cost neutral threshold reached when RR_a:b = Z/(c(1 − Z) + 1/RR_1:1), where Z = b(a/b + 1)²/4a. Trial costs are lowered under the alternative allocation ratio when RR_a:b is larger than this amount.

Using Equation (4)

RelMC calculates the modifiable cost of a trial under a:b allocation as a percentage of the MC of a clinical trial of equal power under 1:1 allocation. Equation (4) requires as input: (1) the elements of the allocation ratio, a and b; (2) the cost per subject in units of the cost per day, c; (3) the presumed baseline recruitment rate under 1:1 allocation, RR_1:1; and (4) the presumed recruitment rate under the alternative a:b allocation, RR_a:b. Values of these parameters can be used to estimate RelMC for a given allocation ratio a:b.

As a practical application of Equation (4), we note that a clinical trial center can estimate c and RR_1:1 for its particular trial context based on experience in prior or active trials, and then use Equation (4) to assess the potential net cost effect of altering the allocation ratio in future trials. For a given design scenario (i.e., for a given allocation ratio a:b and cost ratio c), the relative cost of a trial with an alternative allocation ratio decreases as the impact of the alternative ratio on recruitment rate increases. For example, for trials with a low referent recruitment rate of 10 subjects per month and low cost ratio c of 2, the 1.5:1 allocation design is cost-effective if the increase in recruitment rate is about 6% or more. Trials with 1.5:1 allocation and c = 5 are cost neutral when the increase in recruitment is 19% and are cost-effective when the increase in recruitment is higher than this amount. Trials with a 2:1 allocation ratio, on the other hand, are not cost efficient unless the recruitment rate is increased by more than 22% when c = 2, and by more than 39% when c = 5.

For trials with a higher referent recruitment rate, the proportional increase in recruitment needs to be more substantial to warrant an unequal allocation ratio design. For example, if the referent recruitment rate is 30 subjects/month, then designs with a 1.5:1 allocation ratio are not net cost-beneficial unless recruitment rates are increased by more than 13% when c = 2 and by more than 29% when c = 5.

Discussion

Costs and consequences of alternative designs are commonly considered in practice. The models presented here are practical tools for comparing the time and monetary cost of using alternative allocation designs to improve recruitment rates. Several conclusions follow from this work. First, these equations emphasize the point that trial time and cost may be increased by alternative allocation designs, even when these designs do improve recruitment rates. Nonetheless, the maximum possible increase in time and cost from increasing the allocation ratio is modest, not more than about 4% for a 1.5:1 allocation ratio and not more than about 12% for a 2:1 allocation ratio. Second, the formulas presented here characterize the relative influence of the factors that determine trial cost under alternative allocation designs. Based on the formulas derived here, the trial contexts most likely to benefit from an alternative allocation ratio design are those with a low baseline recruitment rate, large target sample size, or low per sample cost. In these scenarios, even small increases in the recruitment rate may lead to meaningful reductions in trial time and cost.

There are considerations beyond trial time and the tangible cost of performing a trial when comparing study designs. In the case of positive trials of promising new treatments, using an alternative allocation ratio to reduce trial time has the intangible benefit of making an effective medication available sooner. For industry-sponsored trials of patented medications, shorter trial time translates to earlier and longer on-patent marketing of the medication. Higher allocation to experimental treatment also has the advantage of increasing the likelihood of observing rare adverse events. Arguing against alternative allocation ratios, on the other hand, is the increased study subject burden implicit in performing a trial with a larger sample size. Limiting subject burden is a goal of both investigators and the Institutional Review Boards charged with approving study designs. For treatments with known side effects, a higher allocation ratio increases in particular the number of subjects exposed to the additional burden of treatment side-effects. Studies of active treatments known to have significant side effects should consider the trade-offs carefully before implementing an unequal allocation scheme. Similarly, all other things being equal, the choice between study designs that are approximately time and cost neutral should always be the design with the smallest total sample size (that is, the design with allocation ratio closest to 1:1).

A second concern with using alternative allocation to improve recruitment relates to the fact that this approach relies on the perception that the experimental treatment offers more hope of a favorable outcome compared to control or currently available standard care. This perception is most likely for diseases where only palliative treatments are currently available, as for example Alzheimer’s disease, or for diseases where the prognosis is poor given current standard of care, as for example certain cancers. Patients seeking active treatment via enrollment in clinical trials are arguably nonrepresentative and may also be more prone to early dropout if treatment effects are not immediately evident or the treatment arm is unmasked. Alternative allocation designs are attractive to patients seeking active treatment via enrollment in clinical trials, and therefore may be more vulnerable to these potential sources of bias.

Finally, we note that there are only limited data on the effect of alternative allocation ratios on recruitment rates. One exception is within the context of Alzheimer’s disease treatment trials, where a recent survey estimated recruitment yield per 100 contacts under a range of likely scenarios [8]. This survey found that 2:1 allocation would increase yield from 47 subjects to 60 subjects recruited per 100 contacts within the context of a low risk experimental treatment with only mild side-effects ((13/47) × 100% = 28% faster recruitment), and from 27 to 42 subjects recruited per 100 contacts within the context of a high-risk experimental treatment (56% faster recruitment). To our knowledge, these are the only estimates of the effect of allocation ratio on recruitment rate published to date, although a number of surveys have indicated that concern over randomization to the nonexperimental arm of a trial is a common potential barrier to recruitment (reviewed in [7]). We expect that the effect of allocation ratio on recruitment is highly context specific in any case, and can only be definitively estimated within the context of an actual trial, that is, by randomizing different allocation ratios to the study sites in a multi-site trial. Absent real data on recruitment rates as a function of allocation ratios, the best course of action is to use plausible but conservative assumptions when considering alternative designs. Given defensible assumptions about recruitment rates, the decision to proceed with an alternative allocation design hinges on whether the improvement in recruitment rate justifies the increased sample size and study subject burden implied. The cost models presented here provide a practical tool for informing this decision process.

Acknowledgments

This study was supported by the Alzheimer’s Disease Cooperative Study, NIA grant number AG10483 and the University of California, San Diego Alzheimer’s Disease Research Center, NIA grant number AG005131.

References

1. Adams J, Silverman M, Musa D, Peele P. Recruiting older adults for clinical trials. Control Clin Trials. 1997;1:14–26. [PubMed]

2. Avins AL. Can unequal be more fair? Ethics, subject allocation, and randomised clinical trials. J Medical Ethics. 1998;24:401–08. [PMC free article] [PubMed]

3. Treves TA, Verchovsky R, Klimovitsky S, Korczyn AD. Recruitment rate to drug trials for dementia of the Alzheimer type. Alzheimer Dis Assoc Disord. 2000;4:209–11. [PubMed]

4. Urban N, Self S, Kessler L, et al. Analysis of the costs of a large prevention trial. Control Clin Trials. 1990;2:129–46. [PubMed]

5. Jenkins V, Fallowfield L. Reasons for accepting or declining to participate in randomized clinical trials for cancer therapy. Br J Cancer. 2000;11:1783–88. [PMC free article] [PubMed]

6. Pocock SJ. Allocation of patients to treatment in clinical trials. Biometrics. 1979;35:183–97. [PubMed]

7. Mills EJ, Seely D, Rachlis B, et al. Barriers to participation in clinical trials of cancer: a meta-analysis and systematic review of patient-reported factors. Lancet Oncology. 2006;2:141–48. [PubMed]

8. Karlawish J, Cary MS, Rubright J, TenHave T. How redesigning AD clinical trials might increase study partners’ willingness to participate. Neurology. 2008;71:1883–88. [PMC free article] [PubMed]

9. Hornberger J. A cost-benefit analysis of a cardiovascular disease prevention trial, using folate supplementation as an example. Am J Public Health. 1998;1:61–67. [PubMed]

10. Claxton K, Posnett J. An economic approach to clinical trial design and research priority-setting. Health Econ. 1996;6:513–24. [PubMed]

11. Berry DA. Introduction to Bayesian methods III: use and interpretation of Bayesian tools in design and analysis. Clin Trials. 2005;4:295–300. [PubMed]

12. Eisenstein EL, Collins R, Cracknell BS, et al. Sensible approaches for reducing clinical trial costs. Clin Trials. 2008;5:75–84. [PubMed]

13. Kocher MS, Henley MB. It is money that matters: decision analysis and cost-effectiveness analysis. Clin Orthop Relat Res. 2003;413:106–16. [PubMed]

14. Meinert CL. Clinical Trials Design, Conduct, and Analysis. Oxford University Press; New York: 1986.

15. Food and Drug Administration Center for Drug Evaluation and Research. Guidance for Industry: Premarketing Risk Assessment. Mar, 2005. http://www.fda.gov/cder/guidance/index.htm.

16. Aisen PS, Schafer KA, Grundman M, et al. Effects of rofecoxib or naproxen vs placebo on Alzheimer disease progression – A randomized controlled trial. JAMA. 2003;21:2819–26. [PubMed]

17. Allison DB, Allison RL, Faith MS, et al. Power and money: designing statistically powerful studies while minimizing financial costs. Psychological Methods. 1997;1:20–33.

18. Dumville JC, Hahn S, Miles JN, Torgerson DJ. The use of unequal randomisation ratios in clinical trials: a review. Contemp Clin Trials. 2006;1:1–12. [PubMed]

19. Torgerson D, Campbell M. Unequal randomisation can improve the economic efficiency of clinical trials. J Health Serv Res Policy. 1997;2:81–85. [PubMed]

20. Schulman K, Burke J, Drummond M, et al. Resource costing for multinational neurologic clinical trials: methods and results. Health Econ. 1998;7:629–38. [PubMed]