|Home | About | Journals | Submit | Contact Us | Français|
To assess the most effective recruitment strategies for an acupuncture clinical trial of reproductive age women.
The underlying study is an acupuncture randomized clinical trial for an ovulatory disorder that affects approximately 6.5% of reproductive age women (Polycystic Ovary Syndrome). Study participation involved 2 months of intervention and 3 months of follow-up with US$170 compensation. Success of each recruitment method used during the first 37 study months was analyzed.
Clinical trial in the Dept. of OB/GYN at the University of Virginia, US. The original geographic residency target was an 80 mile radius around a college town in Virginia (population 155,000), and was expanded to the state capital (population 850,000) in recruitment year 2.
Number of study inquiries (phone calls or emails) over time and by recruitment source.
In the first 37 months of recruitment (Jan 2006 – Jan 2009), there were 800 study inquiries (582 by phone, 218 by email), of which 749 were screened via telephone questionnaire. The most successful recruitment methods were flyers (28% of inquiries and 26 % of participants) and direct mailing to targeted zip codes (26% and 27%, respectively). The direct mailing cost US$110/inquiry, while the flyers cost less than US$300 in total. Study inquiries were least likely in May and November. Almost all prospective participants (94%) were acupuncture-naïve.
Posters/flyers and direct mailings proved to be the most successful recruitment methods for this CAM study. Active recruitment with multiple methods was needed for continual enrollment.
Recruitment is a vital and integral part of all clinical trials and contributes largely to both the statistical power and overall success of the study. Additionally, unrepresentative recruitment can yield results that are not generalizable to other populations or, more seriously, can result in selection bias due to a “volunteer effect”1 or diagnostic bias.2 Many researchers agree that recruitment is one of the most challenging aspects of a clinical trial.3 Recruitment for complementary/alternative medicine (CAM) trials theoretically could have greater recruitment challenges due to the fact that CAM therapies, by definition, are not part of conventional medicine.4
This report is focused on OB/GYN research because marketing for a female-only study is different than studies for male or pediatric populations. Women seem more accurate in decoding nonverbal cues and are considered to be more visually oriented, more intrinsically motivated, and more romantic compared to men and tend to have more associative, imagery-laced interpretations of marketing materials.5
Only three prior publications of recruitment strategy successes from any OB/GYN-related CAM clinical trials were located6,7,8, and their “best marketing methods” differed. Included among the list of most effective techniques are: press releases, community referrals, doctor referrals, direct mail, radio ads, newspaper ads, and referrals from family or friends. Among the least successful strategies reported are: referrals from clinicians or family or friends, posters, cable TV bulletin board ads, and press releases. Note that some techniques are contained on both lists, hence, the inconsistency on this topic. These disparate findings may indicate that marketing strategies need to be tailored to the specific study, may be geographically dependent (e.g., rural/urban; cultural differences by country), and/or may be related to socio-economic status. A low frequency of participation in research has been reported for some race/ethnicity minority groups. 9–11 It has also been reported that recruitment strategies may need to be tailored12,13 or require more investment14 for some census groups.
The source for this report is an ongoing randomized clinical trial of acupuncture for women with Polycystic Ovary Syndrome (PCOS). PCOS is the most common endocrine disorder of reproductive age women and is the most common endocrine cause of infertility.15 Approximately 6.5% of women of reproductive age have PCOS16,17, which is characterized by irregular or absent menstrual periods, and hyperandrogenic manifestations such as acne and hirsutism.18 The clinical presentation of PCOS is generally irregular cycles and/or anovulatory infertility.
The goal of this paper was to evaluate the marketing success of this OB/GYN acupuncture clinical trial by both recruitment method and calendar time period in greater detail than prior reports. A British Medical Journal editorial19 and others20 have called for publications detailing recruitment methods.
The underlying study was a randomized, double-blind, placebo-controlled clinical trial of acupuncture in women diagnosed with PCOS. The primary goals of this study were to see if acupuncture, as compared to a sham acupuncture treatment, normalizes ovarian hormones and increases the frequency of ovulation in reproductive-age PCOS women.
Inclusion criteria were: a) a diagnosis of PCOS, as confirmed through the study using the U.S. National Institute of Child Health and Human Development criteria of oligomenorrhea, non-diabetic, with self-reported hirsutism and/or acne and/or elevated free testosterone21, b) aged 18 to 43 years, and c) weight of 250 pounds (113 kg) or less. Participants were not allowed to take hormonal medications 60 days prior to entry into the study nor during the course of the trial.
The study involved a 5-month protocol: a 2-month intervention (1:1 randomization of real:sham acupuncture) and 3 months of follow-up. Blood and/or urine was collected routinely throughout the 5 months. There were 16 clinic visits in total, which represent a combination of acupuncturist appointments and serum sampling clinic visits. The study is still recruiting, thus, the sham protocol is not described. US$170 was offered as compensation for study completion. This study was conducted by the OB/GYN Department at the University of Virginia Health System (UVa), which is located in Charlottesville, VA, USA. The UVa Internal Review Board approved this protocol (#12045). This report is restricted to the first 37 months of recruitment (Jan 2006 – Jan 2009).
The target sample size for the underlying study was 89 randomized with 78 completers (i.e., 12% drop-out rate anticipated). The original goal was to recruit and retain 3–4 women per month for 24 months, which was based on the three years of funding received minus 3 months for start-up and 9 months for analysis.
Recruitment was initially targeted to Charlottesville and the surrounding counties (population of 155,000). In Year 2, marketing efforts were expanded to cover the Richmond metropolitan area (state capital) with a population of 850,000. In addition to the larger population base, the Richmond vicinity was targeted due to their sizeable proportion of minority residents. A study acupuncturist was hired to improve participation convenience for Richmond women; participation still required three visits to the Charlottesville study clinic. Based on the National Certification Commission for Acupuncture and Oriental Medicine 22, there are at least 18 licensed acupuncturists in the Richmond vicinity. In Year 2, we also expanded recruitment with direct mailings to the counties between Charlottesville and Harrisonburg, VA (metropolitan population 70,000), where there is only one licensed acupuncturist. In Year 3 we added a third acupuncturist in Charlottesville to accommodate evening appointments, and expanded marketing efforts to the Lynchburg, VA metropolitan district (population 65,000), where there is one licensed acupuncturist.
Our initial recruitment philosophy was to target university students, under the assumption that younger populations would have a low proportion of needle-phobic individuals due to the preponderance of body piercing and tattoos, and thus, be open to acupuncture treatment. We also aimed to minimize expenses and, thus, primarily relied upon posters, emails, and website listings for the first nine months. Over time, a wider variety of recruitment strategies were employed, as described below, and an increasing amount of money spent on marketing.
The analysis examined the count of study inquiries and enrolled participants, as well as the distribution within both those categories by recruitment method. The term “screened” refers to the women who completed a phone screening questionnaire. The term “enrolled” refers to women who signed an informed consent, screened positive for PCOS with blood tests, and were randomized. Month-by-month recruitment success was examined graphically. Recruitment success by source was qualitatively compared with published literature for other CAM trials. Organizational characteristics that potentially impacted recruitment19 were described.
There were 800 study inquiries (582 by phone, 218 by email) in the first 37 months of recruitment. Some women made an initial contact and never again had any communication with the study office; there were n=749 women with whom we communicated regarding their eligibility, and this is the population available for this analysis. A total of 70 women were enrolled in this time period. The mean age of inquirers was 29.5 years (sd 7.9).4 The geographic distribution of eligible women was 54.7% from the Charlottesville vicinity, 22.3% Harrisonburg, 16.2% Richmond, 2.8% Lynchburg, 2.8% Northern Virginia, and 1.1% other areas.
The most successful recruitment methods were flyers (28% of inquiries and 26% of enrolled women) and direct mailing to targeted zip codes (26% of inquiries and 27% of enrolled women); see Table 1. The direct mailing cost US$110/inquiry, while the printing expense for the posters/flyers was minimal due to the availability of a high-technology color printer. The TV recruitment yield was lower than for direct mail (8–9 inquiries per week of commercials vs. 13–17 inquiries per direct mailing to 50,000–60,000 households), with a cost of US$201 per inquiry. Contrary to expectations, commercial air time during local news reports had a greater response than either daytime talk shows or evening prime time. Advertisements in the local, weekly newspaper were totally unsuccessful in recruitment month 2 (zero inquiries after three weekly ads), but more successful when used again in Year 3. Daily newspaper ads were deemed unsuccessful (5 inquiries from 4 ads at a total cost of US$880). The public service TV ads ($0 cost) yielded approximately the same number of inquiries as the commercial TV ad for 1 year, and then ceased to be effective. Radio advertisements were unsuccessful (3 inquiries after 3 weeks of advertisement, US$270 per inquiry). Physician referral efforts were not particularly effective, with the exception of referrals from the UVA Student Health Center, although the cost for these efforts was minimal.
As summarized in the Introduction, the prior literature on the success or failure of recruitment strategies in CAM clinical trials for OB/GYN patient concerns indicates inconsistency in their effectiveness (Table 2). Following the structure of organizational characteristics that might impact recruitment success presented by Foy et al19, this study team consisted of junior faculty members with advanced training in research methods, variable research experience of the primary OB/GYN referring clinicians, use of systematic patient identification through computerized records albeit this produced a minority of the enrollees, virtually no workload impact on primary OB/GYN providers from this study, and use of university research network for press releases and newspaper ads.
Study inquiries fluctuated depending on the time of year (Figure 1), with the least number of inquiries around holiday and vacation times and a slightly higher volume at other time points. The monthly inquiry volume range from 4 to 59. Charlottesville, Virginia is a university-centered town. As such, the study inquiry volume was generally best in the first 2–3 months of each university semester, and our marketing efforts became increasingly targeted to those months over the course of the study. The recruitment success in May and November was predictably low, presumably due to preoccupation with year-end finals and holidays, respectively. Direct mail in December did prove to be worthwhile; mailings occurred after Christmas/Hannukah in 2007, so a December mailing served as a post-holiday initiative for women. The marketing outreach and the volume of inquiries were directly related to each other but less related to season, with the exception of May and November, as reviewed above.
Nearly all (94%) of the prospective participants were acupuncture-naïve. There was little difference by recruitment source in women’s ages: the mean age by marketing strategy ranged from 27 years to 34 years except for the university electronic newsletter (21.7 years). Hirsutism symptoms ranged from 44% among university students responding to the electronic newsletters up to over 90% of clinic patients and women who responded to newspaper articles or press releases. For most of the recruitment sources, half (+/− 12%) of the inquiries reported acne symptoms. The most common reasons for ineligibility were women who had fewer than 39 days between their periods (19%), did not want to stop using hormonal medication for study purposes (15%), or were over the weight limit for the study (8%).
The best recruitment technique overall for this acupuncture clinical trial for reproductive age women was the use of posters/flyers in clinics, bathrooms, and bulletin boards, based upon the large number of study inquiries and very low direct costs. Direct mailings to targeted zip codes produced nearly as many inquiries as posters/flyers, but cost approximately US$110 per inquiry. Direct mailing was more cost-effective than TV commercials, as has been reported by others for non-CAM interventions.14 The number of inquiries was not directly related to the dollar amount spent for each of the recruitment strategies.
Strengths of this report include the close monitoring of the marketing technique that prompted the potential participant to contact the study office, and the attention by the study staff to revise an advertisement and to develop new materials/sources. The limitations are (a) this study’s experience may not be applicable to other CAM interventions in other geographical areas, and (b) no formal methodological testing of marketing techniques was employed.
Smith & Coyle6 in 2006 summarized five OB/GYN studies of acupuncture or ginger interventions. Their most successful strategies for recruitment were press releases (yielding approximately 30% of the inquiries), community referrals (yielding ~20%) and doctor referrals (yielding ~20%). In contrast, none of these marketing techniques were beneficial in this clinical trial. Cambron et al7 in 2004 found that the most successful recruitment methods for a chiropractic multi-center study for chronic pelvic pain were direct mail (39% of inquiries), radio (34%) and newspaper advertisement (19%); our research supports their success with direct mail only. Recruitment through referrals made by family, friends or physicians was not particularly successful in the 2004 Cambron report7, and our research supports that finding. A study of chiropractic treatment for menstrual cramps found that newspaper advertisements (46% of inquiries), radio advertisements (18%) and referrals from family or friends (17%) were the best methods for recruitment8; in contrast, none of those methods was effective in our community.
In terms of geography, there was mixed success at expansion to neighboring metropolitan areas. The rate of inquiries and ultimate enrollment was low from Richmond. Factors that may have contributed to poor recruitment: (a) Richmond city and the surrounding county are 57% and 28% African-American, respectively, and that racial group has lower participation rates in scientific studies in the US9–12, and (b) the driving distance to Charlottesville (approximately 65 miles). In contrast, we found recruitment from the Harrisonburg vicinity (approximately 35 miles to Charlottesville) to be quite good. Factors that may have contributed to recruitment success from that geographic region are: few acupuncturists in Harrisonburg, and many people commute to Charlottesville for work.
An important implication of this analysis is the budgetary and staff retention issues that arise with it. Due to the variation in volume of inquiries over the calendar year especially in “university towns” (lower in summer, highest at the beginning of college semesters), the staff and/or acupuncturists associated with the study could not be kept busy at an even pace month by month. If this study and future research can be combined to determine a predictable trend in alternative medicine trials (or other clinical trials in general), pre-planning of work hours and/or study activities can occur. With such information, recruitment strategies could be planned 4–6 months in advance and thus save time and effort once the study begins, yielding an improved allocation of scarce monetary support.
The actual recruitment did not keep pace with the anticipated recruitment plan, which is true for other studies. As reported in a review of seven studies of dyspepsia19, three studies closed prematurely and the remaining four studies continued enrolling an average of 27% longer than the original estimate with a final sample size that was 74% of the original sample size (range 29% - 100%).
A focus group study assessed the willingness of individuals (both men and women) to participate in CAM clinical trials for craniofacial conditions.24 That qualitative study reported these key findings:
The experience of this acupuncture trial in these regards is as follows:
In summary, this trial found that flyers/posters and direct mailing were the two most worthwhile recruitment activities for enrolling reproductive age women into this acupuncture clinical trial. Given the time and expense required for participant enrollment, the relative effectiveness of marketing strategies for CAM interventions is important for future researchers to utilize in developing recruitment plans. As suggested by others19, 23, methodological research that evaluates recruitment techniques in an unbiased fashion would provide evidence based guidance for researchers. The following “lessons learned” were derived from this study:
We thank Martin Phillips, who provided graphic artist services for this study through the University of Virginia General Clinical Research Center. This publication was made possible by grant number R21 AT002520 from the National Center for Complementary and Alternative Medicine (NCCAM) at the National Institutes of Health. Its contents are solely the responsibility of the authors and do not necessarily represent the official views of NCCAM. The project described was also supported by Grant Number M01RR000847 from the National Center for Research Resources. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Center for Research Resources or the National Institutes of Health.
Sources of support: Grant R21 AT002520 from the National Center for Complementary and Alternative Medicine (NCCAM) at the National Institutes of Health. Grant M01RR000847 from the National Center for Research Resources.
Conflict of interest statement: No conflicts of interest.
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.