PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
Addict Behav. Author manuscript; available in PMC 2018 January 1.
Published in final edited form as:
PMCID: PMC5143209
NIHMSID: NIHMS815141

An Ounce of Prevention: A Pre-Randomization Protocol to Improve Retention in Substance Use Disorder Clinical Trials

Thomas F. Northrup, Ph.D.,1 Tracy L. Greer, Ph.D.,2 Robrina Walker, Ph.D.,2 Chad D. Rethorst, Ph.D.,2 Diane Warden, Ph.D., M.B.A.,2 Angela L. Stotts, Ph.D.,1,3 and Madhukar H. Trivedi, M.D.2

Abstract

Background

Missing data in substance use disorder (SUD) research pose a significant threat to internal validity. Participants terminate involvement or become less likely to attend intervention and research visits for many reasons, which should be addressed prior to becoming problematic. During a 9-month study targeting stimulant abuse, early dropouts and participant reported attendance barriers led to implementing a structured, pre-randomization protocol with participants about retention and solution-focused strategies (the “Fireside Chat”). Our aim is to outline this approach and present data on intervention participation and research visit attendance after implementation.

Methods/design

STimulant Reduction using Dosed Exercise (STRIDE) was a two-arm, multisite randomized clinical trial testing treatment-as-usual for stimulant abuse/dependence augmented by Exercise or Health Education. For both groups, study intervention visits at the site were scheduled 3/week for 12 weeks followed by 1/week for 24 weeks. During The Chat, research staff thoroughly reviewed participants’ expectations, and barriers and solutions to retention. Fifteen participants were randomized (to Exercise or Health Education) prior to and fourteen were randomized after Chat implementation. Intervention and monthly follow-up attendance (before and after implementation) were compared at the site (N=29) that developed and rigorously implemented The Chat.

Results

Individuals who participated in The Chat (n=14) attended significantly more intervention visits during weeks 1–12 (p<0.001) and weeks 13–36 (p<0.05) and attended more research visits (p<0.001).

Discussion

Proactive discussion of expectations and barriers prior to randomization was associated with greater study attendance. SUD researchers should consider tailoring this approach to suit their needs. Further investigation is warranted.

Keywords: Retention, Longitudinal Research, Substance Use Disorder, Attrition, Dropout

1. Introduction

Missing data in longitudinal substance use disorder (SUD) research pose a significant problem for drawing valid inferences, with 20% of participants routinely lost to follow-up after 3 months of participation and nearly one-third of participants missing data beyond 12 months (Hansen, Tobler, & Graham, 1990). This poses a severe threat to internal validity, and researchers are frequently faced with the dilemma of how to handle missing observations, particularly for rigorous, longitudinal SUD research. All commonly employed imputation methods within SUD research (e.g., last-observation-carry-forward, missing equals positive, multiple imputations, and full-information maximum likelihood) are subject to error. Further, these methods assume data are missing at random (MAR) or missing completely at random (MCAR). When the data are not missing at random (NMAR; i.e., the probability of a missing value depends on the variable that is missing) the missing data mechanism must be modeled to obtain valid parameter estimates. For this reason and others, missing information should be minimized through proactive, ongoing study staff efforts to refine and improve the recruitment and retention process and facilitate participant attendance; however, little formal research guides best practices for retaining participants.

Participants join studies for a variety of reasons (Scott, Walker, White, & Lewith, 2011), including personal relevance, altruism, and monetary compensation (Kost, Lee, Yessis, Coller, & Henderson, 2011). Also, participants terminate involvement or become less involved for many reasons, including perceived stigma, scheduling difficulties, SUD symptoms, and waning motivation (Ball, Carroll, Canning-Ball, & Rounsaville, 2006; Claus, Kindleberger, & Dugan, 2002). Most reasons can and should be addressed prior to becoming a retention issue, particularly for longitudinal research designs.

Engaging participants in a comprehensive, collaborative discussion prior to randomization about real-world barriers to research attendance may hold promise to maximize retention. Evidence supports participant comprehension as an important factor of involvement in multisite RCTs (Lipton et al., 2011). Further, team recruitment approaches have had success in a variety of studies (e.g., weight gain prevention; Stockton, McClanahan, Lanctot, Klesges, & Beech, 2012).

In response to dropouts and participant-reported attendance barriers, we initiated a structured protocol with ongoing participant collaboration, called the “Fireside Chat.” This approach augmented the informed consent process with participants and was applied rigorously within the context of a multi-site, longitudinal intervention study with stimulant abusers to maximize available data and participant retention. The aims of this secondary data analysis are to describe the Fireside Chat and present data on intervention and research visit attendance for participants who did and did not participate in a Fireside Chat at one of the study sites.

2. Study Background & Methods

2.1 STimulant Reduction using Dosed Exercise (STRIDE) Design Overview

A multisite RCT (STimulant Reduction using Dosed Exercise [STRIDE]) was conducted by the National Drug Abuse Treatment Clinical Trials Network at nine U.S. residential substance abuse treatment programs (RTPs). STRIDE (N=302) compared Exercise versus Health Education as augmentation to addiction treatment-as-usual in individuals with stimulant use disorders (Trivedi et al., in press). Men and women (ages 18–65) who were admitted to RTPs, used stimulants in the 30 days prior to admission, met DSM-IV criteria for stimulant abuse/dependence, and medically cleared to exercise, were eligible.

Randomization to Exercise or Health Education occurred soon after treatment admission to maximize days of study participation during the RTP stay and increase the likelihood of intervention attendance. Study visits (i.e., Exercise or Health Education intervention and research visits) occurred 3/week for three months (weeks 1–12; acute phase), followed by six months of weekly visits (weeks 13–36; continuation phase). Exercise was prescribed at approximately 50 minutes, 3 days per week. Health Education consisted of online, video, and written educational materials for an equivalent period of time. Complete design and rationale are described elsewhere (Greer et al., 2012; Stoutenberg et al., 2012; Trivedi et al., 2011).

2.2 Study Initiation and Identification of Study-Implementation Challenges

At study initiation, staff proactively discussed and monitored participants’ treatment status (e.g., outpatient treatment plans) to help ensure decisions to randomize were fully informed. Further, due to the large participant-time investment (>50 hours for highly adherent participants, not including travel time), incentives were provided to increase intervention and research visit attendance. Specifically, participants received monetary compensation for weekly research visits ($15–$25) to offset participation costs (e.g., travel) and received additional adherence incentives, such as water bottles, notebooks, and monetary compensation, for completing adherence milestones.

Due to a high frequency of early dropouts at the beginning of the trial and participant reported attendance barriers that became evident only after randomization of participants had begun, the study team determined that a more structured and formal retention-related protocol should be added prior to randomization. The Fireside Chat was therefore developed to use with every participant by study teams at each site, including the site PI, before a decision to randomize was made by the team and participants. The Chat was conceptualized and developed by Houston, Texas-based study team members and study leadership.

2.3 Proactively Addressing Challenges with the Fireside Chat

Following implementation of the Fireside Chat (approximately midway through STRIDE recruitment at the Houston site), The Chat occurred with each potential participant and included all members of a site study team, to identify barriers to retention, minimize negative effects from these barriers and build a foundation for ongoing retention efforts. It also helped both potential participants and staff determine the “fit” between the participant and study prior to randomization. The Chat was scheduled after informed consent was obtained and after most or all screening measures were completed and all other eligibility criteria were met. This avoided overburdening participants prior to determining their likelihood of meeting eligibility criteria. Parts of the informed consent were revisited during The Chat; however, it was not simply a more in depth informed consent discussion. The Site PI typically led the discussion, using a structured guidance tool (see Table 1), over a 30- to 60-minute period. This provided the opportunity for open, non-judgmental exploration of specific barriers and solutions in a thorough and structured way, and included educating participants about their study responsibilities and evaluating their understanding.

Table 1
Sample pre-randomization “Fireside Chat” by seven discussion topics

During The Chat, the study team first began by describing the reasons for The Chat and the importance of the research question. Further, study staff explicitly detailed the structure and time requirement for each visit type. Participants were asked to describe how the study might fit into their lives over nine months and were asked to prospectively identify solutions to staff-presented and participant-elicited attendance barriers during The Chat. Staff were mindful about superficial and socially desirable answers, and continued to re-state barriers to elicit a thoughtful, in-depth discussion about concerns regarding study involvement raised by both staff and participant. “What if” questions were frequently used to challenge participants to find alternative solutions to attendance barriers (e.g., What if your work schedule changes? How else could you make it to visits?). Sources of outside social support were investigated as well (e.g., What friends and family could you ask for help if you have competing demands on your time?), which often gave a clearer picture of participants’ abilities to get rides from friends/family or receive childcare assistance, in order to complete study visits. The opportunity to discuss the plausible course of participation helped participants fully envision the experience prior to randomization. Participants were often randomized in the STRIDE study the day following The Chat, with some randomized on the same day.

The team considered participants’ reactions before final STRIDE randomization decisions were made. Consistent with eligibility criteria in this hybrid efficacy-effectiveness trial, candidates could be excluded if deemed to be at high risk for dropping out. For example, some candidates were excluded if at risk for non-attendance due to onerous study-related travel. While this criterion was in place prior to The Chat protocol implementation, The Chat helped evaluate this in a thorough, thoughtful manner. If a participant did choose to be randomized and had difficulty making visits during the trial, follow-up Chats revisited barriers and potential solutions. The Chat was often revisited at the beginning of the continuation phase (week 13). Chat topics, talking points, questions, and the intent of questions are thoroughly described in Table 1.

2.4 Participants

The Chat was disseminated to all nine sites via national training phone calls and e-mails from study leadership. Participant study attendance before and after Chat implementation, however, are only available from the Houston-based site (N=29), which rigorously implemented The Chat, and will be presented here. Fifteen participants had already been randomized in the STRIDE study when The Chat protocol was implemented. Thus, 14 participants who enrolled in the STRIDE study at the Houston site following implementation of the Chat protocol participated in a STRIDE pre-randomization Chat, and the 15 previously enrolled participants did not.

2.5 Measures

During the acute phase (weeks 1–12), 36 intervention visits were possible (for Exercise and Health Education participants). During the continuation phase (weeks 13–36), Exercise participants could complete a total of 72 sessions (24 weekly intervention visits at the RTP and 48 home-based exercise sessions), while Health Education participants could complete a total of 24 (RTP-based) visits. Beginning with the continuation phase (week 13) participants completed a longer assessment visit every 4 weeks until study completion (week 37); attendance at these 7 visits were used to compare research-visit retention.

2.6 Statistical Analyses

Means of intervention and research visit attendance were compared across the participants who did not receive The Chat (n=15 [n=7 Exercise; n=8 Health Education]) and those who did (n=14 [n=9 Exercise; n=5 Health Education]). Statistical comparisons were made using T tests (PROC TTEST; SAS 9.3 [Cary, NC]; Satterthwaite variance method) and evaluated with 2-tailed tests at the alpha=0.05 level.

3. Results

The mean age of participants at the Houston site was 34.4 years (SD=10.8) and participants were predominantly male (62.1%, n=18) and White, non-Hispanic (82.8%, n=24). The remaining participants were Black (17.2%, n=5).

Participants in both intervention conditions who received The Chat attended significantly more intervention visits during the acute phase compared to those who did not receive The Chat, t(20)=−4.28, p<0.001 (32.2 visits [SD=5.3] vs. 18.3 visits [SD=11.3]; see Figure 1). Sensitivity analyses found that these results did not differ by condition within Chat status. Specifically, among participants who did not receive The Chat, Health Education participants attended more acute phase intervention visits than Exercise participants (22.8 [SD=12.7] vs. 13.3 [SD=8.5] visits, respectively) but the difference was not significant (t[12]=−1.77, p=0.10). Similarly, among participants who received The Chat, no significant differences were found between Health Education and Exercise participants for intervention visits in the acute phase (34.6 [SD=1.7] vs. 30.9 [SD=6.2] visits, respectively, t[10]=−1.69, p=0.12).

Figure
The mean number of intervention visits attended during the acute phase (maximum=36 visits) by Fireside Chat status.

In the continuation phase, Exercise participants who received The Chat attended significantly more intervention visits, t(10)=−2.38, p<0.05 (24.7 visits [SD=22.2] vs. 6.0 visits [SD=6.7], respectively). Similarly, Health Education participants who received The Chat attended more intervention visits during the continuation phase, t(11)=−3.02, p=0.01 (18.0 [SD=3.6] vs. 8.8 [SD=7.4] visits, respectively). Participants in both conditions who received The Chat attended significantly more research visits (out of 7 possible visits) as well, t(24) =−4.46, p<0.001 (6.4 [SD=1.3] vs. 3.5 [SD=2.2] visits). Sensitivity analyses revealed that Health Education and Exercise participants who did not receive The Chat did not differ on research visit attendance (3.9 [SD=2.2] vs. 3.0 [SD=2.2] visits, respectively, t[13]=−0.77, p=0.45). This finding was consistent among participants who received The Chat (6.8 [SD=0.4] vs. 6.2 [SD=1.6] visits, respectively, t[10]=−0.99, p=0.34).

4. Discussion

This secondary data analysis demonstrated the potential for a thorough, highly engaging retention protocol with participants (prior to randomization) to potentially mitigate missing data, particularly NMAR data (e.g., missing due to issues with intervention adherence). Indeed, the results at our site were dramatic, with approximately 90% of acute phase intervention and research visits completed after implementation of The Chat—an excellent marker of study attendance and retention in this field (Hansen, et al., 1990).

Post-hoc analyses of factors associated with attendance and attrition are common in this area of research (e.g., Claus, et al., 2002; Northrup, Green, Evans, & Stotts, 2014); however, prospective studies (secondary to the main trial) focused on increasing retention are less so. A weight-loss trial (with 6-, 12-, and 18-month follow-up visits) employed Motivational Interviewing and “active learning” in a group format, prior to randomization, to address participant retention and expectations. Only the first 51 participants (of 162 total) took part in the pre-randomization groups, and the researchers reported ≥94% attendance at all three visits, regardless of whether a participant took part in the pre-randomization groups (Goldberg & Kiernan, 2005). Similar efforts to prevent individuals from dropping off waitlists for publicly funded substance abuse treatment have not met with success (Donovan, Rosengren, Downey, Cox, & Sloan, 2001) but these studies may differ from the present study in important ways. For example, administering a retention/problem-focused protocol close to the time of research participation may capitalize on when participants’ motivation peaks and fully engage their problem-solving skills; whereas, patients’ motivation for change while on a waitlist may wane while their treatment status remains in limbo.

Our preliminary data suggest the Fireside Chat is promising; however, our sample size was modest and our pre-post design precludes drawing a stronger conclusion about the association between The Chat and improved study visit attendance. Other explanations and potential sources of bias cannot be ruled out. For example, there may be time effects in that our staff may have improved in efficiency and knowledge over the life of the trial. Procedural changes in response to identified barriers may also have affected attendance. Prospectively designed studies with random allocation are needed to fully assess the efficacy of this approach. We acknowledge that other research designs may not allow for a multiple-day baseline, with at least a day for participants to consider the full meaning of research involvement. Abbreviated Chat adaptations may prove feasible depending on the research context, and this approach could be modified to address other research-specific problems (e.g., medication adherence); however, The Chat may be inappropriate for some research designs. In spite of these limitations the exemplary attendance/retention markers following Chat implementation at our site suggest promise for a formalized retention protocol to augment routine consenting and screening procedures with participants and the need for further evaluation of this method.

5. Conclusion

In closing, the myriad barriers to research engagement that participants face require careful consideration by all stakeholders involved in research. The time investment for The Chat by potential participants and research staff is not trivial but it could produce important benefits for the study. Indeed, the increased validity of study results when less data are missing and staff time savings as the study progresses (e.g., less need for efforts to contact participants who have missed visits) are both quite important. Cooperative dialogue, beginning early in the recruitment phase and prior to randomization, is critical to engaging and retaining participants in SUD research as the collaborative partners we intend them to be.

Highlights

  • Missing data are a problem for valid inference in all areas of research.
  • Substance use disorder researchers routinely report over 20% missing data points.
  • Before randomization, a thorough, real-world discussion of all potential retention threats may improve attendance and adherence.
  • Adherence and complete data points were each over 90% following implementation of pre-randomization retention protocol.

Acknowledgments

Role of Funding Sources

STRIDE (CTN-0037) was supported by the National Institute on Drug Abuse of the National Institutes of Health under Award Number U10DA020024 and UG1DA020024 (PI: Trivedi). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

Dr. Greer has received research funding from NARSAD and consulting fees from H Lundbeck A/S and Takeda Pharmaceuticals International, Inc. Madhukar H. Trivedi is or has been an advisor/consultant to: Alkermes, AstraZeneca, Bristol-Myers Squibb Company, Cephalon, Inc., Cerecor, Concert Pharmaceuticals, Inc., Eli Lilly & Company, Evotec, Forest Pharmaceuticals, GlaxoSmithKline, Janssen Global Services, LLC/Janssen Pharmaceutica Products, LP/Johnson & Johnson PRD, Lundbeck, MedAvante, Medtronic, Merck, Mitsubishi Tanabe Pharma Development America, Inc., Naurex, Neuronetics, Otsuka Pharmaceuticals, Pamlab, Pfizer Inc., Phoenix Marketing Solutions, Rexahn Pharmaceuticals, Ridge Diagnostics, Roche Products Ltd., Sepracor, SHIRE Development, Sunovion, Takeda, Vivus, and Wyeth-Ayerst Laboratories. In addition, he has received research support from: Agency for Healthcare Research and Quality (AHRQ), National Alliance for Research in Schizophrenia and Depression, National Institute of Mental Health, and National Institute on Drug Abuse.

The authors wish to thank the research and clinical staff at the Houston site for their support of this research.

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

Conflict of Interest

Drs. Northrup, Walker, Rethorst, Warden, & Stotts declare that they have no conflicts of interest.

Contributors

All authors developed and assisted with implementation of the pre-randomization protocol (i.e., The Fireside Chat). Dr. Northrup conducted the statistical analyses for this manuscript and wrote the first draft, in consultation with Drs. Greer, Walker, Rethorst, Warden, Stotts, and Trivedi. All authors contributed toward and provided edits on several versions of the manuscript as well. All authors approved the final manuscript.

References

  • Ball SA, Carroll KM, Canning-Ball M, Rounsaville BJ. Reasons for dropout from drug abuse treatment: Symptoms, personality, and motivation. Addictive Behaviors. 2006;31(2):320–330. [PubMed]
  • Claus RE, Kindleberger LR, Dugan MC. Predictors of attrition in a longitudinal study of substance abusers. Journal of Psychoactive Drugs. 2002;34(1):69–74. [PubMed]
  • Donovan DM, Rosengren DB, Downey L, Cox GB, Sloan KL. Attrition prevention with individuals awaiting publicly funded drug treatment. Addiction. 2001;96(8):1149–1160. [PubMed]
  • Goldberg JH, Kiernan M. Innovative techniques to address retention in a behavioral weight-loss trial. Health Education Research. 2005;20(4):439–447. [PubMed]
  • Greer TL, Ring KM, Warden D, Grannemann BD, Church TS, Somoza E, Rethorst C. Rationale for using exercise in the treatment of stimulant use disorders. The journal of global drug policy and practice. 2012;6(1) [PMC free article] [PubMed]
  • Hansen WB, Tobler NS, Graham JW. Attrition in substance abuse prevention research a meta-analysis of 85 longitudinally followed cohorts. Evaluation Review. 1990;14(6):677–685.
  • Kost RG, Lee LM, Yessis J, Coller BS, Henderson DK. Assessing Research Participants' Perceptions of their Clinical Research Experiences. Clinical and Translational Science. 2011;4(6):403–413. [PMC free article] [PubMed]
  • Lipton LR, Santoro N, Taylor H, Kidwai N, Isaac B, Magnani M, Pal L. Assessing comprehension of clinical research. Contemporary Clinical Trials. 2011;32(5):608–613. [PubMed]
  • Northrup TF, Green C, Evans PW, Stotts AL. Characteristics associated with intervention and follow-up attendance in a secondhand smoke exposure study for families of NICU infants. The Journal of Maternal-Fetal & Neonatal Medicine. 2014;0:1–6. [PubMed]
  • Scott C, Walker J, White P, Lewith G. Forging convictions: the effects of active participation in a clinical trial. Social Science and Medicine. 2011;72(12):2041–2048. [PubMed]
  • Stockton MB, McClanahan BS, Lanctot JQ, Klesges RC, Beech BM. Identification of facilitators and barriers to participation in weight gain prevention research by African American girls. Contemporary Clinical Trials. 2012;33(1):38–45. [PubMed]
  • Stoutenberg M, Rethorst CD, Fuzat G, Greer TL, Blair SN, Church TS, Trivedi MH. STimulant Reduction Intervention using Dosed Exercise (STRIDE)–Description of the exercise intervention and behavioral program to ensure adherence. Mental health and physical activity. 2012;5(2):175–182. [PMC free article] [PubMed]
  • Trivedi MH, Greer T, Rethorst CD, Carmody T, Granneman BD, Walker R, Nunes EV. Randomized trial comparing exercise to health education for stimulant use disorder: Results from STimulant Reduction Intervention using Dosed Exercise (CTN-0037; STRIDE) Journal of Clinical Psychiatry. (in press)
  • Trivedi MH, Greer TL, Grannemann BD, Church TS, Somoza E, Blair SN, Nunes E. Stimulant reduction intervention using dosed exercise (STRIDE) - CTN 0037: Study protocol for a randomized controlled trial. Trials. 2011;12:206–220. [PMC free article] [PubMed]