In its first ten years, the Clinical Trials Network (CTN) built a national research infrastructure to address the gap between research and practice in addiction treatment (1
) and conducted over 25 randomized trials involving over 11,000 participants in over 200 community treatment programs (2
). These trials involved pharmacological, behavioral and combined interventions and focused on a range of addiction-related problems and populations. This large research-clinical collaboration generated a wealth of research reports describing primary and secondary outcomes, novel dissemination materials and innovations, as well as multiple ancillary studies. The CTN has also generated a number of methodological advances and contributions relevant to Stage III dissemination and hybrid research. To set the stage for this special issue on methodological issues in the CTN, this article will provide an overview of some of the most important methodological lessons learned from the first ten years of the CTN, discuss implications for future studies, and briefly speculate as to potential contributions over its next ten years.
Top Ten Take Home Lessonss from the CTN’s First Ten Years
1. The CTN has successfully established a research-clinical partnership and built a multifaceted infrastructure
The 1998 Institute of Medicine Report (1
) on the gap between research and practice in addiction treatment, which was the impetus for NIDA’s development of the CTN (3
), articulated the need for more collaboration between researchers and practitioners, but also emphasized considerable tensions between these groups. Several features in the structure of the CTN led to rapid progress in initiating clinically-relevant protocols. The most important of these was ‘bidirectionality’, which ensured that the community-based treatment providers, or CTPs, were on equal footing with investigators regarding governance and protocol development. The structure provided for equal numbers of investigators and providers on all major decision-making bodies, facilitating selection of clinically-relevant protocols with designs informed by considerations of public health impact. Substantial CTP roles in protocol development and planning prevented common problems that often undermine community based research. While early trials were comparatively simple two-group designs, the research capability of the CTN has matured such that it has successfully completed complex FDA-level medication trials, genetic studies, and trials with sophisticated adaptive designs (4
2. CTN has developed methodology for effectiveness research
Another major contribution of the CTN has been in articulation of methodological strategies for dissemination and effectiveness research, particularly for behavioral trials. Prior to the CTN, strategies for standard efficacy trials were comparatively well established (5
), and NIDA’s Behavioral Therapies Development branch played a critical role in articulating the Stage model of behavioral therapies development (7
). Although methodological standards for Stage I (feasibility and initial evaluation) and II (efficacy testing) research were well established, standards and strategies for Stage III (dissemination) research had been much less clear (9
) The CTN made multiple contributions to this area, including articulation of the ‘hybrid’ model for effectiveness research, which retains key features of efficacy research (such as randomization, integrity of the independent variable) but which includes multiple features of effectiveness research (10
). Other contributions have been made in articulation of models for training clinicians (11
), strategies for selection of comparison conditions (4
), and consideration of dissemination and sustainability (10
3. The CTN has demonstrated the value of secondary analyses
Well-designed trials yield far more than confirmations or refutations of primary hypotheses, in that secondary analyses often produce findings that address important questions and spark new areas of research. CTN trials have already produced scores of reports based on secondary analyses (www.CTNdisseminationlibary.org
). These include findings pointing to staging of therapies for substance using women with PTSD (13
), the efficacy of contingency management extends to methamphetamine users (14
), and that empirically validated therapies (EVTs) are efficacious among subgroups including ethnic minorities (15
4. CTN research has provided support for EVTs in real world settings
Another important conclusion drawn from the initial set of CTN trials is that treatments with strong prior efficacy data fared well in CTN trials, whereas more novel therapies or those with less substantial evidence bases tended not to outperform comparison conditions (16
). While this may reflect the efficacy of those therapies that are ‘tried and true’ prior to evaluation in the CTN, it may also reflect that treatments with extensive histories of evaluation carry with them refined training, implementation and outcome selection protocols that may facilitate their evaluation in multi-site trials. Furthermore, although many of the effect sizes from CTN trials have been modest, it should be noted that the bulk of CTN trials have used ‘active’ control comparisons (e.g., treatment as usual), and hence effect sizes are expected to be smaller than if less stringent control conditions, such as attention or wait list controls had been used (17
5. CTN research has confirmed the power of incentives
Consistent with a strong and consistent prior evidence base (19
), contingency management (CM) interventions demonstrated large short-term effects on stimulant use in both methadone and outpatient settings (21
). These trials also extended the evidence base by demonstrating the cost-effectiveness of prize-based CM (23
) and highlighted the broad utility of CM effectiveness by demonstrating efficacy in a subgroup of methamphetamine users (14
). Successful implementation of contingency management in 11 CTPs and positive results of the CTN trials appears to have fostered dissemination efforts and greater acceptance of CM principles in CTN-affiliated community programs (26
6. CTN research has provided a peek into ’treatment as usual’ (TAU)
Another important by-product of CTN trials is extensive data on the content and efficacy of TAU. In the many CTN trials which have used TAU as a comparison condition(4
), outcomes are typically quite positive in terms of retention and level of substance use (27
). For example, while multiple previous studies noted extremely high levels of attrition among Spanish-speaking substance users in outpatient treatment (28
), retention in both arms of the CTN study of MET delivered in Spanish was excellent, with over 90% of participants still enrolled one month after randomization (29
). It is not clear if this reflects (1
) particularly high quality treatment associated with CTN-affiliated programs (30
) the influence of issues related to clinical trials methods, including participant selection factors or the comparatively high level of patient and protocol monitoring that occurs in many randomized clinical trials, or (3
) standard treatments may be more effective than they often assumed to be. Thus, even when modest, the presence of significant effects over TAU for the EVT evaluated in the CTN is noteworthy.
7. The importance of retention
Another conclusion that may be drawn from CTN studies is that retention remains a major issue in substance abuse treatment (16
), particularly for those trials that have focused on detoxification and the early phases of treatment (32
). Retention remains a good predictor of longer-term outcome in the addictions, and multiple CTN studies have found modest rates of treatment exposure and retention in heterogeneous populations and settings. This suggests the field still has a long way to go to develop interventions, and a treatment system, that attracts and retains larger proportions of those who need treatment.
8. Contributing to the debates regarding safety of behavioral treatments
CTN trials include independent monitoring and review, involving multiple university Institutional Review Boards (IRBs) and Data Safety Monitoring Boards (DSMBs). These procedures followed the NIH 1998 policy requiring all Stage III trials, including behavioral trials, to have a DSMB to ensure safety of participants. While utilization of NIH procedures originally designed for medication trials resulted in a tremendous volume of adverse events reports, it also generated an opportunity to systematically evaluate rates of FDA-defined serious adverse events (SAEs) in large behavioral trials. Using data from four CTN studies (21
) which involved nearly 1700 participants, SAEs were found to be common, occurring in about 10% of all participants. However, not only were the rates of SAEs comparable in the usual care and experimental conditions (CM and motivation interviewing), but the DSMB also determined that none
of the SAEs were study-related (36
). A similar analysis of the incidence of gambling in the trials of prize-based CM also indicated no increase in gambling (37
). These data have led to articulation of more refined approaches for evaluating SAEs in behavioral trials see (36
9. Bridging the gap takes time and sustained attention
Another finding emerging from the CTN’s unique ability to conduct comprehensive evaluations of TAU is evidence that EVTs are still not yet broadly implemented in practice. For example, independent processs ratings of almost 500 sessions of TAU from early CTN trials suggested that interventions associated with specific EVTs were implemented infrequently and rarely at levels of adequately levels of fidelity (39
). Moreover, with respect to ratings by independent raters and expert supervisors, clinicians consistently overestimated the level of EVTs they used in sessions (42
) Multiple, sustained efforts are still needed to bridge the gap between research and practice (43
10. Effort, support, and commitment are essential for adopting and sustaining EVTs
The CTN has also demonstrated that adoption of EVTs carries with it substantial cost and organizational commitment. Costs of behavioral therapies include the interventions themselves, as well as training and supervision. The CTN has included evaluations of costs in several trials (23
); these have moved the field forward in demonstrating that the extra costs associated with implementing specific EVTs such as contingency management may vary widely across different clinics (24
). Although complex, these analyses are critical in helping policy makers decide if the extra costs of adopting EVTs are justified.
Top Ten Recommendations for the CTN’s Next Ten Years
Even with its many accomplishments and successes, the CTN has only been able to address a portion of the many challenges in addictions treatment. We next focus on a sample of methodological and research issues that are ripe for development for the CTN and more broadly for the field.
1. Need for a common outcome measure in drug addiction treatment trials
Although a common battery of assessments was used in many CTN trials, comparisons of participant outcomes across studies are not straightforward because different outcomes were used (e.g. percent of days abstinent from primary drug, self-report versus urinalysis, retention; see (16
) for overview). As each study selected a primary outcome based on its hypotheses and aims, there has been no ‘common denominator’ outcome measure that would provide a means of comparing outcomes across CTN trials. This has made it difficult to interpret findings when experimental treatments are not significantly different from TAU, to understand site effects, or to ‘benchmark’ outcomes across studies. Common outcomes have been adopted in other areas of mental health, including trials for nicotine (45
) and alcohol dependence (46
). The variability in patterns of substance use and their half lives are some factors that make it difficult to identify a single outcome measure; nevertheless, lack of a common outcome is a critical weakness in the drug treatment field. The various datasets generated by CTN trials should be used to inform this process, by identifying proxy markers of long-term outcome and reporting these consistently.
2. Incorporating performance indices and measures of clinical significance
CTN trials have relied on traditional methods of significance testing and estimations of effect size. These are important in many respects but also marked by clear limitations in terms of informing practical decisions about treatment choice, policy, and impact (48
). Indices marking clinically significant improvement have increasingly been adopted in treatment for alcohol use disorders (51
) but remain rare for illicit drug use (53
). Both the addiction research and treatment systems also lack consensus indicators of quality and performance that are now common in most areas of medicine. However, addiction treatment programs are not likely to have, or share, data regarding their outcomes so that individuals seeking treatment can make informed decisions regarding likely outcomes or provides success rates for different procedures or programs. Defining a set of common outcome-based performance measures would be extremely complex, but it is likely that development of consensus performance indicators would be likely to spur significant efforts to improve treatment outcome.
3. Comparative effectiveness studies
Progress in research on treatment for a range of addictions has generated a large number of science-based treatments, but comparatively little data that can be used to help clinicians and policy makers choose among available validated pharmacotherapies or behavioral therapies (12
); that is, there are relatively few direct comparisons of different active therapies (i.e., comparative effectiveness research) (54
). The size and scope of the CTN makes it well positioned to address complex issues regarding the relative efficacy of available treatments, in line with NIH priorities (55
) and calls from the public.
4. Addressing comorbidity
To date, the CTN has completed only two trials on populations with co-occurring psychiatric illness (56
). Continued attention to comorbidity research in the CTN is very important, as a substantial number of substance-dependent patients have psychiatric illness, and optimal approaches to so-called “dually diagnosed” patients have not been identified.. Studying comorbid patients is difficult, in part because of their heterogeneity in terms of issues such variability in different Axis I and II disorders, drugs of abuse, illness severity, and use of psychotropic medications. Although studying more homogeneous populations (e.g., patients with bipolar I disorder with cocaine dependence who are taking valproate) is methodologically “cleaner,” such findings are unlikely to be generalizable. The CTN, with its capacity to involve multiple sites with large numbers of diverse patients, is an ideal platform for clinically meaningful comorbidity research.
5. Mechanisms of action
Clinical trials are generally designed to answer basic questions of clinical or comparative effectiveness. While this type of information is essential for clinical decision making, it does not address the underlying mechanisms influencing drug use and other outcomes. This is especially so in the case of behavior therapies, since medications generally (though not always) have a known pharmacological action directly related to the outcome of interest (e.g. blockade of opioid reinforcement by naltrexone). Behavioral therapies are usually predicated on a theoretical or conceptual underpinning that suggests a mechanism of action. In fact, however, the mediating mechanisms of behavior change have been difficult to identify (58
). CTN trials provide a rich opportunity for more in-depth investigation of mechanisms of behavior change, and hence the potential to enhance the power and efficiency of treatment.
6. Studying dissemination as a means of building implementation science
The transfer of knowledge from bench to bedside is a universal challenge in medical practice, with substance abuse treatment being no exception (9
). There are at least two ways in which this transfer is encouraged within the world of providers. The first is professional licensing requirements for continuing education, but this does not ensure such practices are learned in sufficient depth to be used in practice or that they will in fact be adopted. A second pathway is through funding requirements, such as program payments that depend on assurance that EVTs are being used. Alternatively, programs may be paid bonuses based on achieving targeted clinical outcomes (60
) with the expectation that adoption of EVTs will improve outcomes. Neither approach, however, is grounded in evidence regarding the most effective, or cost-effective methods of improving treatment (e.g., what are the best strategies to train practitioners in EVTs? What supports are needed for clinical programs to encourage their adoption of EVTs or to implement them with adequate fidelity?) As in other areas of medicine, there is a tremendous need to understand more about the most effective methods for promoting adoption of evidence-based practices (61
). The CTN, with its branching network of community treatment providers, presents an ideal infrastructure in which to conduct implementation science.
7. Reaching those we don’t reach
Only a minority of individuals who meet diagnostic criteria for substance use disorders access specialty care services (63
). There are multiple reasons for this gap, including lack of access to care, lack of treatment financing (66
), scarcity of trained clinicians, lack of interest in seeking treatment, and stigma. The CTN has begun to address this issue in a new protocol evaluating the efficacy of screening, brief intervention and referral to treatment (67
) in emergency department settings. Future CTN studies should explore integration of treatment in settings such as primary medical, HIV, or dental care. Technology also holds promise for increasing treatment access, effectiveness, and availability, particularly for underserved populations (68
); the CTN has initiated a study evaluating computer-based treatment (70
). Additionally, strategies developed to improve healthcare by making them more patient-centered and customized to patient needs (e.g., adaptive treatment strategies) have been advocated (71
) and provide another potential CTN focus.
8. Sustainability of treatment effects
In addition to better understanding initial effects of interventions, a more thorough evaluation of long-term effects is necessary. To date, CTN trials have focused primarily on short-term approaches to treat addiction. For example, the bulk of CTN pharmacotherapy trials have focused on detoxification, and many of the behavioral trials have evaluated brief approaches. Follow-up evaluations have also been comparatively short, usually up to six months after the end of treatment (16
). Although this short-term focus has been appropriate and pragmatic, addiction is now widely acknowledged as a chronic relapsing condition (72
). Thus, understanding how these interventions fare over the long-term is important from patient and societal perspectives.. This is especially true since many interventions require training and implementation costs, which public payors ultimately will have to support. A large diverse network like the CTN could accurately capture long-term success rates as well as societal benefits that may be realized from implementation of effective treatments.
9. Longer-term trials and benchmarking
Adding longer term-follow ups to future CTN studies, while expensive, may be justified in helping both the scientific and treatment communities better understand the course of addiction across the lifespan and the role of interventions in affecting its course in individuals. Data from longer-term outcome and longitudinal studies are needed to design adaptive interventions and evaluate continuing care models (73
). Data from longer-term trials could also be used to develop benchmarks, that is, empirically-grounded expectations regarding outcome that can be used to set standards by which new or existing interventions could be compared. This could provide important guidance to policy makers, treatment providers and patients about what to expect from different programs.
10. Conducting trials more efficiently
A major challenge facing the CTN is the need to enhance efficiency and produce more findings more rapidly in the years ahead (74
). Application of contingency management principles may be particularly helpful in this regard, in that the flexibility of behavioral targets for CM enables these interventions to be used to enhance retention in trials, treatment exposure, and medication adherence in CTN trials. Multisite trials are very expensive; hence, substantial delays in recruitment, low rates of adherence, and loss of participants to follow-up add to these costs. Contingency management might be used as an optimal behavioral platform in large scale trials to enhance treatment adherence and foster higher rates of data acquisition and availability by reinforcing adherence with assessments (75
), particularly for efficacy or proof-of-concept studies where generalizability is less crucial.