|Home | About | Journals | Submit | Contact Us | Français|
This article provides a comprehensive review of research studies that have examined the diffusion of evidence-based treatments (EBTs) within the field of substance abuse treatment. Sixty-five research studies were identified and were grouped into one of three major classifications: attitudes toward EBTs, adoption of EBTs, and implementation of EBTs. This review suggests significant progress has been made with regard to the advancement of the fields’ knowledge about attitudes toward and the extent to which specific EBTs have been adopted in practice, as well as with regard to the identification of organizational factors related to EBT adoption. In an effort to advance the substance abuse treatment field towards evidence-based diffusion practices, recommendations are made for greater use of methodologically rigorous experimental or quasi-experimental designs, psychometrically sound instruments, and integration of quantitative and qualitative data collection.
In 1998 the Institute of Medicine (IOM) issued the now landmark report Bridging the Gap Between Research and Practice: Forging Partnerships with Community-Based Drug and Alcohol Treatment (IOM, 1998). Several tasks were charged to this committee, including the identification of promising research strategies which would help lessen the disparity between research and practice within the field of substance abuse treatment. Among the committee’s recommendations to improve implementation of research-based interventions in practice was the development of an infrastructure to facilitate research within a network of community-based treatment programs and the suggestion for states and federal agencies to develop financial incentives to encourage the inclusion of evidence-based treatments (EBTs) in community-based programs.
The National Institute on Drug Abuse (NIDA) has responded to these needs in a number of ways. One method of promoting greater diffusion of EBTs was publishing treatment manuals for several different approaches including: cognitive behavioral treatment (Carroll, 1998), the community reinforcement approach plus vouchers (Budney & Higgins, 1998), and individual drug counseling (Mercer & Woody, 1999). In 1999, NIDA established the Clinical Trials Network (CTN), which now has produced several articles demonstrating the effectiveness of different substance abuse treatments in community-based treatment settings (e.g., Amass et al., 2004; Ling et al., 2005; Peirce et al., 2006; Petry et al., 2005). In 2001 NIDA worked with the Substance Abuse and Mental Health Services Administration (SAMHSA) to create what is called the NIDA/SAMHSA Blending Initiative. A collaboration between NIDA CTN’s and SAMHSA Addiction Technology Transfer Centers (ATTCs) this initiative represents one of the most innovative efforts to date to improve the diffusion of research into practice. The general technology transfer strategy used as part of the Blending Initiative includes: 1) identification of promising CTN and/or other NIDA-funded findings that address gaps in the treatment field, 2) formation of “blending teams” (composed of representatives of the NIDA research and representatives from the ATTCs), which work closely together to develop training curricula, supervisory manuals, and strategic dissemination plans. Thus far, five different blending projects have been initiated and include: a) Short-Term Opioid Withdrawal Using Buprenorphine; b) Buprenorphine Treatment: Training for Multidisciplinary Addiction Professionals; c) Treatment Planning M.A.T.R.S.: Utilizing the Addiction Severity Index (ASI) to Make Required Data Collection Useful; d) Motivational Interviewing Assessment: Supervisory Tools for Enhancing Proficiency (MIA-STEP); and e) Promoting Awareness of Motivational Incentives (see Condon, Miner, Balmer, & Pintello, 2008 or http://www.nida.nih.gov/Blending for more details on each of these blending projects).
A historical overview of the federal government’s role in drug abuse technology transfer over the past 30 years has been provided by Brown and Flynn (2000). In addition to concluding that in order for progress to be made technology transfer must be embraced as a major responsibility of the federal government, Brown and Flynn posited a technology transfer model which emphasized the federal role, and included four elements: technology development (i.e., development of research agenda and selection of research projects), transfer preparation (i.e., selection of research findings appropriate for transfer), transfer implementation (i.e., conduction of core tasks of technology transfer), and transfer stabilization (i.e., maintenance of implementation efforts). Similar efforts to advance the transfer of research to practice, have produced conceptual models of the diffusion process (also referred to as technology transfer) (e.g., Simpson, 2002; Thomas, Wallack, Lee, McCarty, & Swift, 2003).
Simpson (2002) proposed a program change model for transferring research to practice, which incorporated findings from several relevant literatures (e.g., Backer et al., 1986; D’Aunno & Vaughn, 1995; Klein & Sorra, 1996; Rogers, 1995). As with the Brown & Flynn’s model, Simpson’s program change model has four main elements. These include exposure (i.e., via training or workshops), adoption (i.e., representing an intention to try an EBT), implementation (i.e., a period of trial use), and practice (i.e., incorporation of EBT into regular use and sustaining it). Within this program change model are several factors, such as organizational climate, staff attributes, and program resources, which are believed to influence the change process and ultimately determine the extent to which changes occur. Since the evaluation and refinement of a model depends on having appropriate assessments available, Simpson and colleagues developed several useful survey instruments, including the Organizational Readiness for Change (ORC; Lehman, Greener, & Simpson, 2002) and the Program Training Needs (PTN; Rowan-Szal, Greener, Joe, & Simpson, 2007). Recently, “heuristic refinements” have been made to Simpson’s (2002) program (see Simpson & Flynn, 2007) by broadening its scope to include strategic program planning and preparation and with the re-labeling of the first stage “training” rather than exposure.
As a decade has now passed since the substance abuse treatment field began its concerted effort to “bridge the gap” (IOM, 1998) it seemed appropriate to review the progress which has been made. Although important reviews related to this topic have been published previously (e.g., Fixsen, Naoom, Blasé, Friedman, & Wallace, 2005; Gotham, 2004; Miller, Sorensen, Selzer, & Brigham, 2005), this review is the first to systematically identify and review research studies focusing on the diffusion of EBTs within the field of substance abuse treatment. That is, of the 419 references included in the often cited review by Fixsen et al. (2005) less than two percent were related to substance abuse treatment. Furthermore, although the review by Gotham (2004) included several substance abuse treatment research studies, its greater emphasis on reviewing how diffusion research and theory could be applied to mental health treatment does not fully capture the state of diffusion research within the substance abuse treatment field. Finally, although Miller and colleagues (2006) provided a good review of methods for disseminating new treatment methods, the availability of new studies during the last two to three years already warrants updating. Thus, the primary purpose of this review was to systematically identify research studies within the field of substance abuse treatment which have examined: a) attitudes towards EBTs (i.e., studies which examined beliefs or attitudes regarding the effectiveness or use of EBTs), b) adoption of EBTs (i.e., studies which examined the extent to which staff and/or agencies reported having adopted specific EBTs), or c) implementation of EBTs (i.e., studies which examined the extent to which and/or the processes through which an EBT was implemented).
A systematic review of substance abuse treatment literature was conducted to identify research studies published between 1998 and July 2008 that examined diffusion of EBTs within U.S. community-based substance abuse treatment centers. In order to meet the definition of a “research study” the paper had to describe the methods and findings from a quantitative or qualitative data-based study. For the purposes of this review, EBTs included both pharmacological (e.g., naltrexone, buprenorphine) and psychosocial interventions (e.g., community reinforcement approach, contingency management) for the treatment of substance abuse or dependence. Two primary methods were used in conducting the literature search. The first, was a keyword search using the PsychINFO, Medline, and Implementation Science databases, where key search terms included: “adopting,” “adoption,” “attitudes,” “Clinical Trials Network,” “diffusion,” “disseminating,” “dissemination,” “evidence-based,” “implementing,” “implementation,” “implementation research,” “implementation science,” “substance abuse,” and “technology transfer.” The second method included searching the bibliographies of articles identified as being relevant. As a systematic review of the effectiveness of workshop training studies already has been conducted (see Walters, Matson, Baer, & Ziedonis, 2005), the current review only includes training-related articles that were not included in or were published subsequent to the Walters et al. (2005) article. Finally, articles with an emphasis on something other than substance abuse treatment (e.g., psychotherapy, mental health treatment) also were excluded from the current review.
A total of 65 research studies were identified based on the methods described above. As previously noted, several articles related to EBT training (e.g., Miller, Yahne, Moyers, Martinez, & Pirritano, 2004; Morgenstern, Morgan, McCrady, Keller, & Carroll, 2001; Sholomskas et al., 2005) were excluded from the current review because they had already been reviewed by Walters and colleagues. Additionally, several articles were excluded (e.g., Carroll et al., 2002; Liddle et al., 2002; Robbins, Bachrach, & Szapocznik, 2002) because they did not meet criteria for classification as a “research study.”
Articles identified were classified into one of three major classifications including: a) attitudes towards EBTs, b) adoption of EBTs, and c) implementation of EBTs (defined previously in the introduction). Although many articles could be classified into only one of these, others could have been classified into more than one. In order to prevent tables from becoming unnecessarily long and redundant, articles were placed in the classification which appeared most relevant. Within each major classification studies have been ordered by year (and alphabetically within year) in order to give readers a feel for how the field has progressed over time. Within each of the three major classifications, studies also have been sub-classified as being focused on: a) pharmacological EBTs only, b) psychosocial EBTs only, or c) both pharmacological and psychosocial EBTs. Membership in each sub-classification has been indicated with superscripts in the tables.
Twenty-five research studies (38% of all studies identified) were classified as having examined attitudes toward EBTs. As indicated by superscripts in Table 1, 12 (48%) of the studies within this classification examined attitudes towards pharmacological EBTs only, 6 (24%) examined attitudes toward psychosocial EBTs only, and 7 (28%) examined attitudes toward both types of EBTs.
Two of the first studies identified as part of this review examined patients’ attitudes towards the use of pharmacological treatment for alcoholism. The first study, conduced by Swift, Duncan, Nirenberg, and Femino (1998), assessed 127 alcoholic patients’ attitudes regarding use of medications such as naltrexone, which received FDA approval in 1994 for the treatment of alcohol dependence. Results of their study indicated 14% of patients felt naltrexone was helpful, but only 6% reported having had taken naltrexone. In a similar study, Rychtarik, Connors, Dermen, & Stasiewicz (2000) assessed the attitudes of 277 members of Alcoholics Anonymous (AA) regarding use of medications to prevent relapse, as well as their experiences with the use of such medication. Results of their study indicated that 73% of members reported the use of medications to prevent relapse either “might be” or “was” a good idea. Additionally, results of the study found AA meeting attendance to be negatively related to attitudes toward use of medications. Thus, based upon these two studies, there appears to be mixed patient support for using pharmacological treatments, such as naltrexone.
In contrast to these first two studies, the majority of studies that examined attitudes toward EBTs focused on staff members, such as: physicians and/or physician assistants (Mark et al., 2003c; Roose, Kunins, Sohler, Elam, & Cunningham, 2008; West et al., 2004), program leaders (Willenbring et al., 2004), and counseling staff (e.g., Forman, Bovasso, & Woody, 2001; Knudsen, Ducharme, & Roman, 2007; Knudsen, Ducharme, Roman, & Link, 2005; McCarty et al., 2007; McGovern, Fox, Xie, & Drake, 2004; Rieckmann, Daley, Fuller, Thomas, & McCarty, 2007). The general state of attitudes toward pharmacological EBTs is best illustrated by the results of an early study conducted by Forman et al. (2001). Using a confidential survey Forman and colleagues assessed the beliefs of staff about addiction treatment prior to the initiation of the CTN research protocols, including the extent to which staff believed “new approaches” and specific medications should be used more. Results indicated although 80% of staff agreed “new approaches” should be used more, less than 40% of staff agreed medications such as naltrexone or methadone maintenance should be used more. Willenbring and colleagues (2004) similarly found although program leaders of Veteran Affairs reported generally positive beliefs about clinical practice guidelines, only 46% agreed naltrexone should be routinely recommended for alcohol dependence.
In addition to studies that assessed attitudes towards pharmacological EBTs, such as naltrexone, research also has assessed attitudes toward buprenorphine, which received FDA approval in 2002 for the treatment of opiate dependence. Early studies indicated generally low support for or knowledge regarding buprenorphine. For example, West and colleagues (2004) found that 81% of the 1,206 physician psychiatrists they surveyed were not comfortable providing buprenorphine treatment, while Knudsen, Ducharme, Roman, and Link (2004) found 86% of the 2,298 counselors surveyed from both private and public community-based treatment programs not aware of the effectiveness of buprenorphine. Fortunately, the extant literature indicates training and involvement in a research network have been found to be associated with more positive attitudes toward pharmacological EBTs (Knudsen, Ducharme, & Roman, 2007; McCarty, Rieckmann, Green, Gallon, & Knudsen, 2004). For instance, McCarty et al (2004) used the 10-steps included in the Addiction Technology Transfer Centers (ATTC, 2000)The Change Book to structure the change process for the Opiate Medication Initiative for Rural Oregon Residents (OMIROR) project. Pre-post measures of attitudes indicated staff reported significantly higher attitudes and beliefs regarding buprenorphine’s effectiveness, ability to save lives, impact on patients’ health, and ability to block heroin cravings following the training. Similarly, Knudsen, Ducharme, and Roman (2007) found counselors who reported greater receipt of buprenorphine training were significantly more likely to also report greater ratings of buprenorphine acceptability. Knudsen et al. (2007) also found CTN-affiliated counselors reported significantly greater acceptability of buprenorphine compared to non-CTN affiliated counselors.
In addition to affiliation with a research network, such as CTN, other organizational factors have been shown to be correlated with attitudes toward pharmacological EBTs. For instance, Fuller et al. (2007) used data from 205 treatment units to examine multilevel relationships between several scales included as part of the ORC instrument and attitudes toward several treatment practices. Results indicated 3 of the 18 ORC scales (i.e., growth, internet, and program needs) were significantly related to attitudes toward medications.
Review of the literature indicates there are generally more positive attitudes and support for psychosocial EBTs, relative to pharmacological EBTs. For instance, whereas only 46% of VA program leaders in the survey conducted by Willenbring and colleagues agreed naltrexone should be routinely recommended, 93% of respondents agreed there should be more routine recommendation of cognitive behavioral relapse prevention. Greater support for specific psychosocial EBTs also has been suggested by the findings of McGovern et al. (2004) who reported clinicians were more likely to say they were motivated to adopt twelve-step facilitation, cognitive behavioral therapy, motivation interviewing, and relapse therapy, relative to contingency management, behavioral couples therapy, or pharmacotherapies.
Treatment manuals are viewed as essential to providing guidance for the implementation of an EBT and research also has examined therapists’ attitudes towards them. As part of the CSAT funded Cannabis Youth Treatment (CYT) project (see Dennis et al., 2002, 2004 for more details on CTY project) Godley, White, Diamond, Passetti, and Titus (2001) used qualitative interviews to examine staff reactions to the use of a manual guided intervention for adolescent marijuana users. Results indicated 100% of clinicians agreed the treatment manuals provided structure and consistency to their therapeutic work, however, 42% also indicated the manuals restricted their ability to respond to individual client needs to some extent. Further, the type of treatment manual (i.e., session-based, procedure-based, or principle-based) was related to how clinicians perceived that the intervention lent itself to a flexible and individualized implementation. More recently, McCarty et al. (2007) surveyed 3,698 staff from 384 treatment units within 106 CTN community treatment programs. On a scale of 1 to 5 (strongly disagree to strongly agree), the average counselor rating regarding the usefulness of treatment manuals for learning new interventions was 3.99 (SD = .66). The average counselor rating regarding whether treatment manuals interfere with treatment was 2.20 (SD = .76). Thus, based upon these findings, it appears there are generally positive attitudes towards treatment manuals.
Similar to the previously described study by Fuller et al. (2007), two studies (Henggeler et al., 2007; Saldana, Chapman, Henggeler, & Rowland, 2007) used the ORC instrument to examine correlates of attitudes toward psychosocial EBTs. Henggeler et al. used data from 432 substance abuse and mental health practitioners to examine predictors of voluntary attendance to a workshop on contingency management. Demographic characteristics, attitudes towards treatment manuals or evidence-based practices were not significantly associated with workshop attendance. Workshop attendance was, however, significantly higher among participants who reported greater program motivation for change, better organizational climates, and lower program resources. Based upon data from 543 community-based therapists Saldana et al. examined the association between the four ORC domains, the Evidence-Based Practice Attitude Scale (Aarons, 2004), and attitudes toward treatment manuals (Addis & Krasnow, 2000) at both the therapist and agency level. Results indicated therapist-level responses on the ORC provided better estimates of therapists attitudes toward evidence-based practices and treatment manuals and that therapist openness to evidence-based practices was significantly associated with more positive ratings on several ORC domains (i.e., Staff Attributes, Training Exposure & Utilization, and Motivational Readiness for Change).
Thirty-one research studies (48% of all studies identified) focused primarily on the adoption of EBTs. As indicated by superscripts in Table 2, 18 (58%) of the studies within the adoption of EBT classification examined adoption of pharmacological EBTs only, 9 (29%) examined adoption of psychosocial EBTs only, and 4 (13%) examined adoption of both types of EBTs.
Most studies of pharmacological EBT adoption have focused on the adoption of naltrexone. Overall, rates of naltrexone adoption are varied depending on the sample used and year the survey was conducted. For example, one of the first publications of the rates of naltrexone adoption in the US was conducted by Roman and Johnson (2002). Based upon this 2000 survey of a nationally representative sample of approximately 450 treatment centers in the US, Roman and Johnson (2002) found 44% of centers reported adoption of naltrexone. Results of a 1999 survey of 1,251 clinicians (135 physicians and 1,116 nonphysicians) in three states (i.e., Massachusetts, Tennessee, and Washington) conducted by Thomas, Wallack, Lee, McCarty, and Swift (2003) found a similar rate of naltrexone adoption, with 45% of physicians reporting that they had prescribed or recommended naltrexone occasionally. In contrast, however, Fuller, Rieckmann, McCarty, Smith, and Levine (2005) found, based upon three surveys (conducted in 1997conducted in 1999, and 2001) of over 200 outpatient substance abuse treatment centers in the northeast, naltrexone adoption rates of 14%, 17%, and 25%, respectively. A recent study by Knudsen, Ducharme, & Roman (2007a), which surveyed the largest number of substance abuse treatment centers in the U.S. thus far, probably provides the best current estimate of naltrexone adoption. Data from 403 privately funded and 363 publicly funded substance abuse treatment centers surveyed between 2002 and 2004 indicates an overall naltrexone adoption rate of 21%. However, naltrexone adoption was significantly lower for public non-profit centers (7%) compared to both for-profit and non-profit private centers (33% each). Thus, the extant literature suggests roughly 1 in 5 substance abuse treatment centers report having adopted naltrexone.
Given the eight year gap between the FDA’s approval for naltrexone (approved in 1994) and buprenorphine (approved in 2002), it is not surprising that fewer research studies have been conducted on the adoption of buprenorphine. In 2003, Koch, Arfken, and Schuster (2006) examined the extent to which buprenorphine was offered in substance abuse facilities during its initial stage of availability. A survey of 13,060 facilities, revealed an overall adoption rate of 5.5%, with opiate treatment programs (OTP) being significantly more likely than non-OTP programs to offer buprenorphine (11% vs. 5%, odds ratio = 2.39). Using data collected from 576 substance abuse treatment centers (299 privately-funded and 277 publicly funded) Knudsen, Ducharme, & Roman (2006) found 6% of centers reported use of buprenorphine at the baseline survey (conducted between 2002 and 2004), while 14% reported use at the 12-month follow-up survey (conducted between 2003 and 2005). Results also showed private treatment centers reported significantly greater rates of adoption at both time points (baseline = 10%; follow-up = 21%) compared to public treatment centers (baseline = 3%; follow-up = 7%).
Although numerous studies have examined organizational correlates of pharmacological EBT adoption the majority of these studies have been conducted by researchers at the Institute for Behavioral Research at the University of Georgia (e.g., Ducharme, Knudsen, & Roman (2006a & b; Ducharme, Knudsen, Roman, & Johnson, 2007; Knudsen, Ducharme, & Roman, 2007a; Knudsen, Ducharme, Roman, & Link, 2006; Knudsen & Roman, 2004; Knudsen, Roman, Ducharme, & Johnson, 2005). Organizational factors significantly correlated with buprenorphine adoption have included: offering detoxification services, current use of naltrexone, being a for-profit organization, and being an accredited organization. It should be noted, support for some of these factors appeared to be related to the sample and analytic method used. For example, for-profit and accreditation status were significant predictors in the final multivariate models of buprenorphine adoption by Knudsen, Ducharme, & Roman (2006). However, neither of these factors reached significance in the multivariate model of Ducharme, Knudsen, Roman, & Johnson (2007). One possible explanation for these differences may be the inclusion of a measure of direct exposure to buprenorphine (via participation in NIDA’s CTN), which Ducharme and colleagues found to have the highest odds ratio among all factors.
Research on the adoption of psychosocial EBTs has focused on a broad range of EBTs (e.g., behavioral couples therapy [BCT], community reinforcement approach [CRA], motivation enhancement therapy [MET]). One of the first studies was conducted by Fals-Stewart and Birchler (2001). Results of their telephone survey with 398 substance abuse treatment program administrators indicated none of the respondents used BCT. Moreover, despite results from multiple published studies demonstrating that BCT is an effective treatment for married or cohabiting alcohol-and drug-abusing patients, only 3% of the respondents reported being aware of BCT for this population. Knudsen, Johnson, Roman, and Oser (2003) compared rural (n = 67) and urban (n = 238) treatment centers with regard to the adoption of several different treatment innovations. Results of their study indicated 37% of rural and 48% of urban centers reported adoption of MET, while 55% of rural and 48% of urban centers reported adoption of CRA. However, rural and urban centers did not significantly differ with regard to their adoption of these EBTs.
Given training workshops are widely used for diffusing EBTs, Dwayne Simpson and colleagues at the Institute of Behavioral Research at Texas Christian University (TCU) have examined the extent to which staff report having adopted innovations learned in workshops. Bartholomew, Joe, Rowan-Szal, & Simpson (2007) examined relationships between workshop evaluations and 6-month post workshop follow-up assessments from over 200 counselors. Training relevance and training engagement were significant predictors of adoption of training materials, while lack of time and already using similar materials were reported as being among the top barriers to adoption of workshop training materials. Using a sample of staff and clients from approximately 60 substance abuse treatment units Simpson, Joe, and Rowan-Szal (2007) examined longitudinal interrelationships among program-level organizational measures of needs and functioning, workshop training, innovation adoption, client functioning, and follow-up changes in training needs collected at six separate time points during a two-year period. Results of their study indicated adoption of workshop training materials was significantly associated with lower perceived barriers to training assessed 12-months prior to training, as well as higher perceptions of organizational climate (e.g., Mission, Openness to Change) assessed 4-months prior to training.
Two studies used experimental methods to test strategies or interventions for impacting the adoption process. Peters et al. (2005) used a quasi-experimental design to examine the effectiveness of intensive counselor training sessions combined with peer opinion leaders relative to a standard training session with only resource materials (e.g., treatment manual) provided to counselors. Although the experimental and standard training groups did not significantly differ with regard to attitudes or the level of knowledge acquired about the treatment, counselors in the experimental group were significantly more likely to report having adopted the treatment manual relative to the standard training group (48% vs. 19%, p < .05). More recently, Squires, Gumbley, and Storti (2008) reported the results of a study based on a change model called the Science to Service Laboratory (SSL), which is designed to increase the perceived attributes of EBTs via training and support, and is grounded on several of principles of The Change Book (ATTC, 2004). Out of 54 agencies that applied to participate in the SSL training, 34 (63%) completed the training. Twenty-eight agencies completed the SSL training and of these, 26 (93%) successfully adopted and implemented contingency management. Agencies with more turnover of their assigned technology transfer specialist were more likely to drop out of the training. Because of the absence of a comparison change strategy it is, however, difficult to assess if the method under study was better or worse than other approaches.
Nine research studies (14% of all studies identified) focused primarily on the implementation of EBTs (see Table 3). As indicated by superscripts in Table 3, only one study (11%) examined implementation of pharmacological EBTs only, eight studies (89%) examined implementation of psychosocial EBTs only, and none of the studies identified as part of this review examined implementation of both EBT types.
In contrast to the two other major classifications of studies (attitudes toward EBTs and adoption of EBTs) few studies were identified that examined implementation of pharmacological EBTs. Amass and colleagues (2004) did include a description of their “general experience bringing buprenorphine-naloxone to community treatment programs, protocol compliance, and regulatory issues,” however, because of the lack of any quantitatively or qualitatively collected data this study did not meet the criteria for inclusion in the current review, nor did three other initially identified CTN trials (Ling et al., 2005; Peirce et al., 2006; Petry et al., 2005). Indeed, only one study was found which met the criteria for being included as a research study on the implementation of pharmacological EBTs (Kovas, McFarland, McCarty, Boverman, & Thayer, 2007). In a retrospective chart review of 200 patients outcomes with either clonidine or buprenorphine, Kovas and colleagues found clients treated with buprenorphine had significantly longer lengths of stay in treatment (5.7 vs. 3.5 days) and “completed” discharges (67% vs. 46%).
Among the eight research studies that examined psychosocial EBT implementation five used qualitative interviews (Brown, 2004; Guydish, Tajima, Manser, & Jessup, 2007; Hayashi, Suzuki, Hubbard, Huang, & Cobb, 2003; Obert et al., 2005; Riley, Rieckmann, & McCarty, 2008) and three used quantitative methods (Andrzejewski, Kirby, Morral, & Iguchi, 2001; Keller & Galanter, 1999; Liddle et al., 2006). Based upon qualitative interviews with 85 clinical and research staff that had participated in CSAT’s Methamphetamine Treatment Project, both Brown (2004) and Obert et al. examined the process of implementing the manual-based Matrix model. Results from these two studies indicated that nearly everyone interviewed expressed concern about the ability to meet clients’ needs within the context of a research study and that conducting a randomized clinical trial within a research-naïve clinical setting may not be an ideal way to introduce evidence-based practices. In another qualitative study Guydish, Tajima, Manser, and Jessup (2007) gathered data to help inform how technologies of multisite clinical trials might be modified to better support adoption of EBTs. Although Guydish and colleagues used the word adoption, this study was classified as an implementation study, as it did not assess rates of adoption, but rather the processes of adopting EBTs, which fits the definition of implementation as defined in this paper. Based upon interviews with 71 individuals representing eight organizational roles (e.g., clinic staff, clinic director, intervention designer) they identified six strategies including: 1) plan for adoption, 2) train senior staff how to deliver the EBT, 3) use a regional training model, 4) use a local supervision model, 5) bring EBT training back to the clinic, and 6) bring study findings back to clinic.
Among the studies that employed quantitative methods to examine the implementation of psychosocial EBTs, one was part of a larger investigation evaluating CM and examined the extent to which graphical feedback (i.e., bar graph representing a counselor’s performance) and positive reinforcement (i.e., cash prize drawing) increased staff implementation of CM with clients (Andrzejewski, Kirby, Morral, & Iguchi 2001). One condition was introduced at a time in order to evaluate the additive influence of each component. Five of the ten counselors were exposed to the graphical feedback condition and six of the ten were exposed to the “drawing” condition. Results indicated graphical feedback increased staff implementation by 69% (from 41% to 71%), while the cash prize drawing increased staff implementation by 93% (from 41% to 81%). There was not, however, a significant difference between these two experimental conditions. In another implementation study, Liddle and colleagues (2006) examined longer-term implementation using a multiphase (i.e., 12-month baseline, 6-month training, 14-month implementation, and 18-month durability phase) interrupted time-series design to test a “technology transfer intervention,” which has been described previously (Liddle et al., 2002). Several significant improvements were noted regarding the number of individual and family sessions, the content focus of sessions, and adherence to the treatment intervention. Additionally, clients reported significantly greater program control, clarity, and practical orientation during the implementation and durability phases as compared to the baseline phase. Perhaps most important, a significantly larger proportion of clients reported abstaining from drugs in the implementation and durability phases compared to those in the baseline phase.
This paper systematically reviewed the last decade of scientific research addressing attitudes towards, adoption of, and implementation of EBTs within the field of substance abuse treatment. Based upon this review significant progress has been made, especially with regard to the advancement of the fields’ knowledge about attitudes toward EBTs and the extent to which specific EBTs are adopted in practice. However, in contrast to the 25 studies on attitudes toward EBT and the 31 research studies on EBT adoption, only nine research studies were found to have examined implementation of EBTs, which suggests the substance abuse treatment field is still in the early stages of development and knowledge regarding implementation of EBTs. Certainly numerous reasons exist for the disparate number of research studies on EBT implementation, and future research on identifying such barriers is clearly warranted. Seemingly, one of the most significant barriers to implementation research may be the lack of objective criteria for what determines when EBT implementation has or has not occurred in practice. For instance, according to Fixsen et al. (2005) implementation is defined as “a specified set of activities designed to put into practice an activity or program of known dimensions.” Importantly, both the activity/program being implemented and specified set of activities are supposed to be described in enough detail to allow independent observers to assess the presence and strength of each. Although EBTs generally have associated manuals and/or treatment fidelity measures, there is typically little guidance on the extent to which modifications (desirable or undesirable) can be made and still allow the EBT to be considered implemented as planned by the treatment model developers (i.e., level of implementation tested and shown to be effective as part of randomized clinical trials). Given such modifications or “re-invention” as labeled by Rogers (2003) is generally the rule rather than the exception in implementation efforts, it would seem to behoove the field as a whole to work towards the development of implementation criteria that would aid future studies of EBT implementation.
Considerable research to date has examined attitudes towards EBTs with the majority of studies having addressed attitudes toward pharmacological EBTs. Interestingly, the literature seems to suggest generally more positive attitudes are held towards the use of psychosocial EBTs, relative to pharmacological EBTs. In fact, a recent study by Thomas and Miller (2007) found 65% of the 84 counselors and administrators surveyed agreed pharmacologic interventions work best only if accompanied by psychosocial interventions. What appears to be one of the primary reasons why staff have less positive attitudes toward pharmacological EBTs is simply lack of knowledge about pharmacological EBTs relative to psychosocial EBTs. That is, unfortunately education and/or training about pharmacological EBTs are not provided to community-based substance abuse treatment staff and which is reflected in the fact that Knudsen et al. (2004) found 1,972 of the 2,298 (86%) substance abuse treatment counselors surveyed to report not knowing about the effectiveness of Buprenorphine. Thus, in order to improve attitudes toward pharmacological EBTs it is imperative that future initiatives target increasing counselors’ knowledge about the effectiveness of available EBTs, both psychosocial and pharmacological. Fortunately, as has been shown by McCarty and colleagues (2004) staff attitudes toward pharmacological EBTs can be improved by providing trainings on the potential benefits of using such practices.
Almost half of studies identified as part of the current review focused on adoption of EBTs with the majority of these studies focused on adoption of pharmacological EBTs such as naltrexone or buprenorphine. Despite the large number of studies conducted it was not easy to determine a single estimate of the rates of adoption, given estimates varied considerably depending on the sample surveyed and the year the survey was conducted. As noted previously, Knudsen, Ducharme, & Roman (2007a) appear to have provided the best current estimate of the rate of naltrexone adoption within US substance abuse treatment centers at 21%, which is based upon data from 766 treatment centers surveyed between 2002 and 2004. However, given less than two percent of the 194,001 Veteran Affairs outpatients diagnosed with alcoholism between October 2000 and March 2001 were prescribed naltrexone (Petrakis, Leslie, & Rosenheck, 2003) suggests that simply because a center has “adopted” a EBT does not guarantee all who may benefit from the treatment will receive it. Nevertheless, rates of naltrexone adoption do appear to be higher than the rates of buprenorphine adoption, which based upon available estimates ranges between 5.5% (Koch et al., 2006) and 14% (Knudsen, Ducharme, & Roman, 2006). Perhaps due to the wider range of available EBTs or possible the generally more positive attitudes held towards them, considerably less research to date has examined adoption of psychosocial EBTs. However, among the studies that have provide rates of adoption for psychosocial EBTs one of the highest rate of adoption was found for CRA at 49% (Knudsen et al., 2003).
Only nine of the 65 research studies (14%) identified in this review focused on implementation of an EBT. As noted above, this relative paucity of implementation research suggests the substance abuse treatment field is ripe for further research in this area. In addition to the previously noted recommendation to develop more objective implementation criteria six strategies to help encourage better EBT implementation were given by Guydish et al. (2007) with a common theme among several of these strategies being related to “training” (e.g., training senior staff how to deliver the EBT, using a regional training model, and bring EBT training back to the clinic). Two good examples of the types of implementation research studies which are needed to significantly enhance the field have been provided by Andrzejewski et al. (2001) and Liddle et al. (2006). However, this is not to say that these studies were not without their limitations. For instance, the study by Andrzejewski and colleagues was based upon a very small sample of only ten counselors, which limits the generalizability of the findings. An important limitation of Liddle and colleagues study was the high degree of staff turnover. Given only one of the five social workers participated during all of the study phases it is not possible to conclude whether the improvements found during the implementation phase of the Liddle et al study are attributed to the training intervention or simply to having hired more competent staff. Nevertheless, this study is notable for being the only study to have examined longer-term implementation (i.e., sustainability).
Although not a primary focus of the current review, it does seem relevant to note that for the most part research studies did appropriately apply the various terminology, which represents a significant advancement within the field given previous reviews have noted the varied and inconsistent usage of terms (Gotham, 2004; Fixsen et al., 2005). Additionally, it seems relevant to note that the term “implementation science,” which was defined by Perl (2006) as “the study of principles and methods to promote the systematic adoption and implementation of scientific advances into real-world practice” was not found to be used in any of the studies identified. Indeed, using implementation science as a keyword in Psychinfo and Medline searches resulted in only 10 articles, none of which were related to substance abuse treatment. Similarly, substance abuse as keyword search in Implementation Science, which is an online journal published by BioMed Central, resulted in only six articles – none of which met inclusion criteria for the current study. Rather, it appears most common for researchers to discuss specific stages, such as adoption or implementation, or the general process of “technology transfer”, which as noted by Gotham (2004) also is sometimes referred to as diffusion of innovations. Given the theoretical basis for many conceptual frameworks is actually based upon Rogers’ (1962, 1983, 1995, 2003) diffusion of innovation theory, it may be appropriate to begin referring to the study of how innovations spread (planned or unplanned) as “diffusology.” Although, adoption, implementation, and sustainability should still be used to be more specific about which stage of the diffusion process is being examined, this broader term would be more consistent with how fields of study are typically labeled. For instance, the study of the origin and spread of diseases is referred to as epidemiology, not disease transfer.
Primary limitations of the current review are that it only includes research studies conducted within the U.S. substance abuse treatment field and published after 1998. Although beyond the scope of the current review, diffusion of EBTs is of considerable interest in other fields of research. As this is particularly true of mental health treatment (e.g., Aarons, 2004, Gotham, 2004; Henggeler, Melton, Brondino, Scherer, & Hanley, 1997; Schoenwald, Sheidow, Letourneau, & Liao, 2003) it behooves substance abuse treatment researchers and clinicians to familiarize themselves with these types of related research. Additional limitations of the findings of the current review include use of cross-sectional survey methods, low survey response rates, and heavy reliance on respondent self reports.
As discussed previously, significant progress has been made in the advancement of knowledge regarding EBT diffusion research within substance abuse treatment. Nonetheless, there remains considerable opportunity for further study. In particular, there is a need for more experimental or quasi-experimental studies which test different methods of improving the adoption and/or implementation of EBTs. One of the best examples to date, which was previously reviewed in Walters and colleagues (2005) systematic review of the effectiveness of workshop training for psychosocial addiction treatment, was a study by Miller et al. (2004). Using a randomized design, Miller and colleagues tested four methods to help therapists learn Motivational Interviewing (MI). These conditions included 1) workshop only, 2) workshop plus practice feedback, 3) workshop plus individual coaching, and 4) workshop, feedback, and coaching. Overall, results of their study indicated only clinicians in the full training condition (i.e., workshop, feedback, and coaching) showed significantly better client responses compared to baseline.
Related to the need for more controlled studies on methods to improve the diffusion of EBTs is the need for better measures of adoption and implementation. To date, diffusion research has relied heavily on “yes” or “no” self-reported responses, with little, if any, external verification of the validity of such reports. Although reliance on self-report remains far from perfect, such methods are at least much more suited for studies of EBT adoption than they are for studies of EBT implementation given the latter generally require more information about the extent of and/or quality of implementation (i.e., implementation fidelity). Indeed, among the studies included in this review, only a couple (Henggeler et al., 2008; Liddle et al., 2006) used an implementation measure to collect this type of information. Thus, in sum it is concluded that in order for the field of substance abuse treatment to further advance its knowledge about how to best diffuse EBTs, future research needs to develop psychometrically sound measures of adoption and implementation, and more importantly use these measures to develop and experimentally test different diffusion interventions. Additionally, future researchers are encouraged to employ mixed methods (integration of both quantitative and qualitative method), which also has been recommended by others (Dennis, Fetterman, & Sechrest, 1994; Dennis, Perl, Huebner, & McLellan, 2000). Use of methodologically rigorous experimental or quasi-experimental design, psychometrically sound instruments, and integration of quantitative and qualitative data collection should hopefully lead one day to the field having not only EBTs, but evidence-based diffusion practices.
This work was supported by contract 270-2003-00006 with Center for Substance Abuse Treatment (CSAT), Substance Abuse and Mental Health Services Administration (SAMHSA), the National Institute on Alcohol and Alcoholism Abuse (NIAAA grant AA017625, AA10368), and the National Institute on Drug Abuse (NIDA grant DA018183). The author wishes to thank Drs. Mark and Susan Godley for their support in preparing this review, as well as for providing feedback on drafts of the manuscript. The author also wishes to acknowledge Christopher Roberts and Kelli Wright for their assistance in preparing this manuscript for submission. The opinions are those of the author and do not represent official positions of the government.
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.