PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
Psychiatr Serv. Author manuscript; available in PMC 2017 March 22.
Published in final edited form as:
PMCID: PMC5361218
NIHMSID: NIHMS850800

A Web-Based Platform to Support an Evidence-Based Mental Health Intervention: Lessons From the CBITS Web Site

Abstract

Objective

To explore the role of Web-based platforms in behavioral health, the study examined usage of a Web site for supporting training and implementation of an evidence-based intervention.

Methods

Using data from an online registration survey and Google Analytics, the investigators examined user characteristics and Web site utilization.

Results

Site engagement was substantial across user groups. Visit duration differed by registrants’ characteristics. Less experienced clinicians spent more time on the Web site. The training section accounted for most page views across user groups. Individuals previously trained in the Cognitive-Behavioral Intervention for Trauma in Schools intervention viewed more implementation assistance and online community pages than did other user groups.

Conclusions

Web-based platforms have the potential to support training and implementation of evidence-based interventions for clinicians of varying levels of experience and may facilitate more rapid dissemination. Web-based platforms may be promising for trauma-related interventions, because training and implementation support should be readily available after a traumatic event.

The tremendous progress in developing evidence-based practices to improve mental health care quality has not been matched by successful implementation of such interventions in community settings (1). Few clinicians enter the workforce trained to implement evidence-based practices (2), and training the current workforce is challenged by limited infrastructure, funding, and support (3). Moreover, the “train and hope” (4) approach (training clinicians without continued implementation support) has resulted in low use of evidence-based practices among community clinicians (3,5). The demand for effective mental health services will likely increase substantially because of recent policies such as the Affordable Care Act (ACA). However, the existing mental health workforce and training infrastructure is unlikely to meet this increased demand, requiring new mechanisms to expand and enhance successful training in and implementation of evidence-based practices (1).

Many clinical fields have increased the use of distance learning via Web sites to enhance workforce skills (6). Recently, there has been a comparable increase in behavioral health care, with Web-based training for substance abuse treatment (7), dialectical behavior therapy (8), and cognitive-behavioral therapy (9). Such efforts have been demonstrated to be as effective as in-person training in improving professionals’ knowledge and skills (7,8,10). However, little is known about who uses such Web sites or how users interact with them.

To increase understanding of the rapidly growing area of distance learning in behavioral health, we examined user characteristics and usage of a Web site that supports training and implementation of an evidence-based intervention for traumatized youths. Findings may inform future efforts to enhance training and implementation support through such Web sites and may begin to inform how Web-based platforms can be used to rapidly disseminate knowledge after traumatic events.

Methods

The Cognitive Behavioral Intervention for Trauma in Schools (CBITS) Web site (cbitsprogram.org) is an online training and implementation support Web site for CBITS, a widely recognized (11), effective treatment for trauma-exposed children (12). The Web site was developed in response to high demand for CBITS training and requests for ongoing implementation support. The site comprises a “public” side that contains general information about CBITS, biographies of the intervention developers, and answers to frequently asked questions and a “private” side that provides registrants with a training course, implementation assistance resources, and a community forum.

To better understand use of the CBITS Web site, we used Google Analytics to examine Web site utilization among all registrants between July 1, 2011, and June 30, 2012 (N=1,406). Average site visit duration, or the average length of time a user spends on the Web site per visit, was used to evaluate Web site engagement. Page views, defined as the total number of pages a registrant viewed during all Web site visits, was the measure used to examine usage of the three sections of the site. Web site user characteristics, including demographic information, training background, clinical experience, purpose for visiting the site, and prior participation in CBITS training, were obtained from the Web site registration survey.

Using SPSS version 21.0, we described registrants’ characteristics and calculated overall Web site engagement and usage of Web site sections. We conducted bivariate analyses, using t tests and one-way analysis of variance (ANOVA), as appropriate, to examine the relationship between user characteristics, Web site engagement, and content usage. The RAND Corporation’s institutional review board (IRB) exempted the study from IRB review.

Results

Registrants were predominantly female (86%; N=1,186 of 1,374) and Caucasian (68%; N=959 of 1,406) and had a master’s degree (68%, N=939 of 1,376) and had not been trained in CBITS (79%; N=1,085 of 1,374). [Further information about user characteristics is available in an online data supplement to this report.] Registrants averaged 13:59 minutes per visit, with registrants not previously trained in CBITS spending significantly longer on the Web site each visit than previously trained registrants (15:11 versus 10:08 minutes, p<.001; Table 1). Registrants with less than four years of CBT experience had significantly longer visits than registrants with four to six years of CBT experience (16:07 versus 11:11 minutes, p<.01), as did registrants with four to seven years of clinical experience compared with those with eight or more years of experience (16:53 versus 11:05 minutes, p<.001). Registrants whose reason for visiting was to work toward becoming a CBITS clinician spent significantly more time per visit than registrants who did not visit the site for this reason (15:29 versus 12:38 minutes, p=.009). We found no other significant differences in visit duration when examining registrants’ purpose for visiting the Web site, including administrators who indicated when registering that they were visiting the site to better understand CBITS.

Table 1
CBITS Web site usage, by user characteristics, July 2011 through June 2012a

Registrants’ interaction with Web site content, as assessed by page views, was greatest for the training course (mean of 21.94 pages viewed), followed by the implementation assistance section (6.90) and the online community forum (.71). Registrants without prior training in CBITS viewed significantly more pages in the course section than those with prior training (24.26 versus 14.05 pages, p<.001), as did registrants with less than four years of CBT experience compared with registrants with four to six years of experience (27.28 versus 16.15 pages; p<.001). Clinicians with less than eight years of experience as a mental health clinician viewed significantly more pages compared with those with eight or more years of experience (16.30 versus 25.47 pages; p<.001). Individuals whose purpose for visiting was to become a CBITS clinician viewed significantly more pages in the course section than those who did not indicate this as a reason for visiting (26.79 versus 17.63 pages, p<.001). Registrants whose purpose for visiting was to understand CBITS as an administrator viewed significantly fewer pages in the course section than did those who did not indicate this as a reason for visiting (15.52 versus 23.36 pages, p=.006). In the implementation assistance and online community sections, those previously trained in CBITS viewed more pages in both the implementation assistance section (10.25 versus 6.16 pages; p<.001) and the online community section (1.44 versus .50 pages visited; p<.001).

Discussion

Evidence-based practices can “languish” 15–20 years before being routinely used (13), but Web sites may facilitate more rapid dissemination. Our examination may begin to inform how Web sites can be used to support training and implementation of evidence-based practices. We found high levels of Web site engagement overall, with the training section getting the most usage across all user types. The substantial use by untrained individuals was unsurprising, given that many clinicians enter the workforce untrained in effective treatments (2). With the need to increase the availability of training in effective interventions (14) and findings showing that distance learning is an effective and cost-efficient approach to enhancing clinical skills (7), initial and repeated exposure to course material through Web sites is one potentially scalable approach to addressing this implementation barrier.

Although not attracting as much traffic as the training course, the implementation support section had substantial usage. The greater use of this section, particularly among individuals previously trained in CBITS, suggests that once trained, clinicians may seek information designed to support implementation. The availability of implementation support materials is likely to result in more successful implementation because ongoing implementation support is essential for establishing evidence-based practices within a system (3).

There was relatively little use of the online community section, likely resulting from a lack of Web site prompts and new material, features that bring individuals back to Web sites on an ongoing basis (15). Research is needed to identify how online communities can enhance implementation and provide real-time feedback regarding implementation barriers or clinical situations as they arise.

Although little is known about how a nonclinical audience interacts with Web-based platforms primarily designed to train clinicians in evidence-based interventions, our findings reveal that administrators had high rates of engagement with the CBITS Web site, with visit duration times comparable with those of clinicians. However, compared with their clinical counterparts, administrators viewed fewer pages in the course section. Implementation is facilitated when administrators are committed to providing evidence-based practices and are knowledgeable about their use (5). These findings may help us understand how Web sites can be used to facilitate administrator support for the intervention. More broadly, research clarifying the needs of specific user groups is necessary to understand how Web-based platforms can more effectively support each user type.

Our study had several limitations. We had no information on whether Web site usage enhanced adoption, implementation, or sustainability of CBITS. Our study did not assess differences in the quality of online versus in-person CBITS training, nor differences in knowledge and skills gained by clinicians trained online versus in person. However, prior research would likely indicate that they are relatively equivalent (7,8,10). Finally, Web analytics data were gathered with a combination of hypertext transfer protocol access logs, sessions, persistent cookies, and Google Analytics user tracking tools. How information obtained through Google Analytics differs from self-reported usage is not known.

Conclusions

Our findings suggest that Web-based platforms have the potential to support the training and implementation of evidence-based practices for clinicians with varying levels of experience and familiarity with the intervention. These findings are particularly relevant at a time when there are greater incentives to disseminate evidence-based practices and a need to train and support a growing mental health workforce. In addition, Web-based platforms may be promising for supporting trauma-related interventions, for which training in or access to implementation assistance should be available rapidly after a traumatic event.

Acknowledgments

Support for this study was provided by grant SM061270 from the Substance Abuse and Mental Health Services Administration and by grant P30MH082760 from the National Institute of Mental Health.

Footnotes

Disclosures

The authors report no competing interests.

Contributor Information

Ms. Pamela Vona, School of Social Work, University of Southern California, Los Angeles.

Mr. Pete Wilmoth, RAND Corporation, Pittsburgh, Pennsylvania.

Dr. Lisa H. Jaycox, RAND Corporation, Arlington, Virginia.

Dr. Janey S. McMillen, 3C Institute for Social Development, Cary, North Carolina.

Dr. Sheryl H. Kataoka, Department of Psychiatry and Biobehavioral Sciences, University of California, Los Angeles.

Dr. Marleen Wong, School of Social Work, University of Southern California, Los Angeles.

Dr. Melissa E. DeRosier, 3C Institute for Social Development, Cary, North Carolina.

Dr. Audra K. Langley, Department of Psychiatry and Biobehavioral Sciences, University of California, Los Angeles.

Mr. Joshua Kaufman, Los Angeles Unified School District, Los Angeles.

Dr. Lingqi Tang, Department of Psychiatry and Biobehavioral Sciences, University of California, Los Angeles.

Dr. Bradley D. Stein, RAND Corporation, Pittsburgh, Pennsylvania. Department of Psychiatry, University of Pittsburgh.

References

1. Final Report. Rockville, Md: US Department of Health and Human Services, President’s New Freedom Commission on Mental Health; 2003. Achieving the Promise: Transforming Mental Health Care in America.
2. Horvitz-Lennon M, Donohue JM, Domino ME, et al. Improving quality and diffusing best practices: the case of schizophrenia. Health Affairs. 2009;28:701–712. [PMC free article] [PubMed]
3. Hoagwood K, Burns BJ, Kiser L, et al. Evidence-based practice in child and adolescent mental health services. Psychiatric Services. 2001;52:1179–1189. [PubMed]
4. Henggeler SW, Schoenwald SK, Liao JG, et al. Transporting efficacious treatments to field settings: the link between supervisory practices and therapist fidelity in MST programs. Journal of Clinical Child and Adolescent Psychology. 2002;31:155–167. [PubMed]
5. Corrigan PW, Steiner L, McCracken SG, et al. Strategies for disseminating evidence-based practices to staff who treat people with serious mental illness. Psychiatric Services. 2001;52:1598–1606. [PubMed]
6. Ruiz JG, Mintzer MJ, Leipzig RM. The impact of E-learning in medical education. Academic Medicine. 2006;81:207–212. [PubMed]
7. Weingardt KR, Cucciare MA, Bellotti C, et al. A randomized trial comparing two models of Web-based training in cognitive-behavioral therapy for substance abuse counselors. Journal of Substance Abuse Treatment. 2009;37:219–227. [PMC free article] [PubMed]
8. Dimeff LA, Koerner K, Woodcock EA, et al. Which training method works best? A randomized controlled trial comparing three methods of training clinicians in dialectical behavior therapy skills. Behaviour Research and Therapy. 2009;47:921–930. [PubMed]
9. National Crime Victims Research and Treatment Center. TF-CBT Web: First Year Report. Charleston: Medical University of South Carolina; 2007.
10. DeRosier ME, McMillen J, Davis NO, et al. Tools to support career advancement of diverse social, behavioral, and mental health researchers: comparison of in-person and online training delivery modes. Journal of Online Learning and Teaching. 2011;7
11. National Registry for Evidence-Based Programs and Practices. Rockville, Md: Substance Abuse and Mental Health Services Administration; 2013.
12. Stein BD, Jaycox LH, Kataoka SH, et al. A mental health intervention for schoolchildren exposed to violence: a randomized controlled trial. JAMA. 2003;290:603–611. [PubMed]
13. Balas EA, Boren SA. Managing clinical knowledge for health care improvement. In: Bemmel J, McCray A, editors. Yearbook of Medical Informatics. Stuttgart, Germany: Schattauer Verlagsgesellschaft mbH; 2000.
14. Pagoto SL, Spring B, Coups EJ, et al. Barriers and facilitators of evidence-based practice perceived by behavioral science health professionals. Journal of Clinical Psychology. 2007;63:695–705. [PubMed]
15. Dennen VP. From message posting to learning dialogues: factors affecting learner participation in asynchronous discussion. Distance Education. 2005;26:127–148.