Search tips
Search criteria 


Logo of nihpaAbout Author manuscriptsSubmit a manuscriptNIH Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
Am Psychol. Author manuscript; available in PMC Mar 13, 2014.
Published in final edited form as:
Published online Apr 30, 2012. doi:  10.1037/a0028180
PMCID: PMC3951837
Disruptive Innovations for Designing and Diffusing Evidence-based Interventions
Mary Jane Rotheram-Borus, Dallas Swendeman, and Bruce F. Chorpita
University of California, Los Angeles
Correspondence concerning this article should be addressed to: Mary Jane Rotheram-Borus, Department of Psychiatry, University of California, Los Angeles, CA 90025. Contact: cchpublications/at/
Mary Jane Rotheram-Borus, Department of Psychiatry, University of California, Los Angeles; Dallas Swendeman, Department of Psychiatry, University of California, Los Angeles; Bruce F. Chorpita, Department of Psychology, University of California, Los Angeles.
The numbers of evidence-based interventions (EBIs) have been growing exponentially, both therapeutic and prevention programs. Yet, EBIs have not been broadly adopted in the United States. In order for our EBI science to significantly reduce disease burden, we need to critically re-examine our scientific conventions and norms. Innovation may be spurred by re-examining the biomedical model for validating EBIs and the compartmentalization of EBIs as disease-specific, institutionally-based, counseling programs. The model of Disruptive Innovations suggests that we re-engineer EBIs based on their most robust features in order to reach more people in less time and at lower cost. Four new research agendas will be required to support disruptive innovations in EBI science: synthesize common elements across EBIs; experiment with new delivery formats (e.g., consumer controlled, self-directed, brief, paraprofessional, coaching, and technology and media strategies); adopt market strategies to promote and diffuse EBI science, knowledge, and products; and adopt continuous quality improvement as a research paradigm for systematically improving EBIs, based on ongoing data and feedback. EBI science can have more impact if it can better leverage what we know from existing EBIs in order to inspire, engage, inform, and support families and children to adopt and sustain healthy daily routines and lifestyles.
Keywords: Disruptive Innovations, evidence-based interventions, prevention science, mental health treatment
Psychologists have a broad array of effective tools that could potentially reduce the burden of both health and mental health problems. In the last 30 years, we have focused on demonstrating that evidence-based intervention (EBI) programs are efficacious. Our focus now needs to shift from solely demonstrating that EBIs can work to interjecting what works into user-friendly and scalable tools, products, and experiences. This article suggests ways to redeploy what we have learned from our EBI science in novel ways to extend our impact, create consumer demand, and permeate the daily lives of children, families, and communities.
Health rests on our daily behavioral routines (Weisner, 2002). Five habits lead to 70% of morbidity and mortality: how much and what we eat, exercise, smoking, and alcohol use (deVol & Bedrosian, 2007). These lifestyle behaviors are significantly and positively related to quality of health and mental health (Walsh, 2011). Delivering care for the chronic illnesses resulting from these habits accounts for more than 75% of medical care costs (Centers for Disease Control & Prevention [CDC], 2009). If we add sleeping, mating, drug use, and relationship habits, we account for another significant proportion of the burden of chronic and infectious diseases.
Concurrently, 27% of Americans experience mental health symptoms and disorders that significantly impact families’ daily routines, workplace productivity, and quality of life (Kessler & Wang, 2008). Mental health symptoms result in at least $500 billion annually in lost workplace productivity (Birnbaum, et al., 2010) and often lead to over-utilizing medical care (O’Donohue & Cucciare, 2005). Containment of America’s health care costs rests not only in health care reform, but also in the daily routines and lifestyles of American families.
Psychologists’ primary technology for influencing Americans’ routines are EBIs, both for prevention and treatment (Kazak, Hoagwood, Weisz et al., 2010). There are now hundreds of EBI programs that can reliably shape or reshape Americans’ habits (National Research Council & Institute of Medicine, 2009) validated by at least a dozen different registries or organizations (e.g., National Registry of Evidence-based Prevention Programs [NREPP] and the CDC Diffusion of Effective Behavioral Interventions [DEBI]). Similarly, psychotherapeutic EBIs achieve significant and sustained improvements for children, families, and adults (Weisz & Kazdin, 2010; Nathan & Gorman, 2007). However, neither preventive nor therapeutic EBIs have been fully scaled nationally (Glasgow & Sanchez, 2011).
Implementation data are not available for the diffusion of the compendiums of EBIs for prevention. Principal investigators or parties with vested interests have typically only tracked the diffusion of their own EBIs. Two EBI programs are perhaps the best examples of diffusing EBIs for prevention. The Nurse-Family Partnership (NFP) is a home visiting program during the first two years of life that has demonstrated significant and substantial improvements in maternal and child outcomes over 15 years in multiple randomized controlled trials (e.g., Olds, Sadler, & Kitzman, 2007). Improvements have been demonstrated in maternal reproductive health, criminal justice contact, and income, as well as children’s behavior problems, substance use, and criminal justice contact. Every day 22,000 low income mothers are reached by the NFP program. Yet, there are about 1.6 million low income women whose families would benefit from these services who do not receive them (Hill et al., 2009). Similarly, Gil Botvin’s Life Skills Training (LST), a school-based drug abuse prevention program for adolescents, has repeatedly demonstrated significant positive impacts on alcohol, tobacco, and drug use, as well as violence and delinquency (Botvin, Griffin, & Nichols, 2006). LST is adopted in 3,000 schools and serves approximately 1 million elementary and middle school children annually (“About the Botvin LifeSkills Training,” n.d.). However, there are over 90,000 public elementary and secondary schools in the U.S. needing drug abuse prevention (U.S. Department of Education, 2004-2005). These successfully diffused EBIs suggest a sizable gap exists between the potential and realized impact of preventive EBIs.
A similar gap exists for EBIs for treatment of mental health problems and disorders. Access to mental health care is highly limited; even though parity legislation exists, mental health and physical health disorders continue to be treated differentially (Wang, Lane, Olfson, Pincus, & Wells et al., 2005). Only 24% of children and 35%–45% of adults with mental health needs have received any mental health services (Ringel & Sturm, 2001; Leatherman & McCarthy, 2005; Goldstein, Olfson, Martens, & Wolk, 2006). When services are received, the service is likely not the most effective option (Zima et al., 2005; Weisz et al., 2006). The gap is even greater for African-Americans and Asian Americans, compared with white Americans (Harris et al., 2005). Under-utilization of effective prevention and therapeutic services contributes significantly to excess morbidity and mortality (Strong, Mathers, Leeder, & Beaglehole, 2005), as well as higher medical costs (DiMatteo, 2004).
Thus, EBI science could be more useful. We are not realizing its potential. To put our progress in perspective, mobile phones and social networking have revolutionized families’ lives globally in under five years. Facebook went from 0 to 500 million members in 6 years; adults are spending an average of 5.5 hours social networking daily ( Private enterprise saturates our daily lives with messages and cues that change our behaviors. Brand loyalty to McDonald’s, for example, is established by the age of nine (Schlosser, 2001). Our EBI science needs to shape families’ daily lives as much as McDonald’s or Facebook shapes Americans’ daily routines. Reconsidering how science can be broadly applied to servicing human behavior may help us achieve this goal.
Many EBIs are relatively similar in their structure, reflecting researchers’ adoption of a similar underlying paradigm for design, validation, and implementation of EBIs. First, scientists ensure validity of EBIs by evaluating impact. Our careful, linear design process is based on a biomedical product development model, widely used by clinical researchers for treatment innovations. Second, EBIs are typically designed and validated to address a disease-specific or situation-specific challenge. Third, most EBIs are individual or small group face-to-face counseling programs delivered in institutional settings. Although this paradigm has afforded our field many advantages and produced a compendium of sophisticated EBIs, its wholesale adoption has also had unintended consequences that present barriers to innovation.
The Biomedical Validation Model
Pharmaceutical companies have employed a four phase validation process to establish safety of any innovation, document benefits, test efficacy under optimal implementation conditions, and finally, test effectiveness under real world conditions, as the original innovation is replicated with fidelity (Flay et al., 2005). The biomedical model validates an innovation tested in a RCT as a discrete unit or product, a model that fits well for drugs, but not as well for complex behavioral interventions that are comprised of many components that contribute to their efficacy (Hawe, Shiell, & Riley, 2004). Both prevention and treatment researchers have adopted this biomedical model for EBIs, particularly the requirement that implementation and dissemination of EBIs are conducted with fidelity to the original, sequenced, scripts and activities as represented in the EBI manual (Flay et al., 2005).
Following this process, EBIs typically take about 20 years to become “certified” as effective and ready for diffusion: two years for intervention pilot testing; a five year efficacy trial; a second five year efficacy trial; a five year effectiveness trial; and then two years to update manuals and develop training programs for providers. Often this sequence does not proceed in a timely manner, as gaps in funding or a principal investigator’s life events intervene to derail or stop the process. In some cases, steps in this sequence are skipped in order to disseminate high priority interventions more quickly.
More problematic is that we do not have an efficient strategy for updating EBIs (cf. Chorpita, Daleiden, & Weisz, 2005a). Changes introduced into program curricula are considered violations of the principle of replication with fidelity. New RCT are then required to verify that each adapted EBI remains efficacious under new conditions or with a new population (Flay et al., 2005). The scientific community typically excludes untested adaptations of an EBI from “certified compendiums” and is not willing to confer the title of “EBI” on such adaptations.
Even more problematic is that EBIs are not typically replicated and disseminated with either fidelity or effectiveness. The programs are not sustained over time with fidelity to core elements (e.g., Collins, Phields, Duncan, & Science Application Team, 2007; Dworkin, Pinto, Hunter, Rapkin, & Remien, 2008), nor are the core elements consistently defined and operationalized (Rotheram-Borus, Swendeman, Flannery, et al., 2009). Direct service providers often adapt EBI or develop and implement their own intuitively-based interventions focused on providing information or creating intense feelings (Hallett, White, & Garnett, 2007). Concurrently, even when adapting this systematic approach, many effectiveness trials do not demonstrate the promise of the preliminary efficacy trial (e.g., Southam-Gerow et al., 2010; Weisz et al., 2009). Nationally, the federal grant review system has consistently tried to minimize risk of unnecessary expenditures of research funding; however, these are offset by the risk of developing something in the laboratory or a well-resourced efficacy trial, before knowing whether it is ultimately attractive or feasible to providers and consumers.
HIV prevention provides a good example of this challenge: some EBIs certified by the CDC DEBI review panel for HIV prevention have high uptake by community-based agencies while others sit on a shelf after validation or training. The length of time required for provider training, attractiveness of the delivery format, and ability to offer EBIs at consumer’s convenience, influence uptake in addition to the quality of the EBI. The careers and resources spent in the four phase validation of an EBI could potentially be reorganized to re-engineer EBI delivery to become even more cost-efficient interventions and to have greater impact, if the design and validation process were not so cumbersome (Wesiz et al., 2006).
Compartmentalized EBIs in Disease-Specific, Institutionally-Based, Counseling Programs
In the 1970s and 1980s, researchers focused on building cognitive, behavioral, and emotional skills (e.g., problem solving, self-control, assertiveness) that were intended to generalize across problem domains (e.g., Rotheram-Borus, Armstrong, & Booraem, 1982). Overall, the programs were too diffuse in foci to have measurable impacts in a single domain (Goldstein, Whitlock, & DePue, 2004). Over time, EBIs became increasingly targeted at narrower outcomes.
There were good reasons to compartmentalize. First, it is easier to demonstrate a significant behavior change within one specific domain and with a high risk population. Second, policymakers are more easily lobbied and legislation targeted to support specific negative outcomes. Third, review committees at the National Institutes of Health, in an effort to responsibly manage risk, often support incremental breakthroughs or small steps forward in science, rather than high risk, high gain programs.
EBIs are now typically designed for a specific behavior problem in specific settings. Experts exist for each disease outcome (diabetes, HIV, violence), each type of institutional setting (schools, criminal justice settings, foster care), and for each developmental transition (divorce, pregnancy) or situational challenge (disasters, wars, migration). The range of expert silos is potentially infinite: occasions, roles, cultural subgroups, and so forth. Each silo perceives or presents its science as unique. We, psychologists, have over-compartmentalized our expertise (Kazdin, 2008; Chorpita, Daleiden, & Weisz, 2005b).
There are significant and unintended costs of compartmentalization (Kodner & Spreeuwenberg, 2002). Our scientific priorities are directed by categorical funding lines, shaping how we frame, understand, and intervene. At the policy level, science is focused on the most politically salient health issues, often unrelated to disease burden. The most highly trained personnel are often siphoned into a single, vertically-integrated health issue. For example, HIV has dominated the global agenda for the last 15 years, at the cost of malaria prevention and treatment, a disease that has higher morbidity and mortality (Lorden, Tang, & Carmignami, 2011).
Often the designs of EBIs do not reflect the full complexity of real families’ and children’s lives. Individuals typically have more than one risky lifestyle behavior (Fine et al., 2004), which requires that each person receive more than one EBI. Physical and daily survival needs, social supports, and environmental and structural conditions operate to create a web of causation for multiple and co-occurring conditions (Krieger, 2003). By only designing for one risk, we fail to address the interrelated or holistic needs of most people. EBIs that address multiple outcomes may have greater impact on public health with reduced costs, compared to EBIs targeting one behavior or problem domain (Barlow, Allen, & Choate, 2004; Prochaska, Spring, & Nigg, 2008). Obesity, cardiovascular disease, cancers, and diabetes, the leading causes of morbidity and mortality in the U.S. (Fine et al., 2004), require intervention with multiple behaviors to remain disease free over time. For each of these conditions, diet and physical activity must be jointly managed (Prochaska et al., 2008), which also supports good mental health (Walsh, 2011). Yet, there are separate teams of experts for nutrition and exercise.
EBIs may lead to long-term, sustained impacts on multiple domains
In addition to NFP and LST, for example, the Social Development Program, an intervention delivered in urban elementary schools, has found long-term impacts well into adulthood, fifteen years after the intervention ended. By their mid-twenties youth from the intervention schools had significantly better educational achievement, employment, mental health, and sexual health compared to young people from the control schools (Hawkins, Kosterman, Catalano, Hill, & Abbott, 2008). Comprehensive interventions with mothers with AIDS and their adolescent children not only improved the parents’ functioning and quality of life, but their adolescent children report less substance use, delay the onset of sexual debut, have fewer babies and at a later age, and report fewer internalizing and externalizing mental health symptoms (Rotheram-Borus et al., 2006). Grandchildren, born years after the death of the grandmother with AIDS, have better home environments and tend to have better cognitive development compared with grand-babies in a control condition. Change processes applied to one behavior can be generalized to other behaviors and cascade over the life-course, typically by enhancing motivation, self-efficacy, and skills (Eccles & Wigfield, 2002). Yet, the long-term and generalized benefits of EBIs are not typically assessed or recognized, nor are the mechanisms influencing these benefits well understood. Research and evaluation are typically only funded for a few years of follow up at most. The biomedical validation model and compartmentalization of EBIs do not support evaluation of long-term impacts across a broad range of potential impacts.
Counseling within institutional settings is a narrow delivery platform
When creating EBIs, a single delivery modality is typically selected and one version of an EBI is designed and evaluated. While a range of potential delivery platforms exist, almost all EBI programs are interpersonal, either delivered in one-on-one counseling, family, or small groups (Kelly, 1999). The delivery format connects EBI-related activities to a “counseling” metaphor, which is often stigmatizing (Raghavan, Bright, & Shadoin, 2008) because many consumers do not want to admit needing help.
Each institution selected for EBI dissemination has its own delivery challenges. Healthcare institutions are over-burdened and face constantly increasing costs of care. The primary mission of educational institutions is academic performance, not prevention or treatment; prevention and treatment are “extras” and in times of economic recession, preventive programs are the first to go. In addition, teachers typically lack training in promoting social emotional health and in managing students’ behavior problems. Community based organizations (CBOs) are also dependent on unstable or unsustainable funding sources. CBOs typically pay staff even less than educational institutions and staffing is often a revolving door, except in the most organized and stable agencies. Public health institutions, also challenged to use public financing effectively, typically emphasize infectious diseases and environmental risks, rather than promotion of healthy daily routines. The sustainability of EBIs in these settings is uncertain. Although each of these delivery settings makes significant contributions to the current diffusion of EBIs, none is ideal for significantly expanding the diffusion of our EBI science. We need to re-examine our delivery vehicles for EBI science.
Bower and Christensen (1995) introduced the concept of disruptive innovations in Harvard Business Review 17 years ago. The model describes how changes to current ways of doing business by reframing our understanding of a problem’s causes and solutions can enable us to meet the essential needs of the majority of consumers in more efficient and accessible ways (Christensen, 1997; Christensen, Grossman, & Hwang, 2009). Disruptive innovations simplify existing services or products that typically “over-serve” the majority of consumers.
America has been particularly successful in designing high-end solutions that can meet the needs of the most demanding consumers. Certainly in health care, no country can offer the comprehensive and technologically advanced care that America can provide (Chernichovsky & Leibowitz, 2010). In fact, it is this characteristic that leads us to have costs that are double those of any other developed nation (Kimbuende, Ranji, Lundy, & Salganicoff, 2010).
Rather than focusing on products and services that can satisfy the full range of needs of the most demanding consumers, a disruptive innovation provides a simpler and less expensive alternative that meets most of the same needs for the majority of consumers. The new service is more accessible, scalable, replicable, and sustainable. For example, automated teller machines (ATM) have been a disruptive innovation in the banking industry that has revolutionized banking (Tomashoff, 2002). Similarly, paying for goods and services via mobile phone airtime credit transfers is a disruptive innovation expanding electronic money transfers to 3 billion persons globally who previously had little access to banks, ATMs, or credit cards (Standage, 2009). The mass distribution of generic $2 eyeglasses is another example. The $2 eyeglasses will not meet the needs of 20% of consumers who need specialized corrective lenses. However, 80% of those who need eye care are served well and inexpensively, including many persons who would otherwise have no access to an optometrist and custom lenses. Telemedicine and community colleges are other examples of disruptive innovations; the service provided may not be as intensive as the traditional option, but it is available to many more people to meet their most essential needs. Notably, in none of these examples did the disruptive innovation entirely replace the traditional specialty options, but in all cases, it dramatically increased the public impact of the underlying technology or service.
EBI science may be more broadly diffused more quickly, if we adopt and experiment with disruptive innovations. In order to generate disruptive innovations, we recommend that we identify the common and robust elements of EBIs; diversify our delivery models; emphasize consumer and provider preferences and capacities from inception of a potential EBI: and experiment with models of continuous quality improvement, rather than requiring replication with fidelity of EBI manuals.
Four new research agendas will be required to support disruptive innovations in EBI science: synthesizing common elements across EBIs; experimenting with new delivery formats; adopting market strategies to diffuse EBI science; and adopting continuous quality improvement as a research paradigm. In order to establish accountability and credibility, we have held EBIs to the criteria of demonstrating behavior change, typically as a packaged program. Because extensions and adaptations to new consumer contexts typically require adapting the packaging and going through the entire test sequence again, we lose immediate opportunities for improving public health. We are not advocating abandoning our current paradigm altogether, however, more active research and debate is needed on how to best design, mount, diffuse and evaluate EBIs over time.
Synthesize and Diffuse Common and Robust Elements of EBIs
In the pursuit of developing a specialized program of research, researchers are often incentivized to emphasize differences in EBI programs and/or their theoretical orientations. Careers are built on these differences. However, there is much more in common across theories and EBI practices than is typically acknowledged (e.g., Fishbein, Bandura, Triandis, et al., 1991). Identifying the common functions, processes, and elements across EBI programs may help us identify the robust theoretical and programmatic components that support program efficacy and cost-efficiency, and allows for a broader synthesis of knowledge that can guide policy and action.
Synthesis of our existing knowledge is as important as generating novel information
Risk and protective factors are shared across a broad range of disorders and symptoms (Atkins & Clancy, 2004; Prochaska et al., 2008). Similarly, common elements underlie EBIs within and across different types of health challenges (e.g., Noar, Chabot, & Zimmerman, 2008; Chorpita et al., 2005b; Grencavage & Norcoss, 1990; Karver, Handelsman, Fields, & Bickman, 2005). There are many strategies for identifying common and robust elements of EBIs. Meta-analyses, research syntheses, Delphi panels, and novel theories have been the traditional strategies for identifying robust EBI features. Analyzing and coding EBI manuals is another strategy. For example, content analysis and coding of adolescent HIV prevention EBIs from diverse developers demonstrates that EBIs share common factors (Rotheram-Borus, Swendeman, Flannery, et al., 2009), principles (Rotheram-Borus, Ingram, Swendeman, Flannery, 2009), and processes that structure small group delivery (Ingram, Flannery, Elkavich, & Rotheram-Borus, 2008). Meta-analyses have also identified robust principles and practice elements (e.g., Hall & Rossi, 2008; Albarracin, Gillette, Earl, et al., 2005). The immediate question arises: are similar components found in EBIs for smoking cessation, alcohol abuse, or obesity?
Similar commonalities have long been observed for psychotherapy and mental health treatment (Kazdin, 2005; Bickman, 2005). Perhaps the most recent programmatic analysis of common therapeutic practices is the work of Chorpita and Daleiden (2009). The practice elements embedded in evidence-based treatments tested in more than 500 RCT have been cataloged into an online searchable database (the PracticeWise Evidence Based Services [PWEBS] database [accessible at]). The PWEBS database systematically aggregates the scientific knowledge across more than 1,000 treatments for children that have been tested in RCTs (1,148 treatments have been coded as of October 2011). Clinicians can search to find what treatments tested in RCTs have in common, with respect to a specific target behavior or diagnosis for persons of a particular age, gender, ethnicity, and/or in a specific setting (Chorpita et al., 2011). Rather than merely getting a reference to a multi-session manual with sequenced activity scripts, clinicians also receive a list of practices summarizing the most common procedures among all relevant EBIs—sometimes dozens—for the same problem (Chorpita & Daleiden, 2009). These practices can then be systematically arranged, as the clinician custom designs and adapts an intervention plan in real time (e.g., Chorpita & Daleiden, 2010). Neither prevention nor clinical researchers have successfully met the definitional challenge of defining robust, common features of EBIs. We need to actively begin this research, if we want to broadly diffuse our EBI science and programs.
The proliferation of EBIs increases the importance of identifying the commonalities across EBIs, rather than the differences
For example, only seven practice elements (exposure, cognitive restructuring, psychoeducation, relaxation, modeling, parent psychoeducation, and self-monitoring) emerge from a list of over 55 as common among 102 EBI protocols for children with anxiety disorders (Chorpita et al., 2011). Likewise, 18 common elements (e.g., praise, time out) characterize 111 EBI protocols for oppositional behavior. Resources that synthesize knowledge are increasingly as important as generating new knowledge: PWEBS is one potential strategy to create a toolkit-based approach to designing treatments in the field in real time (Chorpita & Daleiden, 2010). That is, training community providers in these 15 to 20 common practice elements and how to arrange them for different presentations could be significantly less complex and less costly than training multiple different and independent EBIs (cf. Weisz, Jensen-Doss, & Hawley, 2006).
The example models for synthesizing common and robust elements from prevention and treatment EBIs address only a few types of elements embedded within EBIs, which are not sufficient to create full programs. There are many other dimensions that go into EBIs, such as their messaging, their structure, their supervision model, their decision logic, their underlying theory, their format and settings, etc. Fortunately, these dimensions can also often be abstracted and aggregated from the collective literature, yielding profiles of “common structures” or “common logics.” Of considerable importance among the common dimensions of EBIs is the manner in which the procedures are chosen, sequenced, repeated, or selectively applied—logic that Chorpita and Daleiden (2010) refer to as “coordination.”
There is emerging evidence of the benefits of this approach. Chorpita and Weisz (2009) recently designed MATCH (Modular Approach to Therapy for Children), a multi-disorder intervention system that incorporates treatment procedures (elements) and treatment logic (coordination) corresponding to three successful EBIs for childhood anxiety, depression, and conduct problems, with modifications to allow the system to operate as a single protocol. Tested in a recent multi-site RCT, MATCH appears to have larger benefits than any of the specific EBIs on which it was based (Weisz et al, in press). Similarly, providers in the Hawaii state mental health system were trained primarily in element-based treatments to complement a small number of manualized interventions, with notable improvements in outcomes over time (Daleiden et al., 2006). Parallel initiatives are occurring within medical care (Aro, Smith, & Dekker, 2008; Hawe, Shiell, & Riley, 2004). Yet, there is not a recognized research agenda or diverse strategies being evaluated to identify and leverage robust EBI features to enhance effective diffusion.
We need to test the limits of existing EBIs by adapting their delivery formats, dose, organization, or coordination while also continuously examining outcomes. Currently, there is limited evidence on how design or delivery format shifts that adapt the structure, roles, dose, rules, responsibility and intensity of an EBI change its impact or effectiveness. By testing the limits of EBIs, we will create a science of design and delivery formats.
Experiment with Novel Delivery Formats
Many delivery formats are potential disruptive innovations for diffusing EBIs for health behaviors and mental health: consumer controlled diagnostics and screening; self-directed or self-management interventions; brief interventions; paraprofessional delivery; coaching for common low- and mid-level problems; and technology and multi-media delivery. Each has nascent examples in the field but none has reached its potential (Kazdin & Blasé, 2011).
Consumer-controlled diagnostics and screening
Shifting from physician or laboratory controlled diagnostics to consumer-controlled diagnostics is a strategy to create disruptive innovations. Pregnancy tests were only available at doctors’ offices as recently as the 1970’s (Leavitt, 2006); the information was perceived as too important to be discovered by oneself. Now, pregnancy tests are broadly available in local grocery stores or pharmacies, enabling pregnant women to stop smoking and drinking alcohol very early in their pregnancies, much earlier than was previously possible. Pregnancy counseling is not immediately available but more people can access the essential diagnostic service, in less time, and with decreasing costs over time. Furthermore, broad accessibility and marketing of diagnostic tests almost immediately reduce stigma, evidenced by rapid uptake of home testing products.
HIV testing, for example, is poised to allow a similar disruptive innovation in screening tests that occurred with pregnancy tests 40 years ago (Greenwald, Burstein, Pincus, & Branson, 2006). Currently, HIV-tests in the United States are only available with third-party notification of test results and are not broadly available in the marketplace. Providing inexpensive and broadly available consumer-controlled HIV tests and other diagnostic screening tools are examples of disruptive innovations. Both China and Africa have rapid HIV tests available for about $0.80, but Americans spend $40 for the test itself and then pay an additional physician, counselor, or technician fee to conduct the test and provide the result and counseling. High stigma has been associated with HIV testing; consumer controlled testing at a low price would allow repeat HIV testing by persons at high risk, without stigma (Rotheram-Borus & Etzel,2003). The opportunities for patient self-screening are broad, encompassing health status indicators (e.g., pregnancy, HIV, HbA1c, high blood pressure, Body Mass Index, depression, anxiety) and lifestyle risk and protective factors (e.g., physical activity, caloric intake, sleep, substance abuse). Consumer controlled self-screening tools begin to shift responsibility for managing health to be shared by consumers and providers. Broad utilization of these monitoring tools by most Americans would facilitate early intervention and be a disruptive innovation.
Self-management and self-help
Self-directed interventions have some of the same benefits of consumer-controlled diagnostics (Harwood & L’Abate, 2010). Last year, 13.5 million Americans bought self-help books (693 million books at a price of $9.1 billion), significantly more than enrolled in therapy ( Some self-help books are grounded in years of evidence-based research and practice: for example, martial therapy has yielded popular self-help books by John Gottman and colleagues (1999). The Writing Cure (Lepore & Smyth, 2002) is based on the many years of research on journaling by Pennebaker and colleagues (1997). Yet, there is substantial variability in the quality of the self-help genre, much of it not based on evidence. Empowering consumers with brief self-management interventions that can serve to motivate, orient, and outline a step-by-step change and maintenance process is likely to be useful to a significant subset of the population, perhaps as high as 15% (Harwood & L’Abate, 2010).
Brief interventions
Brief interventions consisting of one or two sessions are also potential disruptive innovations, in that they provide a lower cost and lower intensity option than typical EBIs. A promising innovation is the Family Check-up (Dishion et al., 2008), a low-cost alternative to a psychiatric referral for child behavior problems that focuses on wellness and strengths rather than targeting the problems specifically. Family Check-ups, which have achieved a 24% decrease in behavior problems, have only been offered in schools serving low income children (Dishion, Nelson, & Kavanagh, 2003). Brief, single session EBI programs have resulted in significant behavior changes in smoking (Prochaska et al., 2008), alcohol use during pregnancy (O’Connor & Whaley, 2006), drug abuse (Longshore & Grills, 2000), and HIV risk acts over the short term (Patterson, Shaw, & Semple, 2003). Yet, the effects are typically not sustained over long periods without additional exposures to intervention. For some consumers, a brief EBI is sufficient to meet their needs, whereas for others it may function as a tool to screen and link to more intensive EBIs.
Paraprofessional delivery of EBIs
The high costs of delivering mental health care (e.g., $275/hour in West Los Angeles, CA) and health care ($185 per office visits for Blue Shield) suggest the need for providers with less training to assume key roles in delivering care. Historically, we have recognized this opportunity for addressing less severe disorders or problems at lower cost. The licensing of Marriage and Family Therapists is one example. However, America has no workforce of community health workers (CHW) that parallels the 40 million CHW globally (Lewin et al., 2010). Promotoras, who are available in about 200 counties in the US are the closest we have to CHW nationally (Lujan, 2009). Similar to calls to “task shift” responsibilities from highly trained medical providers to CHWs (World Health Organization, 2008), we propose that “task sharing” from mental health professionals to CHW (Patel, 2009) is one possible disruptive innovation for current delivery models. Most CHWs and paraprofessionals are trained for a specific task (e.g., to assist with breast feeding problems, to encourage adherence to diabetes medications, etc.), but what remains to be demonstrated is whether generalist CHWs can effectively deliver EBIs. A primary challenge is identifying, selecting, and training paraprofessionals. Selecting CHWs based on their being positive role models in their local community is one successful selection strategy that has been used for the last 30 years globally (Rotheram-Borus et al., 2011), but not in the US (e.g., Schoenwald et al., 2010). A paraprofessional workforce could also broadly implement low cost behavioral interventions (Bennett-Levy, Richards, Farrand, Christensen, Griffiths, Kavanaugh, et al., 2010). In particular, CHW could saturate local communities (e.g., with home visits) and healthcare settings with brief EBIs and screening assessments. This workforce could support consumers to implement self-management strategies for a broad range of chronic diseases including low- and mid-level mental health problems.
Coaching programs for common low- and mid-level problems
“Coaching” is one format or reframe on a counseling model that may be a more acceptable label for delivery of EBIs by CHW. Coaching, in particular, permeates families’ lives, including men, and embeds the intervention in a sports or professional development metaphor, rather than stigmatized counseling or therapy (Green, Oades, & Grant, 2006). Extending this potential opportunity, sports programs may be a way to engage men who would otherwise not be reachable for intervention (Rotheram-Borus, Swendeman, & Chovnick, 2009). For example, an organized soccer and job training program that integrates EBI practices and principles for HIV prevention and pro-social masculine identity among young men in South African townships, delivered by the soccer team coaches. We are examining male centered consumer engagement in EBI science.
Technology and media driven delivery platforms
Telemedicine, telehealth, and computerized interventions are examples of disruptive innovations in healthcare dating back decades, including for mental health diagnoses and treatments (DelliFraine & Dansky, 2008) and health behavior change (Krebs, Prochaska & Rossi, 2010). The potential range of delivery formats is currently exploding. Internet and mobile phone mediated intervention programs are rapidly expanding the reach of EBI science (Swendeman & Rotheram-Borus, 2010).
Media, whether through mobile phone, web, television, video consoles, or social networking applications, penetrate into our daily lives. The web has already become an important platform for health education with high variability in the scientific credibility of this work. There are more than 4 billion mobile phone users globally (Central Intelligence Agency, 2007) and the number is anticipated to reach six billion by 2013 (Standage, 2009). Mobile phone teledensity (i.e., number of phones per person) is over 100% in many developed and low income countries.
Technology platforms can enable diffusion of EBI science at low cost, in a highly engaging manner at an unprecedented scale. The information automatically recorded as consumers move through their daily lives is of much higher quality, validity and reliability than any information previously available--and will include more diverse, specific, sensitive, and scalable data types. Real-time data collection that is geographically and temporally stamped provides novel assessment and intervention opportunities. Investing in the software and methodological infrastructure to support a broad range of preventive innovations on this platform may improve and diffuse EBI science through a broader range of applications (Estrin & Sim, 2010).
Adopting Market Strategies to Promote EBI Science
Private entrepreneurs know how to create products and services that consumers want (Duan & Rotheram-Borus, 1999), often informed by researchers and psychologists. For example, inspiration is not a component of social cognitive theories or EBIs, yet, inspiration is basic to marketing, and many EBIs are embedded with inspirational strategies and experiences. Creativity, attractiveness, accessibility, demand, and utilization are basic prerequisites for products that create sustained habits and loyalty over time in the private enterprise world (Curtis et al., 2007). Our EBIs must share these attributes.
Private entrepreneurs also know how to implement a consistent quality product in millions of sites concurrently. Prevention and clinical researchers typically do not. The experts knowledgeable on these areas are product distribution specialists and business planners. Rarely do research teams include these specialists, to our detriment. For researchers to scale-up EBIs broadly or globally we also need reliable and consistent product delivery across thousands or millions of providers or households.
Research teams need to diversify expertise
There is at least one famous example of successful scientific-business-artist collaboration: the Children’s Television Workshop (Mielke, 1990). More than 95% of American preschoolers surveyed in the mid-1990s watched the Sesame Street series by the age of three years; 77 million American adults watched the program series as children; and 120 million people watch Sesame Street daily (Truglio & Fisch, 2001). Researchers and television experts joined together to create one of the most significant vehicles to influence children’s development in the last 40 years (e.g., Mielke, 1990; Truglio & Fisch, 2001). Government and private foundation resources initially funded the project aimed at supporting the development of disadvantaged preschoolers. The Children’s Television Workshop has generated more developmental research than any other initiative (Murphy, 2009) and has been self-sustaining for more than 30 years. The process of how to transform educational curricula into high impact media that diversified from television to a broader range of media (books, CDs, games, etc.) was itself studied and diffused (Mielke, 1990).
In contrast to most efforts to take science to the marketplace, The Children’s Television Workshop was led by the creative team. It was easier to train comic writers to know science than to train educators to know comedy. Similar to the investments made in the Children’s Television Workshop more than 40 years ago, EBIs for prevention and treatment now need investments that foster novel partnerships: scientists, artists, and entrepreneurs who are willing to learn from each other. The cultural norms, values, perspectives, and operating styles will clash. Not all scientists, artists, or entrepreneurs will be able to tolerate the ambiguity in roles, processes, and outcomes that such innovation will require. Yet, if we fail to invest in novel strategies and approaches, we will surely miss critical opportunities to increase the public impact of EBI science.
Pursue existing markets for delivery of EBI science
If we were to innovate in how we design EBIs, a market appears to exist, especially among families. Families spend billions of dollars each year on edutainment (delivering education via media, technology, toys, etc.; Singhal, Cody, Rogers, & Santelli, 2004) that parents perceive as an effective strategy for increasing their children’s cognitive, social, and emotional development. Marketers can guide family spending priorities by encouraging children to demonstrate “pester power” (Sutherland & Thompson, 2001). There is a potential market for our EBIs, yet we lack much of the expertise to tap this market. We are armed with science, but do not know what, when, and how to deliver the science broadly at low cost and with high impact. Fundamental innovation is needed with regard to who is involved with prevention and clinical science design and implementation.
Private entrepreneurs understand better than scientists how to engage families over time and to influence their preferences and habits to establish brand loyalty (Curtis, Garbrah-Aidoo, Scott, 2007). If we could socialize children to healthy daily routines for eating, exercising, and goal setting early, dramatic and sustained drops in our health care expenditures and improved length and quality of life are likely. Our daily lives are permeated with cues, triggers, messages, and experiences encouraging unhealthy habits. These experiences in turn influence habits; private enterprise has been utilizing this principle to increase profits (Pine & Gilmore, 1999). For example, children view an average of 40,000 commercials annually to buy a variety of typically unhealthy products (Schor, 2004), which has changed families daily routines within a generation (Schlosser, 2001). We psychologists need to saturate daily life with as many health promoting cues and experiences as do private entrepreneurs and marketers.
Create a brand for EBI knowledge and tools
Consistent product use is often sustained by brands (Moore & Rogers, 2002). All well-known entities have brands: World Health Organization, Habitat for Humanity, Gates, Coca-Cola (Cone, Feldman, & DaSilva, 2003). These brands are valuable; the Habitat for Humanity brand was worth $3.1 billion in 2006 ( This familiarity with a trusted and familiar brand can elicit consumers’ loyalty and utilization. Brands make the introduction of new products easier and more potent. Today’s EBI programs are mainly branded on the reputations of principal investigators, and such brands typically resonate only with academic and professional audiences. Psychological interventions need a brand (or brands) as broadly diffused and as accepted as the Good Housekeeping Seal of Approval once was.
Diffuse robust elements of EBIs, in addition to programs
Currently, we certify or brand programs as evidence-based, usually as an entire package of sequenced activities and scripts. We propose that scientific principles can be applied across a broad range of settings and contexts. The simple tools that are broadly used in EBI programs have been under-diffused to consumers, despite evidence of repeated validation over 40 years. For example, star charts have been demonstrated to improve children’s behaviors, if used consistently. Yet, there is only limited direct to consumer marketing of star charts. There is a large untapped consumer market for the specific tools that are embedded within many existing EBI programs. If we diffused robust elements of EBI science, in addition to EBI programs, each element may not only result in producing a sustainable behavior change but also in permeating the environment with robust science-based experiences can help continue to create health promoting contexts and environments.
It will also be necessary to think of building distribution channels not simply around EBI products, but around EBI knowledge (i.e., what has been learned from the development of those products). For example, Chorpita, Bernstein, and Daleiden (2011) recently combined client data from a statewide public mental health system with study data from the PWEBS database and demonstrated that even if all EBI protocols (i.e., products) from 437 randomized trials were available in that state system, over 30% of youth would still not be served with any EBI. That is, for 30% of youth there was no EBI in the literature (let alone within that service system) that matched the child on all of only three dimensions: primary problem, age, and gender. Nevertheless, those 437 studies likely provide useful information that could potentially be adapted or extended to serve that 30% of youth for whom there is no specific EBI product. Thus, we need to think broadly about not only product or program distribution, but about knowledge distribution more generally (Graham et al., 2006). We can do far better at using what we know to inform what we do (Chorpita, Rotheram-Borus, et al, 2011).
Create new marketplace delivery settings for EBI knowledge
Disruptive innovations can help move healthcare services outside of clinical settings. For example, “Minute Clinics” in retail pharmacies that provide treatment for the ten most common health problems by nurse practitioners is a disruptive innovation in healthcare (CA Healthcare Foundation, 2006). There are currently up to 1,000 minute clinics in the US. In three million minute clinic visits as of 2008, consumers indicated a 90% satisfaction rating in terms of quality of care, convenience, and cost (Laws & Scott, 2008). In particular, minute clinics provide low cost and easily accessible alternatives to emergency rooms, especially for the uninsured. Services are accessible, mainstreamed into everyday commercial enterprises, and, hopefully, destigmatized. Minute clinics could be developed to provide mental health care at lower cost for the most common, low-level mental health problems: attention deficit disorders, depressive symptoms or disorders, anxiety symptoms, and conduct problems. Linking EBI science to a broad range of experiences (e.g., sports, cultural activities, science explorations, date nights, theater performances, service tourism) are strategies for overcoming this stigma, embedding prevention in daily life and engaging experiences.
Develop Family Wellness Centers for delivery of EBI science
Experimenting with this approach, our group has been promoting the creation of a platform of Family Wellness Centers staffed by the community members whose families are thriving, based on a theory of positive peer deviants (Rotheram-Borus, Flannery, Rice, & Lester, 2005). Rather than focus on the standard counseling delivery format, we are experimenting with experiential approaches to deliver EBI science: martial arts, mindfulness, yoga, dance, crafts, and music, summer camps, expert series, and coaching. Experiential public health information, prevention and treatment programs, and recreational and vocational activities that engage the community are promoted in order to create a “health promoting community” (Campbell, Nair, & Maimane, 2007). Capitalizing on local entrepreneurial promotion of relaxation strategies (such as yoga, massage), the power of the “wellness” framework may be harnessed to promote healthy daily routines. We currently have three different contexts to experiment with wellness framed behavioral health services in alternative delivery settings: (1) a retail setting in a middle class shopping center that integrates behavioral health (e.g., coaching) with child and family focused wellness services (yoga, mindfulness) and extra-curricular activities (martial arts, music & dance, summer camps;; (2) a similar retail setting focused on behavioral health but integrated with a mental health and physical health care clinic in an upscale shopping center; and (3) behavioral health services delivered in a set of six K-12 schools co-located at one site and in collaboration with teachers, nurses, and local community-based agencies. Rather than deliver manualized EBI programs in these settings, we are instead experimenting with disseminating products and services embedded with EBI knowledge aggregated from across the evidence-base (i.e., through common elements and science syntheses).
Let market research guide EBI design as well
Consumers’ and providers’ needs, wants, preferences, and feelings must be equal to theoretical considerations in the design process to increase EBI science utilization. The private sector can teach us many lessons about how EBI science can evolve to create programs, structures, and experiences that inspire, engage, and retain consumers’ healthy daily routines. However, scientists are typically unilaterally in charge of any design of EBIs, and often lack the market research on families and consumers or on service organizations (e.g., school systems, public clinics) needed to inform those designs. Scientists often also lack sustainable distribution platforms (Maibach, 2006) and have never been accountable to create demand for their interventions, as would be the case with private enterprise.
Market data is not typically accessible to researchers (e.g., the Harte-Hanks Market Intelligence surveys). To create a new toy or to sell rice, entrepreneurs are provided with information about the characteristics of the target market. Demographics do not typically define the target market, psychogenic profiles define markets. As scientists, we need to be as well-equipped as businesses to serve the needs of our customers (providers or consumers). Every small research team reinvents the information for their own specific EBI with a small-scale formative research phase, rather than accessing the deep knowledge held by private enterprise in a systematic way.
Continuous Quality Improvement to Guide EBI Implementation and Innovation
A continuous quality improvement model (Daniels, Sandler & Wolchik, 2008; Duan & Rotheram-Borus, 1999) will allow us to innovate within EBI programs and shorten timelines from design to broad diffusion, particularly when EBI programs are designed as “continually updatable” from the outset (Chorpita et al., 2005a).
In contrast to the norm of replication with fidelity, medical settings and private enterprise have consistently adopted an iterative quality improvement paradigm to improve the efficiency and impact of practices, experiences, and programs over time (Deming, 1981). Replication with fidelity requires that we know what aspects of an EBI are robust and cannot be abandoned and which aspects can be tailored to local contexts. Thus, changing a paradigm from replication with fidelity to iterative quality improvement involve—at the very least—informed guesses about how EBI programs “work”, i.e. what are the key ingredients. It is typically not clear what components make EBI programs efficacious because: (1) we have not consistently identified or experimented with the components of efficacious EBI programs; and (2) we do not monitor outcomes or delivery processes over time under a range of field conditions. Both are necessary to identify and deploy “robust” features of an EBI, yet, these strategies are rarely implemented.
We would be well served to think of the scale-up of EBI programs as challenges in distribution systems, similar to distributing products and services at Wal-Mart. Private enterprise has routinely mined sales data (Duan & Rotheram-Borus, 1999) to understand how to increase attractiveness, acceptability, and demand for their products and programs, while simultaneously decreasing the costs of production. We need to be able to deliver consistent EBI products and services, but can only do this when informed by data. A paradigm shift is needed to support integration of monitoring and evaluation into the design and dissemination of EBI programs, as the programs are broadly diffused (Bickman, 2008; Kluger & Denisi, 1996; Lambert, 2005).
Equally important are continuous feedback loops that link patient outcome data to provider practice data that can shape and inform iterations of EBIs on an ongoing basis. Distribution networks of our largest corporations (Northrop-Grumman, Walmart) have at their core this opportunity to constantly improve their products and service. Prevention and clinical science need these data and feedback loops (Bickman, 2008; Chorpita, Bernstein, & Daleiden, 2008; Kluger & Denisi, 1996; Lambert, 2005). Evaluation and monitoring must be unobtrusively and automatically integrated into our ongoing EBI programs and science in order to realize this vision cost-effectively. Mobile and web-based technologies, for example, offer us this opportunity.
Our existing arsenal of EBIs is likely to improve public health if those EBIs could be broadly diffused, embedded in families’ everyday life, utilized and sustained over time. For this to occur, we are challenged to address a research agenda that requires us to substantially innovate how we conduct our research and practice. As noted in the September 2011 APA Monitor on Psychology, psychotherapy is decreasing in popularity and utilization (Clay, 2011). Concurrently, the economic recession has reduced pay lines at the National Institutes of Health, cut state and local budgets substantially, and decreased the likelihood that prevention services are offered or reimbursed. Although health care reform has some funding lines for preventive interventions, the current global economic setting motivates us to identify how to intervene successfully with more people at less cost more efficiently. Psychology is well positioned to initiate disruptive innovations.
If psychologists are able to diversify our delivery formats for each EBI in a manner that is highly attractive to consumers, we will begin to permeate our culture with tools, products, and experiences. Rather than working solely to establish dissimilar and specialized technologies among a handful of scientists, we should be able to collaborate (hopefully, under a few major brands) on platforms that allow data from scientists, consumers, and providers to create divers and contextually individualized health applications of psychological knowledge.
Overall, we need to be more ambitious in setting goals for domestic and global diffusion, adoption, and routine implementation of EBI science in families’ daily lives. We, psychologists, need new research paradigms, expanded training models, and broader expertise. Scaling up will require a focus on both simplification and local customization to develop more robust methods that are highly attractive to consumers and embedded in everyday life. Overall, we need disruptive innovations in how, what, to whom, by whom, and with what the brands we use to diffuse what we have learned so far from EBIs.
  • About the Botvin LifeSkills Training (LST) Program. Botvin LifeSkills Training: Frequently Asked Questions. n.d. Retrieved from
  • Albarracin D, Gillette JC, Earl AN, Glasman LR, Durantini MR, et al. A test of major assumptions about behavior change: a comprehensive look at the effects of passive and active HIV-prevention interventions since the beginning of the epidemic. Psychol Bull. 2005;131(6):856–97. [PMC free article] [PubMed]
  • Aro AR, Smith J, Dekker J. Contextual evidence in clinical medicine and health promotion. European Journal of Public Health. 2008;18(6):548–549. [PubMed]
  • Atkins D, Clancy C. Multiple risk factors interventions-are we up to the challenge? American Journal of Preventive Medicine. 2004;27:102–103. [PubMed]
  • Barlow DH, Allen LB, Choate ML. Toward a unified treatment for emotional disorders. Behavior Therapy. 2004;35:205–230.
  • Bennett-Levy J, Richards D, Farrand P, Christensen H, Griffiths K, Kavanagh DJ, Klein B, Lau M, Proudfoot J, Ritterband L, White J, Williams C, editors. The Oxford Guide to low intensity CBT interventions. Oxford: Oxford University Press; 2010.
  • Bickman L. A Common Factors Approach to Improving Mental Health Services. Mental Health Services Research. 2005;7(1):1–4. [PubMed]
  • Bickman L. A measurement feedback system (MFS) is necessary to improve mental health outcomes. Journal of American Academy of Child and Adolescent Psychiatry. 2008;47(10):1114–1119. [PMC free article] [PubMed]
  • Birnbaum HG, Kessler RC, Kelley D, Ben-Hamadi R, Joish VN, Greenberg PE. Employer burden of mild, moderate, and severe major depressive disorder: Mental health services utilization and costs, and work performance. Depression and Anxiety. 2010;27(1):78–89. [PubMed]
  • Botvin GJ, Griffin KW, Nichols TD. Preventing youth violence and delinquency through a universal school-based prevention approach. Prevention Science. 2006;7(4):403–408. [PubMed]
  • Bower JL, Christensen CM. Disruptive technologies: Catching the wave. Harvard Business Review. 1995 Jan-Feb;:43–53.
  • California Healthcare Foundation. The minute clinic movement: Model for the future or 60 seconds of fame? 2006 Jul 18;
  • Campbell C, Nair Y, Maimane S. Building contexts that support effective community responses to HIV/AIDS: A South African case study. American Journal of Community Psychology. 2007;39:347–363. [PubMed]
  • Centers for Disease Control and Prevention. Behavioral Risk Factor Surveillance System: 2008 Summary Data Quality Report. 2009 Retrieved from
  • Central Intelligence Agency. World Fact Book. Washington, D.C: Potomac Books, Inc; 2007.
  • Chernichovsky D, Leibowitz AA. Integrating public health and personal care in a reformed US health care system. American Journal of Public Health. 2010;100(2):205–211. [PubMed]
  • Chorpita BF, Bernstein AD, Daleiden EL. Network on Youth Mental Health Research. Driving with roadmaps and dashboards: Using information resources to structure the decision models in service organizations. Administration and Policy in Mental Health and Mental Health Services Research. 2008;35:114–123. [PubMed]
  • Chorpita BF, Bernstein AD, Daleiden EL. Empirically guided coordination of multiple evidence-based treatments: An illustration of relevance mapping in children’s mental health services. Journal of Consulting and Clinical Psychology. 2011;79:470–480. [PubMed]
  • Chorpita BF, Daleiden EL. Mapping of evidence-based treatment for children and adolescents: Application of the distillation and matching model to 615 treatments from 22 randomized trials. Journal of Consulting and Clinical Psychology. 2009;77(3):566–579. [PubMed]
  • Chorpita BF, Daleiden EL. Building evidence-based systems in children’s mental health. In: Kazdin AE, Weisz JR, editors. Evidence-Based Psychotherapies for Children and Adolescents. New York: Oxford; 2010. pp. 482–499.
  • Chorpita BF, Daleiden E, Weisz JR. Modularity in the design and application of therapeutic interventions. Applied and Preventive Psychology. 2005a;11:141–156.
  • Chorpita BF, Daleiden EL, Weisz JR. Identifying and selecting the common elements of evidence based interventions: A distillation and matching model. Mental Health Services Research. 2005b;7(1):5–20. [PubMed]
  • Chorpita BF, Rotheram-Borus MJ, Daleiden EL, Bernsein A, Cromley T, Swendeman D, Regan J. The old solutions are the new problem: How do we better use what we already know about reducing the burden of mental illness? Perspectives on Psychological Science. 2011;6:493–497.
  • Chorpita BF, Weisz JR. Modular approach to therapy for children with anxiety, depression, trauma, or conduct problems (MATCH-ADTC) Satellite Beach, FL: PracticeWise; 2009.
  • Christensen C. The innovator’s dilemma: When new technologies cause great firms to fail. Boston, MA: Harvard Business Press; 1997.
  • Christensen CM, Grossman J, Hwang J. The innovator’s prescription: A disruptive solution for health care. New York, NY: McGrawHill; 2009.
  • Clay RA. Advocating for psychotherapy: An APA resolution, treatment guidelines and other efforts seek to reverse the decline in psychotherapy’s use. APA Monitor. 2011;42(8):48–51.
  • Collins C, Phields ME, Duncan T. Science Application Team. An agency capacity model to facilitate implementation of evidence-based behavioral interventions by community-based organizations. Journal of Public Health Management and Practice, Supplement 1. 2007;(13):S16–S23. [PubMed]
  • Cone CL, Feldman MA, DaSilva AT. Causes and effects. Harvard Business Review. 2003;81(7):95–101. [PubMed]
  • Curtis VA, Garbrah-Aidoo N, Scott B. Ethics in public health research: masters of marketing. Bringing private sector skills to public health partnerships. Am J Public Health. 2007;97(4):634–41. [PubMed]
  • Daleiden EL, Chorpita BF, Donkervoet CM, Arensdorf AA, Brogan M. Getting better at getting them better: Health outcomes and evidence based practice within a system of care. Journal of the American Academy of Child and Adolescent Psychiatry. 2006;45:749–756. [PubMed]
  • Daniels V, Sandler I, Wolchik S. Use of Quality Management Methods in the Transition from Efficacious Prevention Programs to Effective Prevention Services. American Journal of Community Psychology. 2008;42(3–4):369. [PubMed]
  • Dellifraine JL, Dansky KH. Home-based telehealth: A review and meta-analysis. Journal of Telemedicine and Telecare. 2008;14(2):62–66. [PubMed]
  • Deming WE. Improvement of quality and productivity through action by management. National Productivity Review. 1981 Winter;:12–22.
  • DiMatteo MR. Variations in Patients’ Adherence to Medical Recommendations: A Quantitative Review of 50 Years of Research. Medical Care. 2004;42(3):200–209. [PubMed]
  • Dishion TJ, Nelson SE, Kavanagh K. The family check-up with high-risk young adolescents: Preventing early-onset substance use by parent monitoring. Behavior Therapy. 2003;34(4):553–571.
  • Dishion TJ, Shaw D, Connell A, Gardner F, Weaver C, Wilson M. The family check-up with high-risk indigent families: Preventing problem behavior by increasing parents’ positive behavior support in early childhood. Child Development. 2008;79(5):1395–1414. [PMC free article] [PubMed]
  • Duan N, Rotheram-Borus MJ. Translating prevention research into social work practice (monograph) Seattle: University of Washington, School of Social Work, Prevention Research Center; 1999. Development and dissemination of successful behavioral prevention interventions: Safety, innovation, essential ingredients, robustness, and marketability.
  • Dworkin SL, Pinto RM, Hunter J, Rapkin B, Remien RH. Keeping the Spirit of Community Partnerships Alive in the Scale Up of HIV/AIDS Prevention: Critical Reflections on the Roll Out of DEBI (Diffusion of Effective Behavioral Interventions) American Journal of Community Psychology. 2008;42(1–2):51–59. [PMC free article] [PubMed]
  • Eccles JS, Wigfield A. Motivational beliefs, values, and goals. Annual Review of Psychology. 2002;53:109–132. [PubMed]
  • Estrin D, Sim I. Health care delivery. Open mHealth architecture: an engine for health care innovation. Science. 2010 Nov 5;330(6005):759–60. [PubMed]
  • Fine LJ, Philogene GS, Gramling R, Coups EJ, Sinha S. Prevalence of multiple chronic disease risk factors. 2001 National Health Interview Survey. American Journal of Preventive Medicine. 2004;27(Suppl 2):18–24. [PubMed]
  • Fishbein M, Bandura A, Triandis HC, et al. Final Report- NIMH Theorists Workshop. Washington, DC: National Institute of Mental Health; 1991. Factors influencing behavior and behavior change.
  • Flay B, Biglan A, Boruch R, Castro F, Gottfredson D, Kellam S, Ji P. Standards of evidence: Criteria for efficacy, effectiveness and dissemination. Prevention Science. 2005;6(3):151–175. [PubMed]
  • Glasgow R, Sanchez M. News from NIH: Global Health. Translational Behavioral Medicine. 1(2):201–202. [PMC free article] [PubMed]
  • Goldstein MG, Whitlock EP, DePue J. Multiple behavioral risk factor interventions in primary care: Summary of research evidence. American Journal of Preventive Medicine. 2004;27(Suppl):61–67. [PubMed]
  • Goldstein RB, Olfson M, Martens EG, Wolk SI. Subjective unmet need for mental health services in depressed children grown up. Administration and Policy in Mental Health and Mental Health Services Research. 2006;33(6):666–673. [PubMed]
  • Gottman JM, Silver N. The seven principals for making marriage work. New York: Three Rivers Press; 1999.
  • Graham ID, Logan J, Harrison MB, Straus SE, Tetroe JM, Caswell W, Robinson N. Lost in knowledge translation: Time for a map? Journal of Continuing Education in Health Professions. 2006;26:13–24. [PubMed]
  • Green LS, Oades LG, Grant AM. Cognitive-behavioral, solution-focused life coaching: Enhancing goal striving, well-being and hope. Journal of Positive Psychology. 2006;1(3):142–149.
  • Grencavage L, Norcross J. Where are the commonalities among the therapeutic common factors? Professional Psychology: Research and Practice. 1990;21(5):372–378.
  • Greenwald JL, Burstein GR, Pincus J, Branson B. A rapid review of rapid HIV antibody tests. Current Infectious Disease Reports. 2006 Mar;8(2):125–31. [PubMed]
  • Hall KL, Rossi JS. Meta-analytic examination of the strong and weak principles across 48 health behaviors. Preventive Medicine. 2008;46:266–274. [PubMed]
  • Hallett TB, White PJ, Garnett GP. Appropriate evaluation of HIV prevention interventions: From experiment to full-scale implementation. Sex Transm Infect. 2007;83:i55–i60. [PubMed]
  • Harris KM, Edlund MJ, Larson S. Racial and ethnic differences in the mental health problems and use of mental health care. Medical Care. 2005;43(8):775–784. [PubMed]
  • Harwood TM, L’Abate L. Self-help in mental health: A critical review. New York: Springer; 2010.
  • Hawe P, Shiell A, Riley T. Complex interventions: how “out of control” can a randomised controlled trial be? British Medical Journal. 2004;328:1561–1563. [PMC free article] [PubMed]
  • Hawkins JD, Kosterman R, Catalano RF, Hill KG, Abbott RD. Effects of social development intervention in childhood fifteen years later. Archives of Pediatrics & Adolescent Medicine. 2008;162(12):1133–1141. [PMC free article] [PubMed]
  • Hill I, Hogan S, Palmer L, Courtot B, Gehshan S, Belnap D, Snyder A. Medicaid Outreach & Enrollment for Pregnant Women: What is the State of the Art? [Report for the March of Dimes Foundation] Washington, DC: Urban Institute; 2009. Mar,
  • Ingram BL, Flannery D, Elkavich A, Rotheram-Borus MJ. Common processes in evidence-based adolescent HIV prevention programs. AIDS and Behavior. 2008;12(3):374–383. [PubMed]
  • Karver MS, Handelsman JB, Fields S, Bickman L. A Theoretical Model of Common Process Factors in Youth and Family Therapy. Mental Health Services Research. 2005;7(1):35–51. [PubMed]
  • Kazak AE, Hoagwood K, Weisz JR, Hood K, Kratochwill TR, Vargas LA, Banez GA. A meta-systems approach to evidence-based practice for children and adolescents. American Psychologist. 2010;65(2):85–97. [PubMed]
  • Kazdin AE. Evidence-based treatment and practice: New opportunities to bridge clinical research and practice, enhance the knowledge base, and improve patient care. American Psychologist. 2008;63:146–159. [PubMed]
  • Kazdin AE. Treatment outcomes, common factors, and continued neglect of mechanisms of change. Clinical Psychology: Science and Practice. 2005;12(2):184–188.
  • Kazdin AE, Blase SL. Rebooting psychotherapy research and practice to reduce the burden of mental illness. Perspectives on Psychological Science. 2011;6(1):21–37.
  • Kelly JA. Community-level interventions are needed to prevent new HIV infections. American Journal of Public Health. 1999;89:299–301. [PubMed]
  • Kessler RC, Wang PS. The descriptive epidemiology of commonly occurring mental disorders in the United States. Annual Review of Public Health. 2008;29:115–129. [PubMed]
  • Kimbuende E, Ranji U, Lundy J, Salganicoff A. US Health Care Costs. 2010 Retrieved from
  • Kluger AN, DeNisi A. Effects of a feedback intervention on performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychological Bulletin. 1996;119(2):254–284.
  • Kodner D, Spreeuwenberg C. Integrated care: logic, meaning, implications and applications. International Journal of Integrated Care. 2002;2:e12. [PMC free article] [PubMed]
  • Krebs P, Prochaska JO, Rossi JS. A meta-analysis of computer-tailored interventions for health behavior change. Preventive Medicine. 2010 Sep-Oct;51(3–4):214–21. Epub 2010 Jun 15. [PMC free article] [PubMed]
  • Krieger N. Genders, sexes, and health: what are the connections – and why does it matter? International Journal of Epidemiology. 2003;32(4):652–657. [PubMed]
  • Lambert MJ. Emerging methods for providing clinicians with timely feedback on treatment effectiveness: An introduction. Journal of Clinical Psychology. 2005;61(2):141–144.
  • Laws M, Scott MK. The emergence of retail-based clinics in the United States: Early observations. Health Affairs. 2008;27(5):1293–1298. doi: 10.1377/hlthaff.27.5.1293. [PubMed] [Cross Ref]
  • Leatherman S, McCarthy D. Quality of Health Care for Medicare Beneficiaries: A Chartbook. New York: The Commonwealth Fund; 2005.
  • Leavitt SA. “A private little revolution”: the home pregnancy test in American culture. Bulletin of the History of Medicine. 2006;80(2):317–345. [PubMed]
  • Lepore S, Smyth J, editors. The writing cure: How expressive writing promotes health and emotional well-being. American Psychological Association press; Washington, DC: 2002.
  • Lewin S, Munabi-Babigumira S, Glenton C, Daniels K, Bosch-Capblanch X, Scheel IB. Lay health workers in primary and community health care for maternal and child health and the management of infectious diseases. Cochrane Database of Systematic Reviews. 2010;(3) [PubMed]
  • Longshore D, Grills C. Motivating illegal drug use recovery: Evidence for a culturally congruent intervention. The Journal of Black Psychology. 2000;26(3):288–301.
  • Lordan G, Tang KK, Carmignani F. Has HIV/AIDS displaced other health funding priorities? Evidence from a new dataset of development aid for health. Social Science & Medicine. 73(3):351–355. [PubMed]
  • Lujan J. Got Hispanic clients? Get a promotora. The Internet Journal of Allied Health Sciences and Practice. 2009;7(3) Retrieved from
  • Maibach EW, Van Duyn MA, Bloodgood B. A marketing perspective on disseminating evidence-based approaches to disease prevention and health promotion. Prev Chronic Dis. 2006;3(3):A97. [PMC free article] [PubMed]
  • Mielke KW. Research and development at the Children’s Television Workshop. Educational technology research and development. 1990;38(4):7–16.
  • Moore JS, Rogers M. Female-controlled prevention technologies. In: O’Leary A, editor. Beyond Condoms: Alternative Approaches to HIV Prevention. New York: Kluwer/Plenum; 2002. pp. 47–76.
  • Murphy Tim. How We Got to ‘Sesame Street’ New York Magazine. 2009 Nov 01; Retrieved 2011-08-23.
  • Nathan PE, Gorman JM, editors. A Guide to Treatments that work. 3. New York: Wiley; 2007.
  • National Center for Education Statistics, Common Core of Data (CCD) Public Elementary/Secondary School Universe Survey 2004–05, Version 1a
  • National Research Council & Institute of Medicine. Preventing mental, emotional, and behavioral disorders among young people: Progress and possibilities. Consensus Report. 2009 Retrieved from [PubMed]
  • Noar SM, Chabot M, Zimmerman RS. Applying health behavior theory to multiple behavior change: Considerations and approaches. Preventive Medicine. 2008;46(3):275–280. [PubMed]
  • O’Connor MJ, Whaley SE. Brief intervention for alcohol use by pregnant women. American Journal of Public Health. 2006;97(2):252–258. [PubMed]
  • O’Donohue WT, Cucciare MA. Pathways to medical service utilization. Journal of Clinical Psychology in Medical Settings. 2005;12(2):185–197.
  • Olds DL, Sadler L, Kitzman H. Programs for Parents of Infants and Toddlers: Recent Evidence from Randomized Trials. Journal of Child Psychology and Psychiatry. 2007;48:355–391. [PubMed]
  • Patel V. The future of psychiatry in low-and middle-income countries. Psychological Medicine. 2009;39:1759–1762. [PubMed]
  • Patterson TL, Shaw WS, Semple SJ. Reducing the sexual risk behaviors of HIV+ individuals: Outcome of a randomized controlled trial. Annals of Behavioral Medicine. 2003;25(2):137–145. [PubMed]
  • Pennebaker JW. Writing about emotional experiences as a therapeutic process. Psychological Science. 1997;8:162–166.
  • Pine JB, Gilmore JH. The Experience Economy. Cambridge, MA: Harvard Business School Press; 1999.
  • Prochaska JJ, Spring B, Nigg CR. Multiple health behavior change research: An introduction and overview. Preventive Medicine. 2008;46(3):181–188. [PMC free article] [PubMed]
  • Raghavan R, Bright CL, Shadoin AL. Toward a policy ecology of implementation of evidence-based practices in public mental health settings. Implementation Science. 2008;3(26) [PMC free article] [PubMed]
  • Ringel JS, Sturm R. National estimates of mental health utilization and expenditures for children in 1998. J Behav Health Serv Res. 2001;28:319–333. [PubMed]
  • Rotheram-Borus MJ, Armstrong M, Booraem C. Assertiveness training in fourth-and fifth-grade children. American Journal of Community Psychology. 1982;10:567–582. [PubMed]
  • Rotheram-Borus MJ, Etzel M. Rethinking the HIV counseling and testing model in the USA. Lancet. 2003;361:611–612. [PubMed]
  • Rotheram-Borus MJ, Flannery D, Rice E, Lester P. Families living with HIV. AIDS Care. 2005;17(8):978–987. [PubMed]
  • Rotheram-Borus MJ, Ingram BL, Swendeman D, Flannery D. Common principles embedded in effective adolescent HIV prevention programs. AIDS and Behavior. 2009;13(3):387–98. [PMC free article] [PubMed]
  • Rotheram-Borus MJ, Lester P, Song J, Lin YY, Leonard NR, Beckwith L, Lord L. Intergenerational benefits of family-based HIV interventions. Journal of Consulting and Clinical Psychology. 2006;74(3):622–627. [PubMed]
  • Rotheram-Borus MJ, Swendeman D, Flannery D, Rice E, Adamson DM, Ingram B. Common factors in effective HIV prevention programs. AIDS and Behavior. 2009;13(3):399–408. [PMC free article] [PubMed]
  • Rotheram-Borus MJ, Le Roux IM, Tomlinson M, Mbewu N, Comulada WS, le Roux K, Stewart J, O’Connor MJ, Hartley M, Desmond K, Greco E, Worthman CM, Idemundia F. Philani Plus (+): a randomized controlled trial of Mentor Mother home visiting program to improve infants’ outcomes. Prevention Science. 2011;12:372–388. [PMC free article] [PubMed]
  • Schlosser E. Fast food nation: The dark side of the all-American meal. New York, NY: Houghton Mifflin Company; 2001.
  • Schoenwald SK, Hoagwood KE, Atkins MS, Evans ME, Ringeisen H. Workforce development and the organization of work: The science we need. Administration and Policy in Mental Health and Mental Health Services Research. 2010;37:71–80. [PubMed]
  • Schor JB. Born to buy. New York, NY: Scribner; 2004.
  • Singhal A, Cody MJ, Rogers SM, Santelli JS, editors. Entertainment-Education and social change: History, research, and practice. Mahwah, NJ: Lawrence Erlbaum Associates; 2004.
  • Southam-Gerow M, Weisz J, Chu B, McLeod B, Gordis E, Connor-Smith J. Does cognitive behavioral therapy for youth anxiety outperform usual care in community clinics? An initial effectiveness test. Journal of the American Academy of Child & Adolescent Psychiatry. 2010;49:1043–1052. [PMC free article] [PubMed]
  • Standage Tom. Mobile Marvels. The Economist. 2009 Sep 26;
  • Strong K, Mathers C, Leeder S, Beaglehole R. Preventing chronic diseases: how many lives can we save? The Lancet. 2005;366(9496):1578–1582. [PubMed]
  • Sutherland A, Thompson B. Kidfluence: Why kids today mean business. Toronto, Canada: McGraw-Hill Ryerson; 2001.
  • Swendeman D, Rotheram-Borus MJ. Innovation in sexually transmitted disease and HIV prevention: Internet and mobile phone delivery vehicles for global diffusion. Current Opinion in Psychiatry. 2010;23(2):139–144. [PMC free article] [PubMed]
  • Tomashoff C. Cash machines spawn a green revolution of sorts. The Los Angeles Times. 2002 Dec 9; Retrieved from
  • Truglio RT, Fisch SM. Introduction. In: Fisch SM, Fisch R, editors. “G” Is for Growing: 30 Years of Research on Children and Sesame Street. Mahwah, NJ: Erlbaum; 2001.
  • Walsh R. Lifestyle and mental health. Psychologist. 2011;66:579–592. [PubMed]
  • Wang PS, Lane M, Olfson M, Pincus HA, Wells KB, Kessler RC. Twelvemonth use of mental health services in the United States: Results from the National Comorbidity Survey Replication. (2005) Arch Gen Psychiatry. 2005;62(6):629–640. [PubMed]
  • Weisner TS. Ecocultural understanding of children’s developmental pathways. Human Development. 2002;45(4):275–281.
  • Weisz JR, Chorpita BF, Palinkas LA, Schoenwald SK, Miranda J, Bearman SK, Daleiden EL, Ugueto AM, Ho A, Martin J, Gray J, Alleyne A, Langer DA, Southam-Gerow MA, Gibbons RD. and the Research Network on Youth Mental Health. Testing standard and modular designs for psychotherapy with youth depression, anxiety, and conduct problems: A randomized effectiveness trial. Archives of General Psychiatry. 2012;69:274–282. [PubMed]
  • Weisz JR, Southam-Gerow M, Gordis E, Connor-Smith J, Chu B, Langer DA, McLeod B, Jensen-Doss A, Updegraff A. Cognitive–behavioral therapy versus usual clinical care for youth depression: An initial test of transportability to community clinics and clinicians. Journal of Consulting and Clinical Psychology. 2009;77:383–396. [PMC free article] [PubMed]
  • Weisz J, Sandler I, Durlak J, Anton B. Promoting and protecting youth mental health through evidence-based prevention and treatment. American Psychologist. 2005;60(6):628. [PubMed]
  • Weisz JR, Kazdin AE, editors. Evidence-based Psychotherapies for Children and Adolescents. New York: Guilford Press; 2010.
  • Weisz JR, Jensen-Doss A, Hawley KM. Evidence-Based Youth Psychotherapies Versus Usual Clinical Care: A Meta-Analysis of Direct Comparisons. American Psychologist. 2006;61(7):671–689. [PubMed]
  • World Health Organization. [Accessed 10 August 2009];Task shifting: global recommendations and guidelines. 2008
  • Zima BT, Hurlburt MS, Knapp P, Ladd H, Tang L, et al. Quality of publicly-funded outpatient specialty mental health care for common childhood psychiatric disorders in California. Journal of the American Academy of Child and Adolescent Psychiatry. 2005;44:130–144. [PubMed]