|Home | About | Journals | Submit | Contact Us | Français|
The promise of widespread implementation of efficacious interventions across the cancer continuum into routine practice and policy has yet to be realized. Multilevel influences, such as communities and families surrounding patients or health-care policies and organizations surrounding provider teams, may determine whether effective interventions are successfully implemented. Greater recognition of the importance of these influences in advancing (or hindering) the impact of single-level interventions has motivated the design and testing of multilevel interventions designed to address them. However, implementing research evidence from single- or multilevel interventions into sustainable routine practice and policy presents substantive challenges. Furthermore, relatively few multilevel interventions have been conducted along the cancer care continuum, and fewer still have been implemented, disseminated, or sustained in practice. The purpose of this chapter is, therefore, to illustrate and examine the concepts underlying the implementation and spread of multilevel interventions into routine practice and policy. We accomplish this goal by using a series of cancer and noncancer examples that have been successfully implemented and, in some cases, spread widely. Key concepts across these examples include the importance of phased implementation, recognizing the need for pilot testing, explicit engagement of key stakeholders within and between each intervention level; visible and consistent leadership and organizational support, including financial and human resources; better understanding of the policy context, fiscal climate, and incentives underlying implementation; explication of handoffs from researchers to accountable individuals within and across levels; ample integration of multilevel theories guiding implementation and evaluation; and strategies for long-term monitoring and sustainability.
Most scientific evidence about improving health and health care stems from single-site and single-level interventions, taking decades to move from clinical trials to new routines at bedsides or clinic offices (1,2). Despite the growing volume of such interventions in the literature, their application in typical practice settings remains stubbornly elusive, rendering the promise of evidence-based practice—widespread implementation of efficacious interventions into routine clinical care—still unrealized (3,4). Also, few interventions address interventions outside health-care settings, limiting potential contributions of community-level interventions to advances in public health (eg, mobile units, neighborhood screening). The heart of the matter, however, is that the evidence itself is insufficient, as single-level interventions have chiefly been tested under highly-controlled and homogenized circumstances, often in academic medical centers or other settings—circumstances unlike those in which most patients obtain their care (5,6). As a result, interventions yielding significant advances under controlled research protocols undergo what has been described as a “voltage drop” when applied to real-world settings (7).
Applying the current state of research evidence to health care (ie, fostering the adoption, implementation, spread, and sustainability of new evidence-based approaches to care) requires explicit attention to the interactions between and among multiple levels of influence surrounding any particular single-level intervention (ie, communities and families surrounding patients; health-care policies and organizations surrounding provider teams) (8,9). Indeed, practice guidelines have increasingly embraced multilevel concepts (eg, tobacco control guidelines incorporate patient-, provider-, and system-level recommendations) (10,11), though rarely based on trials that themselves were multilevel (12,13).
Though seldom reported (14), the contextual influences underlying intervention success (or failure) have been the subject of increasing study, as each contextual layer potentially becomes the target for additional intervention components (15–19). Greater recognition of such influences has motivated the design and testing of multilevel interventions that target them (20,21). However, relatively few multilevel interventions (comprising ≥3 levels) have been conducted along the cancer care continuum, and fewer still have been implemented, spread, or sustained in practice (20).
The purpose of this chapter is to illustrate and examine the concepts underlying the implementation and spread of predominantly single-level interventions into the multilevel context of routine practice and policy using a series of cancer and noncancer examples. The examples span different levels and stages of the care continuum, from community-based primary prevention to screening in diverse clinical practices to treatment implementation and spread in large integrated health-care systems.
Efficacy studies place primary emphasis on internal validity to maximize the certainty with which claims may be made that the intervention was responsible for the observed differences in outcomes. Effectiveness (and implementation) studies must generalize from efficacy studies, recognizing all the ways in which they lack external validity and particular relevance to the local circumstances in which they would be applied and necessarily adapted (22) (Table 1). Adaptations require setting-specific evaluations as efficacy studies provide no assurance that the adaptations will achieve the same effects in different settings, circumstances, populations, cultures, and political environments (23). These differences account for much of the diminished impact when interventions from efficacy trials are implemented more broadly.
Implementation and spread are neither direct nor intuitive when patients are selected to reduce complexity, when interventions are tested only in the most favorable environments, when context is factored out, and when researchers work to ensure strict protocol adherence and control (that will not typically be feasible during implementation in other sites/levels). Rather than being entirely controlled by researchers, interventions implemented in real-world settings must involve and engage policymakers, managers, providers, nurses, clerks, and usually patients and their families, as key stakeholders in the new processes underlying implementation at each level. These stakeholders are directly engaged in working to determine how to adapt intervention elements to their practice and routines and within their social norms and settings (ie, their context). Researchers’ capacity to influence such adoption is acutely determined by the nature of the “handoffs” and support they construct through negotiation with the people, places, and circumstances of each environment they seek to improve. Each aspect of change (for stakeholders at each level), therefore, requires consideration of how individuals contribute to (or hinder) implementation. To further spread interventions to achieve a universal and permanent new way of doing business, new organizational units and/or fiscal policies may be required or new legislation enacted (24).
Furthermore, not all contextual factors are modifiable, requiring adaptation that stretches beyond the available evidence base (eg, urbanization, family structure) and commonly beyond investigators’ comfort zones (18). As adaptation extends to less familiar levels in which investigators have less influence, the inevitable drift from the seeming simplicity of the original evidence bases to accommodate increasingly diverse practices and communities complicates virtually everything (5,25). Hawe et al. (26) recommend, instead, starting with an understanding of the community first and studying how phenomena are reproduced in that system, rather than focusing on mimicking processes from the original controlled setting. Either way, it is essential to bridge the gaps between evidence-based practice and practice-based evidence (27).
Much of the research evaluating implementation of interventions in real-world settings has lacked strong theoretical foundations, thereby ignoring the contributions of different social science disciplines to their design and implementation (28,29). While theoretical frameworks represent an important resource for designing implementation efforts (30–32), no integrative theories have been developed to specifically guide implementation across multiple levels. Designing effective approaches for multiple levels may require a collection of theories addressing behavior and behavior change at each component level, whereas others have recommended a consolidated framework across often overlapping theories to help explain implementation in multilevel contexts (33). Related fields offer additional guidance in identifying theories for use in multilevel implementation [eg, patient (31,34), professional (35), and organizational behavior change (15)] and are addressed in other chapters (36,37). Theories in political science and policy studies also are available to help researchers address levels of government and regulatory agencies (30,32). In addition to theories offering detailed depictions of causal relationships, a number of planning frameworks (eg, Predisposing, Reinforcing and Enabling Constructs in Educational Diagnosis and Evaluation [PRECEDE]-Policy, Regulatory, and Organizational Constructs in Educational and Environmental Development [PROCEED]) and conceptual models (eg, Chronic Care Model) identify broad categories of factors to consider, although many stop short of specifying individual causal relationships and influences of these factors (38,39).
To grapple with these issues, we drew on our combined experience with a series of interventions whose implementation and spread spanned different levels (Figure 1 and Table 2). Because the use of multilevel interventions in cancer care is still developing, we also included a noncancer example that has spanned virtually all levels. Use of theory is well reflected in the examples chosen.
The Pool Cool Diffusion Trial tested a three-level skin cancer prevention program at recreational swimming pools (44). To implement, disseminate, and evaluate the program, the project team had to build effective relationships with professional organizations and recreation sites at national, regional, and local levels. This was achieved by participating in aquatics and recreation conferences, developing career opportunities and encouraging local media coverage of program activities (55,56), and providing resources to conduct the program after research participation concluded (57).
The Pool Cool program drew from social cognitive theory (58), diffusion of innovations theory (59–61), and theories of organizational change (49). These models are complementary, with considerable overlap among them (50,58). The investigators’ intent was not to test a single model but to apply the most promising constructs from each to the problem of skin cancer prevention and program diffusion in aquatics settings.
Three examples focused on CRC screening, one of which also focused on follow-up.
CHOICE combined patient activation through decision aides and brochures among health plan members with academic detailing to prepare practices to facilitate CRC screening for activated patients (40,41). A cluster randomized trial, CHOICE required extensive engagement and partnership development at all levels. The CHOICE intervention relied on social cognitive theory (multiple levels) and the transtheoretical model of change (stages of change) (for the decision side, patient/member level) (51,62).
Sequist et al. designed randomized multilevel systems interventions to assess whether CRC screening could be increased among overdue adults. The study was conducted at HVMA, a large integrated medical group in Eastern Massachusetts. Screening rates were higher for patients who received mailings compared with those who did not, and the effect increased with patients’ age (42). Screening rates were similar among patients whose physicians received electronic reminders and those whose physicians were in the control group. However, reminders tended to increase screening rates among patients with three or more primary care visits over the 15-monthintervention. Adenoma detection tended to increase with both patient mailings and physician reminders. With a cost-effectiveness ratio of USD $94 for each additional patient screened, patient mailings were deemed cost-effective for continued use by the organization (43).
Improving CRC screening has been a longstanding national priority in the VHA health-care system, followed by more recent emphasis on managing timely, complete endoscopic follow-up and treatment. These examples span the VHA Colorectal Cancer Care Collaborative (C4) and Veterans Affairs (VA) Colorectal Cancer Quality Monitoring System (54,63), which grew out of QUERI (Quality Enhancement Research Initiative) (64,65).
The HVMA and VHA examples were more explicitly anchored in principles of continuous quality improvement during implementation phases, guided by Plan-Do-Study-Act (PDSA) cycles. Originally proposed by Langley et al. (66), amplified by Berwick (67), and applied in QUERI (64,65), PDSAs have often been used for smaller-scale rapid-cycle improvements. PDSA has also been adopted for broader organizational initiatives to improve quality of care (68). Stone et al. (53) have augmented this approach to guide quality improvement interventions to promote cancer screening services, identifying several key intervention features, including top management support, high visual appeal and clarity, collaboration and teamwork, and theory-based tailoring of interventions based on current needs and barriers. In both HVMA and VHA examples, these insights were primarily used during planning and pilot phases, when study interventions at different levels were refined with input from organizational leaders and pilot testing at one or more health centers or practices.
The Office on Smoking and Health of the Centers for Disease Control and Prevention (CDC) examined the experience of several successful statewide tobacco control programs in the early to mid-1990s (particularly California and Massachusetts, but also specific lessons drawn from Arizona, Oregon, Florida, and Mississippi in the mid- to late-1990s). They blended these programs with the evidence-based literature on tobacco control from other sources to produce a widely adopted document titled Best Practices for Comprehensive Tobacco Control Programs (45). A second edition was published in 2007, based on the growing evidence from other states after following the lead of the initial states and another CDC document, Introduction to Program Evaluation for Comprehensive Tobacco Control Programs (69). On reviewing the evidence of effective comprehensive statewide programs, CDC concluded that no single intervention by itself, other than sharply increased prices on cigarettes through taxation, could account for the significant changes in tobacco consumption found over time. California and Massachusetts, in particular, doubled, tripled, and then quadrupled the rate of decline in tobacco consumption of the other 48 states while implementing their comprehensive statewide programs. Less comprehensive programs had successes in specific subpopulations, on specific outcomes, at specific levels of their states, but not as dramatic as California's or Massachusetts's comprehensive, multilevel programs. This example spans national and state policy changes as an overlay to organizational-level interventions that occurred, for example, in schools, worksites, and restaurants within statewide programs. The overriding theoretical framework for the tobacco control programs was social normative theory, which drove the mass media and smoke-free policy initiatives and which, in turn, undermined the tobacco industry's promotions and the acceptance of smoking in public (70–72).
The TIDES initiative, which began with a planning phase in 2001 and enrolled its first patients in 2002, used evidence-based quality improvement (EBQI) methods as the basis for redesigning, adapting, and spreading collaborative care models for improving outcomes among primary care patients with depression. Collaborative care models have been shown to be effective and cost-effective based on more than 35 randomized trials and meta-analyses. Also supported by the VA QUERI program, TIDES was a multiregion EBQI effort to adapt and implement the research-based depression collaborative care models to the context of the large national VA health-care system.
The EBQI approach used regional and local iterative meetings to adapt and tailor collaborative care evidence—a multicomponent intervention directed at primary care patients who screen positive for depression—to the VA context. Key intervention features included a depression care manager supervised by a mental health specialist, structured assessment and follow-up of depressed patients, and patient self-management support. Key EBQI features are regional leadership priority setting, a research/clinical partnership with involvement of technical experts, and iterative intervention development with provider- and practice-level feedback on collaborative care intervention performance. The overriding goal of the series of projects that comprised the TIDES initiative was to use regional and local adaptation of the evidence-based care model as the basis for national VHA implementation, which occurred in 2006. The VHA-only SharePoint website, which houses TIDES tools and methods, continues to be accessed about 2000 times per month from all VHA regions across the country, in addition to an internet site sponsored by the Substance Abuse and Mental Health Services Administration (http://www.ibhp.org). In addition to continuous quality improvement, TIDES also relied on the Chronic Illness Care model (39) and tenets of social marketing (73).
Table 3 provides a summary of the lessons learned about the implementation and spread of interventions into multilevel practice and policy. Several key themes emerged from our examination of these diverse examples.
Attention to the nature of stakeholders at each level is key to successful implementation of a multilevel intervention, as is a strong understanding of how levels may interact. For example, in CHOICE, academic detailing was designed to prepare providers for patients activated by the decision aide. The HVMA delivered patient and provider reminders in parallel. Creating interdependencies also can be beneficial, for example, when local programs received tobacco control funding for mapping to state-level program activities or where local facilities received incentives for achieving compliance with CRC follow-up performance monitors. Determining the quality of the evidence (and continually integrated new evidence) for the interventions being deployed at each level also is important. However, when the evidence is lacking, blending scientific literature with experience from successful programs can be especially useful. Use of social marketing strategies also provided interventional messaging that penetrated multiple levels, though messages often have to be honed for each level's target audience (ie, what rivets the attention of patients likely differs from that of providers or policymakers). Several projects emphasized rapid cycle improvement pilots to test functions and effectiveness of implementation efforts within and across levels. This approach is especially important given the size and complexity of multilevel interventions and the importance of balancing fidelity and flexibility when adapting to local contexts.
Implementation also benefited from staged approaches, beginning with pilot testing within levels at a single practice or community followed by broader implementation as details and needs at each level become clearer (5). Recognition of the time needed for changes to penetrate each level's members’ knowledge and behavior is often underappreciated. For example, many multilevel interventions rely on champions, which requires education/training of the champion and then their peers or constituents (either by the champion or project team) through formal or informal social networks (76).
The direction of implementation—top–down vs bottom–up—also is an important distinction. In the Pool Cool program, the demand for and interest in the program went in different directions at different levels of the intervention. In some regions, motivated leaders at the top sometimes dictated program involvement, whereas in other regions, someone from a “lower level” (eg, a specific pool) was resourceful enough to find other sites and resources to bring the program to the local area. Tobacco control successes clearly moved from local and state levels to the national level for dissemination to other states that could emulate successful states’ practice-based experience, blended with evidence-based practices from controlled trials on specific interventions. TIDES also grew from a bottom–up intervention design guided by regional priorities and later was adopted nationally. Experiences from these programs, as well as others, also point to the importance of comprehensive process evaluations to measure the levers and directions of implementation, as well as the processes used, if any, to promote activity and align interests at different levels.
The importance of partnerships within and across levels and between researchers, clinicians, and managers was a clear and consistent theme across the examples, reflecting in large part the reduced control that researchers have over implementation dynamics on each level and the need to hand off intervention activities to nonresearchers—otherwise, it would not be “routine care” (5). To fit local conditions, proactive and intentional adaptations to the environmental and organizational milieu represented by each partnership level (eg, practice tailoring) reduce the risk of failed implementation (77–84). Such partnerships require shared knowledge, trust, and role specification; require time spent in relationship- and team-building before, during and after implementation (with changing roles over time); and continual identification of a growing network of stakeholders who will ultimately maintain and be responsible for the intervention components at their level. Few studies have documented the costs associated with such implementation, with the exception of TIDES, which demonstrated substantial contributed time by implementers and researchers (85).
Strong support from senior leaders is also essential. Policy, community, practice, and other leaders help ensure engagement of members at their respective levels and frequently secure and allocate resources while also encouraging other participants who may need to be involved (eg, engaging gastroenterology and/or radiology specialists in primary care–based efforts to improve CRC screening). Senior leaders also are accountable for implementation and maintenance activities between research team contacts and may play a major role in coalition building. Partnerships with health information technology staff also were considered key, especially in settings with electronic medical records (EMRs).
Consistent with the Institute of Medicine's Crossing the Quality Chasm report (86), our examples point to the importance of organizational supports for implementation. In some scenarios, such supports may be centralized across a large number of sites (eg, computerized decision support in practices with a shared EMR or state-level media campaigns for tobacco control) and may include direct grants, special funding allocations, and/or protected time for quality improvement and training. The degree of leadership control over a particular level may also increase the consistency of implementation, especially when supported by regular feedback of evaluation data. For example, in the HVMA CRC screening intervention, organizational leaders fully endorsed the programs being developed, allowing key quality improvement staff to participate actively in their design and implementation. However, implementation that requires interdisciplinary cooperation may be met with resistance when members at a particular level compete for resources or control or operate in silos where communication and coordination mechanisms may not have been developed. The perceived importance or value of implementation goals must be balanced with competing demands among busy members at any given level (87,88). These kinds of implementation barriers may not be predictable, underscoring the value of planning phases, “pre-work,” and PDSA cycles as integral components of implementation efforts.
Insofar as all behavior is affected by context, our examples demonstrated the vital importance of understanding the contextual influences surrounding players at each level of implementation. For example, the policy context in Massachusetts during the time of the HVMA CRC screening initiative was a virtual “perfect storm” in favor of implementation, as confirmed in structured interviews with HVMA chief medical officers, another large integrated provider network in the same region, and two regional insurers. The National Committee for Quality Assurance (NCQA) had introduced a new Healthcare Effectiveness and Data Information Set (HEDIS) measure for CRC screening in 2004 (89), with two of Massachusetts's four major insurers having participated in NCQA's field testing of the new measure. Pay-for-performance incentives for CRC screening rates also were being incorporated in some health-plans’ provider contracts, and a statewide quality monitoring program, Massachusetts Health Quality Partners (http://www.mhqp.org), was preparing to release statewide public reports on medical groups’ CRC screening rates. In other states without this policy context, the same level of adoption and participation might not have been seen.
Similarly, the rapid adoption and implementation of practice-based evidence for tobacco control from California and Massachusetts was accelerated by the Master Settlement Agreement between the states’ attorneys-general and the tobacco industry, which infused large amounts of earmarked funds into state tobacco control budgets. Implementation in settings where the fiscal climate is more difficult requires advance assessment of practice priorities and placement of the intervention among competing demands, in addition to adapting to local constraints.
Few examples of intervention spread are generally available. Among our examples, the spread of successful tobacco control programs benefited from CDC's best practices document as a touchstone for planning programs at a time when the Master Settlement funds became available from the lawsuit filed against the tobacco industry, making its publication both timely and immediately applicable. Although such timing may occur serendipitously, implementation clearly benefits when advances at different levels of influence co-occur.
In the 4 years since the HVMA CRC screening interventions were originally implemented, the CRC screening rates have continued to rise from 63% to about 85%, which is one of the highest publicly reported rates for any medical group, health-plan, or region in the United States. This high rate was achieved through a strong organizational commitment to CRC screening, an advanced EMR for tracking CRC screening and other preventive services, and an expanded capacity to perform screening colonoscopy (by about 300 procedures per month) at a new HVMA endoscopy center.
Champions can support spread in addition to implementation, for example, through initial practices’ sharing of their experiences and troubleshooting with spread practices. Such person-to-person support, however, may best be accomplished when augmented with tools that facilitate adoption in new locations (eg, tracking tools, compendia of evidence, listservs, resource websites), adaptation to new populations (or subgroups), and measurement and evaluation.
However, one of the keys to implementation and spread based on these examples is the explication of the handoffs of multilevel intervention activities from researchers to accountable individuals within and across levels. When researchers support implementation by offloading certain activities from providers, they are unintentionally creating a nonsustainable situation. Furthermore, when multilevel interventions engage several clinical disciplines and multiple levels of leadership, no single handoff strategy is likely to succeed. Better assessments of usual practice, development of explicit memoranda of understanding (ie, spelling out the details of new roles and responsibilities), and continual management of research–clinical partnerships help alleviate at least some of these issues.
Implementation of current evidence remains painfully slow, and the evidence base itself may not change as fast or as dramatically as often implied. Nonetheless, one of the reasons it is difficult to implement and spread evidence-based practice is that the levels of implementation are often changing. Practices face provider and staff turnover and leadership changes, and the political environment is always evolving. Just as multilevel influences are in perpetual motion, so is the evidence base to support interventions. New trials are completed, whereas observational studies contribute new information to our understanding of the factors involved in patient, provider, or organizational behavior and beyond. It is therefore important to continually scan and integrate new evidence over time: Sustainability may be a myth as there is always new evidence to consider, new people to train, practices opening and closing, communities adapting to new contexts, and state and federal agencies and their priorities changing. Unfortunately, systematic reviews, in their typically exclusive reliance on randomized controlled trials, will not close the information gap in the strategies for implementation, spread, and sustainability.
Based on the examples we reviewed, the best evidence for sustainability is long-term and continual attention to influences within and across all levels, enabled by engagement of people and places with ever increasing and overlapping spheres of influence (90). Integration of evidence into new national norms, regardless of how such norms are fostered or reinforced (eg, through performance measures, new reimbursement policies or legislation), is an essential method for sustaining multilevel change, though the path at the national level is complex and circuitous at best.
While full treatment of the range of study design and other methodological issues rooted in implementation and spread research are beyond the scope of this monograph, Table 2 provides insights into the methodological approaches each example used, as well as the challenges they faced. Key issues span study design complexity, geographic scope, measures and data collection mapped to multiple levels and over multiple waves, and the inherent value of EMR systems for supporting evaluation and monitoring.
In this chapter, we used several exemplary studies to illustrate key concepts underlying the implementation and spread of interventions into the multilevel contexts of routine practice and policy. Lessons from these studies provide insights into approaches for handling implementation of interventions and partnerships within and across levels, as well as facilitators and barriers for their implementation, spread, and sustainability.
Advancing implementation will continue to be a challenge for the foreseeable future. Discomfort with the compromises inherent in the naturalistic rollout of intervention activities at multiple levels (in contrast to experimental control focused on reducible variation) slows our ability to meet these challenges. Criticisms against multilevel intervention research are also misguided when they are based on the contention that it is inherently difficult to discern the relative contributions of each intervention component. Experience from our examples suggests that they produce synergies and complementary effects, which require mixed methods and may benefit from hybrid designs to yield useful information. Furthermore, implementation requires expertise in politics and diplomacy, skills rarely taught in scientific curricula, in addition to flexibility, comfort with uncertainty, and persistence.
Experiences from our examples offer a potential roadmap for improving the design and evaluation of multilevel interventions focused on the cancer care continuum. The methodological challenges will require ongoing investment in interdisciplinary mixed methods of research and evaluation, and greater emphasis on the training/education of a growing cadre of investigators and research teams skilled at building and bridging diverse partnerships, without which most implementation will not be systematically studied. Such an investment should pay dividends by increasing the number of levels effectively combined and examined, the quality of the evidence deployed at each level, and, ultimately the impact on routine practice and population health outcomes.
This work was coordinated by the National Cancer Institute at the National Institutes of Health. EMY's time was supported by the US Department of Veterans Affairs (VA) through the Veterans Health Administration Health Services Research & Development (HSR&D) Service through a Research Career Scientist award (05-195). JZA was supported by the National Cancer Institute (R01 CA112367).
The authors would like to thank and acknowledge the planning group in the Behavioral Research Program at the National Cancer Institute for their review of and input on multiple revisions of this monograph chapter. The authors are also grateful to Thomas D. Sequist and Eric C. Schneider for interviewing clinical leaders regarding the HVMA CRC screening interventions. The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the US Department of Veterans Affairs.