|Home | About | Journals | Submit | Contact Us | Français|
Theories and frameworks (hereafter called models) enhance dissemination and implementation (D&I) research by making the spread of evidence-based interventions more likely. This work organizes and synthesizes these models by: (1) developing an inventory of models used in D&I research; (2) synthesizing this information; and (3) providing guidance on how to select a model to inform study design and execution.
This review began with commonly cited models and model developers and used snowball sampling to collect models developed in any year from journal articles, presentations, and books. All models were analyzed and categorized in 2011 based on three author-defined variables: construct flexibility, focus on dissemination and/or implementation activities (D/I), and the socio-ecological framework (SEF) level. Five-point scales were used to rate construct flexibility from broad to operational and D/I activities from dissemination-focused to implementation-focused. All SEF levels (system, community, organization, and individual) applicable to a model were also extracted. Models that addressed policy activities were noted.
Sixty-one models were included in this review. Each of the five categories in the construct flexibility and D/I scales had/contained at least four models. Models were distributed across all levels of the SEF; the fewest models (n=8) addressed policy activities. To assist researchers in selecting and utilizing a model throughout the research process, the authors present and explain examples of how models have been used.
These findings may enable researchers to better identify and select models to inform their D&I work.
Vast resources are invested in the development of interventions to prevent and treat disease; however, only a fraction of research products is translated to practice and policy in order to affect population health.1–3 Dissemination and implementation (D&I) science seeks to understand how to systematically facilitate deployment and utilization of evidence-based approaches to improve the quality and effectiveness of health promotion, health services, and health care.4 Although this discussion is framed largely around health fields, much of the work in D&I stems from other industries and disciplines. As the field of D&I research grows, the number of existing theories and frameworks informing this research continues to expand.
Although theories and frameworks are often presented as synonymous, they are distinct concepts. Theories present a systematic way of understanding events or behaviors by providing inter-related concepts, definitions, and propositions that explain or predict events by specifying relationships among variables.5 Moreover, theories are abstract, broadly applicable and not content- or topic-specific.5 On the other hand, frameworks are strategic or action-planning models that provide a systematic way to develop, manage, and evaluate interventions.6 Despite their differences, theories and frameworks both enhance effectiveness of interventions by helping to focus interventions on the essential processes of behavioral change, which can be quite complex.5,7–10 For example, public health interventions that utilize health behavior theories, such as social cognitive theory and the theory of planned behavior, are more effective than interventions without a theoretic base.5,10
The importance of theories and frameworks in other areas of research (e.g., individual-level, behavioral intervention) suggests that success in D&I research will also benefit from the use of theories and frameworks. This is supported by research that demonstrates that the use of theories and frameworks in D&I research enhances interpretability of study findings and ensures that essential implementation strategies are included.11–13 For simplicity, the current paper refers to theories and frameworks (both of which are important for D&I research) collectively as models.
The roots of D&I research cut across many disciplines, including agriculture, medicine, public health, organizational behavior, psychology, political science, and marketing. The field has grown and changed since its origins several decades ago.14 The mounting interest in transdisciplinary research and increasing ease of information-sharing encourages collaboration of these diverse, but inter-related specialties. Since 2004, at least ten peer-reviewed journals across a range of scientific disciplines have devoted special issues or sections to the topic of dissemination or implementation of evidence-based practices.15 Due to the interdisciplinary nature of D&I research, there is a need to collect, organize, and synthesize the many models used to integrate evidence-based interventions and healthcare information into practice.
This paper seeks to further D&I science by providing a narrative review of models used in D&I research. D&I science is notably different from the simple dissemination of research findings that occurs at the end of a study (e.g., a press release, an issue brief, a peer-reviewed publication). Instead, D&I science seeks to investigate and better understand the complex task of spreading ideas across multiple levels of the socio-ecological framework (SEF), which may include groups at the organizational and community levels.
Models were aggregated from published literature and scientific presentations. To facilitate selection of the most-appropriate model to inform D&I study design and execution by researchers, these models are organized based on: the flexibility of a model’s constructs; whether the model is more focused on dissemination and/or implementation; and the socio-ecological level to which a model is applicable (system, community, organization, or individual); as well as whether or not the model addresses policy creation or use. Additionally, case studies are included to illustrate how models can be used to inform D&I research.
Dissemination and implementation research is described using several terms, many of which are used interchangeably; for example: knowledge translation, knowledge exchange, and knowledge utilization.16 The diverse range of disciplines contributing models to D&I research leads to a tremendously wide range of sources. These factors prohibited establishing a scope for this review that would comply with traditional systematic review guidelines. Therefore, a narrative approach was determined to be most appropriate for this review. Narrative reviews are useful for summarizing studies and describing “what we know,” informed by reviewers’ experiences and existing theories.17,18 The authors’ aim was to capture and carefully review a large number of existing models within the D&I field. This was accomplished through an approach divided into several phases: initial sampling; snowball sampling from the initial sample; consulting with experts; identifying categories into which models could be placed; arranging the models based on the categories; and contacting a subset of model developers to ensure that the categories were valid.
Without consensus terminology in D&I research, the starting point for the narrative review was determined by two of the study authors, who generated a list of commonly used models and model developers. Snowball sampling was then used to identify new articles through existing reviews, reference lists, and presentations delivered by the authors and available online. The search was not exhaustive but did attempt to identify every model. To ensure comprehensiveness, U.S. NIH officials who advise researchers submitting grant proposals for D&I research were queried for additional models.
Models published in peer-reviewed and non–peer reviewed sources in this review are from many disciplines including innovation, organizational behavior, and research utilization. Several criteria were used to define the scope of models included in this review. The following parameters were informed by two of the study authors, who are experts in the field, and were developed to provide a succinct list of models to D&I researchers that would be of the highest value.
The first criterion was that the model be designed for use by researchers, in contrast to practitioners or clinicians. Although the distinction between researchers and practitioners is ambiguous, researchers have been described as “knowledge creators,” and practitioners have been described as those applying knowledge in service.19 The second criterion was that the model be applicable to local-level dissemination targeting communities and organizations. Thus, models that applied only to national-level plans were excluded; these were models for which the unit of dissemination or implementation would be at a national level (e.g., a country’s dissemination plan).
The authors also excluded models that applied to only individual behavior change with no application to community- or organization-level dissemination. Since this review focuses on models for D&I research, models designed to assist only in the dissemination that occurs at the end of a research study were also excluded. Lastly, the included publications were limited to those written in English. As narrative reviews are best conducted by a team,18 two of the authors reviewed publications as well as reports of D&I research. The authors convened regular meetings to discuss the categorization and inclusion/exclusion of models.
In the process of reviewing the models, several groupings emerged. Therefore, to assist researchers in selecting a model, three author-defined variables were used to categorize the models: construct flexibility, focus on dissemination and/or implementation activities (D/I), and SEF level (Table 1). First, models were categorized based on their construct flexibility on a 1–5 scale, where 1=broad and 5=operational. Models falling between these categories were scored as 2, 3, or 4.
Broad models are those that contain constructs that are more loosely outlined/defined, thereby allowing researchers greater flexibility to apply the model to a wide array of D&I activities and contexts. This also places more responsibility on the researcher to carefully think through how to operationalize, implement, and use the model. Operational models provide detailed, step-by-step actions for completion of D&I research processes. These are clearly defined for a particular context and activity. Models between these two extremes contain constructs that are more detailed than broad models but not as detailed as operational models. This made the models less flexible across all contexts, but more conducive to visualizing how the model may assist with study design.
To further facilitate selection, models were also categorized on a continuum from dissemination to implementation. Dissemination is the active approach of spreading evidence-based interventions to the target audience via determined channels using planned strategies. Implementation is the process of putting to use or integrating evidence-based interventions within a setting.20 Models informing D&I research fall along the spectrum from dissemination to implementation. Therefore, models were split into five categories: models that focused entirely on dissemination (D-only); dissemination more than implementation (D>I); both activities equally (D=I); implementation more than dissemination (I>D); and only on implementation (I only).
The last variable used to classify these models was the level of the SEF at which the model operates. The use of the SEF recognizes that D&I strategies may focus on changing behavior at a specific level (e.g., clinician, organization) or may cut across multiple levels. Therefore, it is important for future use of models to identify the level at which each model operates. Models were assigned as many SEF levels as were applicable, including individual, organization, community, and system. Models addressing policy, such as policy use and creation of policy, were also labeled as such.
Based on these three categories, models were classified by two independent reviewers. Initial agreement for categorization of models along the spectrum from dissemination to implementation was 84% (Kappa coefficient=0.79). Initial agreement for the construct flexibility scale was considerably lower: 43% (Kappa coefficient=0.25). These categorizations were discussed by the independent reviewers, and discrepancies were resolved via consensus. To ensure that models were accurately described and that definitions were clear to experts in the field, a sample of model developers were contacted and presented with the category definitions and assignment for the model they developed. Further, all model developers for whom contact information could be identified were contacted to assure that the models presented below have an accurate name and all appropriate citations.
After finalizing the list of models and their categorization (Table 2), additional information about the model was abstracted: the original field in which the model was developed, the number of times the original publication has been cited, and a subset of studies, if any, that used the model to inform their design. The field of origin was ascertained by determining the model developers’ stated intended use for the model. Google Scholar was used to determine the number of times the original publication had been cited. Articles identified by Google Scholar as citing the model were abstracted to identify studies in which researchers had used the model to inform the study design. The model’s field of origin, the number of articles that cite the model, and studies that use the model are included in Appendix A (available online at www.ajpmonline.org).
Five examples of model use, selected to represent a broad range of fields, are described in greater detail within this work. As models can be applied retrospectively to inform an evaluation or prospectively to inform study design, examples of both types of model applications are provided. One example, or case study, is provided here (Figure 1), with the remaining four available in Appendix B (available online at www.ajpmonline.org). Each case study provides background about the model; how the model was applied to the specific research setting; and when possible, information related to construct measurement.
From a total of 109 models, 26 were excluded due to a focus on practitioners, rather than researchers; 12 were excluded because they were not applicable to local-level dissemination (communities or organizations); and eight were excluded because they focused on dissemination at the end of a research study rather than D&I research. Two models were identified as duplicates, and combined for inclusion. A total of 61 models were included in this review. A complete list of the models, including all three types of categorization can be found in Table 2. This table also includes the original reference for the model as well as references to publications updating the model. The models in Table 3 are organized first by classification along the D/I continuum, then by construct flexibility. Appendix A (available online at www.ajpmonline.org) provides additional information about the field of origin of each model, the number of times a model was cited, and studies that use the model (where available).
Table 2 shows that each of the five categories within the construct flexibility variable was assigned to at least four models, with the greatest number of models (25 models) categorized as three. Similarly, each of the five categories within the D/I variable was assigned to at least five models, with slight skewing towards the dissemination end of the D/I continuum. Models were distributed across all levels of the SEF, with an emphasis on the community (52 models) and organization (59 models) levels. In addition, eight models addressed policy activities.
The models are presented in Table 3 based on their classification in two categories: construct flexibility and D/I. When these two categories were cross-tabulated, a number of findings are apparent. Models with a greater emphasis on implementation tended to have constructs that are more operational. In contrast, there was a greater quantity and variety of dissemination-focused models (D-only, D>I). Of note, broad models were identified only for D-only or D=I activities. It is important to acknowledge that while these models are presented as being distinct from each other, many of the models evolved from and/or were informed by other models. Thus, although the models were divided into discrete categories, the differences among models are much more fluid.
The case studies presented in Figure 1 and Appendixes B–E explain some models in greater detail, discuss how each model was applied to the specific research setting, and, when possible, provide information on measurement. (Note that only two of the studies, shown in Figure 1 and Appendix E, included measures.) The five case studies show the diversity present in D&I research. Within this handful of examples, the fields of study represented include: obesity policy, substance use disorder treatment, and teen pregnancy prevention (Appendixes B, C, and E, respectively, available online at www.ajpmonline.org). Further, the case studies demonstrate the many ways that a model can be applied. In three cases, a model was retrospectively applied to evaluate an existing intervention, wehereas in two cases, the researchers prospectively applied models to design an intervention. Further, Appendix A (available online at www.ajpmonline.org) provides references for studies that use a given model, where they could be identified.
The importance of using models in D&I studies cannot be overstated. Use of models not only makes a study more likely to be successful, but if an existing model is used, this application also contributes to the literature on a particular model and enables continued distillation and better understanding of model constructs.10–13 This paper presents 61 existing models (as well as information regarding the settings and approaches to which these models are suited) to assist researchers seeking to utilize an existing model to inform their work. Although some D&I models are likely missing from this review, the models presented in Table 2 represent the entire spectrum in the construct flexibility, D/I, and SEF categories. At least four models are in each of the five D/I and construct-flexibility groups. Table 3 displays the diversity of the models and suggests the need for guidance on using the information presented in this review. Issues to consider when using Tables 2 and and33 to inform the design of a D&I study are presented below.
The first consideration is the decision to use an existing model or develop an entirely new model. As the number of models presented in this review shows, researchers can choose from a wealth of existing models. There are many benefits to using an existing model. It encourages researchers to build on previous findings. Demonstrating a new application of the model increases the generalizability of the model thereby enhancing the field’s understanding of a model and its constructs.
Since D&I research crosses numerous disciplines, finding the right fit between research needs in a particular field and existing models can be a challenge. It is possible that no existing model is well suited for a given field. In these cases, the researcher can choose to develop a new model or adapt an existing model. As this review identified 61 models, any researcher considering developing a new model should note the considerable overlap between existing models and document that the new model truly addresses a gap in the literature. Based on face validity and expert experience, when adaptation of an existing model is considered, it is essential to review the goal, setting, population, and other contextual conditions for which the model was originally developed.110 The process for selecting and using or adapting an existing model is described below.
By classifying the models using three categories (construct-flexibility, D/I, and SEF), the authors sought to provide useful information to aid in the selection of an appropriate model for a D&I study. For scientists new to D&I research, who may need additional support in designing their study, the construct-flexibility variable may assist in selecting models that will provide additional guidance. Researchers that are considering a study that targets system, communities, organizations, and/or individual level changes may select models that include applications at those levels. Studies that are aimed at the entire dissemination-to-implementation spectrum can be informed by models that address both dissemination and implementation research. Lastly, researchers with interest in policy-related D&I issues may also identify models that will assist with their thinking on policy.
The inclusion in this review of the field of origin of each model provides D&I researchers additional information when selecting a model. The innovation of a research study can be enhanced by utilizing models originally developed in different disciplines, but which may be well suited to an alternative field. This also prevents duplication of models across disciplines.
The authors believe that the provided information will improve the process of selecting an appropriate model for a D&I study. By using Table 3, based on the considerations described above, researchers can identify a list of models most appropriate for their study. If necessary, the list of potential models can be further refined by using additional information (such as SEF and field of origin) found in Table 2 and Appendix A (available online at www.ajpmonline.org). To envision how a model can be used in their research study, researchers can look to the articles that describe the model as well as studies that have used the models; these papers should provide guidance on how exactly the model is used and the availability of measures for the model’s constructs.
Selection of a model should occur as part of study planning and design. Once the appropriate model has been selected, it should be applied throughout the study. Several resources, including the Veteran Affairs’ Quality Enhancement Research Initiative111; the National Cancer Institute’s Implementation Science Team112; Training Institute for Dissemination and Implementation Research in Health113; and the Canadian Knowledge Translation Clearinghouse114 websites provide more-detailed guidance on how to use a selected model to inform a D&I study.
In general, the model should be considered in a study’s design, aims, activities, methods, measures and evaluation. Models can be used directly or after some modification to make them more appropriate for the study. If using the model directly, with minimal adaptation, it is important to ensure that the model is appropriate for the proposed intervention and cultural preferences of the target population. Use of a model primarily implies conversion of the model into measurable components. This allows researchers to quantify mediators, moderators, and outcomes.20, 115, 116 This is easier when measures that capture the specific model constructs are available. Unfortunately, as discussed below, available measures are often lacking.
A researcher will almost always adapt a model in some way; therefore, adaptation is often an important part of using a model. Adaptation often improves the appropriateness of the selected model to the intervention being disseminated or implemented, the population, and the setting.117 Further, adaptation contributes to the field by testing modifications to existing models, such as disregarding pieces shown to be ineffective or adding ones with additional evidence. Models should be viewed as living documents, or works in progress, not as static entities.
For researchers considering adapting an existing model, a number of issues are important to note. Initial identification of a D&I model to adapt should consider factors that influence the fit of a model such as the target population and/or setting (sociodemographics, geography, language, and culture) and the technology and resources needed for intervention delivery (e.g., high-speed Internet connection, media skills). In making adaptations, several types are possible.
Modifications that can be made without much hesitation include: wording to suit the audience, timeline (based on adaptation guides), or cultural preferences based on the population. Adaptations that may be possible, but should be made with caution, include: substituting activities or changing the order of the steps. Adaptations that compromise the core elements of the model should not be attempted without substantial evidence to support the adaptation. This includes changing the health communication model/theory or the health topic/behavior; deleting core elements; or putting in strategies that detract from the core elements. As long as model adaptations do not become a weakness of the proposed study, when drastic changes are made to a model, it provides an excellent opportunity for model testing. In studies that adapt a model, adaptations should be documented and monitored so that the impact of changes on model applicability can be reported and incorporated into the literature.
A particularly important aspect to consider in model-informed studies is the availability of measures to assess a model’s constructs. Without measures, it is impossible to operationalize a model and conduct D&I research. As a developing field, many constructs are currently assessed as open-ended questions (or not assessed at all) because standard measures are lacking.
In addition, the small sample size of many studies prevents the development, evaluation, and use of standard measures. This difficulty is discussed by a number of authors. Damschroder et al. lay out common, overlapping constructs, which are found in many models, and note that reliable and valid measures to assess these common constructs, regardless of the model, would enhance the rigor of D&I research.104 Chamberlain et al. also discuss elements outside the constructs of the individual models that should be measured.118 Use of meta-analysis to enhance D&I measures has been inhibited by weaknesses in information about outcomes, use of dichotomous measures, and unit of analysis.119,120
Given the complexity of the issue of measurement, the authors attempted to provide examples of measurement use in the case studies. Unfortunately, only two of the case studies provide a detailed discussion of measures; this illustrates the difficulty of construct measurement. Readers can refer to the two specific case studies (Figure 1; and Appendix E, available online at www.ajpmonline.org) for a more detailed discussion of how to measure constructs. Although there are few published studies that discuss in detail the use of construct measures, two new, increasingly important resources for researchers looking for relevant measures are: the Seattle Implementation Research Conference Measures Project121 and the Grid-Enabled Measures developed by the National Cancer Institute,122 both of which are initiatives to compile, enhance, and help harmonize D&I measures.
The models described in this review have been organized using a number of categories. These divisions are intended to assist the reader in model selection, rather than to provide actual classifications for models. There is substantial overlap between models, as the included constructs are often similar. This may be due to the similarity of the theoretic underpinnings (such as organizational theory, diffusion of innovation theory, and political science theory), which broadly inform D&I research.123,124 These common theoretic foundations come from many fields, provide overarching roots for many models, and further emphasize the transdisciplinary nature of the field.
This study is strengthened by the face validity and reliability provided by model-developer agreement on the categorization of the models they developed for a subset of models. Further, receiving input from project officers at the NIH, who guide D&I researchers on model selection, ensures that the most commonly recommended models were considered by this review. Contacting all available model developers to ensure that the correct model names, original citations, and updated citations were included increases confidence in the findings. Finally, this review drew from models being used across the many disciplines conducting D&I research and will facilitate innovative, transdisciplinary use of models by D&I scientists.
Since this is not a systematic review, it is impossible to ensure all available models were included. As mentioned above, the lack of terminology in the D&I research field as well as the diverse range of disciplines contributing models to D&I research made this type of search prohibitively broad in scope. Further, it is likely that models from fields outside of health, such as education, business, and political science, may have been missed or under-represented. In addition, as it is difficult to measure the use of models in grant applications and unpublished research projects, the citation number for each model provided in Appendix A (available online at www.ajpmonline.org) can serve as only a proxy for the popularity and use of any given model. Finally, only models published in English were included.
The current review suggests that much work remains to be done in the field of D&I research. These findings need to be spread to not only D&I researchers but also scientists who are less versed in D&I research. Nonresearchers would also benefit from this knowledge, so they become aware of D&I science as a field and how D&I researchers can help them deliver the best care to those they serve. As it was beyond the scope of this review to include models targeted at practitioners, such models should be similarly inventoried and synthesized. As mentioned above, the science of D&I research is severely limited by the lack of measures available to assess the constructs in the included models; future studies in this area should work to review and compile available measures and identify gaps. There is also a lack of consistency in the terminology used to discuss this type of work. Rabin et al. have created a glossary of terms to clarify this discussion, and consistent use of language would help the field as it moves forward.20
An additional characteristic to assess in future research is whether a model is designed to guide D&I intervention development, evaluate interventions, or both guide and evaluate efforts. Further directions for considerations in evidence-based decision-making may look to less-traditional methods such as dynamic simulation to inform implementation decision-making, as suggested by Hvitfeldt Forsberg et al.125 This is a truly transdisciplinary work, which charges researchers with the task of working across fields; this can bring benefits and challenges, both of which must be tackled as the field of D&I research continues to grow.
The authors are grateful to numerous model developers who commented on their models and the variables used for classification. They also appreciate the feedback of the Washington University Network for Dissemination and Implementation Research (WUNDIR).
This project was funded in part by cooperative agreement number U48/DP001903 from the CDC, Prevention Research Centers Program and Grant 1R01CA124404-01 from the National Cancer Institute at the NIH. It was also supported in part by the National Center for Research Resources and the National Center for Advancing Translational Sciences, NIH, through Grant TL1RR024995 and UL1RR024992. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH.
No financial disclosures were reported by the authors of this paper.
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.