A total of 32 articles were identified [9
], which described 258 unique modifications (see Table ). The types of intervention included preventive and health promotion interventions (n
15), mental or behavioral health (n
13), behavioral medicine (n
3), and a multidimensional complex care coordination intervention (n
1). Settings in which the interventions were delivered included hospitals and medical clinics, mental health clinics, substance abuse treatment programs, human service organizations, housing shelters, community organizations, employment settings, bars, and schools. Twenty-three articles provided author descriptions of modifications, four identified modifications through interviews with providers, two utilized observation or fidelity rating, and three based findings on a combination of observation and interviews.
Articles included in coding procedure
Classification of modifications
Our coding process resulted in the identification of modifications to the context of program or intervention delivery, modifications to the intervention or program content itself, and modifications made during an implementation effort to training or evaluation processes. Furthermore, we included a code specifying who made the decision to make each modification. Figure represents the coding system that emerged from this process, which is described in greater detail below. A comprehensive coding manual that includes decision rules and instructions regarding how to code each level is available by request from the first author. Contextual modifications include format, setting, channel of delivery and intervention recipients, and are about ‘setting the stage’ for an intervention to be delivered. Content modifications focus on the actual delivery of the intervention content. Training and evaluation modifications represent changes made ‘behind the scenes’ during an implementation effort. Although modifications to context and training/evaluation codes were not always accompanied by substantial changes to the intervention content, we included them because it is possible that such changes could have an impact on fidelity, clinical outcomes, or the success of an implementation effort. Table includes the frequency with which each modification occurred, along with rater agreement statistics.
System of classifying modifications to evidence-based programs or interventions.
Modifications and adaptations made to programs and interventions
By whom was the decision to modify made?
This code indicates the individual or group of individuals who made the decision regarding whether or how to modify the intervention. Cohen’s kappa for this code was 0.80, indicating substantial agreement.
1. Provider, practitioner, or facilitator: The individual who delivers the intervention made the modification.
2. Team/multiple providers: A group of providers modified the treatment (e.g., either an intervention that requires multiple providers is modified by those providers, or a unit of providers decide together to deliver a program or intervention in a different way).
3. Administrator or supervisor: The individual responsible for oversight of an individual provider, team, unit, organization or system decided how to modify the intervention or program.
4. Researcher: A researcher determined how to modify a program or intervention for the purposes of research (e.g., to study the impact of a particular adaptation or set of adaptations).
5. Purveyor or intervention developer: The individual who developed the intervention or an (often external) individual with expertise in the intervention who was tasked with supporting the implementation determined how to adapt or modify the treatment. If the purveyor and researcher are the same individual, the coding decision is made based on whether the modification is for research or implementation purposes.
6. Coalition of Stakeholders: A group of stakeholders actively participated in the decision-making regarding the types of modifications that are made to an intervention. If the purveyor or researchers used focus groups, interviews, or other means of gathering input to guide their decisions regarding modifications, this code was NOT used, unless stakeholders also directly participated in the process of using that information to adapt the intervention.
Similar to Castro and colleagues’ description of differing forms of delivery [27
], contextual modifications were defined as changes made to delivery of the same program content, but with modifications to the format or channel, the setting or location in which the overall intervention is delivered, or the personnel who deliver the intervention. We also include in this category the population to which an intervention is delivered. Modifications were only coded as contextual if an intervention was specifically designed for a particular context or population and then applied elsewhere or delivered in a different format than originally designed. Modifications were considered to be contextual if one of the elements described below was changed, whether or not alterations to the content of the intervention were made. When content-level changes were also made, they were coded separately. A total of 41 contextual modifications (16% of the total sample of modifications) were described in the sample of articles. The subset of segments that was double-coded for reliability purposes indicated perfect agreement for the presence of contextual modifications.
1. Format: Changes are made to the format or channel of treatment delivery (e.g., a treatment originally designed to be used one-on-one that is now delivered in a group format).
2. Setting: The intervention is being delivered in a different setting or location (e.g., a treatment originally designed to be used in a mental health clinic setting that is now delivered in primary care).
3. Personnel: The intervention is being delivered by personnel with different characteristics (e.g., a treatment originally designed to be administered by a mental health professional is now delivered by clergy).
4. Population: An intervention that was specifically developed to target a particular population is being delivered to a different population than originally intended (e.g., an intervention developed for patients with Borderline Personality Disorder is now being delivered to individuals with Substance Dependence).
Modifications to training and evaluation processes
Changes made to the procedures for training personnel or evaluating the program are classified separately from content or contextual modifications, as they occur ‘behind the scenes’ and do not necessarily impact intervention content or the context of delivery. Examples include expanding training from a single day to a three-day workshop, or making changes to the type of evaluation data or procedures for collecting evaluation data.
Content modifications are changes made to the intervention procedures, materials or delivery. They appear to occur at multiple levels and in differing contexts, ranging from changes made for an individual recipient to changes made uniformly across an entire network, community or system. Therefore, we included a code for both the level at which the modification was made (e.g
., for a single patient vs. across the entire clinic), and the nature of the modification itself. A total of 217 content modifications (84% of all identified modifications) were described in the articles that were reviewed. Table summarizes the frequency with which modification occurred at the following levels. Cohen’s kappa for agreement on levels was 0.79, indicating substantial agreement [52
Levels at which content modifications occur
1. Individual recipient level: The intervention is modified for a particular recipient (e.g., simplifying language if a patient has cognitive impairment or if language barriers exist; changes to increase cultural relevance for an individual recipient).
2. Cohort level: The intervention is modified for individuals grouped within the intervention setting into a treatment group, a class, or other type of cohort (e.g., a specific psychotherapy group, grade or classroom).
3. Population level: The intervention is modified for application to a particular cultural, ethnic, clinical or social group (e.g., repetition of intervention components for all patients with cognitive impairments; development of culturally relevant vignettes to be used with all individuals of a particular ethnic identity).
4. Provider/facilitator level: Modifications are made by a clinician/facilitator for all of their participants (e.g., ‘I never set an agenda when I do cognitive therapy’).
5. Unit level: A modification is made by all of the facilitators in a unit (e.g., clinic/department/grade) within a larger organization (e.g., ‘We can only do 60-minute intervention sessions instead of 90-minute sessions in our clinic’).
6. Hospital/Organization level: Modifications are made by an entire organization.
7. Network/Community level: Modifications are applied by an entire network or system of hospitals/clinics/schools (e.g., a Veterans Affairs VISN; school district) or community.
Types of content modifications
We identified 12 different types of content modifications. Cohen’s kappa for the nature of modifications was 0.87, suggesting that rater agreement for these categories was in the ‘almost perfect’ range [52
]. Regarding reliability for individual codes, raw agreement was at least 80% for each code that was applied more than 15 times in our dataset; less frequently-applied codes were not subjected to reliability analyses.
1. Tailoring/tweaking/refining: This code was assigned to any minor change to the intervention that leaves all of the major intervention principles and techniques intact while making the intervention more appropriate, applicable or acceptable (e.g., modifying language, creating slightly different versions of handouts or homework assignments, cultural adaptations).
2. Adding elements (intervention modules or activities): Additional materials or activities are inserted that are consistent with the fundamentals of the intervention (e.g., adding role play exercises to a unit on assertiveness in a substance abuse prevention intervention).
3. Removing elements (removing/skipping intervention modules or components): Particular elements of the intervention are not included (e.g., leaving out a demonstration on condom use in an HIV prevention intervention for adolescents).
4. Shortening/condensing (pacing/timing): A shorter amount of time than prescribed is used to complete the intervention or intervention sessions (e.g., shorter spacing between sessions, or shortening sessions, offering fewer sessions, or going through particular modules or concepts more quickly without skipping material).
5. Lengthening/extending (pacing/timing): A longer amount of time than prescribed by the manual/protocol is spent to complete intervention or intervention sessions (e.g., greater spacing between sessions, longer sessions, more sessions, or spending more time on one or more modules/activities or concepts).
6. Substituting elements: A module or activity is replaced with something that is different in substance (e.g., replacing a module on condoms with one on abstinence in an HIV prevention program).
7. Re-ordering elements: Modules/activities or concepts are completed in a different order from what is recommended in the manual/protocol. This code would not be applied if the protocol allows flexibility in the order in which specific modules or interventions occur.
8. Integrating another approach into the intervention: The intervention of interest is used as the starting point, but aspects of different therapeutic approaches or interventions are also used (e.g., integrating an ‘empty chair’ exercise into a ‘CBT for Depression’ treatment protocol).
9. Integrating the intervention into another approach: Another intervention is used as the starting point, but elements of the intervention of interest are introduced (e.g., integrating motivational enhancement strategies into a weight loss intervention protocol).
10. Repeating elements: One or more modules, sessions, or activities that are normally prescribed or conducted once during a protocol are used more than once.
11. Loosening structure: Elements intended to structure intervention sessions do not occur as prescribed in the manual/protocol (e.g., the ‘check-in’ at the beginning of a group intervention is less formally structured; clinician does not follow an agenda that was established at the beginning of the session).
12. Departing from the intervention (‘drift’): The intervention is not used in a particular situation or the intervention is stopped, whether this stoppage was for part of a session or a decision to discontinue the intervention altogether (e.g., ‘this client was so upset that I just spent the rest of today’s session letting him talk about it instead of addressing his health behaviors’).