|Home | About | Journals | Submit | Contact Us | Français|
This article describes a program of research on effectively transporting the New Beginnings Program (NBP), a university-tested prevention program for divorced families, to community settings. The status of four steps in this research are described: (1) Selecting a community partner; (2) Developing effective methods of engaging parents; (3) Redesigning the NBP to be easily delivered with high quality and fidelity in community agencies, and (4) Adapting the NBP to meet the needs of the full population of divorcing families. The article concludes with a discussion of plans for an effectiveness trial to evaluate the NBP when delivered in community settings.
Parental divorce is one of the most prevalent adversities experienced by adults and children in the U.S. Each year, over 1.5 million youth experience parental divorce (US Census Bureau, 1999). Currently, it is estimated that 34% of children in the U.S. will experience parental divorce before reaching age 16 (Bumpass & Lu, 2000). The high prevalence of divorce and the fact that parental divorce is a risk factor for multiple mental health and substance abuse problems means that its impact on population rates of problem outcomes in youth is substantial (Scott et al., 1999). Thus, the widespread implementation of effective preventive interventions for children from divorced families could have significant public health benefits.
Although several interventions have been rigorously evaluated and shown to improve short-term outcomes for children in divorced families (e.g., Forgatch & DeGarmo, 1999; Pedro-Carroll & Cowen, 1985; Wolchik et al., 2000), these interventions are not widely available to the population of divorcing families. In this article, we focus on the New Beginnings Program (NBP), a theory-based preventive intervention targeting parenting skills which has been shown to improve multiple youth outcomes in two efficacy trials. The purpose of efficacy trials is to evaluate the benefits of an intervention when delivered under tightly controlled conditions. In efficacy trials, multiple steps are taken to ensure high quality delivery of the program to strengthen inferences about program effects. These steps include providing detailed descriptions of each program session, conducting extensive training of leaders, and assessing whether and how well the leaders complete the program activities. In addition, multiple eligibility criteria are used, such as parents and children not being involved in other interventions, an assurance that the family will remain in the geographical area where the study is conducted throughout the planned assessments, and families having children in a particular age range. Prior to widespread dissemination of programs that have been shown to have positive effects in efficacy trials, it is critical to conduct effectiveness trials of these programs. Effectiveness trials differ from efficacy trials in that the targeted population is defined as those served by the natural service delivery system. Also, program delivery is typically more variable in effectiveness trials than it is in efficacy trials (Kellam & Langevin, 2003). Research on other programs has shown that their implementation in community settings typically deviates markedly from that in the efficacy trials (e.g., Gottfredson & Gottfredson, 2002). Also, research has consistently shown that fidelity of implementation, which is typically defined as the proportion of program content delivered, and quality of implementation, which refers to the clinical and interactive skills providers use to convey the program information, are positively related to program effects (e.g., Mayer & Davidson, 2000). Effectively transporting interventions evaluated in efficacy trials into community settings requires understanding aspects of these settings that present barriers to high levels of fidelity and quality of implementation.
The goal of the program of research described here is to proactively identify such barriers and develop strategies to maximize the likelihood that the NBP will be delivered with high levels of fidelity and quality in natural service delivery systems. Below, we describe four steps in this research: (1) Selecting a community partner; (2) Developing effective methods of engaging parents; (3) Redesigning the NBP so it is easy to deliver with high quality and fidelity in community settings, and (4) Adapting the NBP to meet the needs of the full population of divorcing families.1 Before describing the status of our work in each area, we briefly describe the conceptual framework underlying the NBP, its format and content and its short-term and long-term effects on youth outcomes. We conclude with a brief discussion of plans for an effectiveness trial to evaluate the NBP when delivered in community settings.
The conceptual model underlying the NBP combines elements from a person-environment transactional framework and a risk and protective factor model. Person-environment transactional models posit dynamic person-environment processes underlying development across time. In this model, changes in aspects of the social environment (e.g., parenting) are viewed as affecting the development of problems and competencies in an individual, which in turn influence the social environment and development of competencies and problems at later developmental stages (e.g., Cicchetti & Schneider-Rosen, 1986; Sameroff, 1975; 2000). Derived from epidemiology (Institute of Medicine, 1994), the risk and protective factor model posits that the likelihood of mental health problems is affected by exposure to risk factors and the availability of protective resources. Using these frameworks, we reviewed the research available in the mid 80’s, the period during which we developed NBP, to identify modifiable, empirically-supported correlates of children’s adaptation outcomes after divorce. This research provided support for the following social-environmental factors being related to better post-divorce adjustment: high quality of the custodial parent-child relationship, use of effective discipline, more contact between the child and noncustodial parent, and more contact with/support from non-parental adults. The research also consistently showed that interparental conflict was related to poorer post-divorce adjustment. See Wolchik et al. (1993) for more information on the empirical support for these risk and protective factors. The research published since the mid 80’s has provided additional support for associations between these factors and children’s post-divorce adaptation.
The NBP has been evaluated in two randomized efficacy trials conducted in a university setting (Wolchik et al., 1993; 2000; 2002). In both trials, the program was delivered to mothers with primary residential custody. The NBP was delivered in 11 weekly group sessions (1.75 hours each) and two individual sessions (1 hour each) designed to tailor the program skills to the family’s needs. Two Masters’ level counselors co-led each group, which consisted of 8 to 10 mothers. To ensure high fidelity of implementation, leaders received extensive training and ongoing supervision, sessions were delivered using detailed manuals and all sessions were videotaped. The orientation of the program was cognitive-behavioral; there was a strong emphasis on skills acquisition and enhancement. All sessions were highly structured and included both didactic and experiential components. Weekly home practice assignments were given in which mothers practiced the program skills with their children.
The content of the program was highly similar in both trials. About three-fourths of the sessions focused on the two modifiable factors most likely to improve when working only with the mother: mother-child relationship quality and effective discipline. One session each was devoted to interparental conflict, father-child contact and maintenance of program skills. In the first trial, an increase in the amount and quality of support children received from nonparental adults (e.g., aunts) was targeted. This component was dropped in the second trial to allow for an expanded discipline component. More information about the program and intervention strategies is provided in Wolchik et al. (2007).
The first efficacy trial (N = 70) was a randomized experimental trial that used an intervention versus wait list control design In a wait-list control design, participants assigned to the intervention condition receive the program shortly after all participants have completed the pre-test and those assigned to the control condition receive the program after the first group of participants have completed the program and all participants have completed the post-test assessment. The second efficacy trial (N = 240) was designed to assess whether the positive program effects achieved in the initial trial could be replicated, whether these effects persisted over time, and whether the addition of a children’s group enhanced the program effects beyond that of the mother program. In this trial, families were randomly assigned to one of three experimental conditions: the mother-only program, the mother plus child program (concurrent, separate groups for mothers and children), and a literature control group. The children’s group focused on increasing effective coping, reducing negative thoughts about divorce stressors, and improving mother-child relationship quality. To ensure that the materials and information were developmentally appropriate, families were invited to participate in the trial only if there was at least one child in the age range of 9 to 12. Mothers and children in the literature control condition were sent books on divorce adjustment and syllabi to guide their reading.
In both of these efficacy trials, random sampling of court records of divorce decrees involving children was the primary method of identifying possible families. Letters and phone calls (and in the second trial, in-home visits) were used to recruit families. Multiple eligibility criteria were used, such as mother had not remarried or did not have a live-in boyfriend, and neither the mother nor child was in treatment for psychological problems. Nearly all the mothers were Caucasian (90% and 88% in the first and second trials, respectively); three-fourths of the mothers in the first trial and half in the second trial had completed college or taken some college courses. See Wolchik et al. (1993, 2000) for more information about recruitment, eligibility criteria, and sample characteristics.
Analyses in both of these trials indicated that participation in NBP significantly reduced child behavior problems at posttest compared to the control condition (Wolchik et al., 1993; Wolchik et al., 2000). Also, positive program effects occurred for mother-child relationship quality and effective discipline. The program had the greatest benefit for families with poorer baseline functioning. The 6-year follow-up of families in the second trial showed positive program effects on a wide range of youth outcomes, including internalizing problems, externalizing problems, mental disorder symptoms and diagnosis, alcohol use, marijuana use, other drug use, polydrug use, number of sexual partners, grade point average (GPA) and self-esteem. Illustratively, 23.5% of adolescents in the control group had a mental disorder in the past year versus 14.8% of those in the NBP. For several of these effects, program benefits were greater for youth with higher baseline risk (Dawson-McClure et al., 2004; Wolchik et al. 2002). Comparisons of the mother versus mother plus child conditions showed that adding the child group did not increase program benefits on mental health outcomes beyond the mother program alone (Wolchik et al., 2000; 2007) at post-test, short-term follow-up or six-year follow-up. Thus, our work on transporting the NBP into community settings focuses on the mother program. Mediation analyses at post-test, short term follow-up and at six-year follow-up consistently found that program effects on youth mental health problems were partially accounted for by program-induced improvements in mother-child relationship quality and effective discipline; the program effect on GPA was mediated by effective discipline (Tein et al., 2004; Zhou, Sandler, Millsap & Wolchik, 2008).
On the basis of these positive short-term and long-term program effects, we have taken steps toward the goal of providing the NBP as on ongoing service for divorced families. Below, we discuss four aspects of this work. For each issue, we describe the data we have collected from multiple stakeholder groups and their implications for the planned effectiveness trial.
We view the family court as an ideal institution for facilitating the widespread delivery of prevention programs for youth in divorced families for several reasons. First, virtually all divorcing families come into the court’s purview. Second, court procedures (e.g., filings for separation, mandated attendance at court services) ensure that all families are potentially reachable during the process of obtaining a divorce. Third, the family courts have considerable authority to affect parents’ and children’s lives (Braver, Cookston, & Cohen, 2002). Fourth, family courts have developed an array of non-judicial programs for divorcing families (Sigal, Sandler & Wolchik, 2008), including parent education programs. In some jurisdictions, such programs are mandated for all divorcing families or only for contested or high conflict cases. In other jurisdictions, parents seek these services voluntarily. Because few courts have the capacity to provide multi-session programs in house, we see the role of the family court as providing information about the availability and potential benefits of the NBP and encouraging attendance for those who are interested. The NBP will be offered by mental health agencies that contract with the courts.
To assess interest among family courts in providing evidence-based, multi-session parenting programs, we collected data from an important group of stakeholders, key informants in charge of parent education programs in the courts (Cookston et al., 2002). In addition to interest, we assessed readiness to adopt programs like the NBP and barriers to implementation of such programs. This survey used a stratified random sample of 154 courts that were most likely to have an interest in the NBP, those delivering short parent education programs. Although only 12.6% of these courts offered multi-session programs similar to NBP, 95% stated that multi-session, evidence-based programs would be helpful to families and 74% believed that such programs should be offered in their court. Further, most respondents believed that providing these services was the court’s responsibility (69%). Respondents predicted that other key stakeholders would favor such programs, including judges (77%), bar members (73%), county supervisors (71%), and state Supreme Court members (87%). About 60% reported that it was likely that their court would adopt an evidence-based multi-session program such as the NBP for custodial, as well as non-custodial parents, and 73% felt that courts would find funding for such programs. Respondents consistently identified two barriers to adopting such programs--funding and attendance--with 71% having concerns about funding and 53% noting that attendance would likely be low.
The data clearly indicate that family courts see themselves as an appropriate setting for offering programs like the NBP. Further, most courts believe that it is their responsibility to provide such programs, and that courts would find funds for these programs. However, this group of stakeholders identified two potentially significant barriers to adoption of the NBP: funding and attendance. To be feasible as an ongoing service, delivery costs of the NBP need to be affordable and methods of engaging parents in program participation that are effective and affordable need to be developed.
In the experimental efficacy trials of the NBP, intensive, expensive engagement methods resulted in 43% of the eligible families enrolling in the program. Although this rate is higher thanthe 20 to 25% observed for manypreventive family programs (Spoth & Redmond, 2000), it places significant limits on the potential population level impact of the NBP (Braver & Smith, 1996). To maximize the public health impact of NBP, engagement approaches that reach a large percentage of the population of divorcing families and are both effective and affordable for courts need to be developed. Because many courts have short (approximately two- to four-hour) parent education classes for divorcing parents with minor children, we plan to use these programs as our major mechanism of engagement.
Most of these courses are available to all divorcing parents but some courts offer short parent education courses only to parents with contested or high conflict divorces. Further, some jurisdictions mandate these courses for all divorcing parents whereas other jurisdictions mandate them only for contested cases or high conflict families or offer them on a voluntary basis. In our work on the NBP, we are particularly but not only interested in recruiting the high risk families who attend these classes for two reasons. First, although parental divorce increases risk for significant mental health problems, the majority of children do not experience serious adjustment problems after divorce (e.g., Hetherington et al., 1998). Second, in both evaluations of the NBP, children at highest risk of developing subsequent mental health problems benefited most from the program(Wolchik et al., 1993; 2000; 2007).
We are currently conducting a study in collaboration with the Maricopa Courts to compare five engagement methods that were utilized in brief state-mandated parent education classes. In all conditions, prior to presentation of information about the NBP, parents completed a 15-item questionnaire assessing their children’s level of risk (e.g., child and parent mental health problems, interparental conflict). Condition 1 of this study consisted of giving the parents a brochure describing features of the NBP (e.g., skills taught, group format) and the positive program effects on youth outcomes found in the efficacy trials (e.g., less aggression; higher GPA). Condition 2 included a DVD presentation (about 12 minutes) covering the same material in the brochure, most of which was presented through an interview with one of the NBP’s co-developers.
Conditions 3, 4 and 5 used social influence principles to increase motivation to participate (Cialdini, 2001). The DVD shown to participants in condition 3 used the principle that people are more likely to comply with requests or offers for a service if the service has been socially validated by others who have participated, it is endorsed by legitimate authority figures, or both (Cialdini, 2001). Thus, testimonials from past NBP participants and group leaders and headlines and quotes from newspaper articles about the NBP were included in the DVD. Information was also provided about the potential negative effects of divorce on children’s outcomes, how the NBP can help reduce children’s risk of problems after divorce, and features of the NBP.
In condition 4, Cialdini’s (2001) examined participation procedure was added to the DVD used in Condition 3. This procedure is based on the social influence principle that people are more likely to take advantage of opportunities that are consistent with their prior commitments and goals (Cialdini, 2001). The examined participation procedure was designed to show parents how NBP coincides with the goals they have for their children. The DVD guided parents through this self-assessment procedure, in which parents reviewed a list of common concerns about children after divorce (e.g., behavior problems) and circled the one that concerned them the most. The DVD prompted parents to raise their hands publicly attesting to their chosen concern and then presented information on how the NBP would be helpful in addressing each concern.
Condition 5 was designed specifically to encourage participation of high-risk families who have been shown to benefit most from the NBP. The DVD in this condition included all the components in condition 4 but added feedback about risk. Parents summed the problems circled on the questionnaire they completed prior to watching the DVD which informed them that, although the NBP can be beneficial for all families, those scoring above a certain number would be especially likely to benefit from participation.
In future work, we plan to use the DVD version that is most efficacious in engaging parents into the NBP, particularly those from high-risk families. This decision will be based on the results of a randomized experimental trial comparing the five recruitment conditions that involves over 1,000 parents who attended the courts’ Parent Information Program. The analyses are not yet completed and thus conclusions about differences in the effects of the five recruitment conditions cannot be drawn. However, some tentative conclusions can be drawn. First, there was a high level of interest in the NBP. Over 50% of parents reported that they either definitely wanted to enroll in the NBP or wanted more information about it. Second, the participation rate was much lower than the level of expressed interest. Only about 10% of the parents attended one or more of the program sessions. Third, parents whose scores on the self-assessment were in the high-risk range showed more interest and were more likely to attend the program than those who scored in the low-risk range. Fourth, mothers were more likely to express interest and attend the NBP than fathers.
The finding that risk status positively predicted participation in the NBP is encouraging given that the beneficial effects of the NBP have been greatest for high-risk families. Also encouraging is the finding that many parents expressed interest in the NBP. However, in all conditions, there were large differences between the percent of parents who expressed intentions to participate and the percent who attended the first session. This intention-behavior gap is a widely observed phenomenon in health promotion research (e.g., intending to versus actually exercising) (Sheeran, 2002). Although conditions 3, 4 and 5 used social influence principles (Cialdini, 2001), other communication, marketing, and health promotion techniques were not used. We believe that the participation rate can be increased by incorporating additional evidence-based strategies, particularly ones shown to promote follow-through on intentions. For example, using multiple engagement channels (e.g., media ads; referrals from judges and court counselors; DVD presentation), rather than relying on only one channel (e.g., DVD presentation), and providing repeated exposure to promotional materials (e.g., presentation in parent-education class and a series of mailings about the program), rather than delivering promotional materials on a single occasion (e.g., presentation in parent-education class) have been shown to increase the effectiveness of social marketing campaigns (Evans & Hastings, 2008; Snyder & Hamilton, 2002).
Also, we plan to do more to address barriers to participation. Currently, we counteract barriers by providing free childcare during sessions and offering scheduling and location choices when possible. Based on strategies that have been used to increase parent attendance in child treatment (Nock & Kazdin, 2005) and increase engagement in healthy behaviors (e.g., exercise) (Gollwitzer & Sheeran, 2006), we are developing a brief procedure to help parents plan how they will resolve obstacles to attendance. This procedure will involve prompting parents to write down solutions they will use to overcome common obstacles to program participation, such as scheduling conflicts and transportation problems.
In the efficacy trials, high quality and fidelity of program delivery were assured through multiple methods. Leaders received extensive training (30 hours before the program and 1.5 hours per week during delivery) and ongoing supervision (1.5 hours per week), sessions were delivered using detailed manuals, and leaders were compensated for time spent preparing for sessions. Because such a high level of training, supervision and leader preparation is not feasible in community settings, we are redesigning the NBP to increase the probability that it will be delivered with high quality and fidelity outside university settings.
The framework of Modern Quality Function Deployment (QFD), derived from the original work of Yoji Akao and Shigeru Mizuno in the 1970s (Akao & Mazur, 2003), was used to guide the redesign process. A primary objective of QFD is to incorporate the “Voice of the Customer” (i.e., product features desired by the customer) by assessing these features early in the design process. Once the features are identified, the needs underlying them are articulated, quantified, prioritized and then used to make design choices (Mazur, 2005). We identified five groups of customers for the NBP: family courts that would engage and refer parents, administrators in mental health agencies that would offer the NBP, providers in agencies who would deliver it, parents who would attend the NBP, and the researchers who developed the NBP. Below, we discuss the process used with family courts and mental health agency personnel to illustrate our use of Modern QFD. For more information see Jones, Sandler and Wolchik (2008).
The first step in Modern QFD is to go to the place where the product interfaces with the customer, known as “going to the Gemba” (Japanese for “shop floor’). The objective is to gather information about what features customers need and want in a product via observation, interviews, and/or archival documentation. For the NBP, this step entailed running three focus groups with clinicians who had previous experience running NBP and/or other parenting groups in a community setting. Participants were asked what they liked and disliked about the NBP intervention “package” (i.e., leader manual, teaching aids [e.g., posters], training program, monitoring of implementation and supervision), and then generated ideas about program features and material formats that would improve the program. This process resulted in 241 customer “verbatims” (e.g., features, desires).
The next step in QFD is to articulate the underlying needs or product requirements suggested by the customer verbatims and to uncover any missing needs. These tasks are accomplished with two tools: the Affinity Diagram for grouping items and identifying the underlying cognitive structure or schema of the needs, and the Hierarchy Diagram for finding needs that are missing from the schema. In an Affinity Diagram, customer verbatims are grouped into similar categories; typically the verbatims are written out on separate cards which are then placed in groups of similar needs by two to ten customers. The activity is done in silence so the cognitive structure is not overlaid on the items but is allowed to surface naturally. Once the groups are constructed and labeled, the Hierarchy Diagram is used to sort the groups into levels starting at the most global and progressing to increasingly detailed levels. Through the process of creating a hierarchical structure, missing needs are identified. In the NBP redesign, three pairs of individuals representing three customer groups (i.e., mental health agency administrators, providers, and program developers) completed an Affinity Diagram with the 241 customer verbatims gathered in the focus groups. These pairs identified 34 underlying needs. To reduce the needs to a number that could be effectively used in the next steps in QFD (Saaty & Ozdemir, 2003), a second affinity diagram was conducted by two providers who had experience running the NBP. This process resulted in seven needs. The same pair of providers then constructed a Hierarchy Diagram with these needs and identified an eighth one. These needs were that the NBP should: (1) fit with agency priorities, needs and resources; (2) be held in a welcoming and comfortable environment; (3) utilize and respect providers’ clinical skills; (4) make reasonable time demands on providers; (5) be appropriate and accessible to the broad range of families; (6) engage and involve parents and meet the families’ needs; and have program materials and delivery formats that are (7) engaging and professional looking and (8) accessible and easy to use.
Once the customer needs are quantified, the next step is to have the customers prioritize the needs to provide direction for the design team. In Modern QFD, the prioritization of needs is accomplished through an Analytical Hierarchy Process (AHP), a tool which provides ratio-scale priorities based on a paired comparison of requirements (Saaty, 1977). In an AHP, comparisons are made between all pairs of the needs being ranked (i.e., to prioritize items 1, 2 & 3 the AHP would include comparisons of 1 vs. 2, 1 vs. 3, and 2 vs. 3). For each comparison, the respondent indicates which response is more important and the degree to which it is more important (e.g., slightly more important, much more important). Analyses of the comparisons allows for a ranking of all the responses in terms of customer priorities. Two customer groups, mental health agency administrators and providers in these agencies, were selected to complete AHPs. Counties were first drawn from a stratified random sampling of counties at three sizes (over 1 million, 300,000-1 million, 100,000–300,000), with counties having higher percentages of Hispanics and African Americans being oversampled. Court administrators then identified mental health agencies that were capable of providing clinical services to divorcing parents (e.g., parent education classes). Twenty seven agency administrators and 27 providers across 36 counties completed an AHP questionnaire.
Using QFD processes and tools, needs from a second customer group, the family court, were also identified and prioritized. Data were collected during the two meetings of our National Court Advisory Board, which consisted of professionals with extensive experience in the courts (e.g., judges, court administrators, a director of a national multidisciplinary organization of family court professionals). Participants in the first meeting identified 65 issues that might affect a court’s decision to adopt the NBP. These issues were grouped into six needs and rank ordered in terms of how much they would affect adoption decisions.
The top two priorities identified by the agency administrators and providers and by the Court advisory board members were highly similar. Given the consistency in the ranking of these priorities across key customer groups, the following priorities were selected to guide redesign choices: that the redesigned NBP should: (1) engage and involve parents, meet their needs, and have a positive impact on divorcing parents and their children, and (2) be appropriate and accessible to a broad range of families (Smith-Daniels, Sandler & Wolchik, 2008). A third priority, derived from the program developers, was added: (3) the NBP should be implemented with a high level of fidelity to its core components. Because there is considerable evidence that fidelity of implementation of evidence-based programs relates to their impact when delivered in community settings (Durlak & DuPre, 2008), this priority reflected the needs of the program developers to disseminate an effective program and was in accord with the agency and court priority of the program having a positive impact. The changes made in NBP that are related to priorities 1 and 3 are described below. The process of adapting the program to make it appropriate and accessible to a broad range of families will be discussed in the next section.
The redesign team consisted of the two developers of the NBP, a project director who had experience with NBP and another highly similar program, a provider who had led several NBP groups, a clinician who had both led NBP groups and supervised leaders conducting the NBP, an expert in research on cultural adaptation, and three instructional design professionals. Working over a two-day period, this group came up with a list of design choices or solutions to address the needs in each of the above three priorities. Suggestions from the focus groups of previous leaders and participants were incorporated into this list. These solutions were then organized into similar groups using an Affinity Diagram. The groups of solutions were then evaluated by five of the team members and rated on the degree to which they would effectively address the above three priorities and their user-friendliness. Figure 1 illustrates the links between the customer needs, the broadly defined solutions and the specific design features that meet the priorities. As can be seen, some solutions address multiple priorities. For example, individualizing program materials was viewed as addressing the priorities of the program being engaging and having a positive impact and it being accessible and appropriate to families from diverse backgrounds.
Before making modifications, the program developers identified NBP’s core components, the aspects that are seen as responsible for its positive effects, and thus, need to be preserved in the redesign process. Similar to other program developers (e.g., Dumas et al., 2001), the developers of the NBP viewed core components as including content (e.g., curriculum components) and process (e.g., methods used to teach the curriculum) features. In terms of core content, findings from the efficacy trials demonstrated that changes in mother-child relationship quality and effective discipline accounted for improvements in children’s outcomes (Tein et al., 2004; Wolchik et al., 1993; Zhou et al., 2008). Thus, the components of the program that targeted relationship quality, such as increasing recognition for positive child behaviors and good listening skills, and discipline, such as having clear expectations and using fair and consistent consequences, were retained. Process core components include active teaching, emphasis on home practice, and a small group format. Active teaching involves multiple components, such as soliciting experiences relevant to the program material from parents and modeling and role playing of program skills. The emphasis on home practice involves encouraging parents to use the program skills at home, record how these activities went, and discuss how the program skills worked with the leader and other mothers. The active teaching approach and emphasis on home practice were retained given extensive support for their effectiveness in changing behavior (e.g., Burns & Spangler, 2000; Kagan, 1992). The small group format was retained because it allows parents to receive support from other divorced parents which may lead to higher attendance.
The solutions identified by the redesign team are being used to modify these four aspects of the NBP: method of session delivery, including modification of the manual; number, type, and quality of teaching aids; number of leaders; and length of the sessions. In the efficacy trials, when conducting the sessions, the leaders used very detailed, lengthy session manuals that included verbatim script, instructions for each activity and suggestions for addressing concerns raised by the parents. The teaching aids in the efficacy trials included handouts and posters on the program skills and a small number of video demonstrations of program skills. In the efficacy trials, two leaders ran groups of 8 to 10 parents and sessions lasted for an hour and 45 minutes.
The most significant revision involved changing the method of delivery from manual-driven to DVD-driven. Each of the ten sessions will be run with the use of a DVD that includes much of the material that had been in the leader manual. Each DVD consists of 25–30 slides, which provide didactic material or serve as prompts for the leader to conduct an activity. Each DVD also includes two to five short (2–3 minute) video segments, which involve demonstrations of program skills or testimonials by previous NBP participants. For example, the segment of the DVD which focuses on the program skill of feeling responses begins with several slides describing what feeling responses are, the benefits of using them in conversations with children and a description of how to use feeling responses. This is followed by a video demonstration where an actor models using feeling responses when talking with a child. A slide then prompts the leader to facilitate a discussion of parents’ reactions to the video demonstration and concerns they may have about using feeling responses and to lead an exercise in which the parents practice identifying feelings and using feeling responses in role plays. To increase the user-friendliness of the revised leader manual, the didactic material is presented in outline format and pictures of the corresponding DVD slides are included. The use of a DVD-driven approach has several significant advantages. First, it enhances the likelihood that program content will be delivered with high fidelity, which increases the probability that the program will positively affect children’s outcomes. Second, it reduces leader burden both in terms of preparing for the sessions, which has implications for cost of delivering the NBP, and delivering the sessions. Third, it allows the group to be led by one leader versus two. This change dramatically decreases the cost of delivering the NBP and makes it more economically feasible as an ongoing service. Changes related to the teaching aids include replacing the posters with slides on the DVD, developing a parent workbook, and increasing the user-friendliness of the home practice materials (e.g., reducing the amount of text; increasing use of graphics). Session length was increased to two hours to allow for engagement activities at the beginning of each session. For example, in the first session on good listening, mothers are asked to talk with each other about someone they know who is a good listener and how it feels when someone really listens to them. The engagement activities serve the dual purpose of enhancing group cohesion and providing a buffer for late arrivals so they do not miss the section on home practice review.
These changes have been or will be piloted using an iterative process. Typically, the team developed a prototype of a program feature (e.g., outline format in manual) and a couple of former group leaders provided feedback on it. The modification was then pilot tested in at least one, and often two or three sessions, led by different leaders. Leaders and parents provided feedback weekly during the program and more extensive feedback at the end of the program. Leaders answered questions about how the changes affected ease of delivery; parents provided feedback about their level of engagement and identified the most helpful components of the NBP. This feedback was incorporated into new prototypes that were then piloted. We have been conducting these pilots for about two years and a half years and expect to finish the revisions in the next six months. Once the revisions are complete, a pilot will be conducted and the feedback will be used to make final adaptations before the effectiveness trial.
Changes are being made to the training, monitoring of implementation and supervision components to make them briefer and less intensive but still effective in bringing about high levels of quality and fidelity of program delivery. We plan to conduct the initial 30-hour training segment at the university research center. Because large-scale implementation limits the ability of program developers to interact face-to-face with providers, providers will participate in weekly telephone contact during the first implementation of the program and have access to web-based tools for receiving ongoing training and supervision and for monitoring implementation. The web-based component will include interactive websites with tips and traps for common implementation issues and video examples of high quality implementation, and an online database tool that can provide feedback about implementation quality. These modifications will be pilot tested, with leaders providing feedback about their user-friendliness and respondent burden and suggesting improvements. These data will be used to refine the training, monitoring and supervision components prior to the effectiveness trial.
To summarize, significant progress has been made toward the goal of redesigning the NBP to facilitate high levels of fidelity and quality of implementation in community settings. Multiple changes have been made in the program format and materials (e.g., DVD-driven format, increased user-friendliness of the manual, new parent workbook, more video demonstrations of skills). These changes have been piloted and have been positively evaluated by group leaders and parents. We are currently revising the training, monitoring, and supervision components.
The NBP was based on the literature available in the mid 80’s. At that time, samples in most studies of divorced families consisted of Caucasian mothers. In the efficacy trials, all participants were female, the samples were nearly exclusively Caucasian and too few ethnic minority families participated to assess whether program effects differed across ethnicity. Because ongoing programs serve the entire population of divorcing families, it was important to ensure the NBP would be engaging to diverse families and sensitive to the values, preferences, needs and contexts of the population of families served by the court.
Although some interventions may be effective across diverse cultural groups even without systematic efforts to adapt the program, we decided to make cultural adaptations in preparation for moving to the effectiveness trial. Our decision was informed by evidence that cultural adaptations may have their greatest impact on engagement, particularly attendance (Kumpfer et al., 2002), rather than program outcomes. Engagement is particularly important for achieving positive program effects of the NBP because of its heavy reliance on active learning, home practice review, and feedback to facilitate skill acquisition and use. Further, there is evidence that naturally occurring adaptations are most often made by community service providers when applied with culturally diverse populations (Ringwalt et al., 2004). Thus, we decided to identify adaptations that service providers might feel most compelled to make when delivering the NBP with diverse groups, and build such changes into our redesign. This proactive strategy allowed us to address perceived needs for cultural adaptation while minimizing the degree to which the core components might be compromised when delivered in the community. We also believe our efforts to address diversity will alleviate concerns of court personnel, agency administrators and service providers who want assurances that the program can be used with a broad range of families. Our goal was to develop a single program that is culturally-broad and competent for delivery across ethnic groups, rather than separate culturally-specific programs for different ethnic groups. See Gonzales, Wolchik, Sandler, Winslow, Martinez, and Cooley (2006) for more information about the process of cultural adaptation.
Similar to our approach to redesigning the NBP to maximize fidelity, we elicited insider points of view from several groups of cultural informants. We focused our initial efforts on African Americans (AAs) and Mexican Americans (MAs), because these are two of the largest ethnic subgroups in the U.S (U.S. Census Bureau, 2002), but subsequently expanded our adaptation efforts to include Asian Pacific Americans (APAs). As the largest Latino subgroup, MAs were selected to circumvent problems associated with ethnic glossing, when the various Latino subgroups are combined and assumed to be equivalent. Research has shown that divorce is associated with increased risk of academic and externalizing problems for AA children (Bussell, 1995; Jenkins & Guidubaldi, 1997). Similarly, parental divorce is associated with poor academic achievement among Latinos (Battle, 1997) and a range of mental health problems (including externalizing and internalizing problems), substance use, low academic achievement and low self-esteem among Asian and Asian American children and adolescents (e.g., Sun & Li, 2007).
Our Cultural Advisory Board (CAB) includes three prevention scientists with experience in adapting and implementing prevention programs for MA and AA families. This group played a central role in our work by helping to refine our overall approach, reviewing data obtained at each stage of the adaptation process, and recommending specific modifications. To begin, each CAB member completed a thorough review of the intervention manual and curriculum materials to identify aspects of the NBP that might require adaptation. A separate sub-committee also reviewed videotapes of sessions from the efficacy trial to evaluate program content and processes that might need adaptation to accommodate socioeconomic and cultural diversity. In a two-day meeting with the program developers, the CAB discussed data compiled from this review and generated a summary of areas most likely to need cultural adaptation. They also discussed the feasibility of a culturally broad approach, in which core components would be maintained and all participants would receive the same culturally adapted version of the program. There was agreement that this approach could work well if specific delivery techniques and processes used to teach skills were broadened to be more compatible for diverse families. The CAB outlined a plan to obtain information from two other types of cultural informants--culturally experienced providers and culturally diverse parents--to further inform the feasibility of this goal and generate specific strategies to achieve it. The CAB also recommended that input from Asian Pacific American cultural experts be obtained to ensure the program would be sufficiently broad with respect to the range of cultural groups served by family courts.
In addition to the CAB members, two researchers with expertise in working with APA families and eight providers who had run parenting interventions for MA, AA and APA families (2 providers for each cultural group) provided their perspectives. Reviewing each session, these individuals assessed whether the language (e.g., phrases, sayings, symbols, images and emotional tone) was familiar, easily understood and comfortable for MA, AA, and APA families; whether the values embedded in the program (e.g., concepts, program goals) and methods (e.g., role plays, home practice, and group discussion) were compatible with the values, customs, beliefs and traditions of each cultural group; and whether the program goals, activities and skills took into account the social and economic contexts in which families may live (e.g., financial strain, family size). This procedure resulted in numerous suggestions for “surface structure” changes, such as using people, language, or product brands familiar to the group. Most of these suggestions were similar to those made by the CAB, such as the need for more culturally relevant examples, broadening activities to incorporate extended family members, and greater diversity of actors in the videotaped skills demonstrations. The issues identified at the level of “deep structure,” or those that involve consideration of how ethnic/cultural norms, experiences, and behavioral patterns can be incorporated in the design and delivery of interventions, were also strikingly similar to those identified by the CAB members. Providers consistently noted that with some revision of aspects of the NBP, it would fit well with the needs, values and preferences of AA and MA parents. Conversely, the consensus among the APA informants was that more extensive revision would be needed for some aspects of the program to fit the needs, values and preferences of APA parents. For example, our APA cultural informants consistently noted incompatibilities in program skills that encouraged children’s open expressions of emotions, particularly negative emotions, which generally are not encouraged in most APA subgroups. Although we incorporated changes to be more sensitive to these differences, it is possible the NBP’s heavy focus on emotional expression will not be culturally compatible or beneficial for more traditional, less acculturated APA families.
Ethnic minority parents who have participated in the NBP are in a unique position to evaluate its cultural compatibility. To date, we have conducted six NBP groups that consisted mostly or exclusively of ethnic minority mothers (two groups for each of the three ethnic groups). Prior to these groups, surface structure adaptations (e.g., changes in examples) recommended by the CAB and other cultural experts were made. Multiple methods (e.g., weekly feedback forms, surveys at the end of the program, and focus groups) were used to collect data on the cultural appropriateness of the NBP from mothers and group leaders. The data indicated that the overall themes and core components of the NBP were generally consistent with the values and preferences of MA, AA, and APA families, and that the program seemed acceptable to and engaging for these groups. Like the other types of cultural informants, mothers and group leaders suggested changes in rationales and examples used to teach skills rather than changes in program skills themselves. Mothers and group leaders acknowledged most of the potential cultural incompatibilities identified by the prevention scientists and providers (e.g., language/metaphors, values). For example, extensive reliance on good listening skills was perceived as conflicting with AA and MA parents’ role-based desires to offer advice and guidance to their children. Our cultural informants suggested that we acknowledge these strong desires and the difficulty that parents might have using listening skills prior to or in place of parental advice. Our cultural informants also recommended that we provide more information to help reluctant participants understand how this new approach would benefit their children. Program skills focused on using assertive requests to protect children from hearing disparaging comments about the other parent (e.g., badmouthing) were also perceived as incompatible for MA and APA families. Several parents and community providers felt that many MA and APA parents would be reluctant to use these requests with friends and extended family members in the manner that was taught in the program. These concerns suggested that other ways to protect children from hearing these comments needed to be added to the program.
CAB members reviewed summaries of the data collected from each group of cultural informants and recommended concrete changes in each session to enhance the cultural sensitivity of the NBP. This information was then used to make modifications that maintained fidelity to core components but allowed tailoring or broadening of the program skills to meet the needs of the parents from diverse cultural groups. Examples of adaptations include: offering culturally sensitive rationales and explanations of program skills that were more compatible with goals and values of ethnic minority parents; acknowledging the potential cultural barriers to specific program skills or recommendations, such as systematic reinforcement for positive behaviors (which might be inconsistent with APAs’ emphasis on modesty) and the NBP’s recommendation against spanking (which might be incompatible with some AA and MA parents’ views about physical discipline); and providing more opportunities for parents to consider ways to make program skills more compatible with their own cultural styles and preferences.
To address differences in reading skills and ensure that reading comprehension would not be a barrier to learning the program skills, two master level 5th grade teachers reviewed all program materials. These individuals suggested replacements for words exceeding a 5th grade reading level. One of the program developers assessed whether these changes altered the intended meaning of the information and then incorporated the suggestions into the program materials. Although we were able to make adaptations to address each concern raised by our AA and MA cultural informants, we were not able to address all concerns raised by APA cultural informants without substantially changing or eliminating some core skills. Thus, one member of our team (Qing Zhou) has developed a culturally-tailored version of the program for low acculturated APA families. We also plan to translate the culturally-broad program for implementation with non-English speaking parents.
The efficacy trial only included mothers. Because family courts serve both mothers and fathers, part of the redesign process involved assessing whether the NBP was appropriate for fathers. Based on feedback from divorced parents and our judgment that parents would feel more comfortable participating in the program and be more able to focus on improve their parenting skills rather than their anger at the ex-partner in same-gender groups than mixed-gender groups, separate groups for mothers and fathers will be conducted in the effectiveness trial.
Two male professionals who were knowledgeable about the NBP and had experience working with divorcing populations reviewed the leader manual and curriculum materials to identify possible places where surface structure changes were needed. A team of these two professionals, one of the program developers, and a clinician who had experience with the NBP used these data to make adapt the NBP for fathers who had weekly contact of at least one overnight a week with their child(ren). A pilot was run to determine fathers’ reactions to the program and their suggestions for improvements. Both the fathers and leaders reported positive responses to the program and noted that the skills and framework of the program were relevant and helpful. Ideas for improvements included creating video demonstrations of males using program skills instead of showing the skills through leader role-plays and adding more time for interacting with other fathers. We conducted five additional groups for fathers. All fathers who provided feedback reported that the NBP was helpful or very helpful and that they felt much more or very much more confident about their parenting as a result of participating in the NBP.
We are planning to conduct a large-scale effectiveness trial of the NBP as implemented in collaboration with the family courts. This study will have four specific aims. The first aim is to develop and test an enhanced method of engaging parents to engage in the NBP. We plan to use multiple avenues for engagement and to modify our strategies to be more consistent with the natural processes through which parents become involved in such programs. One avenue will be through referrals from lawyers, judges and court-related social service providers. A second avenue will be brochures or media announcements about the NBP. A third avenue will be an invitational DVD about the NBP shown in short parent education classes. We plan to enhance the DVD used in our previous work to be more effective, particularly for fathers.
A second aim is to test program effects across the sub-populations seen by the courts. Separate groups will be conducted for mothers and fathers; groups for mothers and those for fathers will include parents of different ethnicities. We will examine whether program effects differ for African American vs, Caucasian families and for mothers vs. fathers on the outcomes at post-test that predict positive longer-term outcomes, improvement in parenting and reduction in youth externalizing problems.
A third aim is to test the relation between measures of quality of implementation or how well the program was delivered and program outcomes. We conceptualize implementation as including multiple dimensions and have developed reliable measures of each dimension (quality of program delivery [e.g., positive engagement of leaders with participants when delivering the session material], fidelity of delivery [e.g., percent of session material covered], and parent responsiveness [e.g., degree of home practice completion]). We will examine whether each dimension of implementation predicts program effects.
A fourth aim is to study a process of engaging courts and community agencies to implement the program. The process will involve developing working groups of key stakeholders to consider how local policies, resources, social connections, and administrative processes can be used to foster the successful implementation of the NBP and its sustainability. It is our view that engaging in a planful, collaborative process early on and proactively is essential for effectively transporting university-tested programs into ongoing services.
This program of research was conducted at the Arizona State University Prevention Research Center. Support was provided by National Institute of Mental Health grants 2R01 MH49155-06, R01MH071707 1R01MH057013-01A1, 2P30 M439246-18 and 5P30MH068685-3. The authors are grateful to numerous colleagues and graduate students who made important contributions to this research. Susan Westover, Lillie Weiss, Jenn-Yun Tein, Roger Millsap, Julie Lustig, Art Martin, Kathleen Hipke, Spring Dawson-McClure, Rachel Haine and Robert Cialdini deserve special mention. We are also thankful to the many parents and children participated in this research.
1Researchers at The Arizona State University Prevention Research Center have also used a community based participatory research model to develop interventions that fit well in natural services delivery systems. This strategy involves collaborating with key community stakeholders and the provider institution at the early stages of program development. The article by Hita et al. (in press) in this volume illustrates this approach. Elsewhere, we described how these two approaches are integrated within a Prevention Service Development Model (Sandler et al, 2005).
This article may not exactly replicate the final version published in the journal. It is not the copy of record.