It is challenging to a conduct a scientifically-rigorous evaluation of a prevention program within a child welfare setting, but necessary if we are to identify the most efficacious interventions for one of our most vulnerable populations. Below, we delineate some of the research and clinical challenges we have confronted in designing and implementing FHF.
Recruitment and Retention
There are many challenges in recruiting and retaining participants for a RCT in a foster care population because youth can change placements and/or legal custody multiple times during the course of the study. In fact, we have over 20 consent, assent, and HIPAA forms to accommodate the fact that youth may change placements and custody at different stages in the research study. Despite these difficulties, we have been highly successful in recruiting and retaining participants. To date, we have recruited 93% of the eligible youth and have retained over 90% of those recruited at subsequent assessment timepoints. We attribute our recruitment success to the fact that we provide the following: (a) no-cost screenings assessments, (b) appropriate financial compensation for participation in research interviews, (c) flexibility in scheduling locations, days, and times for interviews, and (d) a chance for youth to participate in an intervention that was specifically designed to meet the needs of children and families.
We have also been successful in engaging and retaining participants for the preventive intervention. To date, almost all of the youth randomly selected to participate in the intervention (95%) have chosen to enroll and only 7% have withdrawn. This is striking, given that more than half of the children changed placements, and 25% reunified, during the 9-month intervention. The rate of participation in the intervention is also very high. During the first three years of the program, youth attended an average of 25 (of 30) skills group sessions and mentors met with youth an average of 26 (of 30) times. Of note is the fact that none of the youth who reunified during the intervention dropped out of the program. This is a tribute to the excellent alliances and trust our mentors establish with biological parents before children reunify.
Several aspects of the intervention design may be particularly compelling for youth and their caregivers. FHF mentors provide all transportation for the program, help children and families access resources, and serve as an ally during an often tumultuous time. In addition, although the intervention is manualized, it allows for flexible and diverse approaches that reorient toward a child’s ever-changing needs (e.g. as they move placements or reunify with biological families). Although the FHF program stresses advocacy on behalf of youth and families, it is important to note that the program does not weigh in on placement decisions or assume other responsibilities that are more traditionally considered the role of caseworkers. Instead, FHF works to empower youth and families to advocate on their own behalf and use available community resources.
Generalizability and Replicability
It is a challenge to design an intervention that is amenable to scientific evaluation but is also generalizable and replicable in diverse community settings. To this end, we have taken steps to maximize the generalizability of our findings by limiting our exclusion criteria. For example, we do not exclude youth with significant mental health and behavior problems, youth with mild developmental delays, or youth in restrictive placements (including psychiatric hospitals and RTCs). The fact that there is not differential attrition by treatment group also suggests that our findings may be generalizable. We believe our program and research methods, coupled with our high recruitment and retention rates, will maximize the generalizability of our findings and enhance our ability to successfully disseminate the program, should the quantitative results demonstrate efficacy.
In order to disseminate the program, however, we must also demonstrate that we can deliver the content and process of the intervention in the same manner to all participants (Dumas, Lynch, Laughlin, Smith, & Prinz, 2001
). To this end, we have developed and revised program manuals to guide the intervention. We provide extensive orientation and ongoing supervision for the students who serve as mentors and research interviewers on the project. We document the duration and content of each mentoring and skills group activity and rate participants’ engagement. For example, the skills group program is comprised of 108 discrete activities delivered over 30 sessions. Group leaders track how many activities are completed in each group each week. During the first 3 years, an average of 96% of the activities were completed. Measuring program process will also enable us to examine which aspects of our program have the greatest impact.
Children involved in the child welfare system are extremely heterogeneous with regard to race and ethnicity. To complicate matters, children in foster care can be placed in neighborhoods strikingly different from their own, with caregivers of different racial, ethnic, and socioeconomic backgrounds; even these differences are not static because they change when children change placements. In the FHF program, these differences are often mirrored in the matches we make between mentors and participating youth. It is not possible to pair all youth with mentors who match on ethnicity, as 80% of our mentors are Caucasian. Nor is it possible to pair all boys with mentors who match on gender, because the social work students who serve as mentors are predominantly female (81%).
For these reasons, we have taken care to design an intervention and assessment protocol that is culturally sensitive. FHF incorporates components of interventions (i.e. PATHS, mentoring) that have been used successfully with heterogeneous populations. All FHF group materials were designed to be linguistically and visually sensitive for mixed cultural backgrounds, using input from experts in these areas. The mentors receive extensive training in multicultural issues through both their graduate school training and our program’s weekly seminar. Because FHF mentors often do not match the children they mentor on gender, race/ethnicity, or socioeconomic status, they are supervised closely with regard to the impact that these differences may have on children, families, and communities. Mentors are encouraged to embrace differences and to explore what these differences mean to their children. Mentors also work to facilitate the involvement of children in culturally-meaningful activities and to learn about and share their own, their mentees’, and others’ cultures. Children, with the support of their mentors, have an opportunity to give a presentation to the group about aspects of their cultural heritage. As already described, children are also paired with “career shadows” who are matched with youth on gender and race/ethnicity.
Experience in the FHF pilot trial provides some evidence that FHF is a culturally-sensitive program. For example, FHF has successfully enrolled multicultural children and families including bilingual children and monolingual Spanish-speaking caregivers. Our recruitment and retention rates do not differ as a function of race or ethnicity. Preliminary evidence from our pilot study suggests that matching youth and mentors on demographic characteristics does not influence outcomes. In the first three years of our pilot study, those ethnic minority children who were not paired with ethnic minority mentors did not differ from those with an ethnic match on rates of group attendance, number of mentor visits, or perceived support from their FHF mentors. Similarly, boys who were paired with female mentors did not differ from boys who were paired with male mentors on these indices.
In a qualitative study conducted with 100% of the first cohort of participants one year after the program ended, we asked youth and caregivers to discuss “what it was like” to have a mentor who matched or did not match on demographic characteristics. None of the children or caregivers reported any concerns about ethnic/racial similarities or differences. Similarly, all of the male youth and their caregivers reported that a mismatch in gender was not a problem, although a few caregivers of boys who had female mentors said that they had been concerned initially. These same caregivers reported, however, that they subsequently valued that their child had developed a positive relationship with a female role model.
Finally, there have been concerns about the ethics of a mentoring component that ends after 9 months. When developing the program, we found no solid empirical evidence suggesting that longer mentoring relationships were more beneficial. Furthermore, we felt that a planned and positive ending could be therapeutic for children who have often experienced unpredictable and traumatic losses. Through focus groups, we learned from youth that some short-term relationships (e.g. with teachers, camp counselors, group home leaders, therapists) had been meaningful and beneficial even though time-limited. We also learned that unplanned endings with mentors could be devastating. We were concerned that if the FHF program allowed mentors to maintain contact with youth after program completion, that they would not be able to sustain the relationship without program support and supervision. We felt that an unplanned, negative ending to the relationship might be harmful.
Our clinical experience, as well as the qualitative interviews with past program participants, suggest that the way we end the program is empowering and positive for youth involved. Children graduate from the program and receive medals and diplomas, as well as highly personalized speeches delineating all they have accomplished in the program. Our qualitative interviews with past program participants suggest that although children and families were sad when the program ended, they felt it was the right length.