|Home | About | Journals | Submit | Contact Us | Français|
Multiple studies in the National Institute on Drug Abuse Clinical Trials Network (CTN) demonstrate strategies for conducting effective substance abuse treatment research with racial/ethnic minorities (REMs).
The objectives of this article are to describe lessons learned within the CTN to (1) enhance recruitment, retention, and other outcomes; (2) assess measurement equivalence; and (3) use data analytic plans that yield more information.
This article includes background information and examples from multiple CTN studies on inclusion, measurement, and data analysis.
Seven recommendations are included for conducting more effective research on REMs.
Research has revealed racial/ethnic differences in the prevalence of substance use (1), the relation of acculturation to substance use (2), legal and health consequences of substance use (3) and the likelihood of participating in and completing substance abuse treatment (4, 5). Therefore, research findings derived from predominantly White substance-using samples may not be generalizable to specific Racial/Ethnic Minorities (REMs). Instead, more focused research on REMs is required to obtain information to reduce and ultimately eliminate racial/ethnic disparities. In this paper, we share lessons from studies conducted within the Clinical Trials Network (CTN) of the National Institute of Drug Abuse (NIDA) about designing and conducting research to maximize the information available on Racial/Ethnic differences.
The NIH Guidelines on the Inclusion of Women and Minorities require both the inclusion of REMs in Phase III clinical trials and valid analysis to detect differences in intervention effects (6). In addition, the Strategic Plan on Health Disparities Research of the National Institute on Drug Abuse (NIDA) encourages research aimed at reducing disparities in treatment and health services research (7). Despite these initiatives, task force reports of the American Psychological Association (APA) both before (8) and after (9) the establishment of the NIH guidelines along with other reviews (2, 10, 11) conclude that the efficacy of evidence-based-treatments (EBTs) for REMs frequently goes unexamined. Moreover, few resources are available to researchers for translating the NIH policy into practice.
The NIDA \CTN conducts randomized clinical trials to evaluate substance abuse treatment interventions with demonstrated efficacy in larger, effectiveness trials. The CTN studies constitute an ideal venue for conducting research on promising treatments for REMs. As of May 1, 2010, the CTN had enrolled 2474 (22%) Black, 1968 (17%) Hispanic, 271 (2%) American Indian/Alaska Natives, 64 (.6%) Asian/Pacific Islanders, and 1510 (13%) Other/Multi-race participants into 25 clinical trials. Most CTN studies meet the rigorous design requirements described by Nathan et al. (12) for randomized clinical trials.
Members of the former CTN Design and Analysis Committee recently published a paper recommending strategies for translating the NIH guidelines, particularly the requirements for valid analysis, into practice (13). This paper compliments that publication by using CTN studies to illustrate the lessons learned about conducting more effective research on REMs in clinical trials. The next section provides a context for the sections that follow by reviewing background information on the inclusion of REMs, measurement issues that jeopardize the accuracy of data collected on REMs, and data analytic strategies that maximize the information gained on REMs in clinical trials. The sections that follow the background address the main objectives of the current paper which are to describe lessons learned within the CTN to (1) enhance recruitment, retention and other outcomes, (2) assess measurement equivalence, and (3) use data analytic plans that yield more information. The summary includes a number of recommendations based on the CTN experience for more effective research with REMs.
A growing body of literature has addressed the formidable challenge of achieving the NIH requirements for the inclusion and valid analysis of data collected on REMs (14, 15). Much of this work focuses on one or more of the following issues: inclusion (recruitment and retention), measurement, and data analysis.
The underrepresentation of REMs in clinical trials limits researchers’ ability to conduct meaningful or valid studies to enhance our understanding of the prevalence and consequences of substance use for REM populations as well as their unique treatment needs (15). Recruitment and retention rates generally are lower for REMs than Whites in addiction (16, 17, 18) as well as other non-addiction clinical trials (19, 20). Some researchers attributed the lower inclusion rates to personal factors (e.g., cultural distrust and attitudes about research), social factors (e.g., travel expenses, competing child care responsibilities, inflexible work schedules), and research-related factors (e.g., lengthy consent documents, insufficient outreach, inconvenient times or locations, use of internet) (14, 15). Other factors such as concerns about the social or employment consequences of disclosing one’s substance use, relapse, and ongoing legal involvement add additional barriers to the inclusion of REMs in substance abuse trials.
The strategies in the literature for including REMs in research fall into two categories. The first category includes efforts to bolster inclusion by involving the community in the research. Community Based Participatory Research (CBPR) exemplifies perhaps the most extreme form of community involvement because the community representatives become full partners in the research in CBPR (21). The second category emphasizes the adaptation of the recruitment and retention procedures along with the intervention itself to be more appropriate for the target group.
Growing evidence suggests that adapting the intervention to the culture of the target group will improve outcomes (22, 23, 24). Even though more effective models for cultural adaptation are available, community treatment programs frequently attempt to adapt the intervention by pairing a client with a clinician from the same racial/ethnic group. This strategy is an example of an approach that Resnicow, Soler, Braithwaite, Ahluwalia, & Butler (2000) label a ‘surface’ adaptation approach. The content of the intervention remains unchanged in a ‘surface’ adaptation even though the delivery method may be modified (25). Research findings on the benefits of racial/ethnic match have been inconsistent (26). The two CTN studies reviewed later in this paper may help to explain the inconsistent findings.
Adequate measurement is essential to eliminating health disparities in substance abuse research (27, 28). The demonstrated adequacy of a measure for one group may not guarantee the adequacy of the measure for another group particularly if differences in cultural norms and practices could influence the relevance and interpretation of items (29, 30). Measurement non-equivalence may result in inaccurate conclusions especially about between-group differences on characteristics such as diagnosis (31). Historically, the reliance on empirical techniques associated with classical test theory led researchers to ignore the fact that violations of measurement equivalence may jeopardize the findings just as much as problems with reliability and validity. Moreover, Strada, Donohue, and Lefforge (2006), in their review of 18 treatment outcome studies of adolescent substance abusers, reported that the translation of several measures into Spanish was the only reference to the cultural appropriateness of the measure. Obviously, the translation of the measures does not ensure measurement equivalence.
The NIH mandate directs the researcher to utilize data analytic strategies that yield the best information about the efficacy of the intervention for REMs. Huey et al. (32) proposed that at least one of the following conditions must be met in order to assume that the overall findings on an intervention apply to the REMs in the sample: at least 75% of the sample must be members of that racial/ethnic group, separate analyses must be conducted to assess the efficacy of the intervention for the specific racial/ethnic group, or additional analyses must be conducted to demonstrate that race/ethnicity does not moderate the relationship between treatment type and outcomes. Our earlier paper demonstrated the pitfalls of data analytic strategies that primarily involve racial/ethnic comparisons, the possibility of misinterpreting findings when REMs are combined for the analysis, and the scientific value of considering the heterogeneity within a group (12).
The NIH guidelines recognize that examining all these issues in every study may not be realistic. However, the NIH guidelines identify two situations in which considering the diversity of the sample in the analysis plan is especially important. The two situations are those areas for which prior studies support the existence of significant differences and those areas for which prior studies neither support nor negate significant differences. In the latter situation, the NIH requirement is less rigorous and can be satisfied with techniques that require less statistical power such as effect size.
The following sections address lessons learned in the CTN about the issues presented above (i.e., issues of inclusion [recruitment and retention], measurement and data analysis).
This section addresses the first objective of our paper---to describe CTN lessons on enhancing recruitment, retention, and other outcomes. The designs of several CTN studies directly and successfully address the issue of inclusion (i.e., recruitment, retention and other outcomes) of REMs in clinical trials. Many lessons about designing clinical trials are embedded in these studies. The researchers in the first study improved both their recruitment and retention by involving the community. The second set of studies examines racial/ethnic differences across multiple CTN trials. The third set of studies provides some helpful information on the efficacy of matching the counselor and client on race/ethnicity.
CTN 0020-A-1 (Job Seekers Training Clients with Substance Abuse at the Na’Nizhoozhi Center, Inc.) (hereafter referred to as the Navajo project) demonstrated the advantage of a community based participatory approach (CBPR) to implement a CTN protocol in a largely Navajo substance abuse treatment facility. The original project, CTN 0020, used standard recruitment and retention methods to implement and evaluate a brief job-training program for the general population of substance dependent participants The Navajo project used a CBPR approach to conduct a similar intervention (33).
The hallmark characteristics of CBPR are collaboration and sharing of decision-making with the target community at all stages of the project from design to implementation to dissemination (21). The initial step taken to implement a CBPR approach was to include Native Americans as full community partners from the beginning. The research team traveled lengthy distances to hold joint weekly meetings to ensure the ongoing involvement of the community partners. Second, the team submitted the project to the Navajo Nation Human Research Review Board. That Board reviews all research on the Navajo nation prior to implementation. Third, Navajo staff edited assessment instruments to ensure the instruments were culturally appropriate. Finally, the project hired counselors and research assistants fluent in both Navajo and English and knowledgeable about the Native American worldview (33). This CBPR approach improved recruitment and yielded impressive retention rates of 98% at the 3 and 6 month follow-ups (33).
The Navajo project was conducted in a community that already had an infrastructure for forming research partnerships and tribal councils to review human and community protection concerns. Other communities may lack such support. In such cases, researchers may seek out community partners in other ways such as involving the research team in community events (34), partnering with existing community agencies (35) ,and other community based organizations (36,37,38).
The available research indicating more retention problems among REMs than Whites is fairly convincing. The NIDA Center for the Clinical Trials Network (CCTN) regularly monitors retention as a component of clinical trial progress. Recently, the CCTN staff examined retention in 17 of the first 20 CTN trials. The analyses included three alternative indicators of retention: the availability of primary outcome data (APOM), the rate of treatment exposure (TE) and the proportion of follow-up (FU) assessments obtained. The numbers in Table 1 represent the cumulative retention rates across the trials in this study. The sample of 6755 was 48% White, 27 % Black, and 15% Hispanic. Overall, the retention rates were slightly higher for Whites than either Blacks or Hispanics on all three indicators. However, the differences were not significant. The largest difference was on the percent of follow up assessments attended (See Table 1). Comparing the CTN results with other substance abuse clinical trials is difficult because many researchers do not provide these separate retention indicators. However, our results are similar to published clinical trials in cocaine dependence, where retention rates were as low as 49% and did not differ by demographic characteristics (39, 40).
The findings in another CTN study by Magruder et al. (41) may be helpful for identifying the subgroups that account for racial/ethnic retention differences among CTN participants. Magruder et al. reviewed the records of 1910 participants from 38 different community treatment programs participating in one of six CTN clinical trials. The sample was 52% White, 36% Black, and 12% Hispanic. An age by race/ethnicity interaction revealed that younger (age 18–35) adult Blacks were harder to retain than older Blacks (age 36+) or any of the other racial/ethnic minority groups. These findings are consistent with previous studies demonstrating the difficulty of retaining younger Blacks (42, 43) and especially younger Black males (5) in treatment.
These findings have implications for increasing retention rates in REMS. Specifically, investigators should be vigilant about retention for all groups. However, the evidence suggests that the traditional strategies are less effective for younger REMs. Therefore, channeling additional resources into improving the retention of younger REMS may be warranted. Fortunately, some previous research suggests that retention can be improved in younger REMs by permitting discussions of cultural themes (5), developing a positive alliance with the participant at the initial contact (42), and by involving the family when appropriate (44).
The findings on the advantages of matching clients with counselors from the same racial/ethnic group have been inconsistent. Some previous research suggests that racial/ethnic matching improves outcomes (45) but other studies suggest that racial/ethnic matching does not improve outcomes (46). Two CTN studies suggest the effect of racial matching may vary across treatment modalities, subpopulations, and disorders. Hence, the appropriate question may be when (e.g., in what treatment modalities, at what point[s] in treatment, with what subpopulations, and with what particular disorders) does racial/ethnic matching improve outcomes rather than does racial/ethnic matching improve outcomes.
The first study was a secondary analysis of CTN 0014, a clinical trial that evaluated Brief Strategic Family Therapy (BSFT) for adolescent substance users (47). Participants included 480 adolescents (44% Hispanic, 31% White, and 23% Black) randomly assigned to BSFT or Treatment as Usual (TAU) at 8 community-based substance abuse treatment facilities. Results indicated that racial/ethnic matching, alone, did not significantly improve treatment outcomes for any racial/ethnic group. Moreover, the Hispanic adolescents with matched counselors had slightly worse outcomes than Hispanic adolescents with counselors from another racial/ethnic group. However, perhaps the most interesting finding is that a third variable, family functioning, influenced the relationship between matching and the engagement outcome (i.e., returning for a second session) for Blacks, Hispanics, and Whites in different ways. As family functioning decreased, matched Black adolescents had a higher likelihood than unmatched adolescents of engaging in treatment. However, as family functioning decreased, unmatched Hispanic adolescents were more likely to engage than matched Hispanic adolescents. Moreover, family functioning did not influence the relationship between racial/ethnic matching and engagement for White adolescents. Racial/ethnic matching may prove more beneficial to Black adolescents in families experiencing considerable conflict and a lack of cohesion, but may be more helpful to Hispanic adolescents in families not experiencing conflict.
Likewise, Ruglass et al. (48) utilized data from CTN 0015 (Women’s Treatment for Trauma and Substance Abuse) (hereafter referred to as Seeking Safety) to conduct a secondary analysis examining the relationship between racial/ethnic match, therapeutic alliance and treatment outcomes among 281 women with posttraumatic stress disorder (PTSD) and substance use disorders (SUD). The original clinical trial evaluated the efficacy of Seeking Safety, a cognitive behavioral intervention designed for female trauma victims, by comparing the outcomes for Seeking Safety and the control, psychoeducational intervention (49). Results did not reveal any overall effects of race/ethnic matching. However, subgroup analyses revealed that White participants with severe PTSD symptoms at baseline reduced their PTSD symptoms more when matched with White counselors. Both Black and White substance users at baseline were less likely to use substances heavily at follow-up when matched with counselors of the same race/ethnicity. Since all clients and counselors were female, gender match was not a confound.
Taken together, the findings from the BSFT and Seeking Safety protocols illustrate the complexity of the impact of race/ethnic matching. The results suggest first that other factors such as family functioning or symptom severity may affect the extent to which racial/ethnic matching improves outcomes. However, the influence of these other factors may differ across racial/ethnic groups. Moreover, although more research on racial/ethnic matching is recommended, other types of matching might also play an important role in treatment outcomes. For example, past research suggests that language match affects treatments outcomes among Hispanics in prior research (50, 51). Therefore, even though only English speaking participants were included in the BSFT intervention, language match could conceivably have played a role in the relationship between racial/ethnic match and BSFT treatment outcomes. Similarly, since previous research revealed that gender matching affected outcomes in other addiction studies (46, 52), gender matching could potentially affect the relationship between racial/ethnic matching and treatment outcomes. More research is necessary to isolate the effects of racial/ethnic matching from the effects of other types of match.
Another variable that influences the outcomes of REMs in clinical trials is measurement equivalence. The following section addresses the second objective of our paper, which is to describe lessons learned within the CTN on assessing measurement equivalence.
Two separate CTN studies described below illustrate the important information that can be gained by using two alternative approaches to assessing measurement equivalence. The first study used Differential Item Functioning (DIF) (item response theory) to evaluate the need to control for race/ethnicity statistically in the analysis of cocaine dependence. The second study conducted Confirmatory Factor Analysis (CFA) to evaluate whether the measures of family functioning were equivalent for adolescents from different racial/ethnic groups.
Wu et al. (31) used DIF and the multiple indicator-multiple causes (MIMIC) analysis to examine whether the dependence criteria from the DSM-IV Checklist represent a similar measure of cocaine dependence among both Black and White substance users. In this case, measurement nonequivalence can potentially affect not only the validity of diagnosis-related findings but also the treatment outcomes by diagnostic status. Wu et al. (31) conducted a secondary analysis of two multisite studies initially designed to evaluate the efficacy of adding contingency management to usual care. The sample consisted of 682 adult cocaine users (mean age = 38.9 years) recruited from 14 community-based substance abuse treatment programs providing either non-methadone (CTN 0006) (53) or methadone maintenance (CTN 0007) (54) treatment. About one half (51%) of the secondary analysis sample were women; the majority (74%) were REMs (Black, 53%; Hispanic, 15%; others, 6%).
The results revealed the presence of DIF in some diagnostic questions for cocaine dependence (i.e., different threshold levels of criterion items) by racial/ethnic status (31). Using the MIMIC modeling to control for the level of cocaine dependence, Hispanics were more likely (lower threshold) than Whites to endorse “tolerance,” but less likely (higher threshold) than Whites to endorse “continued use despite resulting problems”. Blacks were more likely (lower threshold) than Whites to report “inability to cut down”. These findings suggest that cocaine users from different racial/ethnic groups may differ in the symptoms they are likely to endorse. Identifying these differences may improve the assessment and analysis of self-reported data for substance dependence in future studies. Specifically, once the DIF was detected, the research team compared the MIMIC model results for a model that did not include DIF effects with the model including DIF results. Consequently, the team observed some changes in the effect size for the race/ethnicity variable. This finding, along with results from similar research, suggests the need to control for race/ethnicity statistically in the analysis of cocaine dependence and to investigate reasons accounting for DIF (e.g., racial/ethnic differences in interpretation of diagnostic questions).
As described earlier, the CTN 0014 study evaluated the efficacy of BSFT versus TAU. The analyses of secondary outcomes revealed that BSFT was more effective than TAU in improving family functioning. A composite that included the Parenting Practices Questionnaire of the Chicago Youth Survey (55) and the conflict and cohesion scales of the Family Environment Scale (56) was used to assess family functioning. Additional analyses indicated that BSFT/TAU differences in family functioning were most evident among Black and Hispanic youth and less so in White youth. To ensure the validity of cross racial/ethnic comparisons, Feaster et al. (57) conducted a series of CFAs to examine measurement equivalence on family functioning.
The findings revealed a surprising degree of invariance in the measurement of family functioning across racial/ethnic groups. Specifically, an increment in any of the subscales of family functioning, for example, implied the same increment to the overall family functioning construct across all racial/ethnic groups (invariant factor loadings). Stated differently, a unit-change in each subscale means the same change in overall family functioning for all racial/ethnic groups. In addition, the particular value of a subscale mapped onto the same value of the family functioning composite across all racial/ethnic groups (scalar invariance or invariant intercepts) for most, but not all subscales. Despite some differences that are culturally interesting and potentially clinically important for cross-cultural research, overall, the minor differences in measurement did not preclude comparisons of family functioning across racial/ethnic groups.
Measurement equivalence is an important issue that sets the stage for an appropriate analysis (and interpretation) of the data. The following section addresses the third objective of our paper, which is to describe lessons learned within the CTN on using data analytic plans that yield more information.
The CTN studies below illustrate issues that researchers face in the data analysis phase. Each study demonstrates that some data analytic plans yield more information than others regarding treatment effects for REMs. The findings from the studies suggest three separate conclusions.
The first conclusion is that it is preferable to utilize a data analytic plan that examines rather than assumes that the findings for the overall sample apply to the REMs in the sample. Several CTN studies support this conclusion by demonstrating that the overall findings did not apply to the REMs in the sample. For example, Calsyn et al. (58) evaluated a five session HIV risk reduction group intervention called ‘Real Men are Safe’ (REMAS) in CTN 0018. Overall, the men randomly assigned to REMAS reported a greater reduction in sex risk behaviors than participants assigned to the control condition. Post-hoc analyses, however, revealed that REMAS did not reduce HIV risk behaviors as well in the Black subgroup. The investigators in both CTN 0013 (MET for Pregnant Substance Users) (59) and CTN 0029 (Osmotic-Release Methylphenidate for smokers with ADHD) (60) demonstrated the opposite pattern. Specifically, these two interventions had positive outcomes for the REMs even though the treatment and control groups did not differ in the overall sample.
CTN 0004 was an evaluation of MET for substance abusers. CTN 0042-S, a secondary analysis of CTN 0004, only included the REMs in the MET study. The REM findings suggest two additional conclusions related to the data analytic plan (61). First, the REM findings supported earlier recommendations by Hettema et al. (62) and Burlew, Feaster, Brecht, and Hubbard (2009). Both teams argued that combining REMs may conceal differences between racial/ethnic groups. Separate analyses conducted on Hispanics and Blacks in the sample support this conclusion. The analyses revealed the two groups had different results. Black females in MET had higher retention rates than Black Females in TAU. However, the analyses for Hispanic females did not reveal any significant differences between Hispanic females in MET and TAU (L. Montgomery, MA, oral communication, March 2010). This finding suggests that information may be lost when the statistical analyses combine REMs. Instead, the heterogeneity across REMs can be examined either by conducting separate analyses for each racial/ethnic group or including race/ethnicity as a moderator.
The third conclusion is that data analytic plans should consider the heterogeneity within a REM group to yield even more information regarding the generalizability of interventions. Obviously, examining the findings for all subgroups would not be feasible. However, using the literature as a guide, the investigator may be able to identify the specific characteristics (e.g., gender, SES, immigration status) to include in subgroup analyses. The Black sample in the MET study also supported this conclusion. Initial findings did not reveal any significant retention differences between the participants in MET and TAU among Black participants overall. However, as mentioned earlier, additional analyses revealed that Black females in MET had better retention rates than Black females in TAU. Yet, no differences were observed among Black males. The finding that MET increases retention for Black females may not have surfaced if the team had not examined the findings for Black males and females separately.
The findings in this section clearly demonstrate that the conclusions would have been misleading for REMs in a number of CTN studies if the researchers had used a data analytic plan that assumed that the overall findings applied to the REMs. Although examining findings among each racial/ethnic group is not always feasible, efforts such as the NIDA CTN provide a unique opportunity to conduct secondary analyses of REMs participating in large, multisite randomized clinical trials (13). In addition to outcome studies, the data sets from large randomized clinical trials also provide good opportunities for follow up studies on other important questions such as identifying potential mediators and moderators of treatment effectiveness. For example, Montgomery et al.’s (61) secondary analysis of the effectiveness of MET among Blacks led to a new project that is currently examining whether potential moderators (e.g., change talk and decisional balance) influence the treatment outcomes among Black substance users. Potential moderators were chosen from the existing literature on MET. The NIDA CTN provides a great platform for secondary studies of specific REMs.
The objectives of this paper were to describe lessons learned within the CTN to (1) enhance recruitment, retention and other outcomes, (2) assess measurement equivalence, and (3) use data analytic plans that yield more information. The first set of recommendations are based on our first objective. The studies within the CTN suggest that researchers should consider REM issues early in the study design. Both the existing literature and the CTN experiences support the need for more research on effective strategies to promote REM inclusion in clinical trials and research more broadly. Embedding studies on recruitment and retention in substance abuse clinical trials may be more efficient than conducting studies that only address inclusion. Based on the information already available, we offer several specific recommendations. The first three recommendations address the design of clinical trials and other research to promote recruitment, retention, and other outcomes. We recommend that researchers (1) Develop community collaborations and adapt the recruitment/retention procedures to the target group to increase the inclusion of REMs in research, and (2) Continue to develop more effective retention strategies for all groups. However, the evidence suggests that retention strategies that work for other groups may be ineffective for younger REMs. Therefore, directing some specific attention to improving retention among younger REMS appears necessary and (3) Conduct more research on when (e.g., in what treatment modalities, at what point[s] in treatment, with what subpopulations, and with what particular disorders), rather than if, ethnic matching improves outcomes.
The next recommendation is directed at our second objective (i.e., assessing measurement equivalence). Specifically, we recommend (4) that researchers establish measurement equivalence before using measures on a different group and especially before making conclusions about group differences. The last three recommendations address our third objective regarding data analytic plans. Specifically, to provide more accurate information on effective substance use treatments for REMs, we recommend that researchers use data analytic plans that (5) examine rather than assume the findings apply to specific REM groups, (6) avoid combining REM groups for data analyses, and (7) add within group analyses to detect key subgroup differences.