|Home | About | Journals | Submit | Contact Us | Français|
While sustained academic and community partnerships can improve relationships between research partners, they could also influence study outcomes. Research on this issue is limited.
We conducted a trial (2010–15) to test two implementation strategies for an evidence-based intervention to promote colorectal cancer (CRC) screening at community organizations in Los Angeles (N = 17). For both strategies, trained community health advisors (CHAs) recruited Filipino Americans (N = 673) who were non-adherent to CRC screening guidelines. The main study outcome was CRC screening status of participants at 6-month follow-up. This case study compares outcomes among organizations that had participated in our prior effectiveness trial and new organizations with which we had no prior relationship. Using multilevel logistic regression with multiple imputation for missing outcomes, we compared CRC screening rates among previous versus new partners controlling for study condition and organizational, CHA and participant characteristics.
Screening rates were substantially higher among participants of previous versus new partner organizations in unadjusted analysis (77% versus 55%, OR 2.8, p = 0.12), after adjusting for organization-level variables (81% versus 42%, OR 7.5, 95% CI [2.0–28.7], p = 0.003) and after additionally adding CHA and participant level factors to the model (79% versus 47%, OR 5.9, CI [1.3–27.3], p = 0.02). Analyses using complete cases and assuming not-screened for missing outcomes indicated similar differences in screening rates (30 and 33 percentage points, respectively).
Study outcomes that are achieved with long-term community partners may not be generalizable to new partners. However, inclusion of new community partners is important for external validity of dissemination efforts in community settings.
Many studies that promote cancer screening or other health behaviors are conducted in partnership with community organizations and are described as community-based participatory research, community-engaged research or community-partnered research (Israel et al., 2001, Holt et al., 2014, Scarinci et al., 2014). One of the key principles of community-based participatory research is a long-term commitment by all partners (Israel et al., 2001). The exact nature of this partnership varies among studies and is shaped by the setting and the context in which the study takes place; the relationship between academic and community partners; their history, if any, of working together; and the study protocol. The community-based participatory approach is often utilized in research with minority communities that may not be familiar with research, and may be difficult to enroll in a study without the contributions of a community partner they trust (Holt et al., 2014, Ma et al., 2009, Wang et al., 2012). Benefits of this research approach include improved quality and validity of research by incorporating the local knowledge of the people involved, and enhanced relevance and use of the research data by all partners (Israel et al., 2001).
We have conducted two large trials to promote colorectal cancer (CRC) screening in the Filipino American community in Southern California (Maxwell et al., 2010, Maxwell et al., 2016). In both trials, we partnered with a large number of community organizations. In the first trial (CRC1, 2004–2009), we developed a multi-component intervention to promote CRC screening and showed that it was effective in increasing CRC screening among members of community organizations (Maxwell et al., 2010). In the second trial (CRC2, 2010–2015), we tested two strategies – a basic and an enhanced strategy – to promote the implementation of the previously developed intervention by community organizations with the help of trained community health advisors (CHAs). Previous analysis of CRC2 data found that participants reported high screening rates at 6-month follow-up in both arms of the study with no significant difference by implementation strategy (Maxwell et al., 2016).
By design, about half of the organizations that participated in CRC2 had also participated in CRC1 and therefore had a prior relationship with our research group. Presumably, these organizations also had some prior knowledge, capacity and positive values regarding CRC screening and an ongoing commitment to promoting CRC screening in their community. We included them in CRC2 in order to provide ongoing technical and financial support, which is crucial for sustaining health promotion efforts of community organizations (Israel et al., 2006).
While a sustained academic and community partnership can improve the working relationship and trust between research partners, it could potentially influence study outcomes and could have important implications for the generalizability of implementation and dissemination of evidence-based interventions. However, we are not aware of any studies that have examined this issue.
This analysis explores the effect of being a new versus a previous research partner on the main study outcome, CRC screening status of participants at 6-month follow-up. We hypothesized that organizations that had partnered with us in a previous study to promote CRC screening by hosting the intervention and helping with recruiting subjects may be more successful in promoting CRC screening than new partners. Guided by the Consolidated Framework for Implementation Research (CFIR), we also explore the influence of other organizational characteristics, including our other stratification variable, faith-based versus social service organization. In addition, we explore the influence of CHA characteristics (e.g., professional background) on CRC screening of participants, since these characteristics may also play a role (Damschroder et al., 2009). This analysis adds to the literature by examining the potential impact of sustained academic-community partnerships on attempts to implement an evidence-based intervention into community practice.
In a prior randomized trial in partnership with 45 Filipino American organizations and churches (CRC1), we determined that an intervention that included an educational session on CRC screening, distribution of free fecal occult blood test (FOBT) kits and print materials, referral of uninsured participants to a community clinic that had agreed to evaluate FOBT kits and charge the study, and a reminder to get screened significantly improved CRC screening among Filipino Americans (Maxwell et al., 2010). The current trial (CRC2) tested two strategies (basic and enhanced, see below) for implementing this multi-component intervention at 17 community organizations with the help of trained CHAs. CHAs recruited Filipino Americans between 50 and 75 years of age who were not adherent to CRC screening guidelines. CHAs administered informed consent and a baseline questionnaire and implemented the intervention.
Restricted randomization (Hayes and Moulton, 2009) was used to promote balance on zip code-level mean income and education both across conditions and within each of four cells defined by two stratification variables, new versus previous community partners and faith-based versus social service organization. The study was approved by the University of California Los Angeles Office of the Human Research Protection Program. Additional details and a CONSORT flow diagram of the trial have been reported elsewhere (Maxwell et al., 2016).
Based on the CFIR and our research questions, we explored the following baseline variables as predictors of CRC screening among participants at follow-up.
Basic versus enhanced implementation (study condition): Organizations and CHAs in each condition received the same amount of training and financial incentives. Organizations that received the enhanced implementation strategy were encouraged to implement additional activities to promote CRC screening among their members, such as celebrating National CRC Awareness Month. They also received three additional site visits in which research staff answered questions and helped to trouble shoot problems with recruitment and intervention implementation. In addition, only leaders of organizations in the enhanced arm participated in workshops and the study's Community Advisory Board.
New versus previous research partner (stratification variable): By design, about half of the organizations had been our partners in the effectiveness trial (CRC1) that preceded this implementation trial. Therefore, these sites had been exposed to the importance of CRC screening and the intervention, and they knew the Filipino American project director, who was responsible for day-to-day activities in both CRC1 and CRC2. New organizations were recruited through online sources and a Filipino Consumer Guide.
Faith-based versus social service organizations (stratification variable): By design, about half of the organizations were faith-based (Catholic churches) and the remainder were social service organizations such as senior centers or adult day care centers.
Organizational readiness for implementation of the CRC screening promotion program: Readiness for implementation is an important component of the inner setting in the CFIR. It consists of access to information and knowledge, leadership engagement and available resources. Organizational readiness was assessed using a baseline questionnaire that was completed by leaders at each organization, with 9 items that were rated from very low (1) to very high (10). Leaders reported the organization's knowledge and awareness of CRC and CRC screening, engagement in the program (interest in prevention of CRC in the Filipino American community; concern for members at risk for CRC; level of preparedness to promote CRC screening; degree of feeling empowered to promote CRC screening among its members), and available resources for CRC screening related activities. The 9 item instrument was developed for this study, based on the work of Plested and colleagues (Plested et al., 2006), had a Cronbach's alpha of 0.94 and factor analysis identified factors consistent with theoretically meaningful dimensions described above.
Other organizational variables: As additional predictors, we examined the estimated number of Filipino Americans served by each organization; years in operation; and the number of health related activities other than promoting CRC screening that were conducted by the organization in the past 6 months that required some degree of planning (e.g., serving fruits and/or vegetables during events; having classes on healthy nutrition or exercise).
Characteristics of individuals who implement the program may influence outcomes (Damschroder et al., 2009). In addition to demographic variables (gender, age, education), we examined whether or not CHAs had a health care background (e.g., nursing, dentistry) and if they had ever received CRC screening. We also considered how many hours per month they reported that they could devote to promoting CRC screening.
In addition to demographic variables (gender, age, education), acculturation (length of US residence), and health care access (health insurance), we examined whether or not participants had ever had a CRC screening test in the past. All of these variables can influence receipt of CRC screening (Beydoun and Beydoun, 2008, Christy et al., 2013, Larkey et al., 2015, Manne et al., 2015, Savas et al., 2015).
The outcome variable, participant receipt of CRC screening within 6 months after receipt of the intervention (yes/no), was assessed through 1) a 6-month follow-up telephone interview by members of the research team who were not involved in intervention implementation and 2) a list of participants for which the participating clinic billed the study for FOBT lab analyses. There was excellent agreement between these methods (Maxwell et al., 2016).
Analyses were conducted using Stata 13.1. Organization-level variables were: study condition (basic/enhanced); stratification variables used for randomization (previous research partner/new; faith-based/social service organization); number of members (500 or more/under 500); years in operation (25 or more/under 25); number of health-related programs (0 or 1/2 or more), and organizational readiness. Organizational readiness was categorized as high (score > 5.5) or low (score ≤ 5.5) for logistic regression analyses. CHA-level variables were: age (60 years or older/younger than 60); gender; education level (graduate work/less); health professional background (yes/no), and ever screened for CRC (yes/no). Participant-level variables were: age (60 years or older/younger), gender; education level (college graduate/not), ever screened for CRC (yes/no), has health insurance (yes/no), and length of US residency (20 years or more/less).
We examined the association of the stratification variables, previous partner versus new and social service versus faith-based organization, with other organizational characteristics and with CHA and participant characteristics using Fisher exact-tests (when the unit of analysis was organization) or multilevel logistic regression models (when the unit was CHA or participant). Next, we conducted bivariate analyses examining the association of each predictor with the CRC screening outcome by fitting multilevel logistic regression models for each predictor singly. From these models we also obtained estimates of the proportions of participants obtaining CRC screening at each covariate level that accounted for clustering using predictive margins. Subsequently, we fit two multivariable logistic models: Model 1 included only variables at the organization level and Model 2 included organization, CHAs and participant variables. At the organization level, for parsimony, we included only the main variables of interest (previous research partner/new; faith-based/social service organization), study condition, and organizational predictors with bivariate p-value < 0.25. At the CHA level, we similarly included only predictors with bivariate p-value < 0.25. Due to adequate sample size at the participant level, all participant characteristics of interest were included. From these multivariable models we used predictive margins to obtain estimates of the proportions of participants obtaining CRC screening at each covariate level that adjusted for other variables and for clustering. Missing outcomes were multiply imputed using chained equations (Raghunathan et al., 2001). Analyses were repeated on complete cases and assuming not-screened for missing.
Out of 22 randomized organizations that had agreed to partner with us, 17 implemented the program. Of the five organizations that dropped out (due to scheduling problems and being busy with other priorities), four were new research partners and four had been randomized to the basic arm. Of 673 participants enrolled, 593 (88%) provided complete information on predictors of interest and were included in analyses. Ninety-one of these participants (15%) were missing on the screening outcome.
As shown in Table 1, about half of the organizations had > 500 members, had been in operation for at least 25 years and had implemented 2 or more health-related activities in the past 6 months. The majority of CHAs was 60 years or older and female. A substantial proportion of CHAs had a health professional background (49%), could devote 10 or more hours per month to CRC promotion (58%) and had been screened for CRC in the past (77%). Participants were predominantly female (63%) with a college degree (67%). Only 60% had health insurance and only 27% had ever received CRC screening at baseline.
As shown in Table 2, new partners and previous partners were similar on many organizational characteristics; however, new partners included more social service organizations (71%) and previous partners included more faith-based organizations (80%), an imbalance that was caused by the drop out of five organizations. In addition, comparison of social service and faith-based organizations showed that social service organizations had been in operation for fewer years and had smaller memberships. Organizational readiness to promote CRC screening was similar among new and previous partners and in both types of organizations, ranging from 5.7 to 6.1 on a 10-point scale. No differences at the CHA or participant level were observed for the two stratification variables (see Appendix).
Using multiple imputation for missing outcome data, the 17 organizations achieved CRC screening rates at 6-month follow-up ranging from 31% to 100%, a threefold difference (median 61%). Table 3 shows the bivariate relationships between organizational, CHA and participant characteristics and participant CRC screening outcomes. Previous research partners tended to achieve higher screening rates than new partners (77% versus 55%, a 22 percentage point difference). In addition, higher screening rates were observed among social service organizations than among faith-based organizations (77% versus 63%), and among organizations that had been in operation for < 25 years versus longer (79% versus 59% screened). However, the number of organizations was small and none of these differences reached statistical significance. Screening rates were almost identical for organizations who reported high versus low readiness to promote CRC screening.
Female CHAs achieved higher screening rates among participants than male CHAs (72% versus 56%, p = 0.05). CHA age, level of education, health professional background, history of CRC screening and willingness to devote over 10 h per month to CRC screening promotion were not significantly related to screening rates, although education was close to significant (p = 0.06). Among participants, those who had ever received CRC screening prior to the study were significantly more likely to report screening than those who never had any CRC screening (78% versus 66%, p = 0.01).
In a multivariate analysis that only included variables at the organization level, previous research partners had significantly higher CRC screening rates among their participants than new research partners, although with a wide confidence interval (81% versus 42%, OR 7.5, 95% CI [2.0–28.7], p = 0.003). After adding CHA and participant level factors to the model, the adjusted CRC screening rates remained significantly higher for previous research partners than for new partners (79% versus 47%, OR 5.9, 95% CI [1.3–27.3], p = 0.02). Screening rates for social service organizations were also higher than for faith-based organizations (82% versus 57%, OR 5.1), but the difference was not statistically significant (p = 0.10). The predictors at the CHA and participant levels that were significant bivariately (or almost so) were significant multivariately: CHAs who were female and had no college degree and participants who had ever received CRC screening previously had higher odds of screening.
When Multivariate Model 2 was fit using complete case analysis, previous research partners had significantly higher CRC screening rates among their participants than new research partners (80% versus 47%, OR 6.2, 95% CI [1.0–38.0], p = 0.05); when not-screened was assumed for missing outcomes, the difference was somewhat attenuated (70% versus 40%, OR 4.7, 95% CI [0.8–28.4], p = 0.09).
Confirming our hypothesis, our findings suggest that an ongoing research partnership between academics and community organizations can affect study outcomes and that outcomes that are achieved with long-term community partners may not be generalizable to new partners. Interpretation of this finding is challenging because new and previous partners reported similar levels of organizational readiness at baseline, including knowledge and awareness about the importance of CRC screening, interest in preventing CRC in their community, and resources and preparedness to actively engage in CRC screening promotion. It is likely that unmeasured characteristics of new and previous partners were the key drivers of study outcomes. In addition, relationships between previous research partner organizations and members of the study team that had developed during CRC1 may have affected the implementation and the outcomes of the study. Another possible explanation is related to self-selection bias: All organizations that participated in the study volunteered for this task; however, this self-selection bias may have been greatest among previous partner organizations that may have had a better understanding of what would be required to implement the program.
Regardless of why screening rates were higher among our previous research partners, this finding has important implications for studies in community settings. Academic investigators who repeatedly partner with the same community organizations in health promotion efforts need to be aware of the possibility that their findings may not generalize to organizations that do not have this research history and/or the relationship with the research team. Thus, there may be a conflict between building sustainable research partnerships in the community, which requires long-term support and funding for organizations, and conducting research that yields findings that are generalizable more broadly. It should also be noted that partnering repeatedly with the same community organizations may increase the risk of “burn-out” among these organizations, while neglecting other organizations that may benefit from a new partnership.
The fact that faith-based organizations achieved lower screening rates than non-faith-based organizations was unexpected since 8 out of 10 faith-based organizations were prior partners, and since 9 out of 10 had a health ministry, an existing structure, that could potentially support health promotion efforts. More research is needed to confirm and better understand this observation. Overall, our findings suggest that organization level variables can substantially influence study outcomes, even after controlling for variables related to the implementers (CHAs) and the participants. A better understanding of these organization level factors may help to achieve better outcomes in various research settings.
As more and more programs utilize trained CHAs for health promotion (Holt et al., 2014, Santos et al., 2014, Shelton et al., 2016), the question arises as to who is most suitable to serve in this role. Our data suggest that female CHAs achieve the highest CRC screening rates among participants. Filipino Americans are unique in that this community has many health professionals such as nurses. However, the fact that health professional background of CHAs did not predict CRC screening among participants suggests that community members with a wide variety of educational and professional backgrounds can be trained to successfully serve as CHAs and promote cancer screening among their peers.
Because we controlled for individual-level predictors, organizations that dropped out and did not recruit any subjects were not included in the analysis. We only included participants who had complete data on the predictor variables entered into the multivariate models. However, included and excluded participants were not significantly different with respect to demographic characteristics, acculturation, health insurance or history of CRC screening. In addition to the factors measured in our study, numerous unmeasured factors could potentially influence screening outcomes, such as the position of CHAs within organizations or the relationship between a CHA and participant.
Although a sustained academic and community partnership is considered beneficial for community research (Israel et al., 2010, Israel et al., 2001), our findings suggest that study outcomes that are achieved with long-term community partners may not be generalizable to new partners. We recommend further examination of this issue in future studies. In particular, more understanding is needed regarding the nature of the relationship between the implementing organization and the entity that provides resources, and the monitoring and evaluation (often the academic partner). Inclusion of this construct in theoretical formulations should also be considered. Inclusion of organizations that do not have an ongoing research relationship with an academic partner may improve external validity of trials in community settings.
No financial disclosures were reported by the authors of this paper.
This research was supported by Grant RSGT-04-210-05-CPPB from the American Cancer Society and by the UCLA Kaiser Permanente Center for Health Equity. CMC was also supported by NIH/NCI grant P30 CA16042 and AEM by U54-CA143931. We would like to thank the members of the Filipino American community who participated in this study.
The Transparency document associated with this article can be found, in online version.
|Characteristics||New partner||Previous research partner||P||Faith-based||Social services||P|
|CHAs (N = 70)|
|60 years or older||15 (58%)||29 (66%)||26 (62%)||18 (63%)|
|< 60||11 (42%)||15 (34%)||16 (38%)||10 (36%)|
|Female||20 (77%)||38 (86%)||36 (86%)||22 (79%)|
|Male||6 (23%)||6 (14%)||6 (14%)||6 (21%)|
|Graduate work||14 (54%)||20 (45%)||18 (43%)||16 (57%)|
|College degree or less||12 (46%)||24 (55%)||24 (57%)||12 (43%)|
|Health professional background||0.50||0.24|
|Yes||14 (46%)||20 (45%)||18 (43%)||16 (57%)|
|No||12 (54%)||24 (55%)||24 (57%)||12 (43%)|
|Hours can devote per month||0.29||0.65|
|10 or more||12 (48%)||27 (64%)||24 (60%)||15 (56%)|
|< 10||13 (52%)||15 (36%)||16 (40%)||12 (44%)|
|Yes||20 (77%)||34 (77%)||34 (81%)||20 (71%)|
|No||6 (23%)||10 (23%)||8 (19%)||8 (29%)|
|Participants (N = 593)|
|60 years or older||116 (51%)||202 (55%)||188 (53%)||130 (55%)|
|< 60||110 (49%)||165 (45%)||168 (47%)||107 (45%)|
|Female||132 (58%)||240 (65%)||222 (62%)||150 (63%)|
|Male||94 (42%)||127 (35%)||134 (38%)||87 (37%)|
|College graduate||156 (69%)||241 (66%)||249 (70%)||148 (62%)|
|No college degree||70 (31%)||126 (34%)||107 (30%)||89 (38%)|
|Ever received CRC screening||0.19||0.99|
|Yes||54 (24%)||108 (29%)||96 (27%)||66 (28%)|
|No||172 (76%)||259 (71%)||260 (73%)||171 (72%)|
|Has health insurance||0.93||0.34|
|Yes||136 (60%)||220 (60%)||122 (51%)||234 (66%)|
|No||90 (40%)||147 (40%)||115 (49%)||122 (34%)|
|20 or more years||104 (46%)||155 (42%)||169 (47%)||90 (38%)|
|< 20 years||122 (54%)||212 (58%)||187 (53%)||147 (62%)|
P-values were obtained using multilevel logistic regression models.