The purpose of this comprehensive review of literature was to identify instruments and measures of characteristics in Wallerstein et al.
] CBPR logic model . While the logic model has offered a consensus framework for CBPR, of necessity, the next steps were to identify measures that can be used to enhance our understandings of how community partners and academic–community partnerships contribute to changes in intermediate outcomes, such as culturally centered interventions, new practices and policy changes, which in turn have been increasingly recognized as leading to improved health outcomes and equity [2
The review of literature identified 273 articles related to CBPR and within these articles were 46 unique instruments containing items that measured some of the characteristics of the CBPR logic model. We then used the CBPR logic model to categorize the 224 measures within these 46 instruments and guided the interpretation and categorization of the instrument items.
While the model does not contain a unified change theory, theories at different levels along the socio-ecologic framework are embedded within the characteristics. The group dynamic measures available to CBPR, for example, have clearly benefited from the previously established and extensive coalition and organizational team literature [5
]. Many of these can be directly used for a study of CBPR. The literature on context measures was much more limited, except for the results related to community capacity (first as part of context and then as an outcome). The existence of community capacity measures is likely due to the long-time interest in the health education field in community capacity and its role in community change (see for example, Goodman et al.
]) and the newer interest in implementation science and the capacity of communities to adapt and adopt interventions and develop strategies to improve their own health. In a comprehensive review of the implementation literature, Fixsen et al.
] suggest that evidence-based interventions may have certain core elements that can be translated to multiple settings (e.g. principles and best-practice components/characteristics), but implementation success itself depends on context and flexibility to incorporate cultural and community knowledge and evidence. That is, the strategies for implementing interventions need to be based on and adapted within existing capacities and cultures of the organizations and communities in each setting [5
]. A recent commentary about how CBPR can support intervention and implementation research reinforces the importance of drawing from the context for enhancing external validity and potential sustainability of programs once the research funding ends [13
Although the number of measures for context was small, it is an important component that should be further researched. The CBPR logic model suggests that differences in context significantly influence processes, which form the core operational partnership features of CBPR [4
]. These differences such as levels of historic trust between partners or levels of capacity of communities to advocate for policy change or integrate new programs matter in terms of partnership success. Though capacity has been identified more in existing measures (largely from the coalition literature), a major area that is lacking are the contextual measures to assess research capacity of communities or the capacity of University and other research institutions to engage in respectful community engagement and CBPR processes.
The lack of measures related to the outcomes of CBPR is not surprising. The majority of CBPR literature to date has focused on case studies describing CBPR partnerships, research design and description of interventions, with articles reporting on how to create and maintain research partnerships, on methods that have been collaboratively adapted and adopted with community partners and on building trust among stakeholders. Impacts and outcomes attributable to CBPR have been less documented [14
]. However, as the field has matured, CBPR has begun to recognize and document the range of outcomes that can be attributed to CBPR partnerships and interventions. Case studies have increasingly documented policy and capacity changes, though links to health outcomes remain a challenge [1
]. Part of the challenge in linking CBPR to any sort of outcomes is the lack of valid instruments for documenting CBPR system and capacity outcomes and social justice outcomes, as well as the ongoing challenge of attributing change to any single effort (even if it fully encompasses the policy or intervention issue) within dynamic community contexts. These challenges will need to be met head on to advance the science of CBPR.
Currently, we are in the second year of a NARCH V National Institutes of Health (NIH) grant (2009–2013) to further explore how contexts and partnering processes impact interventions, research designs and intermediate system, capacity and health outcomes. Based on a partnership between the National Congress of American Indians Policy Research Center, the University of Washington Indigenous Wellness Research Institute and the University of New Mexico Center for Participatory Research, this research project is investigating the promoters and barriers to success in health and health-related CBPR policy and intervention research within underserved communities. This will be achieved through cross-site evaluation of CBPR projects nation-wide assessing variability in partnership processes, under varying contexts and conditions, which may be associated with system changes, new policies and other capacity-related outcomes. Plans currently include surveys for up to 318 sites, identified through NIH, Centers for Disease Control and Prevention and other databases; and six to eight in-depth case studies. The study is using many of the items identified here and constructing others to assess characteristics where measures are lacking. A variable matrix of the instrument items has been constructed which is linked to a web-based interactive version of the CBPR logic model (The interactive web-based matrix can be accessed at the following URL: http://hsc.unm.edu/som/fcm/cpr/cbprmodel/Variables/CBPRInteractiveModel/interactivemodel.shtml
In sum, the database of instruments and measures presented here provides what is currently available in the literature to advance the art and science of CBPR and demonstrates the need for continuing development of CBPR measures. In addition to the need for measures of context and outcomes in CBPR, there is a need for more work in assessing the reliability and validity of the measures. Overall, only about 25% of the measures evaluated had information regarding either reliability or validity. Further, the majority of measures that had this information only provided minimal information about validity. Future assessment tools of CBPR should address issues of validity [i.e. face, content and may consider use of factorial, construct (concurrent and discriminant) and possibility predictive validity] [15
]. Often instruments constructed for CBPR purposes do not address these validity issues because of the small sample sizes involved and their use for single partnerships.
Future research could establish validity of the measures through two approaches. First, measures could be tested on teams in clinical practice settings, agencies, not-for-profit organizations and inter-organizational coalitions or other alliances. While these partnerships may not be the same as CBPR partnerships, they would provide a reasonable comparison group to at least establish measurement validity. Second, CBPR researchers would benefit by pooling databases to establish a large enough sample size to test the validity of measures. Advancing the capacity to study CBPR will be contingent on strengthening the measurement quality of the measures being utilized.
This review of literature has some limitations. First, the search parameters resulted in a large number of articles but relatively few instruments (N
= 46). While we utilized a number of databases and ‘word of mouth’ solicitations and a systematic procedure in searching, there are likely instruments that were missed or are in development. Second, the matrix is the result of our interpretations of the measures and not the instruments’ original intent. While we demonstrated a consistent coding of instruments, we also realize that other researchers may assign certain instruments in different categories. Additionally, some researchers may find that measures we included are not effective measures of the characteristics. Rather than being a judge of the quality of the instruments, we chose to provide as much information about reliability, validity and population for researchers to make their own judgments. In addition, it was difficult to provide a judgment of quality when so few instruments include measurement validity. In general, we recommend that researchers consider instruments that have sufficient evidence of internal consistency (e.g. Cronbach’s alpha of 0.70 or above) and concurrent and factorial validity. In addition to using validity and reliability data, we do suggest two instruments for consideration based on the high face validity and in-depth use within their respective populations, Schulz et al.
] and Kenney and Sofaer [19
]. These instruments were carefully crafted with their respective partners and thus hold promise for future use. The validity and reliability of these measures will need to be assessed.
An additional limitation may be the challenge of replicating our search process because we incorporated other articles (see ‘outside search dates’ column in supplementary Tables) than those that were identified from our database search terms. Although the database search itself can be replicated, we also chose to incorporate later recommendations from CBPR experts and from other sources, recognizing this is an evolving field, which will require further research. For these other sources, we applied the same inclusion criteria however and believe that these instruments accurately represent the balance of process and outcome measurement tools.
The focus of this article has been on measures. However, we recognize that CBPR change cannot be fully captured only using quantitative instruments. Interviews and focus groups and other qualitative approaches enable partners to share their perceptions of how and why outcomes have changed, including enhanced community capacity or new organizational practices or policies, or how CBPR processes have transformed contexts for collaborative research; these qualitative approaches become a critical and necessary complement for enhancing the science [2
]. The opportunity for community partner reflections on their participation in the CBPR research (using the conceptual model as an evaluation and collective reflection tool) also enables a privileging of community voice as to which characteristics may have been more salient in any particular project’s success. Through community partners critically examining their lived realities in a dialogical encounter with researchers, a contextual and specific ‘face validity’ can emerge, which can only add to our knowledge base of how CBPR research processes can be generalized across communities.
In conclusion, this review of literature and categorization of instruments resulted in a large compendium of research tools for those interested in CBPR. A key aspect of advancing the study of CBPR is having adequate research tools and providing access to the range of tools available. This matrix provides a state of the art list, current up to 2008 about research instruments and should serve as a useful resource for academic and community researchers and practitioners of CBPR.