|Home | About | Journals | Submit | Contact Us | Français|
Community Action Against Asthma (CAAA) is a community-based participatory research (CBPR) project that assesses the effects of outdoor and indoor air quality on exacerbation of asthma in children, and tests household- and neighborhood-level interventions to reduce exposure to environmental asthma triggers. Representatives of community-based organizations, academia, an integrated health system, and the local health department work in partnership on CAAA's Steering Committee (SC) to design and implement the project.
To conduct a process evaluation of the CAAA community–academic partnership.
In-depth interviews containing open-ended questions were conducted with SC members. Analysis included established methods for qualitative data, including focused coding and constant comparison methods.
Community setting in Detroit, Michigan.
Twenty-three members of the CAAA SC.
Common themes identified by SC members relating to the partnership's ability to achieve project goals and the successes and challenges facing the partnership itself.
Identified partnership accomplishments included: successful implementation of a complex project, identification of children with previously undiagnosed asthma, and diverse participation and community influence in SC decisions. Challenges included ensuring all partners' influence in decision-making, the need to adjust to “a different way of doing things” in CBPR, constraints and costs of doing CBPR felt by all partners, ongoing need for communication and maintaining trust, and balancing the needs of science and the community through intervention.
CBPR can enhance and facilitate basic research, but care must be given to trust issues, governance issues, organizational culture, and costs of participation for all organizations involved.
In recent years, there have been growing calls for a more community-based participatory approach to health research as researchers have recognized the value of including the intended beneficiaries in the planning, implementation, and evaluation of research.1,2 Although this has resulted in an increasing application of community-based participatory research (CBPR), much of the literature on CBPR addresses topics such as the conceptual underpinnings, design issues, methodology, and results of this approach to research and action.1–6 There is little in the published literature that evaluates the process of conducting research using a partnership approach. Process evaluation questions are rarely asked that examine partnership issues such as: How do the researchers and community participants work together, make decisions, and negotiate? What are the benefits and challenges of doing this type of research for all parties involved? What lessons can be learned through reflection on these partnership processes to guide the development of successful university–community collaborations? A better understanding of these questions is needed to ensure the effectiveness of CBPR as a key strategy for conducting basic and intervention research.
In this article, we present results of a process evaluation of the Community Action Against Asthma (CAAA) project that used qualitative methods. CAAA is an epidemiological and intervention research study of environmental influences on asthma in urban children that is being conducted within a community-based participatory research partnership. The purpose of this article is to describe and analyze the CAAA partnership process with a focus on how it developed and evolved. We examine the evaluation results, identifying the challenges the partnership has experienced, the successes it has achieved, and the lessons that have been learned about CBPR.
The Community Action Against Asthma project grew out of an already existing community-academic partnership, the Detroit Community-Academic Urban Research Center (URC).7,8 In 1998, as a result of identifying childhood diseases related to the environment as a priority area for future action, the URC Board submitted a grant proposal with a focus on environmental triggers of childhood asthma and received funding to establish the Michigan Center for the Environment and Children's Health (MCECH). CAAA is a project of MCECH and focuses its research activities in neighborhoods in Eastside and southwest Detroit that were selected initially by the URC. The east side of Detroit is predominantly African American, and southwest Detroit is the area of the city in which the largest percentage of Latinos reside.9 To identify eligible participants for CAAA, we mailed self-administered screening questionnaires to parents of all children aged 6 to 10 enrolled in the Detroit public elementary schools in the east side and southwest neighborhoods involved. Three hundred twenty-eight families agreed to participate in the project after one of their children was identified through the questionnaire as having symptoms of moderate to severe or mild persistent asthma (TCL, unpublished data, 2003). CAAA is in its last year of a 5-year funding cycle.
The epidemiological component of CAAA examined the combined effects of indoor and outdoor air quality on childhood asthma exacerbation. Data collection included continuous measurements of ambient PM2.5 (particulate matter less than 2.5 μm in diameter), ozone, and meteorological parameters gathered on the rooftops of 2 community school sites. In addition, for a 2-week intensive period during each season, the following data were collected: 1) PM10 (particulate matter less than 10 μm in diameter) and PM2.5 measurements made indoors at a subset of 15 of the participating households and at the 2 community schools; 2) continuous daily personal monitoring of the PM10 exposures of the children in the subset of 15 households; 3) health outcome data collected for all the children enrolled, including a daily diary of symptoms and medications used, and morning and evening lung function assessment for a subsample of the children.
The intervention research component included both household- and neighborhood-level interventions to reduce exposure to the triggers of childhood asthma. The household intervention consisted of a minimum of 12 visits over a 2-year period by a “Community Environmental Specialist” (CES), a community outreach worker. During these home visits, the CES provided education and materials that relate to the reduction of exposure to asthma triggers (e.g., integrated pest management, vacuum cleaners, mattress covers), and referrals for a range of issues, such as medical care and tenant rights. In the neighborhood intervention, community organizers worked with community residents to reduce neighborhood-level physical and psychosocial environmental stressors associated with childhood asthma.
For the intervention, we used a staggered randomized research design in which one half of the participants received the household intervention immediately after the collection of baseline data and the other half received the intervention the following year. Annually, we conducted a household environmental assessment and administered questionnaires with the caregivers and children to assess health outcomes (e.g., asthma symptoms, quality of life) and psychosocial factors (e.g., social support, stressors) to evaluate the impact of the household intervention on asthma outcomes. Outcome evaluation of the intervention is currently underway.
The CAAA Steering Committee (SC), comprised of representatives from community-based organizations (CBOs) and agencies, the local health department, an academic institution, and an integrated health care system, guides the work of CAAA. CAAA Steering Committee representation is based on organizational affiliation, and each organization identifies, at minimum, a primary and alternate representative for the SC. CAAA has 1 SC member (WB-C) who does not have an organizational affiliation. This member expressed interest in and was asked to remain on the SC as a volunteer after retiring from her organization. The University of Michigan School of Public Health normally has more than 2 organizational representatives attending SC meetings, because their presence is required to answer and explain different aspects of the research activities.
The 8 original partner organizations involved in the URC Board agreed to be on the CAAA Steering Committee. Based on the need for additional expertise in epidemiological asthma research and environmental and housing issues, these original organizations identified 5 new partner organizations for the SC (see Table 1 for adescription of SC partner organizations).
The SC meets monthly and is co-facilitated by members of the university at the initial request of the SC. At the beginning of the project, the SC established operating norms through brainstorming and discussing characteristics of effective groups. Examples of some of the norms adopted include starting and ending meetings on time, being respectful of others' opinions and experiences, having equal participation in meetings from all members, and using a consensus model of decision making.
The SC adopted the URC's set of community-based participatory research principles to guide the research of CAAA (see Table 2). In accordance with these principles, and as described in Table 3, the SC has been actively involved in all major phases of the research and intervention.
Evaluation researchers suggest the value of a process evaluation to determine the extent, fidelity, and quality of intervention implementation.10 In contrast to outcome evaluation, which examines an intervention's influence on health outcomes, process evaluation focuses on examining how an outcome is achieved, e.g., the internal dynamics of program operations.10 We report here the results of one component of the CAAA process evaluation: qualitative interviews conducted with SC members to evaluate how well CAAA is implementing a CBPR approach. Qualitative in-depth interviews are recognized as an effective method for answering process evaluation questions.10,11
Face-to-face interviews were conducted with 23 of the 25 SC members. Of the 23 members interviewed, 12 were community organization representatives and 11 were affiliated with the University of Michigan. Data from 22 interviews are included in this report. The data from one full interview and a portion of a second interview were lost due to a malfunctioning tape recorder and thus could not be analyzed. The questions asked in these in-depth interviews focused on strengths of the partnership and challenges it has faced, relationships and communication among partners, the project's adherence to norms of operation and the principles of CBPR, the participation of SC members in the project, and suggestions for the improvement of project activities and operations. Furthermore, information about the context of the project and recommendations for the project's future were collected.
The Institutional Review Board of the University of Michigan approved the interview process, and all interviewees gave their consent to be interviewed. The interviews were conducted from June until September 2000 by a School of Public Health graduate student who was not a member of the Steering Committee but a paid research assistant with the CAAA project. The interviews took an average of 1 hour to complete.
The interviews were tape recorded with members' permission and transcribed. The interviews were then analyzed using a focused coding method12,13 by a 3-member team that included the graduate student who conducted the interviews and 2 School of Public Health faculty members involved in CAAA. The text was divided into in vivo restatements13 that were assigned to focused code categories (e.g., challenges, accomplishments). These categories and the data assigned to them were then analyzed using a method of constant comparison12 for the identification of more specific themes. These themes, and the quotations that exemplified them, served as the basis for a written report that was shared and discussed at an SC monthly meeting for members' feedback. This feedback was then incorporated into a final report, the results of which are reflected in this article. The SC is presently considering action strategies that are needed to address some of these evaluation results.
Given that the data for this article were collected at the end of the second year of the grant funding cycle, this article will focus on the first 2 stages of partnership development, the formation/mobilization stage and the implementation stage.14–16 Within each stage, a discussion of major accomplishments and challenges of the partnership identified by SC members will be presented.
Steering Committee members were very positive in their description of the formation/mobilization stage. They noted the sense of accomplishment of getting the project started and having an SC with good working relationships, diverse representation in terms of ethnicity, organizational representation, and member interest, and committed members.
The biggest accomplishment is that we have a very good working relationship between the community partners and the people from [the university] and that's probably the single most important thing. If that hadn't happened nothing else would have worked.3
They also were impressed with the relatively quick time that it took for the SC to come together and establish a level of trust and good communication. One community partner noted that CAAA “had come to speed” a lot quicker than the other university-affiliated projects with which she had been involved. Several participants attributed this quick time in establishing trust to the previous relationships that existed among many of the SC members through their URC participation.
Despite the satisfaction with the SC membership and performance, some participants did raise concerns about the composition of the SC, suggesting that in the future, the SC might recruit new members to: strengthen its ties with local health care systems; include more representation from parents of children with asthma; increase community representation from specific ethnic groups and geographical locations within the neighborhoods involved; and increase collaboration with other local universities and colleges. However, some participants cautioned that there were some risks, such as loss of the group's focus, associated with adding too many partners and having too many different viewpoints represented.
Accomplishments of the research and intervention implementation stage included: the recruitment of 331 families to participate in the project; the collection of data (including the amount of data collected, establishing methods for collecting the data, and building the machinery and instruments for collecting the data); the hiring and training of all staff; and the identification, through the screening instrument, of previously undiagnosed children with asthma.
Steering Committee members identified the crucial role of the community partners in the success of this phase. Through the input of the community members on the SC, university researchers felt they were better able to tailor the research to be more sensitive, acceptable, and beneficial to the participating families. Community partners themselves identified their role and influence in implementation decisions, as noted by this community partner:
I think we [community partners] have influence in the writing of the grant. I think we have influence in picking the community partners. You know we help make a decision about that. I think we had influence in hiring the CESs that we got. I think that we had influence in what the design of the questionnaires was going to be because we went through drafts of those like nobody's business. I think that we have a lot of input into what is going to be disseminated, how it's going to be disseminated, how these papers are going to be written, what it's going to look like. I think we have a lot of input into that and so I think that the community has been involved in every step of the way, really.… So, I feel like we have been empowered really.
Participants generally thought the project was doing a good job of adhering to the CBPR principles; however, some suggested there was room for improvement in involving community members in the decision-making process.
Participants noted that the project had begun to disseminate some of the data findings to the SC and at national conferences, and that they were pleased to see community members participating in the dissemination process and were genuinely interested in the findings:
You know lately I think the best part [of SC meetings] has been the reports from the Air Watch systems, and the presentations from the [university] … you know they've been doing several presentations for the last meetings and I thought those were most informative.
There were benefits also to the university and university partners that were identified that included the following: the opportunity to conduct the research itself; funding (including 52% in indirect costs that went directly to the university); publicity and recognition for doing community work; recognition received for using CBPR within the School of Public Health as a result of securing a large federal grant; opportunities for interdisciplinary learning; the data collected and ability to disseminate findings, which translates into career advancement and recognition for the university members; and opportunities for students and researchers to be involved in a CBPR effort.
Steering Committee members identified the following benefits to CBOs: the research lends credibility to other activities CBOs have been working on; increased knowledge and understanding of asthma; ability to disseminate the information to the community; participation in conferences and the recognition CBOs receive for that participation; involvement in something that benefits the community; and the opportunity to contribute programmatically to CAAA.
Steering Committee members identified numerous challenges related to the process of implementing the partnership and project. Challenges included: 1) for both community members and researchers, learning how to conduct research and work together in different ways than they were used to; 2) constraints and costs of doing CBPR across different organizational cultures; 3) ensuring community member participation in day-to-day research and governance decisions; 4) the tension between the time, energy, and resources spent in conducting basic research and the intervention component; 5) feeding back research data in a timely and understandable fashion; 6) sharing information and learning ways to communicate with each other; and 7) maintaining trust. Due to space limitations, only 4 of these challenges that have been particularly salient to the CAAA partnership are explored in depth.
Steering Committee members identified several costs and organizational constraints that could hamper their involvement in CBPR projects, and noted the importance of making each other aware of those costs and constraints. Although all partners faced costs, community partner organizations appeared to face more financial costs from involvement both in terms of reimbursement for their time spent participating as well as opportunity costs for taking time away from grant writing and other job responsibilities. The costs for the university partners were more specific to individual researchers' professional costs resulting from the time and intensity of CBPR and how that might affect tenure and promotion decisions.
Steering Committee members articulated the importance of knowing and considering the constraints under which the various partners operate. For example, a community partner expressed the following about the importance of understanding faculty members' constraints:
A challenge is not being clear about each other's expectations, clarity for roles and responsibilities…. There was some tension around who got to be out front in terms of the research data releasing and that kind of stuff. And as a community person looking on I saw that it was a real value on the university's side to be the one to be able to say, “This is my piece of work.” And it took a lot of time just going through that, and if you don't understand that people are looking for tenure, and they want to be recognized for their research—I understand it, but that's a norm that is not something on our [community organization's] end.
Participants discussed the challenges of ensuring community member participation in the day-to-day research and governance activities, given that the SC only met once a month and that decisions must sometimes be made quickly in between the monthly meetings. One example of this challenge concerned a SC decision about participant incentives that was modified by university partners outside of a meeting without involving or informing the other members of the SC in a timely way. As one partner noted:
When those kinds of issues happen [now] Committee folks or partners have to be apprised of that as soon as possible…. Even though we don't have any legal obligation we do have our reputation to think about and once you lose your reputation you're pretty much done.
After this incident, the SC developed a regular process for handling similar situations when they arose. SC members identified other areas in which community partners have less day-to-day involvement and control, including facilitation, financial decisions, and the more technical and scientific aspects of the project.
Another challenge identified by participants was the tension between the time, energy, and resources spent in conducting basic research in the form of data collection, and providing services to community members, in the form of the intervention. SC members noted that a several-month delay in the start of the intervention gave the impression to some of the community SC partners that the exposure assessment research mattered more than the intervention research. Noted one community member:
Most folks in the community wanted intervention right away. You know they want a cure for asthma whereas the researchers want an understanding or something to that effect. So giving a middle ground between those two points has been a challenge.
Some SC members noted incidents in which it appeared that partners did not trust one another, and noted that the incorporation of new community and academic partners might have affected trust.
A lot of us knew each other before this particular project through similar projects, but still coming together as a group in this project has been a challenge. We still have that situation of mistrust between the university and the group because we obtained some new players from the university…. The cohesiveness of the group—I think that groups have to just merge. I don't think there's anything we could have done different.
Several participants mentioned the writing of a grant proposal and the challenges associated with maintaining trust during this process. In the second year of CAAA, the SC decided to respond to a request for proposals from the National Institute of Environmental Health Sciences to expand the community organizing intervention component of CAAA. A subcommittee of the SC was formed consisting of university and community-based organization representatives from east and southwest Detroit. The group decided on a community-organizing and policy-advocacy strategy that would include hiring 2 neighborhood/community organizers and 1 policy organizer. The university partners strongly urged that the organizers be hired as university employees housed at 3 of the CBO partner organizations and co-supervised by the university and the SC members from the host CBOs. While this was ultimately the strategy that was agreed upon, some of the community partners felt the university should have pursued an arrangement in which the organizers would be employees of the CBOs and solely under their supervision. Noted an SC member from a CBO:
And I think that the thing that strikes me during that process …the two words that sort of jumped out at me, and that was lack of trust, and lack of respect for our [community partners'] work. And no matter how folks spin it, it still comes out the same way: lack of trust and lack of respect for one's work… Because …the ideal model in my mind would have been to subcontract with the community partners to do the work, and then manage the relationship with the community partner based on…what you needed for them to do, right?
University partners acknowledged that although the requirements of the grant precluded funding going directly to a CBO, subcontracts could have been used. As the following comments note, university partners expressed their concerns about not having control over employees for whose performance they would ultimately be accountable. From their perspective, this issue of control, rather than trust, was the basis of their point of view.
We had a number of discussions with community partners around the staffing of community organizers … and there were some tough discussions…and it was all about control needs, ours and theirs, and I think we laid those out … But I think there's still some ruffled feathers….and I think that's a challenge around control issues and particularly when it has to do with an intervention that you want to be able to supervise folks and you want to be able to make sure that things are happening that need to according to a fairly standard, rigid protocol and not that it couldn't happen if they're CBO employees, but also some concerns if that is the case that they get pulled into other activities in their organization that aren't related to the project which was our main reason for going that route.
Based on the results of the evaluation data presented here on the experiences of the CAAA project, we offer a number of lessons learned and recommendations for establishing and maintaining effective CBPR partnerships.
Community Action Against Asthma differs from many CBPR projects in that it developed from an existing partnership, the URC. Thus, instead of forming a project around a funding opportunity, CAAA took advantage of the funding opportunity to form a project around an already identified community need. This allowed community partners to participate in the actual development of the research and intervention design and grant proposal-writing activities. This also facilitated dialog among community and university SC members about the advantages and disadvantages of using a randomized staggered research design and resulted in CAAA being the first URC project to use a “controlled” design. In addition, because many of the CAAA partners had worked together previously with the URC, there was already an existing level of positive working relationships and trust that facilitated CAAA's ability to move quickly from its formation stage to its implementation stage (approximately 4 months), a relatively brief time period compared with other examples from the partnership/coalition literature.2
The CAAA's ability to build upon the URC's formation stage speaks to the need for funders to build in time and support to focus on the “process” of building a research partnership before focusing on the “outcomes” (e.g., research, intervention) of the partnership. This could be done through the funding of planning grants and/or grants to specifically support infrastructure development.17
Despite the establishment of a fair level of trust in the formation stage of the project, when CAAA moved into the implementation phase, issues of lack of trust between the community partners and the university partners, especially around the community organizing grant, began to surface. CAAA's experience demonstrates the danger in assuming that trust, once established, is easily maintained. Instead, partners engaged in CBPR must continue to monitor issues of trust and influence throughout all of the stages of development of the project, and continually review and re-assess issues of governance and decision making.
The inclusion of process evaluation activities is crucial to ensuring that important issues such as trust, influence, governance, and decision making are properly monitored and changed as needed. The use of qualitative in-depth interviews can allow SC members to express their opinions and feelings confidentially. A key component of the process evaluation procedure is ensuring time for feeding back the data and reflecting upon that data. For example, the evaluation interviews allowed community partners to voice their frustrations and concerns about the community organizing grant. This allowed the partnership to engage in conversation about how to address these concerns. In addition, evaluation of the partnership process provided the structure and time needed to reflect upon and celebrate the successes of the partnership.
Partnership members need to recognize the true costs of participation in CBPR projects for both community and academic partners, and consider strategies to minimize those costs. Participants suggested 3 strategies for reducing these costs for community partners: 1) university partners could assist community partners in applying for grants for their regular programs; 2) either grants or subcontracts of grants could go directly to the community organizations to conduct the intervention aspects of the research project; and 3) ensure that the research grant includes a stipend for community partners' participation that more accurately reflects their time involvement.
For academic partners, costs are mostly related to time demands and tenure and promotion decisions. SC partners have suggested that CBPR partnerships consider ways to ensure that the university supports and values faculty involved in CBPR. For example, community partners in CAAA and the URC have articulated the benefits the university receives from faculty involvement in CBPR, such as indirect costs and faculty salaries, and have raised this issue in several venues with university leadership.
With the challenges and costs to academics and community members alike, why undertake CBPR? CAAA Steering Committee members articulated many benefits to their organizations, to themselves personally, to the research itself, and to the community members involved in the intervention. Participants noted that the research conducted by CAAA benefits from, is improved by, and is made possible by the input of community partners. Participants stressed that the success of the intervention and research components of the project resulted from the partnership approach.
Partners also recognized that community partner involvement ensures that research and intervention strategies fit the context of the community. As one community member described:
Our struggle around incentives points out one of the reasons why this [CBPR approach] is important. Some of the stuff [articles in the literature] that is out there is not applicable. Community members are important in providing context. Community members force researchers to look at the local context and gear the research that way. This is a lesson that academics have to hear.
Community Action Against Asthma is in its last of 5 years of initial funding. Although final outcome data for the basic and intervention research questions are still being analyzed, preliminary data suggest that the project has achieved improvement in environmental and health outcomes. In addition, results of this qualitative process evaluation of the partnership suggest the value of both building off of existing partnerships in conducting new research projects, and undertaking traditional epidemiological and intervention research design projects within a CBPR partnership. CAAA's experience highlights the iterative nature of CBPR and the need to constantly monitor, evaluate, and feed back the results concerning the process of the partnership to ensure the maintenance of trust between and influence of all partners in all phases of the research and intervention. It is hoped that, as those of us engaged in community-based participatory research continue to describe and analyze our university–community collaborations, we will enhance the quality and relevance of our research and the benefits to the communities involved.
Community Action Against Asthma is a community-based participatory research project of the Michigan Center for the Environment and Children's Health (MCECH) aimed at investigating the influence of environmental factors on childhood asthma. We acknowledge the contributions of all of the partners involved in this collaborative effort: University of Michigan Schools of Public Health and Medicine, the Detroit Health Department, the Michigan Department of Agriculture, Plant and Pest Management Division, and 9 community-based organizations in Detroit (Butzel Family Center, Community Health and Social Services Center(CHASS), Detroiters Working for Environmental Justice, Detroit Hispanic Development Corporation, Friends of Parkside, Kettering/Butzel Health Initiative, Latino Family Services, United Community Housing Coalition and Warren/Conner Development Coalition), and Henry Ford Health System. MCECH, established in 1998, is funded by the National Institute of Environmental Health Sciences and the U.S. Environmental Protection Agency (grant number P01-ES09589). MCECH is affiliated with the Detroit Community-Academic Urban Research Center (URC), a collaboration among partners from academia, the local health department, community-based organizations, and an integrated health system. The Detroit URC is funded by the Centers for Disease Control and Prevention (CDC). Please refer to www.sph.umich.edu/urc for more information. We also thank Sue Andersen and Kathy Edgren for their assistance in preparation of this manuscript.
Funding for this project comes from the National Institute of Environmental Health Sciences and the U.S. Environmental Protection Agency (grant number P01-ES09589).