|Home | About | Journals | Submit | Contact Us | Français|
Improving community health “from the ground up” entails a comprehensive ecological approach, deep involvement of community-based entities, and addressing social determinants of population health status. Although the Centers for Disease Control and Prevention, the Office of the Surgeon General, and other authorities have called for public health to be an “inter-sector” enterprise, few models have surfaced that feature local health departments as a key part of the collaborative model for effecting community-level change. This paper presents evaluation findings and lessons learned from the Partnership for the Public’s Health (PPH), a comprehensive community initiative that featured a central role for local health departments with their community partners. Funded by The California Endowment, PPH provided technical and financial resources to 39 community partnerships in 14 local health department jurisdictions in California to promote community and health department capacity building and community-level policy and systems change designed to produce long-term improvements in population health. The evaluation used multiple data sources to create progress ratings for each partnership in five goal areas related to capacity building, community health improvement programs, and policy and systems change. Overall results were generally positive; in particular, of the 37 partnerships funded continuously throughout the 5 years of the initiative, between 25% and 40% were able to make a high level of progress in each of the Initiative’s five goal areas. Factors associated with partnership success were also identified by local evaluators. These results showed that health departments able to work effectively with community groups had strong, committed leaders who used creative financing mechanisms, inclusive planning processes, organizational changes, and open communication to promote collaboration with the communities they served.
A number of comprehensive, community-based health initiatives have been implemented that take a broad-based approach to improving community health, including Community Care Networks,1 Health Improvement Initiative,2 Work in Health,3 and Partnerships for Health.4 This approach is best represented in the “Healthy Cities and Communities” movement,5–7 in which “social determinants of health,” such as economic development, housing, and education are seen as important factors influencing community health.8 These initiatives typically emphasize community collaboration and partnerships in shaping intervention approaches and building community capacity.9–12
Although a number of these initiatives have involved local public health departments as partners, few have explicitly included health departments as a key part of the collaborative model for effecting community-level change. Turning Point13 created coalitions that included community representation but focused more on systems change in state and local public health. There are a number of reasons why local public health departments can be effective partners for communities as they strive to improve the overall health and well-being of their citizens, including a mission focused on community-level health improvement, the presence of developed infrastructure and programs, staff that often are trained and interested in population health, and an understanding of governmental processes for making policy and systems changes.
This paper presents evaluation findings and lessons learned from the Partnership for the Public’s Health (PPH), a comprehensive community initiative that featured a central role for local health departments. Funded by The California Endowment (TCE), PPH provided technical and financial resources to 39 community partnerships in 14 local health department jurisdictions to promote community and health department capacity building and community-level policy and systems change designed to produce long-term improvements in population health. We report on findings regarding the progress made towards meeting the initiative’s goals, factors associated with high and low levels of progress, and partnership sustainability.
The PPH was a $40 million, 5-year Initiative funded by TCE to develop partnerships between California communities and local health departments. Fourteen county and city health departments were funded under the PPH Initiative along with 39 local community groups. The Initiative goals were to: (1) strengthen the capacity of communities to engage residents to act on their own and in partnership with health departments and other institutions to protect and improve the community’s health and well-being; (2) enhance the capacity of health departments to respond to community-based and community-driven priorities; (3) create sustainable partnerships between communities and health departments that promote and define mutual responsibility for improving community health; and (4) develop state and local policies that support and sustain local capacity to improve community health. Although there was a statewide policy effort that was part of PPH, most of the Initiative resources went to building and supporting the work of local partnerships. Each local partnership was funded for a total of 4 years. Funding was allocated to both the community group and the health department. Community groups received approximately $80,000/year and health departments (HD) received between $150,000 and $180,000/year depending on the number of community groups with which they had partnerships. Each health department partnered with anywhere from two to five separate community groups within their jurisdiction; none of the partnerships involved multiple community groups.
The four broad PPH goals noted above were revised over time and made more concrete to help the partnerships formulate their community action plans and to guide the evaluation. In the final iteration, the partnerships between HDs and community groups were expected to create an action plan that included activities in each of five major PPH goal areas: (1) community group capacity building, (2) health department capacity building, (3) partnership capacity building, (4) community health improvement, and (5) policy and systems change. Community group capacity building focused on organizational development and skill building for the community-based organizations and residents involved in the partnership. Health department capacity building involved modifying internal structures and processes to allow the health department to work more effectively with community partners. Partnership capacity building worked on the interface between the community groups and health departments, striving to break down traditional barriers to collaboration. Community health improvement involved programmatic activities ranging from educational classes to improving the delivery of social and health services. Policy and systems change focused on more long-term structural approaches to community-level health promotion. More examples of activities in each of the five goal areas are given below in “Results.”
We identified four broad categories of community groups within PPH based on their structure and whether they involved primarily non-professional community residents, local nonprofit agencies, or a combination of the two: (1) neighborhood/grassroots—primarily made up of community residents, with resident-driven decision making; (2) agency/resident collaborative—combination of nonprofit agencies and residents, with power and decision making shared more or less equally between the two groups; (3) agency collaborative—organizationally based, a coalition of service agencies/nonprofits; (4) program/service agency—dominated by a single nonprofit agency or program. The number of community groups that fell into each of these categories is presented in “Results” below.
The PPH Office was established as a grant-making office to develop, implement, test, and disseminate model community-based public health approaches in California. The PPH Office was established before community grants were awarded. The responsibilities of the PPH Office included management of the community-level grants, oversight of evaluation efforts, development of technical assistance resources, support of contractors, and development, and advocacy for statewide policy efforts.
There were two broad goals of the PPH evaluation: (1) document the impact of the Initiative and provide formative feedback and (2) disseminate lessons from PPH to public health practitioners, community organizers, and funders interested in promoting community-based public health. The primary questions guiding the PPH evaluation focused on community impact and how the combination of PPH strategies and local partnership actions contributed to the goals of the initiative. Two additional evaluation questions addressed the role of the PPH office in bringing about health-related policy and systems changes at the state and local-level in California and assisting the partnerships in achieving their goals.
The PPH evaluation team included PPH staff, an external Initiative-level evaluator—the Center for Community Health and Evaluation (CCHE)—and local evaluators based in each of the 14 health jurisdictions. CCHE created the overall evaluation design, gathered selected initiative-wide data, and was responsible for overall coordination of the evaluation, data analysis, and report writing. The local evaluators were responsible for implementing data collection tools designed by CCHE, creating local evaluation plans and logic models, documenting accomplishments and challenges, providing timely formative feedback to grantees, and supporting the initiative-level evaluation through feedback and insights drawn from working with local partnerships.
Multiple data sources were used in the PPH evaluation including open-ended key informant interviews, closed-ended surveys of partnership members, participant observation, and document review (including partnership progress reports):
The analysis of the data for this paper consisted primarily of (1) designing and implementing a process to rate the progress of each partnership in each of the five goal areas and (2) identifying factors associated with more and less successful partnerships.
To capture progress at the local partnership level, a brief case study (Partnership Summary) was developed that summarized individual partnership accomplishments in each of the five goal areas. Data sources used in creating the Partnership Summaries were described in the previous section, including interviews, surveys, participant observation, and document review (progress reports). Local evaluators were trained to prepare the final Partnership Summaries using a participatory process involving local partnership members in drafting and reviewing the summaries. Each Partnership Summary used the same template with sections for community characteristics, partnership history, accomplishments to date, factors associated with success (or lack thereof) in achieving partnership goals, resident involvement, sustainability, lessons learned, and recommendations for program improvement. The Partnership Summaries were then used as the primary source of data in a process to rate progress in each of the five PPH goal areas. Progress was rated independently by four separate groups: the partnerships themselves, CCHE, local evaluators, and PPH staff. Representatives from PPH staff, CCHE, and the local evaluators met to resolve differences between the four sets of raters. This consensus rating was labeled the “Initiative-level” rating. If the initiative-level score differed from the partnership’s rating of their own progress, the partnership was given an opportunity to provide a rationale for their score. In some cases, this process resulted in a revision of the initiative-level rating.
The progress ratings were based on a four-point scale: high, high moderate, moderate, and low. The criteria used in assigning the ratings varied slightly by goal area, but in general, the definitions were the following: high progress—implementation of activities that either substantially strengthened the organization/entity or were likely to significantly improve long-term community health; high moderate progress—implementation of activities with the potential to move into the “high” category with a modest level of additional effort/funding; moderate progress—a limited number of activities and/or activities of modest scope; and low progress—little or no activity in the indicated area.
To systematically identify factors associated with partnership success, local evaluators rated the extent to which a list of 17 positive and 17 negative factors were associated with partnership success (or lack of success) in each of the goal areas. The list of factors was compiled from the previous years’ Partnership Summaries and cross-referenced with the literature on success of community partnerships. For example, one positive factor was “having strong leadership;” one negative factor was “inability to develop and/or maintain clear purpose and vision.” Some factors were associated with a specific goal area; others applied to more than one goal area. Local evaluators rated the factor as a “major factor,” a “minor factor,” or “not a factor” in the partnership’s ability to make progress (or not make progress in the case of the negative factors). The number of times a positive factor was mentioned for high or high moderate performing partnerships was then tabulated (e.g., “for 70% of high performing partnerships strong leadership was a major factor in their success”). Similarly, the number of times a negative factor was mentioned for low or moderate performing partnerships was tabulated (e.g., “for 50% of low performing partnerships lack of a clear vision/mission was a major barrier to their success”).
Results are presented in this section describing the PPH partnerships and communities, summarizing partnership progress in each of the five goal areas, listing factors associated with successful (and less successful) partnerships, and describing progress toward sustainability.
Table 1 shows selected characteristics of the 39 PPH partnerships selected for funding (note: two of the partnerships were de-funded during the Initiative and are not included in the progress ratings shown in Tables 3 and and44 below). The table is organized by the size of the health department jurisdiction, divided into three groups: small (<300,000 population), large (>300,000), and Los Angeles (LA) county as a special case (with almost 10 million total population, but divided into smaller administratively independent Service Planning Areas [“Spas” each with a population of approximately 1–2 million]). Five of the eight LA SPAs were included in PPH, and four of those were funded continuously throughout the Initiative.
Each partnership specified a geographic target area that defined their community of focus. These ranged from areas with small populations [e.g., 12 partnerships (30%) focused on areas with less than 10,000 population] to relatively large cities or small counties [e.g., six partnerships (15%) had target areas of 100,000 people or more]. Smaller HDs tended to have smaller target communities: 71% of HD partnerships in small jurisdictions had target areas with fewer than 10,000 people vs. 5% of larger HDs, and 20% (one out of five) of the LA SPAs.
Many of the PPH partnerships were in areas with underserved, low-income, and ethnically diverse populations. As indicated in Table 1, only 28% of the target communities were majority Caucasian; the remainder were either majority Latino (21%) or a roughly equal mix of ethnic groups (51%). Smaller HDs were less likely to be working in ethnic minority communities: 57% of the communities were majority Caucasian for the small HDs vs. 10% and 20%, respectively, for larger HDs and LA. Larger HDs were working mostly in communities (80%) with mixed ethnicity and LA communities were predominately Latino (60%).
Regarding the type of community organization participating in the partnership, the largest number were agency collaboratives (n=15 or 38%), as opposed to more resident-driven groups or single agencies. There were six community groups (15%) that could be classified as purely “grassroots” or resident-driven; and nine (23%) where there was a mix of resident and agency involvement. The nine remaining community groups (23%) were dominated by a single agency or program.
As described above, the progress rating process involved an independent assessment by the partnerships, evaluators, and PPH staff, which resulted in an Initiative-level rating that attempted to reconcile the views of all raters. The progress ratings reported in this study are based on the Initiative-level ratings, which differed somewhat from the self-ratings by the partnerships themselves. A comparison of the partnership self assessment ratings to the Initiative-level ratings showed that the partnerships were more positive in their assessments. However, partnership- and initiative-level ratings were identical about half the time (48%), and 77% of the disagreements were by only one assessment category (e.g., high vs. high moderate, or moderate vs. low).
Before presenting the results for partnership progress, some examples of high performing partnerships may be helpful in illustrating the activities that the partnerships were carrying out in each of the five goal areas. Table 2 provides brief descriptions of the activities of one high performing partnership in each goal area. One LA community group capacity-building effort involved increasing networking and communication among partner agencies, creating a “Neighborhood College” for resident leadership training, and implementing a more effective group governance structure. The Contra Costa health department created new capacity in several areas, including a computerized referral system, a new unit to consolidate the expertise of HD staff working in community projects, and new personnel procedures that reflected an increased commitment to community engagement. One of the San Joaquin County partnerships increased access to and use of each partner’s resources (staff and materials), developed clear governance structures, increased trust and conflict resolution skills, engaged in joint programmatic activities, and developed a sustainability plan.
Examples are also presented in Table 2 for the two community-change goals: community health improvement and policy/systems change. The Shingletown partnership in Shasta County carried out community improvement activities, including increasing the availability of food for seniors and creating an emergency food closet. This partnership also enhanced preventive services for seniors and increased awareness of traffic safety through data collection, public education, and advocacy. The South Bay partnership in San Diego County worked to promote policy change around alcohol use, including successfully protesting new liquor licenses, limiting alcohol distribution at a local water park, and participating in a national anti drunk-driving campaign.
Table 3 shows the progress ratings for the capacity building activities of the PPH partnerships. Considering the results overall, the area of greatest improvement was in community group capacity building where 15 partnerships were rated “high” in terms of progress (41% of the 37 partnerships rated). Only 4 (24%) of the 17 health jurisdictions (counting each of the four continuously funded LA SPAs as a separate jurisdiction) made high progress in health department capacity building; 11 (30%) were rated high in partnership capacity building.
Although the sample size was too small to make meaningful statistical comparisons, there were generally more successes in community group capacity building in the smaller health department jurisdictions: 86% made high or high/moderate progress vs. 55% in the larger health departments. Three of the four continuously funded SPAs in LA made high or high/moderate progress in community group capacity building. There were no major differences in health department capacity building between large and small jurisdictions. The four LA service planning areas had difficulty making progress—all were rated moderate or low in HD capacity building. The smaller HD’s made more progress in overall partnership capacity building—76% rated high or high/moderate vs. 60% of the larger ones.
Table 4 presents results for the community change outcomes: community health improvement and policy and systems change. Overall, 9 (24%) partnerships made high progress in community health improvement and 12 (32%) made high progress in policy and systems change. More partnerships struggled with policy and systems change; 15 (41%) were rated as having low or moderate progress in the area. There were no clear patterns by size of HD; although smaller HDs were more likely to make high or high/moderate progress in community health improvement (76% vs. 50% of larger HDs).
Table 5 lists the factors in each goal area that contributed to or were barriers to progress, as identified by the local evaluators. The first column shows the most frequently mentioned factors for partnerships making high or high moderate progress; the second shows factors most frequently mentioned among the less successful partnerships (moderate or low progress). Frequently cited factors that cut across multiple goal areas included working in a small and/or well-defined community, open communication and sharing of information, and strong leadership skills and commitment to working with community. Communication challenges, lack of resident engagement, and other challenges of working with community appear in all five goal areas as a factor related to low progress. Leadership issues and the ability to develop and maintain a clear vision are also mentioned in multiple goal areas. Inadequate funding and resources is mentioned but at a lower frequency.
Improvement in long-term health outcomes requires that the activities and relationships created or improved during PPH are sustained beyond the period of grant funding. Initial results were encouraging. At the end of PPH funding (August 2004), almost 70% (25 out of 37) of health department and community groups had specific plans to continue working together, including six local partnerships that had signed Memoranda of Understanding specifically outlining key elements of their continuing relationship. Thirteen local partnerships had specific jointly implemented projects they were continuing. In most cases, because of the lack of resources to sustain regular partnership meetings, it did not appear that the specific partnership structures developed under PPH would continue. However, several partnerships indicated that they would rely on the same committee structure to carry out their activities.
All but three of the partnerships had plans to continue at least one of the major activities begun under PPH. Twenty-six local partnerships (70%) had PPH-related activities for which they already had secured continuation funding. By the final year of the Initiative, nearly 80% (29 out of 37) of partnerships had applied for funding to sustain the work that was started under PPH. At the end of the PPH Initiative, partnerships had applied for over 18 million dollars in funding. Over 4.5 million dollars had been received and an additional 4.7 million was pending.
The PPH was a large, complex Initiative with over 50 separate grantee organizations and ambitious goals for community change promoting long-term improvements in population health. In general, the PPH Initiative was able to manage the complexity effectively and produce a number of substantial achievements. In particular, of the 37 partnerships funded continuously throughout the initiative, in each of the five goal areas, between 25% and 40% were able to make a high level of progress.
Other efforts have been made to promote collaboration between health departments and community groups, including Mobilizing for Action through Planning and Partnerships (MAPP)14 and Turning Point. MAPP is a community assessment process developed collaboratively by the National Association of City and County Health Officials and the Centers for Disease Control and Prevention. It was designed using principles similar to PPH but is strictly an assessment process with no funding attached for the implementation of joint HD/community projects. PPH funded the use of MAPP in PPH sites as one tool of many that health departments could use to get the community and other stakeholders in the broader health system engaged. As noted in the introduction, Turning Point13 created coalitions that included community representation but focused more on systems change in state and local public health, vs. PPH which focused on more community improvement that was also programmatic.
As with many multi-site community initiatives,2,15,16 some partnerships were more successful than others. Understanding the factors associated with partnership success may help in designing future initiatives, and for this reason, a systematic attempt was made to identify success factors in each goal area. The following expands briefly on the results presented earlier for two of the five PPH goal areas: partnership development and HD capacity building.
Partnership development was a key goal area because a strong, long-term relationship between HDs and community groups was viewed by PPH as the primary pathway to sustainable community-level programs and policy and systems changes. Many of the important factors identified in this study have been found elsewhere in the extensive literature on partnership development and sustainability.17–24 Trust among members and effective methods of communication have been shown to be key elements in partnership success factors in a number of studies.24 Devising effective means of gathering input from community members is another key element in successful coalitions.17,22 The other factors we identified—working in a small or well-defined community and having a stable group of core members—have not been identified explicitly in the literature. However, many of the structural and process factors that have been identified as important for success, e.g., setting up workable decision-making processes and finding a match between community priorities and partnership activities,20,21 are made easier in smaller communities with a stable partnership membership.
A second goal area was building HD capacity to work more effectively with communities. Only two factors were identified consistently by the local evaluators as associated with HD capacity building: strong leadership committed to working with the community and working collaboratively with the community on data collection and dissemination. A number of factors worked against HD capacity building, including the bureaucratic nature of HDs, lack of leadership, and communication challenges, related to differences in institutional culture between HDs and community groups.
We conducted a more in-depth investigation using the Partnership Summaries and other data in an attempt to understand the factors that led some HDs to be more successful in working with communities. The most critical ingredient was leadership—health department leaders with a strong commitment to a community-based approach to public health (CBPH). Strong leadership was needed to overcome the bureaucratic and HD cultural factors that are barriers to working flexibly and creatively with community groups. All the health departments we examined that were effectively implementing the CBPH had dynamic executive leadership (i.e., health director and/or health officer) that was strongly committed to changing the way public health approached its mission. These leaders took risks to work with community (e.g., agreeing to approach other agencies on the community’s behalf) and were flexible in the consideration and approval of the types and/or scope of projects the health department worked on with the community.
Successful HD leaders used financing of CBPH, planning, organizational change, and communication to promote effective collaboration with community groups. Regarding financing, “model” health departments (i.e., those making high progress in their building capacity to do CBPH) were able to find innovative ways around categorical funding constraints to support CBPH efforts. Model HDs built the capacity of community partners to apply for grants, designated a portion of categorical funding toward work in CBPH, and used their limited flexible funds to support their work with communities. Model HDs demonstrated a strong commitment to including community members in their planning processes, including planning related to revising mission statements, making major organizational changes, and promoting changes in background and deployment of their workforce. Organizational changes made by model HDs included increasing organizational flexibility, increasing the workforce resources dedicated to working with community, creating units or offices designed to work with community partners, and developing mechanisms for community input into health department planning and practice. Finally, model health departments communicated with the community rather than to the community. Communication strategies for these health departments were consistently designed to build capacity or support community needs for policy and systems change.
A significant limitation in our evaluation was our inability to track long-term changes in population-level health outcomes. This was a deliberate decision based on the relatively short 5-year time frame of the initiative, and it resulted in ratings of significance of the community change outcomes that were largely subjective. Other limitations of the evaluation included difficulties in gathering comprehensive data given the large number of partnerships, the open-ended nature of partnership activities, and the limited data tailored specifically to the geographic areas designated as PPH communities.
A significant strength of the evaluation design was the role of the local evaluators. The local evaluators in many cases became integral partnership members, attending meetings regularly and participating in discussions and decision making. This enabled them to gather better data on partnership structure, processes, activities, and outcomes, and also helped them feed back the data they were collecting more effectively to the partnerships. The relationships they developed with the partnerships assisted in the progress rating process, which required trust that being candid about shortcomings would not affect future partnership funding.
In conclusion, PPH showed that, given the right circumstances and support, health departments can be effective partners with community groups in broader health improvement efforts. These HD–community partnerships led to substantial programmatic, policy and systems changes that, if sustained, can be expected to lead to long-term improvements in community health outcomes.
Cheadle is with the Department of Health Services, University of Washington, Seattle, WA, USA; Hsu, Pearson, and Beery are with the Center for Community Health and Evaluation, Seattle, WA, USA; Schwartz is with the Kaiser Foundation Health Plan and Hospitals, Oakland, CA, USA; Greenwald is with the School of Policy, Planning, and Development, University of Southern California, Los Angeles, CA, USA; Flores is with the The California Endowment, San Francisco, CA, USA; Casey is with the Partnership for the Public’s Health, Oakland, CA, USA.