|Home | About | Journals | Submit | Contact Us | Français|
We evaluated organization- and network-level factors that influence organizations’ perceived success. This is important for managing interorganizational networks, which can mobilize communities to address complex health issues such as physical activity, and for achieving change.
In 2011, we used structured interview and network survey data from 22 states in the United States to estimate multilevel random-intercept models to understand organization- and network-level factors that explain perceived network success.
A total of 53 of 59 “whole networks” met the criteria for inclusion in the analysis (89.8%). Coordinators identified 559 organizations, with 3 to 12 organizations from each network taking the online survey (response rate: 69.7%; range: 33%–100%). Occupying a leadership position (P < .01), the amount of time with the network (P < .05), and support from community leaders (P < .05) emerged as correlates of perceived success.
Organizations’ perceptions of success can influence decisions about continuing involvement and investment in networks designed to promote environment and policy change for active living. Understanding these factors can help leaders manage complex networks that involve diverse memberships, varied interests, and competing community-level priorities.
Interorganizational networks, including coalitions, community collaboratives, and partnerships,1 are formed throughout the public health sector as a mechanism for mobilizing communities to address complex health issues.2 Public health professionals, planning officials, and community-based organizations each bring unique resources to networks, while possessing inherently distinctive organizational goals.3
A key aspect of networks is the ability of community partners to work together to reach goals that cannot be achieved by an organization acting alone.4 Networks offer many advantages—assembling diverse stakeholders, pooling resources, and increasing organizational capacity to achieve goals.5–7 This approach allows organizations within networks to develop “cognitive capability” or joint learning, which enhances capacity building to achieve desired results.8 Goal congruence among organizations is thought to be essential because when individual organizational goals are aligned with the group’s goals, organizations are more likely to commit time and resources to the group.3,5,6 A diverse membership, comprising both formal and informal partners from a range of sectors, can bring resources and political support to the group, increasing the group’s success.9 Engaging with policymakers and public officials to garner support for collaborative activities increases both the visibility of the network and the likelihood of success.10–12
Although diversity in network expertise and resources is cited as essential for collaborative advantage,5 it has been suggested that too much heterogeneity can complicate group effectiveness.3 This caveat has implications for public health practice. Specifically, data about how networks develop and function can inform stakeholders involved in interorganizational networks including government, practitioners, and funders about how to build, manage, and evaluate effective networks. A growing expectation today for public health personnel is that they engage in partnerships with other organizations, particularly outside public health, as a way to achieve stated goals. Although engaging in partnerships has long been a predominant activity for public health personnel,13 the extent to which collaboration is expected today seems to be reaching levels greater than in the past.14–17 O’Leary et al. noted that “public managers now find themselves not as unitary leaders of unitary organizations … instead they find themselves convening, facilitating, negotiating, mediating, and collaborating across boundaries.”18(p8) Moreover, within the public health sector, there is little guidance on how public health personnel might consider the cost of this new expectation, or an evidence base to inform how they might manage the complex relationships that come with increased collaboration.19,20
Interorganizational networks have become essential in the area of physical activity promotion. Physical activity levels across the US population remain insufficient with only 20.6% of adults aged 18 years and older meeting national guidelines for both aerobic and muscle-strengthening physical activity.21 Public health systems are attempting to address the decline in physical activity by reshaping the environment to promote active transportation and active lifestyles. Collaborations among organizations from diverse sectors including public health, planning, transportation, public works, parks and recreation, schools, city government, and community-based organizations are being forged to develop policy and built environment solutions to promote active lifestyles.22–25
Although the prevalence of interorganizational networks has risen rapidly, little is known about their effectiveness in creating lasting change.10,26,27 Insight into these relationships will enable community networks to be more strategic in their involvement of partners and use of various tactics and activities to achieve system-level changes, including policy and related environmental improvements. The present analysis is part of the Coalitions and Networks for Active Living study from the Physical Activity Policy Research Network, a thematic research network of the Centers for Disease Control and Prevention. We aimed to characterize active living collaboratives (operationalized as interorganizational networks) by identifying organization- and network-level factors associated with networks’ effectiveness in advancing related environmental improvements and policies.10,22,28
We used a systems science approach that enables investigators to examine dynamic interrelationships of organizations and between organizations and the broader network of organizations as they relate to system-level change.29 Specifically, we applied 1 type of system science methodology, social network analysis, which is a methodology used to gather and analyze data to explain the degree to which network actors connect to one another and the structural makeup of collaborative relationships of organizations within networks.30,31
Figure 1 presents the framework for conducting this analysis. Whole network data can be analyzed at 3 levels—the organizational, dyadic, and network levels (Figure 1). Provan et al. assert “only by examining the whole network can we understand such issues as how networks evolve, how they are governed, and ultimately how collective outcomes might be generated.”32(p480) The organization level looks at the characteristics of the organizations and accounts for the attributes (which also include their relationship attributes, e.g., number of partners in the network). Then, dyads are identified as any 2 organizations and the relationships between them. Finally, the whole network comprises all the organizations of the network and the relationships among them (including the number and quality of the relationships). We analyzed data in this analysis at the organization and whole level.
Figure 1 also presents the explanatory factors, which included variables at the organization- and whole network levels such as most important outcome, leadership role, and agreement on most important goals or outcomes (e.g., goal congruence).33 We operationalized our main outcome, network effectiveness, as reported network success by responding organizations.
The study sample included representatives from interorganizational networks focused on promoting active living or physical activity within the United States. We identified networks through multiple channels including a nomination process by members of the Physical Activity Policy Research Network, outreach to alumni of the Physical Activity and Public Health Practitioners course and the National Society for Physical Activity Practitioners in Public Health, and advertisements in newsletters and Web sites including the Centers for Disease Control and Prevention’s Prevention Research Centers’ Web sites. Through this recruitment process, which began in 2011, we identified 96 networks and, of these, 59 (61%) agreed to participate. Reasons for not participating included timing (n = 6), cancellations (n = 3), and nonresponse (n = 28).22 Some examples of networks that participated included LiveWell Colorado communities, Action Communities for Health, Innovation, and Environmental change (ACHIEVE) communities, Mass in Motion communities (Massachusetts), Shape Up San Francisco, Active Living Hennepin County, and Activate Omaha, among others. Details on the composition and focus of the networks are available elsewhere.10,22,28
After completing in-depth interviews with coordinators, we asked them to invite core organizations to participate in an online survey called Program to Analyze, Record, and Track Networks to Enhance Relationships (PARTNER; http://www.partnertool.net). Organizations were initially informed of the survey by the coordinator and contacted up to 3 times via e-mail reminding them to take the survey. The online survey asked about relationships among organizations, the activities they engaged in, and the kinds of resources they exchanged. In addition, organizations rated the success of their networks. We collected data between May and September 2011.
We asked organizations to indicate how successful the network has been at reaching its goals on a 5-point Likert-type scale (1 = not successful; 5 = very successful).
The online survey included 18 questions about organizational-level characteristics, including length of time involved with the collaborative, role, organization type, area of expertise, resources contributed, and most important network outcomes. We coded organizations by sector and categorized them as transportation, land use, community, health, school, academic, parks and recreation, central government, other, or don’t know.
Respondents rated their organizations’ most important contributions from a predefined list on a Likert-type scale (1 = none; 4 = a great deal). We categorized responses into 3 domains based on the community capitals literature.34 The first—political, social, or cultural capital—included variables on information and feedback, community connections and networking, facilitation and leadership, and advocacy. The second, built or human capital, included in-kind resources, data resources, information technology or Web resources, paid staff, volunteers, planning expertise, health expertise, legal expertise, and other expertise. The third, financial capital, included funding and fiscal management.
We also asked respondents to indicate the most important outcome of the network, based on a list of 9 activities ranging from the identification of health needs to expanding network of partners to changes in policy. We organized activities into 4 groups: policy or environmental change, networking and communication, policy and political activities, or identifying needs of the community.
We derived goal congruence, a measure of organizations’ agreement on the most important network outcome, by summing the total number of different network outcomes identified by members of each group. Scores were categorized as high (1–3 outcomes), medium (4–6 outcomes), or low (≥ 7 outcomes) agreement.
We collected additional network-level characteristics through the coordinator interview, including collaborative age, size, number of community events, and engagement activities. Collaborative age is included because networks with more experience may be effective in achieving change.35,36 We assessed political and policy activities with 4 items: engaging with elected or appointed officials to author policy, receiving endorsement or support from community leaders, offering testimony in policy or legal hearings, and engaging in media communication.11 Respondents rated how frequently their group engaged in each activity for each of these variables (1 = never to 5 = very frequently).10
We calculated summary statistics, frequencies, means, standard deviations, and ranges for all variables. We categorized the outcome variable, perceived success (range: 1–5) into 3 levels for cross-tabulation with explanatory variables: unsuccessful (1–2), successful (3), and very successful (4–5). We used Mantel–Haenszel χ2 tests to test for differences between stratified groups.
Although this study was not powered to test differences in perceived success across interorganizational networks, we conducted a post hoc exploratory analysis of these relationships, by using hierarchical linear models, which allow organization- and network-level factors to be examined simultaneously, account for the nonindependence of errors, and control for the problem of downwardly biased standard errors that could occur if regular multiple regression was used to analyze data with a nested structure.37
We constructed several multilevel random intercept models to explore how characteristics of the whole network affected the perceptions of organizations embedded within them. First we estimated an unconditional model, which reflected variation in the intercept, and calculated the intraclass correlation coefficient to estimate the variation attributable to differences between groups. Next we used a random intercept model to regress perceived success onto each organization- and network-level explanatory variable. We modeled significant organization-level effects (P < .05) from the bivariable analysis with perceived success in model A. We then modeled significant organization- and network-level effects (P < .05), adjusted for group age, simultaneously in model B. We used the PROC MIXED procedure in SAS version 9.3 (SAS Institute, Cary, NC) for the hierarchical analysis.
We included a total of 53 of 59 “whole networks” from 22 states in the analysis (89.8%). Coordinators identified a total of 559 organizations, ranging from 5 to 17 per network with 3 to 12 organizations from each network participating in the online survey (overall response rate: 69.7%; total n = 389; respondent range per organization: 33%–100%). We included only networks with 4 or more organizations completing the survey (n = 53) and only those respondents who answered the online network survey question on success in the analysis (n = 381).
Of the 53 coordinators who took the online survey, more than half represented the health sector (57.7%). Descriptive statistics for all variables are presented in Table 1. More than half of the organizations selected policy or environmental change as their collaboratives’ most important network outcome (56.2%) and approximately a quarter of the organizations thought that networking and communication activities were the most important network outcome (23.9%). The most important resource contribution was split fairly evenly with 47.5% contributing human or built capital and 46.2% contributing social, political, or cultural capital. Networks consisted of organizations from diverse sectors including health, planning, and community (30.7%, 24.9%, and 17.6%, respectively). Among networks, the average age of networks was 5.8 years and almost half of the groups had a membership of 11 to 30 partner organizations. Level of engagement with elected officials varied among networks with almost one third of groups reporting very frequent engagement and another third reporting limited engagement.
Table 2 presents a summary of bivariable and multivariable hierarchical random-intercept models. The significant relationship between network organizations in leadership positions and perceived success remained after we accounted for the nested structure of the data (b = −0.30; P < .01). That is, the effect of leadership on perceived success was significant and its coefficient was negative indicating that respondents in a nonleadership position, when compared with respondents in a leadership position, were more likely to report a lower rating of network success. Length of time in network was significant in models A and B (each model: b = 0.04; P < .05). Participation in network-level policy engagement activities, as measured by asking whether community leaders, which included prominent elected or appointed officials, participated in or endorsed network-sponsored events, was significant in model B, (b = 0.17; P < .05). Our multivariable models demonstrated that both organization- and network-level factors were associated with perceived success.
Our findings suggest that, in active living networks across the United States, perceptions of network success depend on a combination of organizational and network characteristics. Three factors emerged as correlates of perceived success—occupying a leadership position (P < .01), the amount of time with the network (P < .05), and reported support from community leaders (P < .05). A leader’s positive perception of the network helps generate investment and commitment among organizations to long-term network goals. It was not surprising that coordinators commonly reported positive perceptions of their networks, particularly when one considers that they are highly committed to outcomes and success, have intimate knowledge of progress, and often motivate organizations around common goals.
This is also true for organizations that have been with the group for longer periods of time. These organizations, like the coordinator, may be more aware of network success and progress over time. Both coordinators and longstanding organizations are likely to have a stronger emotional investment in the success of the group. These findings are notable because organizations’ perceptions of success affect their decisions about their continuing involvement and investment.7 Policy and system-level change require long-term investment of time, resources, and relationship building. Fostering relationships to keep organizations engaged in the collaborative is important for the sustainability and success of groups.
Overall, perceptions of success varied significantly among groups. Although approximately a quarter of the variation in organizations’ perceptions of success was attributed to network-level effects, only 1 statistically significant relationship between a network-level variable and perceived success emerged from our model. Consistent with our previous findings, this study confirms that external engagement in the political and policy process is important for success.10 Organizations within networks that frequently solicit endorsements and support from community leaders perceive their groups as more successful. For example, participation by community leaders (e.g., elected officials) in network-sponsored events, such as open street events or community forums, is an important indicator of community support.38 Community and political leaders have the ability to influence the progression of the work of the group (e.g., policy development strategies), by either blocking or clearing the way for change. When community leaders participate in network-sponsored events, it sends a signal of recognition and validation of the networks’ efforts. Engagement in political and policy activities may offer significant opportunities to involve leaders and elected officials and positions networks for success.10,22,39
Although the literature asserts that goal congruence is associated with success,3,5 our data suggest some variation in this assumption. That is, networks with both high goal diversity or congruence are still perceived by the collaborative as successful. Organizations engaged in collaboration may have differing expectations and goals based on their organizations’ agenda, individual expertise, and available resources.3,5 As described by Vangen and Huxham,3 both goal congruence and goal diversity are important for success.
This work contributes to the growing evidence base of the field of public health systems and services research, particularly in the area of public health system structure and performance. A major strength of this study is the size of the sample (53 whole networks). In comparison, a review of 32 empirical studies on whole network studies in Public Affairs and Public Health (completed by the research team) found that the number of whole networks analyzed ranged from 1 to 12, with the majority analyzing 2 or fewer. One study by Faust and Skvoretz compared 42 whole networks; however, the composition of the variables included and methods to collect these data varied considerably.40 The large sample size of this study allowed us to address what factors contribute to public health strategies at local, state, and national levels, in a way that is rare in the study of interorganizational networks. In addition, previous research on network effectiveness has generally aggregated individual responses to the network level.4 Our study is unique in that we used a multilevel framework and social network analysis to examine both organization- and network-level factors in the same model.
A second contribution is to the evidence base to inform public health personnel as they continue to nurture and grow networks as a viable approach to population health problems. In this case, identifying factors that have an impact on organizations’ perceptions of success informs network leaders about potentially modifiable means to keep organizations engaged in the network. For example, we found that organizations who frequently solicited endorsements and support from government leaders perceived their groups as more successful.
An ongoing challenge for a network manager is to keep the network organizations working in a collective process in which each one sees the benefit of his or her participation. Understanding how organizations’ perceptions contribute to the success of the network can help a manager tap into those perceptions and either (1) attempt to mitigate any perceptions resulting in negative perceptions or, alternatively, (2) work to get organizations’ perceptions aligned along factors leading to successful outcomes. This analysis suggests that a combination of organization- and network-level factors contribute to perceptions of success. It also suggests that agreement on the most important outcome is not necessarily required for organizations to view their group as successful. This differs from what some previous research has shown, and suggests that it is not diversity of organizations itself that makes it complex to manage a network, but rather that managing the perceptions of organizations and working to expose and discuss perceptions is a key component to network success.
There are several limitations worth noting. Our sample was constructed through a nomination process. Results, consequently, may reflect the work of more mature groups that have experienced more success. Thus, these findings may not be generalizable to all active living groups.
Because of limited resources, we asked for approximately 10 partners per group. Coordinators likely nominated their most influential or active organizations. Had we asked for an exhaustive list of organizations, we might have identified more variation in the structure of the networks (e.g., more organizations’ responses might have identified more bridging relationships or a more defined core–periphery structure). Thus, we may have lost some of the nuances of the interrelationships between the organizations and success reported by organizations. In addition, this analysis may have only detected the strongest theory-informed relationships. Other, less robust, but potentially important theory-informed relationships were not statistically significant.
System-level changes, including policy and built environment improvements, require strategic, interorganizational partnerships that are cultivated over time. Tapping into the varied strengths of organizations and their diverse areas of expertise will increase the capacity of groups to achieve goals and broaden the array of available resources. Organizations’ perceptions of network success are an important indicator of network effectiveness. When organizations feel that their investments have had a positive impact on collaborative activities, then they are likely to justify additional investments and continue supporting the group. These findings inform the practice of forming and sustaining interorganizational networks, and offer insights about managing complex relationships among diverse organizations with varied interests, expectations, and skill sets that are critical for mobilizing change and realizing health improvements.
The study was reviewed and approved by the Colorado Multiple Institutional Review Board and the Human Research Protection Office at Washington University in St Louis.
This study was supported by the Centers for Disease Control and Prevention, Prevention Research Center Program, Special Interest Project 09–10: Physical Activity Policy Research Network (5U48DP001938-02).
We are thankful for the support and guidance from our partners at LiveWell Colorado and Kaiser Permanente, and Sharon Scarbro for analytic support. We would like to thank all of the groups who participated in this study. Our work would not be possible without them. We would also like to thank the anonymous reviewers for their careful review and suggestions to improve the article.
ContributorsJ. Litt conceptualized the original study and oversaw all aspects of the study. J. Litt, H. Reed, D. Varda, and J. Retrum designed the analysis plan. H. Reed led the implementation and evaluation, including data collection and statistical analysis, and drafted the article. R. Tabak was involved in the implementation, including data collection, and reviewed and revised drafts of the article. D. Varda, J. Retrum, J. Gustat, and N. O’Hara-Tompkins contributed to the design and implementation of the study. All authors reviewed and revised drafts of the article and approved the final version.
Jill Litt, Colorado School of Public Health, Aurora.
Danielle Varda, University of Colorado, School of Public Affairs, Denver.
Hannah Reed, Colorado School of Public Health, Aurora.
Jessica Retrum, University of Colorado, School of Public Affairs, Denver.
Rachel Tabak, Prevention Research Center, Brown School of Social Work, Washington University in St Louis, St Louis, MO.
Jeanette Gustat, Department of Epidemiology, Prevention Research Center, School of Public Health and Tropical Medicine, Tulane University, New Orleans, LA.
Nancy O’Hara Tompkins, West Virginia Prevention Research Center, Department of Social and Behavioral Sciences, School of Public Health, West Virginia University, Morgantown.