|Home | About | Journals | Submit | Contact Us | Français|
To study informal skill transfer via staff networks as a complement to formal training among afterschool childcare providers implementing a health promotion program.
Cross-sectional, sociometric network analysis.
Boston Young Men’s Christian Association (YMCA) afterschool programs implementing the iPLAY program.
All 91 staff members at 20 sites were eligible; 80 completed the survey (88% response rate).
At the network level, network density measured system-level connectedness. At the staff level, the independent variable was out degree, the number of individuals to whom respondents noted a program-related connection. The dependent variable was skill gains, the number of key implementation skills gained from the network.
We mapped the staff program-related social network. We utilized multiple linear regression to estimate the relationship between out degree and skill gains, and we adjusted for clustering of staff in sites.
Most staff (77%) reported gaining at least one skill from the network, but only 2% of potential network connections were established. The regression model showed that out degree (i.e., number of program-related contacts) was significantly associated with skill gains (β = .48, p < .01) independent of other variables.
Informal skill transfer in staff networks may be a useful complement to formal training for implementation of health promotion programs, but informal skill transfer was likely underutilized in this network. Future research employing longitudinal and/or multisite data should examine these findings in greater detail.
Across the field of public health, the call to address the gap between research and practice has been issued repeatedly and emphatically.1,2 In the area of health promotion, much impetus stems from the concern that successful programs have not been disseminated widely, which thereby limits the impact on population health.3 Dissemination often stalls during implementation, the collection of efforts to scale up and incorporate a program or to practice within an organization.4 Many implementation barriers are organizational in nature, such as staff training strategies.5 This study focuses on opportunities to improve training for nonprofit organizations that provide afterschool child-care, and used the Young Men’s Christian Association (YMCA), as a case study.
When implementing health promotion programs in afterschool settings, the development of staff skills to support faithful implementation poses significant organizational challenges.2,4,6 First, health promotion implementation often requires intense, interactive training, especially for complex or novel skills.7 This challenge occurs frequently in afterschool programs, which often use existing staff who were recruited for childcare skills8 rather than implementation-related skills,4 such as program evaluation. Second, limited financial resources often prohibit extensive formal training in these settings.9 Third, afterschool childcare programs typically have high staff turnover rates (i.e., approximately 25% of general staff replaced annually in the United States),10 which results in loss of skills and knowledge when employees leave the organization.11 Given these challenges, organizations must find cost-effective ways to support training for successful implementation.
In their review of implementation research, Fixsen et al.4 noted that staff training models tend to focus on transfer of knowledge and skills. They suggest that receiving immediate feedback while testing new skills in the work environment is critical for success. Although much of the literature suggests that such training must be facilitated by an official consultant or coach,7 this is not always an option for low-resource programs, which prompts the need to seek low-cost, effective methods of training.
Informal training, including peer knowledge sharing, can be cost-effective compared with formal, expert-delivered training12 and may be a useful alternative. On-the-job feedback from peers often occurs via informal connections among employees13,14 as components of the staff social network, which is a web of relationships that exists among employees.15 Peer-to-peer contacts are important channels for spreading innovative ideas and are critical for knowledge transfer.5,16 Social network theory suggests that the number and quality of staff members’ contacts can drive individual and team performance.17 Thus, by supporting informal training through staff social networks, organizations may be able to support successful implementation of novel programs.
We studied the impact of informal training and social networks in the context of a range of implementation outcomes put forth by the review by Fixsen et al.4 Supports and barriers related to implementation processes versus those related to the program must be isolated via evaluations that use multiple ecological levels and categories of outcomes.18 Implementation outcomes are process-specific changes in the following: (1) practitioner behavior, knowledge, and skills; (2) organizational structure to support practitioner behavior change; and (3) relationships with important partners, such as consumers or systems partners. Intervention outcomes are changes in target audience behavior and health outcomes. This study focuses on outcomes related to practitioner skills, an important implementation outcome.
This study addresses the dearth of published research on the utility of staff networks for knowledge transfer in community/health promotion settings,6,19 and it uses the example of a network of YMCA afterschool child-care staff who implement a health promotion program in Boston, Massachusetts.
We were interested in ways that a staff network might serve as a resource for informal training to strengthen practitioner skills and, ultimately, the implementation of a health promotion program. We had three goals: (1) to describe the network of staff implementing a health promotion program; (2) to describe perceived skill transfer within the staff network; and (3) to examine the relationship between staff program-related connections and perceived skill transfer.
The YMCA is the largest private, nonprofit childcare provider in the United States, as it serves approximately 500,000 children annually.20 The organization is striving to become a learning organization, in which employees acquire, adapt, and disseminate information routinely21 to continuously advance the organization. One improvement target is supporting the health and well-being of children,22 and the YMCA of Greater Boston has taken up the national organization’s goals in this area.
In the fall of 2005, the YMCA of Greater Boston selected 24 of 37 urban afterschool sites to implement a set of health promotion and organizational changes. The sites were chosen by Boston YMCA management on the basis of past success with delivery of novel curricula and programs. Changes were guided by the YMCA of the United States and by the Institute for Healthcare Improvement Breakthrough Series model,23 with evaluation assistance from the Harvard School of Public Health. The iPLAY program helped staff create environmental changes that support child health in four areas: physical activity, nutrition, time spent with television and videos, and staff connections with children and parents/guardians. Staff members were also encouraged to use data-driven decision making and experimentation to improve program offerings as part of the learning organization movement. The program placed explicit emphasis on peer knowledge sharing as a means of enhancing implementation. To this end, coordinators (i.e., staff members responsible for spearheading implementation) received mandatory quarterly training sessions and were expected to share information informally (i.e., no formal training mandated) with colleagues at their sites. Technical assistance was provided by the program director, an individual hired to support this program. Staff reported on their personal characteristics and the characteristics of their professional relationships in November and December 2007 by using a self-administered survey. The survey included 25 items and took approximately 20 minutes to complete. The Human Subjects Committee at the Harvard School of Public Health approved this study.
Of the 24 original sites, 20 were still implementing the program 26 months after program inception, which occurred when the study began. The sites served underserved populations of urban children; approximately 70% of children received financial assistance with program fees. The racial/ethnic makeup of the population of children served was estimated as follows: 45% were white, 37% were African-American, 15% were Hispanic/Latino, and 3% were Asian/Pacific Islander or other.24
All 91 staff members at program sites who participated directly in childcare and who were on the staff roster on November 1, 2007 were eligible to participate in the study. A total of 80 staff members took the survey, which yielded a response rate of 88%. Non-responders were absent during survey administration (n = 10) or left the organization before being surveyed (n = 1). Respondents included 53 general staff members who supported implementation, 20 coordinators who were responsible for leading implementation efforts, and 7 supervisors, who were childcare managers for the site and who oversaw coordinators and general staff.
Following the established methods,19,28 we defined our network on the basis of one function: iPLAY program-related connections. Using methods established in the literature,19,25 we asked a series of linked questions to gather our data regarding connections between staff members. Step 1 was as follows: Staff members nominated (listed) colleagues with whom they interacted for sharing information, skills, or talking through challenges and successes regarding the program. If the staff member did not nominate anyone, no additional questions were asked. Step 2 was as follows: Staff members noted skills gained from these colleagues in the following areas: (1) program planning, (2) implementing new programs or activities, (3) evaluating new programs or activities, (4) analyzing data, (5) connecting with children, and (6) connecting with parents/guardians. These six skills represent the focus of formal training initiatives as described by the program director.
This measure assessed network connectedness, or the extent to which collaborative information sharing is taking place.26,27 By using data from step 1, we calculated the percentage of potential ties realized in the network.15 In other words, all members can theoretically connect with all other members of the network, and we compared the number of ties reported to the number of ties theoretically possible. Network density scores can range from zero to one hundred percent; higher values represent more highly connected networks.28 Though clearly defined functional thresholds did not exist, network density levels of approximately 15% to 20% were expected to support knowledge sharing in a network of this size (i.e., roughly 100 members).26
To capture program-related connections, we utilized two forms of the social network analysis measure of degree, or the number of network ties an individual possesses.29 Data from step 1 were input into a matrix, and network software assessed connection patterns for each respondent. Relevant measures with established reliability and validity were not available; however, reports of network connections on the basis of this methodology have shown strong construct validity through triangulation between individual and peer reports.30 We focused on routine program-related interactions, rather than on a specific time period (e.g., the past week), for increased reliability of responses.31
This measure counts all ties between the respondent and others in the network. Each link is counted once regardless of whether the respondent, the colleague, or both individuals reported the connection.29 The in/out degree measure provides a holistic picture of information flow across the network.26
This measure only counts connections reported by the respondent,29 and it was used to assess the relationship between program-related connections and skill gains. Compared with the in/out degree, this measure narrows the focus to connections that may be perceived as functionally useful to respondents25; here, these connections involve the set of individuals from whom respondents may seek and gain skills.
Given our interest in using the staff network to complement formal training processes, we assessed knowledge transfer in terms of skill gains from colleagues. From data collected in step 2, we assigned one point for each skill the respondent gained from at least one colleague. The resulting six-point summary score reflects the list of skills defined by the program director as the focus of formal training initiatives. We focused on reported transfer of target skills to assess the potential of using informal training to complement formal training. Although we do not have reliability or validity data for this measure, studies of skill gains among teachers suggest that individuals are able to accurately self report gaining novel skills.32
We used social network analysis to assess the structure of the staff network and resources created by relationships in the network.33 We utilized a sociometric analysis, which encompassed all members of the network, to track implementation processes and resources holistically.33 Network analysis requires dedicated software to assess data describing connections between respondents; we used UCINET-634 for this purpose.
This analysis included all respondents and nominated colleagues in the network. Thus, if a respondent reported a connection to a network member who was not surveyed, that tie was included in the data, and the tie affected the estimate of connections in the network and the in/out-degree and out-degree values of the respondent. Also, the network dataset included reported links with the program director, who was ineligible for the general survey but who was a key component of the network as the main trainer for the program.
For this analysis, we treated all connections and reports of skill transfer as equal, regardless of staff position. This reflects the fact that, although coordinators were charged with leading implementation at their site, they were not asked to provide formal training to their colleagues. Additionally, the coordinator role did not include managing other staff; thus, all staff were peers in the context of this program.
The network analysis yielded a network-level attribute (i.e., network density) and individual-level staff attributes (i.e., in/out degree and out degree). We conducted our analyses with SAS version 9.0 and UCINET.34,35 We constructed a multiple linear regression model to estimate the relationship between out degree and skill gains. Network data and observations are not independent, therefore the assumptions of classical statistics cannot be met. We utilized correlation and regression techniques developed for network data.36,37 The significance tests are based on random permutations of matrices (here, 10,000 permutations). The data met requirements for linear regression in the original form. The multiple linear regression model included our predictor of interest, out degree, as well as several covariates selected because of their theoretical relevance in the implementation literature, including the following: staff gender, tenure with the YMCA, job position (i.e., general staff, coordinator, or supervisor), and size of the site (i.e., number of staff members). We removed covariates that were nonsignificant (p > .05) and for which removal did not change coefficients of remaining variables by more than 10%. The final model included the independent variable of interest, out degree, as well as staff position and staff tenure.
As seen from Table 1, most individuals involved with the program were young (i.e., younger than 25 years), worked part-time, and had a high school education (44%) or some college or an associate’s degree (48%). More than half of the staff members (56%) had less than 2 years of experience with their current YMCA afterschool programs, which suggests that they began working after program implementation began.
As seen in Figure 1, the network had a hub-and-spoke form, with many connections to one individual, the program director, who was represented by the center circle. The network density was 2.21%, which means that only about 2% of all possible connections between network members were reported. A density score this close to zero reflects a very sparse set of connections within the network15 and is much lower than the 15% to 20% expected to support knowledge sharing.26
On the basis of the measure of in/out degree, which counted all connections reported to and/or by a respondent, all staff members were part of the program-related network. The average number of such connections for staff members was 3.78 (standard deviation [SD] = 2.72), and in/out degree was positively correlated with the number of afterschool program staff at the site in which the individual was based (r = .34, p = .002). In this network, staff members belonged to 20 afterschool childcare sites, which were nested within 8 YMCA branches. The YMCA of Greater Boston is comprised of 16 branches. We noted many connections among individuals at the same site (62% of connections), fewer connections among individuals at different sites in the same branch (18%), and very few connections among individuals at different sites in different branches (6%). An additional 14% of connections existed between staff members and the program director. The average number of connections for coordinators and supervisors was 5.84 (SD = 3.27) compared with 3.24 (SD = 1.71) for general staff; this was a statistically significant difference (p < .0001).
We also assessed out degree (i.e., connections reported by the respondent) and found that 10 staff members (13%) reported having zero connections to others in the network. The average number of connections for out degree was 2.42 (SD = 2.39). The patterns of out degree by position were similar to that found for in/out degree. The correlation between in/out degree and out degree was positive and statistically significant (r = .89, p < .0001).
Overall, about three-fourths of staff members (77%) reported gaining at least one skill from the network. Of the individuals who reported zero skill gains (n = 19), 10 of these individuals were general staff members who reported that they had zero connections with other staff members. The average number of skills gained from the network was 3.58 (SD = 2.39). As seen in Table 2, there was a range in report of skill gains by skill type: highest for program planning (69%) and lowest for program evaluation (50%).
In the multiple linear regression analysis (Table 3), out degree (i.e., connections reported by staff members) was positively associated with skill gains (β = .48, p < .01) independent of other variables. Covariates were not significantly associated with skill gains. The R2 value for the model was .28. Similar results were found when the analysis was conducted by using in/out degree as the independent variable (data not shown). We additionally explored the relationship between out degree and skill gains by restricting the sample to general staff (n = 53). We found positive and statistically significant relationships between skill gains and: connections to one’s site coordinator (β = 1.82, p = .04) and between skill gains and connections to other staff (β = .87, p < .001).
Our findings suggest that informal skill sharing occurred in this staff network and may be a useful complement to formal training among afterschool childcare staff who implemented a health promotion program. Most staff reported at least one program-related connection, and most reported gaining skills from the network. Moreover, program-related connections reported by staff members were positively associated with skill gain reports at a ratio of about one skill transfer reported for every two connections. This is promising in the context of the organizational goal of using the network to transfer six target skills. We also found that the relationships between connections and perceived skill gains held among general staff for their connections to the coordinator at their sites, as well as their connections to other peers. This suggests that informal skill transfer occurs beyond the set of connections between individuals charged with implementation and the peers at their sites. These findings are consistent with similar studies in a range of settings, from large, for-profit corporations14 to hospitals38 and afterschool settings, such as the Boys and Girls Clubs.9 These findings are also consistent with the suggestion that employers who are committed to the transfer of new ideas and skills among staff may find great value in dedicating resources to support informal training.14
This study focused on the potential for staff social network connections to serve as channels for informal training, which resulted, in this instance, in perceived skill transfer. Our findings, though preliminary, provide insight to drive additional research and experimentation in practice in two areas: (1) identifying ways organizations can increase staff connectedness to support network-based informal training and (2) understanding the mechanisms by which connections support informal training.
Given the association between network connections and reports of skill gains, researchers and implementation leaders may be interested in ways in which staff connectedness can be increased. Cross-sectional data prevented us from determining whether individuals tapped into existing connections for skill gains or if they actively sought experts in areas of needed skill, which would simultaneously increase connections and skill gains. In either case, increasing an individual’s connectedness is expected to improve ability to access needed knowledge.25 There is likely a threshold, but, because this network had low density (i.e., only 2% of potential connections realized, which was far less than the theoretical level of 15% to 20% that supports collaborative information sharing26), it follows that increasing network density could improve information sharing above this baseline level.
One way to increase connectedness is to decrease structural and cultural barriers to collaboration.17,39 In this network, coordinators and supervisors were given opportunities and encouragement during training sessions to develop program-related connections with staff members at other sites and in their own sites. This likely explains higher levels of connections (both within and across sites) for these staff members compared with general staff. Creating a knowledge-sharing network will likely require opportunities for staff at multiple levels to establish and maintain connections. Despite these barriers, we found that general staff reported skill gains from coordinators at their sites as well as from peers at other sites.
Given that more than half of respondents were hired after implementation began, the organization also may need to actively facilitate networking and informal skill sharing among recent hires. Across afterschool child-care programs, recent hires make up a large percentage of the workforce at any given time, which drives the demand for continuous and informal training opportunities.40 Increasing connectedness will be challenging in a high-turnover environment41 but may offer two benefits: redundant connections to protect against loss of knowledge as a result of turnover26 and reduced turnover rates if a person is employed as part of a professional development strategy.42
Organizations can also increase connectedness by targeting isolated individuals and subgroups,17 such as staff members who reported zero program-related connections in this network. Interestingly, all of the individuals who reported isolation from the network (on the basis of the out degree measure) were noted as program-related connections by at least one other colleague (on the basis of the in/out degree measure), which suggests that there may be a gap between the source and receiver of program-related information and assessments of available resources.
Organizations also may be able to use their structures to support connectedness. In this system, afterschool programs were based in sites, which were nested in branches, which comprised the YMCA of greater Boston. Although sites are the unit of interest for program goals, the network diagram showed high numbers of connections within branches, which suggests that these connections may be important units of interest for targeted networking. In this analysis, we included all connections in the network and did not distinguish between connections within versus across sites. There are important functional differences between these connections,43 which are explored in subsequent analyses.
Organizations must also engage in a cultural shift towards knowledge sharing. For example, organizations can encourage members to determine what skills they may lack and to seek them in the staff network.39 This can be coupled with linking of networking and knowledge sharing to staff rewards and review processes to encourage informal training.17 Organizations that view staff connectedness as a form of social and human capital can target highly connected staff with retention strategies so that the network is enhanced by their continued membership.26
The findings should be interpreted with a few key limitations in mind. The first relates to validity of our independent and dependent variables, which were collected via self report. Social desirability bias may have affected our data, and this is an anticipated limitation of respondent-driven sociometric analysis. Also, we do not have reliability or validity data for these measures, though we have support from the literature for our choices. Second, the dependent variable describes staff perceptions of skill transfer rather than actual skill transfer, and future research might include a more objective measure. Third, the data in this study were cross-sectional; therefore causation cannot be determined, though alternative explanations still point to the importance of program-related connections. Last, the analysis is limited to one network of non–randomly selected sites; the 24 program sites were chosen by the YMCA of greater Boston because of their past success with delivery of new programs and curricula. The study site environments may be more conducive to implementation efforts than those that were excluded from participation. Network connections are expected to have greater impact in supportive environments5; thus, findings may not be applicable to environments that are less conducive to implementation support.
Despite these limitations, this study contributes to the literature by offering insight into opportunities to support training processes that use staff networks. The theoretical framework that guided this study is unique in combining implementation science in health promotion and health systems settings. Also, the study is strengthened by a high response rate (88%) as well as by sociometric analysis of a clearly defined network. Last, the study provides additional support for use of network analyses in the field of health promotion.
This study serves as a starting point for additional assessments of ways that staff network development may be a useful complement to formal training during implementation of health promotion programs. Additional research should examine the relationship between program-related connectedness and skill transfer by using longitudinal data, ideally across multiple organizations, and also by utilizing a range of implementation and intervention outcomes at staff, site, and organizational levels. Additionally, researchers should examine mechanisms by which skill transfer occurs. As the relationship between staff networks and skill transfer is confirmed and is better understood, organizations may be able to build support for informal skill transfer into the system by creating opportunities for staff to network and creating a knowledge-sharing climate.
Use of the staff network for knowledge transfer may provide a cost-effective method12 to provide necessary training to staff, particularly for new hires, who make up a large share of the workforce in high-turnover environments. Additionally, network analyses focused on skill transfer may provide useful monitoring and evaluation data. These applications may be relevant in other low-wage, high-turnover, restricted-resource environments, such as nursing homes or community groups that rely on volunteers.
The authors are grateful to John Hirliman and Donna Sullivan of the YMCA of Greater Boston for their insight and support as well as to the staff members who implemented the iPLAY program. The authors also appreciate Dr. Betsy Bradley’s insight and suggestions during the editing process. Funding support was provided for the lead author (S.R.) through the National Cancer Institute by the Harvard School of Public Health Education Program in Cancer Prevention Control (5 R25 CA057711-14) and for the last author (K.V.) through the Dana Farber/Harvard Cancer Center.