|Home | About | Journals | Submit | Contact Us | Français|
We describe our stakeholder engagement process for grant application development that occurred as part of our integrated knowledge translation plan and make recommendations for researchers.
In phase 1, a stakeholder consultation group was developed. In phase 2, surveys regarding knowledge gathering, research agenda, and research collaboration preferences were sent to 333 cross-sectoral youth-serving organizations in Ontario, including family and consumer organizations.
In phase 1, 28 stakeholders from six sectors participated in the consultation group and provided input on multiple aspects of the proposal. Through this process, 19 stakeholders adopted formal roles within the project. In phase 2, 206 surveys were received (response rate = 62%). Survey responses supported the grant focus (concurrent youth mental health and substance use problems). Respondents also prioritized project goals and provided specific feedback on research and knowledge translation. Finally, although some stakeholders chose greater involvement, most survey respondents indicated a preference for a moderate level of participation in research rather than full team membership.
Despite short timelines and feasibility challenges, stakeholders can be meaningfully engaged in and contribute to the grant proposal development process. Consideration is needed for the practical challenges that stakeholder organizations face in supporting and participating in research.
Nous décrivons le processus de recrutement de nos intervenants pour la rédaction de demande de subvention qui s’est inscrit dans le cadre de notre plan intégré de transfert des connaissances, et nous faisons des recommandations aux chercheurs.
À la phase 1, un groupe de consultation des intervenants a été formé. À la phase 2, des sondages sur les préférences en matière de collecte des connaissances, de programme de recherche, et de collaboration à la recherche ont été envoyés à 333 organismes de services à la jeunesse intersectoriels de l’Ontario, y compris aux organismes familiaux et de consommateurs.
À la phase 1, 28 intervenants de six secteurs ont participé au groupe de consultation et émis des idées sur de multiples aspects de la proposition. Pendant ce processus, 19 intervenants ont adopté des rôles officiels dans le cadre du projet. À la phase 2, 206 sondages ont été reçus (taux de réponse: 62%). Les réponses au sondage appuyaient l’accent mis sur les subventions (problèmes concurrents de santé mentale et d’utilisation de substances chez les jeunes). Les répondants ont aussi accordé la priorité aux buts du projet et fourni une rétroaction spécifique sur la recherche et le transfert des connaissances. Enfin, bien que certains intervenants aient privilégié une participation accrue, la plupart des répondants au sondage ont indiqué une préférence pour un niveau modéré de participation à la recherche plutôt que pour l’adhésion de l’équipe complète.
Malgré les brefs délais et les problèmes de faisabilité, les intervenants peuvent participer significativement et contribuer au processus de rédaction de demande de subvention. Il faut prendre en compte les problèmes pratiques que doivent résoudre les organismes des intervenants pour soutenir la recherche et y participer.
Substantial literature highlights the gap between knowledge produced by research regarding mental health and application of that knowledge in mental health services (e.g., McHugh & Barlow, 2010; National Institute of Mental Health (NIMH), 1999; Weisz, Weiss, & Donenberg, 1992). Research has enhanced our understanding of psychopathology, providing both information for clinicians regarding services to provide to clients, and information for policy-makers to consider in the structuring and funding of mental health services. The impact of this literature, however, has been limited by a gap in the exchange of knowledge between researchers and these various stakeholders.
Awareness of this research-practice gap, concern regarding its impact on effective service provision, and recognition of the resulting reduction in returns on public funds invested in research have led granting bodies to place greater emphasis on knowledge translation and exchange activities. The mandate of the Canadian Institutes of Health Research (CIHR, 2006), a funding body for health researchers in Canada, emphasizes both knowledge creation and the translation of that knowledge into improved health for Canadians. Other granting bodies are similarly placing greater emphasis on knowledge translation and exchange activities (e.g., NIMH, 1999). As such, researchers applying for funding face increased pressure to develop effective knowledge translation plans for sharing their findings not only with the research community, but also with the broader community of stakeholders and citizens.
Recently, recognition of the limitations of traditional knowledge translation models that operate in a unidirectional or hierarchical manner from researchers to stakeholders has increased. Knowledge translation activities based on these traditional models can increase resistance among recipients and have had limited success in promoting uptake of empirically supported treatments (Herschell, McNeil, & McNeil, 2004). Accordingly, to address these challenges, there has been a move toward integrated models of knowledge translation and exchange involving reciprocal exchanges and enhanced collaboration between researchers and stakeholders over the course of an initiative or project (Becker, Stice, Shaw, & Woda, 2009; Bowen & Martens, 2005; Graham et al., 2006; Green & Mercer, 2001; Hatgis, Addis, Krasnow, Khazan, & Jacob, 2001; Jansson, Benoit, Casey, Phillips, & Burns, 2010; Miller, Sorensen, Selzer, & Brigham, 2006). Various approaches to establishing collaborative researcher-stakeholder relationships have been described in the literature, including participatory action research (PAR; e.g., Kidd & Kral, 2005; Southam-Gerow, Hourigan, & Allin, 2009), community-partnership research (e.g., Becker et al., 2009), and community-based participatory research (e.g., Macauley & Nutting, 2006). Each of these approaches aims to involve the potential users of research in the formulation and application of the research (Green & Mercer, 2001) with general acknowledgement that actual implementation of research-stakeholder collaborations may vary from very little involvement to full and equal involvement by stakeholders.
The reported benefits of these collaborative models include increases in stakeholders’ perceptions of the relevance of research findings and in turn, increases in their application of research findings (Macaulay & Nutting, 2006; Walter, Davies, & Nutley, 2003). Other reported benefits include gains in practical and conceptual knowledge for both researchers and stakeholders (Hatgis et al., 2001; Walter et al., 2003), and improved quality and external validity of research (Bartunek, Trullen, Bonet, & Sauquet, 2003; Herschell et al., 2004). Indeed, the involvement of stakeholders in the development of research plans has been noted as essential for successful community-based, externally valid, research (Bartunek et al., 2003; Reback, Cohen, Freese, & Shoptaw, 2002); if the research questions are not relevant to stakeholders, neither will the answers be (e.g. Jansson et al., 2010). Notably, integrated knowledge translation in research planning and project development is now a CIHR requirement for many funding opportunities (Graham & Tetroe, 2009).
The grant development phase of the research process, however, is characterized by practical limitations, most notably short timeframes and limited funding for research assistance. As such, researchers may perceive a number of constraints in the extent to which they are able to engage stake-holders in consultation and collaboration during this phase, and may feel limited to strategies such as obtaining letters of support from stakeholders (Frazier, Formoso, Birman, & Atkins, 2008). This strategy, however, is unlikely to result in the level of collaboration necessary for these benefits nor to address the research-practice gap (Frazier, et al., 2008). Accordingly, as part of an integrated knowledge translation plan, we implemented a multi-step process of engaging a broad cross-section of stakeholders during the grant application development phase of a CIHR Emerging Team grant focused on understanding and addressing concurrent disorders in children and youth. We aimed to develop a program of research that would involve knowledge users in its formulation, thereby increasing its likely relevance and utility to stakeholders. We illustrate these integrated knowledge translation methods here to demonstrate an approach that was found to be feasible within the practical constraints of the grant development process, and to encourage the development and discussion of other methods of engaging stake-holders during this important phase of the research process. We also report on the results of these consultations, including stakeholders’ perspectives on researcher-stakeholder collaboration.
We conducted a two-phase stakeholder consultation during development of a grant application focused on child and youth concurrent disorders (CD). In phase one, a stakeholder consultation group was developed; phase two consisted of surveying a large cross-section of stakeholders from youth-serving agencies across Ontario, Canada.
Consultation group members (N=28) included representation from 28 organizations from six youth sectors: mental health, addictions, justice, child welfare, community health, and youth engagement/advocacy; and four roles (e.g., consumers, frontline service providers, agency decision-makers, policy-level decision-makers) from across Ontario. Participants were recruited via emails sent to distribution lists of networks of youth-focused services. Additional targeted recruitment occurred to ensure representation from consumers and policy-makers based on suggestions from early consultation group members.
Consultation group members were asked for input regarding: 1) project objectives; 2) study hypotheses; 3) project methods (including sample, design, measurement and data collection strategies); 4) feasibility; and, 5) strategies for researcher-stakeholder collaboration. Their input was sought through: 1) providing materials via email for review; 2) teleconferences; and, 3) in-person meetings.
In addition, we sought input on research goals, methods, and preferred researcher-stakeholder collaboration strategies from a larger body of stakeholders through a web-based survey of youth-serving organizations and consumer-related (family and youth advocacy/engagement) organizations in Ontario. Through this we also sought to expand our stakeholder network.
We developed a comprehensive list of youth-serving organizations across sectors by obtaining organizational information from provincial sources, such as Children’s Mental Health Ontario, Drug and Alcohol Registry of Treatment, Ontario Association of Community Health Centres, and Ontario Association of Children’s Aid Societies.
Through this process we identified 412 stakeholder organizations. We phoned each organization (three calls maximum) and obtained the email addresses of 282 program managers (or equivalent); the remainder were not obtained due to out-of-service numbers or lack of response. For unreachable organizations, general email addresses from the internet were used. In order to increase representation of consumer-related groups (family and youth advocacy/engagement), we used a snowball sampling approach, asking existing stakeholders to identify consumer-related organizations to include in survey sampling. Twenty additional organizations were identified using this method.
Informed by the consultation group the survey included the following domains: 1) preferences for knowledge exchange including current strategies for accessing clinical and research evidence and preferences for receiving clinical and research information; 2) preferences for research collaboration; and, 3) perspectives on the proposed research agenda, including the perceived presence of concurrent disorders amongst the children, youth and families they serve and opinions regarding the most important issues to be addressed by the project. Service provider and consumer versions of the survey differed in wording and on the inclusion of some service-specific questions.
Invitations for survey participation (with survey link) were emailed successfully (i.e. did not bounce back) to 333 stakeholder organizations; 206 survey responses were received (62% response rate; n=188 service provider version; n=28 consumer version). Service providers who indicated their organization’s primary service sector were from mental health (38%), health (25%), addictions (19%), child welfare (7%), education (3%), justice (3%), and other, such as housing/shelters, outreach, or employment services (5%).
The stakeholder group contributed significantly to the grant development process and their input was reflected in the grant proposal in numerous ways. For example:
In addition, some researcher-stakeholder collaboration roles were formalized through members’ participation as grant co-applicants (n=3) or collaborators (n=3), through volunteering to be considered as potential study sites (n=5), or through membership on an Advisory Committee for project implementation (n=8).
Through this survey we sought information about stake-holder knowledge gathering practices and preferences to allow us to match our knowledge exchange activities to stakeholder preferences. Results revealed that workshops/presentations and discussions with colleagues were the most commonly used strategies for obtaining clinical information across both respondent groups (see Table 1). Similarly, workshops/presentations were the most commonly preferred method for receiving research-based knowledge and evidence, far outweighing any other strategies.
We also sought to examine stakeholder preferences regarding models of researcher-stakeholder collaboration to inform efforts to work meaningfully with stakeholders. Table 2 shows responses to the question, “How can we best involve you and your organization in helping us to identify the most important research questions and the best approaches to use in future clinical research projects?” The most commonly endorsed service provider responses were surveys (67%) and focus groups (55%). Substantial involvement such as full research team membership was only endorsed by 11% of service providers, even though multiple responses were permitted. Among respondents from consumer-related organizations who responded to this item (n=12) surveys were the preferred option (50%) for being involved in research; full research team membership was endorsed by 25%.
Table 3 summarizes responses from both service providers and consumer-related groups regarding the proposed research agenda. Both groups identified improving access, promoting the coordination or integration of services across service sectors, and enhancing youth and family engagement in services as the most important goals for the team’s proposed project. Indeed, 83% of the consumer respondents ranked improving how easily people can access services as the most important issue to address through this project. Among service provider respondents, improving access was also commonly chosen as the most important issue to address (58%), but overall there was greater variability in responses, with a number of topics rated as important.
Consistent with these priorities, almost two thirds (63%) of service provider organizations reported that children and youth with clinically significant concurrent mental health and substance use concerns present for service at their agencies while only approximately half (55%) of those agencies, or 35% of all participating agencies, reported that they offer services to address these co-occurring concerns. Moreover, respondents who reported that youth with CD present to their agencies for service indicated that, on average, approximately 45% of their total volume of youth clients have clinically significant problems with both substance use and mental health concerns.
Similarly, according to consumer-related responders 43% of people involved with their organizations are ‘extremely impacted’ by CD and almost half of those individuals (47%) are ‘not at all’ able to access services to address all their concerns through one service provider. Notably, this is consistent with service providers’ reports that only 55% of agencies who see youth with CD offer services to directly address youth CD concerns.
Greater collaboration between researchers and stakeholders has been identified as an important strategy for closing the research-practice gap (Hatgis et al., 2001; Herschell et al., 2004). Such collaborations may take many forms, with varying levels of stakeholder involvement at different times during the research process. One of the points at which collaboration is particularly important, yet often lacking, is during the development of research or grant proposals (Green & Mercer, 2001). Notably, there are a number of obstacles to stakeholder collaboration at this step of the research process. First, timeframes are often short between funding announcements and application deadlines and may not lend themselves to meaningful collaboration with stakeholders who have numerous and usually pressing clinical and administrative commitments. Indeed, research has cited time and resource demands on community-based stakeholders to be an obstacle to stakeholder collaboration in research (Bowen & Martens, 2005). Second, the activities described here are very time and resource intensive.
Some grants provide some funds for proposal development; however, many grants offer no funding prior to the grant being awarded. This can limit the collaborative activities that are feasible during this phase, including meaningful stakeholder collaboration.
In this paper, we describe ways in which we collaborated with stakeholders to inform grant application development. Of course, these activities represent only a small proportion of the work required to prepare a full grant application. Given the considerable workload associated with application preparation and the short timeframe, it was necessary to tailor our stakeholder collaboration activities to those that were feasible during this timeframe, yet would also be meaningful.
Our multi-level stakeholder consultation process informed our research proposal development in many valuable ways. For example, the consultation group provided insight into a number of issues relevant to the feasibility of our research methods, which allowed us to modify our methods to reflect the practical issues related to working with community-based agencies and the clinical population of interest. As well, our consultation group activities and survey allowed us to obtain stakeholder perspectives regarding the relative importance of potential issues to address in our proposed research which we were then able to integrate with information from literature reviews to produce research questions and hypotheses that were both evidence-informed and of practical importance to the field. Furthermore, stakeholders provided information about how they would like to be involved in research, and how they prefer to receive information allowing us to tailor our proposed knowledge translation and exchange activities.
These findings are limited by the survey response rate (62%) and the lack of capacity to gather detailed information from stakeholders who provided input through the stakeholder survey. As well, the practical aspects of preparing a grant application, especially with a relatively short turn-around time constrained our capacity to document in detail the consultation process (e.g., number of emails, number of responses) with the stakeholder consultation group. Future efforts should record such details as the process unfolds.
This study confirms that stakeholder participation can provide valuable input to research planning and is possible even within tight timelines and budgets. Given its importance in shaping useful research, even limited involvement within such tight constraints should be considered. Notably, survey and focus group participation are preferred methods of involvement for many stakeholders perhaps because many agencies report working at or beyond capacity and substantial involvement in research may constitute a significant resource burden. Online surveys are relatively inexpensive, providing an attractive option where no or very limited funding is available to support the grant development process.
Based on our experiences we recommend the following to researchers: 1) begin planning stakeholder involvement from project conceptualization; 2) consider and minimize the burden on participating stakeholders; and, 3) include different collaboration options, allowing different degrees of stakeholder involvement, recognizing that more intensive collaboration options (e.g., focus groups; larger meetings) with fewer stakeholders allow for more detailed input and less intensive options, such as surveys allow for input from a broader range of stakeholders.
Although higher levels of involvement have been advocated to achieve sustainable capacity building in community-based organizations (Spoth, Greenberg, Bierman, & Redmond, 2004), effective and meaningful researcher-stakeholder partnerships requiring full stakeholder participation in all aspects of a research project may not be necessary nor feasible. Instead, identifying effective strategies to ensure that stakeholders have meaningful opportunities to shape research questions, methods and interpretations of findings (Green & Mercer, 2001), while at the same time allowing for achievement of project goals within the confines of the practical constraints of research and clinical contexts may be our most pressing challenge.
This project was supported by CIHR grant #CBG 101832. The authors thank Eddie Chu for his assistance. The authors have no conflicts to disclose.