Decisions regarding health systems are sometimes made without the input of timely and reliable evidence, leading to less than optimal health outcomes. Healthcare organizations can implement tools and infrastructures to support the use of research evidence to inform decision-making.
The purpose of this study was to profile the supports and instruments (i.e., programs, interventions, instruments or tools) that healthcare organizations currently have in place and which ones were perceived to facilitate evidence-informed decision-making.
In-depth semi-structured telephone interviews were conducted with individuals in three different types of positions (i.e., a senior management team member, a library manager, and a ‘knowledge broker’) in three types of healthcare organizations (i.e., regional health authorities, hospitals and primary care practices) in two Canadian provinces (i.e., Ontario and Quebec). The interviews were taped, transcribed, and then analyzed thematically using NVivo 9 qualitative data analysis software.
A total of 57 interviews were conducted in 25 organizations in Ontario and Quebec. The main findings suggest that, for the healthcare organizations that participated in this study, the following supports facilitate evidence-informed decision-making: facilitating roles that actively promote research use within the organization; establishing ties to researchers and opinion leaders outside the organization; a technical infrastructure that provides access to research evidence, such as databases; and provision and participation in training programs to enhance staff’s capacity building.
This study identified the need for having a receptive climate, which laid the foundation for the implementation of other tangible initiatives and supported the use of research in decision-making. This study adds to the literature on organizational efforts that can increase the use of research evidence in decision-making. Some of the identified supports may increase the use of research evidence by decision-makers, which may then lead to more informed decisions, and hopefully to a strengthened health system and improved health.
Evidence-informed health policymaking logically depends on timely access to research evidence. To our knowledge, despite the substantial political and societal pressure to enhance the use of the best available research evidence in public health policy and program decision making, there is no study addressing availability of peer-reviewed research in Canadian health ministries.
To assess availability of (1) a purposive sample of high-ranking scientific journals, (2) bibliographic databases, and (3) health library services in the fourteen Canadian health ministries.
From May to October 2011, we conducted a cross-sectional survey among librarians employed by Canadian health ministries to collect information relative to availability of scientific journals, bibliographic databases, and health library services. Availability of scientific journals in each ministry was determined using a sample of 48 journals selected from the 2009 Journal Citation Reports (Sciences and Social Sciences Editions). Selection criteria were: relevance for health policy based on scope note information about subject categories and journal popularity based on impact factors.
We found that the majority of Canadian health ministries did not have subscription access to key journals and relied heavily on interlibrary loans. Overall, based on a sample of high-ranking scientific journals, availability of journals through interlibrary loans, online and print-only subscriptions was estimated at 63%, 28% and 3%, respectively. Health Canada had a 2.3-fold higher number of journal subscriptions than that of the provincial ministries’ average. Most of the organisations provided access to numerous discipline-specific and multidisciplinary databases. Many organisations provided access to the library resources described through library partnerships or consortia. No professionally led health library environment was found in four out of fourteen Canadian health ministries (i.e. Manitoba Health, Northwest Territories Department of Health and Social Services, Nunavut Department of Health and Social Services and Yukon Department of Health and Social Services).
There is inequity in availability of peer-reviewed research in the fourteen Canadian health ministries. This inequity could present a problem, as each province and territory is responsible for formulating and implementing evidence-informed health policies and services for the benefit of its population.
Health care; Information science; Library science; Knowledge transfer; Research evidence
Community-based organizations are important health system stakeholders as they provide numerous, often highly valued programs and services to the members of their community. However, community-based organizations are described using diverse terminology and concepts from across a range of disciplines. To better understand the literature related to community-based organizations in the health sector (i.e., those working in health systems or more broadly to address population or public health issues), we conducted a scoping review by using an iterative process to identify existing literature, conceptually map it, and identify gaps and areas for future inquiry.
We searched 18 databases and conducted citation searches using 15 articles to identify relevant literature. All search results were reviewed in duplicate and were included if they addressed the key characteristics of community-based organizations or networks of community-based organizations. We then coded all included articles based on the country focus, type of literature, source of literature, academic discipline, disease sector, terminology used to describe organizations and topics discussed.
We identified 186 articles addressing topics related to the key characteristics of community-based organizations and/or networks of community-based organizations. The literature is largely focused on high-income countries and on mental health and addictions, HIV/AIDS or general/unspecified populations. A large number of different terms have been used in the literature to describe community-based organizations and the literature addresses a range of topics about them (mandate, structure, revenue sources and type and skills or skill mix of staff), the involvement of community members in organizations, how organizations contribute to community organizing and development and how they function in networks with each other and with government (e.g., in policy networks).
Given the range of terms used to describe community-based organizations, this scoping review can be used to further map their meanings/definitions to develop a more comprehensive typology and understanding of community-based organizations. This information can be used in further investigations about the ways in which community-based organizations can be engaged in health system decision-making and the mechanisms available for facilitating or supporting their engagement.
Large multi-day conferences have often been criticized as ineffective ways to improve social outcomes and to influence policy or practice. Unfortunately, many conference evaluations have also been inadequate in determining the impact of a conference on its associated social sector, with little evidence gathered or analyzed to substantiate or refute these criticisms. The aim of this scoping review is to investigate and report stakeholders’ objectives for planning or participating in large multi-day conferences and how these objectives are being evaluated. We conducted a scoping review supplemented by a small number of key informant interviews. Eight bibliographic databases were systematically searched to identify papers describing conference objectives and/or evaluations. We developed a conference evaluation framework based on theoretical models and empirical findings, which structured the descriptive synthesis of the data. We identified 3,073 potential papers for review, of which 44 were included in this study. Our evaluation framework connects five key elements in planning a conference and its evaluation (number in brackets refers to number of themes identified): conference objectives (8), purpose of evaluation (7), evaluation methods (5), indicators of success (9) and theories/models (8). Further analysis of indicators of success identified three categories of indicators with differing scopes (i.e. immediate, prospective or follow-up) as well as empirical links between the purpose of evaluations and these indicators. Conference objectives and evaluations were largely correlated with the type of conference (i.e. academic, political/governmental or business) but diverse overall. While much can be done to improve the quality and usefulness of conference evaluations, there are innovative assessments that are currently being utilized by some conferences and warrant further investigation. This review provides conference evaluators and organizers a simple resource to improve their own assessments by highlighting and categorizing potential objectives and evaluation strategies.
Conferences and symposia; Conference objectives; Program evaluations; Learning and behaviour theories; Indicators of success; Evaluation methods
Health systems evidence can enhance policymaking and strengthen national health systems. In the Middle East, limited research exists on the use of evidence in the policymaking process. This multi-country study explored policymakers’ views and practices regarding the use of health systems evidence in health policymaking in 10 eastern Mediterranean countries, including factors that influence health policymaking and barriers and facilitators to the use of evidence.
This study utilized a survey adapted and customized from a similar tool developed in Canada. Health policymakers from 10 countries (Algeria, Bahrain, Jordan, Lebanon Oman, Pakistan, Palestine, Sudan, Tunisia, and Yemen) were surveyed. Descriptive and bi-variate analyses were performed for quantitative questions and thematic analysis was done for qualitative questions.
A total of 237 policymakers completed the survey (56.3% response rate). Governing parties, limited funding for the health sector and donor organizations exerted a strong influence on policymaking processes. Most (88.5%) policymakers reported requesting evidence and 43.1% reported collaborating with researchers. Overall, 40.1% reported that research evidence is not delivered at the right time. Lack of an explicit budget for evidence-informed health policymaking (55.3%), lack of an administrative structure for supporting evidence-informed health policymaking processes (52.6%), and limited value given to research (35.9%) all limited the use of research evidence. Barriers to the use of evidence included lack of research targeting health policy, lack of funding and investments, and political forces. Facilitators included availability of health research and research institutions, qualified researchers, research funding, and easy access to information.
Health policymakers in several countries recognize the importance of using health systems evidence. Study findings are important in light of changes unfolding in some Arab countries and can help undertake an analysis of underlying transformations and their respective health policy implications including the way evidence will be used in policy decisions.
Knowledge translation; Eastern Mediterranean region; Health systems research; Evidence- to- policy
One of the most consistent findings from clinical and health services research is the failure to translate research into practice and policy. As a result of these evidence-practice and policy gaps, patients fail to benefit optimally from advances in healthcare and are exposed to unnecessary risks of iatrogenic harms, and healthcare systems are exposed to unnecessary expenditure resulting in significant opportunity costs. Over the last decade, there has been increasing international policy and research attention on how to reduce the evidence-practice and policy gap. In this paper, we summarise the current concepts and evidence to guide knowledge translation activities, defined as T2 research (the translation of new clinical knowledge into improved health). We structure the article around five key questions: what should be transferred; to whom should research knowledge be transferred; by whom should research knowledge be transferred; how should research knowledge be transferred; and, with what effect should research knowledge be transferred?
We suggest that the basic unit of knowledge translation should usually be up-to-date systematic reviews or other syntheses of research findings. Knowledge translators need to identify the key messages for different target audiences and to fashion these in language and knowledge translation products that are easily assimilated by different audiences. The relative importance of knowledge translation to different target audiences will vary by the type of research and appropriate endpoints of knowledge translation may vary across different stakeholder groups. There are a large number of planned knowledge translation models, derived from different disciplinary, contextual (i.e., setting), and target audience viewpoints. Most of these suggest that planned knowledge translation for healthcare professionals and consumers is more likely to be successful if the choice of knowledge translation strategy is informed by an assessment of the likely barriers and facilitators. Although our evidence on the likely effectiveness of different strategies to overcome specific barriers remains incomplete, there is a range of informative systematic reviews of interventions aimed at healthcare professionals and consumers (i.e., patients, family members, and informal carers) and of factors important to research use by policy makers.
There is a substantial (if incomplete) evidence base to guide choice of knowledge translation activities targeting healthcare professionals and consumers. The evidence base on the effects of different knowledge translation approaches targeting healthcare policy makers and senior managers is much weaker but there are a profusion of innovative approaches that warrant further evaluation.
In the third paper in a three-part series on health systems guidance, Simon Lewin and colleagues explore the challenge of assessing how much confidence to place in evidence on health systems interventions.
In the second paper in a three-part series on health systems guidance, John Lavis and colleagues explore the challenge of linking guidance development and policy development at global and national levels.
In the first paper in a three-part series on health systems guidance, Xavier Bosch-Capblanch and colleagues examine how guidance is currently formulated in low- and middle-income countries, and the challenges to developing such guidance.
Limited research exists on researchers' knowledge transfer and exchange (KTE) in the eastern Mediterranean region (EMR). This multi-country study explores researchers' views and experiences regarding the role of health systems and policy research evidence in health policymaking in the EMR, including the factors that influence health policymaking, barriers and facilitators to the use of evidence, and the factors that increase researchers' engagement in KTE.
Researchers who published health systems and policy relevant research in 12 countries in the EMR (Bahrain, Egypt, Iran, Jordan, Lebanon, Libya, Morocco, Oman, Palestine, Sudan, Syria, and Yemen) were surveyed. Descriptive analysis and Linear Mixed Regression Models were performed for quantitative sections and the simple thematic analysis approach was used for open-ended questions.
A total of 238 researchers were asked to complete the survey (response rate 56%). Researchers indicated transferring results to other researchers (67.2%) and policymakers in the government (40.5%). Less than one-quarter stated that they produced policy briefs (14.5%), disseminated messages that specified possible actions (24.4%), interacted with policymakers and stakeholders in priority-setting (16%), and involved them in their research (19.8%). Insufficient policy dialogue opportunities and collaboration between researchers and policymakers and stakeholders (67.9%), practical constraints to implementation (66%), non-receptive policy environment (61.3%), and politically sensitive findings (57.7%) hindered the use of evidence. Factors that increase researchers' engagement in KTE activities in the region were associated with involving policymakers and stakeholders at various stages such as priority-setting exercises and provision of technical assistance.
Researchers in the EMR recognize the importance of using health systems evidence in health policymaking. Potential strategies to improve the use of research evidence emphasize two-way communication between researchers and policymakers. Findings are critical for the upcoming World Health Report 2012, which will emphasize the significance of conducting and translating health research to inform health policies.
Research evidence is not always being disseminated to healthcare providers who need it to inform their clinical practice. This can result in the provision of ineffective services and an inefficient use of resources, the implications of which might be felt particularly acutely in low- and middle-income countries. Malaria prevention is a particularly compelling domain to study evidence/practice gaps given the proven efficacy, cost-effectiveness and disappointing utilization of insecticide-treated nets (ITNs).
This study compares what is known about ITNs to the related knowledge and practices of healthcare providers in four low- and middle-income countries. A new questionnaire was developed, pilot tested, translated and administered to 497 healthcare providers in Ghana (140), Laos (136), Senegal (100) and Tanzania (121). Ten questions tested participants' knowledge and clinical practice related to malaria prevention. Additional questions addressed their individual characteristics, working context and research-related activities. Ordinal logistic regressions with knowledge and practices as the dependent variable were conducted in addition to descriptive statistics.
The survey achieved a 75% response rate (372/497) across Ghana (107/140), Laos (136/136), Senegal (51/100) and Tanzania (78/121). Few participating healthcare providers correctly answered all five knowledge questions about ITNs (13%) or self-reported performing all five clinical practices according to established evidence (2%). Statistically significant factors associated with higher knowledge within each country included: 1) training in acquiring systematic reviews through the Cochrane Library (OR 2.48, 95% CI 1.30-4.73); and 2) ability to read and write English well or very well (OR 1.69, 95% CI 1.05-2.70). Statistically significant factors associated with better clinical practices within each country include: 1) reading scientific journals from their own country (OR 1.67, 95% CI 1.10-2.54); 2) working with researchers to improve their clinical practice or quality of working life (OR 1.44, 95% CI 1.04-1.98); 3) training on malaria prevention since their last degree (OR 1.68, 95% CI 1.17-2.39); and 4) easy access to the internet (OR 1.52, 95% CI 1.08-2.14).
Improving healthcare providers' knowledge and practices is an untapped opportunity for expanding ITN utilization and preventing malaria. This study points to several strategies that may help bridge the gap between what is known from research evidence and the knowledge and practices of healthcare providers. Training on acquiring systematic reviews and facilitating internet access may be particularly helpful.
Systematic reviews are increasingly seen as helpful “knowledge support” for managers and policy makers, and deliberative processes are starting to be seen as promising, locally contextualized “decision support.” Increases to the flow of systematic reviews should be complemented by efforts to facilitate the retrieval, and adapt the presentation, of the available stock of systematic reviews. Research and other evidence should be combined in transparent ways to facilitate cross-context learning. The challenge for managers and policy makers in moving forward will be to avoid the confusion that comes from the branding of both systematic reviews and deliberative processes.
Globally, healthcare systems are attempting to optimize quality of care. This challenge has resulted in the development of implementation science or knowledge translation (KT) and the resulting need to build capacity in both the science and practice of KT.
We are attempting to meet these challenges through the creation of a national training initiative in KT. We have identified core competencies in this field and have developed a series of educational courses and materials for three training streams. We report the outline for this approach and the progress to date.
We have prepared a strategy to develop, implement, and evaluate a national training initiative to build capacity in the science and practice of KT. Ultimately through this initiative, we hope to meet the capacity demand for KT researchers and practitioners in Canada that will lead to improved care and a strengthened healthcare system.
Although measures of knowledge translation and exchange (KTE) effectiveness based on the theory of planned behavior (TPB) have been used among patients and providers, no measure has been developed for use among health system policymakers and stakeholders. A tool that measures the intention to use research evidence in policymaking could assist researchers in evaluating the effectiveness of KTE strategies that aim to support evidence-informed health system decision-making. Therefore, we developed a 15-item tool to measure four TPB constructs (intention, attitude, subjective norm and perceived control) and assessed its face validity through key informant interviews.
We carried out a reliability study to assess the tool's internal consistency and test-retest reliability. Our study sample consisted of 62 policymakers and stakeholders that participated in deliberative dialogues. We assessed internal consistency using Cronbach's alpha and generalizability (G) coefficients, and we assessed test-retest reliability by calculating Pearson correlation coefficients (r) and G coefficients for each construct and the tool overall.
The internal consistency of items within each construct was good with alpha ranging from 0.68 to alpha = 0.89. G-coefficients were lower for a single administration (G = 0.34 to G = 0.73) than for the average of two administrations (G = 0.79 to G = 0.89). Test-retest reliability coefficients for the constructs ranged from r = 0.26 to r = 0.77 and from G = 0.31 to G = 0.62 for a single administration, and from G = 0.47 to G = 0.86 for the average of two administrations. Test-retest reliability of the tool using G theory was moderate (G = 0.5) when we generalized across a single observation, but became strong (G = 0.9) when we averaged across both administrations.
This study provides preliminary evidence for the reliability of a tool that can be used to measure TPB constructs in relation to research use in policymaking. Our findings suggest that the tool should be administered on more than one occasion when the intervention promotes an initial 'spike' in enthusiasm for using research evidence (as it seemed to do in this case with deliberative dialogues). The findings from this study will be used to modify the tool and inform further psychometric testing following different KTE interventions.
This study examines research knowledge infrastructures (RKIs) found in health systems. An RKI is defined as any instrument (i.e., programs, interventions, tools) implemented in order to facilitate access, dissemination, exchange, and/or use of evidence in healthcare organisations. Based on an environmental scan (17 key informant interviews) and scoping review (26 studies), we found support for a framework that we developed that outlines components that a health system can have in its RKI. The broad domains are climate for research use, research production, activities used to link research to action, and evaluation.
The objective of the current study is to profile the RKI of three types of health system organisations--regional health authorities, primary care practices, and hospitals--in two Canadian provinces to determine the current mix of components these organisations have in their RKI, their experience with these components, and their views about future RKI initiatives.
This study will include semistructured telephone interviews with a purposive sample region of a senior management team member, library/resource centre manager, and a 'knowledge broker' in three regional health authorities, five or six purposively sampled hospitals, and five or six primary care practices in Ontario and Quebec, for a maximum of 71 interviewees. The interviews will explore (a) which RKI components have proven helpful, (b) barriers and facilitators in implementing RKI components, and (c) views about next steps in further development of RKIs.
This is the first qualitative examination of potential RKI efforts that can increase the use of research evidence in health system decision making. We anticipate being able to identify broadly applicable insights about important next steps in building effective RKIs. Some of the identified RKI components may increase the use of research evidence by decision makers, which may then lead to more informed decisions.
Community-based organizations (CBOs) are important stakeholders in health systems and are increasingly called upon to use research evidence to inform their advocacy, program planning, and service delivery. To better support CBOs to find and use research evidence, we sought to assess the capacity of CBOs in the HIV/AIDS sector to acquire, assess, adapt, and apply research evidence in their work.
We invited executive directors of HIV/AIDS CBOs in Ontario, Canada (n = 51) to complete the Canadian Health Services Research Foundation's "Is Research Working for You?" survey.
Based on responses from 25 organizations that collectively provide services to approximately 32,000 clients per year with 290 full-time equivalent staff, we found organizational capacity to acquire, assess, adapt, and apply research evidence to be low. CBO strengths include supporting a culture that rewards flexibility and quality improvement, exchanging information within their organization, and ensuring that their decision-making processes have a place for research. However, CBO Executive Directors indicated that they lacked the skills, time, resources, incentives, and links with experts to acquire research, assess its quality and reliability, and summarize it in a user-friendly way.
Given the limited capacity to find and use research evidence, we recommend a capacity-building strategy for HIV/AIDS CBOs that focuses on providing the tools, resources, and skills needed to more consistently acquire, assess, adapt, and apply research evidence. Such a strategy may be appropriate in other sectors and jurisdictions as well given that CBO Executive Directors in the HIV/AIDS sector in Ontario report low capacity despite being in the enviable position of having stable government infrastructure in place to support them, benefiting from long-standing investment in capacity building, and being part of an active provincial network. CBOs in other sectors and jurisdictions that have fewer supports may have comparable or lower capacity. Future research should examine a larger sample of CBO Executive Directors from a range of sectors and jurisdictions.
Health-system policy makers need timely access to synthesised research evidence to inform the policy-making process. No efforts to address this need have been evaluated using an experimental quantitative design. We developed an evidence service that draws inputs from Health Systems Evidence, which is a database of policy-relevant systematic reviews. The reviews have been (a) categorised by topic and type of review; (b) coded by the last year searches for studies were conducted and by the countries in which included studies were conducted; (c) rated for quality; and (d) linked to available user-friendly summaries, scientific abstracts, and full-text reports. Our goal is to evaluate whether a "full-serve" evidence service increases the use of synthesized research evidence by policy analysts and advisors in the Ontario Ministry of Health and Long-Term Care (MOHLTC) as compared to a "self-serve" evidence service.
We will conduct a two-arm randomized controlled trial (RCT), along with a follow-up qualitative process study in order to explore the findings in greater depth. For the RCT, all policy analysts and policy advisors (n = 168) in a single division of the MOHLTC will be invited to participate. Using a stratified randomized design, participants will be randomized to receive either the "full-serve" evidence service (database access, monthly e-mail alerts, and full-text article availability) or the "self-serve" evidence service (database access only). The trial duration will be ten months (two-month baseline period, six-month intervention period, and two month cross-over period). The primary outcome will be the mean number of site visits/month/user between baseline and the end of the intervention period. The secondary outcome will be participants' intention to use research evidence. For the qualitative study, 15 participants from each trial arm (n = 30) will be purposively sampled. One-on-one semi-structured interviews will be conducted by telephone on their views about and their experiences with the evidence service they received, how helpful it was in their work, why it was helpful (or not helpful), what aspects were most and least helpful and why, and recommendations for next steps.
To our knowledge, this will be the first RCT to evaluate the effects of an evidence service specifically designed to support health-system policy makers in finding and using research evidence.
To support the use of research evidence by community-based organizations (CBOs) we have developed 'Synthesized HIV/AIDS Research Evidence' (SHARE), which is an evidence service for those working in the HIV sector. SHARE consists of several components: an online searchable database of HIV-relevant systematic reviews (retrievable based on a taxonomy of topics related to HIV/AIDS and open text search); periodic email updates; access to user-friendly summaries; and peer relevance assessments. Our objective is to evaluate whether this 'full serve' evidence service increases the use of research evidence by CBOs as compared to a 'self-serve' evidence service.
We will conduct a two-arm randomized controlled trial (RCT), along with a follow-up qualitative process study to explore the findings in greater depth. All CBOs affiliated with Canadian AIDS Society (n = 120) will be invited to participate and will be randomized to receive either the 'full-serve' version of SHARE or the 'self-serve' version (a listing of relevant systematic reviews with links to records on PubMed and worksheets that help CBOs find and use research evidence) using a simple randomized design. All management and staff from each organization will be provided access to the version of SHARE that their organization is allocated to. The trial duration will be 10 months (two-month baseline period, six-month intervention period, and two month crossover period), the primary outcome measure will be the mean number of logins/month/organization (averaged across the number of users from each organization) between baseline and the end of the intervention period. The secondary outcome will be intention to use research evidence as measured by a survey administered to one key decision maker from each organization. For the qualitative study, one key organizational decision maker from 15 organizations in each trial arm (n = 30) will be purposively sampled. One-on-one semi-structured interviews will be conducted by telephone on their views about and their experiences with the evidence service they received, how helpful it was in their work, why it was helpful (or not helpful), what aspects were most and least helpful and why, and recommendations for next steps.
To our knowledge, this will be the first RCT to evaluate the effects of an evidence service specifically designed to support CBOs in finding and using research evidence.
Systematic reviews have the potential to inform decisions made by health policymakers and managers, yet little is known about the impact of interventions to increase the use of systematic reviews by these groups in decision making.
We systematically reviewed the evidence on the impact of interventions for seeking, appraising, and applying evidence from systematic reviews in decision making by health policymakers or managers. Medline, EMBASE, CINAHL, Cochrane Central Register of Controlled Trials, Cochrane Methodology Register, Health Technology Assessment Database, and LISA were searched from the earliest date available until April 2010. Two independent reviewers selected studies for inclusion if the intervention intended to increase seeking, appraising, or applying evidence from systematic reviews by a health policymaker or manager. Minimum inclusion criteria were a description of the study population and availability of extractable data.
11,297 titles and abstracts were reviewed, leading to retrieval of 37 full-text articles for assessment; four of these articles met all inclusion criteria. Three articles described one study where five systematic reviews were mailed to public health officials and followed up with surveys at three months and two years. The articles reported from 23% to 63% of respondents declaring they had used systematic reviews in policymaking decisions. One randomised trial indicated that tailored messages combined with access to a registry of systematic reviews had a significant effect on policies made in the area of healthy body weight promotion in health departments.
The limited empirical data renders the strength of evidence weak for the effectiveness and the types of interventions that encourage health policymakers and managers to use systematic reviews in decision making.
We conducted a print media analysis in 44 countries in Africa, the Americas, Asia, and the Eastern Mediterranean in order to understand one dimension of the climate for evidence-informed health systems and to provide a baseline for an evaluation of knowledge-translation platforms. Our focus was whether and how policymakers, stakeholders, and researchers talk in the media about three topics: policy priorities in the health sector, health research evidence, and policy dialogues regarding health issues.
We developed a search strategy consisting of three progressively more delimited phases. For each jurisdiction, we searched Major World Publications in LexisNexis Academic News for articles published in 2007, selected relevant articles using one set of general criteria and three sets of concept-specific criteria, and coded the selected articles to identify common themes. Second raters took part in the analysis of Lebanon and Malaysia to assess inter-rater reliability for article selection and coding.
We identified approximately 5.5 and 5 times more articles describing health research evidence compared to the number of articles describing policy priorities and policy dialogues, respectively. Few articles describing health research evidence discussed systematic reviews (2%) or health systems research (2%), and few of the policy dialogue articles discussed researcher involvement (9%). News coverage of these concepts was highly concentrated in several countries like China and Uganda, while few articles were found for many other jurisdictions. Kappa scores were acceptable and consistently greater than 0.60.
In many countries the print media, at least as captured in a global database, are largely silent about three topics central to evidence-informed health systems. These findings suggest the need for proactive-media engagement strategies.
Little is known about the extent to which research evidence informs the development of recommendations by international organizations.
We identified specific World Health Organization (WHO) and World Bank recommendations on five topics (contracting, healthcare financing, health human resources, tuberculosis control and tobacco control), catalogued the related systematic reviews and assessed the recommendations to determine their consistency with the systematic reviews that were available at the time of their formulation.
Only two of the eight publications examined were found to cite systematic reviews, and only five of 14 WHO and two of seven World Bank recommendations were consistent with both the direction and nature of effect claims from systematic reviews. Ten of 14 WHO and five of seven World Bank recommendations were consistent with the direction of effect claims only.
WHO and the World Bank – working with donor agencies and national governments – can improve their use of (or at least, their reporting about their use of) research evidence. Decision-makers and clinicians should critically evaluate the quality and local applicability of recommendations from any source, including international organizations, prior to their implementation.
Many international statements have urged researchers, policy-makers and health care providers to collaborate in efforts to bridge the gaps between research, policy and practice in low- and middle-income countries. We surveyed researchers in 10 countries about their involvement in such efforts.
We surveyed 308 researchers who conducted research on one of four clinical areas relevant to the Millennium Development Goals (prevention of malaria, care of women seeking contraception, care of children with diarrhea and care of patients with tuberculosis) in each of 10 low- and middle-income countries (China, Ghana, India, Iran, Kazakhstan, Laos, Mexico, Pakistan, Senegal and Tanzania). We focused on their engagement in three promising bridging activities and examined system-level, organizational and individual correlates of these activities.
Less than half of the researchers surveyed reported that they engaged in one or more of the three promising bridging activities: 27% provided systematic reviews of the research literature to their target audiences, 40% provided access to a searchable database of research products on their topic, and 43% established or maintained long-term partnerships related to their topic with representatives of the target audience. Three factors emerged as statistically significant predictors of respondents’ engagement in these activities: the existence of structures and processes to link researchers and their target audiences predicted both the provision of access to a database (odds ratio [OR] 2.62, 95% CI 1.30–5.27) and the establishment or maintenance of partnerships (OR 2.65, 95% CI 1.25–5.64); stability in their contacts predicted the provision of systematic reviews (OR 2.88, 95% CI 1.35–6.13); and having managers and public (government) policy-makers among their target audiences predicted the provision of both systematic reviews (OR 4.57, 95% CI 1.78–11.72) and access to a database (OR 2.55, 95% CI 1.20–5.43).
Our findings suggest potential areas for improvement in light of the bridging strategies targeted at health care providers that have been found to be effective in some contexts and the factors that appear to increase the prospects for using research in policy-making.
Gaps continue to exist between research-based evidence and clinical practice. We surveyed health care providers in 10 low- and middle-income countries about their use of research-based evidence and examined factors that may facilitate or impede such use.
We surveyed 1499 health care providers practising in one of four areas relevant to the Millennium Development Goals (prevention of malaria, care of women seeking contraception, care of children with diarrhea and care of patients with tuberculosis) in each of China, Ghana, India, Iran, Kazakhstan, Laos, Mexico, Pakistan, Senegal and Tanzania.
The proportion of respondents who reported that research was likely to change their clinical practice if performed and published in their own country (84.6% and 86.0% respectively) was higher than the proportion who reported the same about research and publications from their region (66.4% and 63.1%) or from high-income countries (55.8% and 55.5%). Respondents who were most likely to report that the use of research-based evidence led to changes in their practice included those who reported using clinical practice guidelines in paper format (odds ratio [OR] 1.54, 95% confidence interval [CI] 1.03–2.28), using scientific journals from their own country in paper format (OR 1.70, 95% CI 1.26–2.28), viewing the quality of research performed in their country as above average or excellent (OR 1.93, 95% CI 1.16–3.22); trusting systematic reviews of randomized controlled trials (OR 1.59, 95% CI 1.08–2.35); and having easy access to the Internet (OR 1.90, 95% CI 1.19–3.02).
Locally conducted or published research has played an important role in changing the professional practice of health care providers surveyed in low- and middle-income countries. Increased investments in local research, or at least in locally adapted publications of research-based evidence from other settings, are therefore needed. Although access to the Internet was viewed as a significant factor in whether research-based evidence led to concrete changes in practice, few respondents reported having easy access to the Internet. Therefore, efforts to improve Internet access in clinical settings need to be accelerated.
Community-based organizations (CBOs) are important stakeholders in health systems and are increasingly called upon to use research evidence to inform their advocacy, program planning, and service delivery efforts. CBOs increasingly turn to community-based research (CBR) given its participatory focus and emphasis on linking research to action. In order to further facilitate the use of research evidence by CBOs, we have developed a strategy for community-based knowledge transfer and exchange (KTE) that helps CBOs more effectively link research evidence to action. We developed the strategy by: outlining the primary characteristics of CBOs and why they are important stakeholders in health systems; describing the concepts and methods for CBR and for KTE; comparing the efforts of CBR to link research evidence to action to those discussed in the KTE literature; and using the comparison to develop a framework for community-based KTE that builds on both the strengths of CBR and existing KTE frameworks.
We find that CBR is particularly effective at fostering a climate for using research evidence and producing research evidence relevant to CBOs through community participation. However, CBOs are not always as engaged in activities to link research evidence to action on a larger scale or to evaluate these efforts. Therefore, our strategy for community-based KTE focuses on: an expanded model of 'linkage and exchange' (i.e., producers and users of researchers engaging in a process of asking and answering questions together); a greater emphasis on both producing and disseminating systematic reviews that address topics of interest to CBOs; developing a large-scale evidence service consisting of both 'push' efforts and efforts to facilitate 'pull' that highlight actionable messages from community relevant systematic reviews in a user-friendly way; and rigorous evaluations of efforts for linking research evidence to action.
Through this type of strategy, use of research evidence for CBO advocacy, program planning, and service delivery efforts can be better facilitated and continually refined through ongoing evaluations of its impact.
A questionnaire could assist researchers, policymakers, and healthcare providers to describe and monitor changes in efforts to bridge the gaps among research, policy and practice. No questionnaire focused on researchers' engagement in bridging activities related to high-priority topics (or the potential correlates of their engagement) has been developed and tested in a range of low- and middle-income countries (LMICs).
Country teams from ten LMICs (China, Ghana, India, Iran, Kazakhstan, Laos, Mexico, Pakistan, Senegal, and Tanzania) participated in the development and testing of a questionnaire. To assess reliability we calculated the internal consistency of items within each of the ten conceptual domains related to bridging activities (specifically Cronbach's alpha). To assess face and content validity we convened several teleconferences and a workshop. To assess construct validity we calculated the correlation between scales and counts (i.e., criterion measures) for the three countries that employed both and we calculated the correlation between different but theoretically related (i.e., convergent) measures for all countries.
Internal consistency (Cronbach's alpha) for sets of related items was very high, ranging from 0.89 (0.86-0.91) to 0.96 (0.95-0.97), suggesting some item redundancy. Both face and content validity were determined to be high. Assessments of construct validity using criterion-related measures showed statistically significant associations for related measures (with gammas ranging from 0.36 to 0.73). Assessments using convergent measures also showed significant associations (with gammas ranging from 0.30 to 0.50).
While no direct comparison can be made to a comparable questionnaire, our findings do suggest a number of strengths of the questionnaire but also the need to reduce item redundancy and to test its capacity to monitor changes over time.