The negative impact of musculoskeletal diseases on the physical function and quality of life of people living in developing countries is considerable. This disabling effect is even more marked in low-socioeconomic communities within developing countries. In Mexico, there is a need to create community-based rehabilitation programs for people living with musculoskeletal diseases in low-socioeconomic areas. These programs should be directed to prevent and decrease disability, accommodating the specific local culture of communities.
The objective of this paper is to describe a research protocol designed to develop, implement, and evaluate culturally sensitive community-based rehabilitation programs aiming to decrease disability of people living with musculoskeletal diseases in two low-income Mexican communities.
A community-based participatory research approach is proposed, including multi and transdisciplinary efforts among the community, medical anthropology, and the health sciences. The project is structured in 4 main stages: (1) situation analysis, (2) program development, (3) program implementation, and (4) program evaluation. Each stage includes the use of quantitative and qualitative methods (mixed method program).
So far, we obtained resources from a Mexican federal agency and completed stage one of the project at Chankom, Yucatán. We are currently receiving funding from an international agency to complete stage two at this same location. We expect that the project at Chankom will be
concluded by December of 2017. On the other hand, we just started the execution of stage one at Nuevo León with funding from a Mexican federal agency. We expect to conclude the project at this site by September of 2018.
Using a community-based participatory research approach and a mixed method program could result in the creation of culturally sensitive community-based rehabilitation programs that promote community development and decrease the disabling effects of musculoskeletal diseases within two low-income Mexican communities.
rehabilitation; community-based participatory research; social change; musculoskeletal diseases; Mexico
There is a scarcity of empirical data on African country climates for evidence-informed health system policymaking (EIHSP) to backup the longstanding reputation that research evidence is not valued enough by health policymakers as an information input.
Herein, we assess whether and how changes have occurred in the climate for EIHSP before and after the establishment of two Knowledge Translation Platforms housed in government institutions in Cameroon and Uganda since 2006.
We merged content analysis techniques and policy sciences analytical frameworks to guide this structured review of governmental policy documents geared at achieving health Millennium Development Goals. We combined i) a quantitative exploration of the usage statistics of research-related words and constructs, citations of types of evidence, and budgets allocated to research-related activities; and (ii) an interpretive exploration using a deductive thematic analysis approach to uncover changes in the institutions, interests, ideas, and external factors displaying the country climate for EIHSP. Descriptive statistics compared quantitative data across countries during the periods 2001–2006 and 2007–2012.
We reviewed 54 documents, including 33 grants approved by global health initiatives. The usage statistics of research-related words and constructs showed an increase over time across countries. Varied forms of data, information, or research were instrumentally used to describe the burden and determinants of poverty and health conditions. The use of evidence syntheses to frame poverty and health problems, select strategies, or forecast the expected outcomes has remained sparse over time and across countries. The budgets for research increased over time from 28.496 to 95.467 million Euros (335%) in Cameroon and 38.064 to 58.884 million US dollars (155%) in Uganda, with most resources allocated to health sector performance monitoring and evaluation. The consistent naming of elements pertaining to the climate for EIHSP features the greater influence of external donors through policy transfer.
This structured review of governmental policy documents illustrates the nascent conducive climate for EIHSP in Cameroon and Uganda and the persistent undervalue of evidence syntheses. Global and national health stakeholders should raise the profile of evidence syntheses (e.g., systematic reviews) as an information input when shaping policies and programmes.
Electronic supplementary material
The online version of this article (doi:10.1186/1478-4505-13-2) contains supplementary material, which is available to authorized users.
Cameroon; Climate for evidence informed health system policymaking; Content analysis; Governance; Health systems; Knowledge translation platform; Low- and middle-income countries; Policy sciences analytical framework; Structured documentary review; Uganda
Deliberative dialogues have recently captured attention in the public health policy arena because they have the potential to address several key factors that influence the use of research evidence in policymaking. We conducted an evaluation of three deliberative dialogues convened in Canada by the National Collaborating Centre for Healthy Public Policy in order to learn more about deliberative dialogues focussed on healthy public policy.
The evaluation included a formative assessment of participants’ views about and experiences with ten key design features of the dialogues, and a summative assessment of participants’ intention to use research evidence of the type that was discussed at the dialogue. We surveyed participants immediately after each dialogue was completed and again six months later. We analyzed the ratings using descriptive statistics and the written comments by conducting a thematic analysis.
A total of 31 individuals participated in the three deliberative dialogues that we evaluated. The response rate was 94% (N = 29; policymakers (n = 9), stakeholders (n = 18), researchers (n = 2)) for the initial survey and 56% (n = 14) for the follow-up. All 10 of the design features that we examined as part of the formative evaluation were rated favourably by all participant groups. The findings of the summative evaluation demonstrated a mean behavioural intention score of 5.8 on a scale from 1 (strongly disagree) to 7 (strongly agree).
Our findings reinforce the promise of deliberative dialogues as a strategy for supporting evidence-informed public health policies. Additional work is needed to understand more about which design elements work in which situations and for different issues, and whether intention to use research evidence is a suitable substitute for measuring actual behaviour change.
Deliberative dialogue; Knowledge translation; Evidence-informed decision-making; Public health policy
Practical solutions are needed to support the appropriate use of available health system resources as countries are continually pressured to ‘do more with less’ in health care. Increasingly, health systems and organizations are exploring the reassessment of possibly obsolete, inefficient, or ineffective health system resources and potentially redirecting funds to those that are more effective and efficient. Such processes are often referred to as ‘disinvestment’. Our objective is to gain further understanding about: 1) whether how and under what conditions health systems decide to pursue disinvestment; 2) how health systems have chosen to undertake disinvestment; and 3) how health systems have implemented their disinvestment approach.
We will use a critical interpretive synthesis (CIS) approach, to develop a theoretical framework based on insights drawn from a range of relevant sources. We will conduct systematic searches of databases as well as purposive searches to identify literature to fill conceptual gaps that may emerge during our inductive process of synthesis and analysis. Two independent reviewers will assess search results for relevance and conceptually map included references. We will include all empirical and non-empirical articles that focus on disinvestment at a system level. We will then extract key findings from a purposive sample of articles using frameworks related to government agendas, policy development and implementation, and health system contextual factors and then synthesize and integrate the findings to develop a framework about our core areas of interest. Lastly, we will convene a stakeholder dialogue with Canadian and international policymakers and other stakeholders to solicit targeted feedback about the framework (e.g., by identifying any gaps in the literature that we may want to revisit before finalizing it) and deliberating about barriers for developing and implementing approaches to disinvestment, strategies to address these barriers and about next steps that could be taken by different constituencies.
Disinvestment is an emerging field and there is a need for evidence to inform the prioritization, development, and implementation of strategies in different contexts. Our CIS and the framework developed through it will support the actions of those involved in the prioritization, development, and implementation of disinvestment initiatives.
Systematic review registration
Electronic supplementary material
The online version of this article (doi:10.1186/2046-4053-3-143) contains supplementary material, which is available to authorized users.
Disinvestment; Critical interpretive synthesis; Health technologies; Health system context; Policy development; Policy implementation; Reassessment; Appropriateness
Ensuring the use of research evidence in health system management and policy decisions is an important challenge in this century. Knowledge transfer and exchange (KTE) has emerged as a paradigm to address the challenges and start closing the ‘know-do’ gap. This area of work is gaining momentum in most developed countries, yet, to date, no work has been performed in Israel within this area. The purpose of this study was to identify which KTE activities health systems and policy researchers in Israel have undertaken.
A cross-sectional web-based survey of researchers who have conducted health systems and policy research in Israel was developed. The survey consisted of a demographics section, quantitative scales, and open-ended questions. The survey was sent to all health systems and policy researchers in Israel (n = 125).
The study response rate (28%) was relatively low as compared to other studies in the same field (range of 42% to 88%). Our survey found that more than a third of the health systems and policy researchers in Israel reported that they were frequently or always involved in the following KTE activities: interactions with target audience through the research process (i.e., during developing a research question or executing the research; 35% to 42%) or through formal or informal meetings during conferences, workshops, or conversations (40%). Less than half of the health systems and policy researchers in Israel are engaged in bridging activities aimed to facilitate target audiences to use research.
This is a fairly new area in Israel and therefore the level of engagement of researchers in KTE activities is not very high. The low response rates could be because KTE is a new field in Israel and minimal KTE initiatives have been undertaken. It is preferable to have higher response rates, yet, after several initiatives, this was the outcome. While the findings are relevant, they may not reflect the total population of health system and policy researchers in Israel. Health system and policy researchers in Israel need to be introduced to the benefits and potential advantages of KTE in an organized and systematic way.
Health systems and policy; Knowledge transfer and exchange; Researchers; Survey
Mobilizing research evidence for daily decision-making is challenging for health system decision-makers. In a previous qualitative paper, we showed the current mix of supports that Canadian health-care organizations have in place and the ones that are perceived to be helpful to facilitate the use of research evidence in health system decision-making. Factors influencing the implementation of such supports remain poorly described in the literature. Identifying the barriers to and facilitators of different interventions is essential for implementation of effective, context-specific, supports for evidence-informed decision-making (EIDM) in health systems. The purpose of this study was to identify (a) barriers and facilitators to implementing supports for EIDM in Canadian health-care organizations, (b) views about emerging development of supports for EIDM, and (c) views about the priorities to bridge the gaps in the current mix of supports that these organizations have in place.
This qualitative study was conducted in three types of health-care organizations (regional health authorities, hospitals, and primary care practices) in two Canadian provinces (Ontario and Quebec). Fifty-seven in-depth semi-structured telephone interviews were conducted with senior managers, library managers, and knowledge brokers from health-care organizations that have already undertaken strategic initiatives in knowledge translation. The interviews were taped, transcribed, and then analyzed thematically using NVivo 9 qualitative data analysis software.
Limited resources (i.e., money or staff), time constraints, and negative attitudes (or resistance) toward change were the most frequently identified barriers to implementing supports for EIDM. Genuine interest from health system decision-makers, notably their willingness to invest money and resources and to create a knowledge translation culture over time in health-care organizations, was the most frequently identified facilitator to implementing supports for EIDM. The most frequently cited views about emerging development of supports for EIDM were implementing accessible and efficient systems to support the use of research in decision-making (e.g., documentation and reporting tools, communication tools, and decision support tools) and developing and implementing an infrastructure or position where the accountability for encouraging knowledge use lies. The most frequently stated priorities for bridging the gaps in the current mix of supports that these organizations have in place were implementing technical infrastructures to support research use and to ensure access to research evidence and establishing formal or informal ties to researchers and knowledge brokers outside the organization who can assist in EIDM.
These results provide insights on the type of practical implementation imperatives involved in supporting EIDM.
Evidence informed decision-making; Knowledge transfer and exchange; Knowledge translation
There is a scarcity of empirical data on institutions devoted to knowledge brokerage and their influence in Africa. Our objective was to describe two pioneering Knowledge Translation Platforms (KTPs) supporting evidence informed health system policymaking (EIHSP) in Cameroon and Uganda since 2006.
This comparative historical case study of Evidence Informed Policy Network (EVIPNet) Cameroon and Regional East African Community Health Policy Initiative (REACH-PI) Uganda using multiple methods comprised (i) a descriptive documentary analysis for a narrative historical account, (ii) an interpretive documentary analysis of the context, profiles, activities and outputs inventories and (iii) an evaluative survey of stakeholders exposed to evidence briefs produced and policy dialogues organized by the KTPs.
Both initiatives benefited from the technical and scientific support from the global EVIPNet resource group. EVIPNet Cameroon secretariat operates with a multidisciplinary group of part-time researchers in a teaching hospital closely linked to the ministry of health. REACH-PI Uganda secretariat operates with a smaller team of full time staff in a public university. Financial resources were mobilized from external donors to scale up capacity building, knowledge management, and linkage and exchange activities. Between 2008 and 2012, twelve evidence briefs were produced in Cameroon and three in Uganda. In 2012, six rapid evidence syntheses in response to stakeholders’ urgent needs were produced in Cameroon against 73 in Uganda between 2010 and 2012. Ten policy dialogues (seven in Cameroon and three in Uganda) informed by pre-circulated evidence briefs were well received. Both KTPs contributed to developing and testing new resources and tools for EIHSP. A network of local and global experts has created new spaces for evidence informed deliberations on priority health policy issues related to MDGs.
This descriptive historical account of two KTPs housed in government institutions in Africa illustrates how the convergence of local and global factors and agents has enabled in-country efforts to support evidence-informed deliberations on priority health policy issues and lays the ground for further work to assess their influence on the climate for EIHSP and specific health policy processes.
Evidence informed health system policymaking; Knowledge translation platform; Health systems; Low- and middle- income countries; Governance; Comparative case study; Cameroon; Uganda
Evidence-informed health policymaking is a goal of equitable and effective health systems but occurs infrequently in reality. Past research points to the facilitating role of interpersonal relationships between policy-makers and researchers, imploring the adoption of a social network lens. This study aims to identify network-level factors associated with the exchange and use of research evidence in policymaking.
Data on social networks and research use were collected from seventy policy actors across three health policy cases in Burkina Faso (child health, malaria, and HIV). Networks were graphed for actors’ interactions, their provision of, and request for research evidence. Exponential random graph models estimated the probability of evidence provision and request between actors, controlling for network- and individual-level covariates. Logistic regression models estimated actors’ use of research evidence to inform policy.
Network structure explained more than half of the evidence exchanges (ties) observed in these networks. Across all cases, a pair of actors was more likely to form a provision tie if they already had a request tie between them and visa versa (θ = 6.16, p < 0.05; θ = 2.87, p < 0.05; θ = 2.31, p < 0.05). The child health network displayed clustering tendencies, meaning that actors were more likely to form ties if they shared an acquaintance (θ = 2.36, p < 0.05). Actors’ use of research evidence was positively associated with their centrality (i.e., connectedness).
The exchange and use of research evidence in policymaking can be partly explained by the structure of actors’ networks of relationships. Efforts to support knowledge translation and evidence-informed policymaking should consider network factors.
Electronic supplementary material
The online version of this article (doi:10.1186/s13012-014-0126-8) contains supplementary material, which is available to authorized users.
Evidence-informed health policy; Knowledge translation; Social network analysis; Exponential random graph model; Policy network
Activity-based funding (ABF) of hospitals is a policy intervention intended to re-shape incentives across health systems through the use of diagnosis-related groups. Many countries are adopting or actively promoting ABF. We assessed the effect of ABF on key measures potentially affecting patients and health care systems: mortality (acute and post-acute care); readmission rates; discharge rate to post-acute care following hospitalization; severity of illness; volume of care.
We undertook a systematic review and meta-analysis of the worldwide evidence produced since 1980. We included all studies reporting original quantitative data comparing the impact of ABF versus alternative funding systems in acute care settings, regardless of language. We searched 9 electronic databases (OVID MEDLINE, EMBASE, OVID Healthstar, CINAHL, Cochrane CENTRAL, Health Technology Assessment, NHS Economic Evaluation Database, Cochrane Database of Systematic Reviews, and Business Source), hand-searched reference lists, and consulted with experts. Paired reviewers independently screened for eligibility, abstracted data, and assessed study credibility according to a pre-defined scoring system, resolving conflicts by discussion or adjudication.
Of 16,565 unique citations, 50 US studies and 15 studies from 9 other countries proved eligible (i.e. Australia, Austria, England, Germany, Israel, Italy, Scotland, Sweden, Switzerland). We found consistent and robust differences between ABF and no-ABF in discharge to post-acute care, showing a 24% increase with ABF (pooled relative risk = 1.24, 95% CI 1.18–1.31). Results also suggested a possible increase in readmission with ABF, and an apparent increase in severity of illness, perhaps reflecting differences in diagnostic coding. Although we found no consistent, systematic differences in mortality rates and volume of care, results varied widely across studies, some suggesting appreciable benefits from ABF, and others suggesting deleterious consequences.
Transitioning to ABF is associated with important policy- and clinically-relevant changes. Evidence suggests substantial increases in admissions to post-acute care following hospitalization, with implications for system capacity and equitable access to care. High variability in results of other outcomes leaves the impact in particular settings uncertain, and may not allow a jurisdiction to predict if ABF would be harmless. Decision-makers considering ABF should plan for likely increases in post-acute care admissions, and be aware of the large uncertainty around impacts on other critical outcomes.
This protocol builds on the development of a) a framework that identified the various supports (i.e. positions, activities, interventions) that a healthcare organisation or health system can implement for evidence-informed decision-making (EIDM) and b) a qualitative study that showed the current mix of supports that some Canadian healthcare organisations have in place and the ones that are perceived to facilitate the use of research evidence in decision-making. Based on these findings, we developed a web survey to collect cross-sectional data about the specific supports that regional health authorities and hospitals in two Canadian provinces (Ontario and Quebec) have in place to facilitate EIDM.
This paper describes the methods for a cross-sectional web survey among 32 regional health authorities and 253 hospitals in the provinces of Quebec and Ontario (Canada) to collect data on the current mix of organisational supports that these organisations have in place to facilitate evidence-informed decision-making. The data will be obtained through a two-step survey design: a 10-min survey among CEOs to identify key units and individuals in regard to our objectives (step 1) and a 20-min survey among managers of the key units identified in step 1 to collect information about the activities performed by their unit regarding the acquisition, assessment, adaptation and/or dissemination of research evidence in decision-making (step 2). The study will target three types of informants: CEOs, library/documentation centre managers and all other key managers whose unit is involved in the acquisition, assessment, adaptation/packaging and/or dissemination of research evidence in decision-making. We developed an innovative data collection system to increase the likelihood that only the best-informed respondent available answers each survey question. The reporting of the results will be done using descriptive statistics of supports by organisation type and by province.
This study will be the first to collect and report large-scale cross-sectional data on the current mix of supports health system organisations in the two most populous Canadian provinces have in place for evidence-informed decision-making. The study will also provide useful information to researchers on how to collect organisation-level data with reduced risk of self-reporting bias.
Health systems; Knowledge translation; Research evidence; Cross-sectional study
To improve quality of care and patient outcomes, health system decision-makers need to identify and implement effective interventions. An increasing number of systematic reviews document the effects of quality improvement programs to assist decision-makers in developing new initiatives. However, limitations in the reporting of primary studies and current meta-analysis methods (including approaches for exploring heterogeneity) reduce the utility of existing syntheses for health system decision-makers. This study will explore the role of innovative meta-analysis approaches and the added value of enriched and updated data for increasing the utility of systematic reviews of complex interventions.
We will use the dataset from our recent systematic review of 142 randomized trials of diabetes quality improvement programs to evaluate novel approaches for exploring heterogeneity. These will include exploratory methods, such as multivariate meta-regression analyses and all-subsets combinatorial meta-analysis. We will then update our systematic review to include new trials and enrich the dataset by surveying authors of all included trials. In doing so, we will explore the impact of variables not, reported in previous publications, such as details of study context, on the effectiveness of the intervention. We will use innovative analytical methods on the enriched and updated dataset to identify key success factors in the implementation of quality improvement interventions for diabetes. Decision-makers will be involved throughout to help identify and prioritize variables to be explored and to aid in the interpretation and dissemination of results.
This study will inform future systematic reviews of complex interventions and describe the value of enriching and updating data for exploring heterogeneity in meta-analysis. It will also result in an updated comprehensive systematic review of diabetes quality improvement interventions that will be useful to health system decision-makers in developing interventions to improve outcomes for people with diabetes.
Systematic review registration
PROSPERO registration no. CRD42013005165
Diabetes care; Knowledge translation; Quality improvement interventions; Complex Interventions; Health system decision-makers; Systematic review; Meta-analysis; Implementation science; Heterogeneity; Hierarchical modeling
The growing recognition of the importance of concisely communicating research evidence and other policy-relevant information to policymakers has underpinned the development of several information-packaging efforts over the past decade. This has led to a wide variability in the types of documents produced, which is at best confusing and at worst discouraging for those they intend to reach. This paper has two main objectives: to develop a better understanding of the range of documents and document names used by the organizations preparing them; and to assess whether there are any consistencies in the characteristics of sampled documents across the names employed to label (in the title) or describe (in the document or website) them.
We undertook a documentary analysis of web-published document series that are prepared by a variety of organizations with the primary intention of providing information to health systems policymakers and stakeholders, and addressing questions related to health policy and health systems with a focus on low- and middle-income countries. No time limit was set.
In total, 109 individual documents from 24 series produced by 16 different organizations were included. The name ‘policy brief/briefing’ was the most frequently used (39%) to label or describe a document, and was used in all eight broad content areas that we identified, even though they did not have obviously common traits among them. In terms of document characteristics, most documents (90%) used skimmable formats that are easy to read, with understandable, jargon-free, language (80%). Availability of information on the methods (47%) or the quality of the presented evidence (27%) was less common. One-third (32%) chose the topic based on an explicit process to assess the demand for information from policy makers and even fewer (19%) engaged with policymakers to discuss the content of these documents such as through merit review.
This study highlights the need for organizations embarking on future information-packaging efforts to be more thoughtful when deciding how to name these documents and the need for greater transparency in describing their content, purpose and intended audience.
Policy briefs; Health systems; Health policy; Synthesis; Summaries; Policymaking; Evidence-based
Meetings and conferences are often used as a tool to disseminate information, network with colleagues, and/or set direction for a field of study, but there is little evidence to support whether such events achieve their objectives. This study evaluates the International Forum on Evidence Informed Health Policymaking (EIHP), a three-day meeting held in Addis Ababa, Ethiopia, in 2012, to determine the success of the meeting based on pre-determined objectives.
The evaluation strategy was developed based on a previously published conference evaluation framework and operationalized as an end-of-conference participant survey that incorporated both process (programme/organization) and outcome measures (potential changes in behaviour).
Sixty seven of approximately 121 attendees filled out a questionnaire (a 55% response rate) and, overall, participants rated the programme components and plenary sessions very highly. The top three benefits reported by participants were: i) sharing experiences and lessons learned (75%); ii) new opportunities for future collaboration (69%); and iii) new knowledge (67%). Conversely, only 25% or less of meeting participants reported an intent to utilize any of the potential benefits highlighted in the questionnaire, with the notable exception of pursuing new opportunities for future collaboration.
The evaluation findings suggest that the International Forum achieved its objectives of sharing experiences with EIHP and providing opportunities for networking among EIHP initiatives, although there are limited prospects for direct improvements to efforts to support EIHP.
Conferences; Evaluations; Meetings; Questionnaires
Little is known about the extent to which research evidence informs the development of recommendations by international organizations.
We identified specific World Health Organization (WHO) and World Bank recommendations on five topics (contracting, healthcare financing, health human resources, tuberculosis control and tobacco control), catalogued the related systematic reviews and assessed the recommendations to determine their consistency with the systematic reviews that were available at the time of their formulation.
Only two of the eight publications examined were found to cite systematic reviews, and only five of 14 WHO and two of seven World Bank recommendations were consistent with both the direction and nature of effect claims from systematic reviews. Ten of 14 WHO and five of seven World Bank recommendations were consistent with the direction of effect claims only.
WHO and the World Bank – working with donor agencies and national governments – can improve their use of (or at least, their reporting about their use of) research evidence. Decision-makers and clinicians should critically evaluate the quality and local applicability of recommendations from any source, including international organizations, prior to their implementation.
Many international statements have urged researchers, policy-makers and health care providers to collaborate in efforts to bridge the gaps between research, policy and practice in low- and middle-income countries. We surveyed researchers in 10 countries about their involvement in such efforts.
We surveyed 308 researchers who conducted research on one of four clinical areas relevant to the Millennium Development Goals (prevention of malaria, care of women seeking contraception, care of children with diarrhea and care of patients with tuberculosis) in each of 10 low- and middle-income countries (China, Ghana, India, Iran, Kazakhstan, Laos, Mexico, Pakistan, Senegal and Tanzania). We focused on their engagement in three promising bridging activities and examined system-level, organizational and individual correlates of these activities.
Less than half of the researchers surveyed reported that they engaged in one or more of the three promising bridging activities: 27% provided systematic reviews of the research literature to their target audiences, 40% provided access to a searchable database of research products on their topic, and 43% established or maintained long-term partnerships related to their topic with representatives of the target audience. Three factors emerged as statistically significant predictors of respondents’ engagement in these activities: the existence of structures and processes to link researchers and their target audiences predicted both the provision of access to a database (odds ratio [OR] 2.62, 95% CI 1.30–5.27) and the establishment or maintenance of partnerships (OR 2.65, 95% CI 1.25–5.64); stability in their contacts predicted the provision of systematic reviews (OR 2.88, 95% CI 1.35–6.13); and having managers and public (government) policy-makers among their target audiences predicted the provision of both systematic reviews (OR 4.57, 95% CI 1.78–11.72) and access to a database (OR 2.55, 95% CI 1.20–5.43).
Our findings suggest potential areas for improvement in light of the bridging strategies targeted at health care providers that have been found to be effective in some contexts and the factors that appear to increase the prospects for using research in policy-making.
Policymakers and stakeholders need immediate access to many types of research evidence to make informed decisions about the full range of questions that may arise regarding health systems.
We examined all types of research evidence about governance, financial and delivery arrangements, and implementation strategies within health systems contained in Health Systems Evidence (HSE) (http://www.healthsystemsevidence.org). The research evidence types include evidence briefs for policy, overviews of systematic reviews, systematic reviews of effects, systematic reviews addressing other questions, systematic reviews in progress, systematic reviews being planned, economic evaluations, and health reform and health system descriptions. Specifically, we describe their distribution across health system topics and domains, trends in their production over time, availability of supplemental content in various languages, and the extent to which they focus on low- and middle-income countries (LMICs), as well as (for systematic reviews) their methodological quality and the availability of user-friendly summaries.
As of July 2013, HSE contained 2,629 systematic reviews of effects (of which 501 are Cochrane reviews), 614 systematic reviews addressing other questions, 283 systematic reviews in progress, 186 systematic reviews being planned, 140 review-derived products (evidence briefs and overviews of systematic reviews), 1,669 economic evaluations, 1,092 health reform descriptions, and 209 health system descriptions. Most systematic reviews address topics related to delivery arrangements (n = 2,663) or implementation strategies (n = 1,653) with far fewer addressing financial (n = 241) or governance arrangements (n = 231). In addition, 2,928 systematic reviews have been quality appraised with moderate AMSTAR ratings found for reviews addressing governance (5.6/11), financial (5.9/11), and delivery (6.3/11) arrangements and implementation strategies (6.5/11); 1,075 systematic reviews have no independently produced user-friendly summary and only 737 systematic reviews have an LMIC focus. Literature searches for half of the systematic reviews (n = 1,584, 49%) were conducted within the last five years.
Greater effort needs to focus on assessing whether the current distribution of systematic reviews corresponds to policymakers’ and stakeholders’ priorities, updating systematic reviews, increasing the quality of systematic reviews, and focusing on LMICs.
John Lavis discusses how health policymakers and their stakeholders need research evidence, and the best ways evidence can be synthesized and packaged to optimize its use.
Decisions regarding health systems are sometimes made without the input of timely and reliable evidence, leading to less than optimal health outcomes. Healthcare organizations can implement tools and infrastructures to support the use of research evidence to inform decision-making.
The purpose of this study was to profile the supports and instruments (i.e., programs, interventions, instruments or tools) that healthcare organizations currently have in place and which ones were perceived to facilitate evidence-informed decision-making.
In-depth semi-structured telephone interviews were conducted with individuals in three different types of positions (i.e., a senior management team member, a library manager, and a ‘knowledge broker’) in three types of healthcare organizations (i.e., regional health authorities, hospitals and primary care practices) in two Canadian provinces (i.e., Ontario and Quebec). The interviews were taped, transcribed, and then analyzed thematically using NVivo 9 qualitative data analysis software.
A total of 57 interviews were conducted in 25 organizations in Ontario and Quebec. The main findings suggest that, for the healthcare organizations that participated in this study, the following supports facilitate evidence-informed decision-making: facilitating roles that actively promote research use within the organization; establishing ties to researchers and opinion leaders outside the organization; a technical infrastructure that provides access to research evidence, such as databases; and provision and participation in training programs to enhance staff’s capacity building.
This study identified the need for having a receptive climate, which laid the foundation for the implementation of other tangible initiatives and supported the use of research in decision-making. This study adds to the literature on organizational efforts that can increase the use of research evidence in decision-making. Some of the identified supports may increase the use of research evidence by decision-makers, which may then lead to more informed decisions, and hopefully to a strengthened health system and improved health.
Evidence-informed health policymaking logically depends on timely access to research evidence. To our knowledge, despite the substantial political and societal pressure to enhance the use of the best available research evidence in public health policy and program decision making, there is no study addressing availability of peer-reviewed research in Canadian health ministries.
To assess availability of (1) a purposive sample of high-ranking scientific journals, (2) bibliographic databases, and (3) health library services in the fourteen Canadian health ministries.
From May to October 2011, we conducted a cross-sectional survey among librarians employed by Canadian health ministries to collect information relative to availability of scientific journals, bibliographic databases, and health library services. Availability of scientific journals in each ministry was determined using a sample of 48 journals selected from the 2009 Journal Citation Reports (Sciences and Social Sciences Editions). Selection criteria were: relevance for health policy based on scope note information about subject categories and journal popularity based on impact factors.
We found that the majority of Canadian health ministries did not have subscription access to key journals and relied heavily on interlibrary loans. Overall, based on a sample of high-ranking scientific journals, availability of journals through interlibrary loans, online and print-only subscriptions was estimated at 63%, 28% and 3%, respectively. Health Canada had a 2.3-fold higher number of journal subscriptions than that of the provincial ministries’ average. Most of the organisations provided access to numerous discipline-specific and multidisciplinary databases. Many organisations provided access to the library resources described through library partnerships or consortia. No professionally led health library environment was found in four out of fourteen Canadian health ministries (i.e. Manitoba Health, Northwest Territories Department of Health and Social Services, Nunavut Department of Health and Social Services and Yukon Department of Health and Social Services).
There is inequity in availability of peer-reviewed research in the fourteen Canadian health ministries. This inequity could present a problem, as each province and territory is responsible for formulating and implementing evidence-informed health policies and services for the benefit of its population.
Health care; Information science; Library science; Knowledge transfer; Research evidence
Differences in medical care in the United States compared with Canada, including greater reliance on private funding and for-profit delivery, as well as markedly higher expenditures, may result in different health outcomes.
To systematically review studies comparing health outcomes in the United States and Canada among patients treated for similar underlying medical conditions.
We identified studies comparing health outcomes of patients in Canada and the United States by searching multiple bibliographic databases and resources. We masked study results before determining study eligibility. We abstracted study characteristics, including methodological quality and generalizability.
We identified 38 studies comparing populations of patients in Canada and the United States. Studies addressed diverse problems, including cancer, coronary artery disease, chronic medical illnesses and surgical procedures. Of 10 studies that included extensive statistical adjustment and enrolled broad populations, 5 favoured Canada, 2 favoured the United States, and 3 showed equivalent or mixed results. Of 28 studies that failed one of these criteria, 9 favoured Canada, 3 favoured the United States, and 16 showed equivalent or mixed results. Overall, results for mortality favoured Canada (relative risk 0.95, 95% confidence interval 0.92-0.98, p= 0.002) but were very heterogeneous, and we failed to find convincing explanations for this heterogeneity. The only condition in which results consistently favoured one country was end-stage renal disease, in which Canadian patients fared better.
Available studies suggest that health outcomes may be superior in patients cared for in Canada versus the United States, but differences are not consistent.
Participants in the Cochrane Collaboration conduct and periodically update systematic reviews that address the question, “What works?” for healthcare interventions. The Cochrane Library makes available quality-appraised systematic reviews that address this question. No coordinated effort has been undertaken to conduct and periodically update systematic reviews that address the other types of questions asked by healthcare managers and policy makers, to adapt existing reviews to highlight decision-relevant information (including the factors that may affect assessments of a review’s local applicability) or to facilitate their retrieval through a “one-stop shopping” portal. Researchers interested in evaluating new methodological developments, health services and policy researchers interested in conducting and adapting systematic reviews, and research funders all have a role to play in making systematic reviews more useful for healthcare managers and policy makers.
Systematic reviews are increasingly seen as helpful “knowledge support” for managers and policy makers, and deliberative processes are starting to be seen as promising, locally contextualized “decision support.” Increases to the flow of systematic reviews should be complemented by efforts to facilitate the retrieval, and adapt the presentation, of the available stock of systematic reviews. Research and other evidence should be combined in transparent ways to facilitate cross-context learning. The challenge for managers and policy makers in moving forward will be to avoid the confusion that comes from the branding of both systematic reviews and deliberative processes.
Community-based organizations are important health system stakeholders as they provide numerous, often highly valued programs and services to the members of their community. However, community-based organizations are described using diverse terminology and concepts from across a range of disciplines. To better understand the literature related to community-based organizations in the health sector (i.e., those working in health systems or more broadly to address population or public health issues), we conducted a scoping review by using an iterative process to identify existing literature, conceptually map it, and identify gaps and areas for future inquiry.
We searched 18 databases and conducted citation searches using 15 articles to identify relevant literature. All search results were reviewed in duplicate and were included if they addressed the key characteristics of community-based organizations or networks of community-based organizations. We then coded all included articles based on the country focus, type of literature, source of literature, academic discipline, disease sector, terminology used to describe organizations and topics discussed.
We identified 186 articles addressing topics related to the key characteristics of community-based organizations and/or networks of community-based organizations. The literature is largely focused on high-income countries and on mental health and addictions, HIV/AIDS or general/unspecified populations. A large number of different terms have been used in the literature to describe community-based organizations and the literature addresses a range of topics about them (mandate, structure, revenue sources and type and skills or skill mix of staff), the involvement of community members in organizations, how organizations contribute to community organizing and development and how they function in networks with each other and with government (e.g., in policy networks).
Given the range of terms used to describe community-based organizations, this scoping review can be used to further map their meanings/definitions to develop a more comprehensive typology and understanding of community-based organizations. This information can be used in further investigations about the ways in which community-based organizations can be engaged in health system decision-making and the mechanisms available for facilitating or supporting their engagement.
Large multi-day conferences have often been criticized as ineffective ways to improve social outcomes and to influence policy or practice. Unfortunately, many conference evaluations have also been inadequate in determining the impact of a conference on its associated social sector, with little evidence gathered or analyzed to substantiate or refute these criticisms. The aim of this scoping review is to investigate and report stakeholders’ objectives for planning or participating in large multi-day conferences and how these objectives are being evaluated. We conducted a scoping review supplemented by a small number of key informant interviews. Eight bibliographic databases were systematically searched to identify papers describing conference objectives and/or evaluations. We developed a conference evaluation framework based on theoretical models and empirical findings, which structured the descriptive synthesis of the data. We identified 3,073 potential papers for review, of which 44 were included in this study. Our evaluation framework connects five key elements in planning a conference and its evaluation (number in brackets refers to number of themes identified): conference objectives (8), purpose of evaluation (7), evaluation methods (5), indicators of success (9) and theories/models (8). Further analysis of indicators of success identified three categories of indicators with differing scopes (i.e. immediate, prospective or follow-up) as well as empirical links between the purpose of evaluations and these indicators. Conference objectives and evaluations were largely correlated with the type of conference (i.e. academic, political/governmental or business) but diverse overall. While much can be done to improve the quality and usefulness of conference evaluations, there are innovative assessments that are currently being utilized by some conferences and warrant further investigation. This review provides conference evaluators and organizers a simple resource to improve their own assessments by highlighting and categorizing potential objectives and evaluation strategies.
Conferences and symposia; Conference objectives; Program evaluations; Learning and behaviour theories; Indicators of success; Evaluation methods
Evidence from health services research (HSR) is currently thinly spread through many journals, making it difficult for health services researchers, managers and policy-makers to find research on clinical practice guidelines and the appropriateness, process, outcomes, cost and economics of health care services. We undertook to develop and test search terms to retrieve from the MEDLINE database HSR articles meeting minimum quality standards.
The retrieval performance of 7445 methodologic search terms and phrases in MEDLINE (the test) were compared with a hand search of the literature (the gold standard) for each issue of 68 journal titles for the year 2000 (a total of 25 936 articles). We determined sensitivity, specificity and precision (the positive predictive value) of the MEDLINE search strategies.
A majority of the articles that were classified as outcome assessment, but fewer than half of those in the other categories, were considered methodologically acceptable (no methodologic criteria were applied for cost studies). Combining individual search terms to maximize sensitivity, while keeping specificity at 50% or more, led to sensitivities in the range of 88.1% to 100% for several categories (specificities ranged from 52.9% to 97.4%). When terms were combined to maximize specificity while keeping sensitivity at 50% or more, specificities of 88.8% to 99.8% were achieved. When terms were combined to maximize sensitivity and specificity while minimizing the differences between the 2 measurements, most strategies for HSR categories achieved sensitivity and specificity of at least 80%.
Sensitive and specific search strategies were validated for retrieval of HSR literature from MEDLINE. These strategies have been made available for public use by the US National Library of Medicine at www.nlm.nih.gov/nichsr/hedges/search.html.