|Home | About | Journals | Submit | Contact Us | Français|
Conceived and designed the experiments: XBC. Performed the experiments: XBC. Analyzed the data: XBC. Wrote the first draft of the manuscript: XBC. Contributed to the writing of the manuscript: XBC JL SL RA JAR DD LB AB FEJ LG SO KW PT RK TP AH. ICMJE criteria for authorship read and met: XBC JL SL RA JAR DD LB AB FEJ LG SO KW PT RK TP AH. Agree with manuscript results and conclusions: XBC JL SL RA JAR DD LB AB FEJ LG SO KW PT RK TP AH.
This is one paper in a three-part series that sets out how evidence should be translated into guidance to inform policies on health systems and improve the delivery of clinical and public health interventions.
Present trends suggest that many of the poorest countries in the world, including many in sub-Saharan Africa, will not meet the health-related Millennium Development Goals  (MDGs), especially MDG 4 (reducing under-five mortality) and MDG 5 (reducing maternal mortality) . Even in those countries that are on track to meet health MDGs, striking inequities exist among countries and among socioeconomic groups within them , despite effective and cost-effective interventions being available to improve population health, including that of vulnerable groups . Such interventions are delivered through health systems, which consist of “all organisations, people and actions whose primary intent is to promote, restore or maintain health” , but, in many settings, interactions between weakened health systems and the sometimes conflicting demands of single-disease intervention programmes are hindering the uptake and implementation of life-saving interventions –. A growing number of governments, international institutions, and funding agencies have therefore recognised the urgent need to coordinate and harmonise investments in health systems strengthening in low- and middle-income countries (LMICs) to provide universal social protection and effective coverage of essential health interventions .
Investments in health systems aim to “enhance [their] performance…for meeting the needs of patients and populations in an equitable and efficient manner”  while reducing the risk of impoverishment due to the costs of care . However, although a number of broad principles have been proposed , there is no wide agreement on the operational definition of health systems strengthening , and it remains unclear how health systems can best be strengthened. Because the evidence base addressing this issue is patchy , health systems research has recently been identified as a priority  and its definition and scope have been outlined . Indeed, the need for greater capacity to produce evidence to inform health systems strengthening was one of the drivers that led to the first global symposium on health systems research (Montreux, Switzerland, November 2010) , at which some of the issues developed in this article were presented and discussed.
Importantly, to be useful to policy makers, research evidence needs to be retrieved, its quality appraised, and the recommended options properly framed in the form of guidance. In an analogy with clinical practice guidelines (“systematically developed statements to assist practitioner and patient decisions about appropriate health care for specific clinical circumstances” ), we define health systems guidance as systematically developed statements produced at global or national levels to assist decisions about appropriate options for addressing a health systems challenge in a range of settings and to assist with the implementation of these options and their monitoring and evaluation (Box 1). We use the term “guidance” rather than “guidelines”, as health systems and the evidence on health systems are highly context sensitive. Health systems guidance statements refer to policy options that are accompanied by assessments of the quality of evidence supporting them and the potential for unintended harms, and by discussions of implementation and contextual issues.
Evidence-informed health systems guidance tackles health systems problems by:
Guidance needs to be transparent, systematic, and adapted to the local contexts. It therefore needs to use validated approaches, to consider all the available evidence and to assess its quality. It also needs to take into account local factors that may influence the effects of all options recommended and to address their feasibility.
Several global institutions have legitimate roles in producing guidance, often in response to requests from national decision makers (see the second paper in this series ). Thus, to avoid unnecessary duplication of the enormous efforts of guidance development, coordination is needed at the global level. However, as decisions on the options recommended are taken at national level, global guidance needs adaptation, ideally through national deliberative processes.
The production of guidance is largely based on knowledge translation approaches that bridge the gap between research evidence and its application to policy-making ,. Figure 1 schematically represents these approaches across the research, policy, managerial, and societal domains.
The need for evidence-informed guidance on policies that impact health systems performance is widely accepted  and is one of the six priorities of the director-general of the World Health Organization (WHO) . Moreover, a World Health Assembly resolution  recently urged member states to use evidence-based approaches to assess “country's health and health systems challenges” and to develop “evidence-based responses to evolving challenges and opportunities, and to involve all relevant stakeholders”. However, although well-established methods exist to develop clinical guidelines , there is little experience in developing health systems guidance and the process poses conceptual and methodological challenges related to the different types of evidence to be considered, the complexity of health systems and the pre-eminence of contextual issues. The current experience is mainly related to the development of different by-products of research syntheses and decision aids  targeted towards policy makers, rather than systematically and transparently developed guidance.
The second article in this series explores the challenge of linking guidance development and policy development at global and national levels and examines the range of factors that can influence policy development , and the third article explores the challenge of assessing how much confidence to place in evidence on health systems interventions .
To assess the need for guidance and how it is formulated, we scrutinised the use of guidance-related terms in national health policy and strategic documents from LMICs (Box 2). We found that the terms “guidance” or “guidelines” frequently appeared in strategic documents but were more often related to clinical matters than to health systems (Box 3).
To assess the need of guidance and how it is formulated:
The scope of guidance could be discerned in 63 statements from the included documents:
The area of guidance could be identified in 407 statements:
In 283 statements (either related to clinical, public health, or miscellaneous issues), references to one or more WHO health systems building blocks  included:
Documents from eight countries had explicit statements suggesting guidance as a strategy to improve the “quality of care”. Guidance statements related to traditional medicine were found in The Gambia, Liberia, Rwanda, and Zimbabwe.
The main bodies developing guidance were international organisations (e.g., WHO), ministries of health, and special national committees usually linked to the ministries of health. Guidance production and use was mainly referred to in connection with high-level health sector entities, or at the sub-national level in decentralised structures, although similar terms were used in relation to implementation strategies or operational instructions. There were also statements emphasising the need for producing guidance, for making it accessible to users, and for reinforcing adherence. Finally, some documents explicitly related the development of guidance to the concepts of evidence-based health care.
Our search had several limitations. First, we may have missed some relevant documents. Second, statements do not necessarily reflect the status that governments or stakeholders ascribe to guidance. Finally, the terms used to refer to health systems guidance can differ between countries and across languages. Our findings nevertheless show that the term “guidance” appeared frequently in strategic documents. In its “technical” meaning (i.e., systematically developed statements to assist decisions), the term seems to be more often related to the clinical field rather than to health systems. This could be because the concept of formal guidance applied to health systems decisions is not yet well-established among policy makers or because there are only few examples of such guidance to draw from. However, the statements linking guidance in a more generic sense to health systems actions were plentiful in relation to the functions typically ascribed to ministries of health.
To improve how WHO responds to requests for guidance on health systems and the quality of the guidance produced, WHO recently commissioned the production of a Handbook outlining approaches to develop health systems guidance . A Task Force on Developing Health Systems Guidance (listed in the Acknowledgments section) was formed from WHO expert panels and from experts working in the field of health systems research at international and academic institutions and oversaw the drafting of the Handbook for Developing Health Systems Guidance: Supporting Informed Judgements for Health Systems Policies  (which was based on existing best practices for the development of clinical guidelines, on approaches and tools described in the literature, and on the expertise of the Task Force members) by reviewing and commenting on its content by email and through regular teleconferences and meetings in person. During the production of the Handbook, because evidence from research (for example, on the effectiveness and acceptability of health systems interventions) is not guidance and is insufficient for optimal decision-making , several conceptual and methodological challenges associated with the production of health systems guidance were identified. The writing group for this paper further considered these issues and produced this manuscript, which was finalised after several iterations of comments by the Task Force and external reviewers. Here, we discuss four specific challenges that were encountered during the production of the Handbook, namely:
Many processes, frequently involving iterations from research evidence into policy formulation and from policy evaluation into research prioritisation, are needed to bridge the gaps between research, policy, and practice. The complexity of these processes demands a dynamic framework  that is comprehensive and incorporates current thinking about evidence, policy formulation, and health systems such as the one shown in Figure 1.
Research is articulated around research questions that can be addressed using particular methods and is driven by research opportunities (e.g., funding), researchers' interests, and feasibility. By contrast, policy decisions and the managerial arrangements needed to put them into practice have to be responsive to population needs and integrated into complex health systems that go beyond the mere aggregation of single interventions. For example, the research question of whether lay health workers are effective in delivering specific health care interventions will become fully relevant for policy only when the essential components of the intervention (e.g., training of lay health workers), related actions (e.g., adaptations needed in the distributions of tasks of cadres), implementation issues (e.g., the preferences of potential clients), and the implications across other health system building blocks (e.g., adaptations to the health information sub-system that may be needed to capture the tasks undertaken by such workers) are all considered.
Ideally, implementation and contextual issues should be considered when conducting the systematic reviews of effectiveness that are needed to translate research into health systems guidance or in other types of reviews addressing them ,. Although there are frameworks to analyse barriers to implementation  and the applicability of evidence , many systematic reviews fail to consider issues pertaining to applicability and equity, or the complexity of interventions in relation to their technical feasibility . This is partially due to the lack of this type of information in the primary research that underlies systematic reviews. Where this evidence is missing or scarce, efforts to search, appraise, and synthesise additional evidence on implementation issues will have to be made. It will also be important to consider how to address inequities in coverage of health care , which are pervasive in many countries.
The production of guidance not only entails putting research findings into context, but also entails taking a health systems perspective. New trends in health systems thinking advocate the construction of conceptual pathways that look at the interventions from a systems perspective, making explicit how interventions may trigger reactions in related components of the health system that may produce unintended consequences ,. For example, increasing the salary of health workers in HIV/AIDS treatment programmes may result in a reduction of the health workforce addressing other conditions who do not receive the same salary benefits.
Finally, it is important to note that the study designs for research (and the methods used to synthesise research findings) that explore different health systems components vary. For example, research exploring the governance and leadership components of health systems is likely to involve qualitative methods, whereas research on health care delivery could utilise both qualitative and quantitative methods (see also the third paper in this series ). Qualitative synthesis methods have addressed the nature of problems such as patient adherence to treatment for tuberculosis . Syntheses of mixed method research can address policy-oriented questions using framework synthesis, a method that has been applied to qualitative and mixed methods research addressing research management , public health ,, and workforce management .
Although research evidence to inform guidance is generated through research synthesis (i.e., systematic reviews) and often uses complex statistical methods, guidance is typically produced through a deliberative process where evidence is interpreted and contextualised (Figure 1, Policy domain). During the deliberative process, which can include a wide range of stakeholders, potential users, and beneficiaries of guidance, knowledge is exchanged, filtered (stakeholders decide on the relevance of evidence), and amplified (stakeholders stress evidence consistent with their views). Colloquial (non-research) evidence (e.g., tacit or experiential knowledge) helps to interpret or contextualise research evidence and addresses issues for which research evidence is not available but that may play a role in the decisions (e.g., considerations about the political implications of decisions, or about potential vested interests of stakeholders) . Making colloquial evidence explicit adds transparency to the guidance development process.
Typically, users and producers of guidance, who may include economists, managers, administrators, social scientists and other professional groups as well as elected politicians, come from very different research traditions or none at all and have very different approaches to research and decision-making . Therefore, it is probably unrealistic to expect that they will all be familiar with issues such as assessing the quality or appropriateness of systematic reviews . Furthermore, even in the ideal situation where policy makers are familiar with research methods, they may choose to ignore evidence that creates uncertainty, questions conventional knowledge, ignores local context , or that is not consistent with pre-determined ideas of preferred policy options.
Although evidence on the effects of health systems interventions is relatively scarce, there are renewed efforts to increase availability and accessibility of this evidence ,,. Such evidence should be presented in user-friendly formats (e.g., minimising technicalities such as complex statistical outputs) to increase its accessibility when producing guidance and to facilitate appropriate interpretation by those without a strong research background. Finally, the preferences of different decision makers for how evidence and guidance should be presented should be taken into account.
Timeliness in the production of evidence and guidance is critical, but systematic reviews can take between one and two years to complete depending on the availability of new evidence, and guidance development can take a year or more . The bottom of Figure 1 also shows the potentially long time frames for the policy decision processes and the evaluation of policies. Furthermore, guidance needs to be regularly updated in the light of new evidence or changes and modifications to existing, previously accepted evidence.
Without timely production of research synthesis and guidance, the gap between research, policy, and practice cannot be bridged, even if primary research is available. Alternative approaches and methods are therefore needed to reduce the lengthy time frames needed for producing systematic reviews and for translating the findings of reviews into guidance. Approaches that may be useful include rapid assessment methods , the use of text mining to speed the search for publications , and the adaptation of existing guidance that can be institutionally endorsed by the organisations producing or using it .
The quality of guidance has to be assessed to ascertain the extent to which the guidance used state-of-the-art and validated methods during its development and is, therefore, balanced and reliable in relation to the evidence that informs it. The AGREE instrument (Appraisal of Guidelines Research and Evaluation) and its recent revision (AGREE II) is used to assess the quality of clinical guidelines but could be adapted to assess health systems guidance. The AGREE instrument, which consists of 23 items, aims to assess the quality of guidelines, to provide a methodological strategy for the development of guidelines, and to inform guideline developers on what and how information ought to be reported in guidelines . Many of its items are applicable to both clinical guidelines and health systems guidance, but some need minor adaptation for the latter. For example, item 16 in the AGREE II instrument mentions the “different options for the management of the condition or health issue are clearly presented” whereas in health systems guidance this would need to be reworded to indicate “different options for addressing the health system topic are clearly presented”. In addition, there should be specific reference to the need for health systems guidance to refer to the contextual factors that would determine the extent to which research evidence is applicable in specific circumstances.
Passive dissemination of guidance will not ensure its uptake by potential users , particularly if it lacks relevance to local or national situations. There are several proven approaches for the effective dissemination of research evidence and the promotion of clinical guidelines (e.g., distribution of educational materials, educational meetings and outreach visits, involvement of local opinion leaders, audit and feedback) that have moderate effects on outcomes . Some of these approaches may also be relevant to health systems guidance, although the challenges relating to dissemination and uptake for these two types of guidance are different. At a national level, a relatively small number of decision makers may need to be influenced to integrate guidance into national health policies and plans. Alternatively, sub-national and local decision makers may be in a better position to influence guidance implementation, particularly in more decentralised systems. “Policy briefs” and “policy dialogues” are examples of tools used to improve the contextualisation and utilisation of guidance  (discussed further in the second article in this series ). Although uptake of guidance depends on how it is presented and disseminated, additional factors may increase the utilisation of scientific evidence by policy-makers. These include early, informal interactions between researchers and policy makers, the relevance and timeliness of evidence, and the consistency between the evidence and the recommended options on the one hand, and the beliefs, values, interests, or political goals of policy makers on the other .
We found that high-level policy and strategic health sector documents frequently mention the need for the development of health systems guidance but suggest that there may still be limited awareness that health systems guidance can provide key inputs into policy-making and that the development of such guidance requires systematic and transparent approaches, inspired by the development of clinical guidelines. However, given the growing amount of evidence on health systems and the current initiatives to address methodological challenges to produce health systems guidance (e.g., the Handbook for Developing Health Systems Guidance ), the international health community at all levels is likely to be faced with increasing demands for guidance on health systems issues.
The approaches we identified for producing guidance for health systems typically mirror validated methods developed for clinical guidelines but with some important differences. Specifically, to develop health systems guidance that is relevant and useful to decision makers, it is necessary to acknowledge and to find ways of incorporating the complex interrelations of the system components, and the numerous contextual factors that may influence the effectiveness of interventions, particularly their effects on disadvantaged populations. These issues are discussed further in the second paper in this series . The uptake of guidance by decision makers and the additional complexity of the decision-making process itself are also important.
Health systems guidance has the potential to improve decision-making and enable more efficient use of resources with consequent improvements in the health of populations. However, such guidance needs rigorous and transparent processes of production and evidence-based approaches to ensure its dissemination and uptake. These processes and approaches are still at a rudimentary stage of development. Importantly, the availability of health systems guidance should encourage better and more comprehensive health systems research. This, in turn, should foster more cross-disciplinary approaches to studying the dynamic interactions within complex health systems and thus help to develop new methods to effectively translate health systems evidence into usable policy guidance that is relevant to stakeholders at global and national levels, that takes into account social and ethical imperatives, and that recognises the complexity of health systems and political systems within which such guidance is introduced.
Translation of the Summary Points into Spanish by Xavier Bosch-Capblanch
Translation of the Summary Points into French Bruno Clary, William Lenoir, and Lise Beck
Translation of the Summary Points into Portuguese by Bruno Viana
Translation of the Summary Points into Arabic by Fadi El-Jardali
Characteristics of guidance in health policy or strategy documents analysed with examples of quotations
We acknowledge other members of the Task Force on Developing Health Systems Guidance, who include (with their affiliations at the time when the Task Force was initiated): Abdul Ghaffar, Alliance for Health Policy and Systems Research, World Health Organization (Switzerland); Timothy Evans, Assistant Director-General, Information, Evidence and Research, World Health Organization (Switzerland); Davina Ghersi, Department of Research Policy and Cooperation, Information, Evidence and Research, World Health Organization (Switzerland); Metin Gulmezoglu, UNDP/UNFPA/WHO/World Bank Special Programme of Research, Development and Research Training in Human Reproduction, World Health Organization (Switzerland); Pierre Ongolo-Zogo, Centre for the Development of Best Practices in Health, Yaoundé Central Hospital, and University of Yaoundé, Yaoundé (Cameroon); Ulysses Panisset, Department of Research Policy and Cooperation, Information, Evidence and Research, World Health Organization (Switzerland); Gunn Vist, Norwegian Knowledge Centre (Norway).
Thanks to Don de Savigny for fruitful exchanges around health systems and guidance.
The Swiss Tropical and Public Health Institute and the Norwegian Knowledge Centre for the Health Services received funds from the WHO for the contributions of DD, LB, SL, and XBC to developing a Handbook to produce health systems guidance, and some of this work is reported in this article. All other authors declare no competing interests.
The development of the Handbook was supported by a grant to the WHO by the Rockefeller Foundation, which had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. The paper represents the views of the authors and neither WHO nor the Rockefeller Foundation.
Provenance: Not commissioned; externally peer reviewed.