This article is part of a series written for people responsible for making decisions about health policies and programmes and for those who support these decision makers.
Policy briefs are a relatively new approach to packaging research evidence for policymakers. The first step in a policy brief is to prioritise a policy issue. Once an issue is prioritised, the focus then turns to mobilising the full range of research evidence relevant to the various features of the issue. Drawing on available systematic reviews makes the process of mobilising evidence feasible in a way that would not otherwise be possible if individual relevant studies had to be identified and synthesised for every feature of the issue under consideration. In this article, we suggest questions that can be used to guide those preparing and using policy briefs to support evidence-informed policymaking. These are: 1. Does the policy brief address a high-priority issue and describe the relevant context of the issue being addressed? 2. Does the policy brief describe the problem, costs and consequences of options to address the problem, and the key implementation considerations? 3. Does the policy brief employ systematic and transparent methods to identify, select, and assess synthesised research evidence? 4. Does the policy brief take quality, local applicability, and equity considerations into account when discussing the synthesised research evidence? 5. Does the policy brief employ a graded-entry format? 6. Was the policy brief reviewed for both scientific quality and system relevance?
There has been growing interest regarding the impact of telemonitoring and its ability to reduce the increasing burden of chronic diseases, including chronic cardiovascular disease (CVD), on healthcare systems. A number of randomised trials have been undertaken internationally and synthesised into various systematic reviews to establish an evidence base for this model of care. This study sought to synthesise and critically evaluate this large body of evidence to inform clinicians, researchers and policy makers.
A systematic review of systematic reviews investigating the impact of telemonitoring interventions in the primary care management of CVD was conducted. Reviews were included if they explored primary care based telemonitoring in either CVD, heart failure or hypertension, were reported in the English language and were published between 2000 and 2013. Data was extracted by one reviewer and checked by a second reviewer using a standardised form. Two assessors then rated the quality of each review using the Overview Quality Assessment Questionnaire (OQAQ).
Of the 13 included reviews, four focused on telemonitoring interventions in hypertension or CVD management and the remaining 9 reviews investigated telemonitoring in HF management. Seven reviews scored a five or above on the OQAQ evidencing good quality reviews. Findings suggest that telemonitoring can contribute to significant reductions in blood pressure, decreased all-cause and HF related hospitalisations, reduced all-cause mortality and improved quality of life. Telemonitoring was also demonstrated to reduce health care costs and appears acceptable to patients.
Telemonitoring has the potential to enhance primary care management of CVD by improving patient outcomes and reducing health costs. However, further research needs to explore the specific elements of telemonitoring interventions to determine the relative value of the various elements. Additionally, the ways in which telemonitoring care improves health outcomes needs to be further explored to understand the nature of these interventions.
Systematic review; Telemonitoring; Heart failure; Hypertension; Primary care
The Ontario health care system is devolving planning and funding authority to community based organizations and moving from steering through rules and regulations to steering on performance. As part of this transformation, the Ontario Ministry of Health and Long-Term Care (MOHLTC) are interested in using incentives as a strategy to ensure alignment – that is, health service providers' goals are in accord with the goals of the health system. The objective of the study was to develop a decision framework to assist policymakers in choosing and designing effective incentive systems.
The first part of the study was an extensive review of the literature to identify incentives models that are used in the various health care systems and their effectiveness. The second part was the development of policy principles to ensure that the used incentive models are congruent with the values of the Ontario health care system. The principles were developed by reviewing the Ontario policy documents and through discussions with policymakers. The validation of the principles and the suggested incentive models for use in Ontario took place at two meetings. The first meeting was with experts from the research and policy community, the second with senior policymakers from the MOHLTC. Based on the outcome of those two meetings, the researchers built a decision framework for incentives. The framework was send to the participants of both meetings and four additional experts for validation.
We identified several models that have proven, with a varying degree of evidence, to be effective in changing or enabling a health provider's performance. Overall, the literature suggests that there is no single best approach to create incentives yet and the ability of financial and non-financial incentives to achieve results depends on a number of contextual elements. After assessing the initial set of incentive models on their congruence with the four policy principles we defined nine incentive models to be appropriate for use in Ontario and potentially other health care systems that want to introduce incentives to improve performance. Subsequently, the models were incorporated in the resulting decision framework.
The design of an incentive must reflect the values and goals of the health care system, be well matched to the performance objectives and reflect a range of contextual factors that can influence the effectiveness of even well-designed incentives. As a consequence, a single policy recommendation around incentives is inappropriate. The decision framework provides health care policymakers and purchasers with a tool to support the selection of an incentive model that is the most appropriate to improve the targeted performance.
Objective. To determine graduate and postgraduate students’ perceptions of a drug use management and policy program that applied wide-ranging policy research skills to inform pharmaceutical decision-making.
Design. Nine cohorts of graduate and postgraduate students from diverse academic and professional backgrounds were paired with health-system preceptors for 4 months, and supported by faculty advisors and administrators, to complete research projects that generated evidence to inform policy decisions.
Assessment. A self-administered survey instrument was sent to all alumni of the program over the previous 10 years. The majority of respondents indicated: their prior academic coursework could be applied to everyday life; service-learning projects complemented university programs; participation led to greater awareness of decision-makers’ needs and appreciation of their tacit knowledge; and communication abilities were enhanced with decision-makers, and academics. Many also reported personal desire to fulfill healthcare-system research needs; personal belief in their ability to make a difference; and increased postgraduation marketability.
Conclusion. A drug use management and policy program allowed graduate students from various disciplines to develop new skills and collaborate with experts to produce research evidence that was relevant to drug policy that addressed real-world problems.
pharmaceutical policy; service-learning; assessment; engaged scholarship; residency; knowledge co-production
Despite evidence showing that adolescent-friendly health services (AFSs) increase young people's access to these services, health systems across the world are failing to integrate this approach. In Latin America, policies aimed at strengthening AFS abound. However, such services are offered only in a limited number of sites, and providers’ attitudes and respect for confidentiality have not been addressed to a sufficient extent.
The aim of this study was to explore the mechanisms that triggered the transformation of an ‘ordinary’ health care facility into an AFS in Ecuador. For this purpose, a realist evaluation approach was used in order to analyse three well-functioning AFSs. Information was gathered at the national level and from each of the settings including: (i) statistical information and unpublished reports; (ii) in-depth interviews and focus group discussions with policy makers, health care providers, users and adolescents participating in youth organisations and (iii) observations at the health care facilities. Thematic analysis was carried out, driven by the realist evaluation approach, namely exploring the connections between mechanisms, contexts and outcomes.
The results highlighted that the development of the AFSs was mediated by four mechanisms: grounded self-confidence in trying new things, legitimacy, a transformative process and an integral approach to adolescents. Along this process, contextual factors at the national and institutional levels were further explored.
The Ministry of Health of Ecuador, based on the New Guidelines for Comprehensive Care of Adolescent Health, has started the scaling up of AFSs. Our research points towards the need to recognise and incorporate these mechanisms as part of the implementation strategy from the very beginning of the process. Although contextually limited to Ecuador, many mechanisms and good practices in these AFS may be relevant to the Latin American setting and elsewhere.
adolescent health; realist evaluation; mechanism; qualitative research; health systems; sexual and reproductive health; gender; Latin America
Better communication is often suggested as fundamental to increasing the use of research evidence in policy, but little is known about how researchers and policy makers work together or about barriers to exchange. This study explored the views and practice of policy makers and researchers regarding the use of evidence in policy, including: (i) current use of research to inform policy; (ii) dissemination of and access to research findings for policy; (iii) communication and exchange between researchers and policy makers; and (iv) incentives for increasing the use of research in policy.
Separate but similar interview schedules were developed for policy makers and researchers. Senior policy makers from NSW Health and senior researchers from public health and health service research groups in NSW were invited to participate. Consenting participants were interviewed by an independent research company.
Thirty eight policy makers (79% response rate) and 41 researchers (82% response rate) completed interviews. Policy makers reported rarely using research to inform policy agendas or to evaluate the impact of policy; research was used more commonly to inform policy content. Most researchers reported that their research had informed local policy, mainly by increasing awareness of an issue. Policy makers reported difficulty in accessing useful research syntheses, and only a third of researchers reported developing targeted strategies to inform policy makers of their findings. Both policy makers and researchers wanted more exchange and saw this as important for increasing the use of research evidence in policy; however, both groups reported a high level of involvement by policy makers in research.
Policy makers and researchers recognise the potential of research to contribute to policy and are making significant attempts to integrate research into the policy process. These findings suggest four strategies to assist in increasing the use of research in policy: making research findings more accessible to policy makers; increasing opportunities for interaction between policy makers and researchers; addressing structural barriers such as research receptivity in policy agencies and a lack of incentives for academics to link with policy; and increasing the relevance of research to policy.
Evidence from several sub-Saharan countries support nurse-initiated antiretroviral treatment as a feasible alternative to doctor-led models characteristic of early responses to the HIV epidemic. However, service delivery models shown to be effective in one country may not be readily adopted in another. This study used an implementation research approach to assist policy makers and other stakeholders to assess the acceptability and feasibility of task shifting in the Namibian context.
The Namibian Ministry of Health and Social Services implemented a Task Shifting Demonstration Project (TSDP) at 9 sites at different levels of the health system. Six months after implementation, a mixed methods evaluation was conducted. Seventy semi-structured interviews were conducted with patients, managers, doctors and nurses directly involved with the TSDP. Physician-evaluators observed and compared health service provision between doctors and nurses for 40 patients (80 observations), documenting performance in agreement with the national guidelines on 13 clinical care indicators.
Doctors, nurses, and patients interviewed believed task shifting would improve access to and quality of HIV services. Doctors and nurses both reported an increase in nurses’ skills as a result of the project. Observation data showed doctors and nurses were in considerable agreement (>80%) with each other on all dimensions of HIV care and ≥90% on eight dimensions. To ensure success of national scale-up of the task shifting model, challenges involving infrastructure, on-going mentoring, and nursing scope of practice should be anticipated and addressed.
In combination with findings from other studies in the region, data from the TSDP provided critical and timely information to the Namibian Ministry of Health and Social Services, thus helping to move evidence into action. Small-scale implementation research projects enable stakeholders to learn by doing, and provide an opportunity to test and modify the intervention before expansion.
Glaucoma is a leading cause of avoidable blindness worldwide. Open angle glaucoma is the most common type of glaucoma. No randomised controlled trials have been conducted evaluating the effectiveness of glaucoma screening for reducing sight loss. It is unclear what the most appropriate intervention to be evaluated in any glaucoma screening trial would be. The purpose of this study was to develop the clinical components of an intervention for evaluation in a glaucoma (open angle) screening trial that would be feasible and acceptable in a UK eye-care service.
A mixed-methods study, based on the Medical Research Council (MRC) framework for complex interventions, integrating qualitative (semi-structured interviews with 46 UK eye-care providers, policy makers and health service commissioners), and quantitative (economic modelling) methods. Interview data were synthesised and used to revise the screening interventions compared within an existing economic model.
The qualitative data indicated broad based support for a glaucoma screening trial to take place in primary care, using ophthalmic trained technical assistants supported by optometry input. The precise location should be tailored to local circumstances. There was variability in opinion around the choice of screening test and target population. Integrating the interview findings with cost-effectiveness criteria reduced 189 potential components to a two test intervention including either optic nerve photography or screening mode perimetry (a measure of visual field sensitivity) with or without tonometry (a measure of intraocular pressure). It would be more cost-effective, and thus acceptable in a policy context, to target screening for open angle glaucoma to those at highest risk but for both practicality and equity arguments the optimal strategy was screening a general population cohort beginning at age forty.
Interventions for screening for open angle glaucoma that would be feasible from a service delivery perspective were identified. Integration within an economic modelling framework explicitly highlighted the trade-off between cost-effectiveness, feasibility and equity. This study exemplifies the MRC recommendation to integrate qualitative and quantitative methods in developing complex interventions. The next step in the development pathway should encompass the views of service users.
Evidence-informed health policymaking logically depends on timely access to research evidence. To our knowledge, despite the substantial political and societal pressure to enhance the use of the best available research evidence in public health policy and program decision making, there is no study addressing availability of peer-reviewed research in Canadian health ministries.
To assess availability of (1) a purposive sample of high-ranking scientific journals, (2) bibliographic databases, and (3) health library services in the fourteen Canadian health ministries.
From May to October 2011, we conducted a cross-sectional survey among librarians employed by Canadian health ministries to collect information relative to availability of scientific journals, bibliographic databases, and health library services. Availability of scientific journals in each ministry was determined using a sample of 48 journals selected from the 2009 Journal Citation Reports (Sciences and Social Sciences Editions). Selection criteria were: relevance for health policy based on scope note information about subject categories and journal popularity based on impact factors.
We found that the majority of Canadian health ministries did not have subscription access to key journals and relied heavily on interlibrary loans. Overall, based on a sample of high-ranking scientific journals, availability of journals through interlibrary loans, online and print-only subscriptions was estimated at 63%, 28% and 3%, respectively. Health Canada had a 2.3-fold higher number of journal subscriptions than that of the provincial ministries’ average. Most of the organisations provided access to numerous discipline-specific and multidisciplinary databases. Many organisations provided access to the library resources described through library partnerships or consortia. No professionally led health library environment was found in four out of fourteen Canadian health ministries (i.e. Manitoba Health, Northwest Territories Department of Health and Social Services, Nunavut Department of Health and Social Services and Yukon Department of Health and Social Services).
There is inequity in availability of peer-reviewed research in the fourteen Canadian health ministries. This inequity could present a problem, as each province and territory is responsible for formulating and implementing evidence-informed health policies and services for the benefit of its population.
Health care; Information science; Library science; Knowledge transfer; Research evidence
To support the use of research evidence by community-based organizations (CBOs) we have developed 'Synthesized HIV/AIDS Research Evidence' (SHARE), which is an evidence service for those working in the HIV sector. SHARE consists of several components: an online searchable database of HIV-relevant systematic reviews (retrievable based on a taxonomy of topics related to HIV/AIDS and open text search); periodic email updates; access to user-friendly summaries; and peer relevance assessments. Our objective is to evaluate whether this 'full serve' evidence service increases the use of research evidence by CBOs as compared to a 'self-serve' evidence service.
We will conduct a two-arm randomized controlled trial (RCT), along with a follow-up qualitative process study to explore the findings in greater depth. All CBOs affiliated with Canadian AIDS Society (n = 120) will be invited to participate and will be randomized to receive either the 'full-serve' version of SHARE or the 'self-serve' version (a listing of relevant systematic reviews with links to records on PubMed and worksheets that help CBOs find and use research evidence) using a simple randomized design. All management and staff from each organization will be provided access to the version of SHARE that their organization is allocated to. The trial duration will be 10 months (two-month baseline period, six-month intervention period, and two month crossover period), the primary outcome measure will be the mean number of logins/month/organization (averaged across the number of users from each organization) between baseline and the end of the intervention period. The secondary outcome will be intention to use research evidence as measured by a survey administered to one key decision maker from each organization. For the qualitative study, one key organizational decision maker from 15 organizations in each trial arm (n = 30) will be purposively sampled. One-on-one semi-structured interviews will be conducted by telephone on their views about and their experiences with the evidence service they received, how helpful it was in their work, why it was helpful (or not helpful), what aspects were most and least helpful and why, and recommendations for next steps.
To our knowledge, this will be the first RCT to evaluate the effects of an evidence service specifically designed to support CBOs in finding and using research evidence.
The Ministry of Health (MOH) launched the National Reproductive Health Policy in 2005, which included recommendations regarding the use of emergency contraceptive pills (ECP). However, ECP have not yet been introduced officially in the public sector of the Lao PDR. Thus, their availability is limited. Understanding the knowledge of ECP and attitudes about their provision, barriers to use, and availability among health providers and policy makers is essential to successfully incorporate ECP into reproductive health services.
Qualitative research methods using in-depth interviews were employed to collect data from policy makers and health providers (auxiliary medical staff, nurses, and medical doctors). Altogether, 10 policy makers, 22 public providers, and 10 providers at private clinics were interviewed. Content analysis was applied to analyze the transcribed data.
The majority of policy makers and health care providers had heard about ECP and supported their introduction in the public sector. However, their knowledge was poor, many expressed inconsistent attitudes, and their ability to meet the demand of potential users is limited.
There is a need to train health providers and policy makers on emergency contraception and improve their knowledge about ECP, especially regarding the correct timing of use and the availability of methods. In addition, the general public must be informed of the attributes, side effects, and availability of ECP, and policy makers must facilitate the approval of ECP by the Lao Food and Drug Administration. These interventions could lead to increased access to and demand for ECP.
In 2007, the Ontario Ministry of Health and Long-Term Care made an investment to support full-time employment for new graduate nurses. This paper describes the collaboration of policy makers and researchers in the creation and implementation of the Nursing Graduate Guarantee (NGG). We provide historical context for the development of the initiative and discuss some of the issues related to its implementation. Relevant stakeholders assisted researchers and policy makers in the creation, implementation and evaluation of the NGG. Researchers continue to work with policy makers in ongoing evaluations of the multi-year strategy, which are informed by stakeholder input.
There has been a strong push over the last decade for health services researchers to become “relevant,” to work with policy makers to translate evidence into action. What has been learned from this interaction? The pooled experiences of health services researchers across the country, including those at the Manitoba Centre for Health Policy (MCHP), suggest five key lessons. First, policy makers pay more attention to research findings if they have invested their own funds and time. Second, researchers must make major investments in building relationships with policy makers, because there are inevitable tensions between what the two parties need and do. Third, researchers must be able to figure out and communicate the real meaning of their results. Fourth, health services researchers need a “back-pocket” mindset, as they cannot count on immediate uptake of results; because the issues never go away, evidence, if known and easily retrievable, is likely to have an eventual impact. Finally, getting evidence into the policy process does not come cheaply or easily, but it can be done. The overriding lesson learned by health services researchers is the importance of relationship-building, whether in formalizing contractual relationships, building and maintaining personal trust, having a communications strategy or increasing the involvement of users in the research process.
Determining the effectiveness of social and psychological interventions is important for improving individual and population health. Such interventions are complex and, where possible, are best evaluated by randomised controlled trials (RCTs). The use of research findings in policy and practice decision making is hindered by poor reporting of RCTs. Poor reporting limits the ability to replicate interventions, synthesise evidence in systematic reviews, and utilise findings for evidence-based policy and practice. The lack of guidance for reporting the specific methodological features of complex intervention RCTs contributes to poor reporting. We aim to develop an extension of the Consolidated Standards of Reporting Trials Statement for Social and Psychological Interventions (CONSORT-SPI).
This research project will be conducted in five phases. The first phase was the project launch, which consisted of the establishment of a Project Executive and International Advisory Group, and recruitment of journal editors and the CONSORT Group. The second phase involves a Delphi process that will generate a list of possible items to include in the CONSORT Extension. Next, there will be a formal consensus meeting to select the reporting items to add to, or modify for, the CONSORT-SPI Extension. Fourth, guideline documents will be written, including an explanation and elaboration (E&E) document that will provide detailed advice for each item and examples of good reporting. The final phase will comprise guideline dissemination, with simultaneous publication and endorsement of the guideline in multiple journals, endorsement by funding agencies, presentations at conferences and other meetings, and a dedicated website that will facilitate feedback about the guideline.
As demonstrated by previous CONSORT guidelines, the development of an evidence-based reporting guideline for social and psychological intervention RCTs should improve the accuracy, comprehensiveness, and transparency of study reports. This, in turn, promises to improve the critical appraisal of research and its use in policy and practice decision making. We invite readers to participate in the project by visiting our website (http://tinyurl.com/CONSORT-study).
CONSORT-SPI; Randomised controlled trial; RCT; Reporting guidelines; Complex interventions
Health policy and systems research (HPSR), which aims to produce reliable and rigorous evidence to inform the many critical decisions that must be made about health systems, is a new concept in Nigeria. In this study, policy makers and other stakeholders in the health sector identified the challenges and the potential intervention strategies to HPSR evidence use in policy making in Nigeria. The major challenges identified included capacity constraints at individual and organizational levels, communication gaps and poor networking between policy makers and researchers, and the non-involvement of healthcare recipients in identifying and planning care delivery needs. The main solutions suggested included promotion of strategies to encourage partnership between researchers and policy makers, improvement of staff incentives and facilities for research activities, improved budgetary provision for research, and sustainable institutional capacity development. These strategies have been shown to improve evidence-based policy making in developed countries and are likely to produce better outcomes in the developing world.
Uncontrolled proliferation of health technologies (HT) is one contributor to the increasing pressure on health systems to adopt new technologies. With limited resources, policy-makers encounter difficulties in fulfilling their responsibility to meet the healthcare needs of the population. The aim of this study is to explore how policy-makers' reason about the diffusion and utilization of health technologies in Iran using magnetic resonance imaging (MRI) and interferon beta as tracers.
This qualitative exploration complements quantitative data generated in a research project investigating the diffusion and utilization of MRI and interferon beta in Iran. Qualitative semi-structured interviews were conducted with 13 informants in different positions and levels of authority in the Ministry of Health (MOH), University of Medical Sciences, Health Insurance Organizations, and Parliament. The data was analysed using the framework approach.
Although policy-makers appeared to be positive to health technology assessment (HTA), the processes of policy-making described by the interviewees did not seem to be based on a full understanding of this (discipline). Several obstacles to applying knowledge about HT and HTA were described. The current official plan for MRI adoption and diffusion in the country was said not to be followed, and no such plan was described for interferon beta. Instead, market forces such as advertising, and physician and consumer demand, appear to have strong influence on HT diffusion and use. Dual practice may have increased the induced demand and also reduced the supervision of the private sector by the MOH.
Management instability and lack of coordination in the MOH were found to be important obstacles to accumulation of knowledge and experience which, in turn, could have led to suboptimal managerial and policy-making processes. Furthermore marketing should be controlled in order to avoid creating unnecessary patient demands and negative influences on physicians' behavior.
This article describes recent events in the governance of standard-setting for 2 areas of US health policy — states' decisions about which prescription drugs to cover under Medicaid and other public programs and making health an aspect of foreign policy — and whether these events offer lessons for policy making. In prescription drug coverage, methodologic advances in research that evaluates health services and the politics of restraining the rate of growth in health expenditures enabled policy makers in most states to establish new public processes for assessing and applying evidence about the effectiveness of competing drugs. Their counterparts in foreign policy, in contrast, made few changes in existing processes for choosing which interventions to support. The history of governance in each area of policy making for health explains the selection of standards to evaluate evidence about interventions and whether and how to use this evidence to guide policy.
Despite the fact that HIV epidemic is mainly driven by injection drug use in Iran, partners of People Who Inject Drugs (PWID) have been seriously neglected in terms of effective preventive interventions. Currently, sexual partners of PWID might have access to some harm reduction services at Voluntary Counselling and Testing (VCT) centers; however, their needs have not been effectively targeted and met. Unfortunately, the current programs implemented by the Ministry of Health have overlooked the importance of this population in the course of the HIV epidemic throughout the country. In this policy brief, we are trying to draw the health policy-makers’ attention to this overlooked population and while reviewing the advantages and disadvantages of some of the readily available options on the table, come up with a recommended action to tackle this problem. Our recommended action that seems to have had promising results elsewhere in Asia would try to implement preventive interventions targeting this particular population through peer prevention programs.
Sexual Partners; People Who Inject Drugs (PWID); HIV; Peer Prevention; Iran
Objective: To explore with UK and international policy advisors how research evidence influences public health policy making, and how its relevance and utility could be improved, with specific reference to the evidence on the production and reduction of health inequalities.
Design, setting, and participants: Qualitative residential workshop involving senior policy advisors with a substantive role in policy development across a range of sectors (mainly public health, but also including education, social welfare, and health services). In four in depth sessions, facilitated by the authors, focused questions were presented to participants. Their responses were then analysed thematically to identify key themes, relating to the availability and utility of existing evidence on health inequalities.
Main results: The lack of an equity dimension in much aetiological and evaluative research was highlighted by participants. Much public health research was also felt to have weak underlying theoretical underpinnings. As well as evaluations of the effectiveness and cost-effectiveness of policy and other interventions, they identified a need for predictive research, and for methodological research to further develop methods for assessing the impact on health of clusters of interventions.
Conclusions: This study reinforces the view that there is a lack of information on the effectiveness and cost-effectiveness of policies, and it uncovered additional gaps in the health inequalities evidence base. A companion paper discusses researchers' views of how the production of more relevant public health evidence can be stimulated.
Participants in the Cochrane Collaboration conduct and periodically update systematic reviews that address the question, “What works?” for healthcare interventions. The Cochrane Library makes available quality-appraised systematic reviews that address this question. No coordinated effort has been undertaken to conduct and periodically update systematic reviews that address the other types of questions asked by healthcare managers and policy makers, to adapt existing reviews to highlight decision-relevant information (including the factors that may affect assessments of a review’s local applicability) or to facilitate their retrieval through a “one-stop shopping” portal. Researchers interested in evaluating new methodological developments, health services and policy researchers interested in conducting and adapting systematic reviews, and research funders all have a role to play in making systematic reviews more useful for healthcare managers and policy makers.
Most Ministries of Health across Africa invest substantial resources in some form of health management information system (HMIS) to coordinate the routine acquisition and compilation of monthly treatment and attendance records from health facilities nationwide. Despite the expense of these systems, poor data coverage means they are rarely, if ever, used to generate reliable evidence for decision makers. One critical weakness across Africa is the current lack of capacity to effectively monitor patterns of service use through time so that the impacts of changes in policy or service delivery can be evaluated. Here, we present a new approach that, for the first time, allows national changes in health service use during a time of major health policy change to be tracked reliably using imperfect data from a national HMIS.
Monthly attendance records were obtained from the Kenyan HMIS for 1 271 government-run and 402 faith-based outpatient facilities nationwide between 1996 and 2004. A space-time geostatistical model was used to compensate for the large proportion of missing records caused by non-reporting health facilities, allowing robust estimation of monthly and annual use of services by outpatients during this period.
We were able to reconstruct robust time series of mean levels of outpatient utilisation of health facilities at the national level and for all six major provinces in Kenya. These plots revealed reliably for the first time a period of steady nationwide decline in the use of health facilities in Kenya between 1996 and 2002, followed by a dramatic increase from 2003. This pattern was consistent across different causes of attendance and was observed independently in each province.
The methodological approach presented can compensate for missing records in health information systems to provide robust estimates of national patterns of outpatient service use. This represents the first such use of HMIS data and contributes to the resurrection of these hugely expensive but underused systems as national monitoring tools. Applying this approach to Kenya has yielded output with immediate potential to enhance the capacity of decision makers in monitoring nationwide patterns of service use and assessing the impact of changes in health policy and service delivery.
This study examined the experience of the Ontario Ministry of Health and Long-Term Care in enhancing its stewardship and performance management role by developing a health system strategy map and a strategy-based scorecard through a process of policy reviews and expert consultations, and linking them to accountability agreements. An evaluation of the implementation and of the effects of the policy intervention has been carried out through direct policy observation over three years, document analysis, interviews with decision-makers and systematic discussion of findings with other authors and external reviewers. Cascading strategies at health and local health system levels were identified, and a core set of health system and local health system performance indicators was selected and incorporated into accountability agreements with the Local Health Integration Networks. despite the persistence of such challenges as measurement limitations and lack of systematic linkage to decision-making processes, these activities helped to strengthen substantially the ministry's performance management function.
The Rehabilitation Research and Training Center on Measuring Rehabilitation Outcomes and Effectiveness, along with academic, professional, provider, accreditor, and other organizations, sponsored a 2-day State-of-the-Science of Postacute Rehabilitation Symposium in February 2007. The aim of this symposium was to serve as a catalyst for expanded research on postacute care (PAC) rehabilitation so that health policy is founded on a solid evidence base. The goals were to (a) describe the state of our knowledge regarding utilization, organization, and outcomes of postacute rehabilitation settings, (b) identify methodologic and measurement challenges to conducting research, (c) foster the exchange of ideas among researchers, policy makers, industry representatives, funding agency staff, consumers, and advocacy groups, and (d) identify critical questions related to setting, delivery, payment, and effectiveness of rehabilitation services. Plenary presentation and state-of-the-science summaries were organized around 4 themes: (a) the need for improved measurement of key rehabilitation variables and methods to collect and analyze this information, (b) factors that influence access to postacute rehabilitation care, (c) similarities and differences in quality and quantity of services across PAC settings, and (d) effectiveness of postacute rehabilitation services. The full set of symposium articles, including recommendations for future research, appear in Archives of Physical Medicine and Rehabilitation.
Health policy; Outcome and process assessment (health care); Outcomes research; Rehabilitation
This article is part of a series written for people responsible for making decisions about health policies and programmes and for those who support these decision makers.
Differences between health systems may often result in a policy or programme option that is used in one setting not being feasible or acceptable in another. Or these differences may result in an option not working in the same way in another setting, or even achieving different impacts in another setting. A key challenge that policymakers and those supporting them must face is therefore the need to understand whether research evidence about an option can be applied to their setting. Systematic reviews make this task easier by summarising the evidence from studies conducted in a variety of different settings. Many systematic reviews, however, do not provide adequate descriptions of the features of the actual settings in which the original studies were conducted. In this article, we suggest questions to guide those assessing the applicability of the findings of a systematic review to a specific setting. These are: 1. Were the studies included in a systematic review conducted in the same setting or were the findings consistent across settings or time periods? 2. Are there important differences in on-the-ground realities and constraints that might substantially alter the feasibility and acceptability of an option? 3. Are there important differences in health system arrangements that may mean an option could not work in the same way? 4. Are there important differences in the baseline conditions that might yield different absolute effects even if the relative effectiveness was the same? 5. What insights can be drawn about options, implementation, and monitoring and evaluation? Even if there are reasonable grounds for concluding that the impacts of an option might differ in a specific setting, insights can almost always be drawn from a systematic review about possible options, as well as approaches to the implementation of options and to monitoring and evaluation.
There are calls for low and middle income countries to develop robust health financing policies to increase service coverage. However, existing evidence around financing options is complex and often difficult for policy makers to access.
To summarize the evidence on the impact of financing health systems and develop an e-tool to help decision makers navigate the findings.
After reviewing the literature, we used thematic analysis to summarize the impact of 7 common health financing mechanisms on 5 common health system goals. Information on the relevance of each study to a user’s context was provided by 11 country indicators. A Web-based e-tool was then developed to assist users in navigating the literature review. This tool was evaluated using feedback from early users, collected using an online survey and in-depth interviews with key informants.
The e-tool provides graphical summaries that allow a user to assess the following parameters with a single snapshot: the number of relevant studies available in the literature, the heterogeneity of evidence, where key evidence is lacking, and how closely the evidence matches their own context. Users particularly liked the visual display and found navigating the tool intuitive. However there was concern that a lack of evidence on positive impact might be construed as evidence against a financing option and that the tool might over-simplify the available financing options.
Complex evidence can be made more easily accessible and potentially more understandable using basic Web-based technology and innovative graphical representations that match findings to the users’ goals and context.
health care systems; financing; policy makers; software tools