Health-system policy makers make important decisions every day about the governance, financial, and delivery arrangements within which programs, services, and drugs are provided and about implementation strategies [1
]. The nature of their decisions will vary according to the setting in which they work (e.g.
, federal, provincial, or local government) and the role they play (e.g.
, political staff, policy analyst, senior policy advisor, Assistant Deputy Minister, or elected official), among other factors. Systematic reviews are increasingly seen as a key source of information to inform these decisions [1
]. Reduced bias and increased precision comprise the main advantages of systematic reviews that address questions about the effects of interventions [2
]. Drawing on a systematic review that addresses any
question constitutes a more efficient use of time for busy policy makers because the research literature has already been identified, selected, appraised, and synthesised in a systematic and transparent way. Additionally, a systematic review makes possible more constructive policy debates because stakeholders can focus on the synthesis and its local applicability rather than on which single study has greater credibility [3
In order to make informed decisions, health-system policy makers need timely access to systematic reviews that can be easily retrieved using terminology that is understandable to them and that are presented in ways that facilitate rapid scanning for relevance, recency of searches for potentially relevant studies, the settings of studies included in the review, and quality of the review [3
]. A systematic review of the factors that influence the use of research in policy making identified timing/timeliness as one of two factors that increased the prospects for research use among health-system policy makers [3
]. However, when attempting to retrieve systematic reviews in a timely fashion, health-system policy makers typically cannot search all of the potential sources of systematic reviews. Moreover, policy makers typically cannot search most sources of systematic reviews, like The Cochrane Library, using terms with which they are familiar. The number and searchability of existing sources of systematic reviews become particularly frustrating when policy makers know there is likely to be a review available on a topical issue. Moreover, search results typically do not highlight the types of decision-relevant information that health-system policy makers are seeking [3
One response to the similar types of issues faced by clinical decision makers has been the development of evidence services that provide regular email alerts about newly identified research products and a searchable database of these products[6
]. However, no 'full-serve' evidence service currently exists to meet the needs of health-system policy makers. Existing evidence services that include health-system policy makers among their target audiences, such as E-watch (http://kuuc.chair.ulaval.ca/english/index.php
) and CHAIN Canada (http://www.epoc.uottawa.ca/CHAINCanada/
), do not focus on systematic reviews. Existing evidence services that focus on high-quality studies (not just systematic reviews), such as Evidence Updates (http://plus.mcmaster.ca/EvidenceUpdates/
), do not target health-system policy makers[6
To address this gap, we developed a full-serve evidence service for health-system policy makers. First, we developed Health Systems Evidence, which contains over 1,400 syntheses about governance, financial, and delivery arrangements within health systems and about implementation strategies relevant to health systems. By syntheses we mean both systematic reviews and two types of review-derived products, namely, policy briefs and overviews of systematic reviews [7
]. A policy brief summarises how the findings from a number of systematic reviews pertain to a pressing problem, select options for addressing the problem, and key implementation considerations, whereas an overview provides a 'map' of all available systematic reviews on a broad health-system topic. The reviews have been (a) categorised by topic (i.e.
, by health-system arrangement or implementation strategy), type of review (i.e.
, policy brief, overview of reviews, Cochrane systematic review, systematic review, or systematic review protocol), and type of question addressed (i.e.
, effectiveness, not effectiveness, and 'many'); (b) coded by the last year in which searches for studies were conducted and by the countries in which included studies were conducted; (c) rated for quality using the AMSTAR (A MeaSurement Tool for the 'Assessment of multiple systematic Reviews') instrument [8
]; and (d) linked to available user-friendly summaries, scientific abstracts, and full-text reviews that are available free online [10
Second, we identified systematic reviews in Health Systems Evidence that are neither available free online nor available through subscriptions held by the Ontario Ministry of Health and Long-Term Care (MOHLTC) and developed a mechanism to reimburse publishers for full-text downloads of these reviews.
Third, we developed the format for monthly email alerts, which (in tabular format) identifies new additions to Health Systems Evidence and describes the type of review, type of question addressed, health-system arrangement or implementation strategy addressed, and title of the review. A hypertext link for each review enables policy makers to view the availability of (and links to) user-friendly summaries, scientific abstracts, and the full-text review. A hypertext link to the online Health Systems Evidence webpage enables policy makers to view additional information about these same recent database additions, including the last year searched, quality rating, the countries in which included studies were conducted, and the complete citation. (Electronic newsletter width restrictions precluded having all fields presented in the monthly email alerts.)
Our goal is to evaluate whether (and how and why) a full-serve evidence service increases the use of synthesised research evidence by policy analysts and advisors in the MOHLTC as compared to a 'self-serve' evidence service. The full-serve evidence service comprises database access (an effort to facilitate policy makers' efforts to 'pull' in research when they need it), monthly email alerts about new additions to the database (a 'push' effort), and full-text article availability (an additional effort to facilitate pull). A systematic review found that simply providing information (in the form of clinical-practice guidelines) can change clinical behaviour,[11
] which leaves us reasonably confident that we have the potential to achieve an increase in evidence use among health-system policy makers. Moreover, the results of a cluster randomised trial indicate that a full-serve evidence service increased practicing clinicians' utilisation of evidence-based information from a digital library [12