|Home | About | Journals | Submit | Contact Us | Français|
To apply social science theory so as to define more explicitly the pathways that influence policy makers' use of health services research.
The analysis builds on a literature review and the author's observations. It identifies important social science concepts relevant to use of research in policy and organizational decision making. It integrates and expands upon existing frameworks to differentiate and analyze 10 pathways that can lead to the use of health services research by policy makers.
The process through which research is applied involves many factors, only some of which are amenable to influence by researchers. Within these constraints, multiple pathways can drive research use; no one of these is likely to perform better in all circumstances. Successful uptake is more likely when these pathways cause findings to be converted into messages meaningful to policy makers. Various intermediaries play an important role in creating effective pathways, while users also can influence them.
The pathways open up what too often is an unexplored “black box” that mediates between health services research and its use by policy makers. Such pathways can help stakeholders to bridge different perspectives in ways that strengthen the possibility that effective research will be supported and used.
Clinical research seeks to identify effective treatments that clinicians can use for individuals, while health services research focuses on the performance of the entire system. Health services research has value in itself, but often its relevance rests in its application (Buxton and Hanney 1996; Gray 2003;).1 Many studies examine how economic, organizational, policy, and market variables influence the way patients and clinicians interact to shape outcomes—such as cost, access, and quality—at the societal or systems levels. The complexity of issues addressed by health services research, and the role of knowledge, values, and ideology in decision making, can challenge and limit use of such research (Hanney et al. 2003), yet the key link between research findings and their applications often goes unexamined.
In this paper, we seek to focus more explicitly on the potential pathways that link research to policy makers' decision-making applications. We review relevant concepts from research on policy formation and organizational behavior, factors influencing research use, and knowledge transfer and exchange strategies. With this base, we provide a framework to highlight processes mediating research and its use and with it identify and discuss 10 diverse pathways that may be relevant in different circumstances. We end by discussing the implications for researchers, users, and those bridging both worlds who want to enhance the use of health services research. The assumption is that researchers and other stakeholders can play a more constructive role if they better understand the links between research and its application.
Interest in whether research is used in policy making—and how to promote such use—is not new; it has been a concern in social policy research for almost as long as such research has been funded (Davis and Salasin 1975; Weiss 1979;). Researchers who wish their work to be relevant can benefit from past efforts to address these concerns, some of which may challenge deeply held assumptions. When researchers still are surprised that rational frameworks of “knowledge transfer and exchange” do not always mesh with the underlying politics of health policy making (Mitton et al. 2007) something is amiss in how social science findings are conveyed to researchers in various disciplines.
For years, policy incrementalism, and the seeming randomness or ambiguities with which some problems and their potential solutions are treated in decision making, have been issues in political science and organizational theory (Lindblom 1959; Kingdon 1984; Cohen, March, and Olsen 1972;). For example, stakeholder-preferred policy solutions may be linked to particular events or problems of current relevance (“windows of opportunity”) to take advantage of opportunities, even when the solutions may not be particularly applicable to those problems. Research findings may be used to define problems that can be solved by or support preferred solutions, regardless of their policy relevance. Research with insights not currently relevant, or in conflict with preferred solutions, may be ignored. Ongoing research may be used as ammunition to promote the status quo by delaying action or deflecting criticism (Weiss 1979).
Because research is not valuable at a particular time, however, does not mean that it will be valueless in the future. Such research can be drawn upon if circumstances or needs change (Cohen, March, and Olsen 1972). Whitehead et al. (2004) term this the “sleeper” effect, reflecting what Hanney et al. (2003) describe as the “research reservoir.”
Typically, there is no single decision maker or audience for research findings; rather, there are many players in a political or organizational process. Each may have different goals and values, and some at least will have predetermined positions, which research is used to support (Allison and Zelikow 1999). This use can sometimes make researchers uncomfortable (Spitz and Abramson 2005), but it may be legitimate, especially if the findings are presented appropriately (Weiss 1979).
The nature of the political process raises issues for researchers seeking to link their work to policy formation. For example, in examining U.S. policy on equity, Gamble and Stone (2006) argue for rejecting the classical model of research use, which assumes that disparities in health care are addressed only if they are first documented as research findings and then disseminated. Instead, the authors argue, health disparities are addressed only if they are converted into political issues that are then defined as something government can address with the tools it has available. Viewed in this way, effective research must frame the problems it reveals as “bad situations and moral wrongs that government can and should fix.” Regardless of whether one agrees with researchers framing the policy program, this function may be essential to the incorporation of research into policy making. The real issue is who should do the framing.
Health services research can support the policy process by generating many kinds of knowledge, including the following: evidence that a problem exists, examples of the impacts of policies on people and organizations, controlled evaluations of policy initiatives, feedback from natural experiments with variation, and historical evidence (Whitehead et al. 2004). Barriers to using this evidence also exist, however, because of the inherent ambiguity of issues and the policy process, institutional constraints that lead to fragmentation of focus, and competing interests (Waddell et al. 2005). Further, there may be persistent entrenched ideas (e.g., managed care is inherently bad because of its constraints on professional behavior) and conflicts between competing philosophies (Waddell et al. 2005). Also, social networks influence the communication of ideas, and both the diffusion and eventual acceptance of research findings and other forms of innovation take place in stages (Gray 1973; Rogers 1983;).
A recent review of research on factors that influence take-up of new ideas (“knowledge”) in health services organizations provides numerous insights into ways that research use can be enhanced or diminished (Greenlaugh et al. 2004; see Table 1).2 For example, findings are more likely to be acted upon if they generate ideas that are unambiguous and easy to apply. Action itself is less likely to be an event than an evolving process of decision making and implementation in which research can play a continuing role. In organizations, take-up of ideas often occurs through informal processes. Some organizations may be more “permeable” to outside knowledge because of their institutional philosophies, the positioning of influential staff members, their financial strength, or the “organizational slack” necessary to identify new knowledge. Although not always the sources of the research, individuals who share common attributes with those they seek to reach may be more effective translators.
The literature reinforces the importance of viewing the translation of research into policy use as a complex, time-consuming process involving the convergence of a number of factors. Researchers can both control parts of the process and influence others by how they conduct and disseminate their research. They also are likely to be more effective when the system infrastructure supports the dissemination process. Such an infrastructure is particularly useful in facilitating the informal transfer of information through the networks it creates and the support it provides for tailoring the research message to needs of policy makers.
Yet a supporting infrastructure, while important, is not sufficient. Some findings will reach a more receptive audience than others, regardless of the supporting infrastructure. Some users and their organizations will be more receptive to information than others. When the demands of a user's environment converge with research findings, uptake may be more likely, so it is valuable to examine not only what is used, but why or why not.
While politics and organizational processes may promote or inhibit the use of research in policy, there are techniques to build bridges between researchers and potential users (Mitton et al. 2007). Lavis et al. (2003; see Table 2) are valuable in identifying five relevant questions:
Lavis' analysis suggests that research-transfer strategies are effective only if they incorporate:
The successful pursuit of transfer strategies requires a substantial investment in understanding the audience and its needs, building credibility, and continuing the exchange of ideas (Lavis et al. 2002), but there are substantial barriers to achieving this goal.
AcademyHealth (2006) assessed users' perspectives on the need for health services research; their work highlights perceived defects in researchers' training as skilled communicators and leaders. Intermediaries may be valuable in addressing this shortcoming, including the following:(1) credible “policy entrepreneurs,” who can translate research into state policy implications in relevant and well-received ways (Coburn 1998); (2) “knowledge brokers,” who can translate research into support for actions, leading—for example—to better care for children (Simpson 2004); and (3) “consultants” funded by users to identify and translate relevant knowledge for specific policy or management questions (Jacobson et al. 2005). Mitton et al. (2007) find that personal contacts and trust established through long-term quality relationships are essential to exchange of information. Hence, effective and credible links between research and its users are important.
To help researchers think more concretely about how translation activities influence whether research findings are used, we have drawn on the literature and our own experience to develop a framework showing the factors that mediate the translation of research findings into messages communicated to and used by policy makers (Figure 1). By “used,” we mean that, at a minimum, policy makers or those seeking to influence them are aware of ideas generated through the process of research, and that those ideas have some influence, either on the debate over policy decisions or on the actual decisions.
Within the framework, we identify nodes that distinguish 10 pathways through which research may get applied in policy making, emphasizing different ways in which messages get framed and communicated, who is involved, and what drives or constrains movement along the pathway. The framework in which the pathways are nested assumes they all operate in a political and professional environment, and that there is a reservoir of knowledge to which researchers contribute and policy makers can use (Hanney et al. 2003). The reservoir's richness and contents are influenced over time by the research funded and its topics. Only research that is completed, documented, and made public contributes to the reservoir; proprietary restrictions, release policies, and publication lags all limit flow into it (AcademyHealth 2004, 2007). Richer, better quality findings enhance potential contributions (Soumerai et al. 1993), so funding streams and their allocation are critical.
Some pathways reflect traditional emphasis on research as “knowledge development” and “enlightenment.” These generally assume that meritorious research findings will find an appropriate audience without much emphasis on the mediating process. Other pathways emphasize the role of intermediary, as well as how users more actively influence the way research is conducted and used. The goal is not to rank pathways, but to create a better understanding of each, and their possible roles in enhancing the use of research.
Traditionally, researchers aim for peer-reviewed publication of their work in journals or other vehicles. Such findings influence policy use directly through at least three pathways.
Some studies produce blockbuster findings that immediately inform or change the policy debate and provide value over time (RAND Health 2006). For example, the National Medical Expenditure Survey's estimates on the uninsured influenced congressional consideration of legislation to address the effects of unemployment on health coverage (Wilensky 1987). McGlynn et al.'s (2003) work on the extent of inappropriate care has been used to enhance policy support for quality improvement. Publication in a prestigious journal can generate press coverage that propels the communication and uptake of key messages from research. This pathway also may be used less visibly; an example is the work Ashe and her colleagues conducted on risk-based predictive modeling, which was honored with AcademyHealth's (2008) HSR Impact Award (AcademyHealth 2008).
“Big bang” research carries a great deal of weight and professional satisfaction. Sometimes, however, important findings may not match current policy interest. Also, publication in prestigious journals may delay the use of important research results because of restrictions on prior public disclosure. Realistically, most research will not have sufficient saliency for a “big bang.”
Researchers may strive for a “big bang,” but it is more common for research to contribute to a knowledge reservoir that accumulates over time and is available for policy use. In some cases, this accumulation of findings is converted into “common knowledge” through what has been termed the “enlightenment/percolation/limestone model” (Buxton and Hanney 1996). A good example involves the gradual build-up of understanding that health insurance coverage influences access to care and outcomes. The absence of formal processes to synthesize and identify “messages” from the pool of research means, however, that there is no quality control (Weiss 1979). (Formal synthesis could enhance “scientific vetting,” as in the Institute of Medicine's (IOM) (2001) synthesis of generally accepted findings on the impact of lack of insurance, which added to the credibility of the research.) Use of accumulated knowledge also is impeded when research is difficult to retrieve because findings are not published in archival sources or indexed in appropriate databases; web-based search engines may mitigate this. Older research also may be forgotten or viewed as dated, which can occur when support for specific research topics waxes and wanes, as with interest in the limit between Medicaid payment rates and physician access (Gold et al. 2006).
Researchers can build on existing knowledge through formal synthesis of existing evidence in a particular area, such as knowledge of performance by health maintenance organizations (Luft 1981), or from two decades of health care research on organizational change (Bazzoli et al. 2004). Although exceptions exist,3 support for such synthesis tends to be limited in health services research, with funders preferring to support new work. Researcher-initiated syntheses run the risk of addressing questions of interest to other researchers, not policy makers, with exceptions (e.g., Soumerai et al. 1993). Syntheses pointing out where more research is needed can be important, but those drawing concrete conclusions, even if constrained, are likely to be valued more by policy makers.
A key barrier to the use of research is the potential users' lack of awareness of a study or body of work and why it may be relevant. Intermediaries or processes can support better connections between the policy needs of users and findings from researchers.
Through this pathway, researchers take steps to make it more likely that policy makers will know about and make use of research. Simple steps can enhance relevance, such as thinking carefully about the conclusions drawn from a study's findings and developing a clear message. Embedding the message in press advisories and other concise documents shared with potential intermediaries and users make it more likely that messages will reach key users. Such actions also allow researchers to help shape the “take away.” Some refer to this as satisfying the “elevator test,” using messages communicated between floors.
Such actions make policy makers aware of researchers' expertise. Ongoing relationships might develop, particularly when researchers actively cultivate them by being responsive or promote them through actions such as writing in mass media vehicles or speaking and spending time at user events. For example, in 2004, Uwe Reinhardt and Stuart Altman received AcademyHealth's Distinguished Investigator Award as recognized leaders in this form of translation (AcademyHealth 2008). This pathway draws on knowledge of a reservoir of research and knowledge reaching beyond particular studies. Effective performance as an active and credible research translator takes skills and judgment, however, particularly when policy questions do not neatly match a given body of research. Researchers often are not trained in such skills, and the same interests drawing them to research may limit their involvement in communication endeavors.
In some instances, formal organizations exist to bridge the gap between the different needs and orientations of the research and policy worlds. At the federal level, examples are the Congressional Budget Office (CBO), Congressional Research Service, Government Accountability Office (GAO), and such specialized entities as the Medicare Payment Advisory Commission. Some, such as GAO, emphasize original research in direct response to congressional requests. Others build on the existing reservoir of research. For example, CBO is required to provide a fiscal estimate of the costs of each piece of legislation; it also analyzes potential legislative options and provides more general guidance that often determines which legislative policy options are feasible. With a staff dominated by economists, CBO draws heavily from existing research to execute its mission. Staff evaluate and synthesize relevant research findings and use research to support the development of cost models. Some states also have established independent policy centers, sometimes affiliated with universities or foundations, to carry out similar functions.
To policy makers, such intermediaries have advantages, notably their responsibility to provide rigorous analysis consistent with policy needs and timeframes. Led by researchers with national reputations, such organizations have the capability and independence to attract and use highly qualified researchers. The value of this pathway is constrained by agency mandates—these organizations exist to serve particular users and their policy concerns. Because reports are framed in policy terms, policy makers also may not appreciate how much these agencies depend on the reservoir of research. Such organizations also depend on people with skills that may differ substantially from common professional training. Researchers able to bridge this gap may be discouraged if their efforts are not appreciated or rewarded by peers.
Formal authority can enhance credibility, as can, less formally, good work that gains recognition. In the executive branch, there are many examples of intramural research or policy offices staffed by “honest brokers”; achieving credibility depends on the caliber of staff and openness of agency leadership to supporting rigorous analysis. Outside of government, groups such as the Kaiser Family Foundations' (KFF) Commission on Medicaid and the Uninsured, the Robert Wood Johnson Foundation's (RWJ) Council on Health Care Economics and Policy, and the Commonwealth Fund's Commission on a High Performance Health System perform similar functions. Credibility may be harder to achieve when such entities are affiliated with interest groups or private sector organizations.
The mass media reaches opinion leaders and helps policy makers to become aware of research, which researchers can use to their advantage in publicizing research findings. Recently, some major media outlets arguably have become knowledge generators, as well as intermediaries, in the communication of research findings. Such media outlets now regularly sponsor surveys or other research analyses that address questions viewed as important to their readers, especially opinion leaders, and then report the findings publicly. For example, the Public Broadcasting System and ABC News have teamed up with the KFF/Harvard School of Public Health to study public opinion. Nationally focused papers, such as the New York Times or Wall Street Journal, sometimes develop long-term investigative reports, including primary analysis of such data as financial records, to increase public visibility of an issue. Media professionals know what is newsworthy and can effectively frame and communicate messages. Such outlets may not necessarily be committed to the standards typical of traditional research, however.
While the pathways already mentioned assume a fixed and generally researcher-determined reservoir of research for policy makers, there also are ways for users to influence the value of research.
This pathway involves active solicitation by users of research syntheses addressing important policy topics. Users define questions and timeframes; they may even specify the format for presentation and discussion of results to make these products relevant to policy makers. Work by the Canadian Foundation for Health Services Research and the RWJ Synthesis Project falls into this category (Clancy, Bilheimer, and Gagnon 2006), as does some work commissioned from the IOM to support expert review of a particular question. The value of user-commissioned work varies with how the question is framed, the underlying quality of available knowledge, and the timeliness and policy relevance of the synthesis.
Organizations responsible for managing programs and executing policy, such as the Centers for Medicare and Medicaid Services (CMS), directly commission research on questions of interest. Buxton and Hanney (1996) term this the “problem solving/engineering model,” although it can be much more. Such work often is likely to be “work for hire” or conducted under contract. The researchers involved vary in their independence in defining methods, presenting findings, and drawing conclusions; how such issues are handled will influence the credibility of the work. Commissioned work is responsible for some of the most well-regarded research studies (e.g., the RAND national health insurance experiment) (Newhouse and the Insurance Experiment Group 1993) or the evaluation of Medicare HMOs (Brown et al. 1993). This pathway also can lead to concentration on studies narrowly defined by context- and time-specific questions. Findings may not always be released publicly, and publication may not be valued, leading to research findings that do not contribute as much to the cumulative base of knowledge available to support policy making and managerial decision making.
Some groups, such as the Canadian Foundation for Health Services Research (Lomas 2000), aim for a middle ground, modifying traditional assumptions of grant review while maintaining peer review. Both users and researchers review grant applications, with awards dependent on requiring minimum thresholds for both scientific merit and practical relevance. Ongoing relationships may be fostered between research organizations and users, as in AHRQ's ACTION program. Processes often are structured so that the researcher and user agree on the criteria for a useful study. Because of differences in culture and priorities between researchers and users, identifying research that satisfies both audiences may be challenging, particularly when data are limited. One concern is that collaboration will lead to the worst of all worlds, that is, research that is neither of good quality nor useful.
On rare occasions, trained and experienced researchers are appointed to key policy or management positions and become users. Familiar with both research and policy worlds, they may initiate efforts that apply research to policy objectives and build supporting analytic capacity. For example, Mark McClellan's appointment to head CMS in the George W. Bush administration resulted in CMS providing more support to performance measures, public reporting, and effectiveness research. When she headed HHS's health reform initiative under President Carter, Karen Davis structured that process to incorporate the use of research. However, most researchers do not have the skills, interests, or political access to obtain such appointments. Once appointed, they also may face demands that generate conflicts between their roles as researcher and policy maker, leading to a loss of professional credibility or policy effectiveness.
In the real world, pathways often do not exist solely in the forms characterized here; rather, a composite of pathways contributes to research use. A good example involves Congress' authorization of a change in Medicare physician payments allowing for resource use; this came about through a mix of one researcher's initiative (Dr. William Hsiao), contracted work to make original findings operational, and expert review by an intermediary (the Physician Payment Review Commission). An infrastructure that recognizes and better supports multiple pathways is likely to best enhance the uptake of health services research.
The process through which health services research is used in policy making is complex and multidimensional, including a fair amount of uncertainty not controlled by researchers. Perhaps because of its complexity, and their competing demands, researchers may treat this process as a “black box” that will absorb their work and make it relevant to policy. Such perspectives work in some circumstances, but not others. In this paper, we have sought to make more explicit the “black box” of activities and considerations that mediate between completion of research and its use. The analysis suggests a number of ways in which the utility of existing health services research can be enhanced.
Publication of research contributes to its ultimate use, but additional efforts by researchers can increase its potential. This analysis reinforces the distinction between research findings and message. Researchers can enhance the utility of their work by thinking as carefully about the meaning of the findings (the “message”) as they do about their methods. The analysis also notes the relevance of communication and use of intermediaries in reaching end users. Press advisories or other notices targeting key audiences may increase the reach and uptake of research substantially. Key audiences include intermediaries who influence what end users hear, as well as the end users themselves. Networking and trust make a difference in knowledge transfer, so researchers willing to engage in communications that go beyond publication are likely to find a greater payoff; expectations, however, should be modest, due to the variety of factors influencing policy decision making.
This analysis highlights the fact that research may be relevant and valuable, even if not immediately useful. It also illustrates how uses of research may be important, but not always visible. The analysis of pathways between research and its use could help those commissioning research to think more carefully about the trade-offs between funding very focused research and permitting flexibility, within constraints, to encourage research that could be valuable in the future. Like researchers, policy makers also can benefit from understanding the distinction between research findings and policy messages. Building an infrastructure that supports better mining and synthesis of research can make the findings more relevant for applications unlikely to be addressed by single studies or that entail some ambiguity. To address this problem, one idea is a publicly accessible repository of research findings that will overcome delays or barriers to publication and make it easier for users to access available studies. But Internet search engines already exist and the potential for information overload is high. Thus, building bridges between individual studies and the knowledge they generate cumulatively about relevant policy concerns may be even more important than a repository in allocating resources. Existing intermediaries can make such links but often are pressed for time and resources. Enhancing available synthesis would help intermediaries take better advantage of cumulative knowledge. It also might expand the audience for research findings and develop linkages not well addressed in the current intermediary structure.
Analyzing pathways that link research to its use shows that the world does not divide itself neatly into “researchers,” who generate knowledge, and “users,” who apply it. Instead, there is a spectrum of often unexplored and intermediate activities and feedback loops that determine what messages, if any, develop from a study or body of work, and how these are communicated and used in the policy process. Organizations that play a leadership role in bridging both worlds, such as AcademyHealth (2006), can help researchers and users alike to better understand each other and create bridges to meet the needs of both audiences. Such organizations can help users to understand why delaying or limiting public disclosure may hurt their interests in the long term (AcademyHealth 2004); transparency and the public reporting of results is essential to a free flow of knowledge. Leadership also can help to address issues of mutual concern, such as balancing safeguards on research quality and independence with policy makers' strong interest in making research more relevant and timely. Leaders also might consider whether the current imbalance between clinical and health services research funding makes sense in an environment where rising health care costs raise continuing issues of how best to think about and encourage value (AcademyHealth 2006).
The work cited in this paper suggests that some nations, such as Canada and the United Kingdom, have been more forthright and aggressive in building bridges between research and policy use than has been the case in the United States. The reasons are unclear, but the gap likely reflects the greater emphasis U.S. policy places on “muddling along” with limited government control over the free market of ideas, and limited public investment in an infrastructure to support more deliberative policy making. This paper may indicate ways to do better.
Joint Acknowledgment/Disclosure Statement: The idea for identifying the pathways that influence policymakers' use of health services research, upon which this paper is based, came from an Expert Panel advising the Agency for Healthcare Research and Quality (AHRQ). The panel's mandate was to oversee Mathematica Policy Research's evaluation of the effectiveness of AHRQ's grant-supported research on health care costs, as well as productivity, organizational, and market forces. Michael Hagan of AHRQ was the project officer for that study. Technical panel members included Sharon Arnold (AcademyHealth), Paul Ginsburg (Center for Studying Health Systems Change), Robert Helms (American Enterprise Institute), John Christensen (University of Minnesota), and Gail Wilensky (Project Hope). We are particularly grateful to Paul Ginsburg, to Erin Taylor (MPR), and to the anonymous reviewers for comments on an earlier version of this paper. Felita Buckner and Alfreda Holmes provided secretarial support. Molly and Jim Cameron provided editorial services.
1To paraphrase the IOM, “health services research is an interdisciplinary field that investigates the structure, organization and processes, and effects of health services delivery and financing on people and institutions” (author's revision of Gray 2003).
2Although the focus of this review is on the diffusion of innovation in organizations, the concepts are broadly relevant to understanding how decisions are made, and what evidence, such as research, is used or not used in that process.
3Many years ago, the National Center for Health Services Research commissioned formal syntheses of research as part of its “User Liaison Program.”
Additional supporting information may be found in the online version of this article:
Appendix SA1: Author Matrix.
Please note: Wiley-Blackwell is not responsible for the content or functionality of any supporting materials supplied by the authors. Any queries (other than missing material) should be directed to the corresponding author for the article.