To help researchers think more concretely about how translation activities influence whether research findings are used, we have drawn on the literature and our own experience to develop a framework showing the factors that mediate the translation of research findings into messages communicated to and used by policy makers (). By “used,” we mean that, at a minimum, policy makers or those seeking to influence them are aware of ideas generated through the process of research, and that those ideas have some influence, either on the debate over policy decisions or on the actual decisions.
Factors, Processes, and Actors That Shape Pathways between Research and Its Use Source: Author's analysis; the concept of a research reservior builds on works by Hanney et al. (2003). Note: Numbers (in circles) refer to pathways in .
Potential Pathways to Research Translation by Key Driver and Limits/Barriers
Within the framework, we identify nodes that distinguish 10 pathways through which research may get applied in policy making, emphasizing different ways in which messages get framed and communicated, who is involved, and what drives or constrains movement along the pathway. The framework in which the pathways are nested assumes they all operate in a political and professional environment, and that there is a reservoir of knowledge to which researchers contribute and policy makers can use (Hanney et al. 2003
). The reservoir's richness and contents are influenced over time by the research funded and its topics. Only research that is completed, documented, and made public contributes to the reservoir; proprietary restrictions, release policies, and publication lags all limit flow into it (AcademyHealth 2004
). Richer, better quality findings enhance potential contributions (Soumerai et al. 1993
), so funding streams and their allocation are critical.
Some pathways reflect traditional emphasis on research as “knowledge development” and “enlightenment.” These generally assume that meritorious research findings will find an appropriate audience without much emphasis on the mediating process. Other pathways emphasize the role of intermediary, as well as how users more actively influence the way research is conducted and used. The goal is not to rank pathways, but to create a better understanding of each, and their possible roles in enhancing the use of research.
Traditional Pathways: Meritorious Findings Drive Use
Traditionally, researchers aim for peer-reviewed publication of their work in journals or other vehicles. Such findings influence policy use directly through at least three pathways.
Some studies produce blockbuster findings that immediately inform or change the policy debate and provide value over time (RAND Health 2006
). For example, the National Medical Expenditure Survey's estimates on the uninsured influenced congressional consideration of legislation to address the effects of unemployment on health coverage (Wilensky 1987
). McGlynn et al.'s (2003)
work on the extent of inappropriate care has been used to enhance policy support for quality improvement. Publication in a prestigious journal can generate press coverage that propels the communication and uptake of key messages from research. This pathway also may be used less visibly; an example is the work Ashe and her colleagues conducted on risk-based predictive modeling, which was honored with AcademyHealth's (2008)
HSR Impact Award (AcademyHealth 2008
“Big bang” research carries a great deal of weight and professional satisfaction. Sometimes, however, important findings may not match current policy interest. Also, publication in prestigious journals may delay the use of important research results because of restrictions on prior public disclosure. Realistically, most research will not have sufficient saliency for a “big bang.”
Gradual Accumulation and Diffusion
Researchers may strive for a “big bang,” but it is more common for research to contribute to a knowledge reservoir that accumulates over time and is available for policy use. In some cases, this accumulation of findings is converted into “common knowledge” through what has been termed the “enlightenment/percolation/limestone model” (Buxton and Hanney 1996
). A good example involves the gradual build-up of understanding that health insurance coverage influences access to care and outcomes. The absence of formal processes to synthesize and identify “messages” from the pool of research means, however, that there is no quality control (Weiss 1979
). (Formal synthesis could enhance “scientific vetting,” as in the Institute of Medicine's (IOM) (2001)
synthesis of generally accepted findings on the impact of lack of insurance, which added to the credibility of the research.) Use of accumulated knowledge also is impeded when research is difficult to retrieve because findings are not published in archival sources or indexed in appropriate databases; web-based search engines may mitigate this. Older research also may be forgotten or viewed as dated, which can occur when support for specific research topics waxes and wanes, as with interest in the limit between Medicaid payment rates and physician access (Gold et al. 2006
Researchers can build on existing knowledge through formal synthesis of existing evidence in a particular area, such as knowledge of performance by health maintenance organizations (Luft 1981
), or from two decades of health care research on organizational change (Bazzoli et al. 2004
). Although exceptions exist,3
support for such synthesis tends to be limited in health services research, with funders preferring to support new work. Researcher-initiated syntheses run the risk of addressing questions of interest to other researchers, not policy makers, with exceptions (e.g., Soumerai et al. 1993
). Syntheses pointing out where more research is needed can be important, but those drawing concrete conclusions, even if constrained, are likely to be valued more by policy makers.
Intermediaries Help to Communicate Research Messages
A key barrier to the use of research is the potential users' lack of awareness of a study or body of work and why it may be relevant. Intermediaries or processes can support better connections between the policy needs of users and findings from researchers.
Researchers as Communicators and Experts Consulted by Users
Through this pathway, researchers take steps to make it more likely that policy makers will know about and make use of research. Simple steps can enhance relevance, such as thinking carefully about the conclusions drawn from a study's findings and developing a clear message. Embedding the message in press advisories and other concise documents shared with potential intermediaries and users make it more likely that messages will reach key users. Such actions also allow researchers to help shape the “take away.” Some refer to this as satisfying the “elevator test,” using messages communicated between floors.
Such actions make policy makers aware of researchers' expertise. Ongoing relationships might develop, particularly when researchers actively cultivate them by being responsive or promote them through actions such as writing in mass media vehicles or speaking and spending time at user events. For example, in 2004, Uwe Reinhardt and Stuart Altman received AcademyHealth's Distinguished Investigator Award as recognized leaders in this form of translation (AcademyHealth 2008
). This pathway draws on knowledge of a reservoir of research and knowledge reaching beyond particular studies. Effective performance as an active and credible research translator takes skills and judgment, however, particularly when policy questions do not neatly match a given body of research. Researchers often are not trained in such skills, and the same interests drawing them to research may limit their involvement in communication endeavors.
Formal Intermediary-Brokered Translation
In some instances, formal organizations exist to bridge the gap between the different needs and orientations of the research and policy worlds. At the federal level, examples are the Congressional Budget Office (CBO), Congressional Research Service, Government Accountability Office (GAO), and such specialized entities as the Medicare Payment Advisory Commission. Some, such as GAO, emphasize original research in direct response to congressional requests. Others build on the existing reservoir of research. For example, CBO is required to provide a fiscal estimate of the costs of each piece of legislation; it also analyzes potential legislative options and provides more general guidance that often determines which legislative policy options are feasible. With a staff dominated by economists, CBO draws heavily from existing research to execute its mission. Staff evaluate and synthesize relevant research findings and use research to support the development of cost models. Some states also have established independent policy centers, sometimes affiliated with universities or foundations, to carry out similar functions.
To policy makers, such intermediaries have advantages, notably their responsibility to provide rigorous analysis consistent with policy needs and timeframes. Led by researchers with national reputations, such organizations have the capability and independence to attract and use highly qualified researchers. The value of this pathway is constrained by agency mandates—these organizations exist to serve particular users and their policy concerns. Because reports are framed in policy terms, policy makers also may not appreciate how much these agencies depend on the reservoir of research. Such organizations also depend on people with skills that may differ substantially from common professional training. Researchers able to bridge this gap may be discouraged if their efforts are not appreciated or rewarded by peers.
Formal authority can enhance credibility, as can, less formally, good work that gains recognition. In the executive branch, there are many examples of intramural research or policy offices staffed by “honest brokers”; achieving credibility depends on the caliber of staff and openness of agency leadership to supporting rigorous analysis. Outside of government, groups such as the Kaiser Family Foundations' (KFF) Commission on Medicaid and the Uninsured, the Robert Wood Johnson Foundation's (RWJ) Council on Health Care Economics and Policy, and the Commonwealth Fund's Commission on a High Performance Health System perform similar functions. Credibility may be harder to achieve when such entities are affiliated with interest groups or private sector organizations.
The Mass Media as Intermediary
The mass media reaches opinion leaders and helps policy makers to become aware of research, which researchers can use to their advantage in publicizing research findings. Recently, some major media outlets arguably have become knowledge generators, as well as intermediaries, in the communication of research findings. Such media outlets now regularly sponsor surveys or other research analyses that address questions viewed as important to their readers, especially opinion leaders, and then report the findings publicly. For example, the Public Broadcasting System and ABC News have teamed up with the KFF/Harvard School of Public Health to study public opinion. Nationally focused papers, such as the New York Times or Wall Street Journal, sometimes develop long-term investigative reports, including primary analysis of such data as financial records, to increase public visibility of an issue. Media professionals know what is newsworthy and can effectively frame and communicate messages. Such outlets may not necessarily be committed to the standards typical of traditional research, however.
Users Seek to Enhance Value of Research
While the pathways already mentioned assume a fixed and generally researcher-determined reservoir of research for policy makers, there also are ways for users to influence the value of research.
Commissioning Synthesis around Policy Problems or Questions
This pathway involves active solicitation by users of research syntheses addressing important policy topics. Users define questions and timeframes; they may even specify the format for presentation and discussion of results to make these products relevant to policy makers. Work by the Canadian Foundation for Health Services Research and the RWJ Synthesis Project falls into this category (Clancy, Bilheimer, and Gagnon 2006
), as does some work commissioned from the IOM to support expert review of a particular question. The value of user-commissioned work varies with how the question is framed, the underlying quality of available knowledge, and the timeliness and policy relevance of the synthesis.
Organizations responsible for managing programs and executing policy, such as the Centers for Medicare and Medicaid Services (CMS), directly commission research on questions of interest. Buxton and Hanney (1996)
term this the “problem solving/engineering model,” although it can be much more. Such work often is likely to be “work for hire” or conducted under contract. The researchers involved vary in their independence in defining methods, presenting findings, and drawing conclusions; how such issues are handled will influence the credibility of the work. Commissioned work is responsible for some of the most well-regarded research studies (e.g., the RAND national health insurance experiment) (Newhouse and the Insurance Experiment Group 1993
) or the evaluation of Medicare HMOs (Brown et al. 1993
). This pathway also can lead to concentration on studies narrowly defined by context- and time-specific questions. Findings may not always be released publicly, and publication may not be valued, leading to research findings that do not contribute as much to the cumulative base of knowledge available to support policy making and managerial decision making.
Users Provide Input into New Research
Some groups, such as the Canadian Foundation for Health Services Research (Lomas 2000
), aim for a middle ground, modifying traditional assumptions of grant review while maintaining peer review. Both users and researchers review grant applications, with awards dependent on requiring minimum thresholds for both scientific merit and practical relevance. Ongoing relationships may be fostered between research organizations and users, as in AHRQ's ACTION program. Processes often are structured so that the researcher and user agree on the criteria for a useful study. Because of differences in culture and priorities between researchers and users, identifying research that satisfies both audiences may be challenging, particularly when data are limited. One concern is that collaboration will lead to the worst of all worlds, that is, research that is neither of good quality nor useful.
Researcher as User
On rare occasions, trained and experienced researchers are appointed to key policy or management positions and become users. Familiar with both research and policy worlds, they may initiate efforts that apply research to policy objectives and build supporting analytic capacity. For example, Mark McClellan's appointment to head CMS in the George W. Bush administration resulted in CMS providing more support to performance measures, public reporting, and effectiveness research. When she headed HHS's health reform initiative under President Carter, Karen Davis structured that process to incorporate the use of research. However, most researchers do not have the skills, interests, or political access to obtain such appointments. Once appointed, they also may face demands that generate conflicts between their roles as researcher and policy maker, leading to a loss of professional credibility or policy effectiveness.
The Relevance of Multiple Pathways
In the real world, pathways often do not exist solely in the forms characterized here; rather, a composite of pathways contributes to research use. A good example involves Congress' authorization of a change in Medicare physician payments allowing for resource use; this came about through a mix of one researcher's initiative (Dr. William Hsiao), contracted work to make original findings operational, and expert review by an intermediary (the Physician Payment Review Commission). An infrastructure that recognizes and better supports multiple pathways is likely to best enhance the uptake of health services research.