|Home | About | Journals | Submit | Contact Us | Français|
The Mission of the Agency for Healthcare Research and Quality (AHRQ) has been to support and conduct health services research and to disseminate those research findings. Recently the Agency has changed its mission to: “Improving the quality, safety, efficiency and effectiveness of health care for all Americans.” For agency personnel working with the topic of patient safety, that change has created a need to develop greater awareness of the current patient safety initiatives underway at leading health care systems in order to determine where AHRQ might best play a role in helping these systems more rapidly adopt new practices to improve patient safety.
In order to make that determination, AHRQ conducted a customer needs assessment of leaders in selected health care systems, asking them questions about their current implementation initiatives and their perceived needs for continued implementation of patient safety initiatives. Although not designed or conducted as a research study, the hour-long interviews produced rich insights into the implementation efforts of patient safety initiatives.
The senior leaders interviewed in each of the health care systems, described implementing patient safety initiatives on multiple fronts—in some systems as many as 15 initiatives were underway. As the number of initiatives attests, there was no lack of knowledge about what patient safety practices should be implemented (CPOE, rapid response teams, reduction in surgical site infections) rather the major struggle these health care systems faced was the “how to” of implementation. Most initiatives were only newly begun, so these leaders were not yet confident about what they had learned from these efforts or whether they could be sustained over time. These health care systems drew many of the ideas for initiatives from outside of health care, for example, the nuclear power industry or aviation. The executives expressed concern about a number of issues including: how patient safety initiatives should be sequenced, the lack of benchmarking data to measure their systems against and the pressing need for IT standardization.
The insights from this customer needs assessment revealed a wealth of implementation knowledge in the field and has led AHRQ to create an opportunity for leading edge health care systems to learn from each other via learning networks.
The primary mission of the Agency for Healthcare Research and Quality (AHRQ) has been to support and conduct health services research and to disseminate those research findings.1 Recently the Agency has changed its mission to: “Improving the quality, safety, efficiency and effectiveness of health care for all Americans.” The need for such a change grew out of the recognition, identified in the Institute of Medicine publication Crossing the Quality Chasm: A New Health System for the 21st Century (2001), that a disparity exists between what we know to be essential to the provision of quality health care and what is actually delivered. That report acknowledges that a large portion of research in this area is not being used in practice to the extent it could be. The Agency's new focus is the attempt to fill this gap—to ensure, for example, that if research has determined that prescribing beta blockers after a heart attack decreases an individual's chances of experiencing another heart attack—that such information is not only disseminated to hospital settings, but it is more fully implemented in hospital settings. This inclusion of implementation within AHRQ's mission is a substantive change, in part, because of the nature of the knowledge involved.
Research findings, which have been AHRQ's primary knowledge base, are by design context free, whereas implementation knowledge is necessarily context specific. It is not possible to understand a specific action that worked successfully in a provider setting, without understanding the setting itself. Whereas research findings are the result of systematic study of scientists under largely controlled conditions, implementation knowledge is the outcome of practical attempts by frontline workers to implement those findings in local settings. Research findings are produced in agreed upon written formats (reports and journals) for the consumption of fellow researchers. Implementation knowledge resides largely in the heads of those frontline workers who have attempted the implementation. Frontline knowledge tends to be communicated primarily through word of mouth or brief “best practice” descriptions (Dixon 2000). This contrast between research findings and implementation knowledge suggested to AHRQ the need for AHRQ to build closer, longer-term working relationships with providers than they have needed in the past. It suggests as well, a need for alternative distribution channels.
As a leader in patient safety research, the Agency has and will continue to have a number of research findings, tools, and products available. With the stronger focus on implementation, the Agency has been determining how best to get this information, much of it newly generated over the past year, out and used by health care providers. At about the same time, the Agency was becoming interested in the concept of HROs. The Agency saw a great opportunity to work with health care systems interested in becoming an HRO—there were AHRQ products to contribute to this field, and the Agency would gain first-hand knowledge of additional products, tools, and research that might be needed by this audience. Equally as important, the Agency would learn more about the “how” implicit in implementation.
The first HRO meeting resulted from discussions that occurred during an AHRQ meeting focused on patient safety where states were invited as teams representing private and public sector. At this meeting, some hospital-based participants expressed interest in promoting the HRO framework in health care organizations. In response, AHRQ held a subsequent 2-day meeting focused on HROs where participants discussed the knowledge and skills that chief executive officers (CEOs) would need in order to better lead their organizations in that direction. After the meeting, participants were eager to learn more about applying these skills and sharing experiences.
As a follow-up to the 2-day meeting, AHRQ, working with the Delmarva Foundation, identified health systems that were interested in achieving a state of high reliability, focusing on health care systems rather than individual hospitals because the spread of knowledge could potentially be much greater. Because the HRO concept was so new to health care, AHRQ decided to work with systems considered to be “early adopters” (Rodgers 2003)2 and within those systems work with senior leadership, recognizing that senior leadership involvement in the diffusion of new medical information in health care settings is more likely to lead to change (Mills, Weeks, and Surrott-Kimberely 2003).
In order to determine the current extent of HRO implementation knowledge among early adopter health care systems, AHRQ conducted in-depth interviews in November and December 2004 using the topics and questions listed in Table 1. The selection of health care systems to interview was based largely on their interest and commitment to patient safety. There was no attempt to create a representative sample of health care systems, excepting some general intent to address geographic diversity, system size, and a mix of rural versus urban settings. All the systems chosen were nonprofit. In all, 10 health systems were chosen, and of those, eight agreed to participate in a 1½ hours in-depth telephone interview. The interviewees were senior staff members, two of whom were Senior VPs, two CMOs, one CQO, two CEOs, and one VP. Table 2 provides a list of the systems that participated in the interviews, a description of each system, and the names of the individuals interviewed.
Dr. Dixon conducted each of the interviews along with a member of AHRQ's staff who explained the purpose of the interview. In each case permission was asked and given to record the interviews. During several of the interviews not all of the questions could be asked because the agreed upon time was over. In those cases it was usually the questions about sources for information and priority populations that were not asked. The interviews were transcribed and a preliminary analysis was conducted by Dr. Dixon to identify themes related to:
Copies of the interview transcripts were also provided to the Delmarva Staff, the patient safety staff of AHRQ, and five knowledge management consultants selected by AHRQ.
An analysis meeting was held with the Delmarva staff to review the preliminary themes. Similarly, a teleconference was held with the knowledge management consultants to gather their insights from reading the interview transcripts. Finally, a face-to-face meeting was held with AHRQ staff, Delmarva staff, and the knowledge management consultants during which Dr. Dixon reviewed the insights gleaned from the interviews and received comments and insights from the attendees.
An early draft of this article was provided to the interviewees, asking for both their corrections to any inaccuracies and for permission to quote them in specific cases. Interviewees responded with a few changes, which were made, and permission was granted for the quotes used in this article. The insights gained through these interviews were based on the report of the interviewees. AHRQ has no specific knowledge of the actual reliability and safety performance record of the organizations interviewed.
This group of health care system leaders exhibited comprehensive knowledge of patient safety constructs. For example, in describing the characteristics of an ideal safe health care system, they talked about concepts such as looking beyond issues of individual competence to system errors, the value of learning from near misses, the need to create redundant systems, and the use of forcing functions. They acknowledged that it was not possible to eliminate human error and talked about the need to provide error-free care, even given the fallibility of human beings. To Err Is Human (Kohn, Corrigan, and Donaldson 1999) was frequently referenced, and they spoke fluently about the work of recognized leaders in patient safety such as James Reason and Jim Baigen. Their descriptions of an ideal safe health care system tended to focus on two critical aspects: (1) the use of technology and automation and (2) the establishment of a robust safety culture. On the basis of these responses, it seemed evident that this select group of health care leaders had internalized the values that are needed to fully support patient safety in their health care systems.
While they were well versed in the concepts of patient safety, only two volunteered the term “HRO” in their descriptions of their patient safety efforts. With one exception they did not reference Weick's (Weick and Sutcliffe 2001) concepts (e.g., preoccupation with failure, sensitivity to operations) about what it takes to become an HRO. When specifically asked about Weick's framework for HROs, most indicated they had heard of it, but only at Sentara was it being used to select and design interventions. Although the HRO framework was not central to the thinking about patient safety for most of these systems, three of the eight systems had originated frameworks for their patient safety work.
All of the hospital systems interviewed were implementing multiple patient safety initiatives: in some systems, as many as 15 different initiatives were being implemented concurrently. Figure 1 lists the initiatives that were frequently mentioned. It was not a part of the interview protocol to elicit a complete list of initiatives for each system, so Figure 1 reflects those initiatives that came most readily to mind as the leaders responded to the interview questions.
Both the number of initiatives within each health care system and the similarity of initiatives across the eight health care systems indicate that these organizations have a clear understanding of the practices that need to be implemented in order to improve patient safety. Where there was considerable uncertainty, but also considerable creativity, was in how they were accomplishing the implementation of these evidence-based practices. Examples of creativity in implementation included:
Most of the projects the health care leaders described were only newly begun, as represented by comments such as: “We just started this 3 months ago,”“We implemented the survey but don't have the results back,” and “We just bought a new 25M dollar computer system to install.” Although some leaders were able to report early results, most did not yet know what they had learned from these efforts or whether they could be sustained over time. Gary Yates of Sentara spoke for many others when he said: “If safety is a marathon, I think we just now maybe know what the two mile marker looks like, but boy, there are a lot of miles between here and there.”
The leaders' sense of pride in their current efforts was obvious. But it was tempered by the realization that they were at the beginning of the safety journey, and at this early stage, there was little about which they could say with confidence, “We have evidence that this works.”
One thing that reflected being at the beginning of the safety effort was their frequent use of the term “struggle” in describing their implementation efforts. For example, Mike Connelly, CEO of CHP said:
One of the issues we are struggling with right now is reporting for boards. In a system of our size, there are multiple levels of boards, e.g., parent board, local boards. We are asking ourselves, What are the various levels of responsibilities for a sentinel event? Who should know about it? Who is going to deal with it? You could easily have the problem of too many cooks stirring the pot. We are trying to work on more clarification of roles and information flows.
The leaders frequently described trying an initiative, getting some results, recognizing that they were not where they wanted to be, and then trying something else to see if that would work. As an example, Jeff Selberg, CEO of Exempla, explains that Exempla put together a task force to develop a protocol to reduce the variability in the way opiate medications were ordered (in terms of frequency, amount, and magnitude). Initially the protocol was voluntary, but when problems continued, the pharmacy was directed by the medical executive committee not to fill any order that did not follow the protocol. This second try worked, as evidenced by having no near misses since the protocol was made mandatory.
As this example illustrates, for most health care systems there did not seem to be a lack of knowledge about what to implement, rather the struggle was around the “how to.” Gary Yates of Sentara described the struggle this way:
I think we are sort of a science experiment. Remember when your kids had the science lab at home? They keep pouring stuff in to see what happens. It sort of feels like that sometimes. We are not running controlled experiments; we do where we can. So we add a little red in this tester and a little blue into that one. So we keep adding.
It was remarkable how many of the implementation processes that these leaders reported were borrowed from other industries. Centra drew heavily on GE's change process and brought in GE “black belts” to help in the implementation. Ascension found Motorola's work on culture useful, and Exempla drew ideas from Lockheed Martin. Sentara's stand-downs were inspired by the Navy, and they made heavy use of consultants from the nuclear power industry as well.
The use of external sources far out shadowed borrowing ideas from other health care systems. The exception was the frequent mention of IHI as a source of useful ideas and the VHA's work in crew resource management (which was, of course, initially derived from work in the airline industry).
In part, the phenomenon of external borrowing could have come about because all of the health care systems found themselves at the same beginning phase of implementation and thus were looking to other industries to find proven practices. It may also be that the health care literature has primarily been focused on evidence-based practices—the “what”—with much less attention being paid to implementation practices—the “how.” Gary Yates, noted:
There still isn't a good way to know what is going on among organizations at the cutting edge that are trying new practices or adopting unusual novel approaches of improving safety in health care. We are members of most of the organizations; we hear about things. But it would be useful if there were a way to learn from organizations that are out there on the edge, thinking about high reliability in a thoughtful way. It would be nice to connect there.
The leaders identified a number of patient safety challenges they face going forward.
The need for transparency in reporting, analogous to what the National Transportation Safety Board requires of the airline industry, was seen by several of the health care leaders as a necessary next step if patient safety is to be taken seriously by the health care industry as a whole. ARHQ was viewed as having a role to play in advocating for reporting and perhaps even creating the methodology through which that reporting might occur.
Transparency was closely tied to the issue of incentive programs for performance in safety. These leaders were generally supportive of having such systems in place, but they cautioned that the measures used would need to be evidence based as well as based on real clinical activities. They acknowledged that an incentive program would not be in place in the near future, given that it would require a national-level integrated electronic medical record database. But they felt that the time for agencies such as AHRQ to begin actively working toward that goal was at hand.
In speaking about the issue of transparency, some of the leaders talked in terms of benchmarks—that is, a set of metrics that everyone reported into and that could guide behavior. Any or all of these efforts at national reporting were viewed as a powerful and needed force for change.
The leaders recognized that computer vendors are evolving but saw them as far from being where they needed to be. They found themselves frequently having to select “the least worst” rather than an IT system that met their needs. Even after a system was purchased, costly adaptations were often required to make the software work in their specific setting. One leader spoke of his institution having a 2-year backlog of modifications.
The lack of standardization across vendors was an especially troublesome problem. Bob Panzer, CQO of Strong Memorial, illustrated the problem this way:
Because we are not one system in Rochester, we can't get the whole community to have the same electronic records or lab system database that would let us see the labs of someone from the other side of town that got sick and went to our emergency room. So the fragmentation of the systems is frustrating. It would make so much sense for somebody to develop low cost tools we could implement, especially when each of the hospitals chose the same lab computer system without taking the logical next step of providing access to each other's lab results.
All of the leaders had, at the top of their list of challenges, IT solutions that were easy to put in, reproducible, and reliable.
The leaders saw a need to have a clear roadmap about how to phase in patient safety initiatives and the order in which to implement them. That included knowing which clinical conditions to address first and which clinical areas would yield the biggest bang for their buck. Several leaders spoke about how easy it was for them to shoot off in many different directions without a blueprint in place to guide their efforts. These leaders were clearly not asking for more rules or regulations, but rather research and information that would better inform their implementation decisions.
Several health care systems expressed the need to partner more effectively with patients on patient safety issues. Most already had pamphlets and videos that they provided to patients and their families but were looking for what the next level of partnership might entail. They raised the possibility of patients contributing to their reporting system as active partners as well as the idea of holding patients accountable for certain safe practices while in the hospital.
The need for learning from others was raised as an issue on several levels: (1) within a single hospital, (2) across a health care system, and (3) among health care systems so they could learn from each other.
David Pryor, Senior VP at Ascension, described their efforts to disseminate the initiatives developed in one hospital setting to other hospitals across their system.
We are seeing three or four different dissemination vehicles occurring as we go forward. Not the active, classic, linear spread that we had initially envisioned: that is, alphas to several betas, refine the package, and then start it across the system. Instead, what we are finding is more rapid adoption of key parts of these underlying packages that are high-yield components. That is happening through a variety of different and unexpected patterns. But having a much more cogent strategy for how you move these things across a system our size will be required, given the magnitude of the change processes that we are talking about, that we don't have completely worked out yet.
The leaders expressed the need for health care systems that are on the leading edge in patient safety to have a mechanism through which they could learn from each other. ARHQ was looked to as an organization that could possibly help hospitals and health care systems figure out how to effectively transmit lessons learned.
As AHRQ moves toward involving itself in implementation in addition to the more traditional role of the dissemination of its research, it anticipates considerable change in its work processes and in the competencies needed to conduct those processes. It will require a broader engagement with users as well as coordinating opportunities for users, who are in the process of implementation, to exchange what they are learning with each other.
Based on the rich interview data, AHRQ decided to build a learning network among leading edge health care systems as a mechanism to help them meet their goals. Learning networks are an avenue for “exchange of views or shared attempts at problem solving” (Bessant, Kaplinsky, and Morris 2003). A learning network would also allow AHRQ to take advantage of learning moments, instances when the group expresses an interest in a research finding, tool, or product that AHRQ has funded. In addition, the Agency will have a group advising it on additional research needs. At the time this article was written, AHRQ was in the midst of selecting learning network participants based on the intersection of user needs and AHRQ's priorities and capabilities.
The HRO project will be evaluated in terms of the effectiveness both of discrete projects within this initiative, and of the project as a whole. It is expected that rich information, such as that found in the needs assessment, will continue to present itself as the project moves forward. The project is expected to be completed within the timeframe of 2007–2009.
The customer needs assessment and analysis of findings discussed in this paper is supported by the Agency for Healthcare Research and Quality under contract number 290-04-0009. This contract is with the Delmarva Foundation for Medical Care.
Disclaimer: The authors of this report are responsible for its content. Statements in the report should not be construed as endorsement by the Agency for Healthcare Research and Quality or the U.S. Department of Health and Human Services.
1The Agency for Healthcare Research and Quality was established in 1989 under its previous name—the Agency for Health Care Policy and Research. It was reauthorized in 1999 under its current name.
2Early adopters are the group of individuals who are second in line to adopt a new innovation after the first group called innovators.