|Home | About | Journals | Submit | Contact Us | Français|
In May of 2001, thirteen experts on computerized provider order entry (CPOE) from around the world gathered at a 2-day conference to develop a consensus statement on successful CPOE implementation. A qualitative research approach was used to generate and validate a list of categories and considerations to guide CPOE implementation.
Reasons cited for computerized provider order entry (CPOE) system adoption include medical error reduction,1–3 and provision of just-in-time feedback to physicians,4–6 but use of CPOE is not widespread.7Presumably, implementation lags because CPOE, by reputation, is hard to implement, expensive, and difficult to coax clinicians (and especially physicians) to use. At organizations with successful CPOE implementations, people instrumental in creating the success possess valuable experiential knowledge. As part of a three-year research grant funded by the National Library of Medicine, a two-day consensus conference enticed experts on CPOE implementation to share their expertise through discussions and generation of recommendations for CPOE adoption and usage to aid clinicians, vendors, hospital administrators, and information technology personnel in addressing the challenges that they will face. Computerized provider order entry was defined for the purpose of the conference as a process by which a clinician with order writing authority sits at a computer to directly enter patient care orders.
The conference convened to develop a consensus set of recommendations for CPOE implementation. A consensus statement is a document, developed and agreed upon by representatives of multiple perspectives, that provides guidance for practice in specific areas.8An important aspect of consensus statement development is the focus on agreement and collaboration. Guidelines differ from consensus statements in that the former may be more prescriptive and use more elaborate branching and hierarchical structures. The process for developing consensus statements has been described well by Gamroth et al.8
A team of Oregon Health & Science University informatics investigators organized and supported the activities of the two-day conference and analyzed the data. To achieve the goal of generating recommendations, such conferences typically follow a commonly used format, with formal talks by experts followed by small group discussions on preselected topics.9,10However, the research team organizing this conference was experienced in the use of qualitative methods, and did not want to restrict discussion to preconceived topics. The goal was to generate new insights as the experts shared their multiple perspectives on CPOE. Formal presentations were replaced with prior reading, and the large group was encouraged to determine the direction of the discussion. A facilitator managed the large group discussions. The research team designed an agenda that included a thinking task before participant arrival, brainstorming after arrival, narrative generation, prioritization of issues, and small team assignments.
Six months before the conference, potential participants were identified through literature searches, citation analysis, and recommendations from known experts. Categories of attendees included administrators, clinicians (physicians, nurses, pharmacists), social scientists, information technology implementers, and vendors (see Appendix for a list of participants and their affiliations). Conference organizers sought geographic diversity, representation of different system types (either commercial or locally developed), and varying implementation site characteristics (university, community, federal). Approximately half of those invited attended. Many attendees represented more than one of the stakeholder groups involved in CPOE implementation. Those who could not attend were asked if they would instead serve as an expert panel to review the final document.
Participants were asked to read a number of papers as background material to establish a shared knowledge base. The goals were to maximize on-site time for the sharing of expertise and experiences, to generate narratives, and to develop consensus statements.
During the two-day conference, participants’ comments were captured on flip charts that were transcribed during the conference. Large group discussions were tape-recorded and later transcribed. Research team members were assigned to each of the small break-out groups to assist with taping and facilitation.
On the first conference day, the organizers described the purpose of the conference and introduced the experts and the research team members. A trained facilitator (who was neither a stakeholder nor, at that time, a central member of the research team) led subsequent activities. Participants were asked to share their perceptions of the “one most important thing,” or success factor, that they had discovered during their own CPOE implementations. Next, participants divided into four small teams, preassigned to promote diversity, which began discussion of the success factors outlined in the large group, prioritizing them and adding others with greater levels of detail. The four teams reported back to the large group. To further explore success factors and to share the lessons each expert had learned through experience, a narrative exercise was devised. The value of sharing lessons learned about CPOE is illustrated by Massaro’s description and analysis of the implementation at the University of Virginia.11,12Each participant was asked to tell both a success story and a failure story. This was an enjoyable way of generating descriptions of additional success factors within an organizational and temporal context. Tales of stumbles, missteps, or pitfalls rather than outright failures were also allowed.
The experts then continued to work in small groups to develop lists of success factors based on the two exercises. For example, one of the failure stories highlighted inadequate levels of readily available support during a pilot implementation; thus, follow-up discussion centered on ideal levels of support. There were several iterations of large and small group work sessions. Thorny issues that provoked disagreement during any of these discussions were placed in a “bank.” The bank served as a way to put aside time-consuming and difficult issues and to deal with them later. Bank issues were discussed during an informal evening session and, after agreement, some were added to the list of considerations.
During the second day of the conference, the large group reviewed the work of the previous day and discussed and prioritized the main discussion points. Plans were made for disseminating the results and gaining subsequent feedback about them.
The data collected (transcriptions of all large and small group interactions) were analyzed with a grounded theory approach, using the words of informants to develop patterns and themes.13Instead of beginning the analysis with a predetermined list of topics, the researchers allow the data, in the form of words in transcripts, to guide code development. The process entails extracting major statements from all data sources and placing them on cards. The primary investigator led a multidisciplinary team of five through a 500 card sort exercise,14iteratively grouping similar ideas together. The sorting process created a taxonomy of high-level themes. A “themes document” was generated and shared with all participants, who engaged in three months of electronic discussion and changes. Experts who could not attend the conference were shown the resulting document and did not suggest any major content changes. The document seems to accurately reflect participating experts’ consensus on CPOE implementation.
The group realized that guidelines or recipes for success would be difficult to create because “CPOE” means different things and raises different concerns at different organizations—academic centers are different from community hospitals, and inpatient care is different from outpatient care. Cost reduction as an overarching goal raises different issues from the goal of patient safety. A desire for high levels of decision support raises different issues from a desire for a basic system. However, the experts also felt that despite such variations, certain themes were common across all CPOE projects and could be addressed. What the experts asked to have available as they first began thinking about CPOE was a menu of possibilities—a list of issues that they might not otherwise think about. Consequently, a list of “considerations” was targeted as the product of the conference rather than a specific guideline or recipe.
The qualitative analysis began with the identification of approximately 500 major statements and ended with ten themes (subsequently condensed) that represent a list of the overarching issues that the experts believe must be considered before CPOE implementation. An overview of the considerations is presented here. Although the complete list included 144 subconsiderations with comments and amplification, the following list offers those derived from them by the authors of this paper. Please visit <www.cpoe.org> for the complete report.
The motivation for implementing CPOE influences where funding will come from, who will provide political support, and who will provide clinical leadership. Implementers should consider the following issues:
Successful CPOE implementations require effective leadership over extended time periods—in different forms and at multiple levels in the organization. Leadership is needed at the executive level to promote a shared vision and provide funding at the clinical level to ensure champions and buy-in and at the project management level to make practical, effective, and useful decisions. Before embarking on the serious undertaking of CPOE, organizations should determine whether the following conditions are met:
Financial considerations are of critical importance. Often, costs are underestimated because purchase of the software is only the beginning of financial outlays; other expenditures such as person-hours for training and support are harder to predict. Decision makers need to consider the following issues:
The manner in which a CPOE application alters and integrates into existing environments and workflows is critical to its success. Users resent disruption of their patient care activities; thus, implementers must consider the following issues:
Constituencies affected by CPOE implementation (e.g. physicians, nurses, ancillary department personnel) must understand the CPOE implementation “value proposition,” i.e., they must do things differently but there will be some benefit in return. One benefit for clinicians is embedded CPOE decision support logic that helps to improve patient care quality and/or to reduce costs. Related issues include the following:
Project management dictates that implementation be completed in carefully planned stages. Key considerations include the following:
Technical details to consider as part of a CPOE implementation include strategic considerations, user considerations, task completion flexibility, and the quality of the application—from customizability to user friendliness. Questions related to the system itself include:
One of the constant themes identified by the experts at the retreat was the importance of live help available “at the elbow” at the time of implementation. In addition to the symbolic importance of supporting the users by being present while they are first using the application, intensive support at “go-live” time allows the implementation team to have direct experience with what is and what is not working well. Most successful implementations have had more post-go-live support than pre-go-live training. Most sites have had 24 × 7 support for at least several days post go-live. Considerations include
CPOE implementation is an ongoing effort that benefits from continuous improvement. It is important that mechanisms for feedback and modification of the system be in place. Questions to consider include
The group used a qualitative approach to create a consensus statement on the specific issues that organizations contemplating a CPOE implementation face. Each consideration should be reviewed by the leadership and implementation team of any organization considering CPOE installation. Some issues will be more easily addressed than others; some will be more relevant to one particular organization than others; and, some are more applicable at different stages in the implementation than others. Some of these questions and issues will have clear and obvious answers, but most will not and will require effort to address. Organizational representatives should focus on the difficult-to-answer questions rather than avoiding them. All of the detailed considerations listed are relevant to a successful implementation.
The qualitative approach allowed us to generate different results than might have been realized with a more prescriptive approach to consensus creation. Of the major categories of considerations, only one grouping was strictly technical. It is possible that the discussion would have centered more on technology if the inductive approach had not been taken. The importance of strong executive leadership at the highest levels in the organization in a CPOE initiative should not be underestimated. Leadership is a thread running through many of the major considerations. Administrative leaders, acting on behalf of the organization, must believe viscerally that CPOE is in the best interest of the institution and be able to communicate that feeling throughout the organization.
Clinical leadership must also be committed to CPOE and communicate this commitment to the clinical staff, who will typically be less than excited about the prospect of CPOE (due to natural resistance to change). Clinical staff will have real concerns that the time to complete work will increase. Arguments about improved safety may appear to end-users as vague and intangible. Clinical leaders must work strenuously to communicate (and physically demonstrate) to their staffs how CPOE provides opportunities for improved quality and efficiency. Administrative and clinical leaders must work together to create a strong sense of “common will” to overcome obstacles that will be encountered during a CPOE implementation.
The expert panel has continued its dialogue about CPOE. It gathered informally during the American Medical Informatics Association 2001 and 2002 Annual Symposia to plan further endeavors. Ongoing efforts will develop suggestions to help organizations find answers to the questions listed under each consideration. There is agreement that future research needs to be done to develop valid tools to measure readiness for CPOE, the effectiveness of the process during implementation, and outcomes during and after CPOE implementation.
The list of considerations presented in this paper and the full text are available at <www.cpoe.org>. This is meant to serve as a guide for organizations to help them make appropriate decisions regarding CPOE. The group agreed that implementation of CPOE is difficult and it must be approached with awareness of the potential problems. The combined wisdom of those who have already experienced successful implementation efforts, summarized in the list of considerations, can serve as a resource for those contemplating future implementations. Health care institutions are being pressured by the Leapfrog Group3 and others to rapidly adopt CPOE. To increase chances of success, leaders are urged to look carefully at the Considerations before they “leap” into CPOE projects.
Consensus conference participants, their current affiliations, and their roles: Jos Aarts (Erasmus University, The Netherlands, social science and international perspective), Marilyn Davis (El Camino Hospital, Mountain View, CA, nurse and implementation specialist at the first POE site), Dick Gibson (Providence Health Systems, Portland, OR, physician and clinician leader), Homer Chin (Kaiser Permanente Northwest, Portland, OR, physician and clinician leader), Paul Nichol (Puget Sound Veterans Administration, Seattle, WA, physician and clinician leader), Marc Overhage (Regenstrief Institute, Indianapolis, IN, physician and clinician leader), Tom Payne (University of Washington, Seattle, WA, physician and clinician leader), Karen Hughart (Vanderbilt University, Nashville, TN, nurse and implementation specialist), Janet Greenman (IDX, Seattle, WA, vendor representative), John Dulcey, MD (Lansdale, PA, physician and clinical systems consultant), Gil Kuperman (Partners Healthcare System, Chestnut Hill, MA, director of research and development), Brian Churchill (Peacehealth, Eugene, OR, nurse and project manager for POE implementation), and Jim Carpenter (Legacy Healthcare, Portland, OR, clinical pharmacist).
Research team members included Joan Ash, P. Zoë Stavri, Paul Gorman, William Hersh, Mary Lavelle, Lara Fournier, and Jason Lyman.
Views expressed by participants are their own and not necessarily those of agencies or organizations with which they are affiliated.
This work was supported by grant LM06942-02 from the National Library of Medicine. Special thanks to Paul Tang, MD, and William Bria, MD, for their comments about the Considerations and to Lara Fournier, MS, for assistance with the manuscript.