A Consensus Statement on Considerations for a Successful CPOE Implementation

doi:10.1197/jamia.M1204

J Am Med Inform Assoc. 2003 May-Jun; 10(3): 229–234.

doi: 10.1197/jamia.M1204

PMCID: PMC342045

A Consensus Statement on Considerations for a Successful CPOE Implementation

Joan S. Ash, PhD, MLS, P. Zoë Stavri, PhD, MLS, and Gilad J. Kuperman, MD, PhD

Oregon Health & Science University, Portland, OR (JSA, PZS); Partners Healthcare System, Chestnut Hill, MA (GJK).

Correspondence and reprints: Joan S. Ash, PhD, Division of Medical Informatics and Outcomes Research, School of Medicine, Oregon Health & Science University, 3181 SW Sam Jackson Park Rd., Portland, OR 97201-3098; e-mail: ash/at/ohsu.edu.

Received August 15, 2002; Accepted December 3, 2002.

This article has been cited by other articles in PMC.

Abstract

In May of 2001, thirteen experts on computerized provider order entry (CPOE) from around the world gathered at a 2-day conference to develop a consensus statement on successful CPOE implementation. A qualitative research approach was used to generate and validate a list of categories and considerations to guide CPOE implementation.

Reasons cited for computerized provider order entry (CPOE) system adoption include medical error reduction,^1–³ and provision of just-in-time feedback to physicians,^4–⁶ but use of CPOE is not widespread.⁷Presumably, implementation lags because CPOE, by reputation, is hard to implement, expensive, and difficult to coax clinicians (and especially physicians) to use. At organizations with successful CPOE implementations, people instrumental in creating the success possess valuable experiential knowledge. As part of a three-year research grant funded by the National Library of Medicine, a two-day consensus conference enticed experts on CPOE implementation to share their expertise through discussions and generation of recommendations for CPOE adoption and usage to aid clinicians, vendors, hospital administrators, and information technology personnel in addressing the challenges that they will face. Computerized provider order entry was defined for the purpose of the conference as a process by which a clinician with order writing authority sits at a computer to directly enter patient care orders.

Methods

Goals of the Conference

The conference convened to develop a consensus set of recommendations for CPOE implementation. A consensus statement is a document, developed and agreed upon by representatives of multiple perspectives, that provides guidance for practice in specific areas.⁸An important aspect of consensus statement development is the focus on agreement and collaboration. Guidelines differ from consensus statements in that the former may be more prescriptive and use more elaborate branching and hierarchical structures. The process for developing consensus statements has been described well by Gamroth et al.⁸

A team of Oregon Health & Science University informatics investigators organized and supported the activities of the two-day conference and analyzed the data. To achieve the goal of generating recommendations, such conferences typically follow a commonly used format, with formal talks by experts followed by small group discussions on preselected topics.^9,¹⁰However, the research team organizing this conference was experienced in the use of qualitative methods, and did not want to restrict discussion to preconceived topics. The goal was to generate new insights as the experts shared their multiple perspectives on CPOE. Formal presentations were replaced with prior reading, and the large group was encouraged to determine the direction of the discussion. A facilitator managed the large group discussions. The research team designed an agenda that included a thinking task before participant arrival, brainstorming after arrival, narrative generation, prioritization of issues, and small team assignments.

Preconference Activities

Six months before the conference, potential participants were identified through literature searches, citation analysis, and recommendations from known experts. Categories of attendees included administrators, clinicians (physicians, nurses, pharmacists), social scientists, information technology implementers, and vendors (see Appendix for a list of participants and their affiliations). Conference organizers sought geographic diversity, representation of different system types (either commercial or locally developed), and varying implementation site characteristics (university, community, federal). Approximately half of those invited attended. Many attendees represented more than one of the stakeholder groups involved in CPOE implementation. Those who could not attend were asked if they would instead serve as an expert panel to review the final document.

Participants were asked to read a number of papers as background material to establish a shared knowledge base. The goals were to maximize on-site time for the sharing of expertise and experiences, to generate narratives, and to develop consensus statements.

Conference Activities

During the two-day conference, participants’ comments were captured on flip charts that were transcribed during the conference. Large group discussions were tape-recorded and later transcribed. Research team members were assigned to each of the small break-out groups to assist with taping and facilitation.

On the first conference day, the organizers described the purpose of the conference and introduced the experts and the research team members. A trained facilitator (who was neither a stakeholder nor, at that time, a central member of the research team) led subsequent activities. Participants were asked to share their perceptions of the “one most important thing,” or success factor, that they had discovered during their own CPOE implementations. Next, participants divided into four small teams, preassigned to promote diversity, which began discussion of the success factors outlined in the large group, prioritizing them and adding others with greater levels of detail. The four teams reported back to the large group. To further explore success factors and to share the lessons each expert had learned through experience, a narrative exercise was devised. The value of sharing lessons learned about CPOE is illustrated by Massaro’s description and analysis of the implementation at the University of Virginia.^11,¹²Each participant was asked to tell both a success story and a failure story. This was an enjoyable way of generating descriptions of additional success factors within an organizational and temporal context. Tales of stumbles, missteps, or pitfalls rather than outright failures were also allowed.

The experts then continued to work in small groups to develop lists of success factors based on the two exercises. For example, one of the failure stories highlighted inadequate levels of readily available support during a pilot implementation; thus, follow-up discussion centered on ideal levels of support. There were several iterations of large and small group work sessions. Thorny issues that provoked disagreement during any of these discussions were placed in a “bank.” The bank served as a way to put aside time-consuming and difficult issues and to deal with them later. Bank issues were discussed during an informal evening session and, after agreement, some were added to the list of considerations.

During the second day of the conference, the large group reviewed the work of the previous day and discussed and prioritized the main discussion points. Plans were made for disseminating the results and gaining subsequent feedback about them.

Postconference Activities

The data collected (transcriptions of all large and small group interactions) were analyzed with a grounded theory approach, using the words of informants to develop patterns and themes.¹³Instead of beginning the analysis with a predetermined list of topics, the researchers allow the data, in the form of words in transcripts, to guide code development. The process entails extracting major statements from all data sources and placing them on cards. The primary investigator led a multidisciplinary team of five through a 500 card sort exercise,¹⁴iteratively grouping similar ideas together. The sorting process created a taxonomy of high-level themes. A “themes document” was generated and shared with all participants, who engaged in three months of electronic discussion and changes. Experts who could not attend the conference were shown the resulting document and did not suggest any major content changes. The document seems to accurately reflect participating experts’ consensus on CPOE implementation.

Results

The Data

There were approximately 11.5 hours of tape to be transcribed.
Two hundred and forty-nine single-spaced pages of transcripts were generated.
Approximately 12 of the 13 participants and seven research team members submitted comments in response to the initial mailing, and each resulted in modifications. The major change suggested by the comments was that the considerations should be put into a question rather than a statement format. The purpose was to become less prescriptive.
Two of the nonparticipant experts offered suggestions that were factored into the final list.

The group realized that guidelines or recipes for success would be difficult to create because “CPOE” means different things and raises different concerns at different organizations—academic centers are different from community hospitals, and inpatient care is different from outpatient care. Cost reduction as an overarching goal raises different issues from the goal of patient safety. A desire for high levels of decision support raises different issues from a desire for a basic system. However, the experts also felt that despite such variations, certain themes were common across all CPOE projects and could be addressed. What the experts asked to have available as they first began thinking about CPOE was a menu of possibilities—a list of issues that they might not otherwise think about. Consequently, a list of “considerations” was targeted as the product of the conference rather than a specific guideline or recipe.

The qualitative analysis began with the identification of approximately 500 major statements and ended with ten themes (subsequently condensed) that represent a list of the overarching issues that the experts believe must be considered before CPOE implementation. An overview of the considerations is presented here. Although the complete list included 144 subconsiderations with comments and amplification, the following list offers those derived from them by the authors of this paper. Please visit <www.cpoe.org> for the complete report.

The Considerations

Consideration 1: Motivation for Implementation

The motivation for implementing CPOE influences where funding will come from, who will provide political support, and who will provide clinical leadership. Implementers should consider the following issues:

Whether local or national authorities may require CPOE at a future date
If administrators and/or clinicians are pressing for CPOE adoption
What stated objectives are for CPOE implementation (e.g., “improve efficiency”)
If outsiders/external conditions are forcing CPOE adoption (e.g., competitors are doing it)

Consideration 2: CPOE Vision, Leadership, and Personnel

Successful CPOE implementations require effective leadership over extended time periods—in different forms and at multiple levels in the organization. Leadership is needed at the executive level to promote a shared vision and provide funding at the clinical level to ensure champions and buy-in and at the project management level to make practical, effective, and useful decisions. Before embarking on the serious undertaking of CPOE, organizations should determine whether the following conditions are met:

Top-level leadership commits unwaveringly and visibly to CPOE.
A shared vision exists in the organization regarding the purpose of CPOE (e.g., to improve patient care) as well as a common understanding of why the current state is suboptimal and change is needed.
A single, clearly identified CPOE project leader exists with realistic attitudes about what can/cannot be accomplished, knowledge of how to educate administrators, and skills to foster teamwork.
There are clearly stated anticipated benefits that staff can embrace.
A realistic overall understanding exists of the efforts required to implement CPOE, coupled with the ability to communicate the vision and articulate tangible objectives to all levels of the organization.
The institution has identified and enrolled the support of physician leaders and clinical champions, respected by their peers, who can communicate the shared vision.
The organization has adequate finances, technical infrastructure, project management expertise, and staff readiness for CPOE—coupled with real and visible commitment of the chief executive and financial officers.
Clinical staff will trust and support the administration through this difficult effort, and, conversely, the leadership will value, have faith in, and depend on the individual clinicians in the organization who will use CPOE.
An organizational culture exists, or can be created, that values constructive feedback, changes made for quality improvement, and continuous learning—kept in balance by leadership that can tell the difference between clinicians’ requests for “what would be nice” versus “what is essential or critical for success.”
The compelling enthusiasm and urgency for CPOE can sustain the organization through the hurdles of implementation—i.e., leaders will maintain momentum through communication of appropriate urgency.
The stability and product quality of the vendor are at least good, if not excellent.
There is deep understanding that CPOE projects are a vendor “marriage,” not a purchase.

Consideration 3: Costs

Financial considerations are of critical importance. Often, costs are underestimated because purchase of the software is only the beginning of financial outlays; other expenditures such as person-hours for training and support are harder to predict. Decision makers need to consider the following issues:

Whether the total cost of ownership has been considered rather than simply the cost of technology
Whether the organization can afford the attendant temporary productivity losses that accompany CPOE implementation
Whether funds have been dedicated solely for CPOE, with the ability to commit additional funds quickly for good (unanticipated) cause

Consideration 4: Integration: Workflow, Health Care Processes

The manner in which a CPOE application alters and integrates into existing environments and workflows is critical to its success. Users resent disruption of their patient care activities; thus, implementers must consider the following issues:

Whether the impact of CPOE on the work processes of physicians, nurses, pharmacists, ward clerks, laboratory personnel, registration personnel, and other hospital staff was carefully considered and will be closely followed during implementation and afterward
Whether an organization-wide change management strategy exists and has been tested under similar stresses previously
Whether CPOE will be used for all orders or simply some categories of orders
How users will view orders during construction, after entry, and after completion
How new, potentially life-saving orders will be communicated reliably to nurses or others who need to be aware of them
How CPOE integrates with other hospital applications such as the laboratory system, pharmacy system, ADT/registration system, and other clinical systems via interface engines and/or messaging protocols
Whether the impact of CPOE on human communication among key employees such as physicians, pharmacists, nurses, and lab technicians has been worked out for both regular use and during CPOE downtime

Consideration 5: Value to Users/Decision Support Systems

Constituencies affected by CPOE implementation (e.g. physicians, nurses, ancillary department personnel) must understand the CPOE implementation “value proposition,” i.e., they must do things differently but there will be some benefit in return. One benefit for clinicians is embedded CPOE decision support logic that helps to improve patient care quality and/or to reduce costs. Related issues include the following:

There must be a plan for the ongoing management of clinical CPOE system content, including decision support.
Users must participate in development of decision support and other “benefits” that affect them, and be adequately trained in their usage.
Users must understand where the CPOE system provides “help” and where it may not, and CPOE behavior must be consistent (e.g., for drug-drug interaction, drug-allergy interaction, duplicate medications, duplicate labs, expensive tests, suggested drug level monitoring).
Order sets—groups of orders to manage a disease state or for a procedure (prebronchoscopy, postbronchoscopy)—must be developed, reviewed, and maintained for personal and/or departmental usage.
Surveillance must be in place to determine that benefits for clinicians and for patients are both easy to see and to describe.
Users must be trained before implementation and on an ongoing basis thereafter as CPOE systems evolve.
Clinical users must be shown that CPOE usage is not clerical work, emphasizing what cannot be done via manual, paper systems.

Consideration 6: Project Management and Staging of Implementation

Project management dictates that implementation be completed in carefully planned stages. Key considerations include the following:

During all stages, “people issues” must have highest priority—keep employees (clinicians) informed, engaged, and content, through planning and communication.
Early in the project, the scope must be defined, with clear, reasonable, measurable goals.
Early milestones must be selected to produce “wins” that help maintain momentum toward more difficult long-term objectives.
Plans should be detailed enough but not overly so
Multiple mechanisms for collecting feedback from users and staff must be in place, and analyzed in real time for appropriate action.
The golden rule should be applied by all involved (do unto others as you would have them do unto you), and leaders should work to develop consensus when disagreements arise (keeping in mind that various ways of doing things may all lead to success).
Use of consultants should be carefully planned with specific objectives before they are employed (if at all).
A critical mass of users must be ready for the implementation.
A plan for involving clinicians must be developed, followed, and evolved.
Metrics for success should be determined beforehand and evaluated over time.
Accountability for objectives, large and small, must be established and maintained, as each new concern arises.
During the pre-implementation phase, the organization should develop a vision, locate funding, identify people who will lead the implementation, solicit involvement from key people, and exhibit strategic and tactical planning skills.
During the implementation phase, the organization should hire staff, deploy staff where and when most needed, keep up staff morale, and use communication, publicity, and personnel management skills effectively to maintain project momentum.
After implementation, the organization should establish maintenance routines, create an environment for ongoing system improvement, and provide management systems for the long term.

Consideration 7: Technology

Technical details to consider as part of a CPOE implementation include strategic considerations, user considerations, task completion flexibility, and the quality of the application—from customizability to user friendliness. Questions related to the system itself include:

Whether there is a plan to authorize all users who need access to the system (e.g., attending and house staff physicians, nurses, medical assistants and unit secretaries)
Whether the system is sufficiently customizable for the organization’s needs, including the ability to provide decision support where needed
Whether the balance between customization and standardization has been considered
Whether individual users can customize some things themselves
If the system can be modified on site
What special considerations have been made for replacing older systems
Whether assurance of high-level data quality is possible and has been implemented
Whether the CPOE system can interface with existing and planned future systems
If a risk analysis of the project has been conducted, with specific attention given to addressing the risks
If there is a need for remote access
How great a burden system use places on end-users from the users’ point of view
Whether the response time is good enough for the users (one expert cited 0.7 seconds as too slow)
If there are escape routes, such as entering free text, for frustrated users
If details of the user interface (UI) have been scrutinized, focusing on aspects of the UI that are most likely to give the users difficulty

Consideration 8: Training and Support 24 × 7

One of the constant themes identified by the experts at the retreat was the importance of live help available “at the elbow” at the time of implementation. In addition to the symbolic importance of supporting the users by being present while they are first using the application, intensive support at “go-live” time allows the implementation team to have direct experience with what is and what is not working well. Most successful implementations have had more post-go-live support than pre-go-live training. Most sites have had 24 × 7 support for at least several days post go-live. Considerations include

Whether there is a training plan for the support staff
If support staff are able to act as translators between clinicians and information technology staff
Whether provisions have been made for online help as well as direct assistance by support staff
If users will train and mentor other users (and with what methods)

Consideration 9: Learning/Evaluation/Improvement

CPOE implementation is an ongoing effort that benefits from continuous improvement. It is important that mechanisms for feedback and modification of the system be in place. Questions to consider include

How the organization can learn from its mistakes
Whether there is a process for responding to problems in a timely manner
How problems will be addressed in a timely manner
How the system will be “test piloted” without putting patients at risk
What the plan is for formal feedback and evaluation
How the system will be improved upon continuously
Whether a plan exists to revisit decisions on a regular basis

Discussion

The group used a qualitative approach to create a consensus statement on the specific issues that organizations contemplating a CPOE implementation face. Each consideration should be reviewed by the leadership and implementation team of any organization considering CPOE installation. Some issues will be more easily addressed than others; some will be more relevant to one particular organization than others; and, some are more applicable at different stages in the implementation than others. Some of these questions and issues will have clear and obvious answers, but most will not and will require effort to address. Organizational representatives should focus on the difficult-to-answer questions rather than avoiding them. All of the detailed considerations listed are relevant to a successful implementation.

The qualitative approach allowed us to generate different results than might have been realized with a more prescriptive approach to consensus creation. Of the major categories of considerations, only one grouping was strictly technical. It is possible that the discussion would have centered more on technology if the inductive approach had not been taken. The importance of strong executive leadership at the highest levels in the organization in a CPOE initiative should not be underestimated. Leadership is a thread running through many of the major considerations. Administrative leaders, acting on behalf of the organization, must believe viscerally that CPOE is in the best interest of the institution and be able to communicate that feeling throughout the organization.

Clinical leadership must also be committed to CPOE and communicate this commitment to the clinical staff, who will typically be less than excited about the prospect of CPOE (due to natural resistance to change). Clinical staff will have real concerns that the time to complete work will increase. Arguments about improved safety may appear to end-users as vague and intangible. Clinical leaders must work strenuously to communicate (and physically demonstrate) to their staffs how CPOE provides opportunities for improved quality and efficiency. Administrative and clinical leaders must work together to create a strong sense of “common will” to overcome obstacles that will be encountered during a CPOE implementation.

The expert panel has continued its dialogue about CPOE. It gathered informally during the American Medical Informatics Association 2001 and 2002 Annual Symposia to plan further endeavors. Ongoing efforts will develop suggestions to help organizations find answers to the questions listed under each consideration. There is agreement that future research needs to be done to develop valid tools to measure readiness for CPOE, the effectiveness of the process during implementation, and outcomes during and after CPOE implementation.

The list of considerations presented in this paper and the full text are available at <www.cpoe.org>. This is meant to serve as a guide for organizations to help them make appropriate decisions regarding CPOE. The group agreed that implementation of CPOE is difficult and it must be approached with awareness of the potential problems. The combined wisdom of those who have already experienced successful implementation efforts, summarized in the list of considerations, can serve as a resource for those contemplating future implementations. Health care institutions are being pressured by the Leapfrog Group³ and others to rapidly adopt CPOE. To increase chances of success, leaders are urged to look carefully at the Considerations before they “leap” into CPOE projects.

Appendix

Consensus conference participants, their current affiliations, and their roles: Jos Aarts (Erasmus University, The Netherlands, social science and international perspective), Marilyn Davis (El Camino Hospital, Mountain View, CA, nurse and implementation specialist at the first POE site), Dick Gibson (Providence Health Systems, Portland, OR, physician and clinician leader), Homer Chin (Kaiser Permanente Northwest, Portland, OR, physician and clinician leader), Paul Nichol (Puget Sound Veterans Administration, Seattle, WA, physician and clinician leader), Marc Overhage (Regenstrief Institute, Indianapolis, IN, physician and clinician leader), Tom Payne (University of Washington, Seattle, WA, physician and clinician leader), Karen Hughart (Vanderbilt University, Nashville, TN, nurse and implementation specialist), Janet Greenman (IDX, Seattle, WA, vendor representative), John Dulcey, MD (Lansdale, PA, physician and clinical systems consultant), Gil Kuperman (Partners Healthcare System, Chestnut Hill, MA, director of research and development), Brian Churchill (Peacehealth, Eugene, OR, nurse and project manager for POE implementation), and Jim Carpenter (Legacy Healthcare, Portland, OR, clinical pharmacist).

Research team members included Joan Ash, P. Zoë Stavri, Paul Gorman, William Hersh, Mary Lavelle, Lara Fournier, and Jason Lyman.

Views expressed by participants are their own and not necessarily those of agencies or organizations with which they are affiliated.

Notes

This work was supported by grant LM06942-02 from the National Library of Medicine. Special thanks to Paul Tang, MD, and William Bria, MD, for their comments about the Considerations and to Lara Fournier, MS, for assistance with the manuscript.

References

1. Hurtando MP, Swift EK, Corrigan JM (eds). Crossing the Quality Chasm: A New Health System for the 21st Century. Institute of Medicine, Committee on the National Quality Report on Health Care Delivery. Washington, DC: National Academy Press, 2001.

2. Sittig DF, Stead WW. Computer-based physician order entry: The state of the art. J Am Med Inform Assoc. 1994;1:108–23. [PMC free article] [PubMed]

3. The Leapfroggroup for Patient Safety. Computer physician order entry (CPOE) factsheet. November 2000. Available at <www.leapfroggroup.org>.

4. Kaushal R, Bates DW. Computerized physician order entry (CPOE) with clinical decision support systems (CDSSs). Chapter 6 in Making Health Care Safer: A Critical Analysis of Patient Safety Practices, Evidence Report/Technology Assessment no. 43, Agency for Health Care Quality and Research, 2001. Available at <www.ahrq.gov/clinic/ptsafety/chap6.htm>.

5. Shea S, DuMouchel W, Bahamonde L. A meta-analysis of 16 randomized controlled trials to evaluate computer-based clinical reminder systems for preventive care in the ambulatory setting. J Am Med Inform Assoc 1996;3:399–409. [PMC free article] [PubMed]

6. Davenport TH, Glaser J. Just-in time delivery comes to knowledge management. Harv Bus Rev 2002;80:107–111. [PubMed]

7. Ash JS, Gorman PN, Hersh WR. Physician order entry in U.S. hospitals. J Am Med Inform Assoc Symposium Suppl 1998 ;235–239. [PMC free article] [PubMed]

8. Gamroth L, Semradek J, et al. (eds). Enhancing Autonomy in Long-Term Care. New York, Springer, 1995.

9. Yasnoff WA, Overhage JM, Humphreys BL, LaVenture M. A national agenda for public health informatics: Summarized recommendations from the 2001 AMIA Spring Congress. J Am Med Inform Assoc 2001;8:535–545. [PMC free article] [PubMed]

10. Stroup DF, et al. Meta-analysis of observational studies in epidemiology: A proposal for reporting. JAMA 2000;283:2008– 2012. [PubMed]

11. Massaro TA. Introducing physician order entry at a major academic medical center. I: Impact on organizational culture and behavior. Acad Med 1993;68:20–5. [PubMed]

12. Massaro TA. Introducing physician order entry at a major academic medical center: II. Impact on medical education. Acad Med 1993;68:25–30. [PubMed]

13. Glaser BG, Strauss AL. The Discovery of Grounded Theory: Strategies for Qualitative Research. Chicago, Aldine, 1967.

14. Lincoln YS, Guba EG. Naturalistic Inquiry. Newbury Park, CA, Sage, 1985, pp 346–347.

Articles from Journal of the American Medical Informatics Association : JAMIA are provided here courtesy of
American Medical Informatics Association