A Cross-site Qualitative Study of Physician Order Entry

doi:10.1197/jamia.M770

J Am Med Inform Assoc. 2003 Mar-Apr; 10(2): 188–200.

PMCID: PMC150372

A Cross-site Qualitative Study of Physician Order Entry

Joan S. Ash, PhD, Paul N. Gorman, MD, Mary Lavelle, MSN, Thomas H. Payne, MD, Thomas A. Massaro, MD, PhD, Gerri L. Frantz, MBA, and Jason A. Lyman, MD

Affiliations of the authors: Oregon Health & Science University, Portland, Oregon (JSA, PNG, ML); University of Washington, Seattle, Washington (THP); University of Virginia, Charlottesville, Virginia (TAM, GLF, JAL).

Correspondence and reprints: Joan S. Ash, PhD, Division of Medical Informatics and Outcomes Research, School of Medicine, Oregon Health & Science University, 3181 SW Sam Jackson Park Rd., Portland, OR 97201-3098; e-mail: <ash/at/ohsu.edu>.

Received February 9, 2000; Accepted September 9, 2002.

This article has been cited by other articles in PMC.

Abstract

Objective: To describe the perceptions of diverse professionals involved in computerized physician order entry (POE) at sites where POE has been successfully implemented and to identify differences between teaching and nonteaching hospitals.

Design: A multidisciplinary team used observation, focus groups, and interviews with clinical, administrative, and information technology staff to gather data at three sites. Field notes and transcripts were coded using an inductive approach to identify patterns and themes in the data.

Measurements: Patterns and themes concerning perceptions of POE were identified.

Results: Four high-level themes were identified: (1) organizational issues such as collaboration, pride, culture, power, politics, and control; (2) clinical and professional issues involving adaptation to local practices, preferences, and policies; (3) technical/implementation issues, including usability, time, training and support; and (4) issues related to the organization of information and knowledge, such as system rigidity and integration. Relevant differences between teaching and nonteaching hospitals include extent of collaboration, staff longevity, and organizational missions.

Conclusion: An organizational culture characterized by collaboration and trust and an ongoing process that includes active clinician engagement in adaptation of the technology were important elements in successful implementation of physician order entry at the institutions that we studied.

Computerized physician order entry (POE) is defined as a process that allows a physician to use a computer to enter medical orders directly. The concept is receiving an increasing level of attention because the Institute of Medicine report “To Err is Human” notes that POE holds potential for decreasing the number of medical errors in hospitals.¹ This is because the system can offer decision support at the time it is most needed. Despite its touted benefits, however, POE is not widely used. In a 1997 mail survey of hospitals, we found that POE reportedly exists in one-third of U.S. hospitals, but is really used in less than two percent of them.² Comments from the survey respondents indicated that administrators are looking forward to POE implementation but fear that physicians will resist it. Historically, physicians have been reluctant users.^3–⁶ We designed a study not only to find out why there is resistance but also to investigate the complex interplay of factors that influence the success of POE implementation.

A description of the process in the context of a teaching hospital illustrates why POE is an intriguing implementation issue:

An intern attends rounds early in the morning with her team of residents and an attending physician. The group discusses each patient either at the bedside or just outside the room. During the course of the discussion, suggestions are made about what tests and medications to order for the patient. The intern writes some notes as a reminder about what to do later. At the conclusion of rounds, the intern is expected to enter orders into the computer for most patients. She seeks an unused machine, logs on and locates the first patient in the system. She may order labs and then begin to order medications. To order a medication, she first needs to find its name in an alphabetical list and select a dosage from a menu and then a schedule for administration from another menu. If the dosage or schedule desired is different from normal in some way, she may need to type in exact instructions and be creative about abbreviating words because the space for free text may be limited. She sends the order for the first medication and, to order a second, goes back to the alphabetical list and starts again. If she orders something that might interact with another substance, she receives an alert when she asks the machine to send the order. When the intern is finished with the orders for the first patient, she pulls up the record on the second patient and starts again. She is pleased that the medication is received on the floor within an hour. Because the data are entered in a structured manner, and because they enter a large database, an accurate record of that order now exists for billing and other tracking purposes.

Without the computer, the physician would write a list of orders in longhand or check boxes on a form, one list per patient, and a ward clerk or nurse would take over the process after that. The clerk may need to clarify the order, especially if the writing is hard to read. It may take the intern ten minutes per patient to write the orders and answer questions. Messengers or a pneumatic tube may be used to deliver the order to the pharmacy. A drawback of the manual system is that it may take six hours before the medication reaches the floor. However, the physician may spend less than a half hour writing orders for all patients. Using POE may take much longer or be perceived as taking much longer.

The purpose of this study is to describe perceptions of POE held by diverse professionals at both teaching and nonteaching sites where POE has been successfully implemented. The professional groups include clinicians, administrators, and information technology personnel. Success is defined as heavy use (over 80% of orders are entered electronically) by a large number of physician users. The reason for studying successful sites is that they can serve as models. The focus of this study is on commercial systems that have the potential for being widely adopted. The present study is designed as a cross-site study so that the perceptions of diverse professionals can be compared both within organizations and among different settings. The field investigators (JSA, PNG, ML, JAL) were an external, multidisciplinary team unaffiliated with any of the selected sites at the time of the fieldwork.

Methods

Selection of Sites

We contacted experts in informatics and asked for suggestions of sites where POE is used with success. For this initial study, three organizations were selected that span a spectrum: an academic institution on the East Coast with a ten-year history of using POE; a West Coast VA hospital with a recent installation and two campuses; and a nonteaching hospital on the West Coast with the longest history of POE use in the country. The first institution was the University of Virginia in Charlottesville, Virginia, which has used the Technicon/Eclypsis system (Eclypsis Corp., Delray Beach, FL) for POE since 1989, when it experienced a difficult implementation.⁷ It is a state institution with a 591-bed hospital and a network of ambulatory service facilities. The Veterans Affairs (VA) Puget Sound Health Care System campuses in Seattle and American Lake were the second site. VA Puget Sound was a test site for the VA’s Computerized Patient Record System (CPRS), which allows note entry and results reporting in addition to order entry. Use of the system on the busiest inpatient wards and critical care units began in December, 1998.⁸ The campuses in Seattle and American Lake are 40 miles apart and together have a total of 347 acute beds and 142 nursing home beds. The Seattle Division is one of five teaching hospitals of the University of Washington. El Camino Hospital in Mountain View, California, was the third site. Most of the literature about POE in the nonteaching hospital setting has El Camino as its focus, but nothing has been published recently. Beginning in 1966, El Camino became a development site for a medical information system originally developed by Lockheed and purchased by Technicon (now Eclypsis).⁹ It was selected by the National Center for Health Services Research in 1971 as a demonstration site of a total hospital information system.¹⁰ A careful evaluation of the system was conducted by researchers from Battelle Columbus Laboratories, with the conclusion that the system was cost-effective.¹¹ The hospital prides itself on being “a true community hospital” and does not have medical residency programs.

Data Collection Methods

Observation, oral history interviews, and focus groups were used because each method has its unique strengths. Participant observation has the advantage of being the most unobtrusive method. It “produces detailed descriptive accounts of what [is] going on (including verbal interaction)”¹² and has been used effectively in prior informatics studies.^13,¹⁴ Experienced qualitative researchers accompanied clinicians in the course of typical daily tasks in the hospital and generally followed such shadowing with informal interviews. Additional observation included watching all activities on certain hospital units. Data were also collected using focus groups. Focus groups provide an efficient way to gather information from a group with the benefit of synergy among participants.¹⁵ Finally, taped oral history interviews provided the longitudinal perspective because informants described past events that occurred during the implementation process.¹⁶ Together, the three methods produce a high level of “trustworthiness,” the qualitative analogue to “validity.” The use of multiple researchers, feedback to and from the informants, and diverse sites contributes even further to trustworthiness.¹⁷

The Data

At the University of Virginia, we shadowed intern-resident pairs in the critical care unit. In labor and delivery, we followed a surgical intern and a medical intern. Because we visited in August, the interns had experienced only one month of POE. At the VA in November, we observed an attending physician in an outpatient primary care clinic, an attending physician in a nursing home unit, and a medical student/intern/resident team on a general medicine inpatient unit. Informal interviews were held with each of these eleven participants plus four nurses and a clinical pharmacist at each hospital. Formal oral history interviews were held with nine people: the chief clinical information officer at each place and a clinical systems specialist with a nursing background, two administrators who are MDs, two faculty who used the system as residents, and two other faculty with MDs at the University of Virginia. Three focus groups were held, two in Virginia and one in Seattle. To maximize participation, we fit the focus groups into the work schedule of the house officers, once at a meeting of about ten pediatrics residents, a second at a meeting of about twenty general medicine residents, and a third at a meeting of a ward team, including a medical student, an intern, and three residents.

At the American Lake Division of the VA Puget Sound Healthcare System, we shadowed a psychiatrist and held informal interviews with a physician’s assistant and nurse. At El Camino, we shadowed physicians in the emergency department, medicine, oncology, the ICU, and CCU. We conducted four formal oral history interviews with a physician, two nurses who practiced when the system was implemented and subsequently worked for the information technology unit, and a person who had helped with the original cost evaluation of the system, then gave demonstrations of the system to visitors from around the world and continues on the financial staff. Informal interviews were held with nurses, administrators, a care coordinator, clinical pharmacists, and a dietician. Table 1

summarizes the method and subjects at each site.

Table 1

Methods and Participant Groups

Data Analysis

The investigators transcribed their own field notes from their handwritten notes. Transcripts of interview and focus group sessions were produced from audiotapes by a transcriptionist with qualitative methods experience. A total of 120 person-hours of observation were conducted, resulting in about 180 single-spaced pages of transcribed data. Twenty-two hours of formal interviews produced 220 pages of double-spaced pages of transcription. A grounded theory approach was used to identify emergent themes. Grounded means “based on and connected to the context-dependent observations and perceptions of the social scene;”¹⁸ the informants’ own words guided development of codes. Two researchers (JSA and ML) independently reviewed the field notes and transcripts, annotating important themes in the margins. Two other researchers (PNG and JAL) carefully reviewed the documentation and noted major themes. The team of four met three times to reach consensus on naming the patterns and themes. Qualitative data analysis software (QSR NUD*IST 4, Sage Publications, Berkeley, CA) was used to review and index these patterns and themes.

Results

Four major themes (Table 2

) emerged from the data:

Table 2

Themes

Organizational issues, including collaboration, organizational and personal pride, organizational culture, and power, politics, and control
Clinical/professional issues
Technical/information technology implementation issues
Issues related to the organization of information.

In addition, some differences between teaching and nonteaching hospitals arose.

The themes described below are patterns that were seen across sites. Quotations have been selected because they are representative and the individual speaking articulated a particular point well. They are quoted verbatim so that the reader can experience the richness of the spoken words. Because participants were promised confidentiality, an effort has been made to report what was said without divulging any information that would let the reader identify the speaker.

Limitations of the study include the number of hours spent observing, which was sufficient in that saturation was reached but perhaps not ideal (according to the standards of traditional cultural anthropologists who spend years in the field). The small number of formal interviews at the VA, the timing of the VA visit (just before a large-scale implementation), and the fact that two sites were using the same basic TDS system (although extensive modifications have been made in each) may be considered limitations. In the tradition of qualitative research, the results may be transferable but are not necessarily generalizable.

Theme 1: Organizational Issues

Organizational issues involve the people within organizations, the relationships among individuals or groups, the culture or shared system of meaning, and the context within which organizations exist.

Organizational Issues: Collaboration

“The effect of [the system] in the whole hospital has caused everybody to become more inter-dependent.”
“It used to be the department could do whatever it wanted and it didn’t have to impact anybody else.”

After initial resistance at the University of Virginia, house officers began sitting at the table with administrators and information technology staff to solve problems related to the system. Such committees are useful, but members need to be heard and want to be useful. One informant lauded a “national group and hierarchy of making suggestions.” Another “enjoyed meeting colleagues at the national level” and had a sense of loss when a development group was discontinued.

Communication among physicians and nurses has changed as a result of POE. From field notes:

“I ask if he feels that the system has changed the way he communicates with nurses. He said, ‘absolutely,’ saying that he felt their interactions were much friendlier because there were less calls from the nurses re: handwriting, e.g. and less going back to the nurses and saying something like ‘how come you didn’t put these orders in yet?’”
“Instead of an order book where you write down your inpatient orders and put it down and the nurse looks it over, now that’s done sometimes from another physical location . . . there are fewer circumstances where there’s a conversation about illegibility.”

Legibility was cited again and again as a positive aspect of POE at all sites.

Organizational Issues: Organizational and Personal Pride

All organizations in the study opted for implementing POE partly because they wanted to be on the cutting edge of technology. Financial efficiency was a goal, but the attractiveness of being viewed as a future-oriented, proactive, technologically advanced organization was also important. One informant said that “part of this was an ambitious project from the university and the hospital side to try to put X at the forefront of information technology in use,” and another at a different institution said about the leadership, “He, from a conceptual point of view, was trying to get the institution at the forefront.” There is now a good deal of pride in having weathered the implementation process, especially in those who helped to modify it. One said proudly, “I have lash marks on my back from the last implementation.” Information technology staff take pride in POE as well: “We had meetings all the time with caregivers and no matter what the concern was, we tried to work on it . . . everybody wanted it to succeed . . . it was a showcase, the hospital was a showcase.”

There is also a sense of personal pride in having mastered the POE system, especially among interns. Residents were proud that they could develop personal order sets: “The ownership became an issue and so now you can construct, construct your own personal order set. And that became yours and you could mold it however you wanted and it would be, you know, your hallmark, whatever.”

Organizational Issues: The Culture

Leadership and top-level support were often cited as success factors:

“What kept us going here during the tough times is the fact that our administration. . . said ‘you know, guys, do whatever you can’ [to I.T.], they realized the importance of keeping it going, ‘just tell how much it’s gonna cost, and do what you have to do.’”
“He was here over 25 years and politically astute and more of a team builder and got people working together.”

The context within which the hospitals operate is also a factor in success. Outside pressures such as managed care and JCAHO influence hospital policy. There are also unique pressures on individual hospitals: the University of Virginia is a state university; the VA sites are part of a federal system; and El Camino Hospital is community-based. The era of the implementation makes a difference as well. In the early 1970s, when the El Camino system was implemented, the pace of change was different, but the technology was new so that different issues surfaced. When Lockheed staff members were explaining to El Camino staff what the new system would be like, “They said we’re going to install this computer, and this is what it’s going to look like, and they had a cardboard box with a hole cut in it, and they had a scroll and they were scrolling butcher paper through and saying this is what it’s going to look like and we were just laughing our heads off.”

Organizational Issues: Power, Politics, and Control Issues

The implementation of physician order entry involves power relationships, with power defined as “a capacity that A has to influence the behavior of B so that B does things he or she would not otherwise do.”¹⁹ In the context of POE, this means that someone has the ability to influence a physician to directly enter orders into a computerized system. Politics is the attempt to influence “the distribution of advantages and disadvantages within an organization.”²⁰ Politics is therefore power in action; it can be an effective force for changing behavior.

Both formal and informal power was evident at all sites. Interestingly, implementation of POE caused shifts in power distributions. As one participant observed, “It changed the balance of power because nurses learned the system and became the ones who taught the doctors the system and that was an interesting dynamic . . . the nurses felt more important and the doctors felt more respectful of nurses, in general.” In a sense, however, nurses seemed to lose some power in that they had less informal input into physician decision making as a result of off-site entry of orders. When nurses overtly gained greater power at one place by generating their own order sets, physicians quickly intervened. Interns felt an increased sense of power and ownership of patients. Because they enter the orders, they sense they have a certain degree of control over the care of the patient: “Part of it’s a control issue, if you don’t do it yourself, you don’t actually know what’s being done to your patient.”

Although power and influence from the top are necessary, mid-level power exerted by department chairs at teaching hospitals may be even more important. Informal political skills exhibited by champions (peers who favor the innovation) and opinion leaders (who may or may not favor it but nevertheless hold influence) were evident. “You need some people to be the champions, people who will ride the white horse through the units and say this is good and don’t be afraid of it.”

Information technology staff are acutely aware of the power of positive politics: “You have to be very political, and you have to be patient and you have to just bend over backwards to give doctors what they want and value.”

A sense of power also comes from being heard. “They listened to nursing carefully in developing the system . . . thirty-two of the care plans were put in at his [the nurse’s] request.” Participation, not buy-in, is needed. Clinicians want to be part of the problem-solving process. Developers at El Camino took the idea of a partnership with clinicians especially seriously, perhaps because they were designing a system and breaking new ground.

When the system is created for someone else’s purposes, for administration in particular, rather than for clinical reasons, it can be greatly resented. A recipe for failure seems to be the imposition on clinicians of a system that will help the hospital but not help them: “Everybody became an agent of the administration by default if they were hired through the hospital and if they were on the receiving end of an order. And there was a lot of tension created.” At the University of Virginia, in the first days of implementation, house staff “felt it was compromising patient care, which is something that is important to them, whereas the hospital’s bottom line is not.” The following is from field notes: “He felt like it was a way for the administration to save money—and that he had to work more because of it. He said it ‘helps other people, not the doctors.’” Factions were evident throughout the histories of implementation. The prime ones were physicians and administration. Feelings towards IT were more often kind, perhaps because training and support were emphasized at all sites and seemed to be appreciated.

Theme 2: Clinical/Professional Issues

There are several dimensions to this issue, including both customization of the system and the ability to do individual customization.

Clinical/Professional Issues: Local Customization

Different units in the hospital are quite unique. One may assume that medical specialties would differ from surgical, but there are also large differences between the emergency department and the rest of the hospital, labor and delivery, mental health, nursing home care, and pediatrics. In addition, there are differences in shifts and system needs during evening and weekend hours.

The differences among units in the hospital were pointed out by nearly everyone at all sites. At the University of Virginia and El Camino, physician order entry is more accepted by surgical than medical specialties. A surgical intern can routinely use order sets, which makes the job easier and perhaps even faster than a manual system. However, the medical specialties are less structured and orders are less routine. There are numerous examples in our field notes of problems with system use when a nonstandard order is needed. The rigidity of the system can cause great frustration in a tense situation.

“The place where we had the most trouble was our medicine service and they have been very cautious, gun shy.”
“I see it in the order entry application in the ICU, [it] is an entirely different animal than one on a ward or [in the] spinal cord [unit]. Even though they have their instances of acute medical catastrophes, they don’t happen predictably day after day.”
“It’s a problem with the dosing in pediatrics.”
“In the ER, patients came and went very quickly . . . so we were writing things on paper towels and stapling paper towels to the chart because that was our medication record.”

Shifts vary considerably. An information-intensive period is during the change of nursing shifts. Nurses must pass on information to the person replacing them in an efficient manner. Each nurse has a preferred way of organizing information on each patient. Space is crowded, and computers tend to be in great demand. In the morning, this may occur at the same time physicians are rounding; bottlenecks occur when computers are in short supply.

Clinical/Professional Issues: Individual Customization

There seemed to be varying opinions about age and ease of acceptance of POE. Other factors may be more important.

“Some people absolutely love it and they want all the bells and whistles . . . age doesn’t matter . . . once you learn something can do something for you, you’ll use it over and over and if you never hear about it, you don’t use it.”
“Computer literacy does matter. The younger physicians catch on much faster and are ready to use it.”
“They like efficiency, and legibility is important to them if it prevents them from getting a second phone call asking what does this mean?”
Customization of training is important for the comfort of users. At El Camino, when a new physician joins the staff, he or she is contacted by a nurse trainer working in information technology for personalized training. Nurses there are trained by nursing educators. The University of Virginia and the VA offer classes. Virginia trains new physicians primarily in the summer when interns and residents arrive on a yearly basis. The Seattle VA has house officers rotating through every month. In addition, customized training is provided at the teaching hospitals by more experienced residents with whom you are partnered. What you learn, then, is dependent on what those residents learned along the way.

Theme 3: Technical/Information Technology Implementation Issues

Implementation is a process of continuously modifying the system. The University of Virginia had to make large-scale changes to make the system acceptable to house staff. They worked hard to anticipate problems but simply could not predict the amount of resistance. In retrospect, the major players point with pride at the hard work and eventual success of the system that they helped to change. At the VA Puget Sound, a new system was being implemented shortly after we were there, but since it was an alternative interface for the VISTA system already in use, one could call it an enhancement. The El Camino system continues to evolve.

Technical/Implementation Issues: Usability

Usability is a systems design concept. In this context, it means how the system fits the workflow. The system must be enabling without constraining. Personal and departmental order sets and clinical pathways were cited numerous times by informants as making their lives easier.

Certain aspects about the systems in the three settings delighted users. Most importantly, they appreciate being able to access the system without being on the unit with the patient at that moment. For the most part, logging on was relatively fast and easy. The observers were surprised that system users at some sites accepted the text-based systems that used all capital letters. With practice, users had become amazingly adept at reading the screens, even though to novice eyes it appeared that screen layouts were jumbled and hard to read.

The threats to usability were based primarily on the number of screens that one has to access to get the job done. One informant said that you have to “think like a computer” to place an order. Tricking the system by using workarounds is common. Workarounds, defined as clever methods for getting done what the system does not let you do easily, are used by clinicians at all three sites. From field notes:

“Notes to a patient in the discharge summary have to be typed using a workaround because the space is so limited. You have to go under ‘diet’ because that part offers free text capability. The poor nutritionist probably has to read all kinds of non-diet comments listed under diet.”
“You just have to use it a lot and sort of figure out little quirks and how to get around it.”
“There’s lots of times I wish I had a pen and I could write this order because I know what I want, and I just can’t find it on this computer.”

Technical/Implementation Issues: Order Sets

At the University of Virginia, it was not until personal order sets were implemented that the system became acceptable to clinicians. Order sets can be time savers for straightforward orders. Collections of orders can also be developed at a departmental level and therefore agreed on by a group of physicians. All sites had personal and departmental order sets. At El Camino, the personal order sets developed by individual specialists were used by internists who would be referring the patient to that specialist. The VA has something called “quick orders,” which are standard orders that are extremely easy and fast to enter. Order sets there are collections of quick orders.

Beyond personal and departmental order sets, El Camino also had over fifty clinical pathways. These were developed using an inclusive and intensive collaboration process among nurses, physicians, pharmacists, and others. They outlined the entire care plan for the patient with a specific problem. They are widely used and constantly updated. The nurses like them because if a physician puts someone on the care plan, he or she often gives greater license to nurses to use their judgement. Physicians need to be called less often. The pathways are agreed on by consensus, but physicians can modify any aspect of them. “Rather than having to write all the admission orders, you just click quickly on.”

Technical/Implementation Issues: POE Training and Support

All sites devoted large amounts of staff time to training and support before and during implementation of physician order entry. They all continue to provide support by phone and in person. At El Camino, physician training can be done on a one-on-one basis because turnover is low. At teaching hospitals, house staff are trained in groups. Because the VA hospital has students and house staff rotating on a short-term basis, training must be held frequently.

At the VA, appointed clinical application coordinators are on call to answer systems-related questions, in addition to information technology staff. At El Camino, nurses take over the role of just-in-time instructors. There is also a phone number to call. Physicians there asked nurses for help before anyone else, and often a nurse spontaneously volunteered to help. From field notes

“He finally said, in frustration, ‘I can’t do this’ and went to find a nurse.”
“The nurses are programmed to help the new doctors, if one is struggling, they go up and say can I help you out?”

Technical/Implementation Issues: Time

Speed of the order entry process is the most important issue from the clinician perspective. Response time is critical. At El Camino, we were told that when the response time increased to anything over 0.7 seconds, the help desk phone started ringing with complaints. Order sets are popular because they save time: “You can literally get somebody admitted and order all their labs and the medicines in, and it, it’s five minutes.”

It makes a great deal of difference whose time is impacted. Nurses are grateful that they no longer need to put in all the orders on behalf of the physicians. Their time is saved, but most users assume that POE takes more physician time than manual order entry. Nurses and pharmacists also save time by not having to call the physician to question an order they cannot read. When time of administration of a needed therapy is positively impacted by POE, clinicians are grateful: “One of the physicians used to tell a story about putting orders in [the system] and another doctor picking up a yellow sheet and writing orders and before the doctor was even through writing, the one entering in [the system] saw the nurse going down the hall to give the medication, because it had printed in pharmacy and they had sent it up before the other guy even got through with his yellow sheet.”

Time perceptions are vastly different depending on the unit. In the emergency department, orders need to be communicated with maximum speed; in the nursing home care unit, there is more time to focus on ordering. “In an emergency situation . . . a really fast order [is needed and] it’s just barbaric to have to go through a system when there should be another kind of mechanism.” Accumulated time—entering orders on a whole series of patients, for example—can indeed add up to hours.

“You had to sit at the computer terminal rather than giving patient care for three hours.”
“It’s not a time neutral activity.”

One informant felt differently, as described in field notes: “He said that speed was not that much of an issue to him. He said that some users get very frustrated when the response is not instantaneous, but joked that he needed the time anyway, because he was getting older.” There is also a system life cycle issue related to time:

“It was introduced, it was dropped, it was re-introduced . . . there’s sort of a lot of fits and starts in the process.”
“It required a huge amount of effort on a lot of people’s part . . . it probably required thousands of hours and people’s time to get it built into what it is now.”

In some ways, all three sites had experienced failures in their clinical systems. The house staff threatened to strike at the University of Virginia. A failure at the VA resulted in withdrawing a POE system in medicine years ago. El Camino experienced physician resistance early on. Leaders at all sites were determined to succeed, and extra resources had to be made available to do so. The VA was the only one of the three that pulled the system out and then replaced it later in a carefully planned effort to succeed anew. At El Camino it took three years before physicians voted to keep the system, but informants thought it took ten years for full acceptance. Interestingly, users at Virginia also cited ten years as the time for acceptance. The VA system was implemented only on a few units and only recently, so the history does not cover ten years. At the other two sites, POE is working and accepted even though it is neither seamless nor beautiful.

Theme 4: The Organization of Information

There seems to be a tension between the need to structure the information about orders so that the computer system can deal with it efficiently and the need for clinicians to express orders in their own way.

Organization of Information: System Rigidity

Instead of fitting the workflow, each system requires that “you gotta think like the guy who programmed it.” For example, a physician was trying to order a coronal sinus CT and thought it did not make sense when he found it under “head CT.” From the field notes: “I think a clinician would typically think of a head CT as a separate entity from a sinus CT, not as a heading for more specific types of scans.”

Although POE systems have been shown to reduce errors, a lack of clear visual cues on some screens may lead to errors of misidentification. There were numerous descriptions of clinicians entering orders on the wrong patient: “If a doctor hits Jane instead of Mary, it will go to the wrong patient, but there are triple checks now.” Systems have safeguards against communication of wrong orders, but in the meantime the clinician has spent extra time and then needs to reenter the same orders for the correct patient. We detected a great deal of redundancy because users are dealing with an error potential. We counted as many as twenty information “systems,” both manual and electronic, in some hospital units. Often clinicians use several to make sure that an order has been communicated. Although order sets tend to improve workflow, there are many examples of the rigidity of POE systems impeding workflow.

Organization of Information: Integration

Every place has several systems to which users would like seamless access. A totally integrated CPR is what everyone would like. None of the systems had seamless access to knowledge resources such as MEDLINE, although the VA system had easy access. The VA was the only system covering both inpatients and outpatients. The ability to use the same system across both inpatient and outpatient is desired: “I’d like to see strong similarities between inpatient and outpatient. I know the process is a little bit different, but I think it’s ludicrous to think that they’re gonna use this when they’re in the inpatient and then they go to ambulatory care and they have to use an entirely different system with a different look and feel, that’s crazy.” The potential of an integrated system is recognized by residents: “I’m thinking of going into practice next year in a community hospital. I think it would be a wonderful tool to have . . . to access from an office even . . . and if it was an integrated user friendly system . . . it would be a very powerful tool to have in any hospital.”

Differences Between Teaching and Nonteaching Hospitals

There were no clearly evident differences between teaching and nonteaching hospitals in relation to POE. The role of nurses is somewhat different and may impact implementation of POE. El Camino had implemented clinical pathways to a greater extent than the other sites; because these pathways are the product of interdisciplinary collaborative effort, teamwork is highly regarded there. It may be that without the presence of residents, nurses are viewed differently.

Differences: Clinical Pathways and Collaboration

A nurse at a teaching hospital said: “I would love to be able to also have a fully integrated system with all the nursing documentation. I see multidisciplinary order sets where we have physicians’ medical orders and we have independent nursing orders together that combine to make the plan of care where all disciplines can chart against all orders in a very quick fashion.” While clinical pathways were a dream at a teaching hospital, they are reality at the nonteaching hospital. “It might be that in a community hospital you don’t need to get consensus from very many people,” one informant reasoned. For example, in labor and delivery, “they have clinical paths for vaginal deliveries, c-sections, preterm deliveries, and hypertension.” It was not easy to gain consensus on these, but the number of physicians involved was fewer than fifteen. Order sets are especially helpful to primary care physicians: “He says if he is seeing someone in the middle of the night who happens to have asthma, ‘I don’t need to wake the pulmonologist in the middle of the night, [I] can use his/her order set for asthma.’”

Differences: Turnover, Longevity, and Stability

There is a sense of permanence at the nonteaching hospitals. POE is more common in teaching hospitals because house officers are a captive audience and can be told what they must do. Having a captive audience in house officers does not necessarily make implementation easy, however. There is the sense that they are merely transients and do not have a long-term stake in the hospital: “The house staff just come here for three years to train. They’re not really institutionally connected.” House officers often need to learn multiple POE systems if they rotate among hospitals. This is particularly difficult: “She’s been on so many different systems it’s a blur.”

There is a sense of permanence at El Camino despite, or perhaps because of, a threat to its viability that was successfully weathered several years ago. We heard the staff referred to several times as “the El Camino family.” We were told that turnover is low. We observed numerous older physicians who were resistant to POE at one time but who happily use it now. A similar sense of permanence was expressed at the American Lake campus of the VA, where there also are no residents.

Differences Including Multiple Missions and Education

Academic health centers have multiple missions of patient care/service, research, and education. Non-teaching hospitals focus on patient care. There are residents and faculty at the teaching hospitals who believe that POE enhances medical education and others who just as strongly believe it is a detriment:

“Order sets . . . you’re really not forced to think through your choices.”
“It’s a mixed blessing because one of the beauties of not having a paper chart is you can do it anywhere and you can do it on the fly . . . but it’s changed the entire teaching exchange.”

Conclusion and Recommendations

The purpose of this study was to identify the perceptions of computerized physician order entry by diverse professionals at sites where POE has been successfully implemented and to identify differences between teaching and nonteaching sites. Types of sites were more alike than different. The themes of (1) organizational issues, (2) clinical/professional issues, (3) technical/implementation issues, and (4) the organization of information and knowledge lead to several recommendations for those planning to implement POE.

Organizational Issues

The results indicate that an organization with a strong culture, a history of collaboration and teamwork, and supportive leadership can provide fertile ground for planting the seeds of a successful system. Organizational culture is deep seated and not easily changed, however. A strong culture can be based on a proud history, as described by informants at the University of Virginia and El Camino. Another important aspect of organizational culture is trust, especially trust in hospital or health sciences center administration. It takes time for a nurturing culture to develop. Organizations would be wise to assess how ready they are according to the subthemes listed under Organizational Issues in Table 2

and try to address areas of weakness before contemplating POE implementation.

Clinical/Professional Issues

Customization and the ability to adapt POE at the local level are vitally important for successful implementation. The more flexibility the system has, the better accepted it will be by different physician specialties. The present study found that the process of local development with broad involvement of clinicians is necessary and healthy for the organization as a whole. Those planning to implement POE need to recognize that this careful customization in partnership with clinicians takes time, however.

Technical/Implementation Issues

Speed was usually cited as the most important aspect of the system. Other critical attributes are the ability to group orders into order sets, to make clinical pathways available to the health care team, and to enter orders from remote locations. Vendors and system designers need to strive to offer products with fast response times and mobility.

Organization of Information

We found that the older systems, in particular, seem to require the user to compartmentalize and think in a rigid hierarchical manner. Users, however, want a meaningful organization to the knowledge, and a system that will make it easier to get things done. To be meaningful to the user, it seems that the organization of knowledge must be local and individualized. The information must be organized in a manner designed to mimic the way in which people use the information. This is generally not in a structured, hierarchical manner. People do not want to be forced to “think like a computer.” Individuals exhibit different ways of chunking or bundling information to summarize it and make it accessible to themselves later. Vendors and system designers should work together with clinicians to develop systems flexible enough to organize information this way.

Teaching vs. Nonteaching Hospitals

Differences between types of hospitals that may have an impact on POE implementation include the level of professional collaboration, the amount of staff turnover, and the diversity of missions. Community hospitals may have closer collaboration between physicians and other health care professionals, a more stable staff, and a more singular mission. Although these hospitals do not have house officers for whom they can mandate POE use, they may capitalize on these success factors instead.

The Future

Several trends may lead to more widespread successful implementation of POE in hospitals. The first is a national effort to reduce medical errors. The Leapfrog Group has called for POE as one of three initiatives that its members, as purchasers of health care, should implement.²¹ In the AHRQ Evidence-Based Practice Center report on interventions that have been proven to improve the quality of care, POE with decision support was shown to have moderate evidence for effectiveness.²² Such pressure may cause hospital administrators to acquire POE too readily, but it also may promote careful consideration and wise implementation.

Several guides to assist in judicious implementation of POE have been published recently. Although they do not offer detailed guidance, they realistically portray the benefits and difficulties of implementing POE. From sources respected by administrators, they advocate cautious movement toward implementing POE. The American Hospital Association Guide to Computerized Physician Order-Entry Systems,²³ the Institute for Clinical Systems Improvement Technol-ogy Assessment Report on POE,²⁴ and the ECRI evaluation of computerized provider order entry systems²⁵ give balanced views of the positive and negative aspects of POE. The authors’ website at cpoe.org also offers guidance from international experts.

In addition to national mandates and published guides, efforts are being made by hospitals that have already implemented POE to share the lessons they have learned. As time goes by, more case studies that explore both failures²⁶ and successes²⁷ are appearing in the literature.

An encouraging trend is that POE is no longer looked upon as simple order transmission, there is greater recognition that POE must be integrated into an electronic medical record system if the benefits of synergy are to be reaped.²⁸ The greatest benefit is the knowledge embedded in clinical decision support systems. Indeed, decision support is becoming the major driver for POE development, implementation, and ongoing improvement.²⁹

Finally, it appears that the technology is improving. Although the ECRI report does not give any of the commercial systems its highest rating, it points out that advances are being made as vendors, clinicians, and health care organizations work together to improve systems.³⁰ Evidence indicates that systems can get faster from the physician’s point of view as a result of improved response time and design improvements.³¹

In conclusion, many of the issues identified in this research are being addressed at the national level by respected organizations and by major systems vendors. However, many of the issues can be addressed only within the hospitals themselves. The data show that an organizational culture characterized by open and trusting communication between administrators and clinicians is necessary. Power and control issues will not disappear, but they can be managed. Collaboration and two-way communication are important and need to be part of the fabric of the organization. Most of the technology issues discovered in our research are being addressed by vendors, and faster, more integrated systems with desired value added features are becoming available. However, the best system can remain unused or misused if clinicians resent it. If it is a good system, resentment may very well stem not from system shortcomings, but from lack of trust in the people deciding that POE is necessary (e.g., administration). POE implementation is an iterative process that takes time. The necessary change process is not easy, but once POE is accepted, users become so dependent on it that they, as one informant said, “couldn’t live without it.”

Acknowledgments

The authors thank Bart Lally, MD, Lynda Winterberg RN, MBA, and others affiliated with El Camino Hospital in Mountain View, California, for their generous cooperation and assistance. Paul Mongerson posed the original research question and provided early funding, for which we are most grateful. William Hersh, MD, initiated the project, and Zoe Stavri, PhD, assisted with preparation of the manuscript. Finally, thanks go to the many house officers, faculty, administrators, and staff who agreed to be observed or interviewed.

References

1. Kohn LT, Corrigan JM, Donaldson MS, for the Institute of Medicine Committee on Quality of Health Care in America. To Err is Human: Building a Safer Health System. Washington, DC, National Academy Press, 2000.

2. Ash JS, Gorman PN, Hersh WR. Physician order entry in U.S. hospitals. Proc AMIA Annu Fall Symp 1998:235–239. [PMC free article] [PubMed]

3. Massaro TA. Introducing physician order entry at a major academic medical center. I: Impact on organizational culture and behavior. Acad Med 1993;68:20–25. [PubMed]

4. Massaro TA. Introducing physician order entry at a major academic medical center. II: Impact on medical education. Acad Med 1993;68:25–30. [PubMed]

5. Ash JS, Gorman PN, Hersh WR, Poulsen SP. Perceptions of house officers who use physician order entry. Proc AMIA Annu Fall Symp 1999: 471–475. [PMC free article] [PubMed]

6. Sittig DF, Stead WW. Computer-based physician order entry: the state of the art. J Am Med Inform Assoc 1994;1:108–123. [PMC free article] [PubMed]

7. Massaro TA. Introducing physician order entry at a major academic medical center. I: Impact on organizational culture and behavior. Acad Med 1993;68:20–25. [PubMed]

8. Payne TH. The transition to automated practitioner order entry in a teaching hospital: The VA Puget Sound experience. Proc AMIA Annu Fall Symp 1999:589–593. [PMC free article] [PubMed]

9. Hodge MH. History of the TDS medical information system. In Blum BI, Duncan K (eds): The History of Medical Informatics. New York, Addison-Wesley, 1990: 328–344.

10. Barrett JP, Barnum RA, Gordon BB, Pesut RN. Evaluation of a medical information system in a community hospital. Columbus, OH, Battelle Columbus Laboratories. Report No. NTIS PB 248 340, 1975.

11. Barrett JP, Hersch PL, Caswell RJ. Evaluation of the impact of the implementation of the Technicon Medical Information System at El Camino Hospital. Part II: Economic trend analysis. Final report. 27 Nov 72–31 Jul 79, 1979.

12. Anderson JG, Aydin CE, Jay SJ. Evaluating Health Care Information Systems: Methods and Applications. Thousand Oaks, CA, Sage, 1994, p 55.

13. Kaplan BK, Duchon D. Combining qualitative and quantitative methods in information systems research: A case study. MIS Q 1988;12:570–586.

14. Forsythe DE. Using ethnography to build a working system: rethinking basic design assumptions. Proc AMIA Annu Fall Proc 1992:505–509. [PMC free article] [PubMed]

15. Morgan D. The Focus Group Guidebook. Thousand Oaks, CA, Sage, 1998.

16. Brunet LW, Morrissey CY, Gorry GA. Oral history and information technology: Human voices of assessment. J Organizat Comput 1991;1:251–274.

17. Janesick VJ. The choreography of qualitative research design: Minuets, improvisations, and crystallization. In Denzin NK, Lincoln YS (eds): Handbook of Qualitative Research, 2nd ed. Thousand Oaks, CA, Sage, 2000, p 393.

18. Crabtree BF, Miller WL. Doing Qualitative Research. Newbury Park, CA, Sage, 1992.

19. Robbins SP. Organizational Behavior: Concepts, Controver-sies, Applications. 7th ed. Englewood Cliffs, NJ, Prentice Hall, 1996, p 396.

20. Robbins SP. Organizational Behavior: Concepts, Controver-sies, Applications. 7th ed. Englewood Cliffs, NJ, Prentice Hall, 1996, p 410.

21. The Leapfroggroup for Patient Safety. Computer physician order entry (CPOE) factsheet. November 2000. Available at <www.leapfroggroup.org>.

22. Kaushal R, Bates DW. Computerized physician order entry (CPOE) with clinical decision support systems (CDSSs). In Making Health Care Safer: A Critical Analysis of Patient Safety Practices. Evidence Report/Technology Assessment #43, Agency for Health Care Quality and Research. Available at <www.ahrq.gov/clinic/ptsafety/chap6.htm>.

23. AHA Guide to Computerized Physician Order-entry Systems. Chicago, American Hospital Association, 2000.

24. Institute for Clinical Systems Improvement. Technology assessment report: Computerized physician order entry. ICSI Technology assessment report #55. June 2001. Available at <www.icsi.org>.

25. Computerized provider order entry systems evaluation. Health Dev 2001;30:323–359. [PubMed]

26. Southon FC, Sauer C, Grant CN. Information technology in complex health services: organizational impediments to successful technology transfer and diffusion. J Am Med Inform Assoc 1997;4:112–124. [PMC free article] [PubMed]

27. Ahmad A, Teater P, Bentley TD, Kuehn L, et al. Key attributes of a successful physician order entry system implementation in a multi-hospital environment. J Am Med Inform Assoc 2002;9:16–24. [PMC free article] [PubMed]

28. Computerized provider order entry systems evaluation. Health Dev 2001;30:323–359. [PubMed]

29. Geissbuhler A, Miller RA. Distributing knowledge maintenance for clinical decision-support systems: The “knowledge library” model. Proc AMIA Annu Fall Symp 1999: 770–774. [PMC free article] [PubMed]

30. Computerized provider order entry systems evaluation. Health Dev 2001;30:323–359. [PubMed]

31. Overhage JM, Perkins S, Tierney WM, McDonald CJ. Controlled trial of direct physician order entry: Effects on physicians’ time utilization in ambulatory primary care internal medicine practices. Am Med Inform Assoc 2001;8:361–371. [PMC free article] [PubMed]

Articles from Journal of the American Medical Informatics Association : JAMIA are provided here courtesy of
American Medical Informatics Association