The Sociotechnical Reality: The Intertwinement of Organizational Environment and Technology
Existing technologies and organizational arrangements are important factors that determine the introduction of new technologies such as the introduction of CPOE. In the case of the university medical center, the organizational considerations that determined the choice of the system could be summarized as follows.
- The existing IBM technologic infrastructure narrowed the choice. The technology itself did not dictate the choice, but the associated organizational arrangements did. For instance, a large body of information technology staff over a long period had gained expertise to develop and support the IBM infrastructure, including the mainframes and the network. It was simply too costly to retrain them for a new computing infrastructure or to replace them.
- Another factor was the fact that investments had been made in nonclinical IBM mainframe applications such as personnel and billing systems that had to be connected to the new clinical application. Again, it would be a destruction of capital investments and human resources if these systems had to be replaced.
The system had of course to be adapted to the Dutch situation. The most visible adaptation was the translation of the screens into Dutch. However, the more subtle part was the translation of the clerical workflows into pathways within the system. This led to an explosion of the number of screens to reduce the rigidity of the implied workflow. Sometimes this adaptation was difficult to achieve. The system was conceived to be used within the American hospital system, which is primarily oriented to its inpatient functions. The CPOE implementation in El Camino Hospital is a good example of that approach.18
It proved to be more difficult to adapt the system to the Dutch situation, where medical specialist diagnostic assessment and therapy take place in ambulatory settings (polikliniek
, ambulatory clinical department or ambulatory clinic). The ambulatory clinics process a high number of patients per day and form an integral part of the medical center (see ). With a referral letter from his or her family physician, a patient usually sees a medical specialist first in the ambulatory clinic. The medical specialist may then decide to admit the patient clinically or continue treatment in an ambulatory mode. Also, after a clinical discharge, a medical specialist would see the patient in the ambulatory clinic for follow-up treatments or checkups. We examine the consequences of that later in this article.
Key Figures of the University Medical Center for the Year 2002*
The screens of the CPOE system were translated into Dutch as shows. The mainframe screens were emulated in a Microsoft Windows environment. Data could be entered with the keyboard or with the help of a mouse and clicking on selected fields on the screen, a key characteristic of the system. The mouse movements were very coarse. The interface was originally conceived for terminal interaction with a light pen. For adequate user interaction, the screen could hold maximally 24 × 40 characters.
Figure 3. Dutch translation of the TDS7000 system screen. This menu screen allows doctors to see standing orders and patient data. Under the heading “Patientgegevens” (patient data), the last arrow is a reference to the DRG-like classification scheme (more ...)
The implementers of the system assumed that the introduction would not significantly affect the organization for two reasons. First, the introduction of CPOE would build on a functionality that already was agreed on in 1983 and that was, although very limited and primitively, implemented in UZIS. The functionality was expected to remain basically the same. Second, the existing IBM infrastructure would remain familiar to most information technology staff, whereas the Windows emulation would make use easier. The implementation focused at first on translating the clerical procedures of UZIS into the new system. In a later stage, the functionality of real-time physician order entry and results reporting would be added.
In reality, the implementation had a drastic impact on the organization. Soon after the implementation, clerical users found that retrieving and entering patient data took much more time because each screen would allow them to handle only a limited amount of data. For most tasks, many more screens now had to be worked through. Also, when making typing errors, they had to go back in the pathway and redo a part of the transaction or even the whole transaction. Furthermore, they discovered that data they held to be essential vanished after a few screens. In the old system, some data such as the patient ID number remained in view to help navigation through the system.
All this caused a severe slow-down of work processes, which created chaos at the ambulatory clinical department desks: long lines of (often angry) patients waiting to be helped, frustrated physicians, and verbally assaulted secretaries: “Are you so stupid that you can't handle a computer?” [Patient, ambulatory clinic]
The problem could only be remedied by increasing the clerical staffing of the ambulatory clinical departments to handle the patient load. This was quite contrary to the expectation that the introduction of the new system would save on personnel, as was projected in the proposal.
The problems that the users encountered with the system showed that the registration work processes were closely interrelated with UZIS, the old system. The screen of that system always held the patient number on the top line. The users were very proficient with use of command and function keys to interact with the system. A telling example was the following. The laboratory system could not be connected to the system after the “big bang.” UZIS remained functional to allow users to see the data of the laboratory information system. Users of the system also accessed UZIS at a terminal next to the personal computer (PC) and used the patient name and number, visible on that system, to navigate through their own.
The recurrent problems also changed the attitude of those physicians who were at first champions of the system. When they saw that the workload of the clerks had increased and were confronted with the practical consequences of the system in use, they turned into opponents of the system. They started to emphasize that it was not their task or interest to spend so much time behind a computer: “The system requires a doctor to send electronic notes. Doctors don't send notes. They have other people doing that for them.” [Physician, former project leader, CPOE implementation]
Weiner et al. reported a similar response of physicians toward CPOE.19
In economic terms, CPOE envisages the most highly trained professionals with the greatest opportunity costs to be placed in the data entry role.20
Given the legal distribution of responsibilities, and the expectation that work procedures would be streamlined, the designers in the medical center had embraced this principle. The professionals, however, now started to rally against this idea because it was clear to them that the system would cost them time. What was much less clear to them was what benefit it would bring and how it would fit into their work practices.
Another problem occurred at departments where patients underwent diagnostic or therapeutic treatments. The budgeting of these departments depended on the number of patients treated. When, after some months, the figures requested from the information systems department turned out to be wrong, it took a few months to find the cause. This endangered their proper enumeration, which caused uproar among the medical specialists involved. (In fact, it turned out that the main problems were due primarily to poor instructions on how to enter patient information correctly into the system.) Other small issues further hurt the acceptance of the system, such as a problem with clinical trial patients who erroneously received bills for their experimental treatments.
Our findings emphasize the intertwinement of organizational and technical elements in information technology. The combination of technical considerations (the preexisting IBM infrastructure) and organizational considerations such as “cost containment,” “being clinically oriented,” and “being a leading medical center” together led to the choice for the CPOE system. The system, subsequently, was translated into Dutch and tinkered with to be able to support the typical Dutch “ambulatory clinic” emphasis in hospital work. When put to work in actual practice, the technical features of the system felt restrictive and cumbersome to the users, especially when more and more people had become adjusted to graphic user interfaces. This restrictive and cumbersome “feel,” however, was not just a technologic issue; the screens could have been configured differently (e.g., more easy to use in admitting a patient), but they were not. Also, organizational routines could have been altered to fit the system better, but that was also not tried. Finally, the choice to have the physician in the data entry role was not a technologic necessity; it was an organizational, largely unquestioned, implementation choice.
Understanding the implementation of an information technology, therefore, requires a simultaneous orientation toward both social and technical aspects. Likewise, the account illustrates how it is useless to attempt to determine whether the experienced problems were ultimately “technical” or “human.” Technical design finds its roots in the organizational conditions and arrangements and organizational conditions are changed as a result of technical design, as the example of the clerical users demonstrates.
The Unpredictable Outcome of Implementation Processes: Emergent Change
In 1999, a few pilots were conducted to assess the feasibility of the order entry and results reporting functionality of the system. The main objectives of the pilots were to understand how and which work practices would be influenced by the system, to identify program errors, and to establish the conditions and to plan for a hospital-wide rollout. The conditions were classified in three categories, which were the description of work processes and possible adaptation of the system's functionality, the technical infrastructure, and the level of support and training for users. It was thought that if the conditions in these categories were judged to be adequately met, then the order entry functionality could be activated.
The departments of the hospital were selected according to a judgment about how well work processes were formalized. The department of radiology and nuclear medicine and the department of pathology were considered good candidates for starting pilots because both departments were thought to have well-formalized work routines that would require little adaptation. The clinical department and ambulatory clinical departments were selected through the informal network of the project leader and other team members and at the suggestions of the radiology and pathology departments. There was little preplanning of the pilots. For example, it was agreed that physician order entry would be implemented as a pilot between the departments of gynecology and pathology because during examinations, physicians would usually take tissue specimens and send them to pathology. The number of orders was expected to be sufficient to make a valid assessment of the feasibility of order entry. A plan was made detailing who would be involved in the project. Only a few days before the start of the pilot, the technical staff discovered that the expected refurbishment of the operating theater was canceled and that it was impossible to install a PC because the whole theater would have to be rewired to comply with electrical safety regulations. The pilot then moved to the ambulatory gynecology clinical department and only one doctor was involved. A PC was activated close to the nursing station and a printer was installed for printing labels. Actually, the participants, including the pathologists, the pathology technicians, and the attending gynecologist and nurses, evaluated the outcomes of this pilot very positively.
The connection of the CPOE system with the laboratory information systems of the hospital was considered crucial for laboratory ordering. As in most modern hospital laboratories, the processing of samples and analysis and collection of data are, to a large extent, automated. The selection of a new laboratory information system in the department of clinical chemistry turned out to be very problematic and was only resolved well after the decision to abort CPOE implementation. No connection was made with the new system, and UZIS was still used to see the data on the old laboratory system. The selection process was not a responsibility of the implementation team; it belonged to the laboratory staff who did not want to be involved with the project because they thought that they already had enough problems of their own. It crucially affected the planned implementation.
The examples of the selection process of the pilots and the laboratory information system show that implementation was fraught with uncertainty because it was dependent on the willingness of individuals and departments to participate and on the local contingencies. In a hospital, there is no central line of command that can align departments and projects. The process contributed to the overall unpredictability of the outcomes of the pilots.
Time influenced the implementation process. Because several years passed after the decision to implement a new system, users got acquainted with Windows-based PCs. The interface became “stone age” in appearance: “The characters look like Braille.” [Nurse, dermatology clinic]
Especially due to the pressure from the physicians, the board of directors started to shift their position with regard to the CPOE system. The board did not move toward “owning” the system, but looked for arguments to halt the implementation. Finally, by having an external review, they effectively blamed factors outside of their influence such as the aged character of the system in the context of rapidly developing Windows-based PC systems.
The end result was a “lock-in” situation. The full-blown implementation of the system was aborted, but the system was not put out of use. Nobody was happy with the system, but, on the other hand, it was clear that for the time being, no alternative was available. The system is still in use as a patient registration system, and in due time will be replaced by a new system. Work is now focused on the development of graphic workstations that allow health care professionals to see patient data in a clinically meaningful way.
Designers and implementers devote much effort to controlling the implementation process. Pilots in this perspective are meant to identify shortcomings of the system that can be remedied before a full-blown implementation. The scaling down of the gynecology pilot was not anticipated, as it was the result of contingencies that they could not know. The implementation team did also not realize when the pilots became futile. They did not recognize how adverse experiences with the patient registration portion of the system had negatively affected a major stakeholder group. In the dynamics of implementing the system, the previously mentioned changes were not anticipated and certainly not planned for. Planning occurred as a response to new arising situations, as the gynecology–pathology pilot showed. The focus on systems implementation as a technical strategy allowed different stakeholders to develop their own agenda, as circumstances would dictate. Changes only emerged as a coming together of contingent events and decisions made on an ad hoc basis. As Orlikowski puts it: “emergent change is the realization of a new pattern of organizing in the absence of explicit, a priori intentions.”9
Therefore, we would characterize the implementation of an information system in a complex organization as “emergent change.”
“Success” or “Failure”: producing “Fit”
The implementation of an information system in clinical practice is not a linear process with a defined starting point, clearly delineated goals, and readily identifiable stages. By most accounts, the implementation of the system was a failure (although the system is still in place). Looking back, the functionality of order entry and results reporting, which constituted the core of the system, was never activated. However, it would be too simplistic to argue that this failure could have been predicted at the beginning of the project. Similarly, it would be too simplistic to say that a checklist with critical success and failure factors could have prevented the problems. The outcome was rather the result of a series of events and contingencies that were not planned for or whose impact was not anticipated. Decisions that looked reasonable when made can thus become constraining and clearly “wrong” in retrospect. Each step in the process of implementation leads to new and partially unpredictable outcomes that have to be judged in the context of the new situation. In the checklist, the box “appropriate technology” would have been ticked at the time the procurement decision was made; the mainframe technology appeared to be highly problematic only later. At the time of the decision, the choice for the IBM mainframe environment made perfect sense; infrastructures based on PC technology for large organizations were far from mature up to the middle of the 1990s. One may conclude that long lead times for implementation are questionable because objectives, goals, and context (organizational and technical) can change drastically over time, yet it is hard to imagine how such expensive and complex implementation processes could reduce their lead time sufficiently. In complex organizations, large-scale implementations are observed to take a long time. Such long times are even deemed to be necessary for the mutual learning process to develop and implement information systems.20,21
There is, then, no simple formula for success or failure. The complexity of the sociotechnical networks, and the inherent unpredictability of information system implementation within complex organizations, is simply too great.11
In addition, what counts
as success or failure is not clearcut, but the outcome of a social negotiation. Many different definitions of “success” or “failure” can and are used by the involved actors.11
In this case study, the proposal to introduce the CPOE system very much focused on savings. Savings could be found in fewer budgets spent on system development and maintenance and reducing the number of clerical staff. The benefits of medical order entry and results reporting were phrased in technical terms of better readable and more complete orders, but the consequences for medical work were not highlighted. Some studies and reports in the public press have shown that cost benefits of strategic information technology in health care and the services industry in general were not achieved.22
In the discussion about medication errors, physicians do not see themselves as part of the problem.23
Other professionals in the loop of ordering medication often correct potential errors so that the ordering physician is not confronted with the potential negative outcome of his or her action.24
In a survey of hospitals that have implemented order entry and results reporting systems, Ash et al. found that physicians were using the system in only 15% of these hospitals.25
The remark of one of the leading physicians in the CPOE system implementation project that physicians do not send electronic notes is a telling example.
From the case study, it became apparent that the system did not fit well with registration work practices. In a study about evaluation of medical informatics applications, Kaplan made a plea to use evaluation methodologies based on social interactionist theories.26
In this paper, she suggested that “fit” is a key factor for the successful implementation of information systems in health care. “Fit” has various dimensions such as compatibility of the system with the workflow, with the level of expertise of the users, with the belief system of the user or the organization, and so forth. We agree with Kaplan that fit with work processes constitutes an important explanation for success. However, we find that whether “fit” exists or not is not due to the technology introduced and the practice in which it is to be used. Rather, this “fit” has to be actively produced
: the technology and the practice have to be made
to fit. An information system has to be adapted to the work practices of the user, but users will have to change their practices as well because of the opportunities and constraints of the technology. To achieve this fit, first of all, requires a thorough understanding of the work practices. Yet an analysis of work practices should not only include an analysis of what people do, but also how they might do it better
Often, a proper understanding of a system's functionality can point to the direction of such improvements. In this case, for example, it was a moot point that the hospital did not attempt to restructure work routines so that an order entry system would be able to articulate with the practice much better.6
An order entry system becomes truly useful, even for doctors, when standardized care paths and protocolized order sets are made part and parcel of working routines. A deeper understanding of clerical work practices, also, might have led to their reorganization and perhaps could have prevented the increase in the number of clerks. In the current example, because “organizational change” was not planned, measures to remedy organizational impacts were now mainly defensive. Alternatively, proper information technology implementation should always
be seen as a process of organizational change and should thus always
be oriented toward a redesign of, in this case, professional working patterns.28