Improving Patient Safety by Identifying Side Effects from Introducing Bar Coding in Medication Administration

doi:10.1197/jamia.M1061

J Am Med Inform Assoc. 2002 Sep-Oct; 9(5): 540–553.

PMCID: PMC346641

Improving Patient Safety by Identifying Side Effects from Introducing Bar Coding in Medication Administration

Emily S. Patterson, PhD, Richard I. Cook, MD, and Marta L. Render, MD

Affiliations of the authors: Ohio State University, Columbus, Ohio (ESP); University of Chicago, Chicago, Illinois (RIC); University of Cincinnati College of Medicine, Cincinnati, Ohio (MLR); VA Midwest Patient Safety Center of Inquiry, Cincinnati, Ohio (ESP, RIC, MLR).

Correspondence and reprints: Emily Patterson, PhD, 210 Baker Systems, 1971 Neil Avenue, Columbus, OH 43210; e-mail: <patterson.150/at/osu.edu>.

Received November 16, 2001; Accepted April 16, 2002.

This article has been cited by other articles in PMC.

Abstract

Objective. In addition to providing new capabilities, the introduction of technology in complex, sociotechnical systems, such as health care and aviation, can have unanticipated side effects on technical, social, and organizational dimensions. To identify potential accidents in the making, the authors looked for side effects from a natural experiment, the implementation of bar code medication administration (BCMA), a technology designed to reduce adverse drug events (ADEs).

Design. Cross-sectional observational study of medication passes before (21 hours of observation of 7 nurses at 1 hospital) and after (60 hours of observation of 26 nurses at 3 hospitals) BCMA implementation.

Measurements. Detailed, handwritten field notes of targeted ethnographic observations of in situ nurse–BCMA interactions were iteratively analyzed using process tracing and five conceptual frameworks.

Results. Ethnographic observations distilled into 67 nurse–BCMA interactions were classified into 12 categories. We identified five negative side effects after BCMA implementation: (1) nurses confused by automated removal of medications by BCMA, (2) degraded coordination between nurses and physicians, (3) nurses dropping activities to reduce workload during busy periods, (4) increased prioritization of monitored activities during goal conflicts, and (5) decreased ability to deviate from routine sequences.

Conclusion. These side effects might create new paths to ADEs. We recommend design revisions, modification of organizational policies, and “best practices” training that could potentially minimize or eliminate these side effects before they contribute to adverse outcomes.

Background

Reducing the risk of adverse drug events (ADEs) is a national priority.¹ Adverse events related to medication errors are common, costly, and often preventable. The Harvard Medical Practice study found that 19% of adverse events were related to medications,² a finding confirmed by the Colorado/Utah study of adverse events during hospitalization.³ Medication administration in the acute care inpatient setting is a complex system requiring coordination among physicians who order the medications, pharmacists who verify and dispense the medications, and nursing personnel who administer medications to the patients. In an analysis of 334 medication errors from 11 acute care wards, 39% of the problems were judged to occur during physician ordering, 12% during transcription and verification, 11% during pharmacy dispensing, and 38% during nursing administration.⁴

Computerized support systems are advocated to reduce the risk of ADEs at each of these stages in medication administration.⁵ These systems, including computerized physician order entry, automated dispensing systems, and bar code reconciliation, are believed to reduce reliance on memory, increase access to information, and increase compliance with “best practice” procedures.⁶ For example, Bates⁷ found an 86% decrease in intercepted serious medication errors associated with the introduction of a computerized order entry system in three medical units at a large urban hospital. Similarly, medication alerts,⁸ a bedside computerized prescribing critiquing system,⁹ a decision support system for antibiotic prescribing,¹⁰ and automated dispensing units integrated with information about a patient’s medication profile¹¹ were associated with improved prescribing outcomes or reduced medication errors.

The Veterans Health Administration (VA), one of the largest health care systems in the United States with 163 medical centers nationwide, is a leader in the use of medical informatics systems. Since 1996, the VA has used an integrated electronic database for patient clinical information, the Veterans Health Information Systems and Technology Architecture (VistA), which enables staff to access administration and demographic data in addition to radiology, laboratory, and pathology information from any computer on the network. In 1997, the VA implemented computerized order entry (CPRS),¹² which is integrated with the VistA database. A Graphical User Interface (GUI) for CPRS was subsequently developed and implemented to replace the original command line interface.

Recently, the VA internally developed bar code medication administration (BCMA) software, based heavily on prototype software deployed at the Topeka VA medical center in 1995. With BCMA, patients wear bar-coded wristbands, and unit dose medications are bar-coded in the pharmacy before delivery to a patient’s medication drawer. Nurses scan the patient’s wristband to access up-to-date medication order data in BCMA, which was input into CPRS by physicians and verified in VistA by pharmacists. When the nurse scans a medication, it is automatically documented as administered at that time if the bar code number matches the number for a medication displayed on the primary BCMA screen. If there is a mismatch, whether it is a different medication, dose, form, or time, an immediate warning pops up as an alert box. Implementation of BCMA and associated hardware was mandated to be completed throughout the VA by 2000. The BCMA software is continuously revised by a national development team, including a second release planned for May of 2002 that is intended to better meet the needs of the Intensive Care Unit (ICU) setting, with expanding IV medication functionality and supporting documentation of STAT verbal orders prior to pharmacy verification.

Even with the most effective systems, the introduction of a new technology into a complex setting is never a panacea.¹³ In addition to providing new capabilities, new technologies have unanticipated side effects on technical, social, organizational, economic, cultural, and political dimensions of work.¹⁴ Kaplan¹⁵ and the Institute of Medicine report¹⁶ recommend a proactive approach to patient safety, including the injunction to “examine new technologies . . . for threats to safety and redesign them before accidents occur.” Even as barcode-enabled point of care medication administration systems reduce the frequency of some failure modes,^17–¹⁹ the transformation of the medication administration process creates new, potentially predictable paths to adverse drug events. We asked the question: What negative, unintended side effects, if any, resulted from the introduction of BCMA that might create new paths to ADEs? This paper describes negative, unanticipated side effects that were identified by targeted ethnographic observation prior to and following system implementation and have implications for system redesign that can be incorporated prior to the occurrence of adverse events.

Methods

Bar Code Medication Administration

Nurses access BCMA software using a laptop permanently fixed to a wheeled medication cart and linked by a wireless network to the VA’s electronic databases. Data entered into BCMA via a bar code scanner are used to verify that the correct patient gets the ordered medications in the correct dose and route at the correct time. To identify the patient, the nurse logs into the BCMA software and scans a bar code representing the patient’s identification number (ID) on his wristband. The computer then displays the medications that are “DUE,” based on the time the wristband is scanned and information in the patient’s active electronic record (Figure 1

). As the bar code of each medication is scanned, if the information about the drug formulation matches the displayed information, the system automatically records the medication as administered by the nurse at the time of scanning. If the scanned information on the medication does not match, the nurse is alerted to the discrepancy by a pop-up dialog box. The nurse cannot override this alert but has the option of typing in a bar code number for either a patient’s wristband or a medication in place of scanning. The scanning of bar coded wristbands is intended to reduce the risk of patients receiving medications intended for another. Scanning each medication barcode is intended to verify that the medication, dose, route, and administration time match what was ordered.

Figure 1

Primary BCMA display for a fictional patient.

Setting

We observed medication administration on acute care and nursing home wards of three VA hospitals selected on the basis of BCMA implementation schedules and willingness to participate (Table 1

). The hospitals varied in size from small (3000 acute care admission per year) to moderate (9000 admissions per year); hospitals 1 and 2 were university affiliated. In all hospitals at the time of observation, the same computerized physician order entry system (CPRS) had been used for at least 1 year before introduction of BCMA, and the mandated BCMA system had been in operational use on all three shifts for at least 1 month. During the observations in hospital 2, full documentation on the paper MAR was maintained in parallel with BCMA for the purpose of providing a backup system, whereas in the other two hospitals documentation was maintained solely in the BCMA software at the time of observation.

Table 1

Characteristics of Observed Facilities

Sites selected hardware (scanners, ethernets, batteries, and computer terminals) from among nationally recommended options. Staff personnel were trained through national “train the trainer” workshops, manuals to direct local training, and conference calls with local experts describing the changes associated with software updates. All nurses received four hours of hands-on training as well as hardware and software maintenance support from Information Resource Management (IRM). This study did not examine training, on-site support, or hardware or physical ergonomic issues associated with local decisions.

Conceptual Framework

Ethnographic observations before and after the natural experiment of introducing a new technology, BCMA, into a complex, sociotechnical setting were used to identify new paths to failure in the complex process of medication administration in the acute care inpatient setting. These observations were targeted in that data collection and analysis were guided by five conceptual frameworks from the human factors literature base²⁰:

Recognition-primed decision making (RPD)
Human-automation interaction
Workload
Authority-responsibility doublebinds
Mutual awareness

RPD is a model of expert decision making under uncertainty, in dynamic settings, and with high consequences for erroneous action.²¹ In contrast to the optimal decision making model, in which careful review of all relevant factors in series results in the selection of the “optimal” choice, the RPD model emphasizes that expert nurses make decisions about which medication to administer to which patient based on past experience by selecting an early workable option that trades off conflicting goals and uncertainties. Thus, for nurses administering medications, we were primed to observe (1) cues to assess the patient state and the identity of the patient and medications, (2) resolution of competing goals, and (3) strategies to reduce uncertainty.

Second, in human–computer interaction²² terms, medication administration is a task that requires the nurse to interact with a computerized system with automated features. Research on automation in aviation cockpits has revealed predictable failures, referred to as “automation surprises,”²³ that occur when the automation initiates actions that are surprising to the human partner. The potential for automation surprises is increased when three factors converge:

The automation can act on its own without direction from its human partner (high autonomy and authority).
There are gaps in users’ mental models of how their machine partners work in different situations.
There is weak feedback about the activities and future behavior of the machine agent (low observability).

During the observations, if a nurse was surprised by actions initiated by the BCMA software, we collected data about the action taken by the software, the displayed information, the nurse’s interpretation of the automated action, the vulnerabilities to failure created by the inability to anticipate that action, and the outcome.

Third, workload is the perception of task demands by a human agent. It depends on performance criteria, task structure, task duration, and the complexity and variability of task demands, among other factors.²⁴ In both aviation and anesthesiology, research finds predictable adaptations to reduce workload.²⁵ From this research base, we anticipated strategies to reduce workload during peak times, including trading accuracy for speed, reducing performance criteria, shedding tasks, deferring tasks, and recruiting resources from other personnel, and noted when these strategies increased the vulnerability to ADEs.

Fourth, the introduction of computerized support systems is often coupled with increased adherence to organizational policies and procedures, increasing opportunities for authority—responsibility doublebind situations, researched primarily in military settings.²⁶ When procedures are not applicable, practitioners must either deviate from the procedures to meet their goals, potentially increasing individual vulnerability to punitive or legal action, or follow the procedures, potentially increasing the vulnerability to undesirable outcomes. When observed nurses encountered authority-responsibility doublebind situations, we took detailed notes about their strategies to resolve the situations, including tailoring procedures, rote rule-following, covert work practices, or activities to reduce the risks of punitive action that might have undesirable effects.

The last conceptual framing is that medication administration is a distributed and cooperative activity conducted in a social setting. Successful coordination across distributed agents depends on a high level of mutual awareness, sometimes referred to as common ground. Common ground is defined as items that are shared among human agents, including knowledge, goals, frames of reference, and beliefs about the current and future state of affairs.²⁷ When electronic systems replace paper-based systems, a commonly observed side effect is the degradation of coordination as a result of privatizing information that was previously publicly accessible (e.g., electronic flight strips²⁸). For example, coordination between controllers of the London underground is aided by lighted fixed line diagram displays that are viewable and easily referenced by physical gesturing. Replacing this display with individual workstation displays degraded coordination by reducing the ability to use the display as a shared frame of reference.²⁹ When a nurse was observed to be surprised by actions of other personnel, we interviewed opportunistically to see if design choices that reduce support for mutual awareness contributed to coordination breakdowns.

Data Collection

One observer (EP), trained in ethnographic field observations in complex settings,³⁰ conducted all observations before and after BCMA implementation (Table 2

). Each observation session lasted from one to seven hours and included observations during all parts of day, evening, and night shifts. To minimize the effect of observations on behavior, we informed the observed nurses that no one, including their nurse managers, would be provided with information that could trigger a punitive response since observations were de-identified for person, place, and time, and we did not collect demographic or medication error data. Prior to BCMA implementation, seven nurses were observed for a total of 21 hours during 10 medication passes at one hospital. Following BCMA implementation, 26 nurses were observed for a total of 60 hours during 23 medication passes at the three hospitals. We targeted timing of the observations to include observations immediately after implementation, after initial problems were resolved, and after the system was fully adopted into the workflow (ranging from 1 week to 13 months).

Table 2

Observations Before and After BCMA Implementation

In addition, the same observer also watched computerized order entry (2 physicians, 2 hours) and order verification by inpatient pharmacists (5 pharmacists, 2 pharmacy technicians, 10 hours) at hospital 1. Twenty complementary interviews were conducted with practitioners, computer support personnel, and nurse managers from the three hospitals. The focus of the interviews was to elicit perceived barriers to intended system use and reactions to possible redesign, organizational, and training interventions to reduce side effects. Finally, critical decision method interviews³¹ about a “near-miss, wrong-patient” case were conducted with nurses, a nurse manager, and a pharmacist to understand why the nurse using the BCMA software did not detect a patient mismatch.

During the observations, we recorded time-stamped, detailed, handwritten field notes of verbal and physical behavior, targeted at the identification of negative, unanticipated side effects, as well as information captured in the electronic patient record.

Analysis

The data were iteratively analyzed using the conceptual theoretical frameworks from the human factors literature to recognize and abstract emerging patterns (Figure 2

). Process tracing protocols³² for all of the observations were created that detailed the activities, grouped into several broad categories. Particularly interesting interaction sequences were represented and analyzed as mini-cases. The mini-cases were then grouped by emerging themes, and supplementary data from the detailed notes based on the themes were collated. Finally, the themes were matched to similar patterns documented in other complex, high-consequence, sociotechnical domains along with design, organizational, or training strategies to reduce the side effects, when available.

Figure 2

Levels of abstraction in the process tracing analysis.

Results

Analysis of the process tracing protocols identified 67 unique mini-cases, which were exclusively grouped into 12 categories based on a prominent theme (Table 3

Table 3

Sixty-seven Mini-cases Grouped by Category

Figure 3

provides an overview of a mini-case that has poor usability³³ as a prominent theme. In this case, workload on an acute care ward was unexpectedly high as a result of unanticipated absences of staff and patient factors (a coworker’s emergency absence in the middle of a shift, attendance of another staff member at a meeting, discharge of a patient “against medical advice,” a new patient admitted from the emergency department with continuing chest pain). Using the computerized order entry system, a physician ordered two medications to be given immediately to treat the new patient’s chest pain (without otherwise informing the nurse of the order, which used to be accomplished by a red color code on the outside of a paper chart before the implementation of computerized order entry but is now communicated less frequently other than through the written order because new orders cannot be highlighted in CPRS or BCMA). The nurse, anticipating that these medications would be ordered, looked in BCMA but found no cardiac medications (note that pending and discontinued medications are not displayed). He then looked in the order entry system and found two “pending” medications. The nurse borrowed and administered one of the two medications (taken from another patient’s medication drawer) and waited for a less critical medication to arrive from pharmacy. He could not document the medication administration in BCMA since the medication was not yet displayed (medications need to first be ordered by physicians and then verified by pharmacists prior to display), and only displayed medications could be documented as administered with the observed version of BCMA. The nurse then turned his attention to preparing another patient for discharge. When the nurse later documented the medication administration, the administration time was automatically documented as the time the medication was scanned, several hours after it was administered. This time was not changed to reflect the actual administration time, partly because changing medication administration times required a long, confusing sequence of activities in a separate, command line interface. Poor usability was a prominent theme in this mini-case. It was difficult for the nurse to accomplish his goals in this nonroutine situation because (1) pending and discontinued medication orders were not displayed, (2) medications that were not displayed could not be documented as administered, (3) it was not possible to see changes to medication orders without first opening a patient record, (4) it was difficult to “undo” actions, and (5) it was difficult to revise database information after it had been entered.

Figure 3

Example of mini-case.

To illustrate the next step of the process-tracing analysis, several elements in this mini-case were re-represented in more abstract, less domain-specific terminology that points to concepts drawn from the human factors literature. First, nursing expertise was brought to bear in ways that resulted in deviations from a “procedural” model of medication administration: rotely and sequentially administering medications displayed on an interface to a series of patients during a “medication pass.”³⁴ For example, knowledge and assessment of the risk of ischemic complications were used to anticipate two pending medication orders and prioritize administering one medication over other tasks. Second, a goal conflict resulted in “violation” of a “best practice” procedure.³⁵ In this case, the goal of following the “best practice” procedure of having a pharmacist verify a medication ordered by a physician prior to administration (to increase order accuracy and decrease the possibility of allergic reactions or interactions with other medications, among other reasons) conflicted with the goal of intervening as quickly as possible to reduce cardiac risk, including the potential for a code. Third, in choosing to meet performance (medical) goals, an authority-responsibility doublebind situation occurred, in which the nurse felt the responsibility to act without having the authority to document the action.³⁶ As a result, the medication was documented at a different time from when it was administered, which could be viewed as a covert work practice. Fourth, the dynamics of the workload distribution encouraged task prioritization strategies of delaying documentation until after a high-tempo period.³⁷ When there was difficulty in documenting information, either because the automatically generated data was incorrect or because medications were not displayed, the nurses moved on rather than take the time to ensure accurate documentation at that moment. Finally, system design choices created the potential for coordination breakdowns.³⁸ The reliance of the physician on BCMA to communicate a new, high-priority order for imminent administration could be viewed as a poor strategy, even though there was no adverse outcome due to the nurse’s anticipation of the order, because the software was not designed to actively highlight priority or new medication orders. In addition, the difficulty in correcting the difference between the actual and documented medication administration time could enable physicians, pharmacists, or other nurses to make incorrect inferences based on the data. For example, a nurse on the next shift could delay medication administration. Similarly, the inability to document administration of medications not displayed creates the potential for coordination breakdowns between multiple nurses (e.g., a covering RN could administer medications that have already been given) and physicians (e.g., a physician might unnecessarily increase the dose of a medication if he bases his inferences on an inaccurate administration time).

Following similar re-representations of each mini-case based on the conceptual frameworks and guided by insights from the interviews, the data were analyzed for recurring patterns across the cases. This iterative top-down and bottom-up data analysis revealed five negative, unanticipated side effects following the introduction of BCMA:

Nurses confused by automated removal of medications by BCMA
Degraded coordination between nurses and physicians
Nurses dropping activities to reduce workload during busy periods
Increased prioritization of monitored activities during goal conflicts
Decreased ability to deviate from routine sequences

Nurses Confused by Automated Removal of Medications by BCMA

There were four cases in which nurses expressed surprise at automated actions initiated by the BCMA software, which is the stereotypic reaction to a classic breakdown in human–computer interaction, referred to as an “automation surprise.”³⁹ Two cases involved IV medications that were automatically dropped from the medication list a number of hours after the scheduled administration time. One was a chemotherapy medication whose administration was delayed due to lack of an IV site, and another was a patient who received Ativan late because he was off the ward for several hours. In another case, a nurse was surprised that an antibiotic was not available as a result of an automatic stop order. In all of these cases, the nurses detected that the automatically removed medications were intended to be given and eventually administered them, but their surprise indicated that their expectation that the medication would be displayed in BCMA was violated, and they reported that this design decision created a new potential path to missed medications.

Degraded Coordination between Nurses and Physicians

A side effect of replacing the paper medication administration record (MAR) with BCMA appears to be degraded coordination between nurses and physicians. During the observations, numerous coordination breakdowns occurred between nurses and physicians, some of which might not have occurred with the previous paper-based system. Prior to BCMA, physicians were observed to have quick access at the bedside to current medication administration information and nurses had immediate access to information about pending and discontinued orders on their paper MAR. Immediately after BCMA implementation, most physicians were not aware of how to access medication administration information, and the display was restricted to fixed 7-day window (Saturday to Sunday). Although the CPRS software was subsequently updated to provide physicians with increased access to BCMA data, in a focus group conducted October 26, 2001, seven of seven residents stated that they did not systematically look at a patient’s medication orders in BCMA unless a question was asked because it is a time-intensive process. In comparison, they estimated that with a paper-based system at another hospital, they reviewed the medication orders for each patient an average of 3 times/week.

Degraded coordination between nurses and physicians can lead to predictable new paths to adverse events, including failing to detect erroneous medication orders, verifications, or administrations, failing to renew automatically discontinued medications, failing to prioritize a STAT medication order over other activities, or failing to explain why laboratory values are unusually high or low for an at-risk patient.

Nurses Dropping Activities to Reduce Workload during Busy Periods

Nurses at all three hospitals consistently used strategies to increase efficiency that circumvented the intended use of BCMA (Table 4

). For example, we observed that all nurses typed in at least one patient identification number rather than scanning the wristband barcode.

Table 4

Observed strategies to reduce workload during medication pass

Before the introduction of BCMA, system developers predicted that an additional benefit of the system would be improved documentation efficiency through scanning barcodes rather than typing information.⁴⁰ Nurses felt that there were efficiency benefits only when scanning was consistently reliable, when there were no deviations from routine actions such as refused medications, and when paper-based systems were not used for parallel documentation. In addition, barcodes are normally scanned during busy, high workload periods. This characteristic works against the commonly observed strategy in event-driven, high-consequence domains, such as take-offs in aviation, to free mental and physical resources during busy periods by shifting other tasks to less busy periods.⁴¹

Scanning wristbands to identify patients was dropped more frequently than scanning medication barcodes, a finding also reported in a study of a different barcode-enabled point of care system in a private institution.⁴² A variety of factors might explain this finding. Obvious factors include that wristband barcodes did not scan as reliably as medication barcodes⁴³ and that wristbands could not be scanned in some cases (e.g., isolation patients, patients who removed wristbands because of swollen limbs or discomfort, particularly in long-term care).⁴⁴ In addition, nurses uniformly believed that typing in a 7-digit number took less time than wheeling a large medication cart into a room and scanning a wristband. In addition to these factors, there were also more subtle barriers to scanning wristbands. Most nurses tried to avoid disturbing sleeping patients, particularly if they anticipated that a patient had no oral medications during the medication pass. They tried to minimize interrupting discussions between patients, family, and health care practitioners. In addition, several nurses explained that they employed strategies to reduce distractions during medication preparation in order to better detect inaccurate medication orders. When they scanned the patient wristband, patients often distracted the nurse by requesting an action or information.⁴⁵ Finally, several nurses expressed concern that other nurses or physicians would log them out of BCMA if they left the computer console.

Increased Prioritization of Monitored Activities during Goal Conflicts

Shortly after BCMA implementation, many nurses stated that they felt timeliness of medication administration was greatly increased in priority. Similarly, nurses at every hospital were observed to be anxious when required to type an explanation for late medications, generally 30 minutes before and 60 minutes after the scheduled administration time. In addition, several nurses stated that they were concerned that analyses of timeliness of medication administration by individual nurses would be much easier to do with the electronic data than with handwritten data. Further research is needed to understand how an increase in emphasis on timely medication administration affects decision tradeoffs during goal conflicts. One nurse stated that she believed that nurses would be more likely to delay other activities during the medication pass that should not be delayed in order to ensure timely medication administration. Figure 4

depicts interruptions that were acted on during medication administration on a chemotherapy ward before BCMA implementation, including fixing restraints on a patient at risk of falling.

Figure 4

Interrup-tions during medication administration prior to BCMA.

Decreased Ability to Deviate from Routine Sequences

Computerized systems, such as BCMA, are often effective at streamlining routine operations, making the system both more efficient and easier to anticipate others’ actions and detect erroneous actions. A frequent tradeoff from streamlining routine operations is that non-routine sequences of activity become more difficult to perform. For example, prior to BCMA, although there was no “taper dose” feature in the computerized order entry system, a physician could enter a description in a text box at the bottom of a single order (e.g., prednisone taper: start at 60 mg decrease 10 milligrams every other day until at 10 mg, then decrease to 5 mg for 2 days). Prior to BCMA, one pharmacist estimated that he would verify a taper dose order in less than 1 minute and pass the free text note on to nursing personnel. After the introduction of BCMA, a pharmacist, in order to create the order in a scannable format, was observed to take 17 minutes to break a taper dose order into daily orders at the exact dose, a total of 14 new orders, and discontinue the original order. In addition, the pharmacist expressed the hope that the physician would not rewrite the order in the original format, as the observed pharmacist explained had occurred in the past when a physician did not understand why the order had been changed. Although a taper order is a specific example of an issue that can be resolved through software enhancements, it illustrates the observed pattern of decreased flexibility when machine algorithms critique human actions, because the “vocabulary” used in communicating with a machine is restricted. When the format is not well known or cumbersome to the practitioner, decreased flexibility might lead to new vulnerabilities and the reduced ability to detect erroneous actions.

One of the greatest challenges with BCMA is how to support flexibility in the event that the system goes down, either for scheduled maintenance or because of an unexpected loss of power. There appears to be great variability in how institutions have adapted BCMA to enable flexibility given the potentially high consequences of being unprepared for the infrequent but predictable eventuality of loss of power. For example, these adaptations have followed the introduction of BCMA in several hospitals in order to provide a backup system:

Maintaining documentation redundancy with the paper-based MAR in parallel with BCMA
Printing out MARs every few hours for informational but not documentation purposes
Creating an electronic backup by logging every change to medication orders in a separate system
Reverting to a paper MAR on an executive decision within 15 minutes of a power outage in parallel with instituting periodic training and manuals to maintain competency with the paper-based system
Capturing health summaries updated every hour to stand-alone computers on emergency power.

Discussion

Bar coding is advocated as a technology to reduce errors in several health care settings.⁴⁶ To improve proactively the effectiveness of the BCMA system in the Veteran’s Health Administration, we identified five negative, unintended side effects with the potential to create predictable new paths to ADEs: (1) nurses confused by automated removal of medications by BCMA, (2) degraded coordination between nurses and physicians, (3) nurses dropping activities to reduce workload during busy periods, (4) increased prioritization of monitored activities during goal conflicts, and (5) decreased ability to deviate from routine sequences.

None of these side effects is inherent in barcode-enabled point of care technology. It is likely that design revisions, modification of organizational policies, and “best practices” training can proactively reduce each of these side effects before accidents occur. “Automation surprises,” for example, are known to result from automated actions that are “strong” in that they occur without immediately preceding direction from a user combined with weak feedback about past and future automation activities. The automation can be made less “strong,” such as by eliminating automated removal of medications without prior approval from the human partner or by providing the capability for users to restore automatically dropped medications. Alternatively, increasing the feedback about automated actions, such as by displaying a “grayed out” medication to indicate that it was removed from the display, would make it easier for users to be aware of the change. Cooperative solutions, such as using pharmacy personnel to remind physicians when orders have been automatically discontinued, could also be explored.

A primary contributor to the degraded coordination between the nurses and physicians appears to be the loss of quick, coded communication via artifacts that can be viewed “at a glance.” This functionality could be designed into BCMA or other software. Specifically, “longshot visualizations”⁴⁷ could be designed that would allow both nurses and physicians to see at a glance when medication orders have changed for multiple patients on a ward, similar to the functionality provided by circular color codes on patient paper charts representing STAT, discontinued, new order, and no change. In addition, the medication administration history window could be redesigned to allow physicians to quickly scan, search, and review medication orders for multiple patients. Similarly, nurses could be provided quicker access to pending and discontinued medication information to make it easier to anticipate orders and detect erroneous actions. Finally, strategies adapted to the paper MAR that had side benefits for coordination, such as “flagging” pages to remember to do an action (that could then be seen by others who looked at the paper MAR), could be supported in BCMA. Design changes to improve coordination can generally be characterized as increasing the “openness” of the workspace, meaning to increase the ability for health care practitioners to be aware of activities being conducted by others. “Open workspaces” have been shown to improve anticipation, synchronization of activities, detection of erroneous actions, and other forms of coordination.⁴⁸

System redesign and best practices can also reduce workload burdens to make it less likely that nurses circumvent the system during busy periods as an adaptation to increase efficiency. Circumventing the system will predictably lead to increased vulnerabilities to mis-identification of patients and medications as well as inaccurate administration data. Obvious strategies to reduce workload, such as decreasing staff/patient ratios, might not always be possible, given the national nursing shortage and increased acuity of hospitalized patients. Alternatively, eliminating unnecessary steps or shifting data entry to lower intensity times could reduce the workload burden. For example, rather than typing the same reason for every late medication for a patient off the ward, data entry might be batched or a default entry provided. In addition, improving the interface usability would decrease workload, such as by making it possible to document refused medications without exiting the system.⁴⁹ Finally, the system could be designed to better fit the natural workflow of providing patient care. For example, wristbands for patients could be scanned after medications have been prepared to minimize interaction between the nurse and patient during medication preparation, particularly when patients are sleeping or prone to interrupt the nurses.

The perception of increased emphasis on timely medication administration will likely drift over time based on how organizations respond to medication administration data. If an organization builds incentive rewards based on medication administration timeliness or punishes individual nurses for taking more time than others, the perception will be reinforced. If an organization clearly communicates that timeliness is a lower priority than other goals, such as reducing patient falls, and restricts access to the time data, the perception will be reduced. Although the organizational context is likely to be the most important contributor to perception of timeliness as a priority, redesign of BCMA might reduce this side effect. Medications that are particularly important to administer at the intended time could be distinguished from others on the display. Dialog boxes that require documentation of why a medication is late could be eliminated. Parameters defining “on-time” medication administration could be enlarged.

Finally, the system could be redesigned to allow an increased ability to deviate from a idealized workflow. As specific deviations are identified, features can be designed to support them, such as by creating a “taper dose” order feature in the computerized order entry system. In addition, flexibility can be increased through generic features such as free text fields, “undo,” “drag and drop,” “select,” “copy and paste,” and “batch.” Flexibility can also be increased by allowing tailoring of options available in dialog boxes, making it easier to revise administration time data and document changes in status from “given” to “refused” or “held.” Finally, features could be added to support the ability to write “free text” information-only notes to document administration of medication orders not viewed on the screen.

It is not surprising that negative side effects were identified after the implementation of BCMA. It is axiomatic that technology change has far-reaching impacts on the way in which work is conducted in a field of practice, even with the most successful technologies.⁵⁰ In opposition to the view that a change in medium (e.g., computerized order entry instead of handwritten prescriptions) can be introduced without greatly altering complex systems embedded in social settings, we believe that little of the former roles, strategies, and paths to failure are preserved across the technology change boundary.⁵¹

This study illustrates how targeted observational studies before and after naturally occurring points of change can be used to better understand processes of transformation and adaptation.⁵² The points of change form natural experiments where a change, the new technology, disturbs a system. Observers calibrated to patterns in human-machine performance watch how the system re-settles into a new equilibrium state. The point-of-change provides opportunities to learn how the system actually functions and sometimes malfunctions by observing how practitioners accommodate and adapt to the technology change. The system is examined to see how the technology transforms roles, judgments, and coordination, as well as how people adapt to the mix of new capabilities and new complexities. This information provides insight for organizational, design, and training post-conditions to make the system more useful and reduce unintended, negative side effects from the change.

In addition, observing at points of change can add to the knowledge base of how changes impact cognition and collaboration in a way that uniquely reveals the complex, inner workings of human-human interactions with a “black box” of technology. This knowledge enables better advance predictions of the effects of a change, especially side effects such as new complexities and vulnerabilities. It also serves as a stimulus to envision alternative design concepts. Armed with this knowledge, we can find higher leverage points for change and develop new concepts for support. If we can better anticipate the reverberations of new technology prior to the process of change, new directions can be pursued when intervention is less difficult and costly.

This study has several limitations. Our objective was to identify proactively negative side effects prior to the occurrence of adverse events. The study was not designed to evaluate the effectiveness of BCMA in reducing the frequency of predefined failure modes at the three participating hospitals or to identify beneficial side effects, such as more efficient and accurate requests for missing medications from pharmacy. Because our findings, as in any analysis of complex, interacting, multifaceted field data, depend on our conceptual frameworks, we likely did not find all side effects. For example, the introduction of BCMA might have impacted institutional views of accountability for adverse drug events. In addition, we observed primarily on acute care wards and nursing homes at three institutions, and it is unclear how these findings would generalize to other settings, such as intensive care units (ICUs) or institutions without 24-hour access to pharmacy. In addition, the ability to generalize these findings to other bar coding applications, such as software designed for the clinical laboratory or blood banking, remains a question to be addressed by future research.

Conclusion

Side effects following the introduction of BCMA were observed that might create new paths to ADEs. These side effects have implications for system redesign, modification of organizational policies, and “best practices” training that might be implemented before the occurrence of adverse events. Investigation of the natural experiment of implementation of new technology can forestall potential accidents in the making by identifying new sources of hazard and developing system redesign, organizational or training initiatives to protect against them.

Acknowledgments

We thank the BCMA development team, led by Mr. Richard Singer, for their hard work and willingness to collaborate to improve the software design, James Bagian, MD, for sharing his knowledge of vulnerabilities with the use of BCMA; David Woods, Ph.D., for sharing findings on research on side effects of technology change; Steve Sawyer, PhD, for reviewing the paper; and Denis Coelho, Ph.D., for conducting pilot research.

Notes

This research was supported by the Department of Veterans Affairs, Veterans Health Administration, Health Services Research and Development Service (Project no. SAF 20-049). The views expressed in this article are those of the authors and do not necessarily represent the view of the Department of Veterans Affairs.

References

1. Doing What Counts for Patient Safety: Federal Actions to Reduce Medical Errors and Their Impact. Report of the Quality Interagency Coordination Task Force (QuIC) to the President, February 2000. Quality Interagency Coordination Task Force. Washington, DC <http://www.quic.gov/report/toc.htm>.

2. Leape LL, Brennan TA, Laird NM, et al. The Nature of Adverse Events in Hospitalized Patients: Results of the Harvard Medical Practice Study II. N Engl J Med. 1991;324(6):377–84. [PubMed]

3. Thomas EJ, Studdert DM, Newhouse JP, et al. Costs of medical injuries in Utah and Colorado. Inquiry. 1999;36:255–64. [PubMed]

4. Leape L, Bates DW, Cullen DJ, et al. Systems analysis of adverse drug events. JAMA. 1995;274:35–43. [PubMed]

5. Bates DW, Cohen M, Leape LL, et al. Reducing the frequency of errors in medicine using information technology. J Am Med Inform Assoc. 2001; 8(4):299–308. [PMC free article] [PubMed]

6. Bates DW. Using information technology to reduce rates of medication errors in hospitals. BMJ. 2000;320:788–91. [PMC free article] [PubMed]

7. Bates DW, Teich JM, Lee J, et al. The impact of computerized physician order entry on medication error prevention, J Am Med Inform Assoc. 1999;6(4):313–21. [PMC free article] [PubMed]

8. Raschke RA, Gollihare B, Wunderlich TA, et al. A computer alert system to prevent injury from adverse drug events: Development and evaluation in a community teaching hospital. J Am Med Inform Assoc. 1998; 280(15):1317–20. [PubMed]

9. Nightingale PD, et al. Implementation of rules based computerised bedside prescribing and administration: intervention study. BMJ. 2000;320:750–3. [PMC free article] [PubMed]

10. Evans RS, Pestonik SL, et al. A computer-assisted management program for antibiotics and other antiinfective agents. N Engl J Med. 1998; 338(4):232–8. [PubMed]

11. Borel JM, Rascati KL. Effect of an automated, nursing unit-based drug-dispensing device on medication errors. Am J Health-Syst Pharm. 1995;52:1875–9. [PubMed]

12. Murff HJ, Kannry J. Physician satisfaction with two order entry systems. J Am Med Inform Assoc. 2001;8(5):499–509. [PMC free article] [PubMed]

13. Barthelemy-Brichant N, Sabatier J, et al. Evaluation of frequency and type of errors detected by a computerized record and verify system during radiation treatment. Radiother Oncol. 1999;53:149–54. [PubMed]

14. Anderson JG, Aydin CE. Theoretical perspectives and methodologies for the evaluation of health care information systems. In: Anderson JG, Aydin CE, Jay SJ (eds). Evaluating Health Care Information Systems: Methods and Applications. Thousand Oaks, CA, Sage Publications,1994, pp 5–29.

15. Kaplan B, Brennan PR, Dowling AF, et al. Toward an informatics research agenda: Key people and organizational issues. J Am Med Inform Assoc. 2001;8:235–41. [PMC free article] [PubMed]

16. Kohn LT, Corrigan JM, Donaldson MS (eds): To Err is Human: Building a Safer Health System. Washington, DC, National Academy Press, 1999.

17. Puckett F. Medication-management component of a point-of-care information system. Am J Health-Syst Pharm 1995; 52:1305–9. [PubMed]

18. Malcolm B, Carlson RA, Tucker CL, Willette C. Veteran’s Affairs: Eliminating medication errors through point-of-care devices. Technical paper for 2000 Annual HIMSS Conference, 2000.

19. Anderson S, Jensen R. Management Case Study: A multidisciplinary approach to decreasing the occurrence of medication errors and variances. Presented at the American Society of Health-Systems Pharmacists Mid-Year Clinical Symposium, 2000.

20. Roth EM, Patterson ES, Mumaw RJ. Cognitive engineering: Issues in user-centered system design. In Marciniak J (ed): Encyclopedia of Software Engineering, 2nd ed. New York, John Wiley & Sons, 2001.

21. Klein GA. Recognition-primed decisions. In Rouse WB (ed), Advances in Man-Machine Systems Research. Greenwich, CT, JAI Press, 1989, pp 47–92.

22. Norman DA. The “problem” with automation: Inappropriate feedback and interaction, not “over-automation.” Philosoph Trans R Soc London. 1990;327:585–93. [PubMed]

23. Sarter NB, Woods DD, Billings CE. Automation surprises. In G. Salvendy (ed): Handbook of Human Factors and Ergonomics, 2nd ed. New York, Wiley, 1997, pp 1926–43.

24. Huey BM, Wickens CD. Workload Transition: Implications for Individual and Team Performance. Washington, DC, National Academy Press, 1993.

25. Woods DD, Patterson ES. How unexpected events produce an escalation of cognitive and coordinative demands. In: Hancock PA, Desmond PA (eds): Stress, Workload and Fatigue. Hillsdale, NJ, Lawrence Erlbaum, 2001, pp 290–304.

26. Woods DD, Shattuck LG. Distant supervision-local action given the potential for surprise. Cogni Technol Work. 2000; 2:86–96.

27. Clark HH, Brennan SE. Grounding in communication. In Resnick, Levine JM, Teasley SD (eds): Perspectives on Socially Shared Cognition. Washington, DC, American Psychological Association, 1991, pp 127–49.

28. Harper RR, Hughes JA, Shapiro DZ. Harmonious working and CSCW: Computer technology and air traffic control. In Bowers JM, Benford SD (eds): Studies in Computer Supported Cooperative Work. Theory, Practice and Design. Amsterdam, North-Holland, 1991, pp 225–34.

29. Heath C, Luff P. Technology in Action. Cambridge:, Cambridge University Press, 2000.

30. Hutchins E. Cognition in the Wild. Cambridge, MA, MIT Press, 1995.

31. Klein GA, Calderwood R, MacGregor D. Critical decision method for eliciting knowledge. IEEE Trans Syst Man Cybernet. 1989;19(3):462–72.

32. Woods DD. Process tracing methods for the study of cognition outside of the experimental psychology laboratory. In Klein G, Orasanu J, Calderwood R (eds): Decision Making in Action: Models and Methods. Norwood, NJ, Ablex Publishing Corporation, 1993, pp 228–51.

33. Rubin J. Handbook of Usability Testing: How to Plan, Design, and Conduct Effective Tests. New York, John Wiley & Sons, 1994.

34. Feltovich PJ, Ford KM, Hoffman RR. Expertise in Context: Human and Machine. Cambridge, MA, MIT Press, 1997.

35. Vicente KJ. Cognitive Work Analysis. Hillsdale, NJ, Lawrence Erlbaum, 1999.

36. Woods DD, Shattuck LG. Distant supervision—local action given the potential for surprise Cogni Technol Work. 2000; 2:86–96.

37. Woods DD, Patterson ES. How unexpected events produce an escalation of cognitive and coordinative demands. In Hancock PA, Desmond PA (eds): Stress, Workload and Fatigue. Hillsdale, NJ, Lawrence Erlbaum, 2001, pp 290–304.

38. Heath C, Luff P. Technology in Action. Cambridge, Cambridge University Press, 2000.

39. Sarter NB, Woods DD, Billings CE. Automation surprises. In Salvendy G (ed): Handbook of Human Factors and Ergonomics, 2nd ed. New York, Wiley, 1997;1926–43.

40. Dinklage KC, White SJ, Lenhart JC, Godwin HN. Accuracy and time requirements of a bar-code inventory system for controlled substances. Am J Hosp Pharm. 1989;46(11): 2304–307. [PubMed]

41. Woods DD, Patterson ES. How unexpected events produce an escalation of cognitive and coordinative demands. In: Hancock PA, Desmond PA (eds): Stress, Workload and Fatigue. Hillsdale, NJ, Lawrence Erlbaum, 2001, pp 290–304.

42. Puckett F. Medication-management component of a point-of-care information system. Am J Health-Syst Pharm. 1995; 52:1305–09. [PubMed]

43. Renner SW, Howanitz PJ, Bachner P. Wristband identification error reporting in 712 hospitals—a College of American Pathologists Q-probes study of quality issues in transfusion practice. Arch Pathol Lab Med. 1993;117(6):573–7. [PubMed]

44. Creasman GL, Dervin D, Widdowson N, Bacher M. Bar-code medication administration: Quality management and medication error evaluation following implementation. Presented at the ASHP Midyear Clinical Meeting. Las Vegas, December 3–7, 2000.

45. Ebright PL, Patterson ES, Chalko B, Render ML. Identification of environmental factors that increase complexity in acute care settings. Proceedings of the Midwest Nursing Research Society 2002 Annual Research Conference, March 1–4, 2002, Chicago, p 148.

46. Bates DW, Cohen M, Leape LL, et al. Reducing the frequency of errors in medicine using information technology. J Am Med Inform Assoc. 2001; 8(4):299–308. [PMC free article] [PubMed]

47. Woods DD, Watts JC. How not to have to navigate through too many displays. In Helander MG, Landauer TK, Prabhu P (eds): Handbook of Human-Computer Interaction, 2nd ed. Amsterdam, Elsevier Science, 1997, pp 617–50.

48. Patterson ES, Watts-Perotti J, Woods DD. Voice loops as coordination aids in space shuttle mission control. Computer Supported Cooperative Work: The Journal of Collaborative Computing. 1999; 8(4):353–71. [PubMed]

49. Murff HJ, Kannry J. Physician satisfaction with two order entry systems. J Am Med Inform Assoc. 2001;8:499–511. [PMC free article] [PubMed]

50. Aydin C. Occupational adaptation to computerized medical information systems. J Health Social Behav. 1989;30:163–79. [PubMed]

51. Cook RI, Woods DD. Adapting to new technology in the operating room. Hum Fact. 1996;38(4):593–613. [PubMed]

52. Woods DD, Sarter NB. Evaluating the impact of new technology on human-machine cooperation. In Wise JA, Hopkin VD, Stager P (eds): Verification and Validation of Complex Systems: Human Factors Issues. New York, Springer-Verlag, 1993, pp 133–58.

Articles from Journal of the American Medical Informatics Association : JAMIA are provided here courtesy of
American Medical Informatics Association