There are very few studies in the medical literature addressing the question of whether CPOE changes patients' medical outcome. Several studies performed with systems designed in the 1970s and 1980s dealt only with antibiotic administration by CPOE and show some benefit in both cost savings and patient outcome.12,13
More recently, a study done at a pediatric hospital in Canada, limited to one medical and two surgical inpatient units on which CPOE was installed, showed a 40% decrease in medication error rates (defined as “any event involving medication prescription, dispensing, administration, or monitoring of medications irrespective of outcome”). Despite this, no difference in actual patient morbidity or mortality was shown,14
a result similar to what was observed in the Harvard studies discussed subsequently.
The only study addressing CPOE of all hospital services (adult and pediatric, including all pharmaceutical agents and intravenous drips and mixtures), quoted extensively by Leapfrog and the IOM, was authored by Bates et al.15
and was performed at Brigham and Women's Hospital (part of the Harvard System) in the early 1990s. Since the IOM study and the initial formation of Leapfrog, Harvard's further experience with CPOE use by housestaff officers (the main users of such systems) over a four-year time span from 1993 through 1997 provided more information regarding patient outcomes.16
Both studies used a control group of hospital admissions to obtain a baseline for rates of “nonintercepted serious medication errors” followed by the same analysis after the institution of a CPOE system. The methods and outcome variables were different in each study and are discussed separately.
Both Harvard studies defined “nonintercepted serious medication errors” as a combination of “preventable adverse drug events (ADEs)” that actually occurred plus the total number of “nonintercepted potential ADEs.” The authors define “nonintercepted errors” as errors in drug dosage, interactions, and so on that were not identified by the ordering physician, the pharmacy system, or the administering nursing or pharmacy staff on the floors. “Potential ADEs” did not cause any adverse patient outcomes, and the “seriousness” of both the potential and real ADEs was determined by two blind reviewers with interobserver correlation. The first study used medical and surgical units as well as intensive-care unit settings for collection of baseline data over six months, followed by institution of CPOE (with a pause for training and acceptance) and recollection of data for nine months. The number of “nonintercepted serious medication errors” decreased 55% with the addition of CPOE. The number of “nonintercepted potential ADEs” decreased 84%. These numbers are very impressive and have been extensively quoted by the Leapfrog Group as well as software vendors and the lay press. However, the institution of CPOE had no statistical impact on the occurrence of actual serious ADEs, which decreased by only 17% after CPOE was instituted. Although the authors of the study comment that 42% of the preventable actual ADEs were attributable to “judgment” errors in the use of multiple sedating medications that the computer program would not have prevented, the reality is that no significant decrease in patient morbidity/mortality occurred as a result of the institution of CPOE.
The second study by the Harvard group used data collected on three medical services and designed to look at the same type of outcomes over a four-year period, during which time the CPOE system was constantly undergoing refinement. Data collection was done over two-month periods in 1992 (baseline), 1993 (period 1), 1995 (period 2), and 1997 (period 3). This study again showed an 86% decline in number of “nonintercepted potential ADEs” from baseline compared with period 3. Of major concern, however, was the increase in preventable ADEs (5 of 1,000 patient-days without CPOE to 15 of 1,000 patient-days with CPOE, an increase of 200%) that occurred during period 1 when CPOE was first instituted. This trended downward over time in the last two CPOE periods studied. However, the absolute numbers of true ADEs were too small to be of statistical significance (5, 15, 2, 2 for baseline, periods 1, 2, and 3, respectively). Additionally, the number of “intercepted potential life-threatening ADEs” (usually caught by nursing staff before administration of a drug or intravenous admixture) dramatically climbed in periods 1 and 2 after the institution of CPOE. These increases were tracked to “bugs” in the CPOE system's mechanism for ordering potassium infusions. After the software was refined in period 3, the problem disappeared. These negative effects of the institution of CPOE stimulated the authors to make the following comments in their discussion: “The increase in the number of intercepted potential ADEs that occurred post-POE during periods 1 and 2 illustrates the potential that any change, especially a systems change with profound effects such as POE, has for causing new errors, even though this particular error was always intercepted and the overall effect was clearly positive.”
“We conclude that computerized POE resulted in a very large decrease in the frequency of non-missed-dose medication errors, the errors that are most likely to harm patients. Systems such as these have the potential to both fix and cause problems, and require evaluation.”16
Review of the Harvard studies indicates that although CPOE does indeed decrease the potential for an ADE to occur, there is no decrease in actual serious ADE events with the institution of CPOE. In these Harvard studies, CPOE appeared to increase the incidence of actual serious ADEs, particularly during the early years of implementation of such systems.