We found that the implementation of vendor CPOE systems in five community hospitals was associated with a reduction of more than a third of all preventable ADEs. The greatest benefit was seen for reducing the frequency of life-threatening and serious preventable ADEs. However, we also identified a significant increase in potential ADEs. Overall, the associated reduction in preventable ADEs following CPOE implementation suggests that vendor-developed applications, which may not have been through as many cycles of refinement as comparable internally developed applications, can still meaningfully reduce the occurrence of drug-related injury and harm. Furthermore, the overall ADE rate actually increased because of an increase in non-preventable ADEs, which, as best as we can determine, was not related to CPOE implementation.
In our study, the following may have contributed to the increase in non-preventable ADEs: physician documentation improved following CPOE, thus facilitating the identification of ADEs; second, over the 5-year study interval, there were changes in prescribing patterns [e.g., a shift towards prescribing hydromorphone over morphine; increased prescribing of beta-blockers and angiotensin-converting enzyme (ACE) inhibitors for coronary artery disease and heart failure]. Accordingly, we found that the increased adoption of these drugs was associated with a consistent increase in harm (e.g., hydromorphone resulting in oversedation; beta-blockers and ACE inhibitors with hypotension and renal dysfunction).
While CPOE convincingly reduces medication errors,5,21
the impact of this technology in reducing preventable ADEs has been more difficult to prove.6,22
Studies of internally developed systems have largely found reductions in ADEs, preventable ADEs, and potential ADEs.19,21,23,24
Only three studies have assessed the effect of commercial CPOE systems with decision support on the occurrence of ADEs and potential ADEs: one, performed in a clinic, showed a reduction in ADEs that did not meet statistical significance25
; the remaining two studies, conducted in a pediatric hospital and an adult critical care unit, found statistically significant reductions in ADEs.26,27
We likewise observed similar benefits in the reduction of preventable ADEs with two vendor-based systems following hospital-wide implementation. However, in most other studies, the rate of potential ADEs fell, while it actually increased in this study.
One particular concern expressed by critics of CPOE systems is the wide range of possible outcomes associated with CPOE, with differences in reported benefits and limitations likely resulting from variations in implementation strategies, differences in technology, and use of decision support.5,6,28,29
It cannot be emphasized enough that CPOE technology should be tailored to specific hospital needs according to workflow and co-existing technology. For example, in our study, most potential ADEs were rooted in therapeutic duplication, often resulting in potentially serious or life-threatening overdoses (e.g., in one case, 20.8 g/day of acetaminophen was ordered). Furthermore, this highlights how the introduction of health information technology may sometimes lead to unintended consequences. In this study, some of the hospital-customized standing order sets had the potential of mishandling many sedatives and analgesics (e.g., opioids, non-steroidal anti-inflammatory drugs) through therapeutic duplication. Consequently, potential ADEs indisputably increased post-CPOE implementation. However, these “near misses” did not appear to be the result of software problems inherent to CPOE as designed by the vendor, but rather from order sets designed by local users.
While CPOE is clearly able to improve patient safety, realizing its potential benefits requires appropriate implementation and use.28
All sites can benefit from tracking issues found post-implementation and then making iterative changes. One approach to this evaluation is to use standardized scenarios with simulated patients to determine whether orders are appropriately transmitted and whether decision support tools function as intended. Simulation tools such as the Leapfrog CPOE Evaluation Tool provide specific performance feedback and help to expose potential safety issues among hospitals implementing CPOE.29
While our study provides evidence that vendor CPOE systems are associated with significant reductions in preventable ADE rates, further research is needed to explore the safest and most effective ways to implement vendor technology. Addressing these needs, our research group is currently investigating the impact of varying levels of decision support in vendor CPOE systems on ADE rates, exploring how orders sets should be used and refined, assessing the economic benefits of CPOE adoption, and identifying barriers to successful implementation.
Our study has limitations, the most important being that it was not a randomized controlled trial. During pre-implementation, six hospitals were studied;14
however, one of these hospital chose not to implement CPOE and further declined to participate in the post-implementation portion of this study as a control site. As such, our study had no contemporaneous control group, and the apparent effect could be the result of a secular trend or an intervention other than CPOE. Furthermore, we were unable to adjust for changing case mix during the study period. Therefore, we cannot exclude the possibility of unmeasured confounding resulting in the reduction in preventable ADEs. However, the introduction of CPOE was the main medication safety-oriented intervention during the study interval, thus arguing against major confounding by co-intervention. The large and significant reductions in preventable ADEs also argue against chance being the explanation for our observations. Second, as with previous studies,1,19,23
we reported the rates of ADEs per admission, rather than per medication order. While these rates facilitate comparisons with previously reported estimates, a limitation is that event rates may be impacted if the typical number of medications prescribed changed substantially between the pre- and post-implementation periods. Third, our study is limited by a heterogeneous intervention as two different vendor systems were studied. However, a stratified analysis by vendor did not change our overall findings, and the effect sizes were broadly similar across all sites. Finally, the systems evaluated did not keep records of electronic transactions for potential orders that were aborted when physicians heeded alerts and cancelled their intended orders before submitting. Consequently, the magnitude of benefit of CPOE in reducing preventable ADEs may be even greater than we observed.
In conclusion, vendor CPOE system adoption in five community hospitals was associated with a reduction in more than a third in the preventable ADE rate, thus achieving comparable benefits as internally developed systems. However, the observed benefit was balanced against an increase in potential ADEs. While this study suggests that the preventable ADE rate may fall after CPOE implementation, it also underscores how monitoring for new problems arising from CPOE is crucial to improving overall patient safety.