In our study, the introduction of CPOE/CDSS led to a large reduction in the incidence of medication errors in line with findings in earlier studies. 3–9
All types of errors were reduced with the exception of therapeutic errors. However, this substantial reduction in errors was not followed by a significant reduction in the incidence of pADEs.
The lack of effect on pADEs may be explained by the lack of effect on therapeutic errors due to the fact that, as we have demonstrated earlier, this is the very type of medication error most strongly associated with an increased risk of pADEs. 19
Another reason for not finding an effect may be that the CDSS in both hospitals was basic: only in case of overdosing, drug–drug interactions and allergies were alerts generated. To prevent other types of therapeutic errors, more advanced decision-making support would be needed such as, for example, adaptive dose support for patients with clinical chemical parameters that are outside the normal range (e.g., renally excreted medication in patients with renal failure), support when drugs are contraindicated (e.g., in case of the frail elderly) or support for drug choice by linking the system to formularies and disease guidelines that could lead to more optimal pharmacotherapy. A further reason could lie in the inappropriateness of the CDSS in respect to the clinical setting, since the CDSS is based on a national drug database for community pharmacies and not for hospital pharmacies. The standard drug safety alerts that are generated may not always be relevant for the particular hospital setting, for example, an alert for the combination of an ACE-inhibitor and a diuretic that gives rise to a risk of orthostatic hypotension or an alert for a high dose of furosemide, both very commonly found in the hospital. This may lead to an overload of irrelevant alerts and may cause alert fatigue. 20
One undesirable effect is that physicians not only override irrelevant alerts but also relevant ones. It is possible that other measurements of decision-making support are needed such as clinical pharmacists attending physicians meetings 21
at the medical ward or more intensive education in prescribing skills for junior physicians. 22,23
On average, fewer patients experienced a pADE in the post-intervention period than in the baseline period (a reduction approximately by half). However, because of the underlying negative trend at baseline, this decrease cannot be attributed to the introduction of CPOE/CDSS. In four recent reviews of the effect of CPOE/CDSS on medication safety, only a few studies evaluated the impact on pADEs or ADEs; this is possibly due to the labor-intensive way the data needed to determine (p)ADEs must be collected and assessed. 3,7,8
The evidence from these studies was inconclusive due to the fact that only half of the studies showed any significant effect on (p)ADEs and those studies that did show an impact primarily used a pre/post analysis. 9,24–26
Our ITS study design with segmented linear regression analysis was more robust because it evaluated the longitudinal effect of an intervention and controlled for trends appearing in the outcome. 12
Thus, differences in the findings between our study and other studies may be explained by the study design chosen and by the data analysis. Although there was no effect on the incidence of pADEs and therapeutic errors, it should be emphasized that the decrease in medication errors in the post-intervention period is likely to contribute to a decreased risk of preventable harm, because medication errors can be considered as process measurements, while pADEs are patient outcome measurements.
With respect to the other types of errors, the largest impact was seen on the rate of administrative and procedural errors due to an improvement in readability and due to the fact that key characteristics of a prescription had to be filled in (required fields), which led to more complete medication orders. Although these types of errors do not frequently lead to patient harm, 19
we would argue that it is worthwhile preventing them; when nurses and pharmacy technicians must correct these errors, a substantial amount of valuable time is wasted, which could be better spent on primary patient care. In hospitals with paper-based systems that do not include nurse transcription—a potential source of MEs—the introduction of CPOE/CDSS might lead to a less impressive reduction in MEs. The same may be the case for hospitals that do include pharmacy review in their paper-based systems, which might lead to a lower number of MEs in the baseline than hospitals that have no pharmacy review. In our study the TweeSteden Hospital made use of pharmacy review. The similar reduction in MEs found in both hospitals would indicate that pharmacy review in itself does not explain the observed reduction. In the baseline, probably other factors might be as or more important than the presence of this kind of pharmacy review, such as the illegibility and incompleteness of MOs.
The significant upward trend observed in MOs with one or more MEs in the baseline period is surprising. This might well be an artifact stemming from a learning effect for both observers in terms of detecting medication errors. When they were assessing data, the observers were not blinded, neither before nor after the introduction of CPOE/CDSS. It was not feasible, in view of the time constraints, to begin to classify errors only after all data (pre- and post-CPOE/CDSS) had been collected, and therefore we could not blind our data. This is thus one limitation of our study. At the start of the study, the observers individually assessed ten pilot patients and then discussed differences in classification. Despite this pilot period and the use of a strict classification scheme, interpretation of medication errors is subjective and a learning curve cannot be excluded. Another explanation could be that, due to the limited number of data points, the baseline was unstable. Although we have adequately fulfilled the Cochrane criteria of 3 data points before and after the intervention, 18
longer time periods and more data points may well result in a more stable and reliable baseline. One-year data collection before and after CPOE implementation would facilitate a correction for seasonality. However, there is no evidence that pADEs are subject to seasonal influences. Longer data collection was not feasible in our case because of the labor-intensive assessment of pADEs, along with financial constraints.
The delay in implementation on the gastroenterology/rheumatology ward was due to management issues and strategic interests. The eventual implementation process on this ward took as long as on the other ward in the University Medical Center Groningen (17 weeks). As on the other wards, data collection started 8 weeks after finishing the implementation process. In another study, we concluded that physicians and nurses were positive about the way CPOE/CDSS was introduced as well as about the system itself. 27
In addition, the CPOE/CDSS users on the gastroenterology/rheumatology ward were also satisfied and did not show any resistance to the system. These findings suggest that the delay would not have had any effect on the results of CPOE/CDSS on MEs and pADEs.
One strength of our study is that we evaluated the impact of CPOE/CDSS in two different types of hospitals with one home-grown and one commercial package. Although these circumstances are considered potential sources of bias, similar effects for medication errors were demonstrated even despite different baseline rates. This emphasizes the robustness of our study findings and implies that our results could be applicable to a wider range of settings than those of studies simply evaluating one type of CPOE system in a single hospital.
Our study was performed in adult-based general medical wards, and findings should not be extrapolated to special-care settings such as intensive care wards. Future research may clarify the effect of CPOE/CDSS in these settings. Since investigating the effect of CPOE/CDSS on the readmission rate would have been interesting, future research is also needed into this effect.