We have identified and described the results of 30 papers on evaluation of CPOE systems in outpatients. The number of such evaluation studies has clearly increased since 2002 (only 10 articles out of 30 before 2002). We used extensive search criteria in order to capture the different ways a CPOE system is referred to in the published literature. However, the following are two limitations of our search. First, because we only addressed studies in which evaluation formed a main objective, we could have missed some studies with a limited evaluation focus. Second, we may have missed some studies that have targeted outpatient CPOE systems in specific specialties such as oncology and pediatrics.
To our knowledge this is the first review exclusively dedicated to the evaluation of CPOE systems in outpatient settings. Existing reviews of CPOE system evaluation studies focused on inpatients, 13,16,49
where advantages of these systems have been reported. Recently, Chaudhry et al. reviewed the impact of health information technology on quality, efficiency, and cost of medical care in inpatient and outpatient settings 50
but only two papers of our review appeared there. Another recent review article focused on overriding drug safety alerts in CPOE, which forms only one aspect of CPOE system evaluation. 51
In contrast, we provide a comprehensive characterization of outpatient studies including description of the study design (with level of evidence), methods, materials, and results.
In spite of the efforts made to enhance safety by introducing CPOE, only four studies evaluated safety in an outpatient setting. Three of them did not show significant reduction in the number of ADEs. Moreover, only one study showed a significant reduction in the number of errors. Recently some observational studies in inpatients described how new medication errors were associated with the use of the CPOE system itself. 52–54
No such studies were found in the outpatient setting, but one should be aware of the possible existence of such associations in outpatients as well.
A plausible explanation for the low number of ADE-related evaluation studies is the difficulty of obtaining quantitative data on ADEs and the high cost associated with chart reviewing in the face of incomplete [and scattered] patient data. 55
The availability of an electronic patient record or a CPOE is only part of the solution because there should also be an infrastructure interconnecting information residing in laboratory and radiology information systems with information residing in the outpatient clinic or GP offices. However, we believe that by focusing on specific patient groups, high risk drugs, simple errors and typical ADEs, one can evaluate the effect of CPOE systems on ADEs with a reasonable level of effort. Another approach would be to identify the few currently existing health care settings with comprehensive paperless systems in place in order to perform evaluation studies with errors and ADEs as outcomes.
Studies on alerts show that alerts were largely ignored by physicians. This does not necessarily mean that safety is compromised; alerts should not be used as proxies for the number of errors or ADEs. Many alerts are not applicable to the patient at hand or they are not clinically important. A valid, although not a new, insight is that the provision of non-patient specific advice is a considerable weakness of CPOE systems, which may lead to low user response and inattention. Five out of the six studies on alerts used the observational study design which has a lower evidence level. Future qualitative and quantitative studies are necessary to show the reasons for overriding alerts and whether redesigning the system is effective in reducing unnecessary alerts and clinician overrides.
Another stated potential benefit of CPOE systems is the reduction of medication cost. There is some indication, although only from two studies with non-RCT design, that advice on equally effective but cheaper drugs and evidence-based messages are more effective at reducing costs than simply displaying a list of drugs with their prices. It is likely that this effect will become more pronounced if the suggestions become patient specific and also target specific expensive groups of medications such as antidepressants and antihypertension medications. 26
Niinimaki and Forsstrom 56
described recommendations for standardization and evaluation of CPOE systems in outpatients. They suggested evaluating technical as well as medical facets of these systems. None of the selected papers in our study evaluated technical facets such as data security and reliability of data transfer solutions, client interface, technical functionality, the checking mechanism for dangerous drug dosage, etc. Technical facets such as the user interface are important as they influence the way users perceive and interact with the system. Such aspects deserve more research attention in the future.
In the selected articles various study designs were used to evaluate CPOE systems in outpatients. The results obtained by non-randomized studies were more likely to report statistically significant improvement in the outcome measures than RCTs. This is possibly an indication that non-randomized studies might be biased.