PMCCPMCCPMCC

Search tips
Search criteria 

Advanced


 
Formats
Article sections
Authors
Related links
Logo of jamiaJAMIA - The Journal of the American Medical Informatics AssociationInstructions for authorsCurrent TOC
 
J Am Med Inform Assoc. 2007 Jul-Aug; 14(4): 400–406.
PMCID: PMC2244893
Copyright © 2007, American Medical Informatics Association
Evaluation of Outpatient Computerized Physician Medication Order Entry Systems: A Systematic Review
Saeid Eslami, PharmD, * Ameen Abu-Hanna, PhD, and Nicolette F. de Keizer, PhD
Department of Medical Informatics, Academic Medical Center, Universiteit van Amsterdam, Amsterdam, The Netherlands.
*Correspondence and reprints: Saeid Eslami, Academic Medical Center, Universiteit van Amsterdam, Department of Medical Informatics, J1b-124, Meibergdreef 15, 1105 AZ Amsterdam, The Netherlands (Email: s.eslami/at/amc.uva.nl).
Received August 2, 2006; Accepted April 2, 2007.
Small right arrow pointing to: This article has been cited by other articles in PMC.
Abstract
This paper provides a systematic literature review of CPOE evaluation studies in the outpatient setting on: safety; cost and efficiency; adherence to guideline; alerts; time; and satisfaction, usage, and usability. Thirty articles with original data (randomized clinical trial, non-randomized clinical trial, or observational study designs) met the inclusion criteria. Only four studies assessed the effect of CPOE on safety. The effect was not significant on the number of adverse drug events. Only one study showed a significant reduction of the number of medication errors. Three studies showed significant reductions in medication costs; five other studies could not support this. Most studies on adherence to guidelines showed a significant positive effect. The relatively small number of evaluation studies published to date do not provide adequate evidence that CPOE systems enhance safety and reduce cost in the outpatient settings. There is however evidence for (a) increasing adherence to guidelines, (b) increasing total prescribing time, and (c) high frequency of ignored alerts.
A 1999 Institute of Medicine report estimated that about 80,000 people are hospitalized and 7,000 die annually in the United States due to medication errors in the inpatient setting. 1 Of these errors, 32–69% are definitely or possibly preventable. From an economic point of view, hospital costs of preventable adverse drug events were estimated at $2 billion per year. Similar reports in other countries show that medication errors indeed have important impact on mortality, morbidity, and cost of care. 2,3 Medication errors have been defined as any error in the medication process, which comprises taking history, ordering, pharmacy management, administration management, and surveillance. A medication error may or may not result in patient harm. Adverse drug events (ADEs) are usually considered to include both medication errors that result in harm (preventable ADEs) and adverse drug reactions (ADRs), which are considered unpreventable. 4
Although failure to monitor patients 5 and work pressure 6 have been reported as possible causes of medical errors, adverse outcomes are more commonly associated with the problems during the medication process. 7 The ordering step is crucial in the process: it is the point at which the physician’s thoughts are transformed to decisions which trigger a series of actions, ultimately resulting in the patient receiving the medication. 7
The Institute of Medicine and other important stakeholders have identified Computerized Physician Order Entry (CPOE) or Electronic Prescription (EP) as a key to reduce medication errors and improve safety. 1,8 In this article, the term CPOE refers specifically to medication ordering.
In an outpatient setting, patient information is often scattered among various paper and electronic information systems, rendering the identification of adverse drug events and other outcomes very difficult. 9 This complicates the evaluation of CPOE systems in the outpatient setting. Not surprisingly, work on evaluation of CPOE systems has focused on the inpatient setting where CPOE systems have been shown to decrease medication errors, especially when the CPOE systems include decision support. 10–16 However, most prescriptions occur in outpatient settings and, due to less control in this environment, more errors might be expected to occur with outpatient prescriptions. A meta-analysis suggested that in 1994, more than one million outpatients in the United States experienced an ADE that required admission to the hospital. 17 Of these ADEs 106,000 were fatal, placing them between the fourth and sixth leading causes of death, although these projections may have been somewhat overestimated. 17,18
The main objective of this systematic review was to identify and summarize published studies of outpatient CPOE systems that evaluated one of six aspects: safety; cost and efficiency; adherence to guidelines; alerts; time; and satisfaction, usage, and usability.
For the purposes of this review, an outpatient CPOE system is a computer-based system that allows clinicians to enter medication orders directly for outpatients or primary care patients. In this context, a decision support system (DSS) is any system designed to directly aid a health professional in decision-making during medication ordering.
We searched for relevant English language articles based on keywords in title, abstract, and MeSH terms, using Ovid MEDLINE® (1950 to March 31, 2006) Ovid MEDLINE In-Process®, and EMBASE® (1980 to March 31, 2006). The final literature search was performed on March 31, 2006.
[triangle] shows the search strategy used to identify the relevant articles. In the first part (A), we applied keywords without quotes and MeSH terms pertaining to electronic prescription. These terms cover old and new ways to refer to CPOE systems. In the second part (B), we searched for medication related terms to identify studies that address prescribing. In the third part (C), we searched for terms related to outpatient care. The results of these three parts were combined using the Boolean operator “and.” Searching was supplemented by scanning bibliographies from identified articles.
Figure 1
Figure 1
Keywords and MeSH terms used in the search strategy (words in bold are MeSH terms) and the search flow.
Two reviewers individually examined all titles and abstracts. Discrepancies among the two reviewers were resolved by consensus involving a third reviewer. Articles were selected if they reported original data from a study in an outpatient setting and if one of their main objectives concerned evaluation (a) of a CPOE system for medication ordering and/or (b) of a DSS used during the medication ordering. Study designs include clinical trials as well as observational studies such as questionnaire surveys on satisfaction, and simulation studies. All studies reporting on alerts, reminders, and DSS which are not part of the CPOE system and which are not triggered during medication order entry were excluded. Opinion papers, letters, and evaluation studies of vaccination ordering systems and stand-alone programs for ordering specific drugs such as Warfarin or Digoxin were excluded.
From the selected papers, the same two reviewers extracted data on the demographics (such as number of physicians, number of patients, and duration of study), study design, outcome measures, and results. Discrepancies between these two reviewers were again resolved by consensus involving the third reviewer.
We listed all reported outcome measures and then categorized them into outcome groups. The measured effects were classified as being one of: statistically significant positive effects; demonstrated positive effects (when the authors report a positive effect but without reporting statistical significance); a mix of statistically significant and demonstrated positive effects; no effect (when reported as such by the authors, with or without statistical arguments); statistically negative effects; and, finally, a mix of positive, absence of, and negative effects. To get insight into the heterogeneous nature of these evaluation studies we classified them according to the hierarchy of study designs developed by the University of California San Francisco Stanford Evidence-Based Practice Center and implemented by Kaushal et al. 16 in their review ([triangle]).
Table 1
Table 1
Table 1 Hierarchy of Study Designs
Searching the online databases resulted in 1,032 articles from Ovid MEDLINE®, Ovid MEDLINE In-Process®, and EMBASE® after removing duplicates. Initial screening of titles and abstracts rendered 52 articles eligible for further full text review. Five additional articles were identified by reviewing bibliographies, yielding a total of 57 articles for full-text review. Based on the full-text review, 19 studies were excluded because they turned out not to address a specific CPOE system or an outpatient setting. One study was excluded because the effects could not be attributed to the CPOE system alone. Two studies were excluded because they only predicted, instead of measuring, the effects. Finally, five studies were excluded because evaluation was not a main objective of the study, leaving 30 articles for detailed analyses. Two of the 30 papers reviewed were published in MEDLINE-indexed conference proceedings. We contacted the authors of these studies, and, to our knowledge, they were not published in a journal before this review was completed. The 30 studies are listed in Table 2 (available as a JAMIA on-line data supplement at www.jamia.org).
For all these studies, we categorized the main outcome measures in six main groups: medication safety; cost and (organizational) efficiency; adherence to guidelines; alerts and appropriateness of alerts; time; and satisfaction, usage and usability. Most papers evaluated multiple outcome measures.
[triangle] shows a summary of all measured effects of the thirty CPOE evaluation articles. Controlled study designs were also described by the effects of CPOE. Articles may have two or more listings in the table because they can describe studies with multiple outcome measures and study designs.
Table 3
Table 3
Table 3 Frequency of All Selected Articles According to Outcome Categories and Study Design
Below we describe the effects per outcome group for all included studies.
Medication Safety
Only four studies assessed the effect of CPOE, all with a DSS, on safety. 19–22 One retrospective observational study showed that there were no ADEs found in a set of randomly selected cases in which the physician accepted the alert on drug allergy or on high severity drug interaction (n = 67). 19 However, among the randomly selected cases in which alerts were ignored there were 3 ADEs found (n = 122, p = 0.55). Since the number of cases (n = 189) was limited, these results did not amount to a significant difference. Another prospective cohort study could not show a statistically significant difference in number of ADEs and preventable-ADEs between computerized and manual prescription systems. 20 An RCT showed that there was no significant difference in the actual number of clinically relevant drug interactions between a control group and the intervention group which received alerts on interactions. However, usage of the system in this study was optional and very low (2.8% of medication orders were prescribed by CPOE). 21
A recent non-RCT study showed that the provider did not complete the medication order of 18 high-volume and high-risk medications when an alert for an abnormal rule-associated laboratory result was displayed (p = 0.03). This study did not show a statistically significant reduction in percentage of definite or probable ADEs (p = 0.23). 22
Cost and (Organizational) Efficiency
Twelve studies evaluated the impact of CPOE on the physician’s office expenses, medication costs and (organizational) efficiency. 21,23–33
Two studies, both non-RCT, showed that there was no significant effect on medication cost when using a price comparison module 23 or prescription cost information 24 but another study with the same study design showed that a CPOE system could reduce the cost of medication by suggesting equally effective but cheaper drugs. 25 One recent cohort study, resulting in two articles, showed that clinicians who used electronic prescribing, with or without receiving a list of prewritten prescriptions and patient specific diagnostic information, had significantly lower prescription costs than those in the control group. 26,27 Rotman et al. performed an RCT and evaluated the cost effect of a CPOE system including an interaction alert module. 21 There was neither an effect on the number of clinically relevant interactions nor a significant effect on the cost of prescribing drugs. Two other RCTs showed that a CPOE system with a DSS did not affect the number of emergency department visits significantly. 28,29 At the same time, one of these two studies showed that there was no significant effect on the cost 28 while the other showed that the cost significantly increased. 29 One study, a non-RCT, evaluated the effect of a CPOE system on the physician’s office expenses and showed that these had increased. 30 Three studies, one observational, one non-RCT, and one RCT, evaluated the effect of electronic prescription on physician office resources and showed a reduction in pharmacist interventions for prescriptions. 30–32 One RCT showed there was no statistically significant effect on consultation rate. 33
Adherence to Guidelines
Eleven studies evaluated the impact of CPOE with a DSS on the adherence to a guideline or another standard. 25,28–30,32–38 Among these, four studies showed that there was a significant positive effect on adherence; 25,35,37,38 two studies showed a positive effect without reporting on statistical significance; 34,36 and five studies 28–30,32,33 did not find a significant difference between the control and the intervention group. Among the studies with a positive effect on adherence to guidelines, one used an RCT design, 37 three used a non-RCT design 25,35,38 and two were observational studies 34,36 whereas in the studies without any effect three were RCTs, 28,29,33 one was a non-RCT 30 and one was observational. 32
Alerts and Appropriateness of Alerts
Six observational studies 19,39–43 assessed the impact of CPOE on the produced, accepted and ignored alerts from two points of view: system weakness and user response. Fernando et al. performed an observational study and showed important weaknesses in generating alerts in four commonly used commercial systems in Britain’s GP offices. 39 None of them was able to generate all 18 predefined established alerts for contraindicated drugs and hazardous drug-drug combinations. 39
The remaining five studies addressed user response to alerts. Four studies showed that most of the alerts (from 55% to 91.2%) were ignored by the physicians. 19,40–42 Two studies showed that “clinically irrelevance” was the main reported reason for overriding alerts. 41,43
Time
Three studies 31,44,45 one RCT and two non-RCTs, showed that the total time for direct and indirect patient care increased due to the introduction of the CPOE system.
Another observational study showed that physicians did not believe that electronic prescription was more time consuming than hand-written prescription. 46
Satisfaction, Usability, and Usage
Fifteen studies evaluated the impact of a CPOE system on user satisfaction and system usage and usability. Among them, five observational studies 25,30,36,43,44 showed that after the introduction of the CPOE system, the majority of users were satisfied with the system and they believed that the system is usable. Although the environment, questionnaire, and target groups were different in these studies, the majority of users believed that CPOE improved drug management and quality of care. Three others, 21,23,35 one RCT and two non-RCT, showed that user satisfaction and usability decreased. Two other RCTs showed that patient satisfaction did not change significantly after introducing CPOE with decision support. 28,29 There was a wide variability in the degree of CPOE usage. Four studies showed that of all prescriptions, 2.8% to 90%, were entered electronically 21,32,46,47 and another study showed that the levels of system usage were low. 33 However, another study showed there was continued improvement in system usage and usability. 47 Finally, a simulation study showed the relationship between usage of two decision support models and the complexity of the cases. 48 During prescribing, physicians were more willing to use on-demand decision support as the clinical situation became more complex while for simple cases the reminder-based support was appropriate.
We have identified and described the results of 30 papers on evaluation of CPOE systems in outpatients. The number of such evaluation studies has clearly increased since 2002 (only 10 articles out of 30 before 2002). We used extensive search criteria in order to capture the different ways a CPOE system is referred to in the published literature. However, the following are two limitations of our search. First, because we only addressed studies in which evaluation formed a main objective, we could have missed some studies with a limited evaluation focus. Second, we may have missed some studies that have targeted outpatient CPOE systems in specific specialties such as oncology and pediatrics.
To our knowledge this is the first review exclusively dedicated to the evaluation of CPOE systems in outpatient settings. Existing reviews of CPOE system evaluation studies focused on inpatients, 13,16,49 where advantages of these systems have been reported. Recently, Chaudhry et al. reviewed the impact of health information technology on quality, efficiency, and cost of medical care in inpatient and outpatient settings 50 but only two papers of our review appeared there. Another recent review article focused on overriding drug safety alerts in CPOE, which forms only one aspect of CPOE system evaluation. 51 In contrast, we provide a comprehensive characterization of outpatient studies including description of the study design (with level of evidence), methods, materials, and results.
In spite of the efforts made to enhance safety by introducing CPOE, only four studies evaluated safety in an outpatient setting. Three of them did not show significant reduction in the number of ADEs. Moreover, only one study showed a significant reduction in the number of errors. Recently some observational studies in inpatients described how new medication errors were associated with the use of the CPOE system itself. 52–54 No such studies were found in the outpatient setting, but one should be aware of the possible existence of such associations in outpatients as well.
A plausible explanation for the low number of ADE-related evaluation studies is the difficulty of obtaining quantitative data on ADEs and the high cost associated with chart reviewing in the face of incomplete [and scattered] patient data. 55 The availability of an electronic patient record or a CPOE is only part of the solution because there should also be an infrastructure interconnecting information residing in laboratory and radiology information systems with information residing in the outpatient clinic or GP offices. However, we believe that by focusing on specific patient groups, high risk drugs, simple errors and typical ADEs, one can evaluate the effect of CPOE systems on ADEs with a reasonable level of effort. Another approach would be to identify the few currently existing health care settings with comprehensive paperless systems in place in order to perform evaluation studies with errors and ADEs as outcomes.
Studies on alerts show that alerts were largely ignored by physicians. This does not necessarily mean that safety is compromised; alerts should not be used as proxies for the number of errors or ADEs. Many alerts are not applicable to the patient at hand or they are not clinically important. A valid, although not a new, insight is that the provision of non-patient specific advice is a considerable weakness of CPOE systems, which may lead to low user response and inattention. Five out of the six studies on alerts used the observational study design which has a lower evidence level. Future qualitative and quantitative studies are necessary to show the reasons for overriding alerts and whether redesigning the system is effective in reducing unnecessary alerts and clinician overrides.
Another stated potential benefit of CPOE systems is the reduction of medication cost. There is some indication, although only from two studies with non-RCT design, that advice on equally effective but cheaper drugs and evidence-based messages are more effective at reducing costs than simply displaying a list of drugs with their prices. It is likely that this effect will become more pronounced if the suggestions become patient specific and also target specific expensive groups of medications such as antidepressants and antihypertension medications. 26
Niinimaki and Forsstrom 56 described recommendations for standardization and evaluation of CPOE systems in outpatients. They suggested evaluating technical as well as medical facets of these systems. None of the selected papers in our study evaluated technical facets such as data security and reliability of data transfer solutions, client interface, technical functionality, the checking mechanism for dangerous drug dosage, etc. Technical facets such as the user interface are important as they influence the way users perceive and interact with the system. Such aspects deserve more research attention in the future.
In the selected articles various study designs were used to evaluate CPOE systems in outpatients. The results obtained by non-randomized studies were more likely to report statistically significant improvement in the outcome measures than RCTs. This is possibly an indication that non-randomized studies might be biased.
In spite of the cited merits of enhancing safety and reducing costs, published evaluation studies do not provide adequate evidence that CPOE systems provide these benefits in outpatient settings. A possible explanation is the small number of such studies conducted to date and the relatively weak study designs used. In contrast, there is more evidence on the ability of CPOE systems to increase adherence to guidelines in outpatient settings. We hence hypothesize that cost reduction can be achieved when guidelines are specifically geared towards this goal and that safety can be improved when guidelines are made more patient-specific. A second conclusion of our study is that there is much to be gained in insight when more direct outcome measures on safety are included. However, this is hard to achieve due to the nature of the scattered patient information and the non-controllable environment in outpatients. A third related conclusion is that although more studies with a randomized controlled design are welcome to demonstrate and confirm the effects of CPOE on the medication process outcomes in outpatients, there are serious difficulties in conducting them in outpatient settings. Hence we believe that there is also room for new and nontraditional methodologies and case studies for addressing the impact of information technology interventions in dynamic environments such as health care.
In sum, we found that CPOE systems seem to support adherence to guidelines which have the potential to influence costs and safety. To date, there is, however, little evidence about improving safety as measured by medical errors and ADEs. Focusing on outpatient subgroups and specific drugs merits more attention in the future. The field would also benefit from efforts to standardize evaluation studies, such as the standard proposed in Bell et al. 57 which aims at facilitating comparisons among studies. Finally, standards for CPOE system requirements and functionality, such as those pertaining to providing alerts, merit more attention as they could facilitate the design and implementation of such systems in the future.
References
1. Kohn LT, Corrigan JM, Donaldson MS. To Err is Human:Building a Safer Health System. Washington, DC: National Academy Press; 1999.
2. Vincent C, Neale G, Woloshynowych M. Adverse events in British hospitals: preliminary retrospective record review Brit Med J 2001;322:517-519. [PMC free article] [PubMed]
3. Wilson RM, Runciman WB, Gibberd RW, Harrison BT, Hamilton JD. The quality in Australian health care study Med J Aust 1995;163:458-471. [PubMed]
4. Franklin BD, Vincent C, Schachter M, Barber N. The incidence of prescribing errors in hospital inpatients: an overview of the research methods Drug Safety 2005;28(10):891-900. [PubMed]
5. Jordan S, Tunnicliffe C, Sykes A. Minimizing side-effects: the clinical impact of nurse-administered “side effect” checklist J Adv Nurs 2002;37(2):155-165. [PubMed]
6. Griffith R, Griffiths H, Jordan S. Administration of medicinePart 1: The law and nursing. Nurs Stand 2003;18(2):47-53. [PubMed]
7. Kilbridge P, Classen D. A process model of inpatient medication management and information technology interventions to improve patient safety. VHA’s research series; 2001.
8. Milstein A, Galvin RS, Delbanco SF, Salber P, Buck Jr CR. Improving the safety of health care: the Leapfrog initiative Eff Clin Pract 2000;3(6):313-316. [PubMed]
9. Gandhi TK, Burstin HR, Cook EF, et al. Drug complications in outpatients J Gen Intern Med 2000;15:149-154. [PMC free article] [PubMed]
10. Teich JM, Merchia PR, Schmiz JL, Kuperman GJ, Spurr CD, Bates DW. Effects of computerized physician order entry on prescribing practices Arch Intern Med 2000;160(18):2741-2747. [PubMed]
11. Bates DW, Leape LL, Cullen DJ, et al. Effect of computerized physician order entry and a team intervention on prevention of serious medication errors JAMA 1998;280(15):1311-1316. [PubMed]
12. Bates DW, Teich JM, Lee J, Seger D, Kuperman GJ, Ma’Luf N, et al. The impact of computerized physician order entry on medication error prevention J Am Med Inform Assoc 1999;6(4):313-321. [PMC free article] [PubMed]
13. Oren E, Shaffer ER, Guglielmo BJ. Impact of emerging technologies on medication errors and adverse drug events Am J Health-Syst Pharm 2003;60:1447-1458. [PubMed]
14. Mekhjian HS, Kumar RR, Kuehn L, et al. Immediate benefits realized following implementation of physician order entry at an academic medical center J Am Med Inform Assoc 2002;9:529-539. [PMC free article] [PubMed]
15. Galanter L, Didomenico RJ, Polikaitis A. A trial of automated decision support alerts for contraindicate medication using computerized physician J Am Med Inform Assoc 2005;12:269-274. [PMC free article] [PubMed]
16. Kaushal R, Shojania KG, Bates DW. Effects of computerized physician order entry and clinical decision support systems on medication safety Arch Intern Med 2003;163:1409-1416. [PubMed]
17. Lazarou J, Pomeranz BH, Corey PN. Incidence of adverse drug reaction in hospitalized patients: a meta-analysis of prospective studies JAMA 1998;279:1200-1205. [PubMed]
18. Bates DW. Drug and adverse drug reactionsHow worried should we be?. JAMA 1998;279:1216-1217. [PubMed]
19. Weingart SN, Toth M, Sands DZ, Aronson, MD, Davis RB, Phillips RS. Physicians’ decisions to override computerized drug alerts in primary care Arch Intern Med 2003;163:2625-2631. [PubMed]
20. Gandhi TK, Weingart SN, Borus J, et al. Adverse drug events in ambulatory care N Eng J Med 2003;348(16):1556-1564.
21. Rotman BL, Sullivan AN, McDonald TW, et al. A randomized controlled trial of a computer-based physician workstation in an outpatient setting: implementation barriers to outcome evaluation J Am Med Inform Assoc 1996;3:340-348. [PMC free article] [PubMed]
22. Steele AW, Eisert S, Witter J, et al. The effect of automated alert on provider ordering behavior in an outpatient setting PloS Med 2005;2(9):864-870.
23. Vedsted P, Nielsen JN, Olesen F. Does a computerized price comparison module reduce prescribing costs in general practice Fam Pract 1997;14(3):199-203. [PubMed]
24. Ornstein SM, MacFarlane LL, Jenkins RG, Pan Q, Wager KA. Medication cost information in a computer-based patient record systemImpact on prescribing in a family medicine clinical practice. Arch Fam Med 1999;8:118-121. [PubMed]
25. Walton RT, Gierl C, Yudkin P, Mistry H, Vessey MP, Fox J. Evaluation of computer support for prescribing (CAPSULE) using simulated cases Brit Med J 1997;315:791-795. [PMC free article] [PubMed]
26. McMullin ST, Lonergan TP, Rynearson CS, Doerr TD, Veregge PA, Scanlan ES. Impact of an evidence-based computerized decision support system on primary care prescription costs Ann Fam Med 2004;2:494-498. [PubMed]
27. McMullin ST, Lonergan TP, Rynearson CS. Twelve-month drug cost savings related to use of an electronic prescribing system with integrated decision support in primary care J Manag Care Pharm 2005;11(4):322-332. [PubMed]
28. Murray MD, Harris LE, Overhage JM, et al. Failure of computerized treatment suggestions to improve health outcomes of outpatients with uncomplicated hypertension: results of a randomized controlled trial Pharmacotherapy 2004;24(3):324-337. [PubMed]
29. Tierney WM, Overhage JM, Murray, MD, et al. Can computer-generated evidence-based care suggestions enhance evidence-based management of asthma and chronic obstructive pulmonary disease?A randomized controlled trial. Health Serv Res 2005;40(2):477-497. [PMC free article] [PubMed]
30. Wogen SE, Fulop G, Heller J. Electronic prescribing: improving the efficiency of the prescription process and promoting plan adherence Drug Benefit Trends 2003:35-40.
31. Beer J, Dobish R, Chambers C. Physician order entry: a mixed blessing to pharmacy? J Oncol Pharm Practice 2002;8:119-126.
32. Ross SM, Papshev D, Murphy EL, Sternberg DJ, Taylor J, Barg R. Effects of electronic prescribing on formulary compliance and generic drug utilization in the ambulatory care setting: a retrospective analysis of administrative claims data J Manag Care Pharm 2005;11(5):410-415. [PubMed]
33. Eccles M, McColl E, Steen N, et al. Effect of computerised evidence based guidelines on management of asthma and angina in adults in primary care: cluster randomised controlled trial Brit Med J 2002;325:941-948. [PMC free article] [PubMed]
34. Chin HL, Wallace P. Embedding guidelines into direct physician order entry: simple methods, powerful results Proc AMIA Symp 1999:221-225.
35. Siegel C, Alexander MJ, Dlugacz YD, Fischer S. Evaluation of computerized drug review system: impact, attitudes, and interactions Comp Biomed Research 1984;17:419-435.
36. Rivkin S. Opportunities and challenges of electronic physician prescribing technology Med Interface 1997;10(8):77-7883. [PubMed]
37. Christakis DA, Zimmerman FJ, Wright JA, Garrison MM, Rivara FP, Davis RL. A randomized controlled trial of point of care evidence to improve the antibiotic prescribing practices for otitis media in children Pediatrics 2001;107(2):15-19.
38. Bernstein SL, Whitaker D, Winograd J, Brennan JA. An electronic chart prompt to decrease proprietary antibiotic prescription to self-pay patient Acad Emerg Med 2005;12(3):225-231. [PubMed]
39. Fernando B, Savelyich BS, Avery AJ, et al. Prescribing safety features of general practice computer systems: evaluation using simulated test cases Brit Med J 2004;328:1171-1172. [PMC free article] [PubMed]
40. Taylor L, Tamblyn R. Reasons for physician non-adherence to electronic drug alerts Proc MedInfo 2004:1101-1105.
41. Magnus D, Rodgers S, Avery AJ. GPs’ views on computerized drug interaction alerts: questionnaire survey J Clin Pharm Therap 2002;27:377-382. [PubMed]
42. Shah NR, Seger AC, Seger DL, et al. Improving acceptance of computerized prescribing alerts in ambulatory care J Am Med Inform Assoc 2006;13:5-11. [PMC free article] [PubMed]
43. Glassman PA, Simon B, Belperio P, Lanto A. Improving recognition of drug interactions, benefits and barriers to using automated drug alerts Med Care 2002;40(12):1161-1171. [PubMed]
44. Overhage JM, Perkins S, Tierney WM, McDonald CJ. Controlled trial of direct physician order entry: effects on physicians’ time utilization in ambulatory primary care internal medicine practices J Am Med Inform Assoc 2001;8(4):361-371. [PMC free article] [PubMed]
45. Murray MD, Loos B, Tu W, Eckert GJ, Zhou XH, Tierney WM. Effects of computer-based prescribing on pharmacist work patterns J Am Med Inform Assoc 1998;5:546-553. [PMC free article] [PubMed]
46. Schectman JM, Schorling JB, Nadkarni MM, Voss JD. Determinants of physician use of an ambulatory prescription expert system Int J Med Inform 2005;74:711-717. [PubMed]
47. Tamblyn R, Huang A, Kawasumi Y, et al. The development and evaluation of an integrated electronic prescribing and drug management system for primary care J Am Med Inform Assoc 2006;13:148-159. [PMC free article] [PubMed]
48. Seroussi B, Bouaud J. Reminder-based or on demand decision support systems: a preliminary study in primary care with the management of hypertension Stud Health Technol Inform 2004;101:142-146. [PubMed]
49. Kuperman GJ, Gibson RF. Computer physician order entry: benefits, costs, and issues Ann Intern Med 2003;139:31-39. [PubMed]
50. Chaudhry B, Wang J, Wu S, et al. Systematic review: impact of health information technology on quality, efficiency, and costs of medical care Ann Intern Med 2006;144(10):742-752. [PubMed]
51. van der Sijs H, Aarts J, Vulto A, Berg M. Overriding of drug safety alerts in computerized physician order entry J Am Med Inform Assoc 2006;13:138-147. [PMC free article] [PubMed]
52. Koppel R, Metlay JP, Cohen A, et al. Role of computerized physician order entry systems in facilitating medication errors JAMA 2005;293(10):1197-1203. [PubMed]
53. Horsky J, Kuperman GJ, Patel VL. Comprehensive analysis of a medication dosing error related to CPOE J Am Med Inform Assoc 2005;12:377-382. [PMC free article] [PubMed]
54. Eslami S, Abu-Hanna A, de Keizer NF, de Jonge E. Error associated with applying decision support by suggesting default doses for aminoglycosides Drug Safety 2006;29(9):803-809. [PubMed]
55. O’Neil AC, Petersen LA, Cook EF, Bates DW, Lee TH, Brennan TA. Physician reporting compared with medical-record review to identify adverse medical events Ann Intern Med 1993;119:370-376. [PubMed]
56. Niinimaki J, Forsstrom J. Approaches for certification of electronic prescription software Int J Med Inform 1997;47:175-182. [PubMed]
57. Bell DS, Cretin S, Marken RS, Landman AB. A conceptual framework for evaluating outpatient electronic prescribing systems based on their functional capabilities J Am Med Inform Assoc 2004;11:60-70. [PMC free article] [PubMed]
Articles from Journal of the American Medical Informatics Association : JAMIA are provided here courtesy of
American Medical Informatics Association