The impact of computerized provider order entry on medication errors in a multispecialty group practice

doi:10.1197/jamia.M3285

J Am Med Inform Assoc. 2010 Jan-Feb; 17(1): 78–84.

PMCID: PMC2995630

The impact of computerized provider order entry on medication errors in a multispecialty group practice

Emily Beth Devine,^1,² Ryan N Hansen,¹ Jennifer L Wilson-Norton,³ N M Lawless,³ Albert W Fisk,³ David K Blough,¹ Diane P Martin,⁴ and Sean D Sullivan^1,⁴

¹Pharmaceutical Outcomes Research and Policy Program, School of Pharmacy, University of Washington, Seattle, Washington, USA

²Division of Biomedical and Health Informatics, School of Medicine, University of Washington, Seattle, Washington, USA

³The Everett Clinic, Everett, Washington, USA

⁴Department of Health Services, School of Public Health and Community Medicine, University of Washington, Seattle, Washington, USA

Corresponding author.

Correspondence to Dr E B Devine, Pharmaceutical Outcomes Research and Policy Program and Biomedical and Health Informatics, University of Washington, Box 357630, Seattle, WA 98195-7630, USA; Email: bdevine/at/u.washington.edu

Supplementary Appendix is available online only at http://jamia.bmj.com/content/vol17/issue1

Preliminary results of this research were presented in two venues in 2008: the 14th Annual National Research Service Award (NRSA) Trainees Research Conference, held in conjunction with Academy Health (podium) and the AMIA Annual Symposium Proceedings 2008 (poster).

Received May 28, 2009; Accepted October 20, 2009.

This article has been cited by other articles in PMC.

Abstract

Objective

Computerized provider order entry (CPOE) has been shown to improve patient safety by reducing medication errors and subsequent adverse drug events (ADEs). Studies demonstrating these benefits have been conducted primarily in the inpatient setting, with fewer in the ambulatory setting. The objective was to evaluate the effect of a basic, ambulatory CPOE system on medication errors and associated ADEs.

Design

This quasiexperimental, pretest–post-test study was conducted in a community-based, multispecialty health system not affiliated with an academic medical center. The intervention was a basic CPOE system with limited clinical decision support capabilities.

Measurement

Comparison of prescriptions written before (n=5016 handwritten) to after (n=5153 electronically prescribed) implementation of the CPOE system. The primary outcome was the occurrence of error(s); secondary outcomes were types and severity of errors.

Results

Frequency of errors declined from 18.2% to 8.2%—a reduction in adjusted odds of 70% (OR: 0.30; 95% CI 0.23 to 0.40). The largest reductions were seen in adjusted odds of errors of illegibility (97%), use of inappropriate abbreviations (94%) and missing information (85%). There was a 57% reduction in adjusted odds of errors that did not cause harm (potential ADEs) (OR 0.43; 95% CI 0.38 to 0.49). The reduction in the number of errors that caused harm (preventable ADEs) was not statistically significant, perhaps due to few errors in this category.

Conclusions

A basic CPOE system in a community setting was associated with a significant reduction in medication errors of most types and severity levels.

Keywords: Computerized provider order entry, ambulatory setting, medication errors, adverse drug events

The Institute of Medicine (IOM) has long been a proponent of electronic health records (EHRs),¹ ² ³ ⁴ making the link between EHR implementation and potential improvements in quality and safety. The vision statement of the National Alliance for Primary Care Informatics endorses the idea that for primary care providers to provide citizens with quality, affordable healthcare, they must be equipped with a fully functional EHR at the point of care.⁵ Research evaluating the impact of EHRs on medication safety has focused largely on evaluation of computerized provider order entry (CPOE) systems, with clinical decision support (CDS) alerts to guide ordering. Several systematic reviews have summarized the benefits of CPOE/CDS systems.⁶ ⁷ ⁸ ⁹ ¹⁰ ¹¹ Collectively, these studies have been conducted primarily in the inpatient settings of academic medical centers, using homegrown systems; most have included CDS alerts to guide ordering. Few have been conducted in the ambulatory setting. Our study addresses some of these gaps, in that we evaluate the impact of implementation of a homegrown, basic CPOE system on medication safety in the ambulatory, community-based setting of a multispecialty, independent medical group.

CPOE systems are computer applications that allow direct, electronic entry of orders for medications, laboratory, radiology, referral, and procedures.¹² CPOE systems for ordering medications are sometimes called electronic prescribing (e-prescribing) systems. CPOE systems are often implemented with clinical decision support (CDS) alerts to guide ordering.¹³ Early research demonstrated the benefits of CPOE/CDS systems in reducing medication errors by as much as 55–86%¹⁴ ¹⁵ and subsequent adverse drug events (ADEs), although the latter occur less frequently and are more difficult to identify.¹⁶ ¹⁷

Evidence from several systematic reviews has shown the benefits of CPOE/CDS systems on medication safety.⁶ ⁷ ⁸ ⁹ ¹⁰ ¹¹ The studies included in these reviews overlap, and vary widely in design and results; few include randomized, controlled trials. The heterogeneity of the studies prevented all but one author from conducting a meta-analysis, but these authors found CPOE/CDS systems were associated with a 66% reduction in the odds of an error occurring (OR 0.34; 95% CI 0.22 to 0.52).¹¹ The authors of the most recent review suggest that additional studies of better quality are needed to cover a wide range of clinical and geographic settings, specifically in the ambulatory care setting.¹¹

Six of the studies in these reviews were conducted in the ambulatory setting, and results were mixed.¹⁸ ¹⁹ ²⁰ ²¹ ²² ²³ Five focused on CDS alerts.¹⁸ ¹⁹ ²⁰ ²² ²³ The sixth explored the effect of a CPOE system on all types of errors, but results revealed that reductions in errors and preventable ADEs were not significant.²¹ These studies were conducted in academic settings and used homegrown systems.¹⁸ ¹⁹ ²⁰ ²¹ Thus, generalizability is limited, as most US healthcare is delivered in community-based settings. Further, sample sizes in these studies were small—for example, fewer than 400 CDS events per study arm,¹⁸ fewer than 2.8% of prescriptions electronically prescribed,¹⁹ or 1879 prescriptions reviewed.²¹ Further research is warranted, as investigators estimate that between 11%²⁴ and 28%²⁵ of ADEs that occur in the ambulatory setting are preventable.

We evaluated the effect of a basic CPOE system in the ambulatory setting on medication errors and subsequent ADEs, employing the definition of a medication error and the severity index of the National Coordinating Council for Medication Error Reporting and Prevention (NCC MERP)²⁶ ²⁷ (table 1). We compared pre- to postimplementation errors. We conducted our study at a physician-owned, multispecialty clinic-system in Everett, Washington, located 25 miles north of Seattle. The clinic system is community-based, not affiliated with an academic medical center. It is the largest independent medical group in the State of Washington. In this real-world setting, we evaluated the impact of implementation of a basic CPOE system, with limited CDS alerts, on medication safety. We hypothesized that the CPOE system would be associated with a reduction in errors of all severity levels, and that the types of errors reduced would be those most affected by a basic system (eg, missing information).

Table 1

Definition of a medication error, error severity index

Methods

Setting

The clinic cares for 250

000 patients, at 14 locations, in 60 clinics. Clinicians admit to the one hospital in the local market. The clinic logs 650

000 ambulatory visits annually, and over 400 providers, approximately equally distributed between primary care and specialty practitioners, prescribe 2.7 million prescriptions, at a retail cost of $140 million (2008). At the time of the study, the clinic owned three retail pharmacies and contracted with 18 health plans, each with unique formularies. Employees of a wholly owned information-systems subsidiary developed the clinic's first EHR in 1995. The EHR contains patient scheduling, chart notes (historical), and laboratory and imaging reports. It can be viewed from the local hospital but is not integrated. Staggered roll out of the CPOE system began in 2003. Each prescriber received a minimum of 30 min of one-on-one training. Skill levels were assessed, and subsequent training was offered as additional one-on-one training, just-in-time training, and site-specific group training. The CPOE system is web-based, uses point-and-click functionality, and integrates e-prescribing into the existing EHR. The CPOE system makes use of the drug database from Multum (Cerner, Denver, Colorado). The Multum database is a comprehensive compendium of drugs, listing each by dosage form, strength, and packaging. It generates new and renewed prescriptions. Prescribers select medications from pull-down menus or from “favorites” lists. Directions can be selected or typed as free-text. During the study, the CPOE system included basic dosing guidance, presented in preference lists, and duplicate therapy checks. When the prescriber entered a child's weight, the system also calculated weight-based, pediatric dosing of drug, strength, and bottle size (if liquid medication). The prescriber could either accept or over-ride the recommendation. Allergy, drug–drug interaction, drug–disease interaction, and laboratory monitoring alerts were added after completion of data collection. Clinic staff can queue prescriptions, but only licensed prescribers can sign and release them. Prescriptions can then be printed or electronically faxed to a pharmacy of the patient's choice. In the course of their usual duties, pharmacists routinely screen for errors, calling prescribers when the need for clarification arises. The dispensing pharmacists continued in this role separate from study activities; these prescriptions were filled once errors were clarified.

Study design

We conducted a quasiexperimental, pretest–post-test study to evaluate prescriptions for errors, comparing the rate, types, and severity of errors that occurred before (handwritten) to after implementation (e-prescribed). Prescriptions were included from all clinic sites and all provider specialties. To facilitate identification and retrieval of prescriptions, we limited the dataset to prescriptions filled at the three onsite retail pharmacies. The number of prescriptions retrieved from each pharmacy reflected the proportion filled at each pharmacy, as a proportion of all prescriptions filled at the three clinic pharmacies combined, during a 12-month period. The CPOE system was implemented at Clinic/pharmacy site A in July 2003. At this site, preimplementation prescriptions were written between March 1 and July 15, 2002; postimplementation prescriptions were e-prescribed between January 14 and July 13, 2004. At clinic sites B, C, and all others, the CPOE system was implemented in July 2004. These sites are served by pharmacies B and C. At these sties, preimplementation prescriptions were written between January 2 and March 4, 2004; postimplementation prescriptions were e-prescribed between July 1, 2005 and April 26, 2006. At each site, preimplementation prescriptions were retrieved in reverse chronologic order (pre-) and chronologic order (post-) until the targeted proportion from each clinic pharmacy was reached. All prescriptions within the stated time frames were evaluated. We evaluated new and renewal prescriptions but excluded prescriptions transferred to/from outside pharmacies, as the transmittal process could cause errors. We also excluded prescriptions for devices and laboratory monitoring supplies.

We limited the evaluation to prescribing errors, creating a list of error types a priori.¹⁵ ¹⁶ ²⁸ ²⁹ ³⁰ ³¹ ³² These are: inappropriate abbreviations; missing information; illegibility; wrong- directions, strength, drug, dose, dosage form, patient, physician, or route; allergy; drug–drug interaction; drug–disease interaction; therapeutic duplication; contraindication in patients ≥65 years of age; and lack of appropriate laboratory monitoring. We adapted the therapeutic drug class index of the Department of Veterans Affairs,³³ and used Beer's criteria for patients ≥65 years old.³⁴ A team of pharmacists and physicians created a laboratory monitoring guide by listing frequently prescribed medications and corresponding monitoring parameters, specifying appropriate frequency, and target (or normal) ranges.³⁵ ³⁶ ³⁷ ³⁸ ³⁹ ⁴⁰ All lists were approved by the clinic's pharmacy and therapeutics committee. Using these tools, we created decision rules for evaluation of errors. The University of Washington Human Subjects Committee approved the study under waiver of consent.

Data collection

Between July 27, 2004 and November 18, 2007, two clinical pharmacists independently evaluated 10

169 prescriptions (RNH, KKH). Both were trained by evaluating 100 prescriptions. Their assessments were compared and discrepancies resolved by members of the research team (EBD, JWN, NML, AWF) before full data collection commenced. Decision rules were clarified and codified. These same team members continued to resolve discrepancies throughout the study. Data were recorded on every prescription, whether or not an error was found. We used a computer-based process to conduct the evaluation.⁴¹

Evaluators collected data from three sources: the prescription (handwritten or e-prescribed), clinical information from the EHR, and the prescription as entered into the pharmacy computer system. Because prescriptions were evaluated after their input into the pharmacy dispensing system, comparing the first and third data sources enabled us both to distinguish prescribing errors from errors that occurred during prescription entry into the pharmacy computer system and to consider pharmacist–prescriber prescription clarifications, as these are documented in the pharmacy computer system. To assess for appropriate monitoring, we reviewed clinical data for 6–12 months prior to the date each prescription was written. The duration was driven by disease-state and drug-specific clinical monitoring guidelines. To determine if an error was associated with a subsequent ADE, we reviewed clinical data for 6 months after. We compared inter-rater reliability to the κ statistic.

In addition, in an exploratory effort to identify ADEs caused by medication errors, we independently identified all patients who experienced an inpatient admission within 90 days of each prescription that appeared in the dataset. We defined an ADE as “an injury resulting from a medical intervention related to a drug”.¹⁶ We reviewed the hospital records of patients whose admission, defined by a discharge diagnosis coded using the International Classification of Diseases 9th Revision Clinical Modification system,⁴² could have been medication-induced, regardless of whether we had identified a prescribing error. A list of the implicated ICD-9 codes, and the corresponding medications, is available from the authors. Using the Naranjo algorithm,⁴³ we assessed the probability that the prescription medication could have contributed to the admission.

Statistical analyses

The prescription was the unit of analysis. The primary outcome was whether or not an error occurred; secondary outcomes were error types or severity. The predictor was the presence of the CPOE system. To make unadjusted comparisons, we used the two-sample test of proportions. To make adjusted comparisons of the effect of the CPOE system on the occurrence of an error (yes/no) and on error type (one of 17 types), we first used alternating logistic regression,⁴⁴ accounting for clustering at the geographic clinic site and prescriber (242 prescribers) levels. As the α level for geographic site clustering was not significant, we then clustered solely at the prescriber level and used generalized estimating equations (GEE) with an independent correlation structure, adjusting for geographic site as a fixed effects variable. We also adjusted for prescriber specialty, patient age and gender, therapeutic drug class, and interaction terms between e-prescribing and each covariate. To control for seasonal variations in prescribing we added a dummy variable that represented each season of the year (fall, winter, spring, or summer) prescribed. We accounted for secular trends by specifying the number of weeks between prescribing of the first prescription, and that undergoing review. We created a best-fitting model by retaining cluster variables and covariates with p values <0.05 (Model 1). If at least one in a group of variables was significant, we retained the group. For infrequently occurring errors, we used reduced models to achieve convergence (Models 2 and 3). When comparing error types, we used the Bonferroni correction to adjust for multiple comparisons, and considered p values <0.005 significant.

We created an inverse probability weighting covariate so that the dataset would better reflect the prescribing patterns of all prescriptions written by clinic providers, regardless of whether these prescriptions were filled on site. The weights adjusted for provider specialty and therapeutic drug class, and were stratified by the onsite pharmacy from which each prescription was retrieved. Details about these weights are available from the authors. Based on our pilot data, using an intraclass correlation coefficient of 0.02 and a two-sided α, we calculated 90% power to detect a 20% decrease in errors from a baseline rate of 25%.

For the severity outcome, we collapsed the NCC MERP categories to better reflect errors wherein harm ensued, in the manner of Snyder et al⁴⁵ (table 1). For this comparison, we used a generalized linear latent and mixed effects model (GLLAMM), specified a multinomial logit link, and adjusted for the same covariates as in the GEE model. Analyses were conducted in SAS 9.1 and Stata 10.1.

Results

Patient and prescription characteristics appear in table 2. Each reviewer evaluated 5016 prescriptions pre- and 5153 postimplementation. The κ for inter-rater reliability was 0.62 (93% agreement) at the error-found level, and 0.7 (97% agreement) at the severity level. The frequency of errors declined from 18.2% to 8.2% with use of the CPOE system, an unadjusted reduction of 55%. The adjusted odds of an error occurring postimplementation was 70% lower than pre-implementation (OR: 0.30; 95% CI 0.23 to 0.40; p<0.001) (table 3).

Table 2

Characteristics of patients and prescriptions

Table 3

Impact of the computerized provider order entry (CPOE) system on medication errors

Types of errors are presented in order of decreasing frequency in the preimplementation period (table 4). The greatest reduction in odds occurred with illegibility (97%), followed by inappropriate abbreviations (94%) and information missing (85%) errors. Also significant were reductions in odds of wrong strength (81%), drug–disease interaction (79%), and drug–drug interaction (76%). errors. Non-significant reductions in odds occurred with wrong drug errors (63%), drugs that were contraindicated in patients ≥65 years of age (56%), and wrong directions errors (34%). There was no decrease in the proportion of errors attributed to lack of appropriate laboratory monitoring, therapeutic duplications, or wrong dose. The numbers of errors in remaining categories were small.

Table 4

Impact of the computerized provider order entry (CPOE) system on medication errors, by error type

Table 5 summarizes prescriptions by error severity, using collapsed categories that represent the potential to cause an ADE.⁴⁵ E-prescribing was associated with a significant, 57% reduction in the odds of an error occurring that did not cause harm, potential ADEs (levels B–D). Level B errors, which by definition did not reach the patient, decreased from 445 to 84 (8.9% to 1.6%). There was a 49% reduction in odds of errors that caused harm, actual ADEs (levels E and F); this was statistically not significant, perhaps due to the small number of errors in this category. The one level F error occurred preimplementation.

Table 5

Impact of the computerized provider order entry (CPOE) system on medication errors, by severity

The results of the best-fitting model appear in table 6. In addition to e-prescribing being associated with a 70% reduction in the odds of an error occurring, the results revealed that prescriptions for patients ≥65 (vs <65) years were more likely to be associated with an error, whereas those written for antibiotics were less likely to be associated with an error than other classes. The interaction terms were significant for hormones, Schedule II–V drugs, and Clinic C. We retained the covariates of antidepressants, and the interaction term between e-prescribing and central nervous system (CNS) agents (and therefore CNS agents), as these were significant in the full model. We retained Clinic sites A and B to keep this group together; and female, for convention. In the full model, the occurrence of an error did not differ over provider type, season of the year during which the prescription was written, or the number of weeks since the first prescription was written; thus, these do not appear in table 6.

Table 6

Impact of the computerized provider order entry (CPOE) system on medication errors, description of multivariate model

The Appendix (online at http://www.jamia.org/) describes the 14 preventable ADEs (levels E and F) found during prescription evaluation. Four of these were attributed to lack of appropriate laboratory monitoring, three to drug–disease interactions, three to wrong directions, and two to wrong dose. For the exploratory ADE analysis wherein we set out to pair a hospital admission with a prescription in the dataset, we identified 59 prescriptions (0.6%) for which the medication prescribed could be associated with the discharge ICD-9 diagnosis. We reviewed the discharge notes of these patients and applied the Naranjo algorithm.⁴³ In all cases, the score on the Naranjo scale was 2 points, out of a possible total of 13 points—indicating a “possible” association between the medication and admission. Indeed, the only criterion that was met was that of temporal association, and this by study design—each prescription was written prior to admission. With only five of these 59 prescriptions was an error found during the prescription review process; all were of severity level C.

Discussion

Our results suggest that implementation of a basic CPOE system in the ambulatory setting is associated with a significant reduction in medication errors. Consistent with our hypothesis, the types of errors most easily mitigated by a basic CPOE system were reduced. Elimination of illegibility is inherent in the e-prescribing process. Forcing routine functions minimizes the use of inappropriate abbreviations and missing information. Pull-down menus mitigate other “wrongs” (eg, wrong strength). Despite the fact that our CPOE system lacked CDS alerts, reductions in numbers of drug–disease and drug–drug interaction errors were significant, although these occurred infrequently at baseline. Types of errors we would have thought would have occurred frequently with handwritten prescribing (eg, drug–drug interactions) occurred in ≤2% of prescriptions. Perhaps, due to our thorough evaluation methods, our evaluators were able to discern clinically meaningful versus theoretically relevant contraindications, interactions, and allergies. That the CPOE system did not affect the number of inappropriate laboratory monitoring errors suggests that CDS alerts are necessary to further decrease some of the more complex types of errors.

Implementation of the CPOE system was associated with a reduction in the number of errors at each severity level. Importantly, use of the CPOE system was associated with fewer errors that reached the patient (level B). Our model reveals that patients ≥65 years experienced more errors, but these were not associated with e-prescribing.

We used a MEDLINE search from 1997 to December 2008 to identify studies similar to ours. Bates' group was one of the first to demonstrate a significant reduction in serious errors (categories C through I) associated with an inpatient CPOE system, from 10.7 to 4.86 events per 1000 patient days (55% reduction; p<0.01). They noted a reduction in potential ADEs (C and D) from 5.99 to 0.98 events (84% reduction; p=0.002), and in actual ADEs of 4.69 to 3.88 (17% reduction; p=0.37).¹⁴ In a follow-on study, they noted an 86% reduction in serious medication errors (p<0.001), using an advanced CPOE/CDS system.¹⁵ Our results demonstrate a similar overall reduction rate of 55%, although our reductions in potential (9.8%) and actual (0.1%) ADEs were lower. These comparisons are not exact, as Bates' group excluded errors in categories A and B, while we included these. Our 70% reduction in odds of an error is similar to that noted in Shamliyan's meta-analysis.⁹

Of the investigations that have taken place in the ambulatory setting,¹¹ ¹⁸ ¹⁹ ²⁰ ²¹ ²² ²³ only one is comparable.²¹ Gandhi retrospectively compared 1879 prescriptions that were handwritten versus e-prescribed using a basic CPOE system, and noted a reduction in errors (11.0% to 4.3%; p=0.31) and potential ADEs (4.0% to 2.6%; p=0.16). Although their results were not significant, the investigators suggested that CDS alerts could have prevented 97% of errors and 95% of potential ADEs. Our corresponding reductions were significant for errors, 18.0% to 8.2% (levels B–F; p<0.001), and for potential ADEs, 17.8% to 8.1% (B–D; p<0.001). Our rates are higher, but reductions similar in magnitude.

Jha et al found that 1.4% of hospital admissions were due to ADEs and that 28% of these were preventable.⁴⁶ Our rate of preventable ADEs was also <1%. Steele et al's pre-, postevaluation assessed the effect on ordering behavior of drug-laboratory CDS alerts in the ambulatory setting.²³ Applying the Naranjo algorithm,⁴⁵ these investigators noted a trend toward fewer “definite” or “probable” ADEs (4.3% vs 10.3%; p=0.23). Using the Naranjo algorithm we found only “possible” associations between admissions and preventable ADEs, and these were tenuous, at best.

Although evidence suggests that well-designed CPOE/CDS systems can reduce error rates and improve care, most errors do not cause harm. However, even infrequently occurring preventable ADEs are unacceptable. Further, the benefits of CPOE/CDS systems are not limited to clinical outcomes; there are also cost implications. The cost of a preventable ADE in older adults has been conservatively estimated at $1983; a national average cost of $887 million (both 2000 $).⁴⁷ These costs include inpatient stays (62%), emergency department visits (6%), outpatient care and physician fees (28%), and prescribed medications (4%). There are also costs associated with medication errors: prescriptions written without an indication, diseases not being treated, lack of adherence, and staff time spent preventing errors from becoming ADEs. These have yet to be estimated.

Some have noted that improved practitioner performance was associated with automatic prompts¹⁸ and homegrown software⁷ ¹¹; that RR reductions were larger when e-prescriptions were compared with those that were handwritten,¹¹ and when the rate of errors with handwritten prescriptions was >12%.⁹ Others noted greater effects in studies that used a manual chart review to detect errors.¹¹ Our system was homegrown, our baseline error rate of handwritten prescriptions was 18%, and our methods included chart review. Perhaps these factors contributed to the magnitude of our error reduction rate.

Our study describes the benefits of a homegrown CPOE system in a community setting in the Northwest. To our knowledge, this is one of the largest studies conducted to date that evaluates this impact. During the study time frame, no other medication safety initiatives were implemented at any site within the clinic. Our sample size gave us power to detect a reduction in errors, although not in preventable ADEs. We limited our postimplementation analysis to only e-prescriptions. Prescribers still had the option of handwriting prescriptions, although few did, as adoption was rapid, and we allowed a 6-month lag time to achieve stability before evaluating prescriptions written postimplementation. Approximately 10% of prescriptions were handwritten at this juncture; these were written by prescribers across clinic sites and specialties. We could not blind our prescription reviewers as they viewed the actual prescriptions, either handwritten or e-prescribed. To achieve study feasibility, we limited our evaluation to prescriptions filled at the three pharmacies owned by The Everett Clinic. These prescriptions may differ from those filled elsewhere. Our analysis methods included a weighting variable to address this limitation. Ours was not a randomized trial, as is evidenced by some of the differences in characteristics of patients, prescribers, and prescriptions illustrated in table 2. To address this limitation we used analytic methods that accounted for clustering on prescriber, effect modification of therapeutic drug class or site, and observable confounding. Finally, our inpatient chart review methods enabled us to capture information about ADEs, although our methods were not ideal for finding definitive links between medications and subsequent hospital admissions.

Research that further illuminates the patient safety benefits of CPOE/CDS systems is ongoing. Evidence that establishes these benefits is substantial for homegrown systems in the inpatient setting, but is not as robust in the ambulatory setting. Our work contributes to this knowledge base in the latter setting.

Conclusion

The results of our study, which demonstrate the association between implementation of a CPOE system and a reduction in medication errors, indicate that even a basic CPOE system, without CDS alerts, can have a favorable impact on medication safety. That this work was conducted in an independent medical group practice suggests that CPOE implementation is associated with improved medication safety in the real-world setting.

Supplementary Material

Web Only Data

Click here to view.

Acknowledgments

The authors gratefully acknowledge the assistance of the following individuals who made valuable contributions to the project: D Meyers, for his project guidance and support; W Hollingworth, for his assistance with study design; P Tarczy-Hornoch and TH Payne, for expertise as prescribers and as health informaticians; K Kelly-Haney, S Milson, S Te, KL Tharp, and C Wong, for assistance with data collection. The authors thank the reviewers for their thoughtful and thorough reviews, and their helpful comments, which have greatly strengthened the manuscript.

Footnotes

Funding: This research was supported by the Agency for Healthcare Research and Quality (AHRQ): Health Information Technology Grant #: 5-UC1 HS015319 (PI: Sullivan); Mentored Clinical Scientist Training Grant #: 5-K08-HS014739 (PI: Devine).

Competing interests: None.

Provenance and peer review: Not commissioned; externally peer reviewed.

References

1. Dick RS. The computer-based patient record: an essential technology for health care. Institute of Medicine. Washington: National Academy Press, 1991.

2. Institute of Medicine Crossing the Quality Chasm: the IOM healthcare quality initiative. Washington: National Academy Press, 2008. http:www.iom.edu/?id=35957 (accessed 3 May 2009).

3. Institute of Medicine To err is human. Washington: National Academy Press, 1999.

4. Institute of Medicine Preventing medication errors: Quality Chasm series. Washington: National Academy Press, 2006.

5. Bates DW, Ebell M, Gotlieb E, et al. A proposal for electronic medical records in US primary care. J Am Med Inform Assoc 2003;10:1–10. [PMC free article] [PubMed]

6. Kaushal R, Shojania KG, Bates DW. Effects of computerized physician order entry and clinical decision support systems on medication safety: a systematic review. Arch Intern Med 2003;163:1409–16. [PubMed]

7. Garg AX, Adhikari NK, McDonald H, et al. Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review. JAMA 2005;293:1223–38. [PubMed]

8. Eslami S, Abu-Hanna A, de Keizer NF. Evaluation of outpatient computerized physician medication order entry systems: a systematic review. J Am Med Inform Assoc 2007;14:400–6. [PMC free article] [PubMed]

9. Shamliyan TA, Duval S, Du J, et al. Just what the doctor ordered. Review of the evidence of the impact of computerized physician order entry system on medication errors. Health Serv Res 2008;43(1 Pt):32–53. [PMC free article] [PubMed]

10. Wolfstadt JI, Gurwitz JH, Field TS, et al. The effect of computerized physician order entry with clinical decision support on the rates of adverse drug events: a systematic review. J Gen Intern Med 2008;23:451–8. [PMC free article] [PubMed]

11. Ammenwerth E, Schnell-Inderst P, Machan C, et al. The effect of electronic prescribing on medication errors and adverse drug events: a systematic review. J Am Med Inform Assoc 2008;15:585–600. [PMC free article] [PubMed]

12. Hook J, Cusack C. Ambulatory computerized provider order entry (CPOE): findings from the AHRQ Health IT portfolio. Center for information technology leadership; AHRQ National Resource Center for Health Information Technology. AHRQ Publication No. 08-0063-EF. Rockville: Agency for Healthcare Research and Quality, 2008.

13. Kuperman GJ, Bobb A, Payne TH, et al. Medication-related clinical decision support in computerized provider order entry systems: a review. J Am Med Inform Assoc 2007;14:29–40. [PMC free article] [PubMed]

14. Bates DW, Leape LL, Cullen DJ, et al. Effect of computerized physician order entry and a team intervention on prevention of serious medication errors. JAMA 1998;280:1311–16. [PubMed]

15. Bates DW, Teich JM, Lee J, et al. The impact of computerized physician order entry on medication error prevention. J Am Med Inform Assoc 1999;6:313–21. [PMC free article] [PubMed]

16. Bates DW, Boyle DL, Vander Vliet MB, et al. Relationship between medication errors and adverse drug events. J Gen Intern Med 1995;10:199–205. [PubMed]

17. Evans RS, Pestotnik SL, Classen DC, et al. A computer-assisted management program for antibiotics and other antiinfective agents. N Engl J Med 1998;338:232–8. [PubMed]

18. McDonald CJ. Protocol-based computer reminders, the quality of care and the non-perfectability of man. N Engl J Med 1976;295:1351–5. [PubMed]

19. McDonald CJ, Wilson GA, McCabe GP, Jr, et al. Physician response to computer reminders. JAMA 1980;244:1579–81. [PubMed]

20. Rotman BL, Sullivan AN, McDonald TW, et al. A randomized controlled trial of a computer-based physician workstation in an outpatient setting: implementation barriers to outcome evaluation. J Am Med Inform Assoc 1996;3:340–8. [PMC free article] [PubMed]

21. Gandhi TK, Weingart SN, Seger AC, et al. Outpatient prescribing errors and the impact of computerized prescribing. J Gen Intern Med 2005;20:837–41. [PMC free article] [PubMed]

22. Tamblyn R, Huang A, Perreault R, et al. The medical office of the 21st century (MOXXI): effectiveness of computerized decision-making support in reducing inappropriate prescribing in primary care. CMAJ 2003;169:549–56. [PMC free article] [PubMed]

23. Steele AW, Eisert S, Witter J, et al. The effect of automated alerts on provider ordering behavior in an outpatient setting. PLoS Med 2005;2:e255. [PMC free article] [PubMed]

24. Gandhi TK, Weingart SN, Borus J, et al. Adverse drug events in ambulatory care. N Engl J Med 2003;348:1556–64. [PubMed]

25. Gurwitz JH, Field TS, Harrold LR, et al. Incidence and preventability of adverse drug events among older persons in the ambulatory setting. JAMA 2003;289:1107–16. [PubMed]

26. National Coordinating Council for Medication Error Reporting and Prevention: about medication errors. http://www.nccmerp.org/aboutMedErrors.html (accessed 3 May 2009).

27. National Coordinating Council for Medication Error Reporting and Prevention: index for categorizing medication errors. http://www.nccmerp.org/medErrorCatIndex.html (accessed 3 May 2009).

28. Bates DW, Cohen M, Leape LL, et al. Reducing the frequency of errors in medicine using information technology. J Am Med Inform Assoc 2001;8:299–308. [PMC free article] [PubMed]

29. Leape LL. Error in medicine. JAMA 1994;272:1851–7. [PubMed]

30. Leape LL, Bates DW, Cullen DJ, et al. ADE Prevention Study Group Systems analysis of adverse drug events. JAMA 1995;274:35–43. [PubMed]

31. Lesar TS. Prescribing errors involving medication dosage forms. J Gen Intern Med 2002;17:579–87. [PMC free article] [PubMed]

32. McNutt RA, Abrams R, Arons DC.; for the Patient Safety Committee Patient safety efforts should focus on medical errors. JAMA 2002;287:1997–2001. [PubMed]

33. Department of Veterans Affairs Pharmacy benefits management services. VA National Formulary; http://www.pbm.va.gov/default.aspx (accessed 3 May 2009).

34. Fick DM, Cooper JW, Wade WE, et al. Updating the Beers criteria for potentially inappropriate medication use in older adults: results of a US consensus panel of experts. Arch Intern Med 2003;163:2716–24. [PubMed]

35. Bakerman S, Bakerman P, Strausbauch P. Bakerman's ABC's of interpretive laboratory data. 4th edn Scottsdale: Interpretive Laboratory Data, Inc., 2002.

36. DrugDex System Micromedex Healthcare series [internet database]. Greenwood Village: Thomson Healthcare, 2009. http://www.thomsonhc.com/hcs/librarian/ (accessed 3 May 2009).

37. Drug facts and comparisons Clin e-guide [internet database]. New York: Wolters Kluwer Health, 2008. http://clinicalresource.ovid.com/autologin.html (accessed 3 May 2009).

38. Fuster V, Alexander RW, O'Rourke RA, et al. Hurst's the heart. 11th edn New York: McGraw-Hill, 2004.

39. Ansell J, Hirsh J, Poller L, et al. The pharmacology and management of the vitamin K antagonists: the Seventh ACCP Conference on Antithrombotic and Thrombolytic Therapy. Chest 2004;126(Suppl 3):204S–33S. [PubMed]

40. Dipiro JT, Talbert RL, Yee GC, et al. Pharmacotherapy: a pathophysiologic approach. 6th edn New York: McGraw-Hill, 2005.

41. Devine EB, Wilson-Norton JL, Lawless NM, et al. Preparing for ambulatory computerized provider order entry by evaluating preimplementation medication errors. In: Henriksen K, Battles JB, Marks ES, et al., editors. , eds. Advances in patient safety: from research to implementation. Vol. 2 Concepts and methodology. AHRQ Publication No. 05-0021-2. Rockville: Agency for Healthcare Research and Quality, 2005. [PubMed]

42. Stat!Ref Online Electronic Medical Library [internet database]. Vols. 1–3 6th edn Jackson, WY: Teton Data Systems, 2009. [Posted 9 Oct 2008] ICD-9-CM. http://online.statref.com/TOC.aspx?grpalias=UofWash (accessed 5 May 2009).

43. Naranjo CA, Busto U, Sellers EM, et al. A method for estimating the probability of adverse drug reactions. Clin Pharmacol Ther 1981;30:239–45. [PubMed]

44. Carey V, Zeger SL. Modelling multivariate binary data with alternating logistic regression. Biometrika 1993;80:517–26.

45. Snyder RA, Abarca J, Meza JL, et al. Reliability evaluation of the adapted national coordinating council medication error reporting and prevention (NCC MERP) index. Pharmacoepidemiol Drug Saf 2007;16:1006–13. [PubMed]

46. Jha AK, Kuperman GJ, Rittenberg E, et al. Identifying hospital admissions due to adverse drug events using a computer-based monitor. Pharmacoepidemiol Drug Saf 2001;10:113–19. [PubMed]

47. Field TS, Gilman BH, Subramanian S, et al. The costs associated with adverse drug events among older adults in the ambulatory setting. Med Care 2005;43:1171–6. [PubMed]

Articles from Journal of the American Medical Informatics Association : JAMIA are provided here courtesy of
American Medical Informatics Association