|Home | About | Journals | Submit | Contact Us | Français|
To examine the association between computerization of physician orders and prescribing medication errors.
Studies published in English language were identified through MEDLINE (1990 through December 2005), Cochrane Central Register of Controlled Trials, and bibliographies of retrieved articles. Of 252 identified in the search, 12 (4.8 percent) original investigations that compared rates of prescribing medication errors with handwritten and computerized physician orders were included.
Information on study design, participant characteristics, clinical settings, and outcomes rates were abstracted independently by two investigators using a standardized protocol.
Compared with handwritten orders, 80 percent of studies (8/10 studies) reported a significant reduction in total prescribing errors, 43 percent in dosing errors (3/7 studies), and 37.5 percent in adverse drug events (3/8 studies). The use of computerized orders was associated with a 66 percent reduction in total prescribing errors in adults (odds ratio [OR]=0.34; 95 percent confidence interval [CI] 0.22–0.52) and a positive tendency in children (p for interaction=.028). The benefit of computerized orders was larger when the rate of errors was more than 12 percent with handwritten orders (p for interaction=.022). Significant heterogeneity in the results compromised pooled relative risks. One randomized controlled intervention demonstrated the greatest benefits of computerized orders on total prescribing errors (OR=0.02, 95 percent CI 0.01–0.02) and dosing errors (OR=0.28; 95 percent CI 0.15–0.52) with 775 avoided prescribing errors (95 percent CI 752–811) per 1,000 orders in a pediatric hospital.
Computerization of physicians' orders shows great promise. It will be more effective when linked to other computerized systems to detect and prevent prescribing errors.
Medication errors are associated with substantial death and injury (Kohn, Corrigan, and Donaldson 2000). More than half a million patients are injured or die each year in hospitals from adverse drug events (ADEs) (Classen et al. 1997) which may cost up to $5.6 million annually per hospital (Bates et al. 1997). In 1992, 5 percent of hospitalized patients experienced medication errors (Bond, Raehl, and Franke 2001). The rate increased substantially to 24 percent from 1995 to 2000 (LaPointe and Jollis 2003).
Physician ordering and transcription were responsible for 50–61 percent of all medication errors (Bates et al. 1995b; Leape et al. 1995). Figure 1 outlines the relationship between medication errors and ADEs. Computerized Physician Order Entry (CPOE) systems were expected to dramatically reduce error rates (Bates et al. 1995b). The Institute of Medicine has identified medication errors as a major threat to patient safety and has endorsed electronic prescribing as a promising approach to combating them (Board on Health Care Services [HCS] et al. 2006). One systematic review reported that CPOE substantially reduced medication errors but not clinical adverse events resulting from errors (Kaushal, Shojania, and Bates 2003).
The implementation of CPOE has not always improved patient safety to expected levels (Han et al. 2005; Mirco et al. 2005; Nebeker et al. 2005; Del Beccaro et al. 2006). The clinical effects of CPOE on medication errors and clinical events have been studied previously without quantitative comparative effectiveness in different clinical settings (Kaushal, Barker, and Bates 2001a; Pape 2001; Wrobel 2003; Hughes and Edgerton 2005; Board on HCS et al. 2006; Jones and Moss 2006; Riskin, Shiff, and Shamir 2006; Joanna Briggs Institute 2006). The present review analyzes published evidence to test the hypothesis that medication errors and adverse clinical events decrease after computerization compared with handwritten physician orders in pediatric and adult patients independent of patient and provider characteristics.
The protocol was created according to the recommendations for Meta-analysis of Observational Studies in Epidemiology (MOOSE) (Stroup et al. 2000). Electronic databases, including Medline, Cochrane Central Register of Controlled Trials, WorldCat, and Internet engines (http://www.Google.com), were searched using MeSH headings and keywords “Medical Records Systems, Computerized,”“Medication Errors,”“Medical Order Entry Systems,” and their combinations. Eligible studies were defined as original observational or interventional investigations published in English from 1990 to 2005.
The level of evidence was estimated using the U.S. Preventive Services Task Force criteria (Harris et al. 2001). Reviews, expert opinions, studies focused on long-term care, and studies comparing different degrees of decision support system in CPOE were excluded. The standardized abstraction protocol (The Cochrane Collaboration 2005) included the first author, year and journal of the publication, target population, clinical settings, randomization in intervention and random sampling in observational investigations, assessment of outcomes, adjustment for confounding factors, and outcomes rates using handwritten and computerized orders.
Pooled odds ratios (ORs) and 95 percent confidence intervals (CIs) were calculated with fixed and random effects models (DerSimonian and Laird 1986). Independent variables were defined as physician order systems (computerized versus handwritten), target population, and clinical settings. Dependent variables were defined as medication errors and adverse events rates among total prescriptions and per 1,000 patient days. Meta-regression models analyzed possible interactions with the year of publication, presence of a control group versus pre- and post-CPOE comparisons, target population (pediatric versus adults), clinical settings (acute versus chronic care), and rate of outcomes in the group with handwritten orders. Publication bias was assessed with Egger's test (Egger et al. 1997), a modification of Macaskill's test (Peters et al. 2006), and the Trim and Fill method (Duval and Tweedie 2000). Heterogeneity between studies was quantified using the I2 statistic (Higgins et al. 2003). Statistical significance was analyzed at the 95 percent confidence level. All calculations were conducted using STATA software (Egger 2001).
From 252 articles included in the initial database (Appendix A), 12 studies reported rates of prescribing errors after CPOE and handwritten orders (Bates et al. 1998, 1999; Teich et al. 2000; Bizovi et al. 2002; Fontan et al. 2003; King et al. 2003; Cordero et al. 2004; Potts et al. 2004; Gandhi et al. 2005; Oliven et al. 2005; Shulman et al. 2005; Upperman et al. 2005), seven compared the effects of CPOE with different decision support systems (Chertow et al. 2001; Mekhjian et al. 2002; Overhage et al. 2002; Colombet et al. 2004; Galanter, Polikaitis, and DiDomenico 2004; Galanter, Didomenico, and Polikaitis 2005; Koppel et al. 2005; Mirco et al. 2005), and four analyzed the likelihood of preventing medication errors (Bates et al. 1995a; Kaushal et al. 2001b; Fortescue et al. 2003; Bobb et al. 2004). Among 12 studies eligible for meta-analysis (Bates et al. 1998, 1999; Teich et al. 2000; Bizovi et al. 2002; Fontan et al. 2003; King et al. 2003; Cordero et al. 2004; Potts et al. 2004; Gandhi et al. 2005; Oliven et al. 2005; Shulman et al. 2005; Upperman et al. 2005), three studies reported rates of medication errors per 1,000 patients days (Bates et al. 1998, 1999; King et al. 2003), and the others provided a number of errors among total prescriptions. Three out of 12 studies analyzed errors related to a wrong drug (Fontan et al. 2003; Potts et al. 2004; Shulman et al. 2005) and seven out of 12 reported errors related to a wrong dose (Teich et al. 2000; Bizovi et al. 2002; Fontan et al. 2003; Cordero et al. 2004; Potts et al. 2004; Oliven et al. 2005; Shulman et al. 2005) (Table 1). Total adverse events rates were obtained from a pooled estimation of eight studies (Bates et al. 1998, 1999; Bizovi et al. 2002; King et al. 2003; Potts et al. 2004; Gandhi et al. 2005; Shulman et al. 2005; Upperman et al. 2005) (Table 2).
Most studies defined CPOE as “direct entry of medical orders into the computer” (Ash et al. 2004). However, within this definition, authors reported different CPOE software packages with and without a decision-making support system (Appendix B). Several studies conducted at the Brigham and Women's Hospital used the Brigham Integrated Computing System (BICS) (Bates et al. 1998, 1999; Teich et al. 2000; Chertow et al. 2001). Potts et al. (2004) analyzed the effects of software (WizOrder) created and implemented by the faculty at Vanderbilt University. Other investigators used a vendor-based system modified for adult and pediatric patients (Mekhjian et al. 2002; Cordero et al. 2004). We could not quantify the functionality of the CPOE software used and assumed that any differences would not modify the impact on outcomes.
The majority of the studies used the definition of medication errors developed by the American Society of Health-System Pharmacists (No authors listed 1998). Medication errors were defined as “any preventable event that may cause or lead to inappropriate medication use or patient harm while the medication is in the control of the health care professional, patient, or consumer.” Prescribing errors are the most common errors and include wrong drug, form, dosage or frequency; wrong route; and contraindicated drug use and interaction (Fontan et al. 2003). Errors in administering drugs, a subcategory of medication errors, include omission of a dose; and wrong dose, route, or time. Bates et al. (1998) further categorized medication errors and adverse events by severity and harm for patients. Potts et al. (2004) defined rules violations as errors not compliant with standard hospital practices (e.g., abbreviations). An ADE was defined as “an injury from a medicine (or lack of an intended medicine).” The U.S. Pharmacopeia Convention classified ADEs by length and severity (Santell et al. 2003). We included in the meta-analyses rates of total prescribing medication errors, of wrong dose and wrong drug errors, and of ADEs.
Only one intervention study randomly allocated patients to CPOE or handwritten orders (Fontan et al. 2003). Few authors used random sampling of patients (Bates et al. 1998) or physician practices (Weingart et al. 2003) to minimize selection bias and increase the external validity of studies. Two prospective studies had formal control groups (Gandhi et al. 2005; Oliven et al. 2005) (Table 1). Adjustment for confounding by patient and provider characteristics was reported in one study (Bizovi et al. 2002).
Macaskill's and Egger's tests for publication bias were not statistically significant for any of the outcomes. The Trim and Fill method indicated a modest amount of publication or small study bias in dosing errors, but without any practical impact on the pooled estimate. There was no apparent bias for medication errors. For adverse events, funnel plot asymmetry was absent if the large study (King et al. 2003) was omitted. The Trim and Fill method is recommended as a tool for sensitivity analysis, and in this case should be interpreted with some caution due to both the small number of studies and the large amount of heterogeneity between studies (Terrin et al. 2003).
We calculated the ORs of medication errors for each study to enable inclusion of all studies with differential reporting of outcomes (Figure 2). All studies reported a reduction in medication errors after CPOE. The strength of the association differed among studies with 98.6 percent of variance between studies attributable to heterogeneity (p<.0001). The presence of a control group versus before–after comparisons, clinical setting, and inclusion of not intercepted errors did not influence the association between CPOE and medication errors. However, two factors significantly modified the effects of CPOE: the rate of medication errors in the group with handwritten physician orders (p for interaction=.022) and target population (p for interaction=.028). The use of CPOE was associated with a 66 percent reduction in medication errors in adults (OR=0.34; 95 percent CI 0.22–0.52) and a similar effect in children, although not statistically significant in a pooled analysis (OR=0.31; 95 percent CI 0.09–1.02) (Table 3). The effect of CPOE was larger when the rate of medication errors was more than 12 percent (highest 20 percent of the distribution) with handwritten physician orders.
The calculated “number needed to treat” and “number of orders needed to prescribe with CPOE to prevent one event” for each study with significant ORs are shown in Table 3. Heterogeneity remained significant in subgroups of adult and children populations and in studies with different baseline rates of medication errors with more than 70 percent of the variation attributable to differences between individual studies. Therefore, pooled ORs are not reliable and valid estimates of effects of CPOE and clinical decisions should be made based on results from individual studies. The greatest benefit of CPOE was demonstrated in one randomized controlled trial with 775 avoided medication errors (95 percent CI 752–811) per 1,000 orders (Fontan et al. 2003).
The rate of prescribing a wrong drug did not decrease after CPOE. Medication errors related to a wrong dose were significantly reduced in three studies with nonsignificant changes in four other studies (Table 3). The results of individual studies varied significantly (p for heterogeneity <.001). A greater decrease in dose errors was observed in studies with a control group compared with pre- and post-CPOE analyses (p for interaction <.001). Randomized intervention demonstrated a 72 percent reduction in risk of wrong dose medication errors using CPOE (OR=0.28; 95 percent CI 0.15–0.52) with 19 avoided medication errors per 1,000 orders.
Total adverse events were lessened after CPOE in three studies (Table 3) with a positive but not statistically significant tendency to reduce ADE in four studies. The test for heterogeneity was significant (p<.001) and could not be explained with meta-regression: neither year of publication (p=.9), the presence of control group (p=.8), the target population (p=.6), nor clinical setting (p=.9) affected the association between CPOE and ADE. The use of CPOE would prevent 9 ADEs per 1,000 prescriptions in pediatric (Potts et al. 2004) and 12 ADEs per 1,000 prescriptions in adult population (Shulman et al. 2005).
The present review of a randomized trial (Fontan et al. 2003) (level of evidence I) and several uncontrolled interventions (Bates et al. 1998; Teich et al. 2000; Cordero et al. 2004; Potts et al. 2004; Upperman et al. 2005) and observational studies (Bizovi et al. 2002; Gandhi et al. 2005; Oliven et al. 2005; Shulman et al. 2005) (levels of evidence II2–II3) confirmed previous contentions (Kaushal, Shojania, and Bates 2003; Koppel 2005) that implementation of CPOE was associated with a significant reduction in medication errors in adult and pediatric populations. The observed heterogeneity in results, not explained by known population and design characteristics, did not provide a valid quantitative estimation of relative risk. The statistical tests did not detect publication bias but the results can be misleading when the number of the studies are small and between-study heterogeneity is significant. (Terrin et al. 2003; Lau et al. 2006) These tests cannot eliminate the possibility of publication bias, but given the interest in this topic it seems likely that negative findings would be reported. Despite a significant reduction in medication errors in most studies, the results should be interpreted with caution. Nonrandomized uncontrolled interventions may provide biased overestimated effects of CPOE (Ioannidis et al. 2001) that are difficult to use for evidence-based decision making (Schulz et al. 1995).
In addition to previous reviews, we statistically examined the impact of study design and clinical settings on comparative effectiveness of CPOE. The beneficial effect of CPOE was larger in studies with greater baseline rates of medication errors in both the present analysis and in previous publications (Choi et al. 1992). The risk of harmful events related to medication errors was three time higher in children compared with adults with the same rate of medication errors (Fortescue et al. 2003), although the effects of CPOE was less potent in pediatric settings. The use of CPOE eliminated medication errors in specialized units (Mekhjian et al. 2002; Cordero et al. 2004) that prescribed fewer groups of drugs, and therefore had a higher probability of positive effects from the interventions. Time of exposure, geographical locations, academic affiliation of the clinics, and changes in drug coverage have been identified as possible effect modifiers and should be included into future analysis (Choi et al. 1992).
The use of CPOE was not associated with a substantial improvement in patient safety (Kaushal, Shojania, and Bates 2003; Oren, Shaffer, and Guglielmo 2003). Nonrandomized controlled interventions examined the effects of CPOE on preventable adverse events associated with medication errors. Two studies showed a significant reduction in potential ADEs in pediatric populations, by 43 percent in an intensive care unit (Potts et al. 2004) (OR=0.57, 95 percent CI 0.44–0.75) and by 76 percent in medical and surgical wards (King et al. 2003) (OR=0.24; 95 percent CI 0.24–0.25). Some authors reported reduction by 84–100 percent in intercepted (prevented) adverse events in adults (Bates et al. 1998, 1999). Differences in definition of outcomes and in interception strategies to prevent patient adverse events (Figure 1) may bias the results of individual studies and pooled analysis.
The variability in CPOE systems complicated comparing the results (Appendix B). Although more than two-thirds (7/10) of the products included in the meta-analysis have incorporated a clinical decision support system into CPOE (Bates et al. 1998, 1999; Teich et al. 2000; Bizovi et al. 2002; Fontan et al. 2003; Cordero et al. 2004; Potts et al. 2004; Gandhi et al. 2005; Oliven et al. 2005), they vary in the degree of decision support capacities. The majority (Bates et al. 1998, 1999; Teich et al. 2000; Bizovi et al. 2002; Fontan et al. 2003; Gandhi et al. 2005; Oliven et al. 2005) provide defaults, acceptable values, and required fields such as dose, duration, and frequency, to facilitate complete and accurate order entry. Several can also perform drug–allergy, drug–drug, drug–disease, and drug–lab interaction checking (Bates et al. 1998, 1999; Teich et al. 2000; Bizovi et al. 2002; Fontan et al. 2003; Cordero et al. 2004; Potts et al. 2004; Gandhi et al. 2005; Oliven et al. 2005). This order-checking function has the potential to reduce drug interactions and contraindications. Only two products have the advanced feature of patient-specific dose calculation (Cordero et al. 2004; Potts et al. 2004) which helps physicians detect prescription errors in a timely manner. Retrospective studies did not analyze differences in availability, inducement, physician's participation, and saturation of CPOE in participating hospitals. According to a survey of 964 randomly selected American hospitals, 83.7 percent of hospitals did not have CPOE available for the physicians, 6.5 percent reported partial availability, and only 9.6 percent reported complete implementation (all orders could be entered in all stations) (Ash et al. 2004).
Authors have used different strategies to detect errors and patient injuries including review of patient charts, incident reports, attending medical rounds to detect undocumented medication errors, interview of medical personnel and patients, comparisons of pharmacists and physicians orders, and follow-up pharmacokinetic examinations of patients (Flynn et al. 2002; Pronovost, Miller, and Wachter 2006). The most common method to measure medication errors—self-report—may underestimate the true rate of outcomes (Kohn and Donaldson 2000; Pronovost, Miller, and Wachter 2006). Direct observations uncovered a 17.9 percent error rate, while chart review found only 0.9 percent (Flynn et al. 2002). Clinically significant medication errors were detected in 71 percent by direct observations, only 9 percent by records review, and none by incident report review. The agreement between evaluators using a κ statistic to measure medication errors differed across studies but was generally around 0.7, with the lowest agreement around the severity of adverse events (0.35). We could not include agreement coefficients in the meta-analysis because not all authors provided this information.
The studies do not allow broad generalizability. As is often the case with innovations, they were primarily conducted in teaching hospitals. CPOE's effects in more typical practice settings where the level of care may be different remains to be determined.
Clinics use other strategies in addition to CPOE to improve patient safety. Some evidence suggests that a combination of CPOE with a decision support system (Chertow et al. 2001; Tamblyn et al. 2003), computerized drug dosing (Hunt et al. 1998) and drug administration systems (Rothschild et al. 2005), automated safety alerts for particular drugs (Galanter, Polikaitis, and DiDomenico 2004; Galanter, Didomenico, and Polikaitis 2005), and clinical pharmacy services (Bond, Raehl, and Franke 2002) may decrease medication errors and preventable adverse events. Future research should assess the effects of combined strategies that include CPOE to improve patient safety.
Existing evidence (levels I–II3) suggests a significant reduction in prescribing medical errors is associated with implementing CPOE systems, but the effects are not consistent among patient populations and clinical settings and are compromised by the design of studies. Sites that use CPOE still experience high rates of medication errors and ADEs. Most of the studies reviewed were conducted in academic centers. The effect of CPOE in more typical practice settings remains to be determined. Future research is needed to investigate the true impact of CPOE on patient safety. Although a randomized trial would be more definitive (Wahrendorf, Blettner, and Edler 1985; Lachin, Matts, and Wei 1988; Signorini et al. 1993), a retrospective multicenter cohort study of randomly selected nationally representative samples of patients and physicians is more feasible and less expensive to estimate the effects of CPOE. In all studies, clear common definitions of intervention and outcomes and valid methods to measure them are essential to detect the effectiveness of CPOE on patient safety.
Study concept and design: Kane R. L., Shamliyan T. A.
Acquisition of data: Shamliyan T. A., Jing Du.
Analysis and interpretation of data: Shamliyan T. A., Duval S., Kane R. L.
Drafting of the manuscript: Shamliyan T. A., Duval S., Jing Du., Kane R. L.
Statistical expertise: Duval S.
Study supervision: Kane R. L.
The following supplementary material for this article is available:
Literature Search Strategies and Eligibility Status of the Reviewed Studies.
A Summary of CPOE Functionality by Product/Vendors.
This material is available as part of the online article from: http://www.blackwell-synergy.com/doi/abs/10.1111/j.1475-6773.2007.00751.x (this link will take you to the article abstract).
Please note: Blackwell Publishing is not responsible for the content or functionality of any supplementary materials supplied by the authors. Any queries (other than missing material) should be directed to the corresponding author for the article.