|Home | About | Journals | Submit | Contact Us | Français|
Healthcare providers previously using older electronic health records (EHRs) with electronic prescribing (e-prescribing) are transitioning to newer systems to be eligible for federal meaningful use incentives. Little is known about the safety effects of transitioning between systems.
To assess the effect of transitioning between EHR systems on rates and types of prescribing errors, as well as provider perceptions about the effect on prescribing safety.
Prospective, case study of 17 physicians at an academic-affiliated ambulatory clinic from February 2008 through August 2009. All physicians transitioned from an older EHR with minimal clinical decision support (CDS) for e-prescribing to a newer EHR with more robust CDS.
Prescribing errors were identified by standardized prescription and chart review. A novel survey instrument was administered to evaluate provider perceptions about prescribing safety.
We analyzed 1298 prescriptions at baseline, 1331 prescriptions 12 weeks post-implementation, and 1303 prescriptions one year post-implementation. Overall prescribing error rates were highest at baseline (35.7 per 100 prescriptions, 95% confidence interval (CI) 23.2–54.8) and lowest one year post-implementation (12.2 per 100 prescriptions, 95% CI 8.6–17.4) (p<0.001). Improvement in prescribing safety was mainly a result of reducing inappropriate abbreviation errors. However, rates for non-abbreviation prescribing errors were significantly higher at 12 weeks post-implementation than at baseline (17.7 per 100 prescriptions, 95% CI 9.5–33.0 versus 8.5 per 100 prescriptions, 95% CI 4.6-15.9) (p <0.001) and no different at baseline than one year (10.2 per 100 prescriptions, 95% CI 6.2–18.6) (p=0.337). Survey results complemented quantitative findings.
Results from this case study suggest that transitioning between systems, even to those with more robust CDS, may pose important safety threats. Recognizing the challenges associated with transitions and refining CDS within systems may help maximize safety benefits.
Improving healthcare safety is a national priority, and e-prescribing is viewed as an important tool in these efforts1–2. Research on the ability of e-prescribing to improve safety has predominantly come from the inpatient setting and on locally developed systems created by organizations for their own use3–5. Fewer studies have been conducted on commercial systems or in the outpatient setting and results have been mixed6–13. To ensure that federal spending is directed toward effective interventions, it is important to evaluate the effect of e-prescribing on safety in the ambulatory setting, where most prescribing occurs and errors are common7,12–15.
Use of e-prescribing in the ambulatory setting has been low, although increased use is expected given federal incentives for meaningful EHR use2,16–20. To demonstrate meaningful use, providers will have to meet certain criteria, including use of e-prescribing21. For providers transitioning from paper, commercial EHRs with e-prescribing are likely to be adopted because they are readily available. Some organizations using locally developed systems are also transitioning to commercial systems because locally developed systems, although uniquely customized, require great initiative to maintain and may not follow national standards in interoperability and function22,23. Healthcare providers may also need to transition to newer versions of existing systems to meet meaningful use requirements.
Implementation of new systems is traditionally challenging and the effect of transitioning between systems on prescribing errors is unknown. Understanding the effects will be informative for those undergoing this type of transition and allow potential safety threats to be better managed24,25. We therefore conducted this study to examine the effects on prescribing safety of transitioning between two systems in an academic-affiliated ambulatory practice.
We conducted this prospective study, approved by the institutional review board of Weill Cornell Medical College, of ambulatory care providers using a pre-post design. We analyzed electronic prescriptions on the older system just prior to implementation of the newer system and at twelve weeks and one year post-implementation. We also administered a novel survey to assess consented provider perceptions of the effect on prescribing safety.
The Institute of Medicine defines medication errors as any error in the medication process (prescribing, transcribing, dispensing, administering, and monitoring)26. We focused only on prescribing errors as defined in Table 17,13,26.
We use the term e-prescribing to describe only the electronic ordering of medications, regardless of whether prescriptions are printed or electronically transmitted to a pharmacy, as we are focused on the prescribing stage. Finally, we use the term CDS to describe support to improve clinical decision-making related to therapeutic processes of care, for example alerts to identify drug-allergy interactions27.
We studied faculty providers at an academically-affiliated, hospital-based adult internal medicine ambulatory practice in New York City. All 19 faculty members working 75% time or more and at least two clinic sessions per week were included.
Providers transitioned from the older to the newer system, whose use was mandatory, in April 2008. The information systems team conducted a large-scale, intensive effort to transition providers, including transferring medication data between systems. All providers were required to attend multiple, 2-hour training sessions prior to go-live. Provider schedules were minimized by 50% for the first two weeks and 75% for the third week of the go-live period. Implementation teams were present on-site during go-live and held weekly sessions for the first month after implementation to answer providers’ questions.
The older PC-based EHR system was developed by one of the providers at the study site and implemented in 1993. The system had order entry, free-text progress notes, flow sheets, laboratory result viewing, clinical messaging, and scheduling and registration capabilities. Usage for prescribing was near-universal for non-narcotic prescriptions. The only CDS was default formulations (the most commonly used formulation would automatically pre-populate when the medication name was entered). The medication database was managed centrally by the developer but allowed “work-arounds” for free-texting of medications. Prescriptions could not be sent electronically to pharmacies.
The newer system is a Certification Committee for Health Information Technology (CCHIT)-certified commercial EHR with e-prescribing28. CCHIT is a federally recognized EHR certification body. In addition to the features offered by the older system, the newer system has additional CDS including alerts for allergies and drug-drug interactions. Prescriptions can be sent electronically to pharmacies. Providers can create preference lists (lists of frequently used orders) and order sets (pre-populated groups of medications). The medication master-file and CDS logic is derived from a third-party supplier.
Electronic prescriptions were extracted from each EHR’s database for a two week period in three study intervals: baseline, 12 weeks post-implementation, and one year post-implementation. We chose 12 weeks because it was early after go-live but allowed providers a brief period to acclimate, and one year to ensure a more prolonged period of system use. We excluded prescriptions written by residents. We obtained a minimum of 75 prescriptions on 25 patients per provider, extending data collection beyond two weeks if necessary.
Two nurse reviewers were trained in an identical manner by R.K. with extensively utilized and standardized methodology7,29–32. This included review of error definitions, test, and actual cases. Both nurses jointly reviewed cases initially; these cases were then reviewed by R.K to ensure proper classification. Reviews were then conducted separately. Conflicts were resolved through discussion and consensus between R.K. and the nurses. Methodology included error classification and identification of ADE trigger drugs29. We determined inter-rater reliability by having both nurses review the same random sample of fifty prescriptions. Inter-rater agreement for overall error and error type was 1.0, indicating excellent agreement.
Of note, our methodology is primarily designed to detect prescribing errors and rule violations, not near misses and preventable ADEs, which are best detected by chart reviews for every prescription or patient surveys.
In 2004, the Joint Commission on Accreditation of Healthcare Organizations (JCAHO) established a “Do Not Use” list for abbreviations with great potential to cause errors33. A ban of these was locally instituted. As a result, the older system was modified to automatically change inappropriate abbreviations into acceptable terms on printed prescriptions (for example “QD” printed as “once daily”). However, providers were not required to upgrade to this modified version and we were unable to ascertain from electronic downloads which system version was used and therefore whether the inappropriate abbreviation had been automatically corrected.
Thus we classified all inappropriate abbreviations from the older system as errors. In the newer system, providers were required to override two separate alerts (at ordering and signing) to use an inappropriate abbreviation, however there was no automatic conversion.
The research nurse performed ambulatory chart reviews for suspected near misses or when a drug often used to treat an ADE was prescribed.
Two physicians independently reviewed all suspected near misses and ADEs. Confirmed ADEs and near misses were rated on preventability using a five-point Likert scale and attribution using the Naranjo algorithm34. ADE severity was rated using a four-point Likert scale. Inter-rater agreement for the presence of prescribing errors and near misses was 0.96 and 0.93, indicating excellent agreement.
We developed a five-page survey based on literature review and consultation with experts in informatics, clinical medicine, and medication safety. Questions focused on physicians’ backgrounds, implementation experiences, and prescribing using the two systems. This confidential survey was administered from January through June 2009 as background for part of a large qualitative study conducted with the same providers35. The survey was piloted with five pediatricians in the same ambulatory care network. We distributed the paper survey and a $10 gift card to eligible physicians. We sent bi-monthly email reminders and visited the site to promote survey completion.
For data management and descriptive statistics, including the survey, we used SAS 9.2 (SAS Institute Inc., Cary, NC). This included comparisons of patient characteristics across the three time points using chi-squared tests for categorical variables and ANOVA for continuous variables. We considered significant findings at the 0.05 level. We used Poisson regression to estimate error rates per 100 prescriptions at the three time points, while adjusting for patient characteristics (age, insurance, and gender). In the model, we compared rates at 12 weeks and 1 year to baseline. We also calculated 95% Poisson confidence intervals for the rates. Using generalized estimating equations, we adjusted for clustering, using provider as the unit of analysis. To reduce the amount of patient clustering, we reviewed a maximum of three prescriptions per patient during data collection. We assumed an independent correlation structure for the Poisson regression. The primary Poisson model was analyzed using Stata 10 (StataCorp, College Station, TX).
Nineteen providers participated in the baseline and 12-week post-implementation data collection periods. Seventeen providers participated at one year (one provider was on maternity leave and one was on sabbatical). Only providers included in all three time points were part of the analysis. Provider characteristics are presented in Table 2.
We reviewed prescriptions for 646 patients at baseline, 736 patients at 12 weeks post-implementation and 715 patients one year post-implementation (Table 2). There was no significant difference in patient gender across time periods. Patients from the baseline period were older than patients at other time periods.
We reviewed 1298 prescriptions at baseline, 1331 prescriptions 12 weeks post-implementation, and 1303 prescriptions one year post-implementation (Table 3). Overall rates of prescribing errors were highest at baseline (35.7 per 100 prescriptions) and significantly lower at 12 weeks (21.1 per 100 prescriptions, p<0.001) and one year (12.2 per 100 prescriptions, p<0.001) post-implementation compared to baseline. Inappropriate abbreviation errors were also highest at baseline (24.1 per 100 prescriptions) and significantly lower at 12 weeks (10.6 per 100 prescriptions, p<0.001) and 1 year (5.9 per 100 prescriptions, p <0.001) post-implementation. Rates of near misses and rule violations did not significantly differ between time periods. No preventable ADEs were detected.
At all three time periods, inappropriate abbreviations constituted a majority of the prescribing errors (Table 4). Direction and frequency errors were also common. Examples of errors are presented in Table 5. Oral diabetic agents, antibiotics, narcotic analgesics, and topical steroids and antifungals were most frequently associated with prescribing errors. Sumatriptans, phosphodiesterase inhibitors, topical anesthetics and narcotic analgesics were most frequently involved in near misses.
Because the majority of prescribing errors were inappropriate abbreviations and later versions of the older system automatically corrected these (although we were unable to ascertain whether this correction occurred), we performed a separate analysis excluding inappropriate abbreviations from error counts. With this analysis, the overall prescribing error rate was lowest at baseline and highest 12 weeks post-implementation (8.5 versus 17.7 errors per 100 prescriptions, p<0.001) (Table 3). Error rates at one year were not significantly different than at baseline (8.5 versus 10.8 errors per 100 prescriptions, p=0.337).
We achieved a 79% response rate (n=15) for the survey. Physician satisfaction regarding implementation of the newer system was mixed. Forty seven percent of physicians reported being very or somewhat satisfied with implementation (n=7), while 40% (n=6) reported being somewhat or very dissatisfied. Two-thirds of providers (n=10), however, felt that training on the new system was rarely or never a problem.
With regard to safety, only one-third (n=5) of physicians felt that the newer system improved safety compared to the older system, despite its more robust CDS. Alert fatigue was also widespread. While 53% of physicians (n=8) reported that alerts were triggered for at least 50% of prescriptions, nearly two-thirds (n=9, 60%) reported that alerts were rarely or never useful. In terms of impact on workflow, two thirds of respondents (n=10) reported that the speed in ordering and refilling medications was much worse or worse with the new system compared with the older system.
Our study is the first to our knowledge to quantitatively evaluate the effect on ambulatory prescribing errors of transitioning between e-prescribing systems. Implementation of the new, commercial system led to a significant and progressive decrease in overall rates of prescribing errors, largely by reducing inappropriate abbreviations.
Rates for non-abbreviation prescribing errors, however, were actually highest at 12 weeks post-implementation, suggesting that transitioning between systems may pose potential patient safety threats even for experienced e-prescribers. Previous studies have identified unintended negative consequences from the introduction and use of computerized systems for ordering medications, including the facilitation of errors36,37. Also in our study, overall error rates for non-abbreviation errors were no different at one year and baseline, despite the fact that the old system had very limited CDS, although due to our small sample size, we may not have been able to detect a significant difference. This is despite a low overall prescribing error rates for non-abbreviation errors compared to rates in other published studies using identical methodology7,13.
One of the major perceived safety benefits of e-prescribing is CDS to aid with medication ordering38. However, the value of CDS is often limited by providers’ lack of alert acceptance. Multiple studies have shown that providers frequently override alerts because they are perceived as irrelevant39–42. The inability of EHR systems to effectively present information for CDS purposes may contribute to the mixed safety benefits observed in studies1,7–10,12,13. In our study, for example, the newer system generated two alerts at separate steps in the ordering process to target inappropriate abbreviations. While this led to an immediate reduction in inappropriate abbreviation use, inappropriate abbreviation errors still constituted the majority of prescribing errors, suggesting that the content or the presentation of the alerts had limited effectiveness in modifying prescribing behavior.
In a companion qualitative study that we did with the providers, findings from semi-structured interviews and field observations complement findings in this manuscript35. We found that almost all providers, even though they were experienced e-prescribers, considered the transition difficult. Consistent with our survey findings, most physicians did not view the newer system as improving safety, despite more CDS features, and alert fatigue led to routine overriding of alerts.
Studying the types and frequencies of errors made using e-prescribing systems can allow for targeted improvements in system design, CDS presentation, and implementation. For example, the second most common error we detected was directions errors. Pre-printed templates with clear patient instructions may help eliminate these errors. By determining drug classes most frequently involved in errors, CDS can be developed specifically to reduce these errors. Targeted alerts for certain prescribing errors have been shown to be effective8–10. In addition, it can also guide calibration of CDS such that there is a higher sensitivity to trigger alerts. Importantly, although the majority of the prescribing errors we detected lacked potential to cause serious harm, these errors can result in inefficiencies (such as pharmacy callbacks) and thus are important to study. Research has shown that pharmacy callbacks are common and often lead to delays in medication dispensing that pose threats to patient safety43.
Our study has several limitations. Providers were not blinded to the study’s purpose and may have been extra careful when prescribing, making our results conservative estimates of true error rates. We were also limited by our methodology to comment on near misses and preventable ADEs. However, given previous research demonstrating that 4% of patients experience a preventable ADE, it is likely that near misses and preventable ADEs occurred that we were unable to capture7. Additionally, because we were unable to ascertain from electronic downloads whether the older system automatically corrected inappropriate abbreviations, we may have overestimated these errors at baseline.
Our study was conducted in one clinic, limiting generalizability. We studied only two systems, although the commercial system is widely utilized and incorporates many features recommended by an expert panel44. We also did not observe providers using both systems, nor do we have data logs tracking the frequency with which alerts were generated or over-ridden. Thus, our ability to comment on usage and usability of the systems is limited and can be derived only from survey data. Finally, due to the small sample sizes for non-abbreviation prescribing errors, we are limited in our ability to detect differences in error rates at different time periods. Future studies should be performed with more providers, multiple systems, and at diverse sites. Longitudinal studies should also be conducted to determine how error rates change over time as iterative refinements are made and providers become more familiar with a new system.
With federal incentives promoting meaningful use of certified EHRs, more organizations will likely transition between e-prescribing systems. Our results suggest that transitioning may lead to unintended negative patient safety consequences, particularly early post-transition. This is despite strategies such as pre-transferring medication data between systems, requiring providers to attend mandatory training sessions, and providing on-site support during and after go-live. Additional strategies may therefore be needed to make transitioning safer. For example, providers may need more individualized training and closer follow-up to address prescribing errors in a timelier manner.
Our study also suggests that organizations and vendors may have to better tailor the design and configuration of CDS to achieve greater safety gains. Focusing CDS toward certain types of errors, such as inappropriate abbreviation errors, may be one such strategy. Greater provider education on rates and types of prescribing errors will complement this strategy. Finally, given the rapidly evolving nature of e-prescribing adoption on a national level and the potential safety issues that may arise, it will be important for organizations to monitor safety issues and iteratively refine systems to ensure that adoption actually leads to safer healthcare delivery.
The authors thank Dr. Fran Ganz-Lord for her assistance enrolling physicians, and Drs. James Hollenberg and Curtis Cole for assistance in retrieving electronic data. This project was supported by the Agency for Healthcare Research and Quality (R18HS017029), Rockville, MD.
Funding/Support This project was supported by the Agency for Healthcare Research and Quality (R18HS017029), Rockville, MD.
Conflict of Interest None disclosed.
Trial Registration ClinicalTrials.gov, ID NCT00603070