Real-time pharmacy surveillance and clinical decision support to reduce adverse drug events in acute kidney injury: a randomized, controlled trial

doi:10.4338/ACI-2012-03-RA-0009

Appl Clin Inform. Author manuscript; available in PMC 2012 June 18.

Published in final edited form as:

Appl Clin Inform. 2012 January 1; 3(2): 221–238.

Published online 2012 June 13. doi: 10.4338/ACI-2012-03-RA-0009

PMCID: PMC3377180

NIHMSID: NIHMS381202

Real-time pharmacy surveillance and clinical decision support to reduce adverse drug events in acute kidney injury: a randomized, controlled trial

Allison B. McCoy, PhD,¹ Zachary L. Cox, PharmD, BCPS,^2,³ Erin B. Neal, PharmD, BCPS,² Lemuel R. Waitman, PhD,⁴ Neeraja B. Peterson, MD, MSc,⁵ Gautam Bhave, MD, PhD,⁶ Edward D. Siew, MD,⁶ Ioana Danciu, BE,¹ Julia B. Lewis, MD,⁶ and Josh F. Peterson, MD, MPH^1,^5,⁷

¹Department of Biomedical Informatics, Vanderbilt University School of Medicine, Nashville, TN

²Department of Pharmacy, Vanderbilt University Medical Center, Nashville, TN

³College of Pharmacy, Lipscomb University, Nashville, TN

⁴Department of Biostatistics, University of Kansas Medical Center, Kansas City, KS

⁵Division of General Internal Medicine and Public Health, Department of Medicine, Vanderbilt University Medical Center, Nashville, TN

⁶Division of Nephrology, Department of Medicine, Vanderbilt University Medical Center, Nashville, TN

⁷Geriatric Research Education Clinical Center (GRECC), VA Tennessee Valley Healthcare System, Nashville, TN

Corresponding author’s current affiliation and address: School of Biomedical Informatics, The University of Texas Health Science Center at Houston (UTHealth), UT Houston-Memorial Hermann Center for Healthcare Quality and Safety, 6410 Fannin St, UPB 1100, Houston, TX 77030, (713) 500-6931, Email: allison.b.mccoy/at/uth.tmc.edu

Abstract

Objectives

Clinical decision support (CDS), such as computerized alerts, improves prescribing in the setting of acute kidney injury (AKI), but considerable opportunity remains to improve patient safety. The authors sought to determine whether pharmacy surveillance of AKI patients could detect and prevent medication errors that are not corrected by automated interventions.

Methods

The authors conducted a randomized clinical trial among 396 patients admitted to an academic, tertiary care hospital between June 1, 2010 and August 31, 2010 with an acute 0.5 mg/dl change in serum creatinine over 48 hours and a nephrotoxic or renally cleared medication order. Patients randomly assigned to the intervention group received surveillance from a clinical pharmacist using a web-based surveillance tool to monitor drug prescribing and kidney function trends. CDS alerting and standard pharmacy services were active in both study arms. Outcome measures included blinded adjudication of potential adverse drug events (pADEs), adverse drug events (ADEs) and time to provider modification or discontinuation of targeted nephrotoxic or renally cleared medications.

Results

Potential ADEs or ADEs occurred for 104 (8.0%) of control and 99 (7.1%) of intervention patient-medication pairs (p=0.4). Additionally, the time to provider modification or discontinuation of targeted nephrotoxic or renally cleared medications did not differ between control and intervention patients (33.4 hrs vs. 30.3 hrs, p=0.3).

Conclusions

Pharmacy surveillance had no incremental benefit over previously implemented CDS alerts

Keywords: Decision Support Systems, Clinical, Electronic Health Records, Randomized Controlled Trial, Medication Errors/prevention & control, Adverse Drug Reaction Reporting Systems

1. Background

Clinical decision support (CDS) within electronic medical records (EMRs) and computerized provider order entry (CPOE) systems has a high potential for reducing medication errors [1–4]. Despite some success, substantial numbers of residual adverse drug events (ADEs) and potential adverse drug events (pADEs) remain, suggesting that further improvements of patient safety will require additional types of well-integrated interventions. Furthermore, recent evaluations of CDS point out the risks of the technology, including provider fatigue and dissatisfaction, and unintended adverse consequences [5–9].

Clinical pharmacy services have traditionally addressed medication safety for hospitalized patients by reviewing medication orders prior to dispensing or rounding with inpatient teams, and the support of a clinical pharmacist has been shown to improve prescribing in multiple settings [10–17]. Pharmacy surveillance has also been shown to detect and potentially prevent ADEs [18–21]. However, the incremental benefit of pharmacy surveillance when added to CDS in reducing ADEs is unknown.

We developed a real-time surveillance tool for medication errors in order to integrate CDS with clinical pharmacy surveillance. The tool recognized high-risk prescribing in patients with acute kidney injury (AKI) and directed a clinical pharmacist to intervene on the highest risk patients, including those where providers were not responding to alerts from an existing CDS system [22]. AKI affects patients and medication regimens across all hospitalized units and prescribing is not well standardized, though CDS has been shown to benefit patients with impaired or rapidly changing renal function [21–26]. We hypothesized that real-time surveillance by a clinical pharmacist using the web-based tool would improve the management of renally-dosed medications. To test this, we conducted a prospective, randomized, controlled study, comparing the effect of enhanced clinical pharmacist surveillance of patients in the intervention group with existing CDS and standard pharmacy services on the occurrence, preventability, and severity of ADEs.

2. Methods

2.1. Study Design and Participants

We performed a parallel group randomized, controlled trial during June 1, 2010 through August 31, 2010 at Vanderbilt University Hospital, a large academic, tertiary care facility with universal utilization of CPOE and extensive integrated CDS [1,25,27–30]. The study included all admitted adult patients who experienced a 0.5 mg/dl increase or decrease in serum creatinine over 48 hours of hospitalization following an active, recurring order for one or more targeted nephrotoxic or renally cleared medications (Table I); these criteria were selected by a local expert nephrology panel to identify significant renal function changes which should trigger reassessment of drug therapy. It included all patients who were eligible to receive a previously implemented CDS alert for AKI [22].

Table I

Targeted nephrotoxic or renally cleared medications for surveillance

Prior to randomization, we excluded patients with end-stage renal disease who received dialysis prior to the first serum creatinine change event or were previously identified as a chronic dialysis patient in the inpatient order entry records. In addition, we excluded patients cared for on services with existing specialty pharmacy support including nephrology services and renal, liver, and bone marrow transplant services. Finally, prior to analysis, a blinded outcomes assessment pharmacist evaluated cases for additional exclusion criteria that could not be determined electronically at the time of randomization, including chronic dialysis patients, transplant patients, palliative care patients, false lab measurements, and no medication administrations. This study was approved by the Vanderbilt Institutional Review Board.

2.2. Surveillance Tool Intervention

The surveillance tool is a dynamic web application, populated by real-time clinical databases and CDS log files, designed to complement traditional CDS and clinical pharmacy services by facilitating monitoring and intervention for high risk patients. The tool consists of two primary view types: the surveillance view and the patient detail view. The surveillance view displays patient details such as demographics, providing service, and hospital location, in addition to most recent creatinine values and alerts about declining or improving renal function, for all currently admitted, eligible patients, allowing pharmacists to identify patients at high risk for harm (Figure 1). The patient detail view (Figure 2) displays a graph of events of interest and a detailed, sortable timeline of orders, order administrations, laboratory values, and CDS logs during the patient’s admission for an individual patient, presented in reverse chronological order to give context for the CDS logs. This view also allows staff to enter notes directly into the EMR and to save comments for reference to other surveillance users.

Figure 1

Figure 2

Prior to formally evaluating the surveillance tool for AKI, we performed a four month pilot implementation. Study personnel independently reviewed select cases, discussed the potential for intervention based on patient and drug factors, and assessed the usefulness of the surveillance tool. Based on the feedback, we made iterative changes to the targeted medication list and the inclusion criteria for both the surveillance tool and AKI CDS.

During the trial, the clinical pharmacist for internal medicine (EN) served as the study pharmacist for the intervention, reviewing the surveillance tool to evaluate patients during each workday. For patients determined to be experiencing AKI and needing an intervention, the study pharmacist contacted the primary provider using a text page, verbal communication, or an EMR note to recommend changes in care. All interactions, including patient’s classification of AKI, communication with provider, recommendations provided, and actions taken, were recorded with a timestamp in a structured form within the surveillance tool for later analysis.

Both control and intervention patients received standard CDS and clinical pharmacy services for all hospitalized patients. Active CDS included guided dosing advisors and CPOE alerts for AKI, which have been described previously [22,27]. A clinical pharmacist rounded on some medical and surgical teams, including the majority of the ICU services, and intermittently on non-ICU services to answer provider questions, independently review medication regimens, and offer verbal recommendations directly to providers during order entry. Clinical pharmacists at the study institution did not typically interact with CPOE based CDS as part of their usual workflow; in rare circumstances, a rounding pharmacist might encounter the AKI CDS when entering medication orders on behalf of providers. Staff pharmacists also reviewed medication orders prior to dispensing during the trial, but these pharmacists did not also monitor laboratory results or changes in serum creatinine.

2.3. Randomization

Patients were automatically assigned to a study group using a pseudo-random number function within the surveillance tool at the time that he or she first met eligibility criteria and remained in the assigned group until discharge.

2.4. Outcomes

Our primary outcome was the rate of AKI-related ADEs and pADEs within the intervention group compared to the concurrent control group. All study events originated from non-intercepted medication errors with a potential for injury that were active at the onset of AKI and continued for at least 24 hours. We report only on ADEs and pADEs associated with renal function, either through direct nephrotoxicity or with a well-described toxicity related to drug accumulation. For example, an active order for a recurring administration of a non-steroidal anti-inflammatory drug during the 24 hours after the initial change in serum creatinine was rated as a pADE, and an episode of bleeding after administration of enoxaparin unadjusted for renal function was rated as an ADE. A subcategory of ADEs, “lab-only” ADEs, was defined as highly elevated drug levels or electrolyte abnormalities consistent with previous ADE literature [31]. We evaluated provider behavior as our secondary outcome, measuring the time to provider reaction to the AKI event. We calculated the time from the first change in serum creatinine to modification or discontinuation of targeted medications ordered prior to the change [22] and the time from the initial order to modification or discontinuation of targeted medications ordered after the change. We assessed all outcomes after completion of the inpatient encounter (either by death or discharge); pADEs or ADEs occurring after patient discharge were not included in the analysis as outpatient follow-up was not routinely available for all patients.

Outcomes assessors were blinded to patient intervention status [32]. A study pharmacist (ZC) reviewed all enrolled cases to identify potential study events. The pharmacist recorded comorbidities present prior to AKI and reviewed each targeted medication order for a potential error or subsequent injury, drawing from the pre-specified lists of drugs (Table I) and potential AKI-related adverse events. An outcomes assessment adjudication committee composed of a nephrologist and an internal medicine physician (GB, NP) independently reviewed cases categorized as having at least one pADE or ADE, using methods previously applied to rate preventability and severity [33–35]. An additional nephrologist (ES) reviewed cases when disagreement occurred. Any residual disagreements within the committee were resolved by joint discussion.

We also measured outcomes describing the use of the surveillance tool. These included number of patients appearing on the tool, number of data items (e.g. drugs, labs, and CDS interactions of interest) for patients, time of day the tool was viewed, duration of views, number of patients with comments or EMR notes submitted, and the number of patients for which pharmacists intervened. We also evaluated comments and EMR notes recorded through the tool.

2.5. Sample Size

We estimated the sample size based on the rate of non-adjudicated medication errors measured from a retrospective database of medication orders and laboratory values, which had a median of 52.5 patient-medication pair events each week and a 45.4% baseline response rate. The planned 13 week trial produced 94% power to detect a 30% reduction in non-adjudicated errors, which we determined to be feasible after a previous analysis of the existing CDS found a significant opportunity for improvement [9].

2.6. Statistical Methods

We used the Pearson chi-square test for categorical variables and the t-test for continuous variables to perform univariate comparisons between the control and intervention groups. To evaluate provider behavior, we applied survival analysis methods for time to provider response, defined as a modification or discontinuation of a targeted drug order by any provider. For this analysis, we defined patient discharge or death as a censoring event. For medications ordered prior to the triggering event, follow-up started at the time of the triggering serum creatinine change, and for medications ordered after the triggering event, follow-up started at the time the medication was ordered. We used the log-rank test to measure the difference between control and intervention groups and provide Kaplan-Meier plots for visualization of the time to event data. Analyses were conducted with Stata 9.2.

3. Results

3.1. Study Population

Figure 3 is a diagram of the selection steps for allocating patients to control or intervention and application of inclusion and exclusion criteria. During the three month trial period, 1,488 of 11,128 admitted adults experienced a triggering change of serum creatinine over 48 hours. We enrolled 540 cases; 278 were randomized to the control group, and 262 were randomized to the intervention group. The blinded outcomes assessment pharmacist identified 82 control and 62 intervention cases that met additional exclusion criteria that could not be determined electronically at the time of randomization; we included 396 cases in the final outcomes assessment. We compared demographic characteristics, including age, sex, race, admitting service, and admission to an intensive care unit, and comorbidities, which the initial outcomes assessment study pharmacist classified, between the control and intervention groups to ensure that the study groups were similar (Table II). We found no statistical difference between groups for any variable evaluated.

Figure 3

Table II

Study population demographics for analyzed acute kidney injury surveillance cases

3.2. Evaluation of Adverse Drug Events

We evaluated 196 control cases with 1303 medication orders and 200 intervention cases with 1396 medication orders. Agreement between the two outcomes adjudication physicians was 93.97% for pADEs (kappa=0.88) and 96.55% for ADEs (kappa=0.93). Table III depicts the detailed breakdown of pADEs and ADEs after adjudication and consensus. The adjudication committee determined 76 (38.8%) control and 67 (33.5%) intervention cases experienced a pADE or ADE (RR=0.86 [0.66, 1.12], p=0.3) and 104 (7.98%) control and 99 (7.09%) intervention medication orders had an associated pADE or ADE (RR=0.88 [0.68, 1.16], p=0.4). Among cases who experienced at least one pADE or ADE in the control group, 55 (72.4%) experienced one, 15 (19.7%) experienced two, 5 (6.6%) experienced three, and 1 (1.32%) experienced four events; in the intervention group, 41 (61.2%) experienced one, 20 (29.9%) experienced two, and 6 (9.0%) experienced three events.

Table III

Evaluation of potential adverse drug events and adverse drug events

The total events included 52 (26.5%) control and 46 (23.0%) intervention cases experiencing a pADE (RR=0.87 [0.61, 1.22], p=0.4), and 68 (5.22%) control and 63 (4.51%) intervention medication orders having an associated pADE (RR=0.86 [0.62, 1.21], p=0.4). Frequent responses for pADEs categorized as “other” included “dose and interval change inappropriate for trough level” and “interacted with another prescribed medication”.

Lab-only ADEs occurred for 13 (6.6%) control and 16 (8.0%) intervention cases (RR=1.21 [0.60, 2.44], p=0.6); 14 (1.07%) control and 16 (1.15%) intervention medication orders had an associated lab-only ADE (RR=1.07 [0.52, 2.18], p=0.9). Actual ADEs occurred for 22 (11.2%) control and 19 (9.5%) intervention cases (RR=0.85 [0.47, 1.51], p=0.6); 22 (1.69%) control and 20 (1.43%) intervention medication orders had an associated actual ADE (RR=0.85 [0.47, 1.55], p=0.6). Severity and preventability of pADEs and ADEs was not significant between control and intervention groups (Table IV).

Table IV

Evaluation of potential adverse drug events and adverse drug events

Among drugs or drug groups with at least ten orders in the control and intervention groups, errors most commonly occurred for vancomycin, beta lactam antibiotics, angiotensin-converting enzyme (ACE) inhibitors, nonsteroidal anti-inflammatory drugs (NSAIDs), quinolones, and angiotensin II receptor blockers (ARBs) comprising 22.2%, 15.3%, 12.3%, 9.4%, 8.9%, and 5.4% of orders resulting in a pADE or ADE respectively.

3.3. Evaluation of Provider Responses

For medications active at the time of patient’s triggering serum creatinine change or ordered after the event, we compared the time to provider response, defined as drug modification or discontinuation, using the log-rank test. While the time to response was shorter in the intervention group compared to the control group for medications, overall response times were highly variable, and we did not find any statistically significant differences between the control and intervention groups. Table V shows the resulting median times to response, hazard ratios, and p-values. Kaplan-Meier curves for these results are shown in Figure 4.

Table V

Evaluation of surveillance and provider response

Figure 4

3.4. Study Pharmacist Interactions with the Surveillance Tool

During the 3-month study period, 273 intervention patients appeared on the surveillance tool. The study pharmacist viewed the surveillance tool on 67 days (56 weekdays). Of the displayed intervention patients, 234 (85.7%) were reviewed by the study pharmacist, with an average of 10.75 patients reviewed each day the surveillance tool was monitored. Monitoring occurred between 08:00 and 16:00, although the study pharmacist primarily checked the surveillance tool in the afternoon, after providing teams had completed rounds, updated medication orders, and entered EMR notes, and laboratory results had returned. During a week of direct observation, the pharmacist spent 71 minutes monitoring the surveillance tool on Monday, and a mean of 16.75 minutes on the remaining days (25 on Tuesday, 9 on Wednesday, 15 on Thursday, and 18 on Friday).

The study pharmacist recommended an intervention for 43 (18.4%) cases, including 70 medication-specific recommendations and 8 patient-specific recommendations. Most cases without an intervention did not require a dose change; frequencies of recommended patient and medication interventions are described in Table VI. Medication recommendations categorized as “other” (9 medications) included correcting the patient’s weight, holding the medication, and monitoring for sedation. Patient recommendations categorized as “other” (2 cases) included redrawing serum creatinine, monitoring for sedation and treatment failure, discontinuing oral potassium, and adding height and weight.

Table VI

Study pharmacist recommendations

The study pharmacist most frequently indicated use of text pages and verbal communication to contact the providing team; recorded contact included 26 text pages, 28 verbal communications, and 1 EMR note. EMR notes were used when the providing team was unavailable (e.g. providing team does not have an attending on campus, or provider did not respond to text page). Providers agreed to make the recommended changes for 24 interventions (77% of recorded responses), the study pharmacist made changes to the orders directly in the CPOE system for 5 interventions (16%), and the provider disagreed with recommendation for 2 interventions (6%). The study pharmacist submitted 157 surveillance tool comments for 102 cases. The comments frequently summarized patient comorbidities, laboratory values and trends, and indications; served as reminders for continued monitoring; and elaborated recommendations.

4. Discussion

We performed a prospective, randomized comparative effectiveness trial to determine whether pharmacy surveillance improved medication safety during AKI when compared to clinical decision support alone. Despite appropriate interventions made by the study pharmacist, we found no significant improvements in the primary outcome, potential and actual ADEs, or the process outcome of timely medication adjustments. Overall, the number of interventions within the CDS far exceeded the interventions by the pharmacist, reflecting the ability for CDS to be active at all times and intervene more frequently and promptly. The findings suggest that, when implemented in a setting with comprehensive CDS, more intensive surveillance by a clinical pharmacist may be necessary to further improve the safety of prescribing during AKI.

The interventions of a clinical pharmacist have been found to be valuable in a variety of settings with manual medication dosing, significantly reducing rates of medication errors and ADEs [10–16]. However, one prior study has suggested no incremental benefit in ADE prevention with a rounding pharmacist when compared to order entry with CDS [36]. Similar to these findings, existing CDS during our intervention, including initial dosing of nephrotoxic and renally cleared drugs, CDS for monitoring of these medications within CPOE, and surveillance by other pharmacists in the event of changing laboratory values, resulted in a large percentage of already prevented errors. While some studies have found that surveillance successfully identifies pADEs and ADEs, they have not evaluated the effect of systems on actual error prevention [18–20,37–41]. Some investigators have evaluated the use of retrospective CDS surveillance and real-time aggregate CDS surveillance [42,43], but no prior study has evaluated the effect of surveillance of CDS in real time on patient or process outcomes. The restriction of our intervention and analysis to ADEs only related to AKI also makes it difficult to compare our results to these studies, which measured all types of ADEs. However, pharmacy use of the surveillance tool for monitoring AKI patients and CDS was similar to use described for a similar tool for aminoglycosides and anticoagulants [44]. Our study also differs from prior research in that we evaluated the surveillance tool in a setting with extensive existing CDS and clinical pharmacy support.

Other real-time surveillance approaches may be more successful. An alternate workflow, such as use of a distributed surveillance tool by front-line pharmacists approving and dispensing medication orders or by pharmacists or other providers participating in rounds might allow earlier, more frequent prevention of medication errors and reduction of ADEs. Because errors still occurred for patients that received an intervention, the timing of the surveillance and the ability of a single clinical pharmacist to monitor all at-risk patients may not have been appropriate. Greater integration of CDS and pharmacist interactions, with communication features, such that physicians and pharmacists could act as a coordinated team might further impact the results. Finally, implementation of surveillance may have a larger impact in institutions without such extensive CDS and clinical pharmacy services.

Several limitations are present. We conducted the trial in an academic, tertiary care medical center with extensive experience in CDS and clinical pharmacy services, which may have reduced the opportunity for surveillance to be effective. Application of the technology in a community hospital setting may yield a different result, particularly given the high number of medication errors related to renal function that have been previously reported in community settings [45]. Reproducing the surveillance tool requires integration of several advanced clinical systems, which many facilities have not implemented. However, most of the functionality can be recreated with access to common electronic sources of data, including patient census, laboratory, and order-entry [20]. The application of the technology to other domains outside of medication management during AKI should also be investigated. One methodological contribution to a negative result includes the possibility of crossover between study arms, since contacted physicians may care for patients randomized to both the intervention and control groups. While overlap of clinical pharmacy coverage occurred, only 3 control (1.5%) and 6 (3.0%) intervention cases were potentially exposed to the intervention pharmacist in the course of completing routine clinical duties. The study was also underpowered to detect smaller differences in ADE rates. A larger study with a higher intensity of surveillance is more likely to demonstrate a significant difference in patient outcomes. Finally, the inability of the system to electronically identify all patients meeting the exclusion criteria is a limitation of the CDS and affects the generalizability of the results. Because the exclusions were designed to eliminate patients at low risk of AKI-related ADEs, including these in the analysis would likely lower the event rates in both groups.

5. Conclusion

In conclusion, we evaluated through a randomized trial the comparative effectiveness of real-time clinical pharmacist surveillance and existing CDS for patients with AKI. Despite interventions made by the study pharmacist and a trend toward improved outcomes during surveillance, we found no statistically significant improvements in occurrence of potential ADEs or ADEs or in provider responses between control and intervention groups. The study emphasizes that, while CDS is effective at preventing pADEs and ADEs in patients with AKI, further research is necessary to determine whether surveillance can improve CDS performance.

Clinical Relevance Statement

Although a variety of informatics approaches to reducing adverse drug events exist, each can be costly to implement and maintain, and institutions or practitioners must often decide which is more advantageous. We sought to determine whether pharmacy surveillance was effective when compared to traditional clinical decision support alerts.

Acknowledgments

The authors were funded in part by National Library of Medicine grants T15 LM007450 and R01 LM009965. Some data collection was supported by NCRR/NIH grant UL1 RR024975. The authors thank Aihua Bian for her statistical review of the manuscript.

Footnotes

Conflicts of Interest

The authors declare that they have no conflicts of interest in the research.

Protection of Human and Animal Subjects

This study was approved by the Vanderbilt Institutional Review Board.

References

1. Miller RA, Waitman LR, Chen S, Rosenbloom ST. The anatomy of decision support during inpatient care provider order entry (CPOE): empirical observations from a decade of CPOE experience at Vanderbilt. J Biomed Inform. 2005 Dec;38(6):469–85. [PMC free article] [PubMed]

2. Galanter WL, Polikaitis A, DiDomenico RJ. A trial of automated safety alerts for inpatient digoxin use with computerized physician order entry. J Am Med Inform Assoc. 2004 Aug;11(4):270–7. [PMC free article] [PubMed]

3. Bates DW, Teich JM, Lee J, Seger D, Kuperman GJ, Ma’Luf N, et al. The impact of computerized physician order entry on medication error prevention. J Am Med Inform Assoc. 1999 Aug;6(4):313–21. [PMC free article] [PubMed]

4. Bates DW, O’Neil AC, Boyle D, Teich J, Chertow GM, Komaroff AL, et al. Potential identifiability and preventability of adverse events using information systems. J Am Med Inform Assoc. 1994;1(5):404–11. [PMC free article] [PubMed]

5. van der Sijs H, Aarts J, Vulto A, Berg M. Overriding of Drug Safety Alerts in Computerized Physician Order Entry. J Am Med Inform Assoc. 2006 Mar 1;13(2):138–47. [PMC free article] [PubMed]

6. Koppel R, Metlay JP, Cohen A, Abaluck B, Localio AR, Kimmel SE, et al. Role of computerized physician order entry systems in facilitating medication errors. JAMA. 2005 Mar 9;293(10):1197–203. [PubMed]

7. Ash JS, Sittig DF, Dykstra R, Campbell E, Guappone K. The unintended consequences of computerized provider order entry: findings from a mixed methods exploration. Int J Med Inform. 2009 Apr;78( Suppl 1):S69–76. [PMC free article] [PubMed]

8. Sittig DF, Singh H. Defining health information technology-related errors: new developments since to err is human. Arch Intern Med. 2011 Jul 25;171(14):1281–4. [PubMed]

9. McCoy AB, Waitman LR, Lewis JB, Wright JA, Choma DP, Miller RA, et al. A framework for evaluating the appropriateness of clinical decision support alerts and responses. J Am Med Inform Assoc. 2012 May 1;19(3):346–52. [PMC free article] [PubMed]

10. Kucukarslan SN, Peters M, Mlynarek M, Nafziger DA. Pharmacists on rounding teams reduce preventable adverse drug events in hospital general medicine units. Arch Intern Med. 2003 Sep 22;163(17):2014–8. [PubMed]

11. Leape LL, Cullen DJ, Clapp MD, Burdick E, Demonaco HJ, Erickson JI, et al. Pharmacist participation on physician rounds and adverse drug events in the intensive care unit. JAMA. 1999 Jul 21;282(3):267–70. [PubMed]

12. Kaushal R, Bates DW, Abramson EL, Soukup JR, Goldmann DA. Unit-based clinical pharmacists’ prevention of serious medication errors in pediatric inpatients. Am J Health Syst Pharm. 2008 Jul 1;65(13):1254–60. [PubMed]

13. Schnipper JL, Kirwin JL, Cotugno MC, Wahlstrom SA, Brown BA, Tarvin E, et al. Role of pharmacist counseling in preventing adverse drug events after hospitalization. Arch Intern Med. 2006 Mar 13;166(5):565–71. [PubMed]

14. Fertleman M, Barnett N, Patel T. Improving medication management for patients: the effect of a pharmacist on post-admission ward rounds. Qual Saf Health Care. 2005 Jun;14(3):207–11. [PMC free article] [PubMed]

15. Falconnier AD, Haefeli WE, Schoenenberger RA, Surber C, Martin-Facklam M. Drug dosage in patients with renal failure optimized by immediate concurrent feedback. J Gen Intern Med. 2001 Jun;16(6):369–75. [PMC free article] [PubMed]

16. Brown G. Assessing the clinical impact of pharmacists’ interventions. Am J Hosp Pharm. 1991 Dec;48(12):2644–7. [PubMed]

17. Bladh L, Ottosson E, Karlsson J, Klintberg L, Wallerstedt SM. Effects of a clinical pharmacist service on health-related quality of life and prescribing of drugs: a randomised controlled trial. BMJ Qual Saf. 2011 Sep;20(9):738–46. [PubMed]

18. Kilbridge PM, Alexander L, Ahmad A. Implementation of a system for computerized adverse drug event surveillance and intervention at an academic medical center. J Clin Outcomes Manage. 2006;13(2):94–100.

19. Classen DC, Pestotnik SL, Evans RS, Burke JP. Computerized surveillance of adverse drug events in hospital patients. JAMA. 1991 Nov 27;266(20):2847–51. [PubMed]

20. Jha AK, Laguette J, Seger A, Bates DW. Can surveillance systems identify and avert adverse drug events? A prospective evaluation of a commercial application. J Am Med Inform Assoc. 2008 Oct;15(5):647–53. [PMC free article] [PubMed]

21. Evans RS, Pestotnik SL, Classen DC, Burke JP. Evaluation of a computer-assisted antibiotic-dose monitor. Ann Pharmacother. 1999 Oct;33(10):1026–31. [PubMed]

22. McCoy AB, Waitman LR, Gadd CS, Danciu I, Smith JP, Lewis JB, et al. A computerized provider order entry intervention for medication safety during acute kidney injury: a quality improvement report. Am J Kidney Dis. 2010 Nov;56(5):832–41. [PMC free article] [PubMed]

23. Chertow GM, Lee J, Kuperman GJ, Burdick E, Horsky J, Seger DL, et al. Guided Medication Dosing for Inpatients With Renal Insufficiency. JAMA. 2001 Dec 12;286(22):2839–44. [PubMed]

24. Sellier E, Colombet I, Sabatier B, Breton G, Nies J, Zapletal E, et al. Effect of alerts for drug dosage adjustment in inpatients with renal insufficiency. J Am Med Inform Assoc. 2009 Apr;16(2):203–10. [PMC free article] [PubMed]

25. Rind DM, Safran C, Phillips RS, Wang Q, Calkins DR, Delbanco TL, et al. Effect of computer-based alerts on the treatment and outcomes of hospitalized patients. Arch Intern Med. 1994 Jul 11;154(13):1511–7. [PubMed]

26. Field TS, Rochon P, Lee M, Gavendo L, Baril JL, Gurwitz JH. Computerized clinical decision support during medication ordering for long-term care residents with renal insufficiency. J Am Med Inform Assoc. 2009 Aug;16(4):480–5. [PMC free article] [PubMed]

27. Cox ZL, Nelsen CL, Waitman LR, McCoy JA, Peterson JF. Effects of clinical decision support on initial dosing and monitoring of tobramycin and amikacin. Am J Health Syst Pharm. 2011 Apr 1;68(7):624–32. [PMC free article] [PubMed]

28. Geissbühler A, Miller RA. A new approach to the implementation of direct care-provider order entry. Proc AMIA Annu Fall Symp. 1996:689–93. [PMC free article] [PubMed]

29. Giuse DA. Supporting communication in an integrated patient record system. AMIA Annu Symp Proc. 2003:1065. [PMC free article] [PubMed]

30. Phillips I, Nelsen C, Peterson J, Sullivan T, Waitman L. Improving aminoglycoside dosing through computerized clinical decision support and pharmacy therapeutic monitoring systems. AMIA Annu Symp Proc. 2008:1093. [PubMed]

31. Gurwitz JH, Field TS, Harrold LR, Rothschild J, Debellis K, Seger AC, et al. Incidence and preventability of adverse drug events among older persons in the ambulatory setting. JAMA. 2003 Mar 5;289(9):1107–16. [PubMed]

32. McCoy A, Peterson JF. The Greasemonkey Firefox Add-On for Altering Display of Data in a Web-Based Electronic Medical Record. AMIA Annu Symp Proc; 2011. p. 1884.

33. Bates DW, Leape LL, Petrycki S. Incidence and preventability of adverse drug events in hospitalized adults. J Gen Intern Med. 1993 Jun;8(6):289–94. [PubMed]

34. Pippins JR, Gandhi TK, Hamann C, Ndumele CD, Labonville SA, Diedrichsen EK, et al. Classifying and predicting errors of inpatient medication reconciliation. J Gen Intern Med. 2008 Sep;23(9):1414–22. [PMC free article] [PubMed]

35. Leape LL, Bates DW, Cullen DJ, Cooper J, Demonaco HJ, Gallivan T, et al. Systems Analysis of Adverse Drug Events. JAMA. 1995 Jul 5;274(1):35–43. [PubMed]

36. Bates DW, Leape LL, Cullen DJ, Laird N, Petersen LA, Teich JM, et al. Effect of computerized physician order entry and a team intervention on prevention of serious medication errors. JAMA. 1998 Oct 21;280(15):1311–6. [PubMed]

37. Jha AK, Kuperman GJ, Teich JM, Leape L, Shea B, Rittenberg E, et al. Identifying adverse drug events: development of a computer-based monitor and comparison with chart review and stimulated voluntary report. J Am Med Inform Assoc. 1998 Jun;5(3):305–14. [PMC free article] [PubMed]

38. Kane-Gill SL, Visweswaran S, Saul MI, Wong A-KI, Penrod LE, Handler SM. Computerized detection of adverse drug reactions in the medical intensive care unit. Int J Med Inform. 2011 Aug;80(8):570–8. [PMC free article] [PubMed]

39. Kilbridge PM, Noirot LA, Reichley RM, Berchelmann KM, Schneider C, Heard KM, et al. Computerized surveillance for adverse drug events in a pediatric hospital. J Am Med Inform Assoc. 2009 Oct;16(5):607–12. [PMC free article] [PubMed]

40. Tinoco A, Evans RS, Staes CJ, Lloyd JF, Rothschild JM, Haug PJ. Comparison of computerized surveillance and manual chart review for adverse events. J Am Med Inform Assoc. 2011 Aug;18(4):491–7. [PMC free article] [PubMed]

41. Forster AJ, Worthington JR, Hawken S, Bourke M, Rubens F, Shojania K, et al. Using prospective clinical surveillance to identify adverse events in hospital. BMJ Qual Saf. 2011 Sep;20(9):756–63. [PMC free article] [PubMed]

42. Reynolds G, Boyer D, Mackey K, Povondra L, Cummings A. Alerting Strategies in Computerized Physician Order Entry: A Novel Use of a Dashboard-style Analytics Tool in a Childrens Hospital. AMIA Annu Symp Proc; 2008. p. 1108.

43. Zimmerman CR, Jackson A, Chaffee B, O’Reilly M. A dashboard model for monitoring alert effectiveness and bandwidth. AMIA Annu Symp Proc; 2007. p. 1176.

44. Waitman LR, Phillips IE, McCoy AB, Danciu I, Halpenny RM, Nelsen CL, et al. Adopting real-time surveillance dashboards as a component of an enterprisewide medication safety strategy. Jt Comm J Qual Patient Saf. 2011 Jul;37(7):326–32. [PubMed]

45. Hug BL, Witkowski DJ, Sox CM, Keohane CA, Seger DL, Yoon C, et al. Adverse drug event rates in six community hospitals and the potential impact of computerized physician order entry for prevention. J Gen Intern Med. 2010 Jan;25(1):31–8. [PMC free article] [PubMed]