Impact of health information technology interventions to improve medication laboratory monitoring for ambulatory patients: a systematic review

doi:10.1136/jamia.2009.000794

J Am Med Inform Assoc. 2010 Nov-Dec; 17(6): 631–636.

doi: 10.1136/jamia.2009.000794

PMCID: PMC3000763

Impact of health information technology interventions to improve medication laboratory monitoring for ambulatory patients: a systematic review

Shira H Fischer,^1,² Jennifer Tjia,^1,² and Terry S Field^1,²

¹Division of Geriatric Medicine, University of Massachusetts Medical School, Worcester, Massachusetts, USA

²Meyers Primary Care Institute, Worcester, Massachusetts, USA

Corresponding author.

Correspondence to Shira H Fischer, University of Massachusetts Medical School, 630 Plantation Street, Meyers Primary Care Institute, Worcester, MA 01605, USA; Email: shira.fischer/at/umassmed.edu

Received August 3, 2009; Accepted August 23, 2010.

Abstract

Medication errors are a major source of morbidity and mortality. Inadequate laboratory monitoring of high-risk medications after initial prescription is a medical error that contributes to preventable adverse drug events. Health information technology (HIT)-based clinical decision support may improve patient safety by improving the laboratory monitoring of high-risk medications, but the effectiveness of such interventions is unclear. Therefore, the authors conducted a systematic review to identify studies that evaluate the independent effect of HIT interventions on improving laboratory monitoring for high-risk medications in the ambulatory setting using a Medline search from January 1, 1980 through January 1, 2009 and a manual review of relevant bibliographies. All anticoagulation monitoring studies were excluded. Eight articles met the inclusion criteria, including six randomized controlled trials and two pre–post intervention studies. Six of the studies were conducted in two large, integrated healthcare delivery systems in the USA. Overall, five of the eight studies reported statistically significant, but small, improvements in laboratory monitoring; only half of the randomized controlled trials reported statistically significant improvements. Studies that found no improvement were more likely to have used analytic strategies that addressed clustering and confounding. Whether HIT improves laboratory monitoring of certain high-risk medications for ambulatory patients remains unclear, and further research is needed to clarify this important question.

Introduction

Since the Institute of Medicine highlighted the impact of medical errors on patient morbidity and mortality in To Err is Human,¹ significant effort has focused on reducing medical errors and improving patient safety in the USA. Medical errors result in 44 000–98 000 deaths per year, a large proportion of which are due to adverse drug events (ADEs).¹ Laboratory monitoring errors are a major cause of potential ADEs, occurring in 60.8% of preventable ADEs in ambulatory older adults² and in 45.4% of preventable ADEs requiring hospital admission.³ Baseline monitoring rates are low, with up to 58% of initial drug dispensings occurring without appropriate laboratory monitoring for ambulatory older adults.⁴ Because patients sometimes miss more than one test for a given drug and often take many drugs, the rate of all potential laboratory-monitoring errors was estimated to be extremely high (~80%) among patients taking chronic medications in 2001.⁵ Because poor adherence to guidelines leads to hospitalizations and significant morbidity,³ ⁶ and because basic human factors make it challenging for clinicians to adhere to complicated monitoring recommendations for a large number of medications, health information technology (HIT) holds promise for improving laboratory monitoring of high-risk medications and may potentially reduce medication errors.⁷ ⁸

Some experts estimate that up to 95% of potential ADEs can be avoided with the adoption of advanced computerized systems.⁸ As a result, tools to reduce errors continue to be developed, many of which are technology based. However, the actual impact of these systems is unclear. Technology and clinical decision support (CDS) systems have been shown to improve patient care and clinical outcomes in many clinical situations.⁹ For example, computerized physician order entry with decision support can reduce medication errors,¹⁰ and interventions to improve laboratory monitoring in the hospital setting can improve outcomes.¹¹ Furthermore, computer access to laboratory data improves the opportunity for pharmacists to monitor medications,¹² ¹³ and systematic reviews of interventions to improve monitoring in the hospital setting show that HIT can reduce errors.¹⁰ ^14–16 Unfortunately, it remains unclear whether HIT CDS alerts in the ambulatory setting are as effective.

To address this gap in the literature, we conducted a systematic review to identify studies that evaluated HIT interventions to improve laboratory monitoring of selected high-risk medications in the ambulatory setting. The specific aims of this review are to answer the following questions regarding high-risk medications (excluding anticoagulants) in the ambulatory setting. (1) Do HIT interventions improve laboratory monitoring? (2) What are the characteristics of HIT interventions that improve monitoring? This review should inform the planning of laboratory monitoring interventions and guide future researchers about important research design elements for HIT interventions.

Methods

Literature search

To identify journal articles for this systematic review, we performed a Medline search of English-language human studies published between January 1, 1980 and January 1, 2009 using keywords for HIT and drug monitoring. The search performed was as follows: ‘drug monitoring’ OR ‘laboratory monitoring’) AND (computerized OR electronic OR informatics OR reminder systems OR ‘Medical Records Systems, Computerized’ (MeSH) OR ‘Decision Support Systems, Clinical’ (MeSH) OR ‘Decision Making, Computer-Assisted’ (MeSH) OR ‘Database Management Systems’ (MeSH). We performed a manual review of relevant authors and journals including bibliographies from identified articles.

Inclusion criteria and selection of studies

We included studies that: were clinical trials, randomized controlled trials (RCTs), or comparative studies; were conducted in an ambulatory setting; had sufficient information about the HIT intervention for it to be assessed separately from other non-HIT interventions; and examined laboratory test monitoring rather than clinical tests (eg, pulmonary function tests). We included studies evaluating the effect of HIT interventions on laboratory test monitoring, defined as laboratory tests to evaluate efficacy, toxicity, or side effects. We excluded studies that examined laboratory testing to evaluate medication adherence, computerized order interventions that did not include laboratory monitoring, at-home patient testing, in-hospital interventions, and literature reviews, meta-analyses, case studies, and opinion pieces. We also excluded studies in which the HIT monitoring intervention was coupled with other interventions (eg, HIT-based medication dosing and appointment scheduling recommendations) because it was not possible to identify the independent effect of HIT on laboratory monitoring; this included all studies evaluating anticoagulation interventions and several multipronged diabetes interventions. Additional studies identified from bibliographies and author searches were also evaluated for inclusion based on the same criteria.

The literature search produced 347 abstracts, of which 314 were in English and published from January 1, 1980 to January 1, 2009 (figure 1). Each study was assessed independently by two investigators (SHF and JT) for inclusion. Disagreements were resolved by consensus. There were 256 studies after exclusion of review articles, and most were excluded after manual review for not meeting the inclusion criteria. Many studies were excluded for more than one reason, such as not being an actual trial and covering a topic other than laboratory monitoring.

Figure 1

Flow diagram of included and excluded studies. HIT, health information technology.

Data abstraction and evaluation

We extracted data from the text and tables of the original publications and classified by clinical setting, targeted medications, time frame, HIT intervention type and duration, randomization, comparison group, and endpoint assessed. In one case, investigators contacted a study author for additional results.

Quality scores were assigned by two investigators (SHF and JT) using an approach outlined by Downs and Black¹⁷ to assess methodological quality. This approach standardizes and rates important aspects of study design and data presentation to assign an overall study quality rating score. The maximum score possible for an original investigation was 27. Disagreements were reconciled by consensus.

Results

A detailed review of the potentially eligible articles identified a total of eight articles for inclusion that evaluated the impact of HIT interventions on laboratory monitoring in the ambulatory settings published between 2003 and 2009. A brief description of these studies is presented in table 1.

Table 1

Characteristics of studies included in the systematic review

Seven studies were conducted in the USA⁶ ¹⁸ ^20–24 and one in Israel.¹⁹ Six of the eight studies were conducted in large, integrated healthcare delivery systems,⁶ ¹⁸ ^20–23 including a series of studies by Raebel et al at Kaiser Permanente¹⁸ ^21–23 and two studies at Partners HealthCare.⁶ ²⁰ Five interventions sent electronic alerts to prescribing physicians alone.⁶ ^19–21 ²⁴ Three sent electronic alerts to a pharmacist who could then order the laboratory test and contact the patient.¹⁸ ²² ²³ One of the three studies that involved pharmacists also included a comparison arm of computerized alerts to physicians only.¹⁸ Seven studies targeted a broad range of medications,⁶ ¹⁸ ^20–24 while the eighth targeted a single medication.¹⁹ Six studies evaluated completion of laboratory test monitoring as the outcome measure,⁶ ¹⁸ ¹⁹ ^21–23 while two evaluated physician test ordering.²⁰ ²⁴ A meta-analysis of the data reported was deemed inappropriate because of the differences between the studies.

Five of the eight studies reported statistically significant improvements in laboratory monitoring attributable to the study intervention,¹⁸ ¹⁹ ^22–24 whether an improvement in appropriate tests ordered or an increase in the completion rate, with the absolute per cent improvement ranging from 3.0% to 26.1%. There was no consistent pattern of intervention efficacy based on outcome measurement. The number of patients enrolled in each study ranged from 196 to 26 586. The smallest study showed the largest absolute improvement in monitoring.¹⁸

Study quality and impact on laboratory monitoring

Six of the eight studies were RCTs, while two were pre–post intervention studies. A brief description of the study methodologies and quality rating score is included in table 1. The study quality rating scores ranged from 16 to 25 (possible score range 0–27). The RCTs were rated higher (quality score=22–25) than the pre–post intervention studies (quality score=16–18). Studies with the highest scores differed from lower quality studies in their analytic approaches by including adjustment for confounding and clustering.⁶ ¹⁸ ²⁰ Interestingly, randomization failed in two of the highest quality studies,⁶ ²⁰ where the intervention and control groups were significantly different on key clinical characteristics such as gender, race, and insurance type.

Both pre–post studies showed statistically significant improvements,¹⁹ ²⁴ while only three of the six RCTs did.¹⁸ ²² ²³ All of the RCTs that showed improvements involved pharmacist-based interventions; this included the only RCT that showed improvement by an alert targeting physicians, and this intervention was evaluated as the comparison arm for more intensive pharmacist-based intervention.¹⁸

All studies enrolled patients nested within providers; two multi-site studies were cluster randomized trials at the level of the clinic, nesting providers within each site.¹⁸ ²⁰ Three studies accounted for clustering at the level of the clinic or provider in the analyses or design,⁶ ¹⁸ ²⁰ and two of these reported no improvements in monitoring with HIT intervention.⁶ ²⁰ While all studies listed some possible patient-level or facility-level confounders, only the same three studies adjusted for these possible confounders in their analyses,⁶ ¹⁸ ²⁰ and two of these studies showed no intervention improvements.⁶ ²⁰ In addition, of the six RCTs, the three with failures in randomization reported no improvement in monitoring, after any adjustment.⁶ ²⁰ ²¹

Study site characteristics and impact on laboratory monitoring

Six of the eight studies were conducted in one of two large integrated healthcare delivery systems, Kaiser Permanente and Partners HealthCare; these included all of the RCTs.⁶ ¹⁸ ^20–23 One study was conducted in a safety-net clinic,²⁴ and one study in multiple health maintenance organization sites in Israel.¹⁹ Baseline rates of appropriate laboratory monitoring varied between study sites, ranging from 14%⁶ to greater than 95%²² depending on the study drug. Sites with lower baseline rates of monitoring reported greater improvements associated with HIT interventions.¹⁸ ²³ ²⁴ Both studies from Partners HealthCare showed no improvements with HIT interventions, but had high baseline rates of monitoring prior to the intervention.⁶ ²⁰ The safety-net clinic study and the Israeli health maintenance organization study had different baseline levels (38.5% and 78.5%), but both showed significant monitoring improvements in their pre–post intervention assessments.¹⁹ ²⁴

Intervention design and impact on laboratory monitoring

All the studies were conducted within healthcare systems with electronic records. Four interventions were based on homegrown electronic medical records programs,⁶ ^19–21 while four were based on modifications to proprietary systems.¹⁸ ^22–24 Six of the eight interventions were built-in systems with computerized physician order entry (CPOE), with the alert going to the physician. Of these, two sent messages via email,¹⁸ ¹⁹ while four provided alerts during patient profile reviews.⁶ ²⁰ ²¹ ²⁴

Of the CPOE interventions, alerts within the electronic record system were either interruptive (requiring the provider to respond to the alert) or non-interruptive (not requiring action). In one study, the intervention was interruptive and required action on the part of the provider to dismiss an alert²⁴; however, this intervention did not shorten the process of test or medication ordering. Other studies had real-time alerts that appeared on the prescribing page as a warning, but they were non-interruptive and did not stop the workflow.⁶ ²⁰ ²¹ No aspect of the CPOE design itself was found to be consistently more effective than any other. Interestingly, the two studies that alerted pharmacists directly, but not physicians, demonstrated significant improvements in monitoring.²² ²³

Most interventions reviewed targeted multiple high-risk medications, while one involved only a single drug.¹⁹ When we examined the impact of interventions on the same drug, diuretics, across all the studies, we found no significant effect of the HIT intervention except in the study for which this was the only drug targeted¹⁹ (table 2). There was no consistency between the medications targeted and whether there was a significant intervention effect.

Table 2

Comparison of serum potassium monitoring for diuretic use across reviewed studies

Discussion

By 2009, eight studies reported the results of HIT interventions to improve laboratory monitoring of medications in the ambulatory setting, including six RCTs. Surprisingly, 50% of the RCTs reported significant improvements in monitoring while 50% did not. A detailed review of each of the studies identified important aspects of study quality, analysis and intervention design that help to explain these conflicting results.

Higher quality studies were less likely to show significant improvements in monitoring with HIT interventions compared with lower quality studies. Studies with lower quality scores¹⁹ ²⁴ used less rigorous study designs (such as pre–post intervention timing rather than RCT) and analytic approaches. These differences may explain some of the differences in intervention efficacy across studies. Because most of the HIT intervention studies were introduced in clinical systems with multiple clinical sites, it is important to account for non-independence of outcomes within each site due to local practice variations that can explain differences between different sites. Likewise, because clinicians cared for multiple patients within a site, it is important to consider non-independence of outcomes (ie, laboratory testing) between patients of the same provider because differences in care delivery between providers can also affect outcomes. Our review found that studies that addressed clustering in their design and analysis were less likely to show improvements in laboratory monitoring.⁶ ²⁰

We also found that all of the RCTs were conducted in one of two large integrated healthcare systems in the USA. Study setting appears to be related to study results in two ways. First, both studies conducted outside of a large integrated healthcare system in the USA were less rigorous pre–post intervention trials,¹⁹ ²⁴ and each showed significant improvements. Second, one of the integrated healthcare systems had high baseline rates of monitoring,⁶ ²⁰ and our review indicates that studies in sites with lower baseline rates of monitoring reported greater improvements from HIT interventions than sites with higher baseline monitoring rates.¹⁸ ²³ ²⁴

Intervention design features may also explain the conflicting study results. Our review revealed that the two interventions that targeted pharmacists were effective, ²² ²³ while only three of six interventions targeting physicians were effective.¹⁸ ¹⁹ ²⁴ One study compared three arms, including an arm with electronic alerts to physicians, a second arm with voice mail messages to patients, and a third with pharmacy team outreach to patients, and found that the physician alert arm was the least effective.¹⁸ Past evidence suggests that changing physician behavior is challenging, and that passive approaches (non-interruptive alerts) to such physicians may not be effective.^25–27 It does not appear that the intrusiveness of the alert explains the difference in study findings, and this is consistent with several studies in which non-intrusive reminders did not improve physician adherence to alert recommendations.²⁸

It is helpful to consider our results in the context of other literature on the effectiveness of HIT interventions and their effects on prescribing errors and ADEs.⁸ ¹⁰ Most reviews included a small number of studies, and many report that the studies reviewed were of low quality. For example, a 2003 review reporting error-rate improvement from CDS-only interventions included seven studies, many of which were under-powered.¹⁰ Another review of HIT interventions to improve drug dosing, mostly in the inpatient setting, found that many studies were of low quality.¹⁵ ¹⁶ None of these studies addressed laboratory monitoring.

Variation in intervention effectiveness is also reported in other reviews of HIT interventions. For example, one review of the effect of CPOE and CDS on ADEs found that only half of the studies showed a reduction in ADEs²⁹, and another systematic review of CPOE and medical errors reported that, while more than half of studies found significant reductions in ADEs, the results varied widely. Although the investigators concluded that CPOE can reduce prescribing errors, they noted, ‘Reporting quality and study quality was often insufficient to exclude major sources of bias.’³⁰ The findings of our review are similar, with a slight majority of studies finding a positive impact of the interventions, but with variation in quality. As with other reviews, the number of studies addressing this issue is still limited.

There are several limitations to our review that should be noted. First, given the relatively small number of studies identified, it is difficult to draw conclusions about the overall impact of HIT intervention on rates of laboratory monitoring. By limiting our search to Medline English-language studies, we may have missed some non-US studies, but this allowed us to adequately review the study methodologies. Further, all studies regarding anticoagulation were excluded because it was not possible to identify the independent effect of interventions in improving laboratory monitoring (ie, INR testing) from dosing recommendations for warfarin. This limits the inferences we can make about HIT interventions on laboratory monitoring overall. Second, the studies were conducted in a limited number of clinical settings: three of the studies were conducted at one site and two at a second site. Further, all but one of the studies were conducted in large managed care organizations, limiting generalizability of the findings outside of these settings. Finally, differences in study design made it difficult to compare outcomes across studies. While we were unable to use meta-analysis to pool the effect sizes, we did compare the effects of the interventions across several studies on a single drug common to all studies, and did not find any consistent effect of HIT interventions on monitoring.

Even though the idea of using HIT to improve quality of care is not new,³¹ this goal has not yet been achieved. Many questions still remain, as posed by Kuperman et al³² in 2007: ‘To what extent does alerting impact on clinician behavior and patient outcomes? What is the optimal way to present alerts to prescribers? Which member of the healthcare team—for example, physician, nurse, pharmacist, other—is the best recipient of any kind of alert?’ These questions have yet to be answered. As more outpatient clinics adopt electronic records and electronic prescribing, it will be increasingly important to know the impact of decision support in this setting in supporting implementation of the most effective interventions. This is particularly true with regard to laboratory monitoring, which is often a locus for preventable adverse effects.

Although numerous reviews and studies have attempted to answer these questions, our systematic search identified more interventions in the inpatient setting than in the ambulatory setting. Of the studies identified, concerns about study quality and design could not exclude sources of bias in the reported results. Future studies of laboratory monitoring should better address patient and provider characteristics and account for fixed physician or clinical site effects by multilevel analysis. Studies can also better clarify outcomes (ie, improvements in test ordering versus test completion), and should also be expanded to include settings outside of the large integrated healthcare delivery systems.

While this systematic review found evidence suggesting that information technology interventions may improve laboratory monitoring for high-risk prescribed medications (exclusive of anticoagulants) in the ambulatory setting, the evidence is conflicting. The well-considered, well-designed studies reviewed appeared to find little improvement in laboratory monitoring for high-risk medications with HIT interventions targeting physicians only. However, five of eight studies found some positive effect, and this suggests that using HIT may be a promising avenue for improving laboratory monitoring. More research is needed to determine how to maximize the full potential benefit of HIT for monitoring high-risk medications and ultimately improving patient safety.

Acknowledgments

We thank Dr Gordon FitzGerald, Dr Patricia Franklin, and Dr Barry Saver for helpful comments on this manuscript.

Footnotes

Competing interests: None.

Provenance and peer review: Not commissioned; externally peer reviewed.

References

1. Institute of Medicine To Err is Human: Building a Safer Health System. Washington, DC: National Academy Press, 2000.

2. Gurwitz JH, Field TS, Harrold LR, et al. Incidence and preventability of adverse drug events among older persons in the ambulatory setting. JAMA 2003;289:1107–16. [PubMed]

3. Thomsen LA, Winterstein AG, Sondergaard B, et al. Systematic review of the incidence and characteristics of preventable adverse drug events in ambulatory care. Ann Pharmacother 2007;41:1411–26. [PubMed]

4. Simon SR, Andrade SE, Ellis JL, et al. Baseline laboratory monitoring of cardiovascular medications in elderly health maintenance organization enrollees. J Am Geriatr Soc 2005;53:2165–9. [PubMed]

5. Hurley JS, Roberts M, Solberg LI, et al. Laboratory safety monitoring of chronic medications in ambulatory care settings. J Gen Intern Med 2005;20:331–3. [PMC free article] [PubMed]

6. Matheny ME, Sequist TD, Seger AC, et al. A randomized trial of electronic clinical reminders to improve medication laboratory monitoring. J Am Med Inform Assoc 2008;15:424–9. [PMC free article] [PubMed]

7. Bates DW, Teich JM, Lee J, et al. The impact of computerized physician order entry on medication error prevention. J Am Med Inform Assoc 1999;6:313–21. [PMC free article] [PubMed]

8. Gandhi TK, Weingart SN, Seger AC, et al. Outpatient prescribing errors and the impact of computerized prescribing. J Gen Intern Med 2005;20:837–41. [PMC free article] [PubMed]

9. Osheroff JA, Teich JM, Middleton B, et al. A roadmap for national action on clinical decision support. J Am Med Inform Assoc 2007;14:141–5. [PMC free article] [PubMed]

10. Kaushal R, Shojania KG, Bates DW. Effects of computerized physician order entry and clinical decision support systems on medication safety: a systematic review. Arch Intern Med 2003;163:1409–16. [PubMed]

11. Rind DM, Safran C, Phillips RS, et al. Effect of computer-based alerts on the treatment and outcomes of hospitalized patients. Arch Intern Med 1994;154:1511–17. [PubMed]

12. Pedersen CA, Schneider PJ, Scheckelhoff DJ. ASHP national survey of pharmacy practice in hospital settings: monitoring and patient education–2006. Am J Health Syst Pharm 2007;64:507–20. [PubMed]

13. Chaudhry B, Wang J, Wu S, et al. Systematic review: impact of health information technology on quality, efficiency, and costs of medical care. Ann Intern Med 2006;144:742–52. [PubMed]

14. Handler SM, Altman RL, Perera S, et al. A systematic review of the performance characteristics of clinical event monitor signals used to detect adverse drug events in the hospital setting. J Am Med Inform Assoc 2007;14:451–8. [PMC free article] [PubMed]

15. Durieux P, Trinquart L, Colombet I, et al. Computerized advice on drug dosage to improve prescribing practice. Cochrane Database Syst Rev 2008;(3):CD002894. [PubMed]

16. Walton RT, Harvey E, Dovey S, et al. Computerised advice on drug dosage to improve prescribing practice. Cochrane Database Syst Rev 2001;(1):CD002894. [PubMed]

17. Downs SH, Black N. The feasibility of creating a checklist for the assessment of the methodological quality both of randomised and non-randomised studies of health care interventions. J Epidemiol Community Health 1998;52:377–84. [PMC free article] [PubMed]

18. Feldstein AC, Smith DH, Perrin N, et al. Improved therapeutic monitoring with several interventions: a randomized trial. Arch Intern Med 2006;166:1848–54. [PubMed]

19. Hoch I, Heymann AD, Kurman I, et al. Countrywide computer alerts to community physicians improve potassium testing in patients receiving diuretics. J Am Med Inform Assoc 2003;10:541–6. [PMC free article] [PubMed]

20. Lo HG, Matheny ME, Seger DL, et al. Impact of non-interruptive medication laboratory monitoring alerts in ambulatory care. J Am Med Inform Assoc 2009;16:66–71. [PMC free article] [PubMed]

21. Palen TE, Raebel M, Lyons E, et al. Evaluation of laboratory monitoring alerts within a computerized physician order entry system for medication orders. Am J Manag Care 2006;12:389–95. [PubMed]

22. Raebel MA, Lyons EE, Chester EA, et al. Improving laboratory monitoring at initiation of drug therapy in ambulatory care: a randomized trial. Arch Intern Med 2005;165:2395–401. [PubMed]

23. Raebel MA, Chester EA, Newsom EE, et al. Randomized trial to improve laboratory safety monitoring of ongoing drug therapy in ambulatory patients. Pharmacotherapy 2006;26:619–26. [PubMed]

24. Steele AW, Eisert S, Witter J, et al. The effect of automated alerts on provider ordering behavior in an outpatient setting. PLoS Med 2005;2:e255. [PMC free article] [PubMed]

25. Grimshaw JM, Shirran L, Thomas R, et al. Changing provider behavior: an overview of systematic reviews of interventions. Med Care 2001;39(8 Suppl 2):II2–45. [PubMed]

26. Graham DJ, Drinkard CR, Shatin D, et al. Liver enzyme monitoring in patients treated with troglitazone. JAMA 2001;286:831–3. [PubMed]

27. Krall MA, Sittig DF. Clinician's assessments of outpatient electronic medical record alert and reminder usability and usefulness requirements. Proc AMIA Symp 2002:400–4. [PMC free article] [PubMed]

28. Majumdar SR, Soumerai SB. Why most interventions to improve physician prescribing do not seem to work. CMAJ 2003;169:30–1. [PMC free article] [PubMed]

29. Wolfstadt JI, Gurwitz JH, Field TS, et al. The effect of computerized physician order entry with clinical decision support on the rates of adverse drug events: a systematic review. J Gen Intern Med 2008;23:451–8. [PMC free article] [PubMed]

30. Ammenwerth E, Schnell-Inderst P, Machan C, et al. The effect of electronic prescribing on medication errors and adverse drug events: a systematic review. J Am Med Inform Assoc 2008;15:585–600. [PMC free article] [PubMed]

31. Elin RJ. Computer-assisted therapeutic drug monitoring. Clin Lab Med 1987;7:485–92. [PubMed]

32. Kuperman GJ, Bobb A, Payne TH, et al. Medication-related clinical decision support in computerized provider order entry systems: a review. J Am Med Inform Assoc 2007;14:29–40. [PMC free article] [PubMed]

Articles from Journal of the American Medical Informatics Association : JAMIA are provided here courtesy of
American Medical Informatics Association