By 2009, eight studies reported the results of HIT interventions to improve laboratory monitoring of medications in the ambulatory setting, including six RCTs. Surprisingly, 50% of the RCTs reported significant improvements in monitoring while 50% did not. A detailed review of each of the studies identified important aspects of study quality, analysis and intervention design that help to explain these conflicting results.
Higher quality studies were less likely to show significant improvements in monitoring with HIT interventions compared with lower quality studies. Studies with lower quality scores19
used less rigorous study designs (such as pre–post intervention timing rather than RCT) and analytic approaches. These differences may explain some of the differences in intervention efficacy across studies. Because most of the HIT intervention studies were introduced in clinical systems with multiple clinical sites, it is important to account for non-independence of outcomes within each site due to local practice variations that can explain differences between different sites. Likewise, because clinicians cared for multiple patients within a site, it is important to consider non-independence of outcomes (ie, laboratory testing) between patients of the same provider because differences in care delivery between providers can also affect outcomes. Our review found that studies that addressed clustering in their design and analysis were less likely to show improvements in laboratory monitoring.6
We also found that all of the RCTs were conducted in one of two large integrated healthcare systems in the USA. Study setting appears to be related to study results in two ways. First, both studies conducted outside of a large integrated healthcare system in the USA were less rigorous pre–post intervention trials,19
and each showed significant improvements. Second, one of the integrated healthcare systems had high baseline rates of monitoring,6
and our review indicates that studies in sites with lower baseline rates of monitoring reported greater improvements from HIT interventions than sites with higher baseline monitoring rates.18
Intervention design features may also explain the conflicting study results. Our review revealed that the two interventions that targeted pharmacists were effective, 22
while only three of six interventions targeting physicians were effective.18
One study compared three arms, including an arm with electronic alerts to physicians, a second arm with voice mail messages to patients, and a third with pharmacy team outreach to patients, and found that the physician alert arm was the least effective.18
Past evidence suggests that changing physician behavior is challenging, and that passive approaches (non-interruptive alerts) to such physicians may not be effective.25–27
It does not appear that the intrusiveness of the alert explains the difference in study findings, and this is consistent with several studies in which non-intrusive reminders did not improve physician adherence to alert recommendations.28
It is helpful to consider our results in the context of other literature on the effectiveness of HIT interventions and their effects on prescribing errors and ADEs.8
Most reviews included a small number of studies, and many report that the studies reviewed were of low quality. For example, a 2003 review reporting error-rate improvement from CDS-only interventions included seven studies, many of which were under-powered.10
Another review of HIT interventions to improve drug dosing, mostly in the inpatient setting, found that many studies were of low quality.15
None of these studies addressed laboratory monitoring.
Variation in intervention effectiveness is also reported in other reviews of HIT interventions. For example, one review of the effect of CPOE and CDS on ADEs found that only half of the studies showed a reduction in ADEs29
, and another systematic review of CPOE and medical errors reported that, while more than half of studies found significant reductions in ADEs, the results varied widely. Although the investigators concluded that CPOE can reduce prescribing errors, they noted, ‘Reporting quality and study quality was often insufficient to exclude major sources of bias.’30
The findings of our review are similar, with a slight majority of studies finding a positive impact of the interventions, but with variation in quality. As with other reviews, the number of studies addressing this issue is still limited.
There are several limitations to our review that should be noted. First, given the relatively small number of studies identified, it is difficult to draw conclusions about the overall impact of HIT intervention on rates of laboratory monitoring. By limiting our search to Medline English-language studies, we may have missed some non-US studies, but this allowed us to adequately review the study methodologies. Further, all studies regarding anticoagulation were excluded because it was not possible to identify the independent effect of interventions in improving laboratory monitoring (ie, INR testing) from dosing recommendations for warfarin. This limits the inferences we can make about HIT interventions on laboratory monitoring overall. Second, the studies were conducted in a limited number of clinical settings: three of the studies were conducted at one site and two at a second site. Further, all but one of the studies were conducted in large managed care organizations, limiting generalizability of the findings outside of these settings. Finally, differences in study design made it difficult to compare outcomes across studies. While we were unable to use meta-analysis to pool the effect sizes, we did compare the effects of the interventions across several studies on a single drug common to all studies, and did not find any consistent effect of HIT interventions on monitoring.
Even though the idea of using HIT to improve quality of care is not new,31
this goal has not yet been achieved. Many questions still remain, as posed by Kuperman et al32
in 2007: ‘To what extent does alerting impact on clinician behavior and patient outcomes? What is the optimal way to present alerts to prescribers? Which member of the healthcare team—for example, physician, nurse, pharmacist, other—is the best recipient of any kind of alert?’ These questions have yet to be answered. As more outpatient clinics adopt electronic records and electronic prescribing, it will be increasingly important to know the impact of decision support in this setting in supporting implementation of the most effective interventions. This is particularly true with regard to laboratory monitoring, which is often a locus for preventable adverse effects.
Although numerous reviews and studies have attempted to answer these questions, our systematic search identified more interventions in the inpatient setting than in the ambulatory setting. Of the studies identified, concerns about study quality and design could not exclude sources of bias in the reported results. Future studies of laboratory monitoring should better address patient and provider characteristics and account for fixed physician or clinical site effects by multilevel analysis. Studies can also better clarify outcomes (ie, improvements in test ordering versus test completion), and should also be expanded to include settings outside of the large integrated healthcare delivery systems.
While this systematic review found evidence suggesting that information technology interventions may improve laboratory monitoring for high-risk prescribed medications (exclusive of anticoagulants) in the ambulatory setting, the evidence is conflicting. The well-considered, well-designed studies reviewed appeared to find little improvement in laboratory monitoring for high-risk medications with HIT interventions targeting physicians only. However, five of eight studies found some positive effect, and this suggests that using HIT may be a promising avenue for improving laboratory monitoring. More research is needed to determine how to maximize the full potential benefit of HIT for monitoring high-risk medications and ultimately improving patient safety.