A large number of RCTs of MMIT exist. They are not uniform in number or intensity of study (eg, length of study, complexity or extent of MMIT and its integration, number of healthcare providers involved) across medication management phases, settings, types of MMIT, personnel, or outcomes. Prescribing (n=74) and monitoring (n=38) phases were much more frequently studied in the 87 RTCs. We identified very few MMIT interventions that targeted order communication, dispensing, administering, continuing professional or patient education, or medication reconciliation. Most trials were conducted in US centers with a strong history of MMIT: Boston, Indianapolis, Washington state, Kaiser Permanente, and Veterans Affairs hospitals. Most trials were published after 1999. The trials concentrated on CDSSs and CPOE systems and few evaluated MMIT systems used by non-physicians. MMIT in long-term care settings such as nursing homes, pharmacies, homes, and community was also not well studied.
Overall, the quality of the articles was poor, with scores indicating that only half reported the use of methods generally accepted as minimizing bias. In addition, we found varying definitions and methods of measurement of outcomes and lack of consensus on reporting. An RCT may not be the only method necessary for a full assessment of the effects of HIT. Programatic analyses or assessment methods for complex interventions can also provide valuable insights into the effects of MMIT, but we did not identify these in our literature review. The full AHRQ report contains studies of various other designs that address important aspects of MMIT systems that are not usually identified using RCT methods.8
For example, qualitative studies identified substantial unintended consequences of MMIT, and case studies of implementations can provide valuable qualitative insights.
Consistent with other reviews of MMIT, most studies measured changes in process and the majority of these showed benefit. Few trials studied clinical outcomes. Despite using a broad definition of clinical outcomes that included physiological measurements (eg, blood pressure and blood glucose levels), very few studies showed improvement in primary outcomes (11 of 26), and even fewer studies that used clinical endpoints as secondary outcomes found benefit (two of 43 showed benefit and one showed harm).
The literature of MMIT is dominated by implementation and demonstration projects (see full report). These were not designed to evaluate and establish clinical benefit for patients in settings that used MMIT systems compared with those who received usual care without MMIT. Few studies measured clinical improvements for patients or the quality of health care provided. Potential harm to patients resulting from the use of MMIT was rarely evaluated. Future research should address these gaps. In particular, it is also important that the effect of MMIT on the consumer be considered.
The consistent application of a set of agreed-upon standards for the assessment, evaluation, and description of MMIT (analogous to CONSORT guidelines for RCT reporting and PRISMA guidelines for reviews) could improve the quality and generalizability of future research. Consistency and greater depth of reporting is also needed in published trials, especially with respect to MMIT itself and settings.
The findings of this review are consistent with those of the full AHRQ report, which included studies using multiple research designs.8
The AHRQ report also includes summaries of qualitative studies, sustainability of MMIT systems, the value proposition of MMIT, and feature sets associated with the likelihood of purchase, implementation and use.
This review has several important limitations. First, the studies reported only limited data on systems, installations, institutions, and targets of the intervention making complex synthesis difficult. We also found problems with methods and analyses, a wide variation in the number of studies in certain areas, and a broad range of MMIT systems. We attempted to address this limitation by basing our work on well-defined analytical frameworks and by identifying not only the systems used but also their functional capabilities. Third, this review summarizes only the RCTs from the 378 studies included in the full report. Although we used only the studies with strong research methods for this overview of MMIT, we feel that they provided a very similar assessment of the effects and effectiveness provided across all articles. Our requirement that MMIT systems were integrated with other HIT may also have led to the exclusion of important evidence. In addition, missing in this review are the qualitative studies that portray a rich understanding of effects of MMIT on clinicians and patients.