|Home | About | Journals | Submit | Contact Us | Français|
Alerts and prompts represent promising types of decision support in electronic prescribing to tackle inadequacies in prescribing. A systematic review was conducted to evaluate the efficacy of computerized drug alerts and prompts searching EMBASE, CINHAL, MEDLINE, and PsychINFO up to May 2007. Studies assessing the impact of electronic alerts and prompts on clinicians' prescribing behavior were selected and categorized by decision support type. Most alerts and prompts (23 out of 27) demonstrated benefit in improving prescribing behavior and/or reducing error rates. The impact appeared to vary based on the type of decision support. Some of these alerts (n = 5) reported a positive impact on clinical and health service management outcomes. For many categories of reminders, the number of studies was very small and few data were available from the outpatient setting. None of the studies evaluated features that might make alerts and prompts more effective. Details of an updated search run in Jan 2009 are included in the supplement section of this review.
Prescribing medication represents an opportunity for providing appropriate and effective treatment, and yet medication errors are responsible for considerable levels of morbidity and mortality. Many errors which result in harm can potentially be prevented by the use of appropriately designed computerized clinical decision support (CDS), in particular computerized alerts and prompts.
Medication errors are common and cause substantial harm; the injuries they cause are called preventable adverse drug events (ADEs). Studies have estimated that 2.4–3.6% of hospital admissions are caused by ADEs of which up to 69% would have been preventable. 1 The incidence rate of ADEs in hospitalized patients is thought to be between 0.7 and 6.5% of which the preventable proportion reported has ranged from 17 to 56%. 2 Since the main error in most preventable ADEs occurs at the stage of drug ordering 3 increasingly sophisticated clinical computer systems have been seen as a major opportunity to prevent inappropriate prescribing.
Clinical provider order entry (CPOE) systems are computer-assisted information systems that enable providers to write all orders, including prescribing medications. Computerized decision support systems (CDSS) are linked with CPOE to provide support for decision-making in patient care by integrating clinical and patient information. They range from simple to more complex decision support systems, 4 with all the more complex using patient-specific data and some of the most complex involving detailed clinical protocols and patient specific data. The category of decision support used and its timing within the clinical workflow are important questions to be addressed when CDSS are implemented. 5 Reminding and alerting represent an important category of providing decision support to the clinician at the point of care. In the area of prescribing, drug reminder or drug alerting systems generate information that warn healthcare staff against potential prescribing errors or support them in their prescribing decisions.
Several systematic and narrative reviews of aspects of CDSS in healthcare have shown general benefits, such as improved practitioner performance and reduced error rates. 2,6–14 A Cochrane review on computerized dosing advice even showed moderate but significant reduction in hospital stay. 15 However, none of the reviews have specifically evaluated the impact of different alerting and reminding systems in computerized medication prescribing.
Therefore, we performed a systematic review to gather evidence of the types of alerting and reminding mechanisms that are in use. Our overall objective was to evaluate the impact of these systems on clinicians' prescribing behavior. In the literature, there are no clear distinctions made between “reminders”, “alerts”, and “prompts”. These terms will therefore be used interchangeably throughout the review.
We searched CINHAL (1982 to May 2007), EMBASE (1980 to May 2007), Inspec (1969 to May 2007), MEDLINE (1950 to May 2007) and PsychINFO (1967 to May 2007) for empiric studies of computerized prescribing alerts and prompts. Our search strategy combined several keywords around electronic prescribing, reminder systems and decision support systems. Articles from all languages were included. The bibliographies of obtained articles were used to identify further relevant studies following guidelines about study selection from the Cochrane Handbook of Systemic Reviews. 16 Previous literature reviews, in particular, were searched in full and any relevant references were added to the electronic selection process if considered appropriate. We rerun the search in Jan 2009 and found two recent studies that would qualify for inclusion in this review. Details concerning these studies, a summary of results and the updated search process can be found in the online appendix of this review (supplements 1–4).
Reference Manager (RefMan Version 10) was used to manage data and eliminate duplicate references. The identified references were distributed among four reviewers (AS, VP, CM, AA) who screened titles and abstracts for relevant studies. Relevant publications were obtained as full text articles and all were assessed by the four reviewers independently applying the following predetermined inclusion criteria:
Studies were included if they were randomized controlled trials (RCTs), time series analysis and before-after studies.
Studies had to target clinicians such as doctors, nurses and other health care professionals, who were actively involved in the prescribing and administration of medications for patients of any age receiving drug therapy for the treatment of a disease or for prevention of a medical condition.
Interventions of interest were computerized alerting or reminding systems consisting in electronic alerts, reminders or prompts related to electronically prescribed medications of a particular drug or class of drug, appearing on the computer screen at the point of prescribing.
Outcome measures were any changes in prescribing related to clinically relevant outcomes such as reduction in medication errors, reduction of risk or harm to patients, decrease of non-recommended medication, increase of recommended medication and increase/decrease of laboratory abnormalities with potential of adverse drug events.
An article was included if it met all the inclusion criteria listed above. Uncertainties or disagreements at any stage were resolved by discussion among reviewers.
Study descriptions and outcomes were extracted using predefined collection tables that had been developed for the review. Data were extracted by four reviewers (AA, VP, AS, CM). Extracted data were verified by two reviewers independently (AS, CM).
We applied a classification system based on a modified version previously proposed by Kuperman, et al. 4 This allowed us to group different types of prompted CDS according to their characteristics and to compare effectiveness between the groups. There were three main categories of alerts and prompts: Basic drug alerts consisting of drug allergy warnings, drug–drug interaction warnings, duplication warnings and basic medication order guidance; advanced drug alerts including drug-laboratory alerts, drug-condition alerts, drug-formulary alerts and dosing guidelines; complex alert systems representing a set of CDSSs containing features of both basic and advanced alerts (see for more details).
Alerts and prompts applied in the studies were grouped in tables and results were compared independently. On the basis of study designs, characteristics of population involved, medication prescribed, types of alerts used and outcome measures reported it was decided whether pooling of outcome data would be appropriate.
Studies were classed according to the appropriateness and quality of their design. Criteria such as study duration, similar before/after outcome assessment and before/after group comparison were applied to evaluate the quality of before-after studies. Randomized controlled trials were assessed considering their main sources of bias including selection bias, performance bias, attrition bias and detection bias. Please see tables in supplement 4 for more detail.
The search identified 14,137 publications of which 14,036 did not fulfill our inclusion criteria when assessing title and abstract (not an experimental trial, not concerned with alerts, not related to e-prescribing of a particular drug/class of drug and not concerned with change in clinician behavior). The 101 remaining full text articles were retrieved and 81 studies were excluded. The 20 remaining studies evaluating 27 different types of alerts addressed our objectives and were included in the review (see flow chart in ).
The geographic setting of most studies was the United States (18 studies). One study came from France and another from The Netherlands. 17,18 Five of the studies took place in a primary care or outpatient setting. The remaining 15 took place in a hospital/inpatient setting; hence most of the providers were hospital doctors. In some studies hospital nurses and nurse practitioners contributed to the pool of providers. 18–22
The medications prescribed in the studies varied widely. Some studies focused only on one class of drugs, such as sedatives, lipid lowering drugs, 18,19 asthma treatment, 18 or antibiotic prescribing. 18,20,23 Five studies evaluated more complex alert systems involving a wide range of prescribing situations. 9,24–28
Four out of 20 studies evaluated clinical outcomes such as drug-induced renal impairment, 29 serious adverse drug events, 25 prescribing related falls 30 and length of hospital stay. 31 Cost savings through modified prescribing were reported by two studies 21,32 (see supplements 2 and 4).
Studies showed considerable variation in characteristics of providers, patients, and health care settings, types of alert interventions and time of observation. More importantly, there was a lack of consistency in the reporting of outcome measures. Studies looked at optimal, suboptimal, dangerous and harmful prescribing using outcome measures such as serious or nonserious medication errors, actual or potential adverse drug reactions, prescribing patterns as per guidelines, increased prescribing of recommended drugs, and decreased prescribing of nonrecommended drugs. As a consequence, pooling of the data or direct comparison between studies was inappropriate. Instead, we classified studies in groups to allow independent comparison.
An RCT represents the most robust method to evaluate the effects of an alert system. In these RCTs, some of the prescribing units are generally randomized to the newly implemented alerting system while the remainders are on the prealert system. Clustered randomized trials in particular have the ability to overcome potential for contamination but require large sample sizes. However, at times when a new and complex computer alert system is introduced in a health care setting, conducting an RCT is often not feasible. This may explain why only four out of 20 studies used RCT designs. 18,19,22,23 These varied in quality with no trials meeting all the criteria to avoid bias (see supplement 3). One study did not provide the appropriate data to evaluate the entire effect of computer prompts. 19
Other methods to compare pre- and post alert prescribing behavior are more open to bias. Quasi experimental designs used in the studies of this review were uncontrolled before-and-after-studies, time series analyses and interrupted time series analyses. Interrupted time series analysis (ITS) in particular is a practical and valuable tool to determine the impact of a complex intervention, both immediately and over time. Multiple measurements are taken before and after an intervention and are tested for their serial correlation. If sequential prescribing data confirm an immediate and long-lasting effect following the introduction of a new alert system, the prompt system is likely to be responsible for the change.
To account for temporal changes in prescribing patterns, such as seasonal prescribing changes, it is desirable to have long sequential baseline and follow up periods. Other important validity factors are similarity of control and intervention group and continuity of pre- and post alert outcome assessment. Four studies used interrupted time series analyses, 21,26,33,34 six studies used time series analyses 17,25,28–31 and six used before-after designs. 20,24,26,27,32,35 Four authors refined the time series analysis approach by alternating control periods and intervention periods to control for other influences. 17,29–31 Important validity criteria were fulfilled by most studies (see supplement 3).
Some of the 20 studies included in this review evaluated more than one type of alert and were therefore classified in more than one category. 24,28,31 In total, 27 types of alerts and prompts were assessed in terms of their ability to improve prescribing. Twenty-three of the 27 alerts resulted in statistically significant improved prescribing behavior and/or reductions in medication errors. Most of the changes noted were clinically relevant. The remaining four showed statistically nonsignificant improvements in prescribing 18,24,33 or inappropriate study design 19 (see and supplement 2).
Of the four alert types that examined clinical outcomes, three had a positive and statistically significant impact leading to decreased prescribing-related renal impairment, 29 fewer falls in elderly people 30 and a reduced length of hospital stay. 31 One alert achieved a reduction in serious and life-threatening adverse drug events, but there was no p value reported. 25 Financial implications of two alerts were evaluated and both achieved annual cost savings of up to US$36,500 32 and US$6,400. 21
Applying the Kuperman framework 4 we classified 27 types of alerts and prompts used across the 20 studies according to their categories and subcategories of alerts as outlined above.
Two papers investigated the effects of four types of basic alerts of which three reported statistically significant beneficial effects on prescribing (table, Supplement 2):
All the 20 papers evaluated more advanced alert types and statistically significant effects were shown in 21 out of 23 types across five categories:
For many alert categories, only a small number of alert types had been performed, and for some categories we found no studies (). Types with few or no published data included drug-allergy warnings (n = 1), drug–drug interaction alerts (n = 1), duplicate medication alerts (n = 0), drug-disease contraindication alerts (n = 0) and dosing guidelines based on age (n = 0), pregnancy/child bearing age (n = 0), pediatric patients (n = 1) and indications (n = 0).
Our study employed rigorous inclusion criteria for the identification of studies relevant to the objectives of the review. In particular, we used a wide variety of potentially relevant literature databases and we carefully selected relevant search terms and adapted these for each database. In addition, we carefully searched the bibliographies of studies and reviews that we obtained to identify other potentially relevant articles. Given the limitations of time and resources we split up the searches of publication titles and abstracts between three reviewers. We did however check three validation samples of 1,000 references and this showed no discrepancies in the selection of main outcome studies. Throughout the process of conducting the review any issues arising, including discrepancies, were resolved through discussion. The data concerning the main outcomes in Tables 1 and 2 were checked by two reviewers independently (AS and CM).
This study had several limitations. The area of computerized decision support systems has become a wide field. In this review, we were specifically interested in studies evaluating point-of-care decision support systems that confront the clinician during the prescribing process. Studies were only included if CDSS were concerned with e-prescribing of a particular drug or class of drugs, not considering systems concerned with improved screening or better responses to abnormal laboratory results. Some might view this intentional narrow focus as an important limitation of the current study.
It is possible that some citations were missed, although the search strategy was comprehensive. The diversity in study characteristics and outcome measures did not allow pooling of the data and even made comparison between categories difficult. The wide variation of alerts and prompts requires a framework to categorize different alert types when reviewing the literature. In this review we applied a classification system based on a version by Kuperman, et al, which is a complex framework of alerts and prompts. 4 Due to the limited number of studies, it was not possible to reveal superiority of any category of alerts. Other categorizations of alerts and prompts according to their functionality, purpose, timing and workflow interruptions have been developed and represent an alternative way of categorization. 5
We identified 20 studies and 27 types of alerts and prompts of which 23 achieved improved prescribing in terms of changes in prescribing behavior and/or reductions in medication errors. In many of the studies, the changes noted were clinically relevant. Positive effects were noted for a wide range of alerts and prompts. Three of the alert types with lacking benefit showed weaknesses in their methodology 18 or design. 19,33 These findings provide support for the continued development and use of computerized alerts and prompts for prescribing. The alert types and categories presented in this review represent relevant information for developers of computerized prescribing systems when setting up of new systems or updating older systems.
The limited number of studies conducted did not cover all alert categories, and it will be useful to target these alerts and prompts in future research. Drug–drug interaction alerts, drug-disease contraindication alerts and dosing guidelines based on age are prescribing categories with great potential for affecting prescribing. However, the lack of study data are surprising. These types of alerts, in particular, need to be further evaluated in future experimental trials using appropriate study designs. Randomized controlled trials are the most valid and robust method to evaluate the effect of an alert systems and should be used whenever possible.
The main obstacle in comparing effects on prescribing was the lack of standardized outcome measures for measuring medication errors. The most common outcomes reported across the studies of this review were medication error (serious and nonserious) and ADEs (actual and potential; preventable and non preventable). A variety of efforts are underway now to build consensus around these and other safety measures. 37 Other outcomes were the impact of alerts on prescribing patterns as per guidelines, prescribing fewer nonrecommended drugs and prescribing more recommended drugs. To improve validity of future studies, an effort has to be made to use more uniform definitions of outcome measures around suboptimal prescribing.
Also, few studies have assessed the impact of computerized alerts and prompts on clinical or health service management outcomes. Such studies need to have a large enough sample size to have sufficient power to assess the effects of these interventions. Evidence from these types of studies is likely to have the greatest impact on the decision making processes of clinical information system implementers.
Perhaps most notably, the current literature provides little evidence from experimental studies about the factors that are most essential to consider when designing computerized alerts and prompts for prescribing. To our surprise, we found no empiric studies assessing the effectiveness of different approaches to the design and display of alerts and prompts. Results from observational studies suggest that paying careful attention to the content of alerts and prompts is likely to improve adherence by clinicians. 38 Regarding the ways in which computerized alerts are displayed, there was support from clinicians for these to be clear, precise, brief and noticeable and for important alerts to be made “intrusive”. 39–41 In addition, the need to improve the signal-to-noise ratio of alerts and prompts appeared to be a clear priority. 42,43 Clinicians expressed reasonably high levels of support for alerts aimed at improving the safety of prescribing and showed strong preference for receiving recommendations rather than just assessments. 44 This suggests that, provided alerts and prompts are carefully designed and selected, clinicians are likely to be accepting of them and that adherence is likely to be improved if they are displayed clearly and the signal-to-noise ratio is reasonably high. It is clear from areas other than medicine that both design and wording of alerts can have a substantial impact on their effectiveness. 45 This review has shown that there are serious gaps in the medical literature regarding the most effective methods for displaying alerts and prompts and that there is a need to investigate how to present and display information at the point of prescribing. These gaps are important, because many electronic records do not include substantial decision support. Models about benefits of electronic health records (EHRs) suggest that substantial benefits will be achieved only with advanced decision support. 46 It also appears that apparently minor issues relating to human factors can have a very large impact on the success of specific alerts and prompts, 47 yet best practices in this area have not been clearly delineated.
We conclude that most empiric studies evaluating the effects of computerized prompts and alerts on prescribing behavior show positive, and often substantial, effects. However, while a wide range of types of alerts and prompts have been evaluated, additional studies should be done to determine the design features that are most strongly associated with improved impact on prescribing and clinical outcomes.
This systematic review was part of a report conducted for NHS Connecting for Health in Jan 2008.