This randomized-controlled trial compared two types of computerized alerts designed to reduce the use of unapproved medication abbreviations by physicians. We demonstrated that alerts embedded within an electronic progress note program reduced the use of abbreviations within the electronic program (as measured by frequency of alerts fired) and within the non-computer-assisted handwritten H & Ps authored by physicians over the same time period. Alerts with a forced correction feature decreased the use of abbreviations to a much greater extent than alerts with an auto-correction feature. Moreover, an unanticipated but particularly interesting finding in our study was that reductions in abbreviation use were observed in a control group who were unexposed to alerts, but who were exposed to the overall study environment.
Eliminating error-prone medication abbreviations has been extremely challenging for hospitals, and there are very few effective interventions in the literature for this vexing problem. An educational intervention designed to reduce prescribing errors in the handwritten medication orders of residents reduced overall prescribing errors among surgery but not medicine residents.11
Enforcement strategies at the level of medical staff leadership proved more effective than education alone in a single study10
; however, enforcing physician accountability for documentation skills is difficult. Given the strong and repeated association between abbreviation use and medication errors,3
it will be necessary and important for healthcare leaders to use multiple strategies to improve this unsafe and therefore unacceptable practice. Health information technology is just one of those strategies. As demonstrated in this study, a clinical decision-support system designed to reduce abbreviations may be an effective addition to administrative oversight and routine education.
Of the 2371 H & Ps reviewed, there were 4191 unapproved abbreviations noted, which equates to approximately two unapproved abbreviations per H & P. On the surface, this average seems low considering the high numbers of patients in our hospital treated with multiple medications. However, when considering the frequency of these occurrences in H & Ps (range 0–17; median=2.5), one can see that significant abbreviation use with the opportunity for medication errors exists. Significant reductions in abbreviations were demonstrated in the non-computer-assisted handwritten notes over time and across all three study groups, further reducing the abbreviation errors in the H & Ps.
The ability for health information technology to intercept unsafe practices and prevent serious medication errors has been described.20–22
Improvements in medication safety with the use of CDSS occur through both direct and indirect effects. Direct effects alter medication prescribing or management at the time practitioners interact with system. Indirect or ‘spillover’ effects result from the carry-over into practice of knowledge or behaviors learned during exposure to the system.23
Few studies of CDSSs have been designed to measure indirect effects. Glassman et al
reported that exposure to automated drug alerts had little effect on the recognition of selected drug–drug or drug–condition interactions as measured by a cross-sectional survey.23
Studies of drug-utilization reviews describe indirect effects of interventions on future clinician behavior.24
For example, a time-series study that involved mailing letters to physicians about drug interactions and monitoring their subsequent prescribing patterns found no effect on future prescribing behavior as a result of the intervention.24
In contrast, our study found large indirect effects by demonstrating significant reductions in the frequency of medication abbreviations in physicians' non-computer-assisted handwritten notes when they were prompted to correct the abbreviations in the electronic notes over the same time period.
The rapid expansion of alert systems in medical informatics calls for more research comparing the effects of different alert systems on the same outcome. Previous studies have described the over-riding of drug safety alerts26
and demonstrated that the nature of alerts influences clinician behavior.22
Thus, the inability to detect significant reductions in abbreviation use in the non-computer-assisted handwritten notes in the auto-correction alert group compared with the forced correction alert group is not surprising. One reason for this is that interns who received auto-correction alerts disregarded the educational message or simply acknowledged the alert without reading the information given human factors such as time pressure, competing priorities, and alert fatigue. In contrast, interns exposed to the forced alert were unable to complete their electronic notes without making manual corrections. It is known that mere repetition facilitates long-term memory,29
and it may be that by forcing physicians to correct abbreviations, their knowledge of these abbreviations was solidified and translated into improvements in written practice. In summary, our study found direct evidence that passive alerts do little to influence clinician behavior. Additional studies will be important to substantiate these findings and advance the field of health informatics.
Reductions in abbreviation use in the control group were not anticipated by the investigators, but there are several possible explanations for the observation. Experimental diffusion, which occurs when a treatment effect applied to one group unintentionally spills over and contaminates another group,31
may explain our findings. Interns in the control group were working in a study environment designed to modify physician behavior. Even though they were not directly exposed to alerts, their behavior may have been influenced by the improving documentation patterns of the interns exposed to the intervention who worked with them. Diffusion of effects threatens the internal validity of research, but it is difficult to control for in quality improvement research. While it is possible that the hospitals' educational strategies to reduce unapproved abbreviations contributed to the documentation improvements, this seems unlikely given the historical failure of routine education related to abbreviation avoidance.10–12
Despite the improvements in documentation practices, we failed to find any significant improvements in physician knowledge of unapproved abbreviations. This apparent ‘disconnect’ in knowledge versus practice is intriguing and has been demonstrated previously by Glassman et al
There are several possible explanations for this finding. Our sample size may have been too small to detect a meaningful difference. The participants had varying degrees of exposure to the abbreviation alerts and thus may have been unable to remember the abbreviations when they were presented to them in the post-test. The alerts may not have been perceived as important or relevant to the interns, especially since they were alerted when writing notes rather than when ordering medications. Nonetheless, it is hard to ignore the substantial reductions in abbreviation use in the non-computer-assisted handwritten notes as a result of the intervention, and this could be interpreted as a surrogate for knowledge acquisition.
The Joint Commission has strictly prohibited the use of seven common and unsafe medication abbreviations,4
and The Institute for Safe Medication Practices has promulgated a list of over 50 abbreviations that have been associated with harm in their error-reporting systems and should never be used.5
However, given the fact that any medication abbreviation creates an opportunity for misinterpretation and error potential, some organizations have attempted to limit all medication abbreviations by creating policies with ‘approved’ (rather then unapproved) abbreviation lists in which all medication abbreviations are prohibited. Since clinicians are in the habit of using medication abbreviations frequently, it is unlikely that one intervention alone will eliminate this practice, and it will be necessary to consider electronic interventions such as this to curb their use in free text entries in prescription writing and medical records.
Our study has several limitations. Because our hospital has an integrated CPOE system, we were unable to assess whether our intervention would have affected handwritten medication prescribing errors related to abbreviations. Handwritten abbreviations in prescriptions present a larger risk to patients than handwritten abbreviations in medical records. However, it is possible that documentation skills learned by physicians in an electronic environment and practiced in handwritten notes will carry over into their future handwritten or electronic free text prescriptions, and recommendations for prescriptions are often made in medical record documentation, so the potential for abbreviation errors exists even outside of the prescription-writing environment. Feasibility issues prevented retrospective reviews of the handwritten medication prescriptions of study participants.
We did not study the documentation practices of the participants after the alerts were turned off. Consequently, we cannot be certain of the long-term sustainability of our intervention and whether the documentation improvements would have improved further, plateaued, or waned had the alerts been turned off or continued. Additionally, given that exposure to the alerts was not continuous over time based upon the sequence of intern rotations and that these alerts varied in frequency among all interns, we may have reduced our ability to detect important differences among the groups and within certain participants. We did not evaluate documentation practices in the year(s) prior to our intervention and thus cannot completely exclude the possibility that a trend towards reduction in unapproved abbreviations occurred from a natural history effect encountered with introducing electronic platforms for documentation.
Finally, our study has several features that may limit its generalizablity. We studied only interns at a single academic medical center with a hybrid information system comprising both paper and electronic documentation. Since many organizations currently practice in hybrid systems, and many physicians practice in multiple information systems over the course of their career, we believe that the information related to the secular trends in physician non-computer assisted handwritten notes as a result of exposure to computerized alerts is relevant. Compared with interns, residents, attending physicians, non-physician providers, or practitioners in community hospitals may have responded differently to the intervention; however, there are elements of practitioner performance that are not unique to interns or academic medical centers, and some generalizations can be made from this study. The undergraduate and graduate medical training years are an ideal time to introduce information technology designed to improve medication safety, since trainees have not yet been influenced by unsafe medication documentation practices in the hospital and may be more open to changes in practice.
In summary, our study contributes important information to the health information technology literature by describing the effect that CDSS can have on physician behavior in the absence of knowledge and demonstrating that an informatics intervention can create large behavioral changes in a control group unexposed to the actual intervention but exposed to the study environment in which the intervention was performed. The methods used in this study to examine the indirect effects of health information systems to modify physician behavior outside of the electronic environment are unique and may have relevance for other health information technology interventions. We have established a methodology within a randomized-controlled trial to evaluate the effects of alerts embedded within a clinical decision-support system on physician knowledge and practice. Estimates for the percentage of unapproved abbreviation use were calculated based on the number of opportunities for error and offer additional endpoints to measure practice changes with technology-based interventions. These estimates can be used to determine samples sizes for adequate statistical power to evaluate the effects of interventions to reduce medication errors and test information systems in patient safety research. We found that alerts for unapproved medication abbreviations within electronic medical record systems are effective in changing physician documentation and thus promoting medication safety. Given that many healthcare organizations do not have fully integrated health information technology systems, researchers and patient safety leaders will continue to be challenged with ways to promote safe medication practices through electronic tools and education.