This is the first study to employ empirically generated audit trails with a computerised medical record to investigate the alert, patient and physician characteristics associated with both the generation of psychotropic alerts from a commercially available drug alert system and physician response in primary care. We found that psychotropic drug alerts were common, with 1.5 alerts occurring per visit, that the majority (97%) were moderate or serious alerts (not severe alerts) and that physicians saw a minority (6.1%) of these alerts because they (82%) set their threshold for viewing alerts to see only the severe alerts. Patients at higher risk of fall-related injuries were more likely to have alerts for their psychotropic drugs, and were also more likely to have medications revised in response to an alert. Alerts were more likely to be generated for patients’ psychotropic medications if their physician was male and had established an alert threshold to view only the severe alerts. Additionally, the severity and type of the alert were associated with a physician's response to an alert.
Our findings are coherent with prior research,12–17
as well as a recent Delphi assessment of important attributes of drug alert systems to improve clinical utility.35
Alerts are considered clinically useful when graded by severity, when patient risk is considered, and when the probability of an adverse event is significant. Physicians in this study responded in a similar manner; preferentially revising severe alerts and in patients who were at higher risk of an adverse medication-related event. Both Isaac and Grizzle noted a similar pattern.17
Physicians were more likely to respond to the most severe alerts, and for patients who were being newly exposed to a drug or drug combination, where the risk would be higher than those who had already demonstrated that they could tolerate the medication. These findings as a group suggest that alerts will be most useful if graded by clinical importance, and if patient-specific estimates of risk are calculated and presented, so that only the most clinically relevant alerts could be presented.
However, one of the greatest barriers to developing more clinically useful alert systems is the paucity of empirical evidence of the risk associated with the vast majority of alerts. Most current alert systems, both commercial and home-grown, base the grading of alerts as well as the probability of harm on lowest level of evidence—expert opinion,36–38
although there are notable recent attempts39
to grade alerts by the strength of the evidence. To make substantial gains in improving drug safety, and reducing the risk of adverse drug events with computerised decision support, real-world evidence needs to be generated on the risks associated with drug alerts, and this evidence needs to be incorporated into a new generation of alert systems. For example, with the increasing use of electronic medical records, an opportunity exists to use clinical and administrative data to monitor adverse drug events and model the actual risks of adverse events associated with prescribed medications. Moreover, priority should be placed on drugs and drug combinations that have the highest population attributable risk, which is a function of the prevalence of use, magnitude of the risk and severity of adverse effects. This would likely include the most prevalent classes of medication—cardiovascular drugs, antibiotics, psychotropic medications, anti-inflammatories and analgesics, as well as those with established high risks such as anticoagulants.
This is the first study to document that physicians who are most likely to generate alerts are also more likely to establish the highest threshold for viewing alerts. This finding, while new, fits with prior observations on the high frequency of overriding computerised prescribing alerts by physicians.12
Alert fatigue created by too many drug alerts is a common complaint,17
and methods of over-riding, or suppressing alerts are highly valued, particularly when alerts are presented in an interruptive manner. However, if physicians with a high volume of drug alerts are more likely to have a higher rate of adverse drug events in their patient population, then facilitating alert over-rides defeats the purpose of implementing computerised decision support in the first place. It has been assumed that reducing the sheer number of alerts to the most serious problems will solve the problem of alert fatigue and alert over-rides. But this assumption may not be correct. A growing body of literature in educational psychology suggests that physicians who have the highest prevalence of quality and safety problems in practice may also be most likely to over-rate the quality of their performance.40
If these factors influence alert response, then the physicians who are most likely to generate severe drug alerts may be least likely to respond to them. This possibility needs to be assessed in future research, as more comprehensive feedback, such as comparative benchmarking for quality of drug management as well as patients outcomes, may need to be included with drug alerts to alter physician's self-assessment of performance so that safety and quality problems will be addressed.41
Our results need to be considered in light of several limitations. First, the number of alerts that were actually seen by physicians and revised was small, and we had insufficient power to identify factors that may have increased the likelihood of responding to an alert by 50% or less. Our sample is limited to primary care physicians and psychotropic drug alerts. The generalisability of these findings to alerts related to other classes of drugs, to other alerting systems, and to other types of physicians or health professionals is not known and would be an important avenue for future research. Lastly, we studied physicians in one jurisdiction, and were unable to assess contextual factors that may modify physician response to alerts such as jurisdictional differences in malpractice litigation risk, funding for the quality of performance rather than for services alone, or the existence of support systems for quality improvement.
In conclusion, to reduce the risk of psychotropic drug-related fall injuries, a new generation of drug alerts should be developed that incorporates empirical estimates of the risk of injury related to psychotropic drugs, as well as patient characteristics. Eventually, these empirical risk estimates could be calculated using clinical and administrative data by systems capable of modelling such data in real time, providing earlier warnings about new drugs associated with increased risks of fall injuries. In addition, the need to incorporate more comprehensive feedback on the quality and outcomes of drug prescribing for physicians that generate a high proportion of alerts should be evaluated in subsequent research.