We implemented and evaluated a vaccine adverse event elicited surveillance and reporting system within a large multi-specialty group practice that used an EMR for routine ambulatory care. Optimal surveillance systems should be able to identify rare adverse events and assess causality. 17
Our system can do the first, and can facilitate the causality assessment by obtaining the clinician’s input while the event is fresh. We believe that this system increased the completeness of reporting because the rate of reporting (0.69 per 1,000 vaccine doses) was several-fold higher than our estimate of the national reporting rate in this age group during the period of our intervention (0.12 per 1,000 vaccine doses), or the overall rate of 0.11 per 1,000 net doses reported in the literature. 1
It is possible, however, that the higher reporting rate we observed was unrelated to the elicited surveillance system. Among other potential explanations is the possibility that this practice’s patients brought more adverse reactions to the attention of their providers than is typical, or that other characteristics of the electronic medical record were responsible for a higher reporting rate. We have anecdotal evidence against the latter explanation, since some clinicians who submitted reports via this system told us that they had never reported a vaccine adverse event before this.
These reports were submitted on the date of the follow-up encounter and contained complete data on patient and provider demographics, adverse outcome(s) and vaccines (vaccination date, vaccine type[s], lot numbers and manufacturers). Our intervention provided reports that were more complete and timely than reports submitted spontaneously to VAERS. This automated system prompted clinicians to consider vaccine adverse events and facilitated reporting to VAERS by auto-populating the VAERS form with data within the EMR and the alert dialog box.
Another EMR-based system recently adapted to assess vaccine adverse events is the MediClass system. 11
This system uses natural language processing and knowledge-based methods to identify potential vaccine adverse events in textual chart notes. This system is able to detect possible vaccine adverse events and may prove useful in epidemiological studies; however, this natural language processing system does not currently operate in real time and so it does not ask the clinician to note explicitly whether a vaccine adverse event is likely.
Computerized provider order entry with clinical decision support has been integrated into EMRs to improve medication safety. 13,14,15,16,18,19
Haller et al. implemented an incident reporting system into their EMR. 12
Our system extends previous work on detection of adverse events and the implementation of clinical decision support systems to the vaccine safety arena, and includes a reporting component so that data are submitted to VAERS. Further validation and monitoring of the alerts would be beneficial to ensure that clinicians continue to use this enhancement, and to monitor for “alert fatigue.” Further work to move this system to a fully electronic messaging system from the EMR to VAERS would be valuable.
There are a number of limitations to this system. First, it is necessary that the EMR includes information about vaccines that have been administered. Second, we excluded diagnoses that occurred on the same day as the vaccination. We did this to avoid generating alerts based on diagnoses entered during the immunization visit that were unrelated to the vaccination. We were mindful that we might miss some immediate hypersensitivity-type reactions by excluding same day follow-up encounters. In addition, since alerts triggered only for follow-up encounters within 14 days following vaccination, we would miss adverse reactions that occurred beyond 14 days.
Finally, the alert is based on an exclusion code list. Therefore, any diagnoses on this list that are true vaccine adverse events would not be detected via this system. For instance, if a vaccine caused otitis media, this system would not prompt the clinician to consider it as a vaccine adverse event. We attempted to minimize this limitation through careful diagnosis code selection and revisions to the code list based on feedback we received from the provider survey. At the same time, we did not want to generate too many false positive alerts. We know that clinicians might stop answering the questionnaire if they receive too many alerts, so we plan to continue monitoring the numbers of alerts and we remain receptive to clinician feedback on codes that trigger alerts frequently. The number of alerts was acceptable after we corrected the error that generated more alerts than intended, as evidenced by the group practice’s decision to continue using this alerting system as part of routine practice after the trial evaluation ended.
There are a number of implications for adoption of this system into routine use. It is based in a proprietary EMR, which limits its portability. We were not able to implement complex logic because of the potential impact it would have on system response time. Thus, we were limited in the degree to which we could fine-tune the code to minimize false positive alerts. Upgrades to the EMR required upgrading our system each time as well. Moving this system outside the EMR would overcome some of these challenges, but would lose the real-time feature of this decision support system.
We believe that elicited surveillance via real time prompts to clinicians holds substantial promise, particularly when coupled with simplified reporting. We therefore believe it is worthwhile to add these capabilities to electronic medical records in the ambulatory setting.