Healthcare organizations use different methods to detect and measure AEs: voluntary incident reports, random chart abstraction, and concurrent clinical surveillance. Prior studies attributed differences between surveillance methods to the data sources used by each method, to differences in the subject matter expertise among human reviewers, and to cognitive challenges faced by the reviewers.15
Other differences, such as timing, scope, and workflow of surveillance, may also contribute to these differences.33
We expected that MCR would detect AEs missed by CSS and, taking advantage of these differences, we could improve CSS identification of AEs. Because agreement between MCR and CSS was less for ADEs than for HAIs, the potential for improving CSS is greater for ADEs than for HAIs. Integrating information from physician narratives with CSS would potentially capture a greater proportion of additional ADEs than HAIs.
Improving CSS with information from physician narratives
Bates et al
suggest that integrating information from physician narratives with automated surveillance methods would increase the number of AEs detected.34
Based on our findings, adding data from physician narratives would have helped CSS detect some, but not all, missed cases. Review of the phrases we collected suggested that detection of LRTIs, SSIs, and ADEs would improve if patient signs, symptoms, interventions, and physician assessments from physician narratives were integrated with CSS.
Using microbiology culture results and the urinary catheter surveillance used at LDS Hospital, CSS detected all of the BSIs and all but one of the UTIs in the study. Thus, detection of these types of events would not benefit much from integration of data from physician narratives. We did not find microbiology culture results for the single UTI, the LRTIs, or several SSIs in either the laboratory system or the physician narratives. The CSS missed some deep incisional and organ space SSIs, because the specimen was entered into the laboratory information system as unstructured freetext as opposed to the expected coded format. In the absence of microbiology data, the signs, symptoms, radiographic evidence (for LRTIs and organ space SSIs), treatment, and diagnoses contained in physician narratives could serve as triggers for HAIs.
The CSS missed ADEs for the following reasons: (a) information needed to trigger an alert was not available to the system, (b) information was available to the system but no alert was triggered, and (c) an assessment of a suspected ADE was not documented by the clinical pharmacist in CSS. We encountered two instances where an intervention (eg, administration of vitamin K or naloxone) was mentioned in the physician narrative but not recorded in the pharmacy system as either an order or an administration event. Thus, physician narratives proved to be an alternate source of information for medication-related events that were not recorded electronically in the pharmacy information system. However, addition of data from physician narratives would not improve CSS for cases for which no alert was generated or where the alert was generated but not reviewed by the clinical pharmacist. For cases where no alert was generated, time-driving the ADE logic and scanning the data from all patients may be more effective rather than depending on using data-driven triggers to activate the logic.
Content of physician narratives
Our analysis of physician narratives revealed challenges in using narrative text to support CSS. Physicians may respond to AEs as part of routine course of care and not document observations and interventions with surveillance in mind.35
In this study, only 58% of HAIs and 34% of ADEs missed by CSS were explicitly documented in dictated reports. If natural language processing could detect these phrases, then CSS would most likely be able to pick up these additional AEs. The lack of explicit physician acknowledgment for the remaining AEs presents a challenge for automated surveillance methods.17
In many AEs, supporting evidence for an LRTI, SSI, or ADE was distributed across multiple physician narratives. In the absence of explicit recognition of the AE, CSS would need to handle information from multiple places in the same document or from multiple documents to identify ADEs and HAIs.
Sources of AE information
By examining the content of physician narratives, we identified those that were likely to contain information about each type of AE. Discharge summaries contained information about more HAIs and ADEs than any other electronic physician narrative. Discharge summaries would be a valuable source of information for retrospective measurement and for confirmation of AEs detected earlier in the admission by other methods. But their benefit to prospective surveillance would be limited, since discharge summaries are not available prior to discharge.
We had to look at the particular subtypes of HAIs to find specific opportunities to improve HAI surveillance by CSS. In order to improve surveillance of SSIs that required readmission, CSS would need access to the information found in emergency department reports and admission history and physical reports. These reports contained information about signs, symptoms, significant white blood cell counts, antimicrobial treatment, bedside interventions, diagnostic imaging, and physician impressions. To improve the surveillance of SSIs that occurred within the current admission, CSS would need access to information found in general surgery reports; these reports contained phrases that suggested the presence of a ‘post-operative wound infection.’ Improving the detection of LRTIs would require access to general consult reports, which contained signs, symptoms, antimicrobial treatment, and physician impressions. In addition to signs of pneumonia (important for LRTIs), diagnostic radiology reports contained important evidence of intra-abdominal and retroperitoneal abscesses, which were important indicators of SSIs that required radiologically-guided drainage. Information about one post-procedural LRTI was found in a death summary report.
Information about outpatient ADEs was found in emergency department reports and admission history and physical reports. Information about anticoagulation-related bleeding events was found in general surgery reports, radiology reports, and discharge summaries. For example, one patient with repeated bleeding episodes on Coumadin received an inferior vena cava filter, which was documented in a radiology report because it was placed under radiographic guidance. Another anticoagulation-related gastrointestinal bleeding event was recorded in an endoscopy report. The ADEs severe enough to require transfer to the intensive care unit were mentioned in a general consult report, which included general anesthesia-related events, cardiac arrests secondary to cardiovascular medications, and opiate-related sedation. Almost all ADEs involving narcotic analgesics were mentioned in a general consult report, which contained signs (eg, mental status changes and decreased respiratory rate), response to naloxone, and physician assessments.
If the clinician did not document their assessment of a suspected case in the CSS, we were unable to distinguish between false positive cases and suspected AEs that were not reviewed. This is an important area for additional investigation, since it would affect the benefit obtained by the integration of additional data from physician narratives.
Recommendations for future work
Physician narratives must be available in electronic form, so that CSS can access their content. The ideal narrative for a concurrent system like CSS is the progress note, since it is typically created daily throughout a hospitalization. As hospitals implement electronic progress notes, we need to understand what information about AEs is more likely to be recorded in progress notes than other physician narratives.
Additional investigation of ADEs missed by CSS is needed to troubleshoot the system and the surveillance workflow. In these cases, improvements may be attained in the cognitive burden, staffing, and prioritization of patient safety activities.