An EHR-based “mandatory” notification of anatomic pathology results improved the proportion of patients who received follow-up at six months. However, an intervention effect on timely follow-up was shown only after accounting for various site, provider and test variables in a logistic regression model. After controlling for facility differences, certain types of specialists and older patients were more likely and trainees were less likely to be associated with timely follow-up. Follow-up was remarkably different in the two study sites despite the use of the same EHR. This likely reflected differences in local practices and workflow features which we are unable to capture using chart review.(17
) Our findings suggest that technology-based interventions to improve test results management in different organizations are likely to exert a highly variable “real-world” effect even when health care systems and technology are similar.
To our knowledge, this is the first study to establish rates of follow-up of anatomic pathology results in the setting of an integrated EHR. Our study also has significant implications for EHR-based interventions targeting effective communication of test results. Despite the same intervention in the same EHR, the intervention had no impact on the pre-existing differences in follow-up patterns between the two sites. Implementation and use of health information technologies in complex systems requires addressing many contextual factors beyond technology for achieving their effectiveness.(18
) Local “socio-technical” factors such as existing workflows or practices, concomitant quality improvement initiatives and other context factors (personnel and organizational features etc.) must be taken into account.(26
Although further qualitative work is essential to fully understand our findings, several contextual factors could likely explain these differences.(27
) For instance, there are few standardized clinical practices or workflows for fail-safe management of test results and the level of institutional support providers receive for test result management activities is variable. Individual provider factors, related to how they manage test results in the EHR, might be especially prominent and need to be explored further.(11
) Some providers might not have been able to access alerts. For instance, certain specialists and trainees who rotate within the VA might not remotely access the EHR. Currently, these alerts reliably only go to a single person (i.e. the ordering provider), who might be off-site. Site-specific differences in management of alerts sent to trainees may exist, but test result follow-up by trainees was still untimely after controlling for these differences. Additionally, many providers can receive over 50 different types of notifications a day (28
) and due to a large number of notifications, a “needle in haystack” phenomenon might result where abnormal pathology reports may be under-prioritized or overlooked.(29
) This might explain why general medicine providers, who typically receive more alerts, were less likely to provide timely follow-up than sub-specialists.
Our study limitations include a lack of control group for comparison to account for temporal trends. This was not feasible because this was a natural experiment throughout the VA. Improvements may occur beyond six months post-intervention, which we did not measure. While our study findings might not be considered generalizable beyond the VA, many EHRs are adopting notification systems similar to the VA and our lessons could be useful for them. Finally, we relied on EHR documentation to determine outcomes and might have missed actions not documented. However, if at all, documentation should have been higher post-intervention because a VA directive co-incidentally also implemented in March 2009 required all test results to be communicated to patients within 14 days of result and for this communication to be documented in the EHR.(15
In conclusion, our study suggests that aggregating the effect of EHR interventions across different institutions and EHRs without controlling for local “socio-technical” contextual factors might underestimate their potential benefits.