|Home | About | Journals | Submit | Contact Us | Français|
The computer interpretation (CI) is now a familiar part of an ECG to most providers. The addition of CI to the readings of ECGs has been offered as a way to improve physicians’ readings and to reduce medical errors.1, 2 Considerable research has compared the performance of CI with that of physicians, with mixed results. Some studies have found CI to be superior to physicians’ readings,3, 4 while others have not,5 but the measured accuracy of CI has been shown to be dependent on the particular ECG diagnoses being studied.2, 6–13
Recent recommendations note that “…errors in computer interpretation are still common, computers should not replace a qualified physician in making patient management decisions.”2 Thus most advocate for CI to serve as an adjunct to physicians’ readings, rather than a replacement. When CI is used as an adjunct it has reduced time spent looking at ECGs,14, 15 and improved cardiologists agreement with an expert benchmark.15 The addition of CI improved Family Practitioners readings to the level of Cardiology consultation, when compared with an expert electrocardiographer.16
In contrast, the addition of CI did not significantly improve a high major error rate among ECG readings of residents.17 In some cases, it has been found that CI adds to errors. With the addition of CI, physicians were more likely to agree with a diagnosis supported by the computer reading, even if it was erroneous.14 In exploring these issues, one author noted that the addition of CI improved residents overall accuracy from 49% to 55%. This gain, not surprisingly, was driven by the subset of ECGs the computer read correctly. When looking only at the subset of ECGs that the computer read incorrectly, however, the addition of CI worsened residents overall accuracy from 57% to 48%.18
Previous research measuring the value of CI has focused on how the CI affects clinician ECG reading,2–18 suggesting that the CI affects clinical decision-making mediated by an effect on ECG interpretation. We hypothesized that erroneous CI may affect clinical decision-making independent of an effect on ECG interpretation. In this study we examined the effect a computer mis-interpretation might have on resident physician ECG interpretation and decision-making.
We presented a true case of a patient with chest pain in a series of teaching conferences for Internal and Emergency Medicine residents. The case consisted of a 62-year-old woman who awoke from sleep at 4 am with severe chest pain. She was sent urgently for a coronary angiogram, before Troponin results were available, which revealed normal coronary arteries. After the angiogram, the patient was clinically diagnosed with pericarditis. We believe she was sent for urgent revascularization, in part, because of an erroneous CI of acute ischemia on her ECG (Figure). The final and accurate official reading of the ECG (by a cardiologist unaware of the study) was “Non-specific ST segment abnormalities.” The erroneous CI read:
ST ELEVATION CONSIDER INFERIOR INJURY OR ACUTE INFARCT ** ** ** ** * ACUTE MI * ** ** ** **
After a short clinical presentation, the residents were asked to select from an unmarked pile of handouts, which included the ECG (some with the erroneous CI and others with no CI), and also included different recommended courses of treatment. Residents were unaware that they were being assigned to different groups, and were told that data were being collected on how residents read ECGs. Responses were written and anonymous. Choices for ECG readings were: “Diagnostic of ischemia or infarct,” “Non-Diagnostic,” or “Normal.” Choices for course of treatment were: “Urgent Revascularization,” “Maximal Medical Treatment for Ischemia,” or “Minimal Medical Treatment for Ischemia.”
To analyze the data, the dependent variables were defined as ECG reading (Diagnostic vs. Non-Diagnostic or Normal) and recommended action (Revascularization vs. Medical Therapy). The major independent variable was the presence or absence of the erroneous CI on the ECG. Comparison of outcomes between groups was performed using Chi-Square tests.
Montefiore Medical Center’s Institutional Review Board approved the study. The funding source had no role in the study.
110 Internal and Emergency Medicine residents received the case presentation and gave written responses. Two surveys in the “erroneous CI present” group and three in the “erroneous CI absent” group were incomplete and were not included in the analysis. Analyses were performed on the remaining 105 complete surveys. The overall reading of the ECGs (Diagnostic vs. Non-Diagnostic or Normal) did not differ significantly between the two groups (p = 0.62). The 56 residents with the erroneous CI reading recommended urgent revascularization more frequently than the 49 residents without the erroneous CI reading (30% vs. 10% p = 0.01, Table 1).
After stratifying by ECG reading, among the subgroup of residents who read the ECG as Non-Diagnostic or Normal (n = 57), the residents with the erroneous CI were still more likely to recommend revascularization than the residents without the erroneous CI, but this difference was not statistically significant (9% vs. 0%, p = 0.15). However, among the subgroup of residents who read the ECG as Diagnostic (n = 48), the residents with the erroneous CI were significantly more likely to recommend revascularization than the residents without the erroneous CI reading (54% vs. 25%, p = 0.048, Table 2).
This is the first study of computer interpretation of ECGs that assessed an outcome other than physician interpretation of ECG. The major finding in this study was unexpected: although an erroneous CI reading did not significantly affect how resident physicians interpreted ECGs, it had a profound effect on management recommendations. The presence of CI reading affected the aggressiveness of the recommended action, even though this was not reflected in the interpretation of the ECG. In other words, the computer reading did not affect the residents’ ECG interpretation; it affected what the residents believed they would do about the ECG reading.
The use of computer interpretation of ECGs as an aid to physician readings is widespread. Earlier research has shown that, with the addition of CI, physicians were more likely to agree with a diagnosis supported by the computer reading, even if the reading was erroneous.14 In addition, Tsai et al noted that in cases where CI was incorrect, residents’ ECG reading accuracy worsened.18
Given our findings we suspect that the computer-induced-reading-error described by Tsai et al underestimates the clinical effects of computer mis-reads. When the computer is mistaken it will cause some mis-reads and, in addition, may cause faulty decision making even if the ECG is read correctly. This raises particular concern about overly aggressive treatment in patients with false-positive CI readings of ECGs, which were present in 12.4% of ECG readings in one study.19 Given the number of ECGs with CI done each year, the potential for misreads is large, and many patients may be inappropriately treated for ischemia where none exists.
The present study has several limitations. First, given that we used only one ECG, it is unclear whether computer influence on action occurs with all false positive computer readings, just a few, or only this one. Although we are not able to estimate how frequently computer mis-reads adversely affect clinical decision making, we were able to demonstrate that this phenomenon does occur. In addition, it is unclear if a false negative CI would lead to overly cautious treatment. Second, residents make more mistakes reading ECGs than attendings, and may be more likely to be affected by computer readings. It is unclear if the effect on decision making, over and above the effect on ECG reading, would also be observed in attendings.
In many teaching institutions, however, residents make management decisions based on their interpretation of ECGs at the bedside. These critical decisions may be influenced in a broader and more profound way by the ECG computer interpretation than has been previously been described. We suspect that the influence of erroneous CI on clinical decision-making has been underestimated.
Financial support: The Institute for Medical Effectiveness Research, a joint project of the Albert Einstein College of Medicine and the North Shore-LIJ Health System, and the Clinical Investigation Core of the Center for AIDS Research at the Albert Einstein College of Medicine and Montefiore Medical Center, funded by the National Institutes of Health (NIH P30 AI51519)
Portions of these data were presented as an oral presentation at the SGIM meeting in April, 2007 in Toronto.