PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
Med Decis Making. Author manuscript; available in PMC 2010 May 1.
Published in final edited form as:
PMCID: PMC2806088
NIHMSID: NIHMS112978

The Effect of Erroneous Computer Interpretation of ECGs on Resident Decision Making

Introduction

The computer interpretation (CI) is now a familiar part of an ECG to most providers. The addition of CI to the readings of ECGs has been offered as a way to improve physicians’ readings and to reduce medical errors.1, 2 Considerable research has compared the performance of CI with that of physicians, with mixed results. Some studies have found CI to be superior to physicians’ readings,3, 4 while others have not,5 but the measured accuracy of CI has been shown to be dependent on the particular ECG diagnoses being studied.2, 613

Recent recommendations note that “…errors in computer interpretation are still common, computers should not replace a qualified physician in making patient management decisions.”2 Thus most advocate for CI to serve as an adjunct to physicians’ readings, rather than a replacement. When CI is used as an adjunct it has reduced time spent looking at ECGs,14, 15 and improved cardiologists agreement with an expert benchmark.15 The addition of CI improved Family Practitioners readings to the level of Cardiology consultation, when compared with an expert electrocardiographer.16

In contrast, the addition of CI did not significantly improve a high major error rate among ECG readings of residents.17 In some cases, it has been found that CI adds to errors. With the addition of CI, physicians were more likely to agree with a diagnosis supported by the computer reading, even if it was erroneous.14 In exploring these issues, one author noted that the addition of CI improved residents overall accuracy from 49% to 55%. This gain, not surprisingly, was driven by the subset of ECGs the computer read correctly. When looking only at the subset of ECGs that the computer read incorrectly, however, the addition of CI worsened residents overall accuracy from 57% to 48%.18

Previous research measuring the value of CI has focused on how the CI affects clinician ECG reading,218 suggesting that the CI affects clinical decision-making mediated by an effect on ECG interpretation. We hypothesized that erroneous CI may affect clinical decision-making independent of an effect on ECG interpretation. In this study we examined the effect a computer mis-interpretation might have on resident physician ECG interpretation and decision-making.

Methods

We presented a true case of a patient with chest pain in a series of teaching conferences for Internal and Emergency Medicine residents. The case consisted of a 62-year-old woman who awoke from sleep at 4 am with severe chest pain. She was sent urgently for a coronary angiogram, before Troponin results were available, which revealed normal coronary arteries. After the angiogram, the patient was clinically diagnosed with pericarditis. We believe she was sent for urgent revascularization, in part, because of an erroneous CI of acute ischemia on her ECG (Figure). The final and accurate official reading of the ECG (by a cardiologist unaware of the study) was “Non-specific ST segment abnormalities.” The erroneous CI read:

ST ELEVATION CONSIDER INFERIOR INJURY OR ACUTE INFARCT ** ** ** ** * ACUTE MI * ** ** ** **

After a short clinical presentation, the residents were asked to select from an unmarked pile of handouts, which included the ECG (some with the erroneous CI and others with no CI), and also included different recommended courses of treatment. Residents were unaware that they were being assigned to different groups, and were told that data were being collected on how residents read ECGs. Responses were written and anonymous. Choices for ECG readings were: “Diagnostic of ischemia or infarct,” “Non-Diagnostic,” or “Normal.” Choices for course of treatment were: “Urgent Revascularization,” “Maximal Medical Treatment for Ischemia,” or “Minimal Medical Treatment for Ischemia.”

To analyze the data, the dependent variables were defined as ECG reading (Diagnostic vs. Non-Diagnostic or Normal) and recommended action (Revascularization vs. Medical Therapy). The major independent variable was the presence or absence of the erroneous CI on the ECG. Comparison of outcomes between groups was performed using Chi-Square tests.

Montefiore Medical Center’s Institutional Review Board approved the study. The funding source had no role in the study.

Results

110 Internal and Emergency Medicine residents received the case presentation and gave written responses. Two surveys in the “erroneous CI present” group and three in the “erroneous CI absent” group were incomplete and were not included in the analysis. Analyses were performed on the remaining 105 complete surveys. The overall reading of the ECGs (Diagnostic vs. Non-Diagnostic or Normal) did not differ significantly between the two groups (p = 0.62). The 56 residents with the erroneous CI reading recommended urgent revascularization more frequently than the 49 residents without the erroneous CI reading (30% vs. 10% p = 0.01, Table 1).

Table 1
The Effect of the Erroneous Computer Interpretation on Resident Interpretation and Actions

After stratifying by ECG reading, among the subgroup of residents who read the ECG as Non-Diagnostic or Normal (n = 57), the residents with the erroneous CI were still more likely to recommend revascularization than the residents without the erroneous CI, but this difference was not statistically significant (9% vs. 0%, p = 0.15). However, among the subgroup of residents who read the ECG as Diagnostic (n = 48), the residents with the erroneous CI were significantly more likely to recommend revascularization than the residents without the erroneous CI reading (54% vs. 25%, p = 0.048, Table 2).

Table 2
The effect of the erroneous computer reading on revascularization rates stratified by ECG reading

Discussion

This is the first study of computer interpretation of ECGs that assessed an outcome other than physician interpretation of ECG. The major finding in this study was unexpected: although an erroneous CI reading did not significantly affect how resident physicians interpreted ECGs, it had a profound effect on management recommendations. The presence of CI reading affected the aggressiveness of the recommended action, even though this was not reflected in the interpretation of the ECG. In other words, the computer reading did not affect the residents’ ECG interpretation; it affected what the residents believed they would do about the ECG reading.

The use of computer interpretation of ECGs as an aid to physician readings is widespread. Earlier research has shown that, with the addition of CI, physicians were more likely to agree with a diagnosis supported by the computer reading, even if the reading was erroneous.14 In addition, Tsai et al noted that in cases where CI was incorrect, residents’ ECG reading accuracy worsened.18

Given our findings we suspect that the computer-induced-reading-error described by Tsai et al underestimates the clinical effects of computer mis-reads. When the computer is mistaken it will cause some mis-reads and, in addition, may cause faulty decision making even if the ECG is read correctly. This raises particular concern about overly aggressive treatment in patients with false-positive CI readings of ECGs, which were present in 12.4% of ECG readings in one study.19 Given the number of ECGs with CI done each year, the potential for misreads is large, and many patients may be inappropriately treated for ischemia where none exists.

The present study has several limitations. First, given that we used only one ECG, it is unclear whether computer influence on action occurs with all false positive computer readings, just a few, or only this one. Although we are not able to estimate how frequently computer mis-reads adversely affect clinical decision making, we were able to demonstrate that this phenomenon does occur. In addition, it is unclear if a false negative CI would lead to overly cautious treatment. Second, residents make more mistakes reading ECGs than attendings, and may be more likely to be affected by computer readings. It is unclear if the effect on decision making, over and above the effect on ECG reading, would also be observed in attendings.

In many teaching institutions, however, residents make management decisions based on their interpretation of ECGs at the bedside. These critical decisions may be influenced in a broader and more profound way by the ECG computer interpretation than has been previously been described. We suspect that the influence of erroneous CI on clinical decision-making has been underestimated.

Acknowledgments

Financial support: The Institute for Medical Effectiveness Research, a joint project of the Albert Einstein College of Medicine and the North Shore-LIJ Health System, and the Clinical Investigation Core of the Center for AIDS Research at the Albert Einstein College of Medicine and Montefiore Medical Center, funded by the National Institutes of Health (NIH P30 AI51519)

Footnotes

Portions of these data were presented as an oral presentation at the SGIM meeting in April, 2007 in Toronto.

Reference List

1. Salerno SM, Alguire PC, Waxman HS. Competency in interpretation of 12-lead electrocardiograms: a summary and appraisal of published evidence. Ann Intern Med. 2003 May 06;138(9):751–760. [PubMed]
2. Salerno SM, Alguire PC, Waxman HS. Training and competency evaluation for interpretation of 12-lead electrocardiograms: recommendations from the American College of Physicians. Ann Intern Med. 2003 May 06;138(9):747–750. [PubMed]
3. Snyder CS, Fenrich AL, Friedman RA, et al. The emergency department versus the computer: which is the better electrocardiographer? Pediatr Cardiol. 2003 July;24(4):364–368. [PubMed]
4. Jakobsson A, Ohlin P, Pahlm O. Does a computer-based ECG-recorder interpret electrocardiograms more efficiently than physicians? Clin Physiol. 1985 October;5(5):417–423. [PubMed]
5. Sekiguchi K, Kanda T, Osada M, et al. Comparative accuracy of automated computer analysis versus physicans in training in the interpretation of electrocardiograms. J Med. 1999;30(1–2):75–81. [PubMed]
6. Widman LE, Tong DA. Validation of the EINTHOVEN model-based computerized electrocardiogram rhythm analysis system with three classes of clinical arrhythmias. Am J Cardiol. 1996 October 15;78(8):927–931. [PubMed]
7. Shirataka M, Miyahara H, Ikeda N, et al. Evaluation of five computer programs in the diagnosis of second-degree AV block. J Electrocardiol. 1992 July;25(3):185–195. [PubMed]
8. Willems JL, breu-Lima C, Arnaud P, et al. The diagnostic performance of computer programs for the interpretation of electrocardiograms. N Engl J Med. 1991 December %19;325(25):1767–1773. [PubMed]
9. Willems JL, breu-Lima C, Arnaud P, et al. Evaluation of ECG interpretation results obtained by computer and cardiologists. Methods Inf Med. 1990 September;29(4):308–316. [PubMed]
10. Massel D, Dawdy JA, Melendez LJ. Strict reliance on a computer algorithm or measurable ST segment criteria may lead to errors in thrombolytic therapy eligibility. Am Heart J. 2000 August;140(2):221–226. [PubMed]
11. de Bruyne MC, Kors JA, Hoes AW, et al. Diagnostic interpretation of electrocardiograms in population-based research: computer program research physicians, or cardiologists? J Clin Epidemiol. 1997 August;50(8):947–952. [PubMed]
12. Heden B, Ohlsson M, Rittner R, et al. Agreement between artificial neural networks and experienced electrocardiographer on electrocardiographic diagnosis of healed myocardial infarction. J Am Coll Cardiol. 1996 October;28(4):1012–1016. [PubMed]
13. Heden B, Ohlsson M, Holst H, et al. Detection of frequently overlooked electrocardiographic lead reversals using artificial neural networks. Am J Cardiol. 1996 September 01;78(5):600–604. [PubMed]
14. Hillson SD, Connelly DP, Liu Y. The effects of computer-assisted electrocardiographic interpretation on physicians' diagnostic decisions. Med Decis Making. 1995 April;15(2):107–112. [PubMed]
15. Brailer DJ, Kroch E, Pauly MV. The impact of computer-assisted test interpretation on physician decision making: the case of electrocardiograms. Med Decis Making. 1997 January;17(1):80–86. [PubMed]
16. Woolley D, Henck M, Luck J. Comparison of electrocardiogram interpretations by family physicians, a computer, and a cardiology service. J Fam Pract. 1992 April;34(4):428–432. [PubMed]
17. Goodacre S, Webster A, Morris F. Do computer generated ECG reports improve interpretation by accident and emergency senior house officers? Postgrad Med J. 2001 July;77(909):455–457. [PMC free article] [PubMed]
18. Tsai TL, Fridsma DB, Gatti G. Computer decision support as a source of interpretation error: the case of electrocardiograms. J Am Med Inform Assoc. 2003 September;10(5):478–483. [PMC free article] [PubMed]
19. Thomson A, Mitchell S, Harris PJ. Computerized electrocardiographic interpretation: an analysis of clinical utility in 5110 electrocardiograms. Med J Aust. 1989 October 16;151(8):428–430. [PubMed]