Our study demonstrates that our patient-note mismatch rate ranged from 0.5% to 0.3%, depending on the user interface. The confidence intervals were very wide because there were only 16 admission notes in the clinician-discovered set of patient-note mismatches. While we could not find analogous patient-note mismatch rates reported in the literature, we found rates related to documentation errors. Nurse documentation errors were found in 6% of documented procedures,6
1.3% of anesthetic records had documentation errors that prevented billing,7
2.0% of radiation treatments had a documentation error,8
and 62% of neonatal ICU resident documentation had some kind of error.9
Therefore, the patient-note mismatch rate appears to make a small contribution to the total error rate. In the neonatal ICU resident documentation study, there was no mention of a patient-note mismatch in the 339 notes that were reviewed9
; the 95% CI of this result (0 of 339) is 0 to 1.1% and thus corroborates our findings.
We also demonstrated that a pop-up window reduced the patient-note mismatch rate by about 40%, as evidenced by the drop in the gender mismatch rate (37%) and in the clinician-reported patient-note mismatches (43%). This confirms that user interface design is important and can affect error rates substantially. While we do not know the patient-note mismatch rate in paper records, we do know that pop-up window interventions are only possible in the electronic record.
The pop-up window may have two components important to reducing the mismatches. First, it interrupted the user action enough to improve the review of information, and second, it made the information more accessible to the user. In our system, documentation tasks are separate from data review tasks in the user interface. However, more advanced, workflow-friendly documentation systems that are integrated into the data review may be more effective at reducing patient-note mismatches. These systems would have more complete information about the patient accessible to the user during note creation, and be better integrated into the workflow so as to not require interruption to access that information. We are currently investigating how such systems could be used to improve the overall documentation process.
While the patient-note mismatch rate is low, it may be possible to reduce it further. For example, on-the-fly natural language processing can detect inconsistencies related to gender and age and warn the user. A patient photograph in the pop-up window might trigger the user's recognition that it is the wrong patient.
This study has several limitations. First, it was carried out at a single teaching hospital on a single clinical information system. The rate is somewhat sensitive to the design of the electronic health record, as demonstrated by the effect of the intervention. Nevertheless, the estimated rates (before and after intervention) offer a rough range that is likely to be near those of other electronic health records, as long as they do not use extraordinary measures (eg, enforce barcode matching for each note, or pop up the patient's photograph).
Second, the patient-note mismatch rate is based on estimates from clinician-discovered errors. The discovery process may have its own unique gender mismatch rate, rendering the assumption invalid. For some clinical specialties that are at least somewhat gender-specific (eg, gynecology or urology), the gender rate assumptions may not be correct, and this will limit the ability to detect mismatches by comparing gender. Similarly, we note that the term in equation 3
may differ by year because it is a posterior probability for mismatches and sensitive to mismatch prevalence. We had insufficient data to estimate it by year, but we note that the bias is toward the null hypothesis (no change due to intervention), so our aggregation across years will tend to yield a more conservative result, and the true effect of the intervention may be higher. This does not affect the estimated overall rate.
Third, the rate for admission notes may differ from that for other notes. We believe, however, that our estimates at least approximate the overall rate. Finally, our method is convenient for patient-note mismatches, but it is not generalizable to most other unintended consequences of electronic health records. Extensive manual review is likely to be needed to vastly expand knowledge of error rates.
In conclusion, our study demonstrates that the prevalence of finding a note in another patient's record, referred to here as the patient-note mismatch rate, is approximately 0.3%. We have demonstrated that the rate depends significantly on the design of the user interface.