illustrates our proposal for a general framework to help select error and adverse event measurement methods. Health care providers, researchers, administrators, and policy makers may find it useful to see these methods as existing on a continuum that illustrates the relative utility of each method for measuring latent errors as compared with active errors and adverse events (). On the left of this continuum are methods that capture the rich contextual issues that surround errors and adverse events and thereby allow detection of the latent errors that lead to them. These include medical malpractice claims file analysis, incident report analysis, morbidity and mortality conferences, and autopsies.
The relative utility of methods for measuring latent errors, active errors, and adverse events.
Using methods like these to identify latent errors has helped improve patient safety in areas like anesthesiology and pharmacy. For example, claims file analyses led to implementation of pulse oximetry in anesthesia,28
and incident reports have led to pharmacy practices such as removal of concentrated potassium chloride from nursing units, or requiring the use of leading zeros when writing medication doses.
Although these methods can provide important and actionable information about systems, they also have weaknesses. They are incapable of providing error or adverse event rates because they are imprecise, primarily because of the various factors that influence whether an error or adverse event leads to a claim, incident report, or autopsy. Therefore, they should be used sparingly, if at all, to assess the efficacy of interventions to improve patient safety. Instead, they can identify the latent errors that need to be addressed. The efficacy of interventions to address these errors and the related active errors and adverse events can be determined with more precise methods to assess baseline rates of errors and adverse events and the efficacy of interventions.
For example, at the far right of the continuum are prospective clinical surveillance and direct observation. These methods can provide precise and accurate estimates of error and adverse event rates in a prospective fashion, and are thus suited to measure incidence, prevalence, and the impact of interventions. However, we believe that direct observation and clinical surveillance alone have relatively limited ability to measure latent errors because such errors may have occurred in a different time or place than is being observed.
The usefulness of our conceptual model is supported by its ability to explain and place in context debates about improving patient safety. For example, indirect support for our model () comes from applying it to two recent articles that discussed the merits of using evidence-based medicine (EBM) principals to evaluate patient safety practices. Leape et al. argued that exclusive reliance on EBM would result in missed opportunities for improving patient safety because many patient safety practices that are believed to be effective have not been, and cannot be, assessed with randomized controlled trials. Leape et al.10
gave examples of patient safety practices such as pharmacy-based intravenous admixture systems, unit dose dispensing of medications, removal of concentrated potassium chloride from nursing units, and various anesthesia safety practices that are believed to improve patient safety but do not meet EBM criteria for acceptance. These practices have come into existence because of errors and adverse events that were initially detected by methods on the left side of our figure (malpractice claims analysis and incident reporting). These methods allowed not only for the adverse event to be detected, but for the latent errors that led to the adverse event (e.g., the presence of concentrated potassium chloride on nursing units) to be detected.
In a companion piece, Shojania et al.11
urged more reliance on EBM principals to determine whether patient safety practices were effective. They identified certain practices that have been tested in clinical trials (for example, perioperative β blocker use and thromboembolism prophylaxis). These types of patient safety practices have been tested by using measurement methods on the right side of our figure (e.g., prospective clinical surveillance to detect postoperative thromboembolic disease or cardiac complications).
Our model illustrates how the contrasting perspectives of Leape and Shojania originate in part from reliance on different measurement methods that vary in their precision, accuracy, and ability to detect latent errors versus active errors and adverse events. Leape urges us to use the methods on the left of our figure because of their ability to detect very important latent errors. Shojania favors reliance on measurement methods on the right because of their ability to provide precise and accurate measurements. Our model accommodates both of these approaches and suggests that they exist on a continuum.
Our model also suggests that a comprehensive monitoring system for patient safety might include combinations of the measurement methods we discussed. For example, ongoing incident reporting, autopsies, morbidity and mortality conferences, and malpractice claims file analysis could be used to identify latent errors and some active errors and adverse events. These methods would not be used to calculate rates, but rather to direct subsequent projects that would use chart review, direct observation, or prospective clinical surveillance to measure explicitly defined errors and adverse events. Combining different measurement methods has been used successfully by hospital epidemiologists to detect nosocomial infections.55
One primary goal of health care is to “do no harm.” Understanding the relative strengths and weaknesses of the error and adverse event measurement methods discussed here can help investigators, clinicians, administrators, and policy makers meet this goal.