PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
AHRQ WebM&M. Author manuscript; available in PMC 2013 May 9.
Published in final edited form as:
AHRQ WebM&M. 2012 May 1; 2012(5): 120.
PMCID: PMC3648871
NIHMSID: NIHMS374579

The Emergence of the Trigger Tool as the Premier Measurement Strategy for Patient Safety

In their landmark 1999 report, To Err is Human: Building a Safer Health System, the Institute of Medicine estimated that avoidable medical errors contribute to 44,000–98,000 deaths, and more than a million injuries, annually in United States hospitals.1 In response to these disturbing data, accreditation bodies, payers, non-profit organizations, governments, and hospitals launched major initiatives and invested considerable resources to improve patient safety.23 Assessing the impact of these patient safety initiatives requires generally accepted, rigorous, standardized, and practical measures of adverse events.45

A number of approaches to measuring adverse event rates have been used, including voluntary reports (“incident” or “occurrence” reports), mining of administrative databases (most notably the Agency for Healthcare Research and Quality’s [AHRQ] Patient Safety Indicators), the two stage review process used in the Harvard Medical Practice Study, and the Institute for Healthcare Improvement’s (IHI) “trigger tool” approach.67 Each of these methods has advantages and limitations. (Table). By identifying clues that guide chart reviewers to specific events during a patient’s hospitalization more likely to contain an adverse event, the trigger tool approach provides an efficient variation on retrospective chart reviews and overcomes many of the limitations of other methods.711 A brief discussion on each of these approaches to patient safety measurement is worth pursuing, as dramatically different adverse event rates are identified depending on the techniques being used to identify and measure harm.

Table
Comparison of four most frequently used methods to identify harm.

Occurrence reports

The most well-known strategy to identify and measure patient safety in US hospitals is the use of occurrence (“incident”) reports, submitted by caregivers. Although these data are relatively easy and inexpensive to obtain, evidence suggests that occurrence reports are underutilized1214 and only identify between 2%–8% of all adverse events in the inpatient setting.7,9,10,12 This underutilization results from the fact that occurrence reports are voluntary, time intensive, and far more likely to be completed by nurses than physicians15, and frequently perceived by staff to result in punitive action.12 While identifying important clues to process flaws, occurrence reports generally identify near misses and sentinel events but rarely reflect the spectrum of adverse events.1618

Administrative data sets

Approaches to measuring patient safety using administrative data sets are appealing, as these data are often routinely available, inexpensive to obtain, and are immediately comparable across sites. However, administrative data sets, which are the source of adverse event rates identified by AHRQ’s Patient Safety Indicators,19 are highly susceptible to variation in coding practices and suffer from harms being easily hidden in the medical record. The end result is that present approaches to identify adverse events using administrative data sets have limited sensitivity and specificity, and should probably only be to help hospitals prioritize chart review and improvement initiatives.7,2021

Retrospective or concurrent chart review

The Harvard Medical Practice study used retrospective chart review to uncover adverse events.22 Another influential study identified adverse events using a combination of “voluntary and verbally solicited reports from house officers, nurses, and pharmacists; and by medication order sheet, medication administration record, and chart review of all hospitalized patients.”17 Several other significant safety studies used similar methods. The most frequently cited adult studies using a retrospective methodology2223 revealed adverse event rates of 3.7 and 2.9 per 100 admissions, respectively. This identification strategy suffers from several problems: inconsistency in defining adverse events; poor, incomplete, confusing or conflicting entries in the medical records; and resource intensiveness. This methodology was valuable in the early days of the patient safety field by highlighting the major patient safety risks present in inpatient health care settings. However, it has largely been replaced by the more efficient and more sensitive trigger tool method described below.7

Trigger-based chart review

The trigger tool methodology has emerged as the premier approach for adverse event detection.7,2425 Triggers, defined as “occurrences, prompts, or flags found on review of the medical record that ‘trigger’ further investigation to determine the presence or absence of an adverse event”,26 have been shown to more efficiently identify adverse events than any other published detection method.7,910,1213,2526 Recent studies using the IHI Global Trigger Tool27 have identified harm rates in adults in US hospitals of 49 per 100 admissions7 (33% of patients), 36 per 100 admissions (28% of patients) in Medicare patients,25 and 25 per 100 admissions (18% of patients) across North Carolina.24 Between 44% and 63% of these adverse events were interpreted as preventable. Examples of “triggers” include abnormal laboratory results such as rising creatinine, prescriptions for antidote medications such as naloxone, and other medical record–based hints that tell the chart reviewer that an adverse event might have occurred, triggering a more thorough review of the medical record.23 The IHI adult Global Trigger Tool,27 the most well studied of the published trigger tools, consistently demonstrates compelling operator characteristics, including excellent inter- and intra-rater reliability, very good to excellent sensitivity, and excellent specificity when compared with the gold standard of detailed expert chart review.7,11,18

A 2011 study by Classen and colleagues highlighted the relative test characteristics of the various adverse event detection methods.7 The authors reviewed 795 closed medical records from 3 large academic medical centers and found that the IHI Global Trigger Tool identified 354 of the 393 adverse events (90%) detected by expert chart review, while the AHRQ Patient Safety Indicators (derived from an algorithm applied to administrative data) identified 35 adverse events (9%), and occurrence reports identified only 4 adverse events (1%). Other studies have demonstrated similar findings.9,10,13,28

In summary, rates of harm in US hospitals remain unacceptably high, with little evidence of significant improvement since To Err is Human was published in 1999.4,7,2425 One major reason for these persistently high rates has been the lack of an accepted, rigorous, standardized, and practical approach to measuring and tracking adverse events over time. The IHI Global Trigger Tool, along with other more patient population–specific triggers tools, were developed to provide practical and reliable measurement approaches to track rates of harm over time7,2425,27 at the local, regional, and national level. Though not perfect, trigger tools have better operator characteristics than other measurement approaches and detect significantly more adverse events than occurrence reports, administrative database–derived harm rates, and concurrent or retrospective chart review.29 Present efforts are underway to automate the IHI adult Global Trigger Tool and to construct and automate a pediatric global trigger tool. Once these two automated global trigger tools are validated, it seems likely that the Centers for Medicaid and Medicare Services (CMS) will require hospitals to report “all cause” harm rates, and perhaps report such results publicly or tie them to reimbursement. Other public and private insurance companies are sure to follow. These will be important next steps to move US hospitals forward toward the real work at hand—reliably improving the safety of patients in our health care system.

Sharek References

1. Kohn LT, Corrigan JM, Donaldson MS, editors. Washington, DC: Committee on Quality of Health Care in America, Institute of Medicine, National Academies Press; 2000. To Err Is Human: Building a Safer Health System. 9780309068376.
2. Rockville, MD: Agency for Healthcare Research and Quality; Patient Safety and Medical Errors. Available at http://www.ahrq.gov/qual/errorsix.htm.
3. McCannon CJ, Hackbarth AD, Griffin FA. Miles to go: an introduction to the 5 Million Lives Campaign. Jt Comm J Qual Patient Saf. 2007;33:477–484. http://www.ncbi.nlm.nih.gov/pubmed/17724944. [PubMed]
4. Leape LL, Berwick DM. Five years after To Err Is Human: what have we learned? JAMA. 2005;293:2384–2390. http://www.ncbi.nlm.nih.gov/pubmed/15900009. [PubMed]
5. Vincent C, Aylin P, Franklin BD, et al. Is health care getting safer? BMJ. 2008;337:1205–1207. http://www.ncbi.nlm.nih.gov/pubmed/19008272. [PubMed]
6. Sharek PJ, Classen D. The incidence of adverse events and medical error in pediatrics. Pediatr Clin North Am. 2006;53:1067–1077. http://www.ncbi.nlm.nih.gov/pubmed/17126682. [PubMed]
7. Classen DC, Resar R, Griffin F, et al. ‘Global Trigger Tool’ shows that adverse events in hospitals may be ten times greater than previously measured. Health Affairs (Millwood) 2011;30:581–589. http://www.ncbi.nlm.nih.gov/pubmed/21471476. [PubMed]
8. Griffin FA, Classen D. Detection of adverse events in surgical patients using the Trigger Tool approach. Qual Safe Health Care. 2008;17:253–258. http://www.ncbi.nlm.nih.gov/pubmed/18678721. [PubMed]
9. Sharek PJ, Horbar JD, Mason W, et al. Adverse events in the neonatal intensive care unit: development, testing, and findings of an NICU-focused trigger tool to identify harm in North American NICUs. Pediatrics. 2006;118:1332–1340. http://www.ncbi.nlm.nih.gov/pubmed/17015521. [PubMed]
10. Takata GS, Mason W, Taketomo C, Logsdon T, Sharek PJ. Development, testing, and findings of a pediatric-focused trigger tool to identify medication-related harm in US children’s hospitals. Pediatrics. 2008;121:e927–e935. http://www.ncbi.nlm.nih.gov/pubmed/18381521. [PubMed]
11. Sharek PJ, Parry G, Goldmann D, et al. Performance characteristics of a methodology to quantify adverse events over time in hospitalized patients. Health Serv Res. 2011;46:654–678. http://www.ncbi.nlm.nih.gov/pubmed/20722749. [PMC free article] [PubMed]
12. Resar RK, Rozich JD, Classen DC. Methodology and rationale for the measurement of harm with trigger tools. Qual Saf Health Care. 2003;12(suppl 2):ii39–ii45. http://www.ncbi.nlm.nih.gov/pubmed/14645894. [PMC free article] [PubMed]
13. Rozich JD, Haraden CR, Resar RK. Adverse drug event trigger tool: a practical methodology for measuring medication related harm. Qual Saf Health Care. 2003;12:194–200. http://www.ncbi.nlm.nih.gov/pubmed/12792009. [PMC free article] [PubMed]
14. Layde PM, Cortes LM, Teret SP, et al. Patient safety efforts should focus on medical injuries. JAMA. 2002;287:1993–1997. http://www.ncbi.nlm.nih.gov/pubmed/11960544. [PubMed]
15. Wild D, Bradley EH. The gap between nurses and residents in a community hospital’s error reporting system. Jt Comm J Qual Patient Saf. 2005;31:13–20. http://www.ncbi.nlm.nih.gov/pubmed/15691206. [PubMed]
16. Suresh G, Horbar JD, Plsek P, et al. Voluntary anonymous reporting of medical errors for neonatal intensive care. Pediatrics. 2004;113:1609–1618. http://www.ncbi.nlm.nih.gov/pubmed/15173481. [PubMed]
17. Kaushal R, Bates DW, Landrigan C, et al. Medication errors and adverse drug events in pediatric inpatients. JAMA. 2001;285:2114–2120. http://www.ncbi.nlm.nih.gov/pubmed/11311101. [PubMed]
18. Classen DC, Lloyd RC, Provost L, Griffin FA, Resar R. Development and evaluation of the Institute for Healthcare Improvement Global Trigger Tool. J Patient Saf. 2008;4:169–177. Available at http://dx.doi.org/10.1097/PTS.0b013e318183a475.
19. Rockville, MD: Agency for Healthcare Research and Quality; AHRQ Quality Indicators: Introduction. Available at http://www.qualityindicators.ahrq.gov/
20. West AN, Weeks WB, Bagian JP. Rare adverse medical events in VA inpatient care: reliability limits to using patient safety indicators as performance measures. Health Serv Res. 2008;43(1 Pt 1):249–266. http://www.ncbi.nlm.nih.gov/pubmed/18211528. [PMC free article] [PubMed]
21. Scanlon MC, Harris JM, Jr, Levy F, Sedman A. Evaluation of the Agency for Healthcare Research and Quality Pediatric Quality Indicators. Pediatrics. 2008;121:e1723–e1731. http://www.ncbi.nlm.nih.gov/pubmed/18474532. [PubMed]
22. Brennan TA, Leape LL, Laird NM, et al. Incidence of adverse events and negligence in hospitalized patients. Results of the Harvard Medical Practice Study I. N Engl J Med. 1991;324:370–376. http://www.ncbi.nlm.nih.gov/pubmed/1987460. [PubMed]
23. Thomas EJ, Studdert DM, Burstin HR, et al. Incidence and types of adverse events and negligent care in Utah and Colorado. Med Care. 2000;38:261–271. http://www.ncbi.nlm.nih.gov/pubmed/10718351. [PubMed]
24. Landrigan CP, Parry GJ, Bones CB, Hackbarth AD, Goldmann DA, Sharek PJ. Temporal trends in rates of patient harm resulting from medical care. N Engl J Med. 2010;363:2124–2134. http://www.ncbi.nlm.nih.gov/pubmed/21105794. [PubMed]
25. Washington, DC: US Department of Health and Human Services, Office of the Inspector General; 2010. Nov, Adverse Events in Hospitals: National Incidence Among Medicare Beneficiaries. Levinson DR. Report No. OEI-06-09-00090. Available at http://oig.hhs.gov/oei/reports/oei-06-09-00090.pdf.
26. Classen DC, Pestotnik SL, Evans RS, Lloyd JF, Burke JP. Adverse drug events in hospitalized patients. Excessive length of stay, extra costs, and attributable mortality. JAMA. 1997;277:301–306. http://www.ncbi.nlm.nih.gov/pubmed/9002492. [PubMed]
27. Griffin FA, Resar RK. IHI Global Trigger Tool for Measuring Adverse Events: IHI Innovation Series white paper. Cambridge, MA: Institute for Healthcare Improvement; 2007.
28. Agarwal S, Classen D, Larsen G, et al. Prevalence of adverse events in pediatric intensive care units in the United States. Pediatr Crit Care Med. 2010;11:568–578. http://www.ncbi.nlm.nih.gov/pubmed/20308932. [PubMed]
29. Medical errors in the USA: human or systemic? Lancet. 2011;377:1289. http://www.ncbi.nlm.nih.gov/pubmed/21497683. [PubMed]