Signal and noise: applying a laboratory trigger tool to identify adverse drug events among primary care patients

doi:10.1136/bmjqs-2011-000643

BMJ Qual Saf. 2012 August; 21(8): 670–675.

Published online 2012 May 23. doi: 10.1136/bmjqs-2011-000643

PMCID: PMC3402751

Signal and noise: applying a laboratory trigger tool to identify adverse drug events among primary care patients

Stacey Brenner,¹ Alissa Detz,² Andrea López,^1,³ Claire Horton,¹ and Urmimala Sarkar^1,³

¹Division of General Internal Medicine, San Francisco General Hospital, University of California, San Francisco, California, USA

²Department of Medicine, California Pacific Medical Center, San Francisco, California, USA

³Department of Medicine, Center for Vulnerable Populations, San Francisco General Hospital, University of California, San Francisco, California, USA

Corresponding author.

Correspondence to Dr Urmimala Sarkar, Medicine, University of California, San Francisco, 1001 Potrero Av, Ward 13, San Francisco, CA 94110, USA; Email: usarkar/at/medsfgh.ucsf.edu

Accepted April 7, 2012.

This is an open-access article distributed under the terms of the Creative Commons Attribution Non-commercial License, which permits use, distribution, and reproduction in any medium, provided the original work is properly cited, the use is non commercial and is otherwise in compliance with the license. See: http://creativecommons.org/licenses/by-nc/2.0/ and http://creativecommons.org/licenses/by-nc/2.0/legalcode.

Abstract

Background

The extent of outpatient adverse drug events (ADEs) remains unclear. Trigger tools are used as a screening method to identify care episodes that may be ADEs, but their value in a population with high chronic-illness burden remains unclear.

Methods

The authors used six abnormal laboratory triggers for detecting ADEs among adults in outpatient care. Eligible patients were included if they were >18 years, sought primary or urgent care between November 2008 and November 2009 and were prescribed at least one medication. The authors then used the clinical / administrative database to identity patients with these triggers. Two physicians conducted in-depth chart review of any medical records with identified triggers.

Results

The authors reviewed 1342 triggers representing 622 unique episodes among 516 patients. The trigger tool identified 91 (15%) ADEs. Of the 91 ADEs included in the analysis, 49 (54%) occurred during medication monitoring, 41 (45%) during patient self-administration, and one could not be determined. 96% of abnormal international normalised ratio triggers were ADEs, followed by 12% of abnormal blood urea nitrogen triggers, 9% of abnormal alanine aminotransferase triggers, 8% of abnormal serum creatinine triggers and 3% of aspartate aminotransferase triggers.

Conclusions

The findings imply that other tools such as text triggers or more complex automated screening rules, which combine data hierarchically are needed to effectively screen for ADEs in chronically ill adults seen in primary care.

Keywords: Patient safety, diagnostic errors

Introduction

An adverse drug event (ADE) is defined by the Institute of Medicine as ‘an injury resulting from medical intervention related to a drug’.¹ It is estimated that 2.4 of every 1000 emergency department visits per year are attributed to ADEs, with approximately 0.32% of hospitalised patients incurring fatal ADEs.^2–6 Recent research estimates between 14.9 and 50.1 ADEs occur per 1000 person months.⁷ ⁸ Approximately 20% of ADEs identified are thought to be preventable and a significant proportion are serious, life-threatening or fatal.⁹ Unfortunately, even though most medical care occurs in the outpatient setting, the incidence of ADEs among ambulatory patients is unknown.

A recent study estimated that approximately 4.5 million ambulatory visits per year in USA are related to ADEs, making this a significant public health concern.¹⁰ However, the full burden of outpatient ADEs remains unclear, because of our inability to detect and monitor these events, especially those that occur when patients are at home between visits. ADEs are normally detected by voluntary reporting and tracking of errors. It is estimated that only 10–20% of all errors are reported.¹¹ In-depth chart review captures significantly more ADEs than voluntary reporting, but this is time consuming and expensive.⁷ ⁹

An ADE trigger tool may make chart review more efficient.¹² Triggers allow for targeted chart review by identifying possible ADEs via abnormal laboratory values, text phrases, or automated ‘rules’ combining multiple data elements available in medical records. Charts identified to have a specific trigger can then be reviewed for ADEs. Trigger tools have been studied in their ability to detect ADEs and research has shown that computerised and non-computerised trigger tools are more time-effective than complete chart review and more sensitive than voluntary reporting.¹³ ¹⁴

Most research on trigger tools used to identify ADEs has focused on hospitalised patients.^15–17 In one of the first studies of a trigger tool in the outpatient setting, Singh et al created a 36-item trigger tool and investigated its efficacy in detecting ADEs among older patients seen in primary care settings. Analysis revealed that six laboratory values and three text triggers had high positive predictive values for identifying ADEs. These nine items accounted for 94.4% of ADEs detected.¹⁸ Because this trigger tool seems promising, we applied it in a single, publicly funded primary care clinic with an ethnically diverse, largely chronically ill population. We sought to determine whether using the six laboratory values as triggers would (1) identify ADEs (2) identify stage of medication use and (3) characterise ADEs detected with respect to preventability and severity. The goal of the study was to determine whether this trigger approach could efficiently identify ADEs, in order to facilitate the clinicians' ability to intervene in incipient ADEs prior to patient harm. We wanted to evaluate whether the triggers would identify ADEs in order to determine whether real-time screening of our laboratory data would be an efficient way of intervening in outpatient ADEs.

Methods

Study design and patients

We conducted a retrospective electronic medical review to validate the use of a six-item trigger tool to detect the occurrence of ADEs. A computer-based search was completed to identify patient charts that had at least one of the six laboratory value triggers (international normalised ratio (INR) >5, serum creatinine (SCr) >2.5, blood urea nitrogen (BUN) >60, alanine aminotransferase (ALT) >84, aspartate aminotransferase (AST) >80, thyroid-stimulating hormone (TSH) undetectable while on levothyroxine).¹⁸ The laboratory triggers were chosen because it was feasible to extract the visit data associated with particular laboratory values. We omitted the three text triggers in the study by Singh et al,¹⁸ because it was not feasible to perform automated text searches of electronic visit notes using the current electronic health record. We searched the electronic medical record, which is used within the city-and-county-wide, publicly funded healthcare system, including all outpatient clinics, urgent care centre, dialysis unit, acute rehabilitation, emergency department and hospital, for the above laboratory values that occurred at any location during the study period from November 2008 to November 2009.

The study evaluated patients of a general internal medicine clinic that operates within an integrated healthcare system of the San Francisco Department of Public Health. Eligible patients were included if they were greater than 18 years old, were prescribed at least one medication, and had at least one laboratory value trigger. As stated above, laboratory values taken at various sites within the healthcare system were identified; however, only patients established in the general internal medicine clinic were included in the study. To determine whether a patient received care from the general internal medicine clinic, patients were included if they were evaluated in this clinic at any time within the study period or within 90 days of the study period. Patients were excluded if their only visit was in the vaccination clinic or in the women's clinic for a routine pap smear. Patients were also excluded if their only visit was with a nurse.

The study protocol was evaluated and approved by the Committee on Human Research at the University of California, San Francisco.

Event detection and classification

Once the above six laboratory values were identified, as a ‘trigger,’ we found that often there were repeat laboratory values on subsequent days as a patient was being monitored by a healthcare provider. We determined that the same abnormal laboratory value identified for the same patient within a 30-day period was classified as one unique ‘episode.’

To determine whether the abnormal laboratory value episode resulted from an ADE, physicians reviewed electronic charts, as in prior studies of ADEs.¹⁹ A physician (SB) independently conducted an in-depth chart review for all charts found to have any of the six laboratory value triggers. A second physician (AD) conducted an in-depth chart review for a randomly selected 15% of the triggers. Differences were resolved by discussion. The two physicians reviewed all patient related documentation from 90 days before, 12 months during and 90 days after the study period. The physician-reviewers first determined the care setting in which the trigger occurred. If any laboratory trigger value occurred during the admission (as long as it was not related to an outpatient medication), it was excluded. For example, if the creatinine was normal on admission but increased during the admission, then that was excluded. In addition, if a laboratory value was taken in the emergency department and was related to their admission, then it was excluded. For example, if someone had an elevated serum creatinine on admission but this resulted from acute renal failure due to sepsis, his/her case was excluded. The physicians first determined whether or not an ADE occurred. For each ADE identified, the physician then determined the stage of the medication process where the event occurred, the severity of the effect on the patient and the preventability of the ADE.

For all ADEs identified, the stage of medication use was classified as prescribing, dispensing, monitoring and patient self-administration. The stage was defined as the point of medication use where the ADE occurred, regardless of whether it was preventable.²⁰ We classified the stage as ‘patient self-administration,’ only when there was clear evidence from the medical chart that the patient took the medication incorrectly. The severity of the effect was divided into no harm, minimal harm (no change in symptoms but have abnormal laboratory results), mild harm (mild reversible complications not requiring hospitalisation or symptoms lasting less than one day), moderate harm (reversible complications not requiring hospitalisation or prolonged symptoms lasting more than one day) and severe harm (irreversible complications or requiring hospitalisation, permanent disability or death).²¹ Finally, the preventability of the ADE was grouped based on established definitions of preventability from prior patient safety studies.²² The preventability of the ADE is classified as follows: (1) ameliorable—severity could have been substantially reduced with different actions or procedures (on the part of the patient, provider or system), (2) preventable—could have been avoided as probably a result of an error or system design flaw or could have been avoided with a patient action that is, reasonable to expect, or (3) nonpreventable.²¹

Finally, we reviewed the medical centre's adverse reporting system, known as the unusual occurrence reporting system, during the study period and found that none of the ADEs identified were voluntarily reported.

Results

Our study included 583 patients. The mean patient age was 55 years with a SD of 14; 64% were men and 70% were English-speaking (table 1). We identified 2662 laboratory values that met one of the following criteria: INR>5, SCr >2.5, BUN >60, ALT >84, AST >80 or TSH undetectable while on levothyroxine. We excluded 1322 laboratory values that were drawn in the hospital setting and related to that hospital admission. There were 1342 triggers that represented 622 unique episodes among 516 patients (figure 1). Agreement between physician-reviewers on ADE occurrence was 94%.

Table 1

Patient characteristics

Figure 1

Flow chart of adverse drug events (ADEs).

From the 622 unique episodes, the trigger tool identified 91 (15%) as ADEs. We evaluated each laboratory trigger separately to determine each trigger's yield in identifying ADEs. Table 2 describes the 622 laboratory triggers and 91 ADEs by individual laboratory trigger. For each laboratory trigger, the percentage of triggers that identified an ADE is also listed. This table highlights that each trigger varied in its yield to predict an ADE. For example, approximately 96% of triggers for INR>5 were associated with an ADE compared with 3% of triggers for AST >80 detecting an ADE. Of note, the laboratory trigger of undetectable TSH while on levothyroxine did not occur in our sample.

Table 2

Adverse drug events (ADEs)

We also evaluated the medication stage and degree of harm when the ADE occurred. Of the 91 ADEs included in our analysis, 49 (54%) occurred during medication monitoring, 41 (45%) during patient self-administration, and one could not be determined. Of the 91 ADEs, 65 (71%) caused minimal harm, 17 (19%) caused mild harm, 5 (5%) caused moderate harm, 3 (3%) caused severe harm and one could not be determined. Examples of severe harm include gastrointestinal bleed requiring transfusion and haemothorax necessitating chest tube placement, with both ADEs leading to hospitalisation.

Finally, of the 91 ADEs, only 3 (3%) were deemed non-preventable. Otherwise, 4 (4%) were preventable and 84 (92%) were ameliorable. An example of a non-preventable ADE that occurred was hydralazine-induced lupus; this was an adverse reaction that could not have been predicted. An example of a preventable ADE was excessive dosing of furosemide leading to an increase in serum creatinine, which could have been avoided.

Discussion

We were able to successfully apply a laboratory-value trigger tool in an ambulatory care clinic and to achieve clinician agreement in ADE ascertainment using the chart review. We found a significant number of ADEs using a trigger tool. While the utility varied among the six laboratory value triggers, INR>5 efficiently identified ADEs.

Our determinations of preventability are consistent with prior literature showing that many ADEs are preventable.⁷ ⁸ ²³ In this sample, only 3% of the triggers were felt to be non-preventable. In terms of clinical severity, the majority of ADEs identified by this trigger tool were in the mild-to-moderate-harm range. Consequently, there is an opportunity to intervene and prevent ADEs in the outpatient setting before significant harm occurs. We believe this underscores the importance of including ADE surveillance in outpatient safety initiatives.

Importantly, we found that most ADEs occurred during the self-management and monitoring stages of medication use, rather than being prescribing or dispensing errors. Therefore, interventions to prevent outpatient ADEs will need to be patient-centred and focus on safe medication self-management. Gaps in medication monitoring clearly led to ADEs, but there is scant evidence underlying current monitoring recommendations, and more study of ambulatory medication monitoring is needed.²⁴ Because the number of medications is strongly associated with ADEs,¹⁰ ²⁵ with population ageing and increasing medication use we expect outpatient ADEs to become more frequent.

There were limitations to our study. We relied upon medical chart review to determine whether an ADE occurred. Similarly, because we relied on medical records, we could not pinpoint the cause of the ADE beyond the medication use stage. As an example, it is often unclear from chart documentation whether the error occurred because of a lack of monitoring or in the course of adhering to recommended medication monitoring. However, the use of a comprehensive electronic medical record mitigated some of the flaws and biases often associated with chart review studies. The patient population is from a single, ethnically diverse, safety net clinic setting and that is both a strength and weakness. It is a strength in that findings from this clinic are likely to have public health importance, but the population may be more ill and less comparable with the usual primary care population in USA.

While the INR >5 successfully identified ADEs, the other laboratory triggers had much lower yield than previously described.¹⁸ Several factors could contribute to this difference. First, our population has a significant prevalence of chronic kidney disease, as reflected in the above-threshold values for BUN and creatinine not attributable to ADEs. Similarly, there is a high prevalence of hepatitis C infection in this population that likely reduces the yield for ALT and AST trigger tools. Second, it is possible that the electronic medical record captures different information that the paper charts from the prior study. Third, because our patients are largely uninsured, virtually all of their laboratory tests and subspecialty care occurred within the integrated public healthcare delivery system. This may account for differences compared with the prior study.

Clearly, the performance of a trigger tool depends on its intended use. For our purposes of identifying patients with incipient ADEs in order to intervene clinically, we concluded that, with the exception of INR>5, using abnormal laboratory values is a ‘noisy’ method to identify the signal of ADEs. INR>5 did detect ADEs efficiently, and surveillance at this threshold may reduce the harm from ADEs associated with anticoagulation. For the other laboratory-based triggers, the labour associated with reviewing many charts in order to identify a single ADE was prohibitive for our clinical practice. For purposes of conducting research or for acquiring prevalence data for operational purposes, this method certainly would be more efficient than comprehensive or randomly selected chart review. Moreover, during the study period, there were 19 066 potentially eligible visits, so using the triggers did significantly reduce the pool of eligible records.

We were encouraged that the method of physician chart review resulted in reliable identification of ADEs, and our findings of problems in self-management and monitoring suggest the need for patient-directed interventions to reduce ADEs. Our results also suggest that in order to use trigger tools effectively, outpatient care settings may need to change the value at which a laboratory value triggers (the ‘trigger threshold’), considering other laboratory-based triggers, and, where feasible, consider testing text triggers. Finally, our findings imply that other tools, such as text triggers, or more complex automated screening rules which combine data hierarchically may be needed to efficiently screen for ADEs in adults seen in primary care.

Footnotes

Contributors: SB performed chart review of all triggers, and wrote the first draft of the manuscript. AD performed secondary chart review and provided intellectual contribution in editing the manuscript. AL conducted analysis and created tables and figures. CH oversaw data collection and provided intellectual contribution in editing the manuscript. US conceived the study, oversaw data collection and provided intellectual contribution in editing the manuscript.

Funding: Funds were provided by the National Centre for Research Resources KL2RR024130 (to US) and Agency for Healthcare Research and Quality K08 HS017594 (to US). None of the funders had any role in study design; in the collection, analysis, and interpretation of data; in the writing of the manuscript; or in the decision to submit the manuscript for publication.

Competing interests: None.

Ethics approval: Ethics approval was granted by the University of California San Francisco Committee on Human Research.

Provenance and peer review: Not commissioned; externally peer reviewed.

References

1. Bates DW, Leape LL, Petrycki S. Incidence and preventability of adverse drug events in hospitalized adults. J Gen Intern Med 1993;8:289–94. [PubMed]

2. Zed PJ, Abu-Laban RB, Balen RM, et al. Incidence, severity and preventability of medication-related visits to the emergency department: a prospective study. CMAJ 2008;178:1563–9. [PMC free article] [PubMed]

3. Budnitz DS, Pollock DA, Weidenbach KN, et al. National surveillance of emergency department visits for outpatient adverse drug events. JAMA 2006;296:1858–66. [PubMed]

4. Johnson JA, Bootman JL. Drug-related morbidity and mortality. A cost-of-illness model. Arch Intern Med 1995;155:1949–56. [PubMed]

5. Pirmohamed M, James S, Meakin S, et al. Adverse drug reactions as cause of admission to hospital: prospective analysis of 18 820 patients. BMJ 2004;329:15–19. [PMC free article] [PubMed]

6. Lazarou J, Pomeranz BH, Corey PN. Incidence of adverse drug reactions in hospitalized patients: a meta-analysis of prospective studies. JAMA 1998;279:1200–5. [PubMed]

7. Gurwitz JH, Field TS, Harrold LR, et al. Incidence and preventability of adverse drug events among older persons in the ambulatory setting. JAMA 2003;289:1107–16. [PubMed]

8. Thomsen LA, Winterstein AG, Sondergaard B, et al. Systematic review of the incidence and characteristics of preventable adverse drug events in ambulatory care. Ann Pharmacother 2007;41:1411–26. [PubMed]

9. Gandhi TK, Weingart SN, Borus J, et al. Adverse drug events in ambulatory care. N Engl J Med 2003;348:1556–64. [PubMed]

10. Sarkar U, Lopez A, Maselli JH, et al. Adverse drug events in U.S. Adult ambulatory medical care. Health Serv Res 2011;46:1517–33. [PMC free article] [PubMed]

11. Griffin FA, Resar RK. IHI Global Trigger Tool for Measuring Adverse Events. IHI Innovation Series white paper. 2nd Edn. Cambridge, MA: Institute for Healthcare Improvement, 2009. http://www.IHI.org.

12. IHI Global Trigger Tool for Measuring Adverse Events Institute for Healthcare Improvement, 2012. http://www.IHI.org (accessed 30 Apr).

13. Jha AK, Kuperman GJ, Teich JM, et al. Identifying adverse drug events: development of a computer-based monitor and comparison with chart review and stimulated voluntary report. J Am Med Inform Assoc 1998;5:305–14. [PMC free article] [PubMed]

14. Classen DC, Pestotnik SL, Evans RS, et al. Computerized surveillance of adverse drug events in hospital patients. 1991. Qual Saf Health Care 2005;14:221–5; discussion 225–6. [PMC free article] [PubMed]

15. Classen DC, Resar R, Griffin F, et al. ‘Global trigger tool’ shows that adverse events in hospitals may be ten times greater than previously measured. Health Aff (Millwood) 2011;30:581–9. [PubMed]

16. Takata GS, Mason W, Taketomo C, et al. Development, testing, and findings of a pediatric-focused trigger tool to identify medication-related harm in US children's hospitals. Pediatrics 2008;121:e927–35. [PubMed]

17. Resar RK, Rozich JD, Simmonds T, et al. A trigger tool to identify adverse events in the intensive care unit. Jt Comm J Qual Patient Saf 2006;32:585–90. [PubMed]

18. Singh R, McLean-Plunckett EA, Kee R, et al. Experience with a trigger tool for identifying adverse drug events among older adults in ambulatory primary care. Qual Saf Health Care 2009;18:199–204. [PubMed]

19. Gandhi TK, Burstin HR, Cook EF, et al. Drug complications in outpatients. J Gen Intern Med 2000;15:149–54. [PMC free article] [PubMed]

20. National Coordinating Council for Medication Error Reporting and Prevention What is a Medication Error? http://www.nccmerp.org/aboutMedErrors.html (accessed 30 Apr).

21. Sarkar U, Handley M, Gupta R, et al. Use of an interactive, telephone-based self-management support program to identify adverse events among ambulatory diabetes patients. J Gen Intern Med 2008;23:459–65. [PMC free article] [PubMed]

22. Sarkar U, Handley MA, Gupta R, et al. What happens between visits? Adverse and potential adverse events among a low-income, urban, ambulatory population with diabetes. Qual Saf Health Care 2010;19:223–8. [PubMed]

23. Hug BL, Witkowski DJ, Sox CM, et al. Adverse drug event rates in six community hospitals and the potential impact of computerized physician order entry for prevention. J Gen Intern Med 2010;25:31–8. [PMC free article] [PubMed]

24. Steinman MA, Patil S, Kamat P, et al. A taxonomy of reasons for not prescribing guideline-recommended medications for patients with heart failure. Am J Geriatr Pharmacother 2010;8:583–94. [PubMed]

25. Budnitz DS, Shehab N, Kegler SR, et al. Medication use leading to emergency department visits for adverse drug events in older adults. Ann Intern Med 2007;147:755–65. [PubMed]

Articles from BMJ Open Access are provided here courtesy of
BMJ Group