|Home | About | Journals | Submit | Contact Us | Français|
National benchmarks for trauma triage sensitivity (≥95%) and specificity (≥50%) have not been rigorously evaluated across broad populations of injured patients. We evaluated the impact of different field triage schemes for identifying seriously injured patients across a range of sensitivity values. Impact metrics included specificity and number of under- and over-triaged patients compared to current triage practices.
This was a retrospective cohort study of injured children and adults transported by 48 EMS agencies to 105 hospitals in 6 regions of the Western U.S. from 2006 through 2008. Hospital outcomes were probabilistically linked to EMS records through trauma registries, state discharge databases and state emergency department databases. The primary outcome was an Injury Severity Score (ISS) ≥16. We evaluated 40 field predictor variables, including 31 current field triage criteria, using classification and regression tree analysis and cross validation to generate estimates for sensitivity and specificity.
89,261 injured patients were evaluated and transported by EMS providers over the 3-year period, of whom 5,711 (6.4%) had ISS ≥16. As the 95% sensitivity target for triage was approached (from the current value of 87.5%), decision tree complexity increased, specificity decreased (from 62.8% to 18.7%), and the number of triage-positive patients without serious injury doubled (67,927 vs. 31,104). Analyses restricted to children and older adults were similar. The most consistent modification to the current triage algorithm to increase sensitivity without a major decrease in specificity was altering the Glasgow Coma Scale (GCS) score cut-point from ≤13 to ≤14 (sensitivity increase to 90.4%).
Reaching the field triage sensitivity benchmark of 95% would require a large decrease in specificity (increase in over-triage). A 90% sensitivity target appears more realistic and may be obtainable by modest changes to the current triage algorithm.
The original Field Triage Decision Scheme for trauma patients was developed by the American College of Surgeons Committee on Trauma (ACS-COT) in 1976, based largely on expert opinion.1,2 The guidelines were a necessary part of early trauma system development and need to concentrate seriously injured patients in specialized regional hospitals (major trauma centers) capable of caring for them. In 1987, the decision scheme was revised and organized into the current algorithmic triage process with multiple “steps,” prioritized by the likelihood of serious injury.3,4 The decision scheme has subsequently been revised five times between 1990 and 2011 based on updated science and expert opinion.5–9 Despite a large body of literature on field triage and multiple revisions of the guidelines, few studies have explored the effects of different triage strategies on healthcare delivery systems (e.g., ambulance transport patterns and trauma center volumes) or data-driven benchmarks for triage accuracy.
The ACS-COT sets the national benchmarks for field triage at ≥95% sensitivity (proportion of seriously injured patients cared for in major trauma centers) and ≥50% specificity (proportion of non-seriously injured patients cared for in non- trauma centers).8 However, the feasibility of reaching these benchmarks and the resulting implications for trauma systems remain unclear. Recent literature suggests that the benchmark for under-triage is not being met,10–15 particularly among older adults, and it remains unknown whether the guidelines could be revised to meet these accuracy targets. It is also uncertain how changes in triage accuracy may impact the volume of injured patients transported to different types of hospitals. These are important health policy questions for trauma systems.
In this study, we sought to empirically derive multiple field triage decision schemes for identifying seriously injured patients (Injury Severity Score [ISS] ≥16) across a range of sensitivity values and to assess their potential impact within trauma systems (under-triage, specificity and over-triage), as compared to current triage practices. Rather than seeking to develop the “best” triage guidelines, this study was designed to provide trauma system policymakers with information about the trade-offs inherent in different strategies for field triage.
This was a multi-region retrospective cohort study. Fourteen Institutional Review Boards at 6 sites approved this protocol and waived the requirement for informed consent.
The study included injured children and adults evaluated and transported by 48 EMS agencies to 105 hospitals (including 12 Level I, 5 Level II, 3 Level III, 4 Level IV, 1 Level V and 80 community/private/federal hospitals) in 6 sites across the Western U.S over a 3-year period (January 1, 2006 through December 31, 2008). The 6 sites included: Portland, OR/Vancouver, WA (4 counties); King County, WA; Sacramento, CA (2 counties); San Francisco, CA; Santa Clara, CA (2 counties); and Denver County, CO. Each site represents a pre-defined geographic “footprint,” consisting of a central metropolitan area and surrounding region (suburban and some rural areas), defined by EMS agency service areas. All sites have established trauma systems.
The study sample included all injured patients (EMS primary impression of “injury” or “trauma”) activating the 9–1-1 EMS system and transported to an acute care hospital (trauma centers and non-trauma centers) with a matched hospital record. Specifying the sample in this manner allowed for an out-of-hospital injury cohort defined by EMS providers and consisting of patients with both minor and serious injuries. This sampling strategy also approximates the denominator of patients to whom field trauma triage guidelines are routinely applied. Excluded patients included: interhospital transfers without an initial presentation involving EMS; EMS runs listed as “cancelled,” “no patient found” or “stand by” (calls without patient contact); and patients that were not transported (e.g., deaths in the field, refusals of transport).
We recorded 40 field-based predictors of serious injury for analysis, including 31 trauma triage criteria in local use in the 6 regions. Most of the triage criteria came from different versions of the Field Triage Decision Scheme,3,5–9 although we also included site-specific triage criteria developed for local use (e.g., steering wheel deformity). We triangulated several different data sources to identify all field “trauma activation” patients (i.e., patients meeting any of the field triage criteria, per EMS personnel) and to comprehensively capture all individual triage criteria used by EMS. We determined field triage information from: (1) trauma triage criteria specified in the EMS chart; (2) EMS provider-documented “trauma activation” (or similar charting, depending on local terminology); (3) EMS-recorded trauma identification number (used at some sites as a mechanism for tracking injured patients entered into the trauma system); (4) a matched record from the local trauma registry specifying “scene” origin (i.e., EMS-identified trauma activations); and (5) matched trauma communication center records (phone records from base hospital communication with EMS providers, required for all field trauma activations in certain sites). We evaluated the presence of triage criteria independent of transport destination.
We also recorded patient demographics (age, gender), initial field physiologic measures (Glasgow Coma Scale [GCS] score, systolic blood pressure [SBP], respiratory rate, and heart rate), field-assisted ventilation (bag-valve mask ventilation, intubation or supraglottic airway use), mechanism of injury (15 categories) and hospital destination. We categorized acute care hospitals as tertiary trauma centers (Level I or II trauma hospitals) or non-trauma centers based on their American College of Surgeons accreditation status and state-level designations.
The primary outcome for this study was “serious injury,” defined as an ISS ≥16. While many definitions of serious injury have been used in previous triage studies, an ISS ≥16 is the metric used by ACS-COT to define under- and over-triage8 and identifies the subset of patients most likely to benefit from care in major trauma centers.16–18
To capture ISS for patients transported to all types of hospitals, we first matched EMS records to hospital records using probabilistic linkage (LinkSolv v8.2, Strategic Matching, Inc., Morrisonville, NY). Probabilistic linkage has been used to link large EMS data files to hospital records,19 has been validated for matching EMS records to trauma registry data20 and has been rigorously evaluated in this sample.21 We used all available sources of electronic hospital information, including trauma registries, state discharge databases (admitted patients) and state emergency department (ED) databases (non-admitted patients). Next, we used a mapping function (ICDPIC 3.0 module for Stata v. 11, StataCorp, College Station, TX) to calculate ISS values from ICD-9-CM diagnoses for all patients.22 Previous studies have validated software for mapping administrative diagnosis codes to anatomic injury scores23,24 and we have validated ICDPIC-generated ISS against chart abstracted ISS in this database.25
We used descriptive statistics to characterize the sample. To derive and cross-validate decision rules, we used classification and regression tree (CART) analysis (CART version 6.0, Salford Systems, San Diego, CA).26 CART analysis is a form of binary recursive partitioning that uses nonparametric methods to classify observations based on a large number of possible predictor variables and is particularly useful in identifying complex, higher-level interactions.26 CART is well-suited for developing high-sensitivity clinical decision rules27,28 and allows for data-driven selection of cut-points for continuous variables (e.g., SBP and GCS), rather than reliance on pre-selected, arbitrary values. We used different misclassification costs (i.e., the misclassification of seriously injured patients as non-injured) and Gini splitting functions26 to derive decision rules across a large spectrum of sensitivity values, including the benchmark of ≥95%. We selected tree complexity parameters to develop practical and sensible decision trees that appeared feasible for use in the prehospital setting.28
We estimated accuracy parameters (sensitivity and specificity) using cross-validation to reduce bias and over-fitting of the dataset.29,30 We omitted confidence intervals around cross-validation point estimates because appropriate variance calculation is not possible with CART-generated cross-validation. Results from the derived decision trees were compared to accuracy estimates for current field triage practices. We also evaluated the CART-generated metric for predictor variable importance (a normalized score from 0 to 100) to further assess the relative utility of individual triage criteria. This score is based on the number of times a given predictor is used as a primary or surrogate decision node “splitter” in the tree-building process.
The primary analysis included injured persons of all ages. However, we also analyzed decision trees for children (0 – 17 years) and older adults (> 60 years) based on the continued debate regarding the utility of age-specific triage guidelines to account for age-based differences in physiology and under-triage. We selected patients > 60 years (rather than ≥55 years) based on previous research demonstrating that under-triage begins to rise in patients over 60 years.13 We managed the database and conducted descriptive analyses using SAS (v 9.2, SAS Institute, Cary, NC).
There were 89,261 injured patients transported by EMS during the 3-year period with matched hospital records, of whom 5,711 (6.4%) had ISS >= 16 and 1,672 (1.9%) died during their hospital stay. Characteristics of the primary sample and age-based subgroups are provided in Table 1.
Figure 1 illustrates resulting changes in specificity, number of over-triaged and under-triaged patients among multiple field triage decision schemes with differing sensitivity values. Triage criteria currently in use demonstrated 87.5% sensitivity and 62.8% specificity. For decision rules with sensitivities > 90% (and therefore fewer under-triaged patients), there was a substantial drop in specificity and consequent increase in the number of over-triaged patients. As the 95% sensitivity target was approached, decision tree complexity (measured by the number of decision nodes in the tree) increased. Compared to current field triage practices, the decision scheme meeting the 95% sensitivity benchmark reduced specificity from 62.8% to 18.7% and doubled the number of non-seriously injured patients over-triaged to major trauma centers (67,927 vs. 31,104).
We replicated similar analyses for children (Figure 2) and older adults (Figure 3). The spectrum of decision rules for children were similar to that of the full sample, although we were able to develop a decision scheme with sensitivity of 93.9% before incurring a notable drop in specificity. Decision rules specific to older adults behaved differently from those of younger patients, with a marked decrease in specificity (from 76.6% to 43.4%) occurring at a lower sensitivity value (89.5%). Older adults also had the greatest potential increase in non-seriously injured patients over-triaged to major trauma centers (22,085 vs. 6,926) using a triage strategy with sensitivity (94.9%) close to the national benchmark.
There were several key findings when comparing high-sensitivity decision trees in the full sample (Figure 4) and age-specific subgroups (children - Appendix 1; older adults – Appendix 2). All decision schemes with sensitivity values approaching the 95% benchmark contained the current triage criteria (no simplification or removal of criteria) and a GCS cut-point of ≤14 (rather than the current GCS ≤13). Changing the GCS criterion increased sensitivity in all age groups (from 87.5% to 90.4% for all ages; from 89.1% to 91.7% for children; and from 78.2% to 82.6% for older adults) with relatively small changes in specificity. The importance of field GCS in building high-sensitivity decision schemes with CART was further illustrated in the variable importance metric (Table 2). Field GCS was the most important predictor of serious injury among the all-age sample and children, and was second in importance only to the age criterion (< 5 or > 55 years) among older adults.
In this multi-region study, we demonstrate the many trade-offs inherent in different strategies for field triage. Current estimates of triage sensitivity fall short of the 95% benchmark established by ACS-COT8 and implementing triage guidelines to reach this benchmark appear to have major implications for over-triage, ambulance transport patterns, shifts in trauma center volumes and the efficiency of trauma systems. Our results raise fundamental questions about the reality of reaching the current benchmarks for field triage and provide insight into modest modifications to the existing triage guidelines to improve sensitivity.
For decades, ACS-COT has set the benchmark for field triage sensitivity in trauma systems at ≥95%.5–8 Several early studies suggested that field triage practices were hitting this sensitivity mark.31–33 However, subsequent research has demonstrated lower sensitivity values for field triage,12–15 particularly among older patients. Our findings confirm the lower sensitivity values for current field triage practices and suggest a “ceiling” for triage sensitivity of 90%. Efforts to push sensitivity toward the 95% benchmark resulted in pronounced and likely unacceptable increases in over-triage with major volume shifts of non-seriously injured patients to trauma centers. Such a change in trauma center volumes risks overwhelming limited trauma resources, undermining the regionalized trauma care delivery system, reducing system efficiency, and increasing costs.34 Because trauma center care is most cost-effective for younger patients with severe injuries35 and treating patients without serious injury in major trauma centers can substantially increase the cost of care,34 an inappropriately high sensitivity benchmark (with resulting increases in over-triage) may not be cost-effective. Major shifts in trauma center volumes may also jeopardize the outcomes of concurrently arriving patients with other acute medical conditions36 and potentially upset the balance of trauma centers serving as specialized centers for patients with other time-dependent medical conditions (e.g., ST-elevation myocardial infarction, stroke, out-of-hospital cardiac arrest). The 90% field triage sensitivity appears to represent a tipping point where trauma centers (and trauma systems) become less efficient and less effective as regionalized care centers.
If such a “ceiling” for the sensitivity of field triage exists, a key question is how to effectively concentrate all seriously injured patients in hospitals most capable of caring for them. Efforts to periodically revise and update the national field triage guidelines have improved field triage processes and should continue, although our findings suggest that secondary (hospital-based) triage is also likely to be important in matching patient need to hospital capability. While previous research on secondary triage has generally focused on more efficient use of internal trauma center resources (e.g., guiding the need for a trauma surgeon on patient arrival), a potentially larger function of secondary triage is the early identification and inter-hospital transfer of seriously injured patients presenting to non-trauma hospitals. Secondary triage outside of major trauma centers has the potential for improving the shortcomings of primary triage.
While the intent of the study was not to develop the “best” triage algorithm, there were key similarities among high-sensitivity decision trees that offer insight into potential modifications to the current Field Triage Decision Scheme.9 First, most of the high-sensitivity decisions trees built on the current triage criteria, rather than simplifying the algorithm or removing certain criteria. Beyond the current triage criteria, the most consistent predictor for identifying patients with serious injury was initial prehospital GCS score ≤ 14, which clearly outperformed the current GCS-based triage criterion (GCS score ≤13). Previous research focused on elders and children have also supported a GCS score ≤14 triage criterion.37,38 While the GCS cut point was increased from ≤ 12 to ≤ 13 in the 1993 revision of the Field Triage Decision Scheme,6 there has been resistance to moving the GCS cut point to ≤14 based on perceived increases in over-triage and overuse of trauma resources. We believe our findings provide evidence against these concerns and demonstrate the added benefit of modifying the GCS criterion (reduced under-triage and simplification of GCS for field use) with minimal increase in over-triage.
Finally, our results provide useful insight into the trade-offs inherent in attempting to resolve the under-triage issue for older adults through the field triage guidelines. The most recent version of the Field Triage Decision Scheme incorporates specific changes designed to minimize under-triage in elders,9 although the result of such modifications remain unknown. We demonstrate that while it may be possible to reduce under-triage through revised triage guidelines, this would also likely lead to redirection of a large number of older adults without serious injuries to major trauma centers. Such a trade-off may be justified through better outcomes in older adults (if these can be demonstrated), although there may also be unintended consequences (e.g., reduced revenue streams for non-trauma hospitals, disruption in care for certain patients and reduced patient autonomy). These trade-offs should be considered by policy-makers before embarking on substantial changes to field triage.
A primary limitation of the study is the retrospective cohort design. We maximized the rigor of the cohort through broad patient sampling, multiple EMS agencies, multiple hospitals, and hospital outcomes matched using multiple data sources (not just trauma registry data). We also triangulated data sources to maximize identification of individual triage criteria and used analytic methods well-matched to the development of clinical decision rules. Despite these methods, the sample was limited to patients who matched to hospital records, which excluded certain subgroups of patients (e.g., those seen and discharged from the ED at sites where ED data was not available).21 A previous study conducted by our group suggested larger estimates for under-triage using a population-based sample with imputed values compared to a matched-only sample.13 These comparisons suggest that our results are conservative and that the magnitude of over-triage (including patient volume shifts to major trauma centers) with high-sensitivity decision schemes may be even greater than presented. Our findings will need to be replicated using prospective, population-based samples.
Also, the primary outcome (ISS ≥16) was calculated using methodology that maps ISS from ICD9-CM diagnosis codes. This was required because we included patients transported to all types of hospitals, including trauma centers and non-trauma centers. While chart abstraction is generally felt to be more accurate for coding injury severity, abstraction was not feasible in this study. Use of ICD9-to-ISS mapping can underestimate ISS in some patients due to injuries not documented as ICD9 codes,25 which could have slightly underestimated injury severity in the sample. Finally, age-based definitions used for “children” and “older adults” differ by trauma system and vary in the trauma literature. We defined the age groups in this study for broad generalizability using common definitions. However, different age definitions (e.g., ≤14 years for children, ≥70 years for older adults) may have produced slightly different results.
Implementing changes to national triage guidelines to reach the sensitivity benchmark of 95% may result in substantial over-triage rates and major increases in the volume of non-seriously injured patients transported to major trauma centers. Whether such changes are feasible and would improve outcomes remains unclear. A more realistic target for field triage sensitivity appears to be 90%, which may be possible through modest modifications to the current triage scheme without substantial changes in over-triage.
Appendix 1. Sample high-sensitivity clinical decision rule for children (n = 8,944).
*Cross-validation results for the decision rule demonstrate sensitivity 91.7% and specificity 48.7%.
Appendix 2. Sample high sensitivity clinical decision rule for older adults ≥ 55 years (n = 31,055).
*Cross-validation results for the decision rule demonstrate sensitivity 94.9% and specificity 25.3%. This decision tree demonstrates that the physiologic cut-points (e.g., SBP) identifying seriously injured patients may change as the sample is narrowed to certain mechanisms of injury and patients not meeting criteria higher in the tree.
Source of Funding
This project was supported by the Robert Wood Johnson Foundation Physician Faculty Scholars Program; the Oregon Clinical and Translational Research Institute (grant # UL1 RR024140); University of California, Davis Clinical and Translational Science Center (grant # UL1 RR024146); Stanford Center for Clinical and Translational Education and Research (grant # 1UL1 RR025744); University of Utah Center for Clinical and Translational Science (grant # UL1-RR025764 and C06-RR11234); and University of California, San Francisco Clinical and Translational Science Institute (grant # UL1 RR024131). All Clinical and Translational Science Awards are from the National Center for Research Resources, a component of the National Institutes of Health (NIH), and NIH Roadmap for Medical Research.
We want to acknowledge and thank all the participating EMS agencies, EMS medical directors, trauma registrars, and state offices that provided data for and supported this project.
Conflicts of Interest
There are no conflicts of interest among authors for this paper.
AUTHOR CONTRIBUTIONSCDN conceived of and designed the study. CDN, RYH, NCM, EBM, NEW, JFH, DZ and KS assisted with acquisition of data. CDN performed all database management and statistical analyses. RF performed ICDPIC mapping of ISS. CDN, RYH, NCM, NEW, JFH, KS and NK helped secure extramural funding for the project. All authors helped interpret the data and results. CDN and NK provided oversight and guidance during the project. CDN drafted the manuscript and all authors participated in critical revision of the manuscript. CDN takes responsibility for the data, results and manuscript as a whole.