PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
J Trauma Acute Care Surg. Author manuscript; available in PMC 2014 May 1.
Published in final edited form as:
PMCID: PMC3726266
NIHMSID: NIHMS454082

The Trade-Offs In Field Trauma Triage: A Multi-Region Assessment of Accuracy Metrics and Volume Shifts Associated With Different Triage Strategies

Craig D. Newgard, MD, MPH,1 Renee Y. Hsia, MD, MSc,2 N. Clay Mann, PhD, MS,3 Terri Schmidt, MD, MS,1,4 Ritu Sahni, MD, MPH,1,5 Eileen M. Bulger, MD,6 N. Ewen Wang, MD,7 James F. Holmes, MD, MPH,8 Ross Fleischman, MD, MCR,1 Dana Zive, MPH,1 Kristan Staudenmayer, MD,9 Jason S. Haukoos, MD, MSc,10 and Nathan Kuppermann, MD, MPH8

Abstract

Background

National benchmarks for trauma triage sensitivity (≥95%) and specificity (≥50%) have not been rigorously evaluated across broad populations of injured patients. We evaluated the impact of different field triage schemes for identifying seriously injured patients across a range of sensitivity values. Impact metrics included specificity and number of under- and over-triaged patients compared to current triage practices.

Methods

This was a retrospective cohort study of injured children and adults transported by 48 EMS agencies to 105 hospitals in 6 regions of the Western U.S. from 2006 through 2008. Hospital outcomes were probabilistically linked to EMS records through trauma registries, state discharge databases and state emergency department databases. The primary outcome was an Injury Severity Score (ISS) ≥16. We evaluated 40 field predictor variables, including 31 current field triage criteria, using classification and regression tree analysis and cross validation to generate estimates for sensitivity and specificity.

Results

89,261 injured patients were evaluated and transported by EMS providers over the 3-year period, of whom 5,711 (6.4%) had ISS ≥16. As the 95% sensitivity target for triage was approached (from the current value of 87.5%), decision tree complexity increased, specificity decreased (from 62.8% to 18.7%), and the number of triage-positive patients without serious injury doubled (67,927 vs. 31,104). Analyses restricted to children and older adults were similar. The most consistent modification to the current triage algorithm to increase sensitivity without a major decrease in specificity was altering the Glasgow Coma Scale (GCS) score cut-point from ≤13 to ≤14 (sensitivity increase to 90.4%).

Conclusions

Reaching the field triage sensitivity benchmark of 95% would require a large decrease in specificity (increase in over-triage). A 90% sensitivity target appears more realistic and may be obtainable by modest changes to the current triage algorithm.

Level of Evidence

III

Study Type

Diagnostic test

Keywords: triage, emergency medical services, serious injury

BACKGROUND

The original Field Triage Decision Scheme for trauma patients was developed by the American College of Surgeons Committee on Trauma (ACS-COT) in 1976, based largely on expert opinion.1,2 The guidelines were a necessary part of early trauma system development and need to concentrate seriously injured patients in specialized regional hospitals (major trauma centers) capable of caring for them. In 1987, the decision scheme was revised and organized into the current algorithmic triage process with multiple “steps,” prioritized by the likelihood of serious injury.3,4 The decision scheme has subsequently been revised five times between 1990 and 2011 based on updated science and expert opinion.59 Despite a large body of literature on field triage and multiple revisions of the guidelines, few studies have explored the effects of different triage strategies on healthcare delivery systems (e.g., ambulance transport patterns and trauma center volumes) or data-driven benchmarks for triage accuracy.

The ACS-COT sets the national benchmarks for field triage at ≥95% sensitivity (proportion of seriously injured patients cared for in major trauma centers) and ≥50% specificity (proportion of non-seriously injured patients cared for in non- trauma centers).8 However, the feasibility of reaching these benchmarks and the resulting implications for trauma systems remain unclear. Recent literature suggests that the benchmark for under-triage is not being met,1015 particularly among older adults, and it remains unknown whether the guidelines could be revised to meet these accuracy targets. It is also uncertain how changes in triage accuracy may impact the volume of injured patients transported to different types of hospitals. These are important health policy questions for trauma systems.

In this study, we sought to empirically derive multiple field triage decision schemes for identifying seriously injured patients (Injury Severity Score [ISS] ≥16) across a range of sensitivity values and to assess their potential impact within trauma systems (under-triage, specificity and over-triage), as compared to current triage practices. Rather than seeking to develop the “best” triage guidelines, this study was designed to provide trauma system policymakers with information about the trade-offs inherent in different strategies for field triage.

METHODS

Study Design

This was a multi-region retrospective cohort study. Fourteen Institutional Review Boards at 6 sites approved this protocol and waived the requirement for informed consent.

Study Setting

The study included injured children and adults evaluated and transported by 48 EMS agencies to 105 hospitals (including 12 Level I, 5 Level II, 3 Level III, 4 Level IV, 1 Level V and 80 community/private/federal hospitals) in 6 sites across the Western U.S over a 3-year period (January 1, 2006 through December 31, 2008). The 6 sites included: Portland, OR/Vancouver, WA (4 counties); King County, WA; Sacramento, CA (2 counties); San Francisco, CA; Santa Clara, CA (2 counties); and Denver County, CO. Each site represents a pre-defined geographic “footprint,” consisting of a central metropolitan area and surrounding region (suburban and some rural areas), defined by EMS agency service areas. All sites have established trauma systems.

Patient Population

The study sample included all injured patients (EMS primary impression of “injury” or “trauma”) activating the 9–1-1 EMS system and transported to an acute care hospital (trauma centers and non-trauma centers) with a matched hospital record. Specifying the sample in this manner allowed for an out-of-hospital injury cohort defined by EMS providers and consisting of patients with both minor and serious injuries. This sampling strategy also approximates the denominator of patients to whom field trauma triage guidelines are routinely applied. Excluded patients included: interhospital transfers without an initial presentation involving EMS; EMS runs listed as “cancelled,” “no patient found” or “stand by” (calls without patient contact); and patients that were not transported (e.g., deaths in the field, refusals of transport).

Variables

We recorded 40 field-based predictors of serious injury for analysis, including 31 trauma triage criteria in local use in the 6 regions. Most of the triage criteria came from different versions of the Field Triage Decision Scheme,3,59 although we also included site-specific triage criteria developed for local use (e.g., steering wheel deformity). We triangulated several different data sources to identify all field “trauma activation” patients (i.e., patients meeting any of the field triage criteria, per EMS personnel) and to comprehensively capture all individual triage criteria used by EMS. We determined field triage information from: (1) trauma triage criteria specified in the EMS chart; (2) EMS provider-documented “trauma activation” (or similar charting, depending on local terminology); (3) EMS-recorded trauma identification number (used at some sites as a mechanism for tracking injured patients entered into the trauma system); (4) a matched record from the local trauma registry specifying “scene” origin (i.e., EMS-identified trauma activations); and (5) matched trauma communication center records (phone records from base hospital communication with EMS providers, required for all field trauma activations in certain sites). We evaluated the presence of triage criteria independent of transport destination.

We also recorded patient demographics (age, gender), initial field physiologic measures (Glasgow Coma Scale [GCS] score, systolic blood pressure [SBP], respiratory rate, and heart rate), field-assisted ventilation (bag-valve mask ventilation, intubation or supraglottic airway use), mechanism of injury (15 categories) and hospital destination. We categorized acute care hospitals as tertiary trauma centers (Level I or II trauma hospitals) or non-trauma centers based on their American College of Surgeons accreditation status and state-level designations.

Outcomes

The primary outcome for this study was “serious injury,” defined as an ISS ≥16. While many definitions of serious injury have been used in previous triage studies, an ISS ≥16 is the metric used by ACS-COT to define under- and over-triage8 and identifies the subset of patients most likely to benefit from care in major trauma centers.1618

To capture ISS for patients transported to all types of hospitals, we first matched EMS records to hospital records using probabilistic linkage (LinkSolv v8.2, Strategic Matching, Inc., Morrisonville, NY). Probabilistic linkage has been used to link large EMS data files to hospital records,19 has been validated for matching EMS records to trauma registry data20 and has been rigorously evaluated in this sample.21 We used all available sources of electronic hospital information, including trauma registries, state discharge databases (admitted patients) and state emergency department (ED) databases (non-admitted patients). Next, we used a mapping function (ICDPIC 3.0 module for Stata v. 11, StataCorp, College Station, TX) to calculate ISS values from ICD-9-CM diagnoses for all patients.22 Previous studies have validated software for mapping administrative diagnosis codes to anatomic injury scores23,24 and we have validated ICDPIC-generated ISS against chart abstracted ISS in this database.25

Data Analysis

We used descriptive statistics to characterize the sample. To derive and cross-validate decision rules, we used classification and regression tree (CART) analysis (CART version 6.0, Salford Systems, San Diego, CA).26 CART analysis is a form of binary recursive partitioning that uses nonparametric methods to classify observations based on a large number of possible predictor variables and is particularly useful in identifying complex, higher-level interactions.26 CART is well-suited for developing high-sensitivity clinical decision rules27,28 and allows for data-driven selection of cut-points for continuous variables (e.g., SBP and GCS), rather than reliance on pre-selected, arbitrary values. We used different misclassification costs (i.e., the misclassification of seriously injured patients as non-injured) and Gini splitting functions26 to derive decision rules across a large spectrum of sensitivity values, including the benchmark of ≥95%. We selected tree complexity parameters to develop practical and sensible decision trees that appeared feasible for use in the prehospital setting.28

We estimated accuracy parameters (sensitivity and specificity) using cross-validation to reduce bias and over-fitting of the dataset.29,30 We omitted confidence intervals around cross-validation point estimates because appropriate variance calculation is not possible with CART-generated cross-validation. Results from the derived decision trees were compared to accuracy estimates for current field triage practices. We also evaluated the CART-generated metric for predictor variable importance (a normalized score from 0 to 100) to further assess the relative utility of individual triage criteria. This score is based on the number of times a given predictor is used as a primary or surrogate decision node “splitter” in the tree-building process.

The primary analysis included injured persons of all ages. However, we also analyzed decision trees for children (0 – 17 years) and older adults (> 60 years) based on the continued debate regarding the utility of age-specific triage guidelines to account for age-based differences in physiology and under-triage. We selected patients > 60 years (rather than ≥55 years) based on previous research demonstrating that under-triage begins to rise in patients over 60 years.13 We managed the database and conducted descriptive analyses using SAS (v 9.2, SAS Institute, Cary, NC).

RESULTS

There were 89,261 injured patients transported by EMS during the 3-year period with matched hospital records, of whom 5,711 (6.4%) had ISS >= 16 and 1,672 (1.9%) died during their hospital stay. Characteristics of the primary sample and age-based subgroups are provided in Table 1.

Table 1
Characteristics of injured patients transported by EMS in 6 regions.*

Figure 1 illustrates resulting changes in specificity, number of over-triaged and under-triaged patients among multiple field triage decision schemes with differing sensitivity values. Triage criteria currently in use demonstrated 87.5% sensitivity and 62.8% specificity. For decision rules with sensitivities > 90% (and therefore fewer under-triaged patients), there was a substantial drop in specificity and consequent increase in the number of over-triaged patients. As the 95% sensitivity target was approached, decision tree complexity (measured by the number of decision nodes in the tree) increased. Compared to current field triage practices, the decision scheme meeting the 95% sensitivity benchmark reduced specificity from 62.8% to 18.7% and doubled the number of non-seriously injured patients over-triaged to major trauma centers (67,927 vs. 31,104).

Figure 1
Sensitivity, specificity and volume shifts with different strategies for field trauma triage (all ages, n = 89,261).

We replicated similar analyses for children (Figure 2) and older adults (Figure 3). The spectrum of decision rules for children were similar to that of the full sample, although we were able to develop a decision scheme with sensitivity of 93.9% before incurring a notable drop in specificity. Decision rules specific to older adults behaved differently from those of younger patients, with a marked decrease in specificity (from 76.6% to 43.4%) occurring at a lower sensitivity value (89.5%). Older adults also had the greatest potential increase in non-seriously injured patients over-triaged to major trauma centers (22,085 vs. 6,926) using a triage strategy with sensitivity (94.9%) close to the national benchmark.

Figure 2
Sensitivity, specificity and volume shifts with different strategies for field trauma triage of children 0 – 17 years (n = 8,944).
Figure 3
Sensitivity, specificity and volume shifts with different strategies for field trauma triage of elders over 60 years (n = 31,055).

There were several key findings when comparing high-sensitivity decision trees in the full sample (Figure 4) and age-specific subgroups (children - Appendix 1; older adults – Appendix 2). All decision schemes with sensitivity values approaching the 95% benchmark contained the current triage criteria (no simplification or removal of criteria) and a GCS cut-point of ≤14 (rather than the current GCS ≤13). Changing the GCS criterion increased sensitivity in all age groups (from 87.5% to 90.4% for all ages; from 89.1% to 91.7% for children; and from 78.2% to 82.6% for older adults) with relatively small changes in specificity. The importance of field GCS in building high-sensitivity decision schemes with CART was further illustrated in the variable importance metric (Table 2). Field GCS was the most important predictor of serious injury among the all-age sample and children, and was second in importance only to the age criterion (< 5 or > 55 years) among older adults.

Figure 4
Sample high-sensitivity clinical decision rule for all ages (n = 89,261).*
Table 2
Top 10 field predictors of serious injury (ISS >= 16) using a measure of variable importance through derivation of high sensitivity (≥95%) classification and regression trees.*

DISCUSSION

In this multi-region study, we demonstrate the many trade-offs inherent in different strategies for field triage. Current estimates of triage sensitivity fall short of the 95% benchmark established by ACS-COT8 and implementing triage guidelines to reach this benchmark appear to have major implications for over-triage, ambulance transport patterns, shifts in trauma center volumes and the efficiency of trauma systems. Our results raise fundamental questions about the reality of reaching the current benchmarks for field triage and provide insight into modest modifications to the existing triage guidelines to improve sensitivity.

For decades, ACS-COT has set the benchmark for field triage sensitivity in trauma systems at ≥95%.58 Several early studies suggested that field triage practices were hitting this sensitivity mark.3133 However, subsequent research has demonstrated lower sensitivity values for field triage,1215 particularly among older patients. Our findings confirm the lower sensitivity values for current field triage practices and suggest a “ceiling” for triage sensitivity of 90%. Efforts to push sensitivity toward the 95% benchmark resulted in pronounced and likely unacceptable increases in over-triage with major volume shifts of non-seriously injured patients to trauma centers. Such a change in trauma center volumes risks overwhelming limited trauma resources, undermining the regionalized trauma care delivery system, reducing system efficiency, and increasing costs.34 Because trauma center care is most cost-effective for younger patients with severe injuries35 and treating patients without serious injury in major trauma centers can substantially increase the cost of care,34 an inappropriately high sensitivity benchmark (with resulting increases in over-triage) may not be cost-effective. Major shifts in trauma center volumes may also jeopardize the outcomes of concurrently arriving patients with other acute medical conditions36 and potentially upset the balance of trauma centers serving as specialized centers for patients with other time-dependent medical conditions (e.g., ST-elevation myocardial infarction, stroke, out-of-hospital cardiac arrest). The 90% field triage sensitivity appears to represent a tipping point where trauma centers (and trauma systems) become less efficient and less effective as regionalized care centers.

If such a “ceiling” for the sensitivity of field triage exists, a key question is how to effectively concentrate all seriously injured patients in hospitals most capable of caring for them. Efforts to periodically revise and update the national field triage guidelines have improved field triage processes and should continue, although our findings suggest that secondary (hospital-based) triage is also likely to be important in matching patient need to hospital capability. While previous research on secondary triage has generally focused on more efficient use of internal trauma center resources (e.g., guiding the need for a trauma surgeon on patient arrival), a potentially larger function of secondary triage is the early identification and inter-hospital transfer of seriously injured patients presenting to non-trauma hospitals. Secondary triage outside of major trauma centers has the potential for improving the shortcomings of primary triage.

While the intent of the study was not to develop the “best” triage algorithm, there were key similarities among high-sensitivity decision trees that offer insight into potential modifications to the current Field Triage Decision Scheme.9 First, most of the high-sensitivity decisions trees built on the current triage criteria, rather than simplifying the algorithm or removing certain criteria. Beyond the current triage criteria, the most consistent predictor for identifying patients with serious injury was initial prehospital GCS score ≤ 14, which clearly outperformed the current GCS-based triage criterion (GCS score ≤13). Previous research focused on elders and children have also supported a GCS score ≤14 triage criterion.37,38 While the GCS cut point was increased from ≤ 12 to ≤ 13 in the 1993 revision of the Field Triage Decision Scheme,6 there has been resistance to moving the GCS cut point to ≤14 based on perceived increases in over-triage and overuse of trauma resources. We believe our findings provide evidence against these concerns and demonstrate the added benefit of modifying the GCS criterion (reduced under-triage and simplification of GCS for field use) with minimal increase in over-triage.

Finally, our results provide useful insight into the trade-offs inherent in attempting to resolve the under-triage issue for older adults through the field triage guidelines. The most recent version of the Field Triage Decision Scheme incorporates specific changes designed to minimize under-triage in elders,9 although the result of such modifications remain unknown. We demonstrate that while it may be possible to reduce under-triage through revised triage guidelines, this would also likely lead to redirection of a large number of older adults without serious injuries to major trauma centers. Such a trade-off may be justified through better outcomes in older adults (if these can be demonstrated), although there may also be unintended consequences (e.g., reduced revenue streams for non-trauma hospitals, disruption in care for certain patients and reduced patient autonomy). These trade-offs should be considered by policy-makers before embarking on substantial changes to field triage.

A primary limitation of the study is the retrospective cohort design. We maximized the rigor of the cohort through broad patient sampling, multiple EMS agencies, multiple hospitals, and hospital outcomes matched using multiple data sources (not just trauma registry data). We also triangulated data sources to maximize identification of individual triage criteria and used analytic methods well-matched to the development of clinical decision rules. Despite these methods, the sample was limited to patients who matched to hospital records, which excluded certain subgroups of patients (e.g., those seen and discharged from the ED at sites where ED data was not available).21 A previous study conducted by our group suggested larger estimates for under-triage using a population-based sample with imputed values compared to a matched-only sample.13 These comparisons suggest that our results are conservative and that the magnitude of over-triage (including patient volume shifts to major trauma centers) with high-sensitivity decision schemes may be even greater than presented. Our findings will need to be replicated using prospective, population-based samples.

Also, the primary outcome (ISS ≥16) was calculated using methodology that maps ISS from ICD9-CM diagnosis codes. This was required because we included patients transported to all types of hospitals, including trauma centers and non-trauma centers. While chart abstraction is generally felt to be more accurate for coding injury severity, abstraction was not feasible in this study. Use of ICD9-to-ISS mapping can underestimate ISS in some patients due to injuries not documented as ICD9 codes,25 which could have slightly underestimated injury severity in the sample. Finally, age-based definitions used for “children” and “older adults” differ by trauma system and vary in the trauma literature. We defined the age groups in this study for broad generalizability using common definitions. However, different age definitions (e.g., ≤14 years for children, ≥70 years for older adults) may have produced slightly different results.

CONCLUSIONS

Implementing changes to national triage guidelines to reach the sensitivity benchmark of 95% may result in substantial over-triage rates and major increases in the volume of non-seriously injured patients transported to major trauma centers. Whether such changes are feasible and would improve outcomes remains unclear. A more realistic target for field triage sensitivity appears to be 90%, which may be possible through modest modifications to the current triage scheme without substantial changes in over-triage.

Supplementary Material

Appendix 1

Appendix 1. Sample high-sensitivity clinical decision rule for children (n = 8,944).

*Cross-validation results for the decision rule demonstrate sensitivity 91.7% and specificity 48.7%.

Appendix 2

Appendix 2. Sample high sensitivity clinical decision rule for older adults ≥ 55 years (n = 31,055).

*Cross-validation results for the decision rule demonstrate sensitivity 94.9% and specificity 25.3%. This decision tree demonstrates that the physiologic cut-points (e.g., SBP) identifying seriously injured patients may change as the sample is narrowed to certain mechanisms of injury and patients not meeting criteria higher in the tree.

Acknowledgments

Source of Funding

This project was supported by the Robert Wood Johnson Foundation Physician Faculty Scholars Program; the Oregon Clinical and Translational Research Institute (grant # UL1 RR024140); University of California, Davis Clinical and Translational Science Center (grant # UL1 RR024146); Stanford Center for Clinical and Translational Education and Research (grant # 1UL1 RR025744); University of Utah Center for Clinical and Translational Science (grant # UL1-RR025764 and C06-RR11234); and University of California, San Francisco Clinical and Translational Science Institute (grant # UL1 RR024131). All Clinical and Translational Science Awards are from the National Center for Research Resources, a component of the National Institutes of Health (NIH), and NIH Roadmap for Medical Research.

We want to acknowledge and thank all the participating EMS agencies, EMS medical directors, trauma registrars, and state offices that provided data for and supported this project.

Footnotes

Conflicts of Interest

There are no conflicts of interest among authors for this paper.

AUTHOR CONTRIBUTIONS

CDN conceived of and designed the study. CDN, RYH, NCM, EBM, NEW, JFH, DZ and KS assisted with acquisition of data. CDN performed all database management and statistical analyses. RF performed ICDPIC mapping of ISS. CDN, RYH, NCM, NEW, JFH, KS and NK helped secure extramural funding for the project. All authors helped interpret the data and results. CDN and NK provided oversight and guidance during the project. CDN drafted the manuscript and all authors participated in critical revision of the manuscript. CDN takes responsibility for the data, results and manuscript as a whole.

References

1. Mackersie RC. History of Trauma Field Triage Development and the American College of Surgeons Criteria. Prehospital Emerg Care. 2006;10(3):287–294. [PubMed]
2. Bulletin of the American College of Surgeons. Sep, 1976.
3. Hospital and Prehospital Resources for the Optimal Care of the Injured Patient. Chicago, IL: American College of Surgeons; 1987.
4. Field categorization of trauma patients. Bull Am Coll Surg. 1986;71:17–21.
5. Resources for the Optimal Care of the Injured Patient. Chicago, IL: Amer College of Surgeons; 1990.
6. Resources for the Optimal Care of the Injured Patient. Chicago, IL: Amer College of Surgeons; 1993.
7. Resources for the Optimal Care of the Injured Patient. Chicago, IL: Amer College of Surgeons; 1999.
8. Resources for the Optimal Care of the Injured Patient. Chicago, IL: Amer College of Surgeons; 2006.
9. Sasser SM, Hunt RC, Faul M, et al. Guidelines for Field Triage of Injured Patients: Recommendations of the National Expert Panel on Field Triage, 2011. MMWR. 2012;61:1–20. [PubMed]
10. Hsia RY, Wang E, Torres H, et al. Disparities in trauma center access despite increasing utilization: data from California, 1999 to 2006. J Trauma. 2010;68:217–24. [PMC free article] [PubMed]
11. Vassar MJ, Holcroft JJ, Knudson MM, Kizer KW. Fractures in access to and assessment of trauma systems. J Am Coll Surg. 2003;197:717–725. [PubMed]
12. Newgard CD, Zive D, Holmes JF, et al. A Multi-Site Assessment of the ACSCOT Field Triage Decision Scheme For Identifying Seriously Injured Children and Adults. J Am Coll Surg. 2011;213:709–21. [PMC free article] [PubMed]
13. Nakamura Y, Daya M, Bulger EM, Schreiber M, Mackersie R, Hsia RY, Mann NC, Holmes JF, Staudenmayer K, Sturges Z, Liao M, Haukoos J, Kuppermann N, Barton ED, Newgard CD. Evaluating age in the field triage of injured persons. Ann Emerg Med. In Press. [PMC free article] [PubMed]
14. Chang DC, Bass RR, Cornwell EE, Mackenzie EJ. Undertriage of elderly trauma patients to state-designated trauma centers. Arch Surg. 2008;143(8):776–81. discussion 782. [PubMed]
15. Baez AA, Lane PL, Sorondo B. System compliance with out-of-hospital trauma triage criteria. J Trauma. 2003;54:344–351. [PubMed]
16. MacKenzie EJ, Rivara FP, Jurkovich GJ, et al. A national evaluation of the effect of trauma-center care on mortality. NEJM. 2006;354:366–378. [PubMed]
17. Mullins RJ, Veum-Stone J, Helfand M, et al. Outcome of hospitalized injured patients after institution of a trauma system in an urban area. JAMA. 1994;271:1919–1924. [PubMed]
18. Mullins RJ, Veum-Stone J, Hedges JR, et al. Influence of a statewide trauma system on location of hospitalization and outcome of injured patients. J Trauma. 1996;40:536–545. [PubMed]
19. Dean JM, Vernon DD, Cook L, et al. Probabilistic linkage of computerized ambulance and inpatient hospital discharge records: a potential tool for evaluation of emergency medical services. Ann Emerg Med. 2001;37:616–626. [PubMed]
20. Newgard CD. Validation of probabilistic linkage to match de-identified ambulance records to a state trauma registry. Acad Emerg Med. 2006;13:69–75. [PubMed]
21. Newgard CD, Malveau S, Staudenmayer K, Wang NE, Hsia RY, Mann NC, Holmes JF, Kuppermann N, Haukoos JS, Bulger EM, Dai M, Cook LJ. Evaluating the Use of Existing Data Sources, Probabilistic Linkage and Multiple Imputation to Build Population-Based Injury Databases Across Phases of Trauma Care. Acad Emerg Med. 2012;19:469–480. [PMC free article] [PubMed]
22. Clark DE, Osler TM, Hahn DR. ICDPIC: Stata Module to Provide Methods for Translating International Classification of Diseases (Ninth Revision) Diagnosis Codes into Standard Injury Categories and/or Scores. Boston: Boston College, Department of Economics; 2009.
23. MacKenzie EJ, Steinwachs DM, Shankar BS, Turney SZ. An ICD-9CM to AIS conversion table: development and application. Proc AAAM. 1986;30:135–151.
24. MacKenzie EJ, Steinwachs DM, Shankar B. Classifying trauma severity based on hospital discharge diagnoses. Validation of an ICD-9CM to AIS-85 conversion table. Med Care. 1989;27:412–422. [PubMed]
25. Fleischman RJ, Mann NC, Wang NE, et al. Validating The Use Of ICD9 Codes To Generate Injury Severity Score: The ICDPIC Mapping Procedure. [Abstract] Acad Emerg Med. 2012 (In Press)
26. Breiman L, Freidman JH, Olshen RA, et al. Classification and regression trees. New York: Chapman & Hall; 1984.
27. McConnochie KM, Roghmann KJ, Pasternack J. Developing prediction rules and evaluating observation patterns using categorical clinical markers: two complimentary procedures. Medical Decision Making. 1993;13(1):30–42. [PubMed]
28. Stiell IG, Wells GA. Methodologic standard for the development of clinical decision rules in emergency medicine. Ann Emerg Med. 1999;33:437–447. [PubMed]
29. Efron B. The regional conference series in applied mathematics. Philadelphia: Society for Industrial and Applied Mathematics; 1982. The jackknife, the bootstrap and other resampling plans.
30. Stone M. Cross-validatory choice and the assessment of statistical predictions. J Roy Statist Soc Ser B. 1974;36:111–147.
31. Norcross ED, Ford DW, Cooper ME, Zone-Smith L, Byrne TK, Yarbrough DR. Application of American College of Surgeons’ field triage guidelines by pre-hospital personnel. J Am Coll Surg. 1995;181:539–544. [PubMed]
32. Wuerz R, Taylor J, Smith JS. Accuracy of trauma triage in patients transported by helicopter. Air Med J. 1996;15:168–170. [PubMed]
33. Henry MC, Hollander JE, Alicandro JM, Cassara G, O’Malley S, Thode HC. Incremental benefit of individual American College of Surgeons trauma triage criteria. Acad Emerg Med. 1996;3:992–1000. [PubMed]
34. Newgard CD, Staudenmayer K, Hsia RY, Mann NC, Bulger EM, Holmes JF, Fleischman RJ, Gorman K, Liao M, McConnell KJ. Acute care costs by hospital type, field triage status and emergency medical services transport patterns for injured patients in 7 regions. Health Affairs (under review at)
35. MacKenzie EJ, Weir S, Rivara FP, Jurkovich GJ, Nathens AB, Wang W, Scharfstein DO, Salkever DS. The value of trauma center care. J Trauma. 2010;69:1–9. [PubMed]
36. Fishman PE, Shofer FS, Robey JL, Zogby KE, Reilly PM, Branas CC, Pines JM, Hollander JE. The impact of trauma activations on the care of emergency department patients with potential acute coronary syndromes. Ann Emerg Med. 2006;48:347–353. [PubMed]
37. Newgard CD, Cudnik M, Warden CR, Hedges JR. The predictive value and appropriate ranges of prehospital physiologic parameters for high-risk injured children. Pediatric Emergency Care. 2007;23:450–456. [PubMed]
38. Caterino JM, Raubenolt A, Cudnik MT. Modification of Glasgow Coma Scale criteria for injured elders. Acad Emerg Med. 2011;18:1014–1021. [PubMed]