This study demonstrates that researchers are increasingly utilizing the NTDB to study multiple domains of trauma research. NTDB data have been the source of approximately 30 peer-reviewed, risk-adjusted outcomes studies per year for the past few years. Most publications (approximately 80%) have relied on mortality as their main outcome measure; however, there are significant differences in the variables and methodologies used by authors to perform risk-adjusted analyses, even though they share the same outcome measure. In fact, more than 40% of studies reviewed did not control for the 5 basic co-variates regarded as necessary to study risk-adjusted outcomes for mortality after trauma. These disparities in study approaches lead to significant variations in the quality of studies generated from the NTDB and suggest the need to develop best practices or guidelines for large database trauma outcomes studies.
The genesis of the NTDB can be attributed to the concerted efforts of the trauma community over the span of several decades.134-137
The forerunner to the current NTDB was established by the American College of Surgeons – Committee on Trauma (ACSCOT) in 1989 with the vision of bringing improvement in “the care of the injured through systematic efforts in prevention, care and rehabilitation.” The NTDB has grown exponentially from 1 million records in 2004 to more than 4 million records currently.6
In 2010, 682 trauma centers submitted data to this trauma registry, including 210 Level I, 220 Level II and 198 Level III or IV trauma centers.6
In fact, it is now a requirement for ACS verification that all Level I and Level II trauma centers submit data to the NTDB. The original data elements of the NTDB were defined in 1995. More recently, the full adoption of the National Trauma Data Standard (NTDS) in 2007 aims to standardize the inclusion criteria, data coding and collection procedures of all data entered into the NTDB. This initiative facilitates the generation of a more homogeneous and standardized national trauma registry database, and its adoption is expected to further strengthen the NTDB by facilitating comparisons between different trauma systems, improving the reliability of national trauma benchmarks and providing a more robust foundation for trauma outcomes research.4
Currently there is no standardized way for investigators to adjust for mortality risk while doing outcomes analyses. This review of published studies suggests that the absence of such standardization or best practices, in turn, limits the interpretability and external validity of scientific findings derived from the NTDB data. Based on these results, we suggest authors follow the three simple, yet key, considerations portrayed in to improve the reliability and generalizability of outcome studies generated from NTDB data.
Important Considerations while Performing Risk Adjusted Analyses using NTDB Data to Study Trauma Outcomes
These basic considerations provide an important starting point for improved standardization of NTDB data analysis. Further quantitative evaluation is needed of co-variates that have been shown to independently influence outcomes and whose incorporation would likely improve the precision and accuracy of prediction models. Thus, the next step to improving NTDB data analysis may be to determine a set of best practices based on consensus of statistical methodology and trauma experts on essential variables and analytical techniques. In time, this could lead to publication of standards that could be used by manuscript reviewers to determine the quality and appropriateness of analyses that utilize NTDB data. Such a level of scrutiny would further help clinicians and policy-makers in determining the level of evidence a paper is presenting and the validity of its findings.
In the present review, it was noted that studies have used NTDB data to investigate a wide variety of topics. Clinical outcomes accounted for more than forty percent of the published studies. With its large sample size, the NTDB allows for the study of common problems such as splenic injury and less common injuries such duodenal trauma. While studying outcomes among patients treated at multiple institutions, it is important to adjust for the effect of inter-hospital outcome variations. One way to do this is the use of clustering by facility during multivariable analyses, which has been shown to impact confidence intervals and therefore, the statistical significance of findings.138
Only a small percentage of the total studies in this review described the use of clustering by facility or similar techniques in their methodology. Quality and public health policy papers were also very common, including studies aimed at quantifying complications. Recent work has demonstrated the importance of selecting the appropriate numerators and denominators when making inferences139
about the prevalence of a specific complication or condition while using NTDB data. Fifteen publications described or reported a race- or health insurance-based disparity in trauma outcomes. The size of the NTDB has been instrumental in describing disparities in trauma outcomes and is another example of its utility.
In this review of studies, mortality was the most widely used outcome measure, which is not surprising, as it is a clearly defined, frequently recorded, reliably documented and readily interpretable measure.41,96
It gives little allowance for error in measurement and can be expected to have minimal bias, despite sampling from geographically diverse trauma centers.96
This review of studies utilizing NTDB data demonstrates that the co-variates used to adjust for patient mix while predicting mortality vary widely and lack standardization. Although certain co-variates such as the ISS (or any measure of anatomical injury severity), age and gender are commonly used, even these measures, which are well-known to impact trauma survival, are not adjusted for by a significant proportion of studies. Similarly, physiologic derangement on ED arrival, which is another well-known predictor of trauma mortality, was only considered by approximately 70% of the studies reviewed. Co-variates such as race, insurance status, mechanism and type of injury, which are also known to impact survival, have been used even less frequently. It is unclear why investigators choose to use only some co-variates in their analysis despite the availability of additional equally important variables in the NTDB. It is important to address this concern in future studies, as inclusion of additional variables known to impact trauma outcomes into statistical models will enhance their ability to deliver accurate results.
It should be acknowledged that researchers must balance the advantages of incorporating additional co-variates into outcome prediction models with the complexities of handling missing data in the NTDB. Any of the statistical techniques commonly used by the authors in this review (ie exclusion of missing data, subgroup sensitivity analyses, etc.) may be appropriate for handling missing data; however, it is critical that researchers appropriately adjust for confounders known to impact mortality. Most investigators exclude patients with missing data from their analysis, leading to considerable loss of sample size.54
This approach can introduce bias and impact the sensitivity and confidence with which differences can truly be ascertained. An analysis by Roudsari et al.138
showed that missing data in the NTDB does not appear to be “completely missing at random,” which raises the possibility of bias with the use of the complete case analysis approach. One method to handle missing information in the NTDB that has recently been successfully utilized in trauma research is the imputation of missing data.36
At least a handful of authors have demonstrated the feasibility of this technique in the NTDB including Oyetunji et al.132
and O'Reilly et al.140
In particular, multiple imputation has been used and validated for missing physiologic data in the NTDB141,142
and holds promise as an excellent technique to decrease the impact of missing data on studies that utilize the NTDB.
Other limitations and challenges with the use of the NTDB have been acknowledged by the American College of Surgeons Committee on Trauma (ACSCOT) and authors in an upfront manner in their reports. The reporting of data to the NTDB is a voluntary undertaking by the trauma centers, making it a convenience sample that is not entirely representative of all trauma centers in the U.S.23,24
This, in turn, creates the potential for selection bias.16
However, the enormous size and cumulative presentation of the sample partially compensates for this limitation. “Unmeasured bias” with the use of NTDB may stem from unknown or residual confounders,8
poor charting, and poor data abstraction.19
The “invisible trauma patient”, who is treated in the emergency room and discharged, is not documented in the NTDB.143
Similarly, patients who died at the scene are not captured by the NTDB.19
Misclassification of variables is possible due to inaccuracy in diagnostic coding of injuries.28
The exact indication for various interventions or diagnostic studies is not mentioned in the NTDB; therefore, it has to be deduced from available information. Furthermore, the potential under-reporting of complications by hospitals submitting data to the NTDB is a significant limitation of the database. Results of studies utilizing the NTDB should be interpreted with the caveat that causal relationships can't be determined with absolute certitude.35
The trends of the associations can be appreciated, but remarks about the precise causes underlying such relationships based on the NTDB data should be made with great care.63
The NTDB represents an exciting chapter in the development of trauma registries. It is a powerful tool that is being used by hundreds of researchers who have published findings in almost every important trauma and surgical journal, as well as non-surgical journals. The aim of this review was to present an organized synthesis of available information about the utilization of the NTDB in the past few years with particular reference to its impact on the study of trauma outcomes. Despite some limitations, the NTDB remains the largest available and most complete national trauma registry61
and has become an invaluable and indispensible data resource for trauma outcomes research. It has not only advanced our understanding of the myriad factors that shape the structure of trauma care in the contemporary medical landscape, but has also provided insight into injury quantification, quality of trauma care, clinical outcomes, burden of injury and disparities prevalent in trauma systems.144
The use of the NTDB for outcomes research is expected to increase in the coming years. Lack of standardization in practices, however, may be a limitation for researchers as they attempt to tap the full potential of the NTDB. Best practices for analyzing data are needed to further improve the quality, reliability and interpretability of research from the NTDB and enhance its impact.