|Home | About | Journals | Submit | Contact Us | Français|
To identify the optimal target of a future intervention to improve physician decision making in trauma triage.
A comparison of the incremental cost-effectiveness ratios (ICERs) of current practice versus hypothetical interventions targeting either physicians’ decisional thresholds (attitudes towards transferring patients to trauma centers) or perceptual sensitivity (ability to identify patients who meet guidelines for transfer).
Taking the societal perspective, we constructed a Markov decision model. We drew estimates of triage patterns, mortality, utilities, and costs from the literature. We assumed that an intervention to change decisional threshold would reduce under-triage but also increase over-triage more than an intervention to change perceptual sensitivity. We performed a series of one-way sensitivity analyses, and studied the most influential variables in a Monte Carlo simulation.
The ICER of an intervention to change perceptual sensitivity was $62,799/ quality-adjusted life years (QALY)-gained compared with current practice. The ICER of an intervention to change decisional threshold was $104,975/QALY-gained compared with an intervention to change perceptual sensitivity. These findings were most sensitive to the relative cost of hospitalizing patients with moderate-severe injuries and their relative risk of dying at non-trauma centers. In probabilistic sensitivity analyses, at a willingness-to-pay threshold of $100,000/QALY-gained, there was a 62% likelihood that an intervention to change perceptual sensitivity was the most cost-effective alternative.
Even a minor investment in changing decision making in trauma triage could greatly improve the quality of care provided. The optimal intervention depends on the characteristics of the individual trauma systems.
Over 30 million people receive trauma care every year in the United States,1 consuming approximately $400 billion annually.2 To optimize the allocation of resources, the American College of Surgeons advocates the regionalization of patients.3 Regionalization requires that physicians at non-trauma centers transfer patients with moderate to severe injuries to specialty trauma centers while admitting those with minor injuries.4 Despite an unprecedented effort at the federal, state, and regional level to increase adherence to clinical practice guidelines supporting regionalization, only one-third of patients with moderate to severe injuries taken initially to non-trauma centers are actually transferred to trauma centers.5 Additional quality improvement interventions are needed.
Physician adherence to clinical practice triage guidelines can be thought of as the product of both decisional thresholds and perceptual sensitivity.6 Decisional thresholds reflect physicians’ preferences for transferring or keeping patients based on norms, attitudes, and incentives. Perceptual sensitivity reflects physicians’ ability to discriminate between patients who do and do not meet clinical practice guidelines for transfer based on their knowledge of the guidelines and their intuitive judgments (heuristics).7 Decisional thresholds and perceptual sensitivity are distinct cognitive processes. Therefore, modification requires targeted interventions, which have different costs and benefits. Current efforts have included both top-down approaches aimed at shifting physicians’ decisional thresholds through a mixture of regulatory and organizational incentives,4,8 and a bottom-up approach aimed at increasing physician perceptual sensitivity through Advanced Trauma Life Support (ATLS) training and certification.
To set priorities for quality improvement in trauma triage, we need to know whether it is more cost-effective to invest additional resources in changing physicians’ decisional thresholds or their perceptual sensitivity. The objective of our study was to compare the cost-effectiveness of current practice with two distinct hypothetical interventions to improve physician adherence to clinical practice guidelines: one that shifted decisional thresholds versus one that increased perceptual sensitivity
Signal detection theory, a well-established behavioral science method, identifies two separate aspects of every dichotomous decision: decisional threshold and perceptual sensitivity (Figure 1). Interventions designed to impact these distinct cognitive processes require different designs, and have different anticipated costs and benefits. An intervention to change decisional thresholds, such as a pay-for-performance initiative, would increase sensitivity. Although relatively inexpensive and easy to implement, shifting physicians’ decisional thresholds to make them more willing to transfer patients with moderate to severe injuries would reduce system efficiency, by increasing the number of patients with minor injuries transferred to trauma centers. On the other hand, an intervention to change perceptual sensitivity, such as a program to modify physician heuristics, would increase specificity. This kind of intervention would be relatively more expensive and difficult to implement, but would increase system efficiency.6,7,9
We constructed a decision model (Figure 2) to compare the clinical and economic outcomes of a 40-year old trauma patient taken initially to a non-trauma center under current conditions of national compliance with trauma triage guidelines compared with compliance resulting from two alternative hypothetical interventions – one targeting physicians’ decisional threshold and one targeting their perceptual sensitivity. As recommended by the Panel on Cost-Effectiveness in Health and Medicine, we took the societal perspective when we considered direct medical and nonmedical costs of obtaining care, and used a one month cycle length and a lifetime time horizon in the analysis. We discounted future costs and benefits at an annual rate of 3%.10 We constructed our model using TreeAge Pro 2009 software (TreeAge, Williamstown, MA).
All clinical parameters, utilities, and costs used in the base case and sensitivity analyses are summarized in Table 1 and described below. We drew probabilities of minor and moderate to severe trauma, of under- and over-triage, and of mortality conditional upon triage from the literature.11–13 In a retrospective analysis of Pennsylvania discharge data from 2001 to 2005, we found that 25% of patients presenting to non-trauma centers after trauma had a moderate to severe injury, defined as either an Injury Severity Score (ISS) > 15 or as an injury categorized by the ACS-COT as “life-threatening” or “critical” (e.g. open long bone fracture). 30% of these patients were transferred to a trauma center. Of patients with minor injuries, defined as the absence of a moderate to severe injury, 15% were transferred to a trauma center.5
We drew age-adjusted probabilities of all-cause mortality from national health statistics.14 We assumed that a minor injury did not affect that probability, but that a moderate to severe injury increased the probability of dying in the first year after the traumatic event.13 We assumed that up to 40% of patients with minor injury and 100% of patients with a moderate to severe injury experienced some period of disability, defined as an inability to return to work. If a patient remained disabled for a full year we assumed that they remained disabled for the rest of their lifetime.
We calculated the number of quality-adjusted life-years (QALYs) associated with treatment of patients at trauma centers and non-trauma centers by multiplying the time spent in each health state and its associated utility.
We equated time spent acutely injured to the number of days spent hospitalized. We used a computer program to calculate an ISS based on trauma International Classification of Diseases, 9th revision – Clinical Modification (ICD9-CM) codes.15 We estimated the mean number of days hospitalized after minor and moderate to severe injuries by using national average length of stay data for each primary ICD9-CM code.16 Based on our analysis of Pennsylvania discharge data, we estimated that LOS would be equal to the average at non-trauma centers, and 0.9 times the average at trauma centers for patients with minor injuries. We estimated that LOS would be 0.9 times the average at non-trauma centers, and 1.4 times the average at trauma centers for patients with moderate to severe injuries. We estimated time spent disabled based on published estimates of rates of return to work after moderate to severe injuries and minor injuries.17–25 Similarly, we estimated utilities associated with health states from published data on functional outcomes after moderate to severe and minor trauma.26,27
Since treatment at trauma centers does not affect functional outcomes, we assumed that treatment at trauma centers and non-trauma centers would not affect the utilities associated with health states.28, 29
Costs for the treatment of minor and moderate to severe injuries included hospitalization, acute disability (inpatient and outpatient rehabilitation, follow-up care, and informal care), chronic disability (inpatient care, follow up care, and informal care), and long-term health care costs. For moderate to severe injuries, we estimated the cost of hospitalizations, acute disability and chronic disability from the literature.30 For minor injuries, we used 2008 national cost data for each ICD9-CM code to identify the average cost of hospitalizations.16 We estimated that care at a non-trauma center would be 0.75 times the national average for each injury, and that care at a trauma center would be 1.3 times the national average.30 We assumed that acute disability costs included ten days in a rehabilitation facility or nursing home, three outpatient physical therapy sessions, and two physician clinic visits. We assumed that chronic disability costs would include four outpatient physical therapy sessions and one physician clinic visit a month.18 We used the 2008 Medicare physician fee schedule to estimate the cost per day for a stay in a rehabilitation facility, the cost per physical therapy session, and the cost per clinic visit.31 We used the 2008 Centers for Medicare and Medicaid estimates of age-adjusted personal health care spending per capita to adjust for the health care costs of patients one year after injury.32
We made three assumptions about the hypothetical interventions. First, based on the literature on quality improvement interventions, we assumed that the decisional threshold intervention would increase the transfer of patients with moderate to severe injuries by 40%.33,34 We estimated that the minimum and maximum improvement in transfer rates were 10% and 65%.35,36 However, changing decisional thresholds would increase tolerance for false positive decisions (i.e. the transfer of patients with minor injuries). We assumed the rate of transfer of patients with minor injuries would be ½ the increase in the rate of transfer of patients with moderate to severe injuries.
Second, we assumed that the complexity of changing individual physician perceptual sensitivity would make the intervention ½ as likely to change rates of the transfer of patients with moderate to severe injuries as an intervention that targeted decisional thresholds.37 However, this intervention would not affect the transfer of patients with minor injuries.
Third, we assumed a fixed budget for implementation of a quality improvement initiative. Based on practice patterns in Pennsylvania, physicians at non-trauma centers treat between five to ten trauma patients a year,5 and spend approximately $200 per year to participate in the ATLS (approximately $40 per patient). We arbitrarily chose an additional $40 per patient as the budget, which would be equivalent to doubling the cost of participating in ATLS to fund the intervention.
We compared the clinical and economic consequences of an intervention that modified physicians’ decision threshold with one that modified physicians’ perceptual sensitivity with current practice patterns by using the incremental cost-effectiveness ratio (ICER), defined as the extra cost imposed by the intervention divided by its extra clinical benefit measured in quality adjusted life years (QALYs). We performed a series of one-way analyses for all variables to assess the effect of varying the baseline estimates on cost-effectiveness. To explore the relationship between the relative cost of the interventions and their effect on under- and over-triage, we varied three parameters (the relative cost of the interventions, the relative effectiveness of the intervention at reducing under-triage, and the relative cost of hospitalization at a trauma center compared with a non-trauma center) in a further sensitivity analysis.
We studied variables that influenced the ICER by more than 10% in the one-way analyses in a multi-variable sensitivity analysis using Monte Carlo simulation. We varied these parameters simultaneously over triangular probability distributions in each of 10,000 Monte Carlo iterations. We computed the percentage of Monte Carlo iterations for which a given strategy was more cost-effective for willingness-to-pay ceilings of $100,000/QALY, a commonly used threshold for cost-effectiveness.38,39
Both a hypothetical intervention to change physicians’ decisional threshold and to change physicians’ perceptual sensitivity would increase costs and improve outcomes compared with the status quo. In the base case, the intervention to change perceptual sensitivity resulted in an additional cost of $1,500 per patient and saved 0.024 QALYs compared with the status quo ($62,799/QALY). The intervention to change the decisional threshold resulted in an additional cost of $2,500 per patient and saved an additional 0.024 QALYs compared with the intervention to change perceptual sensitivity (ICER $104,975/QALY). The cost-effectiveness of an intervention to change perceptual sensitivity remained below the $100,000/QALY threshold as long as the intervention cost less than $800 per patient.
As shown in Table 2, in one-way sensitivity analyses, variation of ten parameters changed the ICER of the intervention to change perceptual sensitivity compared with current practice patterns by more than 10%. The ICER was most sensitive to the age of the patient, the severity of the injury, the relative cost of hospitalization after a moderate to severe injury at a non-trauma center compared to at a trauma center, and the relative risk of dying at a non-trauma center compared to at a trauma center after a moderate to severe injury.
In a three-way sensitivity analyses, we varied the relative effectiveness of the interventions at reducing under-triage, cost of the interventions, and the cost of hospitalization at a trauma center versus a non-trauma center (the primary driver of the cost associated with over-triage). As shown in Figure 3, in the base case, the intervention to change perceptual sensitivity remained the most cost-effective strategy, with an ICER less than $100,000 per QALY gained, as long as the intervention was at least 40% as effective as the intervention to change the decisional threshold. If the intervention to change perceptual sensitivity was at least 40% as effective as the intervention to change the decisional threshold, it could cost up to three times as much and would remain the most cost-effective intervention. As the effectiveness of the intervention to change perceptual sensitivity increased to equal that of the intervention to change decisional threshold, the cost could be 25 times as much as the intervention to change decisional threshold and it would remain the most cost-effective strategy.
Probability-based sensitivity analyses indicated that the intervention to change perceptual sensitivity was cost-effective in 62% of simulations, using the $100,000 willingness-to-pay threshold.
In this cost-effectiveness analysis, we compared current practice in trauma triage with hypothetical interventions that would modify physician decision making. We found that even an expensive intervention to change the cognitive biases of individual physicians would be economically reasonable compared with either allowing current practice to continue or another top-down intervention to change physicians’ decisional thresholds. However, we also found that given different conditions, such as trauma centers that could deliver less expensive inpatient care, an intervention to change decisional thresholds might be the better alternative. Therefore, the intervention choice should be customized to the characteristics of the trauma system.
The clinical uncertainty involved in trauma triage results in two different types of errors in physician decision making: under-triage (the admission of patients with moderate to severe injuries to non-trauma centers) and over-triage (the transfer of patients with minor injuries to trauma centers). Under-triage denies patients the benefits of care at trauma centers, while over-triage increases the costs of care without a commensurate improvement in outcomes. MacKenzie et al. have shown that care provided at trauma centers to patients with moderate to severe injuries is cost-effective compared with care provided at non-trauma centers.30 However, this analysis does not account for the degree to which rates of over-triage affect the costs and clinical outcomes of regionalization. Moreover, since quality improvement interventions have different effects on under- and over-triage, the optimal strategy for improving regionalization is similarly unclear.
As a precursor to developing a quality improvement program in trauma triage, we performed a thought experiment to identify the best target for intervention at the physician level. We used principles of behavioral science to categorize different cognitive processes that might affect physician compliance with clinical practice guidelines,6,7 and limited the analysis to interventions that targeted a single cognitive process. Our experiment produced several key observations. First, we found that an intervention that could change the perceptual sensitivity of individual physicians, such as a program to recalibrate physicians’ cognitive biases, would be cost-effective compared to either current practice or an intervention to change decisional thresholds. Improving physicians’ ability to discriminate between patients who do and do not meet the reference standard for transfer would cost $62,799 per QALY gained. By comparison, an intervention to improve influenza vaccination rates among the elderly costs $49,000 per QALY gained; placing automatic cardiac debrillators in public places costs $57,000 per QALY gained, and dialysis costs $129,000 per QALY gained.30
Second, we found that such an intervention could cost up to $800 per patient and would meet societal standards for cost-effective care. To put that number into context, non-trauma centers in Pennsylvania hospitalize approximately 25,000 trauma patients each year.5 Consequently, the state of Pennsylvania could spend up to $20 million helping physicians at non-trauma centers adhere to clinical practice guidelines and still provide cost-effective trauma care. Even a moderate additional investment in quality improvement could therefore have a profound impact on public health.
Third, we found that the most cost-effective target for a quality improvement intervention varied based on the relative effectiveness and costs of care at trauma centers compared with non-trauma centers. For example, if trauma centers could provide less expensive care, an intervention to change decisional thresholds would surpass the alternatives. As the costs of over-triage decrease, the relative benefit associated with reducing under-triage increases. In other words, the need for nuanced discrimination among patients diminishes. Under these conditions, we speculate that excluding the physician from the decision making process entirely might offer the best method of improving regionalization in trauma. For example, a mid-level provider could possibly use a computerized decision support tool, which screens patients based on a set of clinical predictors, to triage patients. Consequently, identifying the best quality improvement intervention for a region depends on the characteristics of the individual trauma system, precluding a single national or state-wide strategy to reduce variability in trauma triage.
This paper has several limitations. First, estimates of triage patterns derive from an analysis of Pennsylvania discharge data. These values may not be generalizable to other regions of the country. However, varying these parameters in sensitivity analyses produced similar results, suggesting the robustness of our conclusions. Second, our assessment of costs associated with regionalization did not include the effect of increasing the number of patients at trauma centers. Given that hospitals routinely operate close to system capacity, raising the census may have implications for patient outcomes. Consequently, the implications of a large-scale redistribution of patients would require a demonstration project to characterize adequately. Third, we tested hypothetical instead of actual interventions. Although a series of quality improvement interventions exist in trauma, their effect on physician decision making is unknown. Sketching the problem broadly as barriers to regionalization eliminated some of the uncertainty involved in our thought experiment. Finally, we did not account for all the potential consequences of injury, including costs that might arise from disabilities such as post-traumatic stress disorder. Based on current evidence that trauma centers do not influence functional outcomes after trauma, we hypothesize that further specification of our model would not have significantly altered our analysis.
In conclusion, investing in an intervention to improve adherence to clinical practice guidelines for the triage of trauma patients would yield significant value even if the intervention were expensive. Our cost-effectiveness analysis highlights that reducing under-triage may require a more nuanced and context-dependent approach than has been previously recognized.
Investing in an intervention to improve the decision making of individual physicians in trauma triage would yield significant value even if the intervention were expensive.
Source of support: This work was supported by grant 1KL2RR024154 from the National Center for Research Resources (NCRR), a component of the National Institutes of Health (NIH) and NIH Roadmap for Medical Research (Dr. Mohan). Its contents are solely the responsibility of the authors and do not necessarily represent the official view of NCRR or NIH. Information on NCRR is available at http://www.ncrr.nih.gov/. Information on Re-engineering the Clinical Research Enterprise can be obtained from http://nihroadmap.nih.gov/clinicalresearch/overview-translational.asp.
This is a pre-publication version of the manuscript that has been accepted for publication in The American Journal of Managed Care (AJMC). This version does not include post-acceptance editing and formatting. The editors and publisher of AJMC are not responsible for the content or presentation of the prepublication of the manuscript or any version that a third party derives from it. Readers who wish to access the definitive published version of this manuscript and any ancillary material related to it (e.g. correspondence, corrections, editorials, etc) should go to www.ajmc.com or to the print issue in which the article appears. Those who cite this manuscript should cite the published version, as it is the official version of record.