|Home | About | Journals | Submit | Contact Us | Français|
A rigorous statistical analysis of the retrospective estimation of radiation dose received using time to emesis and its uncertainty is provided based on 108 observations associated with accidents with significant exposures to ionizing radiation in the period 1956–2001. The standard error, confidence interval, specificity and sensitivity, and Receiver Operating Characteristic (ROC) curve are used to characterize the uncertainty of the dose prediction. The relative error of the dose prediction using time to emesis data is about 200%. Consequently, if D is the dose assessment, the 95% confidence interval is approximately (D/4, 4D). Our assessment of the precision is applied to computation of the probabilities in triage medical management in the case of a nuclear terrorism event. We also note several factors that indicate that there are additional problems in the use of time to emesis for triage, including a lack of consideration of individuals that do not vomit, differences between the conditions under which the data were obtained and the conditions under which they are likely to be used, and the potential for the incidence of vomiting to be altered by factors unrelated to radiation exposure such as psychogenic factors and the use of emetic agents. In summary, while time to emesis is a rapid and inexpensive method for estimating the radiation dose, it should be used with caution because it is imprecise and may lead to a very high false positive rate. More reliable methods for after-the-fact assessment of radiation dose are needed to complement the use of time to emesis.
There is an urgent need to provide guidance to first responders and managers in the aftermath of an event in which large numbers of people may fear that they have been exposed to high doses of ionizing radiation, such as in the case of nuclear terrorism. In view of the large numbers of people that may be involved and the potential disruption from such incidents, it is essential to use methods that can provide dose estimates immediately in the field, without the need to send samples to a remote site and/or to wait for many hours or days to get the information. It has been suggested that time to emesis correlates with dose, and therefore it has been a prominent component in guidance documents for initial responses to such an event. The end point is clear, and no technology or special skills are required to obtain the data. The key question, of course, is what the accuracy and reliability of this end point is for guiding effective triage. For example, what is the uncertainty in dose estimation for an individual who has vomited within 4 h of the exposure, and for discrimination based on a given threshold dose, what would be the expected false positive and false negative rates?
While data have been published previously that show the relationship between radiation dose and time to emesis, there has not been a rigorous analysis of the precision of this method, including computation of the standard error of the dose prediction, specificity and sensitivity. This paper aims to provide a quantitative assessment of this method along with its precision expressed through the standard error of the dose prediction, confidence interval, and the Receiver Operating Characteristic (ROC) curve (1). In our analyses we pay particular attention to the calculation of the uncertainty in the prediction of dose based on the time to emesis, with a comprehensive discussion of the validity of the underlying statistical assumptions.
Three analyses of dose estimation using time to emesis are presented:
The paper is organized as follows: First we describe the data and give an overview of the methods used in previous studies. We suggest that it will be useful to use the reverse relationship between dose and time to emesis on the log scale for statistical reasons. We assess the uncertainty of the dose reconstruction, calculating the standard error of prediction and confidence interval. We compute the interval probability for triage medical decision-making and assess how properly analyzed data would predict the proportion of the population involved in the case of terrorism using radiation. The specificity and sensitivity and the ROC curve of the time-to-emesis approach are estimated.
Several authors have investigated the relationship between time to emesis and radiation dose received. An excellent summary of the practical application of time to emesis for triage in the event of nuclear terrorism was recently provided in ref. (2).
These data have been used to provide practical guidelines for triage. A good example of the use of the data is provided by Goans and Waselenko (3), who concluded that when the time to emesis is <2 h, the effective whole-body radiation dose is >3 Gy, and that vomiting <1 h after exposure indicates a dose of >4 Gy. Another relevant use of the data is in Table VIII of the IAEA Safety Report Series no. 2 (4), which concludes that if a person vomits 2 h after the event, he/she received ≤2 Gy, and that if the emesis occurred in the period 1–2 h after the event, he/she received 2–4 Gy. There also are online resources that indicate that one can compute the estimated whole-body radiation dose based on time to vomiting, for example, http://www. remm.nlm.gov/ars_wbd.htm. There are several government documents, such as at the website of Department of Homeland Security [see ref. (5), p. 23], where time to vomiting is included as a quick and rough assessment of the radiation dose received after a radiation terrorism event.
Close examination of these recommendations indicates some inconsistencies, with different sources suggesting different estimates of dose with the same time to emesis. These recommendations also seem incomplete, because they do not provide estimates of the error of the estimate or the expected false positive rate.
In the present work, the evaluation of the effectiveness of time to emesis as an estimator of absorbed dose is based on the data set that has been used for most of the calculations. These data have been collected over the years after radiation accidents in the U.S. and the USSR in the period 1945–2001; they consist of 108 individual observations of time to emesis and radiation dose received (see Fig. 1). The majority of the data (about 80%) are from the Chernobyl accident (USSR) in 1986. The rest of the data are from the REAC/TS Radiation Accident Registry (U.S.). These data were initially presented at the fourth REAC/TS conference on the medical basis for radiation-accident preparedness in 2002 and published in ref. (6). We have reassessed the data, providing a more sophisticated statistical analysis with an emphasis on the uncertainty of the dose reconstruction. In Fig. 1, we plot the data with time to emesis in hours as a function of the radiation dose measured in Gy. It should be noted that the distribution of doses received in these data does not resemble the expected distribution of doses found during triage after a terrorist or nuclear event, where the number of subjects receiving negligibly low doses is expected to dominate the population.
In previous studies, two types of analysis have been done. Anno et al. (7) and Anno (8) used a linear least-squares method to estimate the relationship between time to emesis and dose on the log scale, while Goans and Waselenlo (3) applied a nonlinear least-squares fit using a power function of the form Hour = A*DoseB. Although the parameter values for A and B for both methods were close, the standard errors were quite different.
For time to emesis to be used to estimate absorbed dose in a triage setting, either a mathematical relationship between the time and dose or a table of related values is needed. The precision of such estimates, in the form of a confidence interval, should also be provided. This information can be derived from the data using standard regression analysis.
Rather than following the causal relationship between exposure and emesis and considering time to emesis as a function of absorbed dose, we reverse this relationship between time to emesis and radiation dose for the following statistical and practical reasons.
First, the main assumption of a standard regression analysis is that the dependent variable, y, contains measurement errors while the independent, or explanatory, variable, x, is free of measurement error (9). But while time to emesis can be recorded fairly precisely, the radiation dose received is typically subject to substantial measurement error. Hence considering the dose as a function of time to emesis complies better with the theory of the least-squares estimation. Although the methods of parameter estimation with measurement errors in x exist, they are typically significantly more complicated and require knowledge of the distribution of x; e.g., see ref. (10). It is well known that when x is measured with error, the regression slope estimated by least squares has a systematic negative bias (attenuated) (10).
Second, it is convenient to do the prediction when radiation dose is expressed as a function of time to emesis. Alternatively, the method of inverse regression could be applied, but this would involve substantial complications to use statistical inference (9).
Third, as have some previous authors, we consider the relationship between dose and time to emesis on a log scale. This transformation has three advantages: First, the log transformation eliminates the skewsness and the heteroscedasticity of the data, making the distribution much closer to a Gaussian distribution that complies with a standard assumption of the regression analysis (9). Second, the nonlinear power function transforms into a linear function, which simplifies the analysis and therefore improves the precision. Third, the log transformation implies that the measurement error is relative; i.e., larger measurement errors occur for larger doses of radiation, which seems adequate in our setting (12). The log transformation implies a relative error; consequently, our estimated uncertainty is expressed as the percentage of the radiation dose.
In short, reversing the casual relationship between time to emesis and dose avoids statistical complications associated with the assessment of uncertainty and simplifies computation of the dose recovery. Consequently, in this paper we study the relationship between dose and time to emesis by linear regression, yi = α + βxi + εi, where α is the intercept and β is the slope; here yi = log10Dosei and xi = log10Houri for i = 1, 2, …, n. The unobserved term εi represents the regression error with zero mean and variance σ2.
The plot of 108 points with the radiation dose as a function of time to emesis in hours on the log scale with 95% confidence band is presented in Fig. 2. We can approximate the relationship between log(dose) and log(time) by a linear function, which on the original linear scale takes the form
Specifically, the least-squares estimates of α and β were a = log10 4.5 and b = -0.57, respectively, with the estimate of σ2 as the regression variance S2 = 0.282. Equation (1) can be used to compute the dose in Gy; the time to emesis is given in hours. For example, as follows from this formula, an individual is calculated to have received 3 Gy if he or she vomits 2 h after a radiation exposure. A similar formula using the natural log has recently been derived in ref. (13). We provide the uncertainty analysis based on the confidence interval of the reconstructed dose.
Given x, the standard error of the predicted value y is computed using a well-known formula [ref. (11), p. 80]:
A comment regarding the term 1 under the square root should be made. In fact, the formula for the standard error of a predicted value usually does not have 1; e.g., see ref. (11), p. 80. Without 1 the formula would give the error of the regression line (mean), not of the individual value y. Since we are interested in predicting the dose for a given time to emesis, Eq. (2) should be used.
The two-sided 95% confidence band (interval) for dose on the log scale is computed using the standard formula as
similar to the rule of two sigma. Then the exact 95% confidence interval for the reconstructed dose is 10 to the power of the lower and upper bounds.
Below we obtain a simpler formula for the confidence interval, using the relative dose prediction defined as
Using the fact that for a normal distribution the probability to be within one standard deviation around the mean is 65%, if the dose prediction is , the true dose is within the interval (), with 65% confidence. Similarly, we can compute the dose prediction using the 95% confidence interval: With 95% confidence the true dose is within the interval (). This result, summarized in Table 1, indicates that while time to emesis is easy to use as a diagnostic measure, it produces an imprecise assessment of the radiation dose received: The upper limit of the 95% confidence interval is an order of magnitude greater that the lower limit.
In Table 1, we present the results of dose prediction with the two-sided 95% confidence interval and the respective distributions of doses received given as a percentage of the included population. While the first part of this table can be used for an individual dose assessment, the second part may be useful for analysis for triage for medical decision-making.
The percentage of the population involved is computed as follows. Given the value of time to emesis, H, the log10 dose has a Gaussian distribution with mean a + b × log10H and variance given by Eq. (2). Having this distribution, we may compute various probabilities that are expressed in terms of the percentage of the population having the dose in the specified interval.
For example, as follows from this table, if vomiting occurred 1.5 h after an accident, we infer with 95% confidence that the person received a dose in the range from 1 Gy to 12.8 Gy. As follows from the second part of the table, among people who vomited 1.5 h after the accident, 19% received a dose less than 2 Gy, 75% received a dose from 2 Gy to 10 Gy, and 6% received a dose higher than 10 Gy. As indicated by each of these results, and by those in the remainder of the table, the variability in the absorbed dose associated with a specific time to emesis is very large.
Another way to assess the quality of the dose prediction is through sensitivity and specificity, represented graphically by a Receiver Operating Characteristic (ROC) curve (1). An advantage of this method is that no assumption is involved in the modeling of the relationship between dose and time to emesis, as in the previous section. The ROC curve is a useful probabilistic representation of the discrimination problem, based on the raw data.
In the present context, time to emesis is used to discriminate between individuals who received less than or greater than a chosen threshold dose of radiation. Generally, any threshold value can be used; in this paper we chose the threshold dose of 2 Gy as an example, because this dose approximately discriminates between nonlethal exposures and those that require treatment.
By definition, the ROC curve is a plot of sensitivity as a function of (1 — specificity), where sensitivity is computed as the rate of the correct identification of individuals who received a dose greater than the threshold dose and specificity as the rate of a correct identification of individuals who received a dose less than the threshold. Figure 3 is an illustration of this approach for a threshold dose of 2 Gy and a time to emesis of 1 h. Using a time to emesis of 1 h to discriminate between persons receiving greater than or less than 2 Gy produces a specificity of 96% but a sensitivity of only 29%. Accordingly, 96% of those receiving <2 Gy have been properly identified, but only 29% of those who received >2 Gy were identified. If instead, a time to emesis of 4 h is applied [as approximately indicated by Eq. (1)], 86% of those with doses >2 Gy are identified but are accompanied by 46% of the people who received <2 Gy.
The theoretical ROC curve is a smooth approximation of the empirical ROC curve. As was suggested in the previous section, the distribution of time to emesis is normal on the log scale, so that the mean and standard deviation were computed in terms of log time to emesis. Let μ0 and σ0 denote the mean and standard deviation of the log time to emesis in the group of individuals who received less than 2 Gy radiation, and respectively μ1 and σ1 for the group of individuals who received >2 Gy (note that μ0 > μ1). Then the sensitivity and the specificity achieved using a time to emesis t to discriminate between the populations (> or <2 Gy) can be expressed by the cumulative normal distribution function Φ with the following probabilities:
Plotting sensitivity as a function of false positive probability over the range of discriminant values t yields the ROC curve. In reality, we do not know means and standard deviations, so we replace them with sample estimates.
The two groups of individuals with time to emesis corresponding to those who received <2 Gy and >2 Gy is presented graphically by two Gaussian densities in Fig. 4. As the discriminant value t (time to emesis) is increased, we obtain a series of values for sensitivity and specificity; plotting the former as a function of 100 — specificity results in the ROC curve. This theoretical ROC curve with the 95% confidence band for the threshold dose of 2 Gy is presented in Fig. 5. The overall probability of discrimination between the dose populations is 0.83 with 95% CI 0.73, 0.90. The false positive rate is shown at the lower x axis and the corresponding time to emesis is shown at the upper x axis. The upper left side of the ROC curve is the most important because we want to minimize the false positive rate while correctly identifying individuals who received a dose higher than 2 Gy. As follows from Fig. 5, if the false positive rate is to be held to less than 15%, the maximum sensitivity is 60%. On the other hand, if one wants to raise the sensitivity to 80%, a higher rate of false positives will also need to be accepted.
The area under the ROC curve (AUROC) estimates the overall probability of discrimination of the two populations using data on time to emesis. The details of the confidence interval calculation can be found in ref. (14), and the interpretation of the AUROC is found in ref. (15).
In Table 2 we compute the sensitivity, specificity and the false positive rate for a cut-off dose of 2 Gy, with several thresholds for time to emesis. For example, if the decision that someone has received more than 2 Gy is based on vomiting within 2 h after the attack, as recommended by the International Atomic Energy Agency, only two-thirds of the individuals receiving >2 Gy will be correctly identified and 14% will be falsely identified as receiving more than 2 Gy. In absolute terms, assuming that 10 of 1000 people being evaluated have been exposed to >2 Gy, this analysis indicates that seven of the 10 subjects will be correctly referred for treatment and 140 people who received doses <2 Gy will also be referred. In summary, while time to emesis can be used as a rapid and inexpensive method of dose assessment, it leads to a substantial false positive rate when the discrimination time is set so that sensitivity is reasonably high.
Time to emesis was used by previous authors to reconstruct the radiation dose that was received. In this work we emphasize the uncertainty associated with this measure. We have provided quantitative estimates of the quality of the dose prediction through the relative standard error, confidence interval and a ROC curve. As shown in our analysis, the dose assessment based on time to emesis is imprecise, with a standard error of approximately 200% and a 95% confidence interval of , where is the dose estimate. Our accuracy assessment is based on the normal/Gaussian distribution. While this assumption apparently does not apply to the original data due to skewness and heteroscedasticity, the log transformation alleviates this negative feature of the data and makes the normal assumption much more appropriate.
Computation of sensitivity and specificity is an important aspect of dose estimation, especially when considering use of the technique for triage, or screening, of potentially irradiated populations. For example, if one wants to detect 66% of the population who received 2 Gy or more, the time-to-emesis discriminant value should be less than 2 h. This criterion comes with the price of including 14% of the population who received less than 2 Gy but will have vomited within 2 h. If sensitivity is chosen to be 86%, then 4 h to emesis should be used, but this would come with a 46% false positive rate. For both cases, the potential number of false positive detections may swamp the medical care infrastructure, effectively rendering the technique impractical.
These analyses are based on the assumption that the available data are complete and are applicable to the situation where they are to be applied, i.e., to an incident involving large numbers of people such as the detonation of an improvised nuclear device with acute exposures from prompt radiation or transient heavy fallout. There are, however, several additional potential sources of uncertainty that should be considered with the use of these data for dose reconstruction based on time to emesis. All of these additional considerations increase the uncertainties calculated in the previous sections.
A substantial limitation of our analysis is that while it uses the only extensive data set that is available, the quality of the data is quite suboptimal and may be a source of the large standard error for dose prediction. More data on time to emesis are needed to make our assessment more precise. The calculations of uncertainties in the estimates of dose from time to vomiting have an implied assumption that there is no measurement error of the onset of time of vomiting in the data sets, and therefore the observed uncertainties reflect the biological variability in time to emesis. These assumptions are certainly not valid for the exposures at Chernobyl, which comprise the majority of the data points (84 of 108). The doses at Chernobyl are based on a combination of measurements and reconstruction, and it is widely recognized that these have a large degree of uncertainty. Also, the doses were delivered at varying dose rates over varying periods, which modifies the biological consequences of the radiation exposure and probably affects the tendency to vomit. The uncertainties in the data on time of onset of vomiting also are unknown but are probably fairly large, in view of the uncertainties that are acknowledged on other aspects of the personal records of the individuals involved in the accident.
The situations from which these data are obtained have several aspects that are likely to be quite different from the situations in which they are to be applied. The majority of the data points are from Chernobyl, where the individuals were first responders and early clean-up workers with minimal training in the hazards of the event. They generally received their doses over a fairly long period (about 24 h in many cases), and many of the individuals also had substantial radiation doses to the skin from particulates that were not removed promptly; both of these factors are likely to have altered the physiological responses to the radiation exposure, including the tendency to vomit. There were also simultaneous exposures to other environmental toxins generated in the accident. These potentially confounding factors were not as uniformly present in the other accidents, but the conditions may also have varied from the types of applications for which the guidelines are likely to be applied. While the data from the 108 individuals may be the best that we have, one might argue that this does not make them applicable for the very different situation in which they are proposed to be used as part of the decision-making process.
Importantly, the data used for the analyses also did not consider individuals who did not vomit. Such individuals, regardless of dose, would be excluded from the medical system if time to emesis was the sole criterion and would tend to be excluded if time to emesis was a major but not exclusive criterion. Waselenko et al. (16) have published a table that provides data (source not given) that estimates the percentage of subjects vomiting at different dose levels. This varies from 19% at 1 Gy to >94% at 6 Gy or higher.
Perhaps an even more important factor is the potential effect of panic and fright in the exposed population. The populations that would be involved in an incident in the future are likely to have much more knowledge of the effects of radiation than did those exposed at Chernobyl. This may significantly enhance the probability of psychogenically induced emesis, especially if there are many cases of true radiation-induced emesis. The contagious nature of emesis is well documented. While such possibilities are mentioned in some of the documents for guidance, these are not factored in when providing the guidance based on time to emesis.
There is also the potential for terrorists to exploit knowledge of the guidelines using time to emesis. For some types of devices, especially a radiation dispersal device, it would be feasible for the terrorists to include emetic agents within the device.
The concern for having false positives (i.e., individuals who have a short time to emesis but who did not receive high doses of radiation) potentially extends beyond the problems of overwhelming the medical response system. There is considerable effort and some indication of success in the development of radiation-mitigating drugs. To the extent that these are developed successfully and administered as soon after exposure as possible (which is very likely) and have potential toxicity (which also is very likely), the potential for harm from false positives could increase substantially.
In view of the uncertainties associated with the use of time to emesis, what might be the most appropriate way to use the parameter in a situation that requires triage of a large population? We suggest that, in view of its simplicity and clear end point, in spite of its limitations, it has potential for some utility. To use it effectively, however, three additional elements should be taken into consideration:
The use of time to emesis as an indication of significant radiation exposure has many uncertainties and problems associated with it. On the basis of the data set that is used widely for relating time to emesis to decisions for triage, a conservative statistical evaluation indicates that the standard error of prediction is about 200%. Further considerations of the applicability of that data set to the proposed uses, the absence of data on the incidence of any vomiting at various doses, and the potential for psychogenic vomiting and/or use of emetic agents in an event, it is clear that this method has significant additional limitations that increase the uncertainty beyond that from the statistical analysis. On the other hand, in the absence of better methods to determine radiation exposures rapidly after the fact, it is prudent to continue to employ this parameter as part of the guidelines for triage for events in which large numbers of individuals potentially have been exposed. For this to be done rationally and effectively, however, the decision-makers responding to the event need to have an excellent understanding of the imprecision of this method, so that appropriate decisions can be made that will decrease rather than increase the morbidity and mortality that might ensue from such an incident. The imprecision of the technique should be strong motivation for the active development of new dose estimation techniques that have greater reliability (18–20).
We would like to thank Ronald Goans and Darrin Edwards for their thoughtful comments on drafts of the paper. This work was supported in part by NIH Grant U19 A1067733 and DARPA Grant HR0011-08-C-0023.