In most epidemiologic research, the amount of error in a measure of effect is presented in a confidence interval, which is simply an indication of random error or the effect measure’s precision. However, the amount of error due to the effect measure’s validity, the systematic error, rarely is presented. A quantitative assessment of the systematic error for an effect estimate can be made by conducting uncertainty analysis [14
In our study of mortality among North American synthetic rubber industry workers, we assessed the impact of potential systematic error due to problems with historical exposure estimation on the observed association between butadiene and leukemia. When comparing the distribution of RRs from the uncertainty analyses to those in our main analysis, in which the exposure estimates were derived from a JEM containing butadiene intensities corresponding to the mean of the approximate probability distribution of estimates for each plant/work area/job group/calendar year combination, the main analysis RRs in the first two exposure categories fell at the low end of the distribution of RRs from the uncertainty analyses and were at the high end of the distribution in the third and fourth exposure categories. Nonetheless, after considering the complete probability distributions of butadiene exposure estimates, the exposure-response association of butadiene and leukemia is maintained.
There are alternatives to the procedures we used to assess uncertainty stemming from exposure estimation. One possible approach could include the arbitrary variation of assumptions made about TWA estimated exposure for particular work area/job groups. The amount of misclassification of exposure most likely varies among work area/job group estimates. Misclassification may be greatest for work area/job groups that are “nonspecific” (i.e., Production Operator, Production Laborer, Maintenance Laborer or Laboratory worker in unspecified work areas). It is reasonable to assume that a relatively large amount of error occurred in assigning exposure estimates to subjects’ person-time in these groups. Uncertainty analyses could assess how important the lack of job title specificity is in adding to the uncertainty of exposure estimation.
In a preliminary set of uncertainty analyses (data not presented), we created a series of alternative exposure profiles, focusing on work area/job groups that were poorly specified, to evaluate the effect of changes in exposure estimation criteria on the association of butadiene ppm-years and leukemia. We assigned each work area/job group to one of four major categories: unskilled labor in maintenance, skilled trades/field assignment, laboratory technicians and other jobs. We then assigned one of three values of the probability distribution of butadiene estimates (5th percentile, mean or 95th percentile) to each of the four work area/job group categories. The analysis included 10 different exposure profiles and indicated that assumptions made in exposure estimation had little impact on the relation between cumulative butadiene exposure and leukemia. The exposure-response association of butadiene with leukemia persisted in analyses of all 10 exposure profiles. However, this crude analysis had a potential problem in that bias due to exposure estimation error is a complicated function of several parameters, and therefore, examining these few scenarios did not capture the true range of possible estimation error bias.
Using our automated exposure estimation system, we were able to create a much broader range of exposure profiles by creating 1,000 JEMs and subsequently preparing 1,000 datasets for the analysis of the association between butadiene ppm-years and leukemia. The resulting set of RRs portrayed a probability distribution of the estimated RR of the butadiene-leukemia association. These uncertainty analyses assessed the global impact of uncertainty due to exposure estimation on the butadieneleukemia association. The approach entailed manipulation of estimated exposure by using JEMs consisting of exposure values that corresponded to randomly selected percentiles of the approximate probability distribution of plant-, work area/job group- and year-specific butadiene ppm.
This approach was limited in that we were not able to identify particular assumptions (i.e., wind speed, distance of operator from point source of emission, probability of operator standing directly in the emission plume, exposure frequency and duration) that contributed the greatest amount of uncertainty to butadiene exposure estimation. We were also limited to selecting percentile values of butadiene ppm from year-specific approximate probability distributions of exposure estimates for well-defined primary work area/job groups. Butadiene estimates for the less well-defined secondary job groups were, in turn, computed after the percentile estimates were selected for primary work area/job groups. Therefore, this analysis directly quantifies only the variability in the butadiene-leukemia association due to uncertainty in the estimation of butadiene in primary work area/job groups.
This uncertainty analysis was designed to provide insight into the impact of limitations due to exposure estimation procedures, but was carried out only for butadiene ppm-years and leukemia. We estimated exposure for two additional agents in our synthetic rubber workers study, styrene and sodium dimethyldithiocarbamate (DMDTC). Additional analyses could use the same techniques outlined above to investigate the effect on leukemia mortality of uncertainty in styrene and DMDTC exposure estimation.
While additional investigations of the effect of uncertainty in our exposure estimation procedures could be performed, this exercise was a unique approach that displayed the possible distortion of the association observed in our main analysis between cumulative exposure to butadiene and leukemia.
Few occupational and environmental epidemiologic studies have made an effort to quantify the amount of systematic error introduced when using quantitative exposure estimates. This exercise is an example of how uncertainty analyses can be used to investigate and support an observed measure of effect when occupational exposure estimates are employed in the absence of direct exposure measurements.