This analysis focused on central methodological questions relating to the performance of self-reported VGDF as a measure of occupational exposure, particularly relative to a JEM approach. As with any other survey approach, the performance characteristics of the VGDF method indicate that it does have limitations that should be borne in mind. It is moderately sensitive against JEM assignment when the latter uses an expert review approach as recommended by Kennedy.[9
] Its specificity, based on that standard, is less and the kappa was low. Moreover, inconsistency (test retest discordance) and inaccuracy (presuming JEM classification to be correct) differed systematically by diagnostic group. Those with chronic rhinitis alone were more likely to report VGDF status differently despite a lack of job change and to report VGDF (based on their baseline interview) in a job for which the JEM assignment was low exposure likelihood, consistent with a confounding effect. Future studies should take this into account with appropriate methods, including stratified analyses where possible.
Self-reported exposure, whether based on a single item, group of items, or a more detailed questionnaire, has limitations. Potential misclassification, due to either under- or over-reporting exposure, presents a serious challenge. Some subjects may be unaware of the substances they have worked with or, conversely, if they have a disease that they associate with workplace factors, may over-report exposures. In our study, this morbidity-driven bias may have been a relatively greater factor among subjects with chronic rhinitis alone, leading them to over-report exposure even though rhinitis is typically a less severe condition than asthma. Differential self-reported exposure misclassification linked to health status can lead to over- or underestimation of effects, as opposed to non-differential misclassification which would tend to bias toward the null.[24
] Although we do not have an obvious explanation for the differential rhinitis effect, we have previously reported that those with rhinitis alone, compared to asthma, have decreased work effectiveness on the job, even though persons with asthma have lower work force participation. [18
Another limitation of our study, as well as most community-based studies, is the lack of independent quantification of exposures. This is a typical problem, given that industrial hygiene measurements or biological monitoring data are rarely available. Other potential limitations of this particular study should be acknowledged as well. Because all the subjects had airway disease, these findings may not be applicable to survey methods of exposure assessment in other conditions, such as cancer, or among those with no chronic health problems. Studying those currently working may have introduced selection effects if former workers would have systematically differed in performance measures. Our data collection method (telephone interviews) may have introduced measurement error not present with other methods, for example, face to face interviews with flash cards. Another source of error may have been subjects who considered VGDF as including relatively minor exposures such as perfumes or the equivalent of house dust on surfaces. This may account for the low percentage reporting VGDF but denying exposure to any of the items on the checklist and is consistent with what we observed in a comparable survey in a different cohort.[25
] Finally, treating as “discordant” a change in VGDF report over two interviews, when there was no change in occupation or industry or a narrative indication of change, could have overestimated inconsistency, if working conditions did indeed vary.
In this analysis, we used a JEM modified by expert review as the “gold standard” against which to measure the performance of a VGDF item. We did not compare this to performance of the JEM prior to such review, although 14% of the assignments did change as a result of this. Furthermore, we did not study a hybrid JEM-self report methodology, for example, using selected data from the 19 item checklist to modify the JEM classifications as a supplement to the expert review step. The data shown in suggest that, for subgroups of checklist items (in particular, irritant gases and fumes), specificity and overall agreement with the JEM are higher than for the single VGDF item. Of interest, a recent case-control study of occupational risk for COPD employed just such a hybrid method of expert review-modified JEM based not on open-ended text as used in our study, but rather on a multi-tem check-list. [26
The JEM approach has been used extensively since the 1980s as an objective method to assess occupational exposures. The use of JEMs based on both European and U.S. occupational coding systems (often called generic JEMs) has been central to a number of case-control studies, primarily in analyses of occupationally-related cancer. In an extensive review of occupational exposure assessment for case control studies, the sensitivity of a generic JEM was low when compared to self-report or expert assessment. [27
A major problem with the generic JEM approach is that it fails to take into consideration the variability of exposures within the same job classification. Expert review can theoretically reduce this problem by modifying the JEM assignment based on additional data on actual job duties and/or exposures reported by the individual subjects.[9
] It is also important to acknowledge that the JEM approach is probabilistic, thus a “low” likelihood of exposure is not the same a “no” exposure.
Expert assessment of exposure and generic and modified JEM assignment have been used as the de facto “gold standard” to assess self-reported VGDF, as shown in . In this tabular presentation, there is range of sensitivity and specificity for self-reported exposure tested against expert interview or JEM or for agreement between self-reported exposure and JEM assessed with the kappa statistic. In analyses other than the current study, among persons with airway disease, sensitivity estimates have ranged from 48% to 65% and specificity from 80%–83%; among general samples or among those without lung disease, the sensitivity has ranged from 42% to 64% and the specificity from 74% to 91%. Kappa values have ranged between 0.30 and 0.71. The values from the current study fall within the range of these other reports, with the exception of the very low kappa (0.12) for self-report against JEM assignment among those with rhinitis alone.
Comparative performance of self-reported exposure to vapors gas dust and fume (VGDF) and job exposure matrix (JEM) in multiple studies
In multivariate analysis, rhinitis was indeed a risk factor for over-report of exposure against the JEM, which would contribute to a poor kappa. Indeed it was only factor in that analysis that did manifest a statistically significant association with over-reporting. Moreover, in multivariate analysis rhinitis was also associated with increased odds of a discordant self-reported exposure on repeated survey, as were increased age and female gender. The test-retest performance characteristics of self-reported occupational exposure has not been well studied, although self-assessed generic “dustiness” has been reported to have better test-retest performance that querying a more specific exposure (asbestos). [29
- Self-reported occupational exposure to vapors, gas, dust and fumes, ascertained as a single survey item, demonstrates performance characteristics that make it applicable to epidemiological research, bearing in mind that reporting may vary in relation to health status.
- When possible, using multiple occupational exposure assessment measures is a preferable analytic strategy to relying on a single metric alone.
- Epidemiological approaches to exposure assessment in occupational studies can provide data applicable to analyzing health outcomes when direct worksite industrial hygiene assessments are not available.
- Population-based analyses across multiple occupations and industries are feasible and can provide valuable public health information.
Our study is uniquely placed to address the methodological uncertainties of survey-based exposure assessment in airway disease in that it used systematic survey data including multiple exposure assessment measures and also afforded longitudinal re-assessment. In summary, based on this analysis, we found that the VGDF single-item approach does have limitations but that, overall, it performed in a manner indicating that it can be a useful exposure assessment method. Attention should always be given to potential biases; our analysis suggests that rhinitis is a health condition that could lead to differential misclassification. No single method is likely to meet all needs. Whenever multiple approaches to exposure assessment can be applied within the same analysis, this is likely to provide greater confidence in interpreting its results.