|Home | About | Journals | Submit | Contact Us | Français|
Methodological problems of retrospective studies
In the 1980s the comfortable belief that air pollution was no longer a public health issue was shaken by the appearance of the Six Cities study from the USA which revealed dose‐related health effects (ranging from symptoms to mortality) at levels of air pollutants at that time considered to be safe.1 Since then there has been a dramatic rise in the number of publications on air pollution from all parts of the world which have resulted in two broad outcomes: a far better understanding of the mechanisms by which ostensibly “low” concentrations of pollutants impact on the lung and increasing awareness within governments of the need to tighten air quality standards. Most epidemiological studies over this time have considered the effects of day‐to‐day changes in air pollution on daily events such as deaths and hospital admissions (so‐called time‐series studies).2 While these studies are in theory easy to undertake, being based on routinely collected data, they usually lack individual information other than cause of death or admission, age and gender. Using this information, in 1998 the UK's Committee on Medical Effects of Air Pollutants (COMEAP) quantified the health impact of air pollution3 as a stepping stone towards determining the cost effectiveness of further pollution control measures. However, at that time they were unable to quantify the impact of long‐term exposures—which even then were thought likely to be far greater than the day‐to‐day effects—for lack of studies.
Subsequently, the Six Cities studies4 and the much larger American Cancer Society (ACS) study of 151 US cities5,6 have provided insights into the effects of long‐term exposure on mortality and, to some extent, morbidity. While these studies have been used by COMEAP in their second quantification report (the first section on mortality is now on the COMEAP website7), there are no UK prospective longitudinal studies aimed at defining the effect of air pollution on health. Longitudinal studies from Norway and France8,9 have shown associations between particles and mortality with coefficients ranging from 1.04 to 1.16 for a range of outcome and pollutant pairings, while a pilot study from the Netherlands10 has shown much larger effect sizes than the ACS study. The Dutch results might be due to small population size or be a real effect; the full study results are awaited with interest.
The paper by Elliott and colleagues in this issue of Thorax11 using a Geographical Information System‐based small area approach should therefore be a welcome addition (see p 1088). This is a retrospective ecological study of total mortality and thus has the inherent problems of such studies, but the findings are intriguing if true, the effect sizes being larger than those in the ACS study and comparable to the Dutch pilot study. Using black smoke as an index of particle exposure, the adjusted relative risks for respiratory mortality were 3.6% per 10 μg/m3 exposure to black smoke and 13.2% per 10 ppb sulphur dioxide. Puzzlingly, for the most recent period from 1994–8 the coefficients were substantially higher at 19.3% and 21.7%, respectively.
However, these findings need to be interpreted with some caution as timing of exposures, accuracy of estimated exposures and confounding may all be playing a part in inflating these effect sizes. This approach does consider past exposures as relevant to mortality, but only the few years immediately before the period of analysis are used which does not account for longer term exposures which will certainly be having an impact. A further issue is that of spatial autocorrelation, where the frequency or size of an exposure or outcome is dependent on adjacent geographical areas, a particular problem in these sorts of studies.12
One important issue in this study is that of confounding. The authors have adjusted to some extent for socioeconomic factors (important as, among other things, poorer individuals are more likely to live on main roads) using the Carstairs index, but the structure of this index varies geographically and over time. While this might to some extent allow for smoking, the lack of individual smoking data is also a source of confounding. In addition, this is not helped by the spatial change in the population studied over time and, even though the study looked at over 420000 deaths, this variation will be a source of bias. In addition, the population is not truly representative of the total UK population as the wards studied were more urban and more deprived than the average UK ward.
There are two other aspects which are difficult to explain: the findings of a larger effect size for sulphur dioxide than black smoke and the larger effect size when considering more recent exposures. Sulphur dioxide keeps providing positive associations with health end points, especially in European studies, but levels in the UK have fallen and matching the mechanisms to any effect of this gas at these low levels is a problem. Why are there apparently greater effects when considering exposure during recent years when levels have continued to fall? If not artefactual (as the authors discuss), it could be explained on the basis that relative rather than absolute increases in a pollutant level day‐to‐day is the driver of a health impact. Studying the relative (or absolute) change in level (eg, from the previous day) would give insight into this possibility. The mechanistic discussion in this paper was limited to the formation of sulphate as a potential pathway without considering how sulphate itself might be toxic. Sulphate is a non‐toxic ion but, while it is conceivable that it acquires an element of toxicity when formed as a salt such as iron or ammonium sulphate, short‐term human exposure studies to higher than ambient doses of a range of sulphates13 have been negative. While a biological explanation is not necessary when attributing a causal impact to an exposure, where the biology seems to point away from a logical explanation, consideration has to be given to alternative explanations which may well in this case be methodological.
If the full Netherlands study produces effect sizes comparable to the pilot, then this might lend support to the effect sizes reported by Elliott et al11 but, if the Dutch findings are more akin to those in the ACS study, then the methodology of retrospective studies with difficulties in confounding and exposure inaccuracies may be the explanation for the high effect sizes reported here. In the latter case, how then might the question of the effect of long‐term exposure to current air pollution be assessed? Consideration could be given to a longitudinal European study which would also address other environmental factors (such as diet and indoor air pollution) on health. Such a prospective study would deal with the problems that beset the use of retrospective datasets, could address gene environment interactions and would provide definitive quantification of the effects of long‐term exposure to air pollution. The study would be very ambitious and expensive and results, by definition, would be slow in coming, but the only option otherwise is to continue to use US coefficients in our assessments or information from retrospective studies from Europe. At present the results using these two options would be disparate—and which would you choose to use?
Competing interests: None.