In the vaccinated cohort compared with the unvaccinated cohort, the tendency of mortality to rise in periods of high influenza infection rates was clearly reduced. This is not easily explicable by chance or confounding. We have substantially reduced vulnerability to confounding by avoiding direct comparison of mortality in vaccinated and unvaccinated groups in favour of comparing vulnerability within each group to increasing mortality associated with high circulation of influenza. It is hard to envisage confounding that would cause spurious patterns of sharply reduced mortality in vaccinated people specifically during the high influenza periods.
This approach also improves specificity of outcome. By estimating deaths attributable to influenza statistically, we avoided a choice between reliance on information from death certificates or settling for a non-specific outcome. A somewhat similar idea has been used in the study of malaria.16
The robustness and improved outcome specificity of our indirect approach is at the cost of low precision. Only for all cause mortality and respiratory mortality was the protection by vaccination statistically significant at conventional benchmarks. Estimation of vaccine effectiveness, still more demanding of information under this approach, was even less precise. Furthermore, the approach is not immune to information bias. Errors in the index of circulating influenza would have reduced associations of circulating influenza with mortality in both vaccinated and unvaccinated people. Misclassification of vaccination status would have blurred differences between the groups.17
Several recent conventional observational studies have reported vaccine effectiveness against all cause mortality in the influenza season. A UK study of people aged 55 and over in 1989-90 reported a vaccine effectiveness of 75% (95% confidence interval 21% to 92%).3
Among studies of vaccine effectiveness against all cause mortality in people aged 65 and over, vaccine effectiveness was 57% (55% to 60%) in a Swedish cohort study in 1998-2000,8
24% (3% to 40%) in 1996-7 in the Netherlands,7
and 50% in the United States in 1998-2000.6
A meta-analysis of 20 earlier observational studies found mean vaccine effectiveness for all cause mortality of 68% (56% to 76%).4
These estimates were all lower than our estimate of 83%. However, these fractions were of all deaths in the influenza season, rather than of the excess associated with high influenza periods measured by the more specific estimate of vaccine effectiveness we used in our study. Furthermore, because no comparison was made with a non-influenza season, estimates were more vulnerable to confounding and may have been overestimated owing to vaccine recipients being healthier than non-recipients. Control for confounding is possible in conventional studies, but only for the limited variables measured. One study noted a 12% (8% to 16%) protective effect against acute respiratory mortality averaged over a several influenza seasons (1989-90 to 1998-9). Mortality outside the influenza season, during which vaccination had no effect on respiratory mortality, was also investigated. An apparent protective effect against all cause mortality was seen.9
The authors concluded that the estimate of vaccine effectiveness against all cause mortality during influenza seasons was probably upwardly biased due to the “healthy vaccinee” effect. Our study also shows the presence of the healthy vaccinee effect (second column of ), but we controlled for this by comparing effects of influenza rather than mortality in vaccinated and unvaccinated people, as described earlier.
What is already known on this topic
Randomised trials have shown effectiveness of vaccination against influenza, but mortality is too rare an end point for a reduction to be clearly established
Observational studies have estimated effects on mortality but are subject to confounding and to problems in identifying deaths due to influenza
What this study adds
Mortality in periods of high circulating influenza was clearly increased in unvaccinated people but not in vaccinated people, strongly suggesting a protective effect of vaccination
By avoiding direct comparisons of mortality in vaccinated and unvaccinated people in favour of comparisons of responses to circulating influenza, we avoided most confounding
The difference in the point estimates of vaccine effectiveness between the results in the literature and our estimate could thus be due to differences in methods as discussed above or to chance, given our wide confidence intervals. In particular, the greater specificity of outcome from the examination of mortality during periods of high influenza activity and adjustment for cold weather may account for the higher effectiveness seen here. In conclusion, this research adds to evidence that influenza vaccination protects against mortality from influenza, although estimates of vaccine effectiveness are imprecise. The novel method we adopted offers improved control of confounding at the cost of some precision and is applicable to most studies of effects of episodic infections on mortality.