We found that flu shots reduced all-cause mortality among elderly Kaiser Permanente members by 4.6% during 9 laboratory-defined flu seasons in Northern California. Other researchers have reported that flu shots reduce mortality by much greater amounts. In a meta-analysis of results from 20 cohort and case-control studies, Voordouw et al. (6
) found that flu shots reduce winter deaths by 50%, on average; and in a more recent study, Nichol et al. (19
) reported a 48% reduction in all-cause mortality among the elderly during flu season. However, Simonsen et al. (11
) found that excess mortality attributable to influenza has only been 5%–10% on average during flu seasons in the past several decades. They argued that flu shots could not possibly have prevented more deaths than the 5%–10% of deaths that were flu-related (11
). Our estimate of excess mortality during flu season was 7.8%, which is consistent with Simonsen et al.’s nationwide estimate but lower than estimates made by others (21
This excess mortality of 7.8% is what we found in a population with over 60% vaccine coverage. Our findings suggest that had none of the elderly been vaccinated, excess mortality during flu season would have averaged about 9.8%. We infer that our 4.6% VE estimate amounts to a 47% reduction (4.6/9.8 = 47%) in the number of flu-attributable deaths that would have occurred had none of the elderly been vaccinated.
Mortality in the Kaiser Permanente elderly population was approximately 3,804 per 100,000 person-years (). On average, 683 of these 3,804 deaths occurred during a laboratory-defined flu season, including 326 deaths in vaccinees. Our VE estimate of 4.6% implies that in the absence of flu shots, there would have been 342 flu-season deaths (326/0.954 = 342) in vaccinees. Thus, vaccination prevented approximately 16 flu-season deaths per 100,000 person-years (342 − 326 = 16) in the Kaiser Permanente population, which amounted to approximately 25 deaths prevented per 100,000 people vaccinated. The corresponding “number needed to treat” was 4,000; in other words, 1 death was prevented for every 4,000 elderly people vaccinated.
Before estimating vaccine effectiveness, our initial goal was to examine who gets flu shots. Whereas Nichol et al. (19
) reported that higher-risk patients were more likely to be vaccinated, Jackson et al. (9
) reported that higher-risk patients were less likely to be vaccinated. We found a curvilinear relation between predictors of mortality and vaccination. Perhaps other investigators overlooked the curvilinearity because they considered mainly dichotomous indicators of risk. In our population, as in Nichol et al.’s populations, patients with heart disease, diabetes, or chronic obstructive pulmonary disease were more
likely, on average, to get flu shots than patients without these chronic conditions. However, most patients with these conditions had only a moderately elevated risk of death, often in the range where vaccine coverage was highest. In higher-risk patients, who drive mortality rates in the upcoming flu season, the propensity to obtain flu shots waned.
It seems plausible that near the end of life, frailty poses barriers to vaccination, and patients (and providers) may tend to “give up” on preventive measures. However, until then, patients with chronic conditions have more reason and opportunity to get vaccinated than healthy people, because patients with chronic conditions tend to be more vulnerable to influenza and have more contact with providers who encourage vaccination.
Within low-risk subgroups as well as high-risk subgroups, mortality was low soon after vaccination and then increased over time in a pattern suggesting selection bias (). It is this rise in mortality with time since vaccination that is especially challenging in the estimation of vaccine effectiveness. One strategy is to strive for better measures of frailty for covariate adjustment and for exclusion of patients known to be near death at the outset of the autumn vaccination campaign. However, and suggest that whatever it is about nearness to death that suppresses vaccination, it varies markedly over time and would be difficult—even with data from charts or interviews—to monitor precisely enough to overcome selection bias.
Rather than seek covariates that might lower the biased “VE” line in and keep it flat at zero outside of flu season, we implemented a difference-in-differences approach: We traced the trajectory of the bias over time and compared the vaccination-mortality association inside flu season with that outside of flu season. What facilitated this approach was: 1) access to data on a large study population over a period of 9 years; 2) substantial year-to-year variation in the calendar dates of flu season ascertained by laboratory data; 3) little year-to-year variation in the calendar dates when flu shots were delivered; and 4) the assumption that real vaccine effectiveness is negligible each year until flu season arrives. The potential confounders of our VE estimate are not the unmeasured aspects of frailty which confounded Nichol et al. (19
); instead, confounders would have to be somehow associated with the difference in differences—that is, the difference that the arrival of influenza makes in the vaccination-mortality association.
We examined the difference in differences using case-centered logistic regression. Case-centered logistic regression has several noteworthy features. First, it is closely related to Cox regression in a cohort study. It is equivalent to a stratified Cox model in which death is regressed on a time-varying indicator of vaccination. Each record in the case-centered model summarizes an entire risk set in the corresponding Cox model. The same likelihood is maximized (see the Web Appendix, which is posted on the Journal
’s Web site (http://aje.oxfordjournals.org/
)). Second, case-centered logistic regression is closely related to matched case-control studies with risk set sampling (also called incidence density sampling). However, there is no sampling: Data are used from all available controls. Third, it simplifies the analysis of changes in the exposure-outcome relation. In effect, it makes the odds ratio the dependent variable, which is then examined in relation to time and other factors. Fourth, case-centered logistic regression reduces computational burdens dramatically. These can be daunting in large studies with time-varying exposures. Fifth, it can minimize privacy concerns in a multisite study. Researchers at the study sites only need to pool aggregated data about each risk set rather than personal data about each person.
Our data and findings have limitations. First, we were missing data on flu shots given outside of Kaiser Permanente if they were never reported to Kaiser Permanente. If we missed flu shots delivered in nursing homes to patients near death, then we exaggerated the bias that we highlighted. Second, Kaiser Permanente's elderly population may differ from other elderly populations. Care-seeking behavior near the end of life may vary across sociocultural settings, and vaccination outreach may vary across practice settings. Third, our VE estimate was conditional on the severity of the flu seasons and the match of the vaccines to circulating strains of the virus. Fourth, we overlooked herd effects. Fifth, we overlooked late effects (if the vaccine prevents complications that increase mortality after flu season). Sixth, the 95% confidence interval around our VE estimate was wide relative to the excess mortality found in flu season: The lower bound (0.7%) was not far from zero, yet the upper bound (8.3%) would amount to the bulk of the excess mortality that would have struck vaccinees. Seventh, our focus on mortality overlooked the impact of vaccination on morbidity.
All-cause mortality is nonspecific. Nevertheless, it is important to consider, especially in the elderly. Although our estimate of 4.6% vaccine effectiveness against all-cause mortality during flu season may seem disappointing, it amounts to approximately 47% of a plausible target: the rise in mortality that would have occurred during flu season had none of the elderly been vaccinated.
Case-centered logistic regression can be a useful way to examine change in the impact of a vaccine or treatment as periods of high risk begin and end. More generally, case-centered logistic regression can be a useful way to examine the exposure-outcome association.