This matched case-control study showed a considerable number of breakthrough infections, resulting in modest VE estimates. These results suggest that the MF-59™ adjuvanted vaccine may have had only a limited impact in preventing pH1N1-related hospitalisation in risk groups. Because pandemic vaccination started around the peak of the pH1N1 epidemic in the Netherlands, missing date of vaccination in hospitalised cases is a severe limitation to this study. Applying different scenarios partly overcame this limitation and provided a range of VE estimates, though, residual confounding by time cannot be excluded.
As pandemics occur unexpectedly, and during pandemics available resources could be heavily stretched, ideally routinely collected data should be used to provide estimates of VE against severe outcomes. We showed that using such data for VE estimates is feasible. However, observational studies to estimate the effectiveness of influenza vaccination are prone to bias [
29-
33]. Our study is not immune to such potential bias and although we matched for potential confounding variables, we had limited possibilities to adjust for residual confounding. The use of different types of routinely collected health care data and the consequent differences in the quality and level of information between cases and matched controls is a limitation of this study. In cases, vaccination status was self-reported whilst in controls vaccination was reported by the GP. GP registered data can be incomplete because of unreliable registration and because vaccination was also offered outside GP practices for certain individuals (e.g. those working in healthcare, those with children under the age of 6 months, children under the age of 5). By including controls who were eligible for vaccination by the GP and who were sampled from reliably-registering GPs we aimed to minimise potential underestimation of the vaccination coverage in controls. Recall bias in cases will have had only limited impact because of the short delay between the vaccination campaign and symptom onset, and because of the substantial attention of the general population for the pandemic influenza vaccine.
Frailty selection, resulting in confounding by severity, can be important in VE studies focussing on severe disease or hospitalisation as outcome [
30,
31]. Because the majority of the study population was relatively young (median age 48 years), and suffered from one kind of underlying medical condition, frailty is likely to have been of lesser importance relative to studies on seasonal influenza. Moreover, by matching our cases with controls on underlying medical conditions, we decreased the probability of confounding. However, because our cases could have been suffering from more severe underlying medical conditions than our matched controls, but without preventing them to obtain the vaccine, we performed a sensitivity analysis
post hoc using controls with more severe underlying conditions. Using this restriction, the VE was estimated to be 35% (95%CI -26-66) or up to 59% (95%CI -65-90) depending on the cut off and method used to define exposure. However, it is known that several of our cases did not use any medication, suggesting only mild underlying medical conditions in those cases. These restricted VE estimates are therefore likely an overestimation of the actual VE.
The estimates of the effectiveness in preventing pH1N1-related hospitalisation of the adjuvanted pandemic influenza vaccines used in Europe ranged from 45% (95%CI 3-69; UK using a ASO3 adjuvanted vaccine [
14]) to 90% (95%CI 48-100; Spain using several vaccines [
18]) or even 100% (95%CI ∞-100; Scotland [
13]). Even our maximum VE estimate was not as high as the Spanish and Scottish estimates. To determine the maximum VE, we assumed for all 35 cases with unknown date of vaccination (51% of all vaccinees) that vaccination took place within 7 days of symptom onset. This is unlikely a realistic assumption and therefore the maximum VE estimate is likely overestimated. A possible explanation for the wide range of VE estimates in Europe is the difference in inclusion criteria, the control group and the used vaccine. In Spain [
18], the vaccine was mainly distributed to the usual influenza risk groups (including those with obesity), but all hospitalised patients were included in the study. The UK study [
14] and our Dutch study focussed on the population most at risk for severe outcome of an influenza infection, and therefore only included individuals eligible for vaccination because of an underlying medical condition or advanced age. It is known that individuals with certain types of underlying medical condition or older age have a reduced response to vaccination [
34,
35]. A lower VE is therefore expected in this susceptible population. Furthermore, earlier published studies on VE against hospitalisation used the test-negative case-control design [
14,
18]. This design is susceptible to imperfect specificity and sensitivity of diagnoses and the vaccine coverage in test-negative hospital cases is possibly not representative for the general population. In addition, differential health care seeking behaviour between test-positives and test-negatives could result in biased estimates. As we used national data of notifications and data of an extensive GP network which are representative for the country, we expect that our cases and controls originate from the same, general population. However, our data have limited possibilities to refute the presence of potential bias which may have led to an underestimation of the VE.
The pandemic influenza vaccine used [
19] contained half the amount of antigen relative to seasonal influenza vaccines plus an adjuvant to increase the immunogenicity. It is therefore not possible to make a direct comparison to seasonal influenza vaccines. However, the effectiveness of seasonal influenza vaccines in preventing influenza-related hospitalisations is also under debate. For seasonal influenza vaccination, a low VE estimate against hospitalisation (9-12%) was observed in those aged > 50 years using a difference-in-differences design [
36]. Additionally, a Cochrane review concluded that seasonal influenza vaccination had no effect on hospital admissions or complication rates [
17]. Taking into account the differences in study design and vaccines used, the effectiveness of influenza vaccines to prevent influenza-related hospitalisation appears lower than the effectiveness in preventing clinical disease [
11-
16]. The fact that those most at risk of complications and hospitalisations due to influenza react less favourably to the vaccine may have contributed to this difference. Adding the adjuvant to the vaccine did not overcome this problem in our population.