Search tips
Search criteria 


Logo of cidLink to Publisher's site
Clin Infect Dis. 2013 September 15; 57(6): 820–827.
Published online 2013 June 20. doi:  10.1093/cid/cit404
PMCID: PMC3749748

Vaccine Effectiveness Against Laboratory-Confirmed Influenza Hospitalizations Among Elderly Adults During the 2010–2011 Season


Background. Although annual influenza immunization is recommended for adults aged ≥65 years due to the substantial burden of illness, the evidence base for this recommendation is weak. Prior observational studies that examined influenza vaccine effectiveness against nonspecific serious outcomes suffered from selection bias and the lack of laboratory confirmation for influenza infection. The objective of this study was to determine the effectiveness of the 2010–2011 seasonal influenza vaccine against laboratory-confirmed influenza hospitalizations among community-dwelling elderly adults, a serious and highly specific outcome.

Methods. We conducted a test-negative study of community-dwelling adults aged >65 years in Ontario, Canada. Respiratory specimens collected between 1 December 2010 and 30 April 2011 from patients admitted to acute care hospitals were tested for influenza using nucleic acid amplification techniques. Influenza vaccination was ascertained from physician billing claims through linkage to health administrative datasets.

Results. Receipt of the 2010–2011 seasonal influenza vaccine was associated with a 42% (95% confidence interval, 29%–53%) reduction in laboratory-confirmed influenza hospitalizations. Vaccine effectiveness estimates were consistent across age groups, by sex, and regardless of outcome severity, timing of testing, and when considering individuals vaccinated <7 or <14 days prior to admission as unvaccinated.

Conclusions. Results of this study will better inform decision making regarding influenza vaccination of elderly adults. Similar analyses are needed annually due to antigenic drift and frequent changes in influenza vaccine composition. The linkage of routinely collected laboratory testing and health administrative data represents an efficient method for estimating influenza vaccine effectiveness that complements prospective studies.

Keywords: influenza vaccine, hospitalization, vaccine effectiveness, elderly adults

Seasonal influenza causes substantial morbidity and mortality among adults aged ≥65 years [1, 2]. Therefore, annual vaccination is recommended [35]. However, sparse evidence from randomized controlled trials (RCTs) in elderly populations guides these recommendations. Furthermore, we cannot anticipate that this evidence will become less sparse, as RCTs of influenza vaccines are very challenging to carry out in this population because placebo controls would generally be viewed as unethical. Among adults ≥60 years, trivalent inactivated influenza vaccine (TIV) has been found to be 58% efficacious (95% confidence interval [CI], 26%–77%) against serologically confirmed influenza [6], whereas live attenuated influenza vaccine has been reported to have an efficacy of 42% (95% CI, 22%–58%) against culture-confirmed influenza infection [7]. The participants of these 2 trials were relatively young elderly individuals (80%–90% were aged 60–74 years), and neither trial used the ideal endpoint of polymerase chain reaction–confirmed influenza infection [8] or assessed vaccine efficacy against serious outcomes.

Numerous observational studies that demonstrated large benefits of influenza vaccination against serious outcomes such as all-cause mortality or hospitalizations for pneumonia or influenza [9] have been refuted as the purported reductions in events attributable to vaccination exceed the total proportion estimated to be attributable to influenza [10]. These studies were affected by selection bias [1113], and not using laboratory-confirmed influenza outcomes to assess vaccine effectiveness (VE) may have magnified the bias [14].

Only relatively recently has the test-negative design been applied for evaluating influenza VE [15, 16]. This study design has several advantages over traditional observational study designs, one of which is the use of influenza-specific study endpoints. A previous study illustrated that test-negative subjects were more similar to individuals testing positive for influenza than traditional, randomly sampled controls, and that VE estimates derived using the test-negative study design were closer to RCT estimates compared with standard observational study designs [17].

The objective of this study was to use the test-negative design to determine the effectiveness of the 2010–2011 seasonal TIV against laboratory-confirmed influenza hospitalizations, a serious and highly specific outcome, among community-dwelling elderly adults.


Study Population, Setting, and Design

We linked results of respiratory specimens tested for influenza by Public Health Ontario (PHO) Laboratories to population-based provincial health administrative data (linkage success rate = 97.8%). This study was restricted to community-dwelling adults aged >65 years tested for influenza using a nucleic acid amplification test between 1 December 2010 and 30 April 2011 and admitted to an acute care hospital at the time of testing. All patients had universal access to physician services, hospital care, prescription medications, and influenza vaccines during the study. We used the test-negative design to assess VE [15, 16].

Ethics approval for this analysis was obtained from the Research Ethics Board of Sunnybrook Health Sciences Centre, Toronto, Canada.

Data Sources and Definitions

Laboratory Data

PHO operates 11 public health laboratories across the province. Respiratory samples are submitted for testing for respiratory viruses from across the healthcare system as part of routine clinical care, and by public health departments as part of outbreak investigations. We extracted viral RNA using NucliSENS easyMAG (bioMérieux, Marcy l'Etoile, France). Samples were tested for influenza A and B using real-time reverse transcription polymerase chain reaction (PCR) [18] and/or a commercial multiplex PCR method (Luminex Respiratory Viral Panel, Luminex Molecular Diagnostics, Toronto, Canada; or Seeplex RV Seegene USA, Rockville, Maryland). In the event of discrepant results between the 2 methods, positive results by either method were considered positive.

Hospitalization Data

The Canadian Institute of Health Information's Discharge Abstract Database (CIHI-DAD) contains detailed information on all admissions to acute care facilities in Ontario [19]. For individuals who had multiple hospitalizations with influenza tests during the study period, we retained the admission with the positive test (or the earliest admission, when there were multiple positive tests or multiple negative tests) for analysis.

Influenza Vaccination

We ascertained receipt of the 2010–2011 seasonal influenza vaccine using physician billing claims for influenza vaccination in the Ontario Health Insurance Plan (OHIP) database, which contains service and diagnostic information for services provided by approximately 94% of physicians in the province [20]. Approximately 75% of elderly adults receive influenza vaccines through physician offices that submit claims to OHIP [21]. The remainder are vaccinated through physician offices that do not submit claims to OHIP, pharmacies, healthcare organizations, workplace clinics, or community-based clinics organized by public health departments. Only the 2010–2011 TIV was licensed for use in Ontario; it included the following strains: A/California/7/2009(H1N1)–like (A/pH1N1), A/Perth/16/2009(H3N2)–like, and B/Brisbane/60/2008(Victoria lineage)–like [5]. In sensitivity analyses, we considered individuals who received the vaccine either <7 days or <14 days prior to admission as unvaccinated.


We used the Ontario Registered Persons Database (RPDB), which contains demographic information on all individuals with a valid Ontario health card [22], to obtain age, sex, rural residence (communities with <10 000 residents), and neighborhood (census area–level) income quintile [23].

The number of hospitalizations in the past 3 years, outpatient visits in the past year, and prescription medications in the past year were determined using the CIHI-DAD, OHIP, and Ontario Drug Benefit (ODB) databases, respectively. The ODB database contains outpatient prescription medication claims for all adults aged ≥65 years [24]. We used the Home Care Database [25] to determine receipt of home care services in the previous year.

We used an adaptation of the ambulatory care group (ACG) classification [26] to determine comorbidities that increase the risk of influenza complications as identified by Canada's National Advisory Committee on Immunizations (heart diseases, respiratory diseases, diabetes, cancers, immunodeficiency due to underlying disease and/or therapy, renal diseases, anemia, and aspiration history) [5]. Any mention of these diagnoses in the outpatient (OHIP database) or inpatient (CIHI-DAD) databases in the 3 years prior to the admission date of the hospitalization were considered.

Statistical Analysis

Crude and adjusted logistic regression models were used to estimate the association between influenza vaccination and laboratory-confirmed influenza hospitalization. Adjusted models controlled for: age, sex, rural residence, neighborhood income quintile, healthcare utilization factors (number of outpatient visits and prescriptions in the past year, number of hospitalizations in the past 3 years, and receipt of home care services in the past year), comorbidities that increase the risk for influenza complications, and month of influenza testing, except when stratifying by one of these variables. Vaccine effectiveness was calculated as (1 – adjusted odds ratio) × 100%.

We performed subgroup analyses by influenza type/subtype (influenza A [A/H1N1, A/H3N2], influenza B), age group (66–75 years, 76–85 years, ≥86 years), sex, and date of testing relative to date of admission (same day as admission, 1–2 days following admission, ≥3 days following admission). To look for evidence of waning immunity, we estimated VE based on the timing of specimen collection, both by month (December 2010, January 2011, February/March/April 2011) and relative to the peak of influenza season (defined as before vs after 4 January 2011, the day when the percentage of specimens testing positive for influenza among the study population peaked). Furthermore, we evaluated VE among hospitalizations with differing outcome severities (hospitalization but no intensive care unit [ICU] admission, hospitalization requiring an ICU admission, death within 90 days following hospitalization). We used established methods to identify ICU admissions and deaths using CIHI-DAD and RPDB, respectively [22, 27].

To demonstrate specificity of the association between influenza vaccination and influenza hospitalization, we examined the association between influenza hospitalization and eye exams by an optometrist as a negative tracer exposure (ie, no association expected).

Statistical analyses were conducted using SAS 9.1 (SAS Institute, Cary, North Carolina). All tests were 2-tailed and we used P < .05 as the level of statistical significance.


We included 569 individuals who tested positive for influenza (536 A [503 A/H3N2, 16 A/pH1N1, 17 not subtyped] and 33 B) and 1661 individuals who tested negative for influenza. Compared to those who tested negative for influenza, test-positive individuals were older, less likely to be male or to live in rural areas, and more likely to live in lower-income neighborhoods (Table (Table1).1). The test-positive group also had less prior healthcare use (in terms of hospitalizations, outpatient visits, and home care services), took fewer prescription medications, were less likely to have certain comorbidities (chronic cardiovascular and respiratory diseases, history of aspiration) or to have received influenza vaccination, and were more likely to have been tested earlier in the study period (ie, December 2010 or January 2011).

Table 1.
Descriptive Characteristics of Influenza Test–Positive and Influenza Test–Negative Hospitalized Patients, 2010–2011

Compared to unvaccinated individuals, vaccinated individuals were more likely to be male, had fewer prior hospitalizations, and took more prescription medications (Table (Table22).

Table 2.
Descriptive Characteristics of Vaccinated and Unvaccinated Hospitalized Patients, 2010–2011

The overall crude estimate of VE was 44% (95% CI, 32%–54%), and it remained unchanged after multivariable adjustment (VE = 42%; 95% CI, 29%–53%; Table Table3).3). After adjustment, vaccination was associated with decreased risk of hospitalization with influenza A/H3N2 (VE = 40%; 95% CI, 26%–52%) and A/H1N1 (VE = 90%; 95% CI, 51%–98%) but not influenza B (VE = 13%; 95% CI, −77% to 58%). VE estimates were similar across age groups, by sex, and regardless of date of testing relative to admission, outcome severity, month of testing, peak of influenza season, and when considering individuals vaccinated <7 days or <14 days prior to admission as unvaccinated. Although VE after the peak of influenza season (37%; 95% CI, 20%–51%) appeared to be lower than VE before the peak (55%; 95% CI, 32%–70%), the interaction test to formally evaluate the difference between these estimates was not significant (P = .32). No association was observed between optometry examinations and influenza hospitalizations (VE =−21%; 95% CI, −50% to 2%).

Table 3.
Unadjusted and Fully Adjusted Influenza Vaccine Effectiveness Estimates


Receipt of the 2010–2011 seasonal influenza vaccine was associated with a 42% reduced risk of laboratory-confirmed influenza hospitalization among community-dwelling elderly adults. Although significant VE against influenza A subtypes (A/H3N2 and A/H1N1) was demonstrated, we detected no significant VE against influenza B. With only 33 individuals who tested positive for influenza B, our study may not have been sufficiently powered to detect VE against influenza B. Our estimates of VE remained robust to numerous subgroup and sensitivity analyses, and the absence of an association between influenza hospitalization and our tracer exposure suggests that these results are not due to underlying differences in healthcare-seeking behavior between test-positive and test-negative individuals.

An estimated VE of 42% is consistent with the only RCT evaluating the efficacy of TIV against serologically confirmed influenza infection among elderly patients (VE = 58%; 95% CI, 26%–77%) [6]. Our results are also consistent with 2 studies using the test-negative design to evaluate the 2010–2011 seasonal influenza vaccine, where a VE of 38% (95% CI, −16% to 67%) was estimated for adults ≥65 years in the United States [28], and a VE of 60% (95% CI, 17%–81%) was estimated for adults ≥60 years in a multicenter European study; both used laboratory-confirmed influenza infection as the outcome [29]. Although our VE estimate is similar to previous evaluations in elderly adults, those studies did not examine VE against influenza hospitalizations.

Only 2 prior studies have examined VE against laboratory-confirmed influenza hospitalizations in older adults. Talbot et al reported a propensity score-adjusted VE of 61% (95% CI, 18%–82%) among community-dwelling adults aged ≥50 years [30]. However, in that study only 28% of test-positive influenza hospitalizations occurred among those aged ≥65 years (most were in those aged 50–64 years), and VE was imprecisely estimated for the ≥65 years age group (VE = 61%; 95% CI, −48% to 90%). Another study estimated a VE of 59% (95% CI, 4%–83%) against laboratory-confirmed influenza hospitalizations for 2010–2011 [31], but only 45% of the test-positive patients were ≥65 years and VE estimates were not stratified by age group.

A recent study of the 2011–2012 seasonal influenza vaccine suggests that vaccine-induced protection may wane over time within an influenza season [32]. In this study, the VE point estimates declined both by month (51% in December 2010 vs 47% in January 2011 vs 34% in February/March/April 2011) and when dividing the study period into periods before and after peak influenza activity, but the confidence intervals overlapped considerably and interaction tests were not statistically significant. Future studies with larger samples sizes will be required to investigate this possibility.

In the most recent Cochrane Collaboration review of influenza vaccines for elderly adults, vaccinated community-dwelling elderly were found to be at reduced risk of pneumonia and influenza (P&I) hospitalizations during defined influenza seasons compared with unvaccinated individuals (VE = 27%; 95% CI, 21%–33%) [9]. However, only 30% of all P&I hospitalizations during discrete influenza seasons have been statistically estimated to be attributable to influenza viruses [33]. Our study estimated VE against laboratory-confirmed influenza hospitalizations, which is a much more specific outcome for serious influenza infection than P&I-coded hospitalizations in discharge abstracts. Estimating VE against nonspecific outcomes can magnify the bias inherent in observational studies of VE in the elderly, as measured and unmeasured characteristics may differ between those receiving the vaccine and those not receiving the vaccine, leading to exaggerated vaccine benefits [14].

Several limitations of this study merit emphasis. First, as symptom onset date was available for only 16% of specimens, we could not restrict our study sample to individuals for whom symptom onset was within 7 days of specimen collection; this is a common inclusion criteria for most test-negative studies to ensure influenza can still be detected [17, 28, 34]. However, among participants with a recorded symptom onset date, 96% of test-positive individuals and 89% of test-negative individuals were tested within 7 days of symptom onset, suggesting the absence of a large difference between the 2 groups. Second, the specimens were collected as part of routine clinical care rather than through systematic screening and enrollment. With no standard case definition for testing hospitalized patients for influenza, it is possible that a physician's decision to order a clinical test may have been influenced by the patient's prior vaccination status. However, in a post hoc analysis, we explored differences in influenza PCR testing rates among vaccinated and unvaccinated hospitalized patients in Ontario. Among 81 398 hospital admissions of vaccinated individuals occurring between 1 December 2010 and 30 April 2011, 1194 were tested for influenza (1.47% of admissions), compared to 1045 individuals tested for influenza during 62 975 admissions of unvaccinated individuals (1.66% of admissions). Therefore, unvaccinated individuals are more likely to be tested for influenza than vaccinated individuals (crude odds ratio = 1.13; 95% CI, 1.04–1.23). In a multivariable model incorporating the variables described in the “Covariates” section of the Methods, the adjusted odds ratio was 1.21 (95% CI, 1.11–1.32). The 21% relative increase in testing for unvaccinated individuals compared to vaccinated individuals suggests that PCR testing for influenza in inpatient settings is not dramatically affected by a patient's vaccination status. Such a difference in testing patterns would likely have only a small impact on VE estimates; using a simulation model, Ferdinands et al demonstrated that if unvaccinated patients are 10% more likely to be tested than vaccinated patients, the estimated VE would be 73% instead of a true VE of 70% [35]. Third, vaccination status could have been misclassified due to receipt of influenza vaccines outside of physician offices (estimated to be about 25% of those vaccinated among elderly adults). Test-positive individuals have been found to be similar to test-negative individuals in terms of healthcare-seeking behavior [17], so any misclassification would likely be nondifferential and cause underestimation of VE, although the possibility of differential misclassification exists. Fourth, we were unable to control for potential bias that might result from viral interference, as infection with a noninfluenza virus may offer protection against influenza infection [36, 37]. Finally, as with all observational study designs, the possibility of residual confounding remains, although no studies have demonstrated substantial confounding bias when using the test-negative design.

Despite the limitations, the test-negative design allowed for the assessment of influenza vaccine benefit against a highly specific and serious outcome of influenza infection. Linking routinely collected laboratory and health administrative data enabled us to conduct, to our knowledge, the largest evaluation of VE against laboratory-confirmed influenza hospitalizations among elderly adults in an inexpensive and efficient manner. Such studies complement prospective test-negative studies that involve primary data collection.

Although the benefits of influenza vaccines for preventing serious influenza outcomes in older adults have been uncertain given the scarcity of RCT evidence and the selection bias and outcome specificity issues noted in prior observational studies [9, 14, 38], the results of this study suggest that the 2010–2011 influenza vaccine was 42% effective in reducing laboratory-confirmed influenza hospitalizations among elderly adults; this estimate may better inform decision making and vaccination policy in this current area of controversy. Future studies that link routinely collected laboratory data to assess influenza VE should incorporate additional clinical information to ensure greater homogeneity of the study population, if it is not possible to institute systematic selection of hospitalized patients for influenza testing. Similar analyses are needed annually due to antigenic drift and frequent changes in influenza vaccine composition. These findings support the current recommendations of vaccinating adults aged ≥65 years to prevent serious outcomes of influenza infection.


Acknowledgments. We thank the Data Quality and Information Management department at the Institute for Clinical Evaluative Sciences (ICES) for linking the influenza testing data from the Public Health Ontario (PHO) Laboratories to the provincial health administrative data, and Allison McGeer, MD MSc, for helpful discussions.

Author contributions. J. C. K. and M. A. C. had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. These datasets were held securely in a linked, de-identified form and analyzed at the ICES. Study concept and design: J. C. K., M. A. C., J. B. G., A. P., A.-L. W., R. O., R. T., N. S. C. Acquisition of the data: J. C. K., M. A. C., J. B. G., A. P., A.-L. W., R. O., N. S. C. Analysis and interpretation of the data: J. C. K., M. A. C., J. B. G., A. P., A.-L. W., R. O., R. T., L. C. R., N. S. C. Drafting of the manuscript: J. C. K., M. A. C. Critical revision of the manuscript for important intellectual content: J. C. K., M. A. C., J. B. G., A. P., A.-L. W., R. O., R. T., L. C. R., N. S. C. Statistical analysis: M. A. C. Administrative, technical, or material support: R. T. Study supervision: J. C. K.

Financial support. This work was supported by an operating grant from the Canadian Institutes of Health Research (CIHR XIN 82408). J. C. K. is supported by a Clinician Scientist Award from the University of Toronto Department of Family and Community Medicine. This study was also supported by the ICES and PHO, which are funded by annual grants from the Ontario Ministry of Health and Long-Term Care (MOHLTC).

Disclaimer. The study sponsors did not participate in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; or the decision to submit the manuscript for publication. The opinions, results, and conclusions reported in this article are those of the authors and are independent from the funding sources. No endorsement by ICES, PHO, or MOHLTC is intended or should be inferred.

Potential conflicts of interest. J. B. G. has received research grants from GSK and Hoffmann-LaRoche for antiviral resistance studies. All other authors report no potential conflicts.

All authors have submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Conflicts that the editors consider relevant to the content of the manuscript have been disclosed.


1. Thompson WW, Shay DK, Weintraub E, et al. Influenza-associated hospitalizations in the United States. JAMA. 2004;292:1333–40. [PubMed]
2. Thompson WW, Shay DK, Weintraub E, et al. Mortality associated with influenza and respiratory syncytial virus in the United States. JAMA. 2003;289:179–86. [PubMed]
3. van Essen GA, Palache AM, Forleo E, Fedson DS. Influenza vaccination in 2000: recommendations and vaccine use in 50 developed and rapidly developing countries. Vaccine. 2003;21:1780–5. [PubMed]
4. Fiore AE, Uyeki TM, Broder K, et al. Prevention and control of influenza with vaccines: recommendations of the Advisory Committee on Immunization Practices (ACIP), 2010. MMWR Recomm Rep. 2010;59(RR-8):1–62. [PubMed]
5. National Advisory Committee on Immunization (NACI) Statement on seasonal trivalent inactivated influenza vaccine (TIV) for 2010–2011. Canadian Communicable Disease Report. 2010;36(ACS-6):1–49.
6. Govaert TM, Thijs CT, Masurel N, Sprenger MJ, Dinant GJ, Knottnerus JA. The efficacy of influenza vaccination in elderly individuals. A randomized double-blind placebo-controlled trial. JAMA. 1994;272:1661–5. [PubMed]
7. De Villiers PJ, Steele AD, Hiemstra LA, et al. Efficacy and safety of a live attenuated influenza vaccine in adults 60 years of age and older. Vaccine. 2009;28:228–34. [PubMed]
8. Petrie JG, Ohmit SE, Johnson E, Cross RT, Monto AS. Efficacy studies of influenza vaccines: effect of end points used and characteristics of vaccine failures. J Infect Dis. 2011;203:1309–15. [PMC free article] [PubMed]
9. Jefferson T, Di PC, Al-Ansary LA, Ferroni E, Thorning S, Thomas RE. Vaccines for preventing influenza in the elderly. Cochrane Database Syst Rev. 2010 CD004876.
10. Simonsen L, Reichert TA, Viboud C, Blackwelder WC, Taylor RJ, Miller MA. Impact of influenza vaccination on seasonal mortality in the US elderly population. Arch Intern Med. 2005;165:265–72. [PubMed]
11. Jackson LA, Jackson ML, Nelson JC, Neuzil KM, Weiss NS. Evidence of bias in estimates of influenza vaccine effectiveness in seniors. Int J Epidemiol. 2006;35:337–44. [PubMed]
12. Campitelli MA, Rosella LC, Stukel TA, Kwong JC. Influenza vaccination and all-cause mortality in community-dwelling elderly in Ontario, Canada, a cohort study. Vaccine. 2010;29:240–6. [PubMed]
13. Wong K, Campitelli MA, Stukel TA, Kwong JC. Estimating influenza vaccine effectiveness in community-dwelling elderly patients using the instrumental variable analysis method. Arch Intern Med. 2012;172:484–91. [PubMed]
14. Simonsen L, Taylor RJ, Viboud C, Miller MA, Jackson LA. Mortality benefits of influenza vaccination in elderly people: an ongoing controversy. Lancet Infect Dis. 2007;7:658–66. [PubMed]
15. Orenstein EW, De SG, Haber MJ, et al. Methodologic issues regarding the use of three observational study designs to assess influenza vaccine effectiveness. Int J Epidemiol. 2007;36:623–31. [PubMed]
16. Jackson ML, Nelson JC. The test-negative design for estimating influenza vaccine effectiveness. Vaccine. 2013;31:2165–8. [PubMed]
17. Belongia EA, Kieke BA, Donahue JG, et al. Effectiveness of inactivated influenza vaccines varied substantially with antigenic match from the 2004–2005 season to the 2006–2007 season. J Infect Dis. 2009;199:159–67. [PubMed]
18. van Elden LJ, Nijhuis M, Schipper P, Schuurman R, van Loon AM. Simultaneous detection of influenza viruses A and B using real-time quantitative PCR. J Clin Microbiol. 2001;39:196–200. [PMC free article] [PubMed]
19. Juurlink D, Preyra C, Croxford R, et al. Toronto, Canada: Institute for Clinical Evaluative Sciences; 2006. Canadian Institute for Health Information Discharge Abstract Database: a validation study.
20. Chan B. Toronto, Canada: Institute for Clinical Evaluative Sciences; 1999. Supply of physicians’ services in Ontario.
21. Kwong JC, Manuel DG. Using OHIP physician billing claims to ascertain individual influenza vaccination status. Vaccine. 2007;25:1270–4. [PubMed]
22. Iron K, Zagorski BM, Sykora K, Manuel DG. Toronto, Canada: Institute for Clinical Evaluative Sciences; 2008. Living and dying in Ontario: an opportunity for improved health information. ICES Investigative Report.
23. Finkelstein MM. Ecologic proxies for household income: how well do they work for the analysis of health and health care utilization? Can J Public Health. 2004;95:90–4. [PubMed]
24. Levy AR, O'Brien BJ, Sellors C, Grootendorst P, Willison D. Coding accuracy of administrative drug claims in the Ontario Drug Benefit database. Can J Clin Pharmacol. 2003;10:67–71. [PubMed]
25. Bronskill SE, Carter MW, Costa AP, et al. Toronto, Canada: Institute for Clinical Evaluative Sciences; 2010. Aging in Ontario: an ices chartbook of health service use by older adults.
26. Starfield B, Weiner J, Mumford L, Steinwachs D. Ambulatory care groups: a categorization of diagnoses for research and management. Health Serv Res. 1991;26:53–74. [PMC free article] [PubMed]
27. Scales DC, Guan J, Martin CM, Redelmeier DA. Administrative data accurately identified intensive care unit admissions in Ontario. J Clin Epidemiol. 2006;59:802–7. [PubMed]
28. Treanor JJ, Talbot HK, Ohmit SE, et al. Effectiveness of seasonal influenza vaccines in the United States during a season with circulation of all three vaccine strains. Clin Infect Dis. 2012;55:951–9. [PMC free article] [PubMed]
29. Kissling E, Valenciano M, Cohen JM, et al. I-MOVE multi-centre case control study 2010–11: overall and stratified estimates of influenza vaccine effectiveness in Europe. PLoS One. 2011;6:e27622. [PMC free article] [PubMed]
30. Talbot HK, Griffin MR, Chen Q, Zhu Y, Williams JV, Edwards KM. Effectiveness of seasonal vaccine in preventing confirmed influenza-associated hospitalizations in community dwelling older adults. J Infect Dis. 2011;203:500–8. [PMC free article] [PubMed]
31. Castilla J, Martinez-Artola V, Salcedo E, et al. Vaccine effectiveness in preventing influenza hospitalizations in Navarre, Spain, 2010–2011: cohort and case-control study. Vaccine. 2012;30:195–200. [PubMed]
32. Kissling E, Valenciano M, Larrauri A, et al. Low and decreasing vaccine effectiveness against influenza A(H3) in 2011/12 among vaccination target groups in Europe: results from the I-MOVE multicentre case-control study. Euro Surveill. 2013;18 pii:20390. [PubMed]
33. Baxter R, Ray GT, Fireman BH. Effect of influenza vaccination on hospitalizations in persons aged 50 years and older. Vaccine. 2010;28:7267–72. [PubMed]
34. Skowronski DM, Janjua NZ, De SG, et al. A sentinel platform to evaluate influenza vaccine effectiveness and new variant circulation, Canada 2010–2011 season. Clin Infect Dis. 2012;55:332–42. [PubMed]
35. Ferdinands JM, Belongia EA, Nwasike C, Shay DK. Influenza vaccination status is not associated with influenza testing among children: implications for observational studies of vaccine effectiveness. Vaccine. 2011;29:1935–40. [PubMed]
36. Cowling BJ, Fang VJ, Nishiura H, et al. Increased risk of noninfluenza respiratory virus infections associated with receipt of inactivated influenza vaccine. Clin Infect Dis. 2012;54:1778–83. [PMC free article] [PubMed]
37. Crowcroft NS, Rosella LC. The potential effect of temporary immunity as a result of bias associated with healthy users and social determinants on observations of influenza vaccine effectiveness; could unmeasured confounding explain observed links between seasonal influenza vaccine and pandemic H1N1 infection? BMC Public Health. 2012;12:458. [PMC free article] [PubMed]
38. Osterholm MT, Kelley NS, Sommer A, Belongia EA. Efficacy and effectiveness of influenza vaccines: a systematic review and meta-analysis. Lancet Infect Dis. 2012;12:36–44. [PubMed]

Articles from Clinical Infectious Diseases: An Official Publication of the Infectious Diseases Society of America are provided here courtesy of Oxford University Press