Importance of background rates of disease in assessment of vaccine safety during mass immunisation with pandemic H1N1 influenza vaccines

doi:10.1016/S0140-6736(09)61877-8

Lancet. Author manuscript; available in PMC 2010 December 19.

Published in final edited form as:

Lancet. 2009 December 19; 374(9707): 2115–2122.

Published online 2009 October 31. doi: 10.1016/S0140-6736(09)61877-8

PMCID: PMC2861912

NIHMSID: NIHMS183100

Importance of background rates of disease in assessment of vaccine safety during mass immunisation with pandemic H1N1 influenza vaccines

Steven Black, Juhani Eskola, Claire-Anne Siegrist, Neal Halsey, Noni MacDonald, Barbara Law, Elizabeth Miller, Nick Andrews, Julia Stowe, Daniel Salmon, Kirsten Vannice, Hector S Izurieta, Aysha Akhtar, Mike Gold, Gabriel Oselka, Patrick Zuber, Dina Pfeifer, and Claudia Vellozzi

Center for Global Health and Division of Infectious Diseases, Cincinnati Children’s Hospital, Cincinnati, OH, USA (Prof S Black MD); National Institute for Health and Welfare, Helsinki, Finland (Prof J Eskola MD); Center for Vaccinology and Neonatal Immunology, Department of Pediatrics, University of Geneva, Geneva, Switzerland (Prof C-A Siegrist MD); Institute for Vaccine Safety, Department of International Health, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA (Prof N Halsey MD); Division of Infectious Diseases, Department of Pediatrics, Dalhousie University, Halifax, NS, Canada (Prof N MacDonald MD); Vaccine Safety Section, Centre for Immunization and Respiratory Infectious Diseases, Public Health Agency of Canada, Ottawa, ON, Canada (Prof B Law MD); Health Protection Agency, Centre for Infections, Colindale, London, UK (E Miller FRCPath, N Andrews MSc, J Stowe BA); National Vaccine Program Office, Department of Health and Human Services, Washington, DC, USA (D Salmon PhD, K Vannice MHS); Center for Biologics Evaluation and Research, Food and Drug Administration, Rockville, MD, USA (H S Izurieta MD, A Akhtar MD); Discipline of Paediatrics, School of Paediatrics and Reproductive Health, University of Adelaide, SA, Australia (Prof M Gold MD); Department of Pediatrics, Faculty of Medicine, University of São Paulo, São Paulo, Brazil (Prof G Oselka MD); Quality, Safety and Standards Team, World Health Organization, Geneva, Switzerland (P Zuber MD, D Pfeifer MD); and Immunization Safety Office, Centers for Disease Control and Prevention, Atlanta, GA, USA (C Vellozzi MD)

Correspondence to: Prof Steven Black, Center for, Global Health, Cincinnati, Children’s Hospital, 3333 Burnet, Ave, MLC 2048, Cincinnati, OH 45229, USA ; Email: Steven.Black1/at/cchmc.org

The publisher's final edited version of this article is available at Lancet

See other articles in PMC that cite the published article.

Abstract

Because of the advent of a new influenza A H1N1 strain, many countries have begun mass immunisation programmes. Awareness of the background rates of possible adverse events will be a crucial part of assessment of possible vaccine safety concerns and will help to separate legitimate safety concerns from events that are temporally associated with but not caused by vaccination. We identified background rates of selected medical events for several countries. Rates of disease events varied by age, sex, method of ascertainment, and geography. Highly visible health conditions, such as Guillain-Barré syndrome, spontaneous abortion, or even death, will occur in coincident temporal association with novel influenza vaccination. On the basis of the reviewed data, if a cohort of 10 million individuals was vaccinated in the UK, 21·5 cases of Guillain-Barré syndrome and 5·75 cases of sudden death would be expected to occur within 6 weeks of vaccination as coincident background cases. In female vaccinees in the USA, 86·3 cases of optic neuritis per 10 million population would be expected within 6 weeks of vaccination. 397 per 1 million vaccinated pregnant women would be predicted to have a spontaneous abortion within 1 day of vaccination.

Introduction

On June 11, 2009, WHO raised the worldwide pandemic influenza alert to its highest level in response to the global spread of a novel influenza A H1N1 virus (pandemic H1N1). The rapid progression of this outbreak and broad population susceptibility make it likely that a substantial part of the global population will be affected over the next 2 years.

Mass vaccination campaigns against this pandemic H1N1 strain are now being considered or have begun in many countries. Different vaccine products, some containing a novel adjuvant, are likely to be available for use from several manufacturers in late 2009. Such mass campaigns will be unprecedented on a global scale and pose many challenges to assessment of vaccine safety. New questions about vaccine safety will undoubtedly arise. Public health officials will need to monitor for previously unrecognised serious adverse events that might be related to the new vaccines. Additionally, the public will need frequent reassurance of vaccine safety when events that are temporally associated with vaccination are identified, even when these events have other causes and occur at the expected background rate. Although scientists know that a temporal association between vaccine exposure and a subsequent adverse event does not prove that the vaccine caused the event, identification of such temporally associated events could nonetheless raise public concern. Adverse event reporting systems such as the Vaccine Adverse Event Reporting System (VAERS) in the USA or the yellow card system in the UK are designed to detect signals of concern but not assess or prove causality.¹

Unfortunately, the availability of the internet together with an increased public concern and engagement in interpretation of vaccine adverse event data have increasingly allowed for spurious associations to be promoted as fact. Widespread beliefs that such false associations are true can and do disrupt immunisation programmes, often to the detriment of public health. For example, when an association between the measles, mumps, and rubella (MMR) vaccine and risk of autism was made,² it had a negative effect on public uptake of measles prevention programmes in the UK and elsewhere, with a consequent rise in morbidity and mortality due to measles.³ In Nigeria, opposition of religious chiefs to oral polio immunisation campaigns because of unfounded concerns resulted in government officials refusing to allow such campaigns and a substantial rise in polio cases.⁴ In Austria shortly after introduction of the human papillomavirus vaccine, there were calls for withdrawal of the vaccine because of the death of one teenage athlete.⁵ However, data on background rates of sudden death in adolescents were available and the vaccine programme continued.

Another relevant example is the interruption of the 2006 seasonal influenza vaccine campaign in Israel after four deaths occurred within 24 h of immunisation. These four patients were at high risk of sudden death because of their age and underlying disorders, and their clinical presentation was consistent with a cardiac cause of death. Post-event analyses suggested that death occurs at a rate greater than 1 per 1000 per week in this population of patients, such that 20 coincidental deaths might be expected by chance alone within 24 h of immunisation. Nevertheless, this clustering of deaths in time and location resulted in global news coverage and interference with the vaccination campaign.⁶ Factors that contributed to the quick acceptance of several of these false assumptions of causal association were a lack of understanding of how passive reports of adverse events can be interpreted¹^,⁷ and poor understanding of the scientific methods used to assess causality.⁸

Inappropriate assessment of vaccine safety data could severely undermine the effectiveness of mass campaigns against pandemic H1N1 2009 influenza. Guillain-Barré syndrome is a good example to consider. Since the 1976–77 swine influenza vaccination campaign was associated with an increased number of cases of Guillain-Barré syndrome, assessment of such cases after vaccination will be a high priority. Therefore, it is important to know the background rates of this syndrome and how this rate might vary with regard to population demographics. The background rate of the syndrome in the USA is about 1–2 cases per 1 million person-months of observation.⁹ During a pandemic H1N1 vaccine campaign in the USA, 100 million individuals could be vaccinated. For a 6-week follow-up period for each dose, this corresponds to 150 million person-months of observation time during which a predicted 200 or more new cases of Guillain-Barré syndrome would occur as background coincident cases. The reporting of even a fraction of such a large number of cases as adverse events after immunisation, with attendant media coverage, would probably give rise to intense public concern, even though the occurrence of such cases was completely predictable and would have happened in the absence of a mass campaign.

Since recent experience with mass immunisation campaigns in developed countries is limited, we have undertaken a review of considerations that we believe should be made in assessment of vaccine safety during pandemic H1N1 vaccine use. This report is not designed to be a comprehensive review or meta-analysis; rather, we provide examples of background rates of selected events that are likely to raise concerns for pandemic vaccination programmes. We report geographical variations of the incidence of such events to inform discussion of inevitable temporal associations between vaccination and adverse health outcomes in the event of a mass vaccination campaign.

Data collection and analysis

A list of possible outcomes that might need to be assessed after receipt of a pandemic H1N1 influenza vaccine was developed by the US Food and Drug Administration (unpublished). We selected disease events from this list on the basis of availability of several published studies that could provide background rates and our ability to develop background rates from automated data sources. Input on the contents of our list was provided by the US Centers for Disease Control and Prevention, FDA, and the UK Health Protection Agency. The selected events and background rates included in this report are not comprehensive but rather are meant to serve as examples to emphasise the importance of being able to develop and access such rates.

For the literature review, authors were each assigned specific disease outcomes to review. The diagnosis specified was entered as a key word in PubMed and articles that included incidence rates were identified. We gave priority to recent articles and those that included sex-specific and age-specific incidence rates. In addition to the literature review, we developed incidence rates for several outcomes to show the difference between such rates and rates from registries or other studies in which case validation through chart review had been done. We searched hospital admission databases in the UK, Finland, and the USA for a defined population to calculate incidence rates of selected disease outcomes. Table 1 shows the International Classification of Diseases (ICD)-9 (USA, Canada, and UK) and ICD-10 codes (Finland) used to identify disease outcomes. For Bell’s palsy in the USA, both outpatient and hospital databases at Kaiser Permanente Northern California (CA, USA) were searched with the specified ICD-9 code for Bell’s palsy, whereas in the UK, READ and Oxford Medical Information Systems (OXMIS) coding 350FA, F310.00, 350F, 350FN, and 350R were used. Chart review was not done for these assessments, and cases identified through these data searches were accepted without further review as might happen in a rapid real-life situation. Where necessary, a 2-year washout period was used to identify individuals who had a condition before the start of the observation period to separate incident events from repeat episodes of the same diagnosis. Finally, we used background rates of selected events at selected sites to calculate rates of events likely to occur within 1 day, 1 week, and 6 weeks after receipt of a hypothetical dose of vaccine by multiplying the number of hypothetically vaccinated people by the background rates and the risk window.

Table 1

ICD-9 and ICD-10 codes used to identify disease outccomes

Background incidence of disease

Table 2 shows background incidence of selected disease events in several countries, with age-specific incidence reported when available. Incidence of Guillain-Barré syndrome and related syndromes was highest in older individuals and in people in Finland, and was in some places about two times higher in men than in women. In Finland, incidence varied from 0·18 per 100 000 person-years in boys aged 0–17 years to 10·13 per 100 000 person-years in men aged 65 years or older. In the UK, incidence was higher in older individuals but substantially lower than it was in Finland, with 4·57 cases per 100 000 person-years in men aged 65 years or older. Incidence of the syndrome in the USA was similar to that in the UK. Brazil had the lowest incidence of the disease. The data from the USA, UK, and Finland are from computerised outcomes databases, whereas data from Brazil are from a disease registry in which case validation criteria might be more stringent. Hospital accessibility might also be lower in Brazil.

Table 2

Incidence of disease events that might be temporally associated with vaccine, by country and age group

Optic neuritis is often the first presentation of multiple sclerosis. Incidence of optic neuritis varies by sex; the risk in girls aged 10–19 years in Sweden is 12 times higher than the risk in boys in the same age group. Incidence of optic neuritis also varies by country with 17·5 cases per 100 000 person-years in 20–39-year-old women in Sweden, 7·5 cases per 100 000 person-years in females (all ages) in the USA, and only 0·83 cases per 100 000 person-years for people of all ages in Singapore. Incidence of transverse myelitis and autoimmune thrombocytopenia also varied by country. Low rates of transverse myelitis were seen in Canada, Australia, and Israel, and higher rates were seen in the USA and Finland. Incidence of autoimmune thrombocytopenia was higher in the UK and Nordic countries than in the USA and Finland. Two sites in Canada had different rates of multiple sclerosis, showing the high level of geographical variability in incidence of this disease, even within one country.

In the UK, incidence of Bell’s palsy in younger individuals was less than half that recorded in adults, which emphasises the importance of age adjustment for background rates.

Table 3 shows mortality in two age groups in selected countries (WHO mortality statistics). Although these estimates span a long period and are not directly relevant to mortality within the short period after vaccination, the variability in these rates highlights the need for locally relevant data for mortality. Even in developed countries with low overall mortality, the mortality rate is not inconsequential and deaths will occur in all age groups. An important subset of overall mortality is sudden death. Table 2 shows rates of sudden death within 1 h of onset of symptoms. These rates vary substantially by country but also by occupation: the rate of sudden death in individuals in the US military²⁹ is much higher than the rate in the general population, despite the military population being younger.

Table 3

Mortality by country, age, and sex (2006)³⁰

The current pandemic H1N1 influenza virus causes increased morbidity and mortality in pregnant women; this population is therefore a high priority for vaccination.³¹ Table 4 and table 5 show the rates of spontaneous abortion and premature labour in several countries. Rates vary by age and country, but the proportion of pregnancies that result in either spontaneous abortion or premature labour are high in all of the countries reported.

Table 4

Rates of spontaneous abortion by country, surveillance year, and age group

Table 5

Selected rates of preterm labour or delivery (<37 weeks gestation), by country and year

Expected number of background disease events after vaccination

Table 6 shows the number of coincident events that might be expected as background rate events within 1 day, 1 week, and 6 weeks after receipt of a hypothetical vaccine. Even within the short 1-week period, a substantial number of events can be expected for rare events. For example, approximately four cases of Guillain-Barré syndrome per 10 million vaccinated people are predicted to occur within 1 week of vaccination as background coincident cases. The exact number would depend on the demographics of the vaccinated population. For some of the other events, such as spontaneous abortion or death, the numbers of expected events are quite large. 397 per 1 million vaccinated pregnant women are predicted to have a spontaneous abortion within 1 day of vaccination. However, rates of vaccination are not uniform throughout pregnancy. For this reason and others, our predicted rate probably overestimates the number of events that would be seen after vaccination in an actual campaign. In fact, the predicted value would have to be adjusted according to the proportion of women vaccinated in each trimester and, if possible, when in that trimester vaccination occurred. For sudden death within 1 h of onset of symptoms, 5·75 such events within 6 weeks of vaccination would be expected as background coincident cases per 10 million vaccinated people.

Table 6

Predicted numbers of coincident, temporally associated events after a single dose of a hypothetical vaccine, based upon background incidence rates

Discussion

Safety should be monitored to detect previously unrecognised serious adverse events that might be related to new vaccines. A timely and thorough analysis of safety concerns will need to account for the likelihood that large numbers of disease events—which might be misinterpreted as causally related to vaccines—can be expected to occur in large pandemic H1N1 influenza vaccination campaigns. In the past, the occurrence of such events has threatened or stopped large vaccination campaigns. Assessment of causality for events associated with vaccines will be aided by knowledge of their background incidence rates. Additionally, it is possible to look for temporal or geographical clustering when assessing causality. However, one should expect that rates of adverse health outcomes that are temporally associated with vaccination, such as spontaneous abortion, might also be clustered geographically and within clinical practices just by chance alone. Even random events can appear to have patterns. The chance occurrence of geographical clustering of rare cancers has been noted repeatedly. For example, 55% of California census tracts will have at least one type of cancer statistically raised by chance (p≤0·01) owing to the multiple hypothesis testing of these data.⁴⁰

With regard to possible outcomes after vaccination, if the practice-level rates of spontaneous abortion after vaccination follow the normal distribution, there will be a small number of practices (eg, about 2%) with a seemingly high rate (>2 SDs above the mean) of spontaneous abortion. Patients—and maybe even the practitioners in these practices—might view this cluster as being higher than background rates and consequently suspect an association with vaccination or even with a specific manufacturer’s vaccine. Clustering of adverse events geographically and within health-care practices after vaccination should be expected, and not interpreted as an indication of a causal relation with vaccination unless supported by more careful study.

Many countries have developed vaccine safety assessment plans. Many of these plans rely on identification of possible adverse events or “signal detection”, but few have the ability to rapidly analyse any signals that are identified. Passive reporting systems such as VAERS in the USA or the yellow card system in the UK⁴¹ identify possible signals through review of the number of reported cases of adverse events. Analysis of data from such systems is complex. Additionally, to identify possible signals, it is not appropriate to rely on a review of the number of cases reported to these systems. In a pandemic vaccination programme, this approach might not be straightforward because heightened public awareness could lead to increased reporting of events after identification of a possible vaccine safety concern. Furthermore, if such signals are compared with events reported after other vaccines or seasonal influenza vaccines from previous years, the signal might seem to be stronger than expected because passive reporting is usually biased towards under-reporting unless there is heightened public awareness. In addition to the systems in use to detect potential adverse events, the USA, the UK, and other countries have been developing more robust and comprehensive vaccine safety systems to study potential causal associations; however, it is beyond the scope of this report to describe these in detail.

During a rapid-paced immunisation campaign, an adverse event reported on day 1 might result in a spate of similar events being reported over the following days, leading to a media reporting bias. Similarly, active or prompted telephone reporting systems that rely on individuals to call in and report adverse events can also identify a higher than usual number of events in close temporal association with vaccination. A key problem with passive reporting systems is that they provide a number of events (or numerator) but do not allow calculation of a rate or an attributable risk because the number of people vaccinated (or denominator) is not known. Because of this drawback, such systems might contribute to concerns about a false vaccine safety association because they can identify possible signals but cannot analyse causality.

In the UK, the USA, and Denmark, the availability of large databases that link medical outcomes with vaccine data provides a means of assessing signals identified passively and can provide estimates of a true incidence of medically attended events after vaccination. However, these systems can be affected by relatively small denominators (compared with the rarity of the event) and a time lag in the availability of data. Even though some of these systems track millions of people, for very rare events such as Guillain-Barré syndrome or for outcomes affecting a subset of the population such as pregnant women, they might still not have sufficient power to assess a possible safety concern. Additionally, the possible misclassification of events due to miscoding⁴² usually requires time-consuming chart review for accurate case ascertainment. The rates of Guillain-Barré syndrome reported from automated data in Finland were higher than those reported elsewhere in the published work, perhaps because of the lack of case validation. This discrepancy highlights the need for cautious interpretation of such results.

One approach to the analysis of events that occur after vaccination is to compare observed rates with expected rates. Background rates can provide the media, the public, public-health officials, and politicians with important information about the expected number of events that can occur in the absence of any vaccination programme. Additionally, they can be used to estimate the number of such events that will occur after immunisation of any number of individuals. In New Zealand, during a mass campaign against meningococcus type B, background rates were used to calculate observed versus expected ratios for adverse events as the vaccine campaign progressed.⁴³ With this approach, scientists and the public could be assured that the number of events observed was not higher than expected.

When calculating background rates and observed versus expected ratios, one must be aware of the geographical, seasonal, ethnic, and age differences in such rates and their dependence on the method used to develop these rates. Such rates are point estimates and, especially for rare events, the uncertainty in such estimates should be taken into account when comparing rates by use of 95% CIs around the estimate or rate ratio. Registries for given diseases such as multiple sclerosis, where all cases are reviewed and validated, will have lower rates of disease incidence than rates calculated from unconfirmed cases identified in large automated databases. For example, during a post-hoc assessment of a possible association between Guillain-Barré syndrome and influenza vaccination in 1992–94, a review of automated hospital data showed that of the cases identified in such databases, only 14% were confirmed as definite cases after chart review, 35% were probable cases, 20% were possible cases, and 31% were not judged to be cases of Guillain-Barré syndrome.⁴⁴ There were also large differences in incidence by sex, age, and geographical location. Expected background rates of events need to represent as far as possible the age, sex, ethnic, and geographical characteristics of the population being vaccinated. Use of US data to assess the risk of Guillain-Barré syndrome in the UK or Brazil could lead to inappropriate conclusions. Similarly, since the pandemic H1N1 influenza vaccination programmes are likely to target priority groups whose age or sex distribution might differ from that of the general population, it will be important to take these differences into account when assessing the risk of any possible vaccine adverse events.

The prospect of large mass immunisation campaigns against pandemic H1N1 influenza in several countries poses unique challenges to the appropriate assessment of vaccine safety. Such assessment needs to detect and analyse vaccine safety signals and take appropriate action to investigate possible unexpected adverse events. However, it is very likely that concerns about disease events that would have occurred even in the absence of vaccination will raise public concern. Uncommon events such as Guillain-Barré syndrome will occur in close proximity to vaccination in substantial numbers if large populations are vaccinated. Additionally, temporal and geographical clustering of such events can occur by chance alone. Misinterpretation of adverse health outcomes that are only temporally related to vaccination will not only threaten the success of the pandemic H1N1 influenza vaccine programme, but also potentially hinder the development of newer vaccines. Therefore, careful interpretation of vaccine safety signals is crucial to detect real reactions to vaccine and to ensure that temporally related events not caused by vaccination do not unjustly affect public opinion of the vaccine. Development and availability of data banks that can provide locally relevant background rates of disease incidence are important to aid assessment of vaccine safety concerns.

Acknowledgments

The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention, the Department of Health and Human Services, WHO, the US Food and Drug Administration, or the UK Health Protection Agency.

Footnotes

Contributors

All authors contributed to the writing, review, and editing of this report and contributed original data where noted in table 2.

Conflicts of interest

SB serves on a data monitoring safety board for pneumococcal conjugate vaccine for GSK and has received honoraria for participation in scientific advisory boards for Novartis. NH has grants from the Centers for Disease Control and Prevention (CDC) to study adverse events after vaccination. He also serves on data safety review committees for clinical trials sponsored by CDC, Merck, and Novartis. C-AS has received honoraria for participation in scientific advisory boards and research grants from GSK, Wyeth, and Sanofi Pasteur. All other authors declare that they have no conflicts of interest.

References

1. CDC. Surveillance for safety after immunization: Vaccine Adverse Events Reporting System (VAERS)—United States, 1991–2001. MMWR Surveill Summ. 2003;52:1–24. [PubMed]

2. Wakefield AJ. MMR vaccination and autism. Lancet. 1999;354:949–950. [PubMed]

3. Jansen VA, Stollenwerk N, Jensen HJ, Ramsay ME, Edmunds WJ, Rhodes CJ. Measles outbreaks in a population with declining vaccine uptake. Science. 2003;301:804. [PubMed]

4. Kapp C. Nigerian states again boycott polio-vaccination drive. Lancet. 2004;363:709. [PubMed]

5. Löwer J. Two unclear cases of death. Can we still recommend HPV vaccination? MMW Fortschr der Medizin. 2008;150:156. [PubMed]

6. Kokia ES, Silverman BG, Green M, Kedem H, Guindy M, Shemer J. Deaths following influenza vaccination—background mortality or causal connection? Vaccine. 2007;25:8557–8561. [PubMed]

7. Varricchio F, Iskander J, DeStefano F, et al. Understanding vaccine safety information from the Vaccine Adverse Event Reporting System. Pediatr Infect Dis J. 2004;23:287–294. [PubMed]

8. Halsey NA. The science of evaluation of adverse events associated with vaccination. Semin Pediatr Infect Dis. 2002;12:205–214. [PubMed]

9. Schonberger LB, Bregman DJ, Sullivan-Boyyai JZ, et al. Guillain-Barre syndrome following vaccination in the national influenza immunization program, United States, 1976–1977. Am J Epidemiol. 1979;110:105–123. [PubMed]

10. Dias-Tosta E, Kückelhaus CS. Guillain Barré syndrome in a population less than 15 years old in Brazil. Arq Neuropsiquiatr. 2002;60:367–373. [PubMed]

11. Klein N. Rates of autoimmune diseases in Kaiser Permanente for use in Vaccine Adverse Event safety studies; Presentation at National Foundation for Infectious Diseases meeting; April 29, 2009; Baltimore, MD, USA.

12. Warren SA, Svenson LW, Warren KG. Contribution of incidence to increasing prevalence of multiple sclerosis in Alberta, Canada. Mult Scler. 2008;14:872–879. [PubMed]

13. Bhan V, Brown M, Fisk J, et al. Incidence and prevalence of multiple sclerosis in Nova Scotia; Poster at American Academy of Neurology 57th Annual Meeting; April 9–16, 2005; Miami Beach, FL, USA.

14. Lim SU, Wong WL, Fu E, et al. The incidence of neuro-opthalmic diseases in Singapore: a prospective study in public hospitals. Ophthalmic Epidemiol. 2009;16:65–73. [PubMed]

15. Jin YP, de Pedro-Cuesta J, Söderström M, Stawiarz L, Link H. Incidence of optic neuritis in Stockholm, Sweden 1990–1995: I Age, sex, birth, and ethnic group-related patterns. J Neurol Sci. 1998;159:107–114. [PubMed]

16. Rodriguez M, Siva A, Cross SA, O’Brien PC, Kurland LT. Optic neuritis: a population-based study in Olmsted County, Minnesota. Neurology. 1995;45:244–250. [PubMed]

17. D’ Souza RM. Retrospective hospital-based searches for cases of acute flaccid paralysis. Aust N Z J Public Health. 2001;26:45–49. [PubMed]

18. Banwell B, Kennedy J, Sadovnik D, et al. Incidence of acquired demyelination of the CNS in Canadian children. Neurology. 2009;72:232–239. [PubMed]

19. Berman M, Feldman S, Alter M, Zilber N, Kahana E. Acute transverse myelitis: incidence and etiologic considerations. Neurology. 1981;31:966–971. [PubMed]

20. Jeffery DR, Mandler RN, Davis LETransverse myelitis. Retrospective analysis of 33 cases, with differentiation of cases associated with multiple sclerosis and parainfectious events. Arch Neurol. 1993;50:532–535. [PubMed]

21. Brotherton JM, Gold MS, Kemp AS, et al. Anaphylaxis following quadrivalent human papillomavirus vaccination. CMAJ. 2008;179:525–533. [PMC free article] [PubMed]

22. Bohlke K, Davis RL, Marcy SM, et al. Risk of anaphylaxis after vaccination of children and adolescents. Pediatrics. 2003;112:815–820. [PubMed]

23. Hauser WA, Beghi E. First seizure definitions and worldwide incidence and mortality. Epilepsia. 2008;49:8–12. [PubMed]

24. Forsgren L, Sidenvall R, Blomquist HK, Heijbel J. A prospective incidence study of febrile convulsions. Acta Pediatr Scand. 1990;79:550–557. [PubMed]

25. Zeller B, Rajantie J, Hedlund-Treutiger I, et al. Childhood idiopathic thrombocytopenic purpura in the Nordic countries: epidemiology and predictors of chronic disease. Acta Paediatr. 2005;94:178–184. [PubMed]

26. Schoonen MW, Kucera G, Coalson J, et al. Epidemiology of immune thrombocytopenic purpura in the General Practice Research Database. Br J Haematol. 2009;145:235–244. [PubMed]

27. Corrado D, Basso C, Rizzoli G, Schiavon M, Thiene G. Does sports activity enhance the risk of sudden death in adolescents and young adults? J Am Coll Cardiol. 2003;42:1959–1963. [PubMed]

28. Bowker TJ, Wood DA, Davies MJ, et al. Sudden, unexpected cardiac or unexplained death in England: a national survey. QJM. 2003;96:269–279. [PubMed]

29. Eckart RE, Scoville SL, Campbell CL, et al. Sudden death in young adults: a 25-year review of autopsies in military recruits. Ann Intern Med. 2004;141:829–834. [PubMed]

30. WHO. World health statistics 2009. Mortality and burden of disease. [accessed Aug 2, 2009]. http://www.who.int/whosis/whostat/EN_WHS09_Table1.pdf.

31. CDC. Pregnant women and novel influenza A (H1N1) virus: considerations for clinicians. [accessed Aug 2, 2009]. http://www.cdc.gov/h1n1flu/clinician_pregnant.htm.

32. Herbert D, Lucke J, Dobson A. Pregnancy losses in young Australian women: findings from the Australian Longitudinal Study on Women’s Health. Womens Health Issues. 2009;19:21–29. [PubMed]

33. Hemminki E, Forssas E. Epidemiology of miscarriage and its relation to other reproductive events in Finland. Am J Obstet Gynecol. 1999;181:396–401. [PubMed]

34. Regan L, Braude PR, Trembath PL. Influence of past reproductive performance on risk of spontaneous abortion. BMJ. 1989;299:541–545. [PMC free article] [PubMed]

35. Wilcox AJ, Treloar AE, Sandler DP. Spontaneous abortion over time: comparing occurrence in two cohorts of women a generation apart. Am J Epidemiol. 1981;114:548–553. [PubMed]

36. Gray RH, Wu LY. Subfertility and risk of spontaneous abortion. Am J Public Health. 2000;90:1452–1454. [PubMed]

37. Papiernik E, Goffinet F. Prevention of preterm births, the French experience. Clin Obstet Gynecol. 2004;47:755–767. [PubMed]

38. Morken NH, Vogel I, Kallen K, et al. Reference population for international comparisons and time trend surveillance of preterm delivery proportions in three countries. BMC Womens Health. 2008;8:16. [PMC free article] [PubMed]

39. Bodnar LM, Simhan HN. The prevalence of preterm birth and season of conception. Paediatr Perinat Epidemiol. 2008;22:538–545. [PubMed]

40. Neutra RR. Counterpoint from a cluster buster. Am J Epidemiol. 1990;132:1–8. [PubMed]

41. Iskander JK, Miller ER, Chen RT. The role of the Vaccine Adverse Event Reporting system (VAERS) in monitoring vaccine safety. Pediatr Ann. 2004;33:599–606. [PubMed]

42. Mullooly J, Drew L, Destefano F, Chen R, et al. Quality of HMO databases used to monitor vaccine safety. Am J Epidemiol. 1999;149:186–194. [PubMed]

43. McNicholas A, Galloway Y, Stehr-Green P, et al. Post marketing safety evaluation of a new group B meningococcal vaccine in New Zealand. Hum Vaccin. 2007;3:196–204. [PubMed]

44. Lasky T. Influenza vaccine and GBS. [accessed Oct 27, 2009]. http://www.iom.edu/~/media/Files/Activity%20Files/PublicHealth/ImmunizationSafety/Lasky.ashx.