Safety should be monitored to detect previously unrecognised serious adverse events that might be related to new vaccines. A timely and thorough analysis of safety concerns will need to account for the likelihood that large numbers of disease events—which might be misinterpreted as causally related to vaccines—can be expected to occur in large pandemic H1N1 influenza vaccination campaigns. In the past, the occurrence of such events has threatened or stopped large vaccination campaigns. Assessment of causality for events associated with vaccines will be aided by knowledge of their background incidence rates. Additionally, it is possible to look for temporal or geographical clustering when assessing causality. However, one should expect that rates of adverse health outcomes that are temporally associated with vaccination, such as spontaneous abortion, might also be clustered geographically and within clinical practices just by chance alone. Even random events can appear to have patterns. The chance occurrence of geographical clustering of rare cancers has been noted repeatedly. For example, 55% of California census tracts will have at least one type of cancer statistically raised by chance (p≤0·01) owing to the multiple hypothesis testing of these data.40
With regard to possible outcomes after vaccination, if the practice-level rates of spontaneous abortion after vaccination follow the normal distribution, there will be a small number of practices (eg, about 2%) with a seemingly high rate (>2 SDs above the mean) of spontaneous abortion. Patients—and maybe even the practitioners in these practices—might view this cluster as being higher than background rates and consequently suspect an association with vaccination or even with a specific manufacturer’s vaccine. Clustering of adverse events geographically and within health-care practices after vaccination should be expected, and not interpreted as an indication of a causal relation with vaccination unless supported by more careful study.
Many countries have developed vaccine safety assessment plans. Many of these plans rely on identification of possible adverse events or “signal detection”, but few have the ability to rapidly analyse any signals that are identified. Passive reporting systems such as VAERS in the USA or the yellow card system in the UK41
identify possible signals through review of the number of reported cases of adverse events. Analysis of data from such systems is complex. Additionally, to identify possible signals, it is not appropriate to rely on a review of the number of cases reported to these systems. In a pandemic vaccination programme, this approach might not be straightforward because heightened public awareness could lead to increased reporting of events after identification of a possible vaccine safety concern. Furthermore, if such signals are compared with events reported after other vaccines or seasonal influenza vaccines from previous years, the signal might seem to be stronger than expected because passive reporting is usually biased towards under-reporting unless there is heightened public awareness. In addition to the systems in use to detect potential adverse events, the USA, the UK, and other countries have been developing more robust and comprehensive vaccine safety systems to study potential causal associations; however, it is beyond the scope of this report to describe these in detail.
During a rapid-paced immunisation campaign, an adverse event reported on day 1 might result in a spate of similar events being reported over the following days, leading to a media reporting bias. Similarly, active or prompted telephone reporting systems that rely on individuals to call in and report adverse events can also identify a higher than usual number of events in close temporal association with vaccination. A key problem with passive reporting systems is that they provide a number of events (or numerator) but do not allow calculation of a rate or an attributable risk because the number of people vaccinated (or denominator) is not known. Because of this drawback, such systems might contribute to concerns about a false vaccine safety association because they can identify possible signals but cannot analyse causality.
In the UK, the USA, and Denmark, the availability of large databases that link medical outcomes with vaccine data provides a means of assessing signals identified passively and can provide estimates of a true incidence of medically attended events after vaccination. However, these systems can be affected by relatively small denominators (compared with the rarity of the event) and a time lag in the availability of data. Even though some of these systems track millions of people, for very rare events such as Guillain-Barré syndrome or for outcomes affecting a subset of the population such as pregnant women, they might still not have sufficient power to assess a possible safety concern. Additionally, the possible misclassification of events due to miscoding42
usually requires time-consuming chart review for accurate case ascertainment. The rates of Guillain-Barré syndrome reported from automated data in Finland were higher than those reported elsewhere in the published work, perhaps because of the lack of case validation. This discrepancy highlights the need for cautious interpretation of such results.
One approach to the analysis of events that occur after vaccination is to compare observed rates with expected rates. Background rates can provide the media, the public, public-health officials, and politicians with important information about the expected number of events that can occur in the absence of any vaccination programme. Additionally, they can be used to estimate the number of such events that will occur after immunisation of any number of individuals. In New Zealand, during a mass campaign against meningococcus type B, background rates were used to calculate observed versus expected ratios for adverse events as the vaccine campaign progressed.43
With this approach, scientists and the public could be assured that the number of events observed was not higher than expected.
When calculating background rates and observed versus expected ratios, one must be aware of the geographical, seasonal, ethnic, and age differences in such rates and their dependence on the method used to develop these rates. Such rates are point estimates and, especially for rare events, the uncertainty in such estimates should be taken into account when comparing rates by use of 95% CIs around the estimate or rate ratio. Registries for given diseases such as multiple sclerosis, where all cases are reviewed and validated, will have lower rates of disease incidence than rates calculated from unconfirmed cases identified in large automated databases. For example, during a post-hoc assessment of a possible association between Guillain-Barré syndrome and influenza vaccination in 1992–94, a review of automated hospital data showed that of the cases identified in such databases, only 14% were confirmed as definite cases after chart review, 35% were probable cases, 20% were possible cases, and 31% were not judged to be cases of Guillain-Barré syndrome.44
There were also large differences in incidence by sex, age, and geographical location. Expected background rates of events need to represent as far as possible the age, sex, ethnic, and geographical characteristics of the population being vaccinated. Use of US data to assess the risk of Guillain-Barré syndrome in the UK or Brazil could lead to inappropriate conclusions. Similarly, since the pandemic H1N1 influenza vaccination programmes are likely to target priority groups whose age or sex distribution might differ from that of the general population, it will be important to take these differences into account when assessing the risk of any possible vaccine adverse events.
The prospect of large mass immunisation campaigns against pandemic H1N1 influenza in several countries poses unique challenges to the appropriate assessment of vaccine safety. Such assessment needs to detect and analyse vaccine safety signals and take appropriate action to investigate possible unexpected adverse events. However, it is very likely that concerns about disease events that would have occurred even in the absence of vaccination will raise public concern. Uncommon events such as Guillain-Barré syndrome will occur in close proximity to vaccination in substantial numbers if large populations are vaccinated. Additionally, temporal and geographical clustering of such events can occur by chance alone. Misinterpretation of adverse health outcomes that are only temporally related to vaccination will not only threaten the success of the pandemic H1N1 influenza vaccine programme, but also potentially hinder the development of newer vaccines. Therefore, careful interpretation of vaccine safety signals is crucial to detect real reactions to vaccine and to ensure that temporally related events not caused by vaccination do not unjustly affect public opinion of the vaccine. Development and availability of data banks that can provide locally relevant background rates of disease incidence are important to aid assessment of vaccine safety concerns.