Home | About | Journals | Submit | Contact Us | Français |

**|**HHS Author Manuscripts**|**PMC3798059

Formats

Article sections

Authors

Related links

Vaccine. Author manuscript; available in PMC 2013 October 19.

Published in final edited form as:

Published online 2012 August 31. doi: 10.1016/j.vaccine.2012.08.045

PMCID: PMC3798059

NIHMSID: NIHMS408883

Mathematical models of disease transmission and vaccination typically assume that protective vaccine efficacy (i.e. the relative reduction in the transmission rate among vaccinated individuals) is equivalent to direct effectiveness of vaccine. This assumption has not been evaluated.

We used dynamic epidemiological models of influenza and measles vaccines to evaluate the common measures of vaccine effectiveness in terms of both the protection of individuals and disease control within populations. We determined how vaccine-mediated reductions in attack rates translate into vaccine efficacy as well as into the common population measures of ‘direct’, ‘indirect’, ‘total’, and ‘overall’ effects of vaccination with examples of compartmental models of influenza and measles vaccination.

We found that the typical parameterization of vaccine efficacy using direct effectiveness of vaccine can lead to the underestimation of the impact of vaccine. Such underestimation occurs when the vaccine is assumed to offer partial protection to every vaccinated person, and becomes worse when the level of vaccine coverage is low. Nevertheless, estimates of ‘total’, ‘indirect’ and ‘overall’ effectiveness increase with vaccination coverage in the population. Furthermore, we show how the measures of vaccine efficacy and vaccine effectiveness can be correctly calculated.

Typical parameterization of vaccine efficacy in mathematical models may underestimate the actual protective effect of the vaccine, resulting in discordance between the actual effects of vaccination at the population level and predictions made by models. This work shows how models can be correctly parameterized from clinical trial data.

Vaccination programs provide both direct and indirect protection against infectious diseases. Direct protection occurs by lowering the probability of vaccine recipients to become infected or by reducing the infectiousness of vaccinated individuals when breakthrough infections occur [1]. Indirect protection arises by reducing transmission within the population, thereby lowering the transmission rate for both vaccinated and unvaccinated individuals.

The interchangeable use of terms used to measure and parameterize vaccine efficacy and effectiveness can lead to inaccurate parameterization of epidemiological models and needs to be made explicit. *Vaccine efficacy* measures the protective effects of vaccination by the reduction in the infection risk of a vaccinated individual relative to that of a susceptible, unvaccinated individual [2]. In contrast, depending upon the study design of clinical trials, population-level vaccine effectiveness can be further categorized into the ‘direct’, ‘indirect’, ‘total’ and ‘overall’ impact of the vaccine [2–4] (Figure 1). Halloran et al. presented a seminal framework relating the different vaccination effects relevant study designs [5]. *Direct effects* compares the direct risk of a randomly selected individual with and without the vaccination program [1]. *Indirect effects* can be estimated from the difference in the degree of protection that unvaccinated individuals receive in the presence versus the absence of a vaccine program. *‘Total’ effectiveness* measures the relative infection risk in vaccinated individuals compared to the infection risk in unvaccinated individuals before a vaccination program is launched [6]. Thus, ‘total’ effectiveness of vaccination is the effect of the vaccination program combined with the effect of the person having been vaccinated [5]. However, ‘total’ effectiveness does not take into account indirect protection of unvaccinated individuals in partially vaccinated population. ‘O*verall’ effectiveness* of a vaccination program is defined as the reduction in the transmission rate for an average individual in a population with a vaccination program at a given level of coverage compared to an average individual in a comparable population with no vaccination program [5, 6]. Thus, the ‘overall’ effectiveness takes into account benefits accrued by both vaccinated and unvaccinated individuals, and it is the measure most commonly used to evaluate the impact of a mass vaccination program at the population level [7, 8].

Measures of vaccination effectiveness and study designs for the evaluation of each measure based on comparison populations. Population A and B are separated in every way relevant to transmission dynamics. In population A, some but not necessarily all **...**

We use a model of transmission dynamics to derive the expressions for the four protective effects of a vaccine (i.e. direct and indirect effects, and ‘total’ and ‘overall’ effectiveness) and to analyse the underlying dynamics of vaccine effectiveness regarding the control of an outbreak. We apply our analysis to two mechanisms of vaccine action, ‘all-or-nothing’ and ‘leaky’ vaccines [9]. An ‘all-or-nothing’ vaccine offers complete protection to a subset of the vaccinated individuals but does not take in the remainder of vaccinated individuals, whereas a ‘leaky’ vaccine offers partial protection to every vaccinated individual. We show potential discordance between the actual effects of vaccination at the population level and predictions made by mathematical models for ‘leaky’ vaccines, which often arises from incorrect parameterization of vaccine efficacy. We demonstrate how the vaccine efficacy as well as the four common measures of vaccine effectiveness ([2]) can be correctly estimated from typical attack rate data for influenza and measles, and determine the threshold vaccine coverage required to attain a specific level of effectiveness for each measure.

We use a simple population dynamic model of an acute directly transmitted disease to take into account indirect effects of mass vaccination. We assume that the transmission occurs from person to person based on random mixing. The population is divided into vaccinated and unvaccinated groups. We assume that *N _{U}*,

We assume that an unvaccinated, susceptible individual is infected at a rate proportional to β, the transmission rate among susceptible individuals. Upon infection, individuals are divided into two groups, *I _{U}*(

For ‘leaky’ vaccines, we assume that vaccination reduces the probability of infection. We also assumed that if individuals who received a ‘leaky’ vaccine are infected, their infectiousness will be reduced (see [10–15] for examples). Thus, vaccinated individuals (*S _{V}*) are assumed to be partially susceptible to infection. Here we define α as the vaccine efficacy for susceptibility, i.e. the relative reduction in the transmission rate among vaccinated individuals. On the other hand, for ‘all-or-nothing’ vaccines, we assume that a fraction α (vaccine efficacy) of the vaccinated individuals becomes immune and the remaining fraction, 1−α, is susceptible (see [15–21] for examples). We define σ as the vaccine efficacy for infectiousness for both ‘leaky’ and ‘all-or-nothing’ vaccines.

Using the definition of variables above, the flow of unvaccinated individuals between the different epidemiological classes in the presence of ‘leaky’ vaccination can be described by the following set of coupled, ordinary differential equations:

Eq. 1

Eq. 2

Eq. 3

Eq. 4

The initial conditions for Eqs (1)–(4) are *S _{U}*(0) =

Eq. 5

Eq. 6

Eq. 7

Eq. 8

The initial conditions for Eqs (5)–(8) are *S _{V}*(0) =

To calculate the attack rate in both unvaccinated and vaccinated groups, we substitute Eq. 3 and Eq. 7 into Eq. 1:

Eq. 9

Because the population size and the length of the infectious period is finite, there exists a time *T* at which no new infections can occur and *I _{U}*(

Similarly,

Using *R _{U}*(

We define the attack rate (cumulative incidence) in the unvaccinated and vaccinated groups as and , respectively. Using the vaccine coverage (), gives

Eq. 10

Eq. 11

Similarly, the expected attack rate in the pre-vaccine era, Ω_{B0}, is expressed in the following implicit equation: Ω_{B0} = 1− exp[−β τ_{1} Ω_{B0}] (see [22] for more generalized arguments).

We can use the attack rates in the presence of vaccination among the unvaccinated and the vaccinated to define the direct effectiveness for a ‘leaky’ vaccine (*VE _{I,1}*):

Eq. 12

Thus, it is clear that the resulting direct effectiveness is *not* equivalent to vaccine efficacy (α) as has typically been assumed. Instead, one can set a direct effectiveness of vaccine (*VE*_{1,1}) in Eq. 12, solve for Ω_{A0,1} (or Ω_{A1,1}), and substitute it into Eqs. 10 and 11, which gives implicit equations for α and Ω_{A0,1} (or Ω_{A1,1}). This substitution will correctly parameterize protective vaccine efficacy (α) from the direct effectiveness of vaccine (*VE*_{1,1}).

The focus of clinical trials has been direct effectiveness of vaccine, which estimates how well vaccinated individuals are protected. The parameterization of vaccine efficacy in mathematical models of disease transmission has often been based on the assumption that vaccine efficacy is equivalent to direct effectiveness, resulting in incorrect estimation of the impact of mass vaccination (Figure 2). For instance, if we parameterize vaccine efficacy (α) with 0.625, the resulting direct effectiveness is predicted to range from 0.11 to 0.32 (not shown), depending on vaccine coverage and relative infectiousness of the vaccinated individuals when τ_{1} = τ_{2} = 4 days, and *R*_{0} = 3.2.

Based on our mathematical model, we distinguish the different measures of vaccine effectiveness (Eqs. 1–8), and extend previous works ([2–5]) to present numerical simulations of the common measures of vaccine effectiveness. Furthermore, we identify potential sources for underestimation of the impact of vaccination at the population level, and demonstrate how to correct this underestimation. The indirect vaccine effectiveness (*VE _{IIA,1}*) refers to the relative decrease in cumulative incidence among

Accordingly, the common estimation of vaccine effectiveness are presented as one minus the respective measure of relative risk (Table 1) [5]:

Therefore, the typical parameterization of the efficacy of ‘leaky’ vaccines often leads to misestimating the actual protective effect of the vaccine. By implicit differentiation of Eqs (10) and (11), it is noteworthy that Ω_{A0,1}/*f* and Ω_{A1,1}/*f* are non-positive as their denominators can be expressed as the secondary attack size when a fraction *f* of the population is vaccinated. Thus, the attack size among unvaccinated individuals decreases with higher vaccine coverage due to herd immunity.

In analyzing the efficacy of an ‘all-or-nothing’ vaccine - assuming that the vaccine coverage is denoted by *f* where *f*=*N _{V}*/

Eq. 13

Eq. 14

Eq. 15

Eq. 16

The initial conditions for Eqs (13)–(15) are *S _{U}*(0) =

Eq. 17

Eq. 18

Eq. 19

Eq. 20

The initial conditions for Eqs (17)–(19) are *S _{V}*(0)=(1−α)

It follows from an analysis on an equivalent ‘all-or-nothing’ vaccine similar to the one performed in Model (1)–(8) that

Eq. 21

Eq. 22

Eq. 23

Here, Ω_{A0,2} and Ω_{A1,2} denote the cumulative incidence among the unvaccinated and vaccinated individuals respectively, in the presence of an immunization program using an ‘all-or-nothing’ vaccine. Ω_{B0} is defined as the expected cumulative incidence in pre-vaccine era. From these definitions, it follows that

Consequently, parameterization for protective vaccine efficacy (α) is equivalent to direct effectiveness of vaccine (*VE _{I,2}*) when vaccine-induced protection is based on an ‘all-or-nothing’ mechanism. For ‘all-or-nothing’ vaccines, different measures of vaccine effectiveness can be defined using cumulative incidence (Eqs. 21–23) in the same way as for ‘leaky’ vaccines (Table 1).

Here, we estimated the measures of vaccination effectiveness (Table 1) in models of influenza and measles vaccination using the cumulative incidence approach. We let *A _{O}* and

We found that for an influenza vaccination coverage of 20%, the direct effectiveness of influenza vaccine is estimated at 52% (Fig 2). Thus, resulting direct effectiveness in our model was found to be much lower than vaccine efficacy, that is, the extent to which the vaccine reduces transmission (70% or α=0.7), which is often assumed to be equal.

Consistent with this example of influenza, we found that the resulting direct effectiveness of measles vaccine is lower than the input parameter of vaccine efficacy. Specifically, when the level of measles vaccine coverage is 60%, the resulting direct effectiveness is 63%, although the vaccine reduces the transmission rate by 95% (α=0.95). This direct effectiveness rose to 82% when the level of vaccine coverage was increased to 90%. Therefore, the discrepancy between the vaccine efficacy (α) and the resulting direct effectiveness fell with increasing vaccine coverage. These findings indicate that the typical parameterization of vaccine efficacy using direct effectiveness of vaccine in mathematical models generally leads to underestimation of the impact of mass vaccination. This underestimation, due to errors in parameterization of individual vaccine efficacy (α), is worse for the higher e fficacy vaccine of the more transmissible measles virus than for influenza.

The estimated indirect and ‘total’ effectiveness of vaccination compare the cumulative disease incidence in population A (with the vaccination program) and that in population B (without the vaccination program). For an influenza vaccine with coverage of 20%, indirect effect and ‘total’ effectiveness of vaccine are estimated at 8% and 56%, respectively. In addition, ‘overall’ effectiveness is estimated at 18%. Thus, the ‘total’ effectiveness of an influenza vaccine is usually higher than the ‘indirect’ or ‘overall’ effectiveness (Fig 2). When the vaccine coverage is increased to 40%, the direct effectiveness of the influenza vaccine is estimated to be 59%. Similarly, ‘total’, indirect and ‘overall’ effectiveness are estimated to be 71%, 29%, and 48% at the same level of vaccine coverage. In general, three estimates of indirect, ‘total’, and ‘overall’ effectiveness increase with vaccine coverage level, as was also shown to be the case for the measles vaccine. Specifically, with 20% vaccine coverage, the resulting indirect, ‘total’ and ‘overall effectiveness’ of the measles vaccine were 0%, 51%, and 10%, respectively. However, with 60% vaccine coverage, ‘total’ and ‘overall effectiveness’ increased to 63% and 38%, respectively. Finally, when the vaccine coverage is increased to 90%, indirect, ‘total’, and ‘overall effectiveness’ rose to 2%, 83%, and 75%, respectively.

We found that when vaccine-induced protection is based on an ‘all-or-nothing’ mechanism, the parameterization for protective vaccine efficacy (α) is equivalent to direct effectiveness of vaccine (*VE _{I,2}*) (Figure 4). In contrast, for ‘leaky’ vaccines, ‘total’ effectiveness is greater than the direct effectiveness at all levels of vaccine coverage. These differences among the measures arise because ‘total’ effectiveness measures the relative transmission rate among the vaccine recipients compared to a comparable population without a vaccination program, incorporating both the direct and indirect protection provided by vaccination. In addition, as a vaccine coverage level is increased, the ‘overall’ effectiveness increases faster than the indirect effect, because ‘overall’ effectiveness incorporates both the direct and indirect effects of vaccination.

In mathematical models, the protective vaccine efficacy is often incorporated as the reduction in the risk of infection at individual level. To evaluate the population level effects, the unit of observation becomes the population. In translating the individual-based measure of vaccine efficacy to population-level measures, mathematical models of infectious diseases typically assume that protective efficacy of vaccine, the relative reduction in the transmission rate among vaccinated individuals, is equivalent to direct effectiveness of vaccine. Our model shows that such parameterization of vaccine efficacy for ‘leaky’ vaccines can underestimate the protective effect of the vaccine computed by the relative attack rates. Varying vaccine coverage and the reduction of infectiousness among vaccinated individuals when vaccine breakthrough occurs affect the degree of underestimation. Specifically, underestimation tends to be greater as vaccine coverage or the reduction of infectiousness among vaccinated individuals decreases. We show how the common measures of vaccine effectiveness and vaccine efficacy can be correctly derived from typical attack rate data obtained in vaccine field studies. We also show how the threshold vaccine coverage required to attain a specific level of effectiveness for each measure can be calculated.

To evaluate population effects of a vaccination program, we presented mathematical models that incorporate the study designs for evaluating various types of vaccine effectiveness ([29]). Based on our model, the protective vaccine efficacy (i.e., the level of reduction in individual transmission rate) was an input into our mathematical models, from which the resulting direct, indirect, total, and overall effects of vaccine expected at different vaccination coverage levels were determined. Herd immunity changes the level of immunity in the population after vaccination, thereby increasing ‘total’ and ‘overall’ effectiveness of a vaccine. For both ‘leaky’ and ‘all-or-nothing’ vaccines, ‘total’ effectiveness is higher than indirect or ‘overall’ effectiveness at all levels of vaccine coverage. This is because ‘overall’ and indirect effect of vaccines account for the partial protection of unvaccinated individuals through herd immunity, which is often less than the direct protection for the vaccinated.

In conclusion, in this study, we reveal the potential pitfalls in the parameterization of mathematical models and the resulting underestimation of vaccination effects. The accurate parameterization of vaccine effectiveness is fundamental to model predictions, including projections of epidemic trajectories, the optimization of vaccination policies and cost-effectiveness analyses. The framework proposed here can provide more precise parameterization of mathematical models used to evaluate the effects of vaccination at individual and population levels.

APG and ES thank NSF award 0624117. ES is also grateful for the support by the National Institute of General Medical Sciences MIDAS grant 5U54GM088491-02.

**Publisher's Disclaimer: **This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

1. Haber M. Estimation of the direct and indirect effects of vaccination. Stat Med. 1999 Aug 30;18(16):2101–2109. [PubMed]

2. Haber M, Longini IM, Jr, Halloran ME. Measures of the effects of vaccination in a randomly mixing population. Int J Epidemiol. 1991 Mar;20(1):300–310. [PubMed]

3. Halloran ME, Struchiner CJ, Longini IM., Jr Study designs for evaluating different efficacy and effectiveness aspects of vaccines. American journal of epidemiology. 1997 Nov 15;146(10):789–803. [PubMed]

4. Halloran ME, Longini IM, Jr, Struchiner CJ. Design and interpretation of vaccine field studies. Epidemiol Rev. 1999;21(1):73–88. [PubMed]

5. Halloran ME, Longini IM, Jr, Struchiner CJ. Assessing Indirect, Total and Overall Effects. Design and Analysis of Vaccine Studies: Springer. 2009

6. Halloran ME. Overview of vaccine field studies: types of effects and designs. J Biopharm Stat. 2006;16(4):415–427. [PubMed]

7. Edmunds WJ, Medley GF, Nokes DJ. Evaluating the cost-effectiveness of vaccination programmes: a dynamic perspective. Stat Med. 1999 Dec 15;18(23):3263–3282. [PubMed]

8. Brisson M, Edmunds WJ. Economic evaluation of vaccination programs: the impact of herd-immunity. Med Decis Making. 2003 Jan-Feb;23(1):76–82. [PubMed]

9. Smith PG, Rodrigues LC, Fine PE. Assessment of the protective efficacy of vaccines against common diseases using case-control and cohort studies. Int J Epidemiol. 1984 Mar;13(1):87–93. [PubMed]

10. Shim E, Galvani AP. Impact of transmission dynamics on the cost-effectiveness of rotavirus vaccination. Vaccine. 2009 Jun 19;27(30):4025–4030. [PubMed]

11. Shim E, Feng Z, Martcheva M, Castillo-Chavez C. An age-structured epidemic model of rotavirus with vaccination. J Math Biol. 2006 Oct;53(4):719–746. [PubMed]

12. Kribs-Zaleta CM, Velasco-Hernandez JX. A simple vaccination model with multiple endemic states. Math Biosci. 2000 Apr;164(2):183–201. [PubMed]

13. Shim E. A note on epidemic models with infective immigrants and vaccination Mathematical Biosciences and Engineering. 2006;3(3):557–566. [PubMed]

14. Arino J, Cooke KL, van den Driessche P, Velasco-Hernández J. An epidemiology model that includes a leaky vaccine with a general waning function. Discrete and Continuous Dynamical Systems - Series B. 2004;4(2):479–495.

15. Anderson R, May R. Infectious Diseases of Humans: Dynamics and Control. New York: Oxford University Press; 1991. See p. 311, eq (12.23) and discussion.

16. White MT, Griffin JT, Drakeley CJ, Ghani AC. Heterogeneity in malaria exposure and vaccine response: implications for the interpretation of vaccine efficacy trials. Malaria journal. 2010;9:82. [PMC free article] [PubMed]

17. Ball FG, Lyne OD. Optimal vaccination policies for stochastic epidemics among a population of households. Math Biosci. 2002 May-Jun;177(178):333–354. [PubMed]

18. Keeling MJ, Shattock A. Optimal but unequitable prophylactic distribution of vaccine. Epidemics. 2012 Jun;4(2):78–85. [PMC free article] [PubMed]

19. Ball F, Britton T. An epidemic model with infector and exposure dependent severity. Math Biosci. 2009 Apr;218(2):105–120. [PubMed]

20. Lugner AK, van Boven M, de Vries R, Postma MJ, Wallinga J. Cost effectiveness of vaccination against pandemic influenza in European countries: mathematical modelling analysis. BMJ. 2012;345:e4445. [PMC free article] [PubMed]

21. Longini IM, Jr, Halloran ME, Haber M. Estimation of vaccine efficacy from epidemics of acute infectious agents under vaccine-related heterogeneity. Math Biosci. 1993 Sep-Oct;117(1–2):271–281. [PubMed]

22. Ma J, Earn DJ. Generality of the final size formula for an epidemic of a newly invading infectious disease. Bull Math Biol. 2006 Apr;68(3):679–702. [PubMed]

23. Liu J, Fau - Kochin BF, Kochin Bf Fau - Tekle YI, Tekle Yi Fau - Galvani AP, Galvani AP. Epidemiological game-theory dynamics of chickenpox vaccination in the USA and Israel. 20111123(1742–5662 (Electronic)). [PMC free article] [PubMed]

24. Preziosi MP, Halloran ME. Effects of pertussis vaccination on transmission: vaccine efficacy for infectiousness. Vaccine. 2003 May 16;21(17–18):1853–1861. [PubMed]

25. Truscott J, Fraser C, Cauchemez S, Meeyai A, Hinsley W, Donnelly CA, et al. Essential epidemiological mechanisms underpinning the transmission dynamics of seasonal influenza. Journal of the Royal Society, Interface / the Royal Society. 2012 Feb 7;9(67):304–312. [PMC free article] [PubMed]

26. Mori N, Ohkusa Y, Ohyama T, Tanaka-Taya K, Taniguchi K, Kobayashi JM, et al. Estimation of measles vaccine coverage needed to prevent transmission in schools. Pediatr Int. 2008 Aug;50(4):464–468. [PubMed]

27. Strebel PM, Papania MJ, Fine PEM. Measles vaccine, and community immunity. In: Plotkin S, Orenstein WA, editors. Vaccine. 4th ed. Philadelphia: WB Saunders; 2003. pp. 389–440. 1443–61.

28. Shim E, Grefenstette JJ, Albert SM, Cakouros BE, Burke DS. A game dynamic model for vaccine skeptics and vaccine believers: Measles as an example. J Theor Biol. 2011 Nov 15;295C:194–203. [PMC free article] [PubMed]

29. Halloran ME, Struchiner CJ. Study designs for dependent happenings. Epidemiology. 1991 Sep;2(5):331–338. [PubMed]

PubMed Central Canada is a service of the Canadian Institutes of Health Research (CIHR) working in partnership with the National Research Council's Canada Institute for Scientific and Technical Information in cooperation with the National Center for Biotechnology Information at the U.S. National Library of Medicine(NCBI/NLM). It includes content provided to the PubMed Central International archive by participating publishers. |