Time-dependent vaccine efficacy estimation quantified by a mathematical model

Abstract

In this paper we calculate the variation of the estimated vaccine efficacy (VE) due to the time-dependent force of infection resulting from the difference between the moment the Clinical Trial (CT) begins and the peak in the outbreak intensity. Using a simple mathematical model we tested the hypothesis that the time difference between the moment the CT begins and the peak in the outbreak intensity determines substantially different values for VE. We exemplify the method with the case of the VE efficacy estimation for one of the vaccines against the new coronavirus SARS-CoV-2.

Introduction

Vaccines play a central role in our aptitude to prevent diseases [1]. Indeed, in a highly connected world, it acts as a fundamental tool to decrease the multiscale interaction that enhances the propagation of viruses between individuals and populations [2, 3]. For the last couple of centuries, vaccine campaigns have proved to be the most effective medical strategy to reduce death and morbidity caused by infectious diseases [4]. It has been estimated that, before the advent of COVID-19, vaccines saved at least 2–3 million lives per year worldwide [5]. Vaccines differ from classical medicinal products, because the intention of a vaccine is to prevent an infection and/or a disease in a healthy population [1]. Therefore, the benefit of the vaccine ought to significantly outweigh any risks. To establish both safety and efficacy Clinical Trials (CT) must then be undertaken [6]. CTs are divided in three major phases: Phase 1 clinical trials are to test the initial safety and tolerability of the candidate vaccine; Phase 2 clinical trials test the safety, immunogenicity, and dose ranges; and Phase 3 clinical trials are for additional safety data, immunogenicity, and efficacy [7]. The most important phase of the trial is the third phase since it is required by regulatory agencies to aprove the vacinne for public use. [13].

The decision made by regulatory agencies on whether to license or not a vaccine is made based on CT results combined with data on the vaccine manufacture. The regulatory agency also provides, based on such data, the vaccine usage information, such as indications, contraindications, and precautions. Once a vaccine is licensed, recommendations on its use are released by the immunization advisory bodies that must be followed in the program implementation. This ensures that those who are recommended to be vaccinated actually receive the vaccine. Furthermore, modifications in the vaccine use policy can be implemented based on post-licensing surveillance data on vaccine safety and effectiveness [6, 8–12]. The objective of this work is to simulate different scenarios for the third phase of clinical trials. To meet this objective, in this paper, the third phase of clinical trials is organized as follows:

1. A group of volunteers who never had the disease is randomly selected from the population. We call this number S(0).

2. This group is divided into two subgroups, a fraction qS(0) who will be injected with the vaccine and a fraction (1 − q)S(0) who will be injected with placebo. We exclude those individuals that get the infection before being vaccinated.

3. The trial begins at times t_c and t_p, when vaccine and placebo, respectively, begins to be injected. Ideally t_c = t_p, but, for logistic reasons, this may not be so. The beginning of the trial is t_c or t_p, depending on the one that occurs first.

If vaccine efficacy (VE) could be evaluated as 1 minus some measure of relative risk (RR) in the vaccinated group compared with the unvaccinated group [13, 14].

We shall elaborate later the biological significance on the vaccine efficacy and possible alternatives in the One Possible Way to Explain Vaccine Efficacy Subsection.

VE can vary substantially in different settings [15], being influenced by many factors, including the time variation in the force of infection (also known as incidence density rate), defined as the per capita rate at which susceptible individuals in a population acquire the infection per unit of time [16, 17]. Hence, in endemic situation, the force of infection does not vary significantly in time and the calculation of VE is straightforward and constant in time. When the infection is new or recurrent, or present an important seasonal pattern, however, the moment along the outbreak course may result in different values of VE.

The purpose of this paper is to calculate the variation of the estimated VE due to the time difference between the moment the CT begins and the peak in the outbreak intensity. In the simple model presented in A Deterministic Model Subsection, substantial differences occur in the estimation in the values for VE, which are exemplified with one of the vaccines against the new virus SARS-CoV-2.

Materials and methods

One possible way to explain vaccine efficacy

In this section we propose a simple model to explain what vaccine efficacy is. This model considers that a test is performed ending with INFECTED-VACCINATED individuals and INFECTED-NOT-VACCINATED individuals.

Let λ(t) be the per capita incidence in this outbreak and S(0) be the number of individuals that are involved at the beginning of this trial.

We now define the vaccination efficacy (VE) and connect it to the relative risk (RR) that will play a crucial role throughout this paper:

In this paper, RR is defined aswhere, ARV and ARU are defined as follows, let a be the number of cases among persons who received the vaccine and b the number of non-cases among vaccinated persons. Let c be number of cases among people injected with placebos and d the number of non-cases among this group. Then

The parameters described above can be better appreciated in Table 1:

[Figure omitted. See PDF.]

From the inputs of Table 1, the vaccine efficacy is estimated as follows [13, 14]:(1)where ARV is the fraction of persons in the trial injected with the vaccine that got the infection and ARU is the fraction of the persons injected with placebo that got the infection. Of course, if ARV = ARU, the VE is zero and the vaccine is useless. On the other hand, if a in the definition of ARV is zero, then VE = 1, and the vaccine is perfect. However, as illustrated by the results below, these two extreme cases are difficult to happen in practice. Let us first examine in more detail the significance of ARV and ARU with a simple calculation. Assume that a, b, c, and d are real variables and not integers. This is a common practice in mathematical biology and will be relaxed below. This type of model is called deterministic. Deterministic models can also be used, for example, to predict disease incidence, to estimate the underreport of infections, and access the impact in the delay of vaccination strategies [18–22].

Assume that we have selected for the trial N_v vaccinnated persons and N_p persons to inject with placebo. Then, clearly a+ b = N_v at any moment since we are assuming no deaths. Also, c+ d = N_p. Of course, during the trial a, b, c, and d vary with time. Assume that the force of infection for vaccinated individuals is λ_v. Then, a(t) obeys(2)whose solution is(3)

Then,(4)

Similarly, assuming that the force of infection among the placebo population is λ_p gives(5)

If λ_v < λ_p, we have the ratio(6)where t is the time the trial ends because of a certain criterion and ARV(t)/ARU(t) is a number between 0 and 1. When λ_v is zero, ARV(t)/ARU(t) = 0, which means that the vaccine is perfect and VE = 1, but if λ_v = λ_p, VE = 0 and the vaccine is useless.

Note that if λ_v > λ_p, we have a vaccine efficacy negative, which means that the vaccine is producing the disease!

In the next section we present a more elaborate model to show how the results of the Phase 3 of the clinical trial depend on the outbreak intensity and the moment in the outbreak when the test is conducted. The model also allows one to compare the result of the test with a pre-determined nominal (true) vaccine efficacy.

A deterministic model

Since a stochastic version of the model as described above is very cumbersome, we shall present only a deterministic version.

The model is an extension of the classical SI models without demography [23] and uses the following variables:

1. Susceptible (S(t)): Number of individuals which can either be vaccinated or receive a placebo shot or they may even acquire the infection before receiving the vaccine or placebo.

2. Vaccinated (V(t)): Number of individuals who are transferred from the susceptible state and administered the vaccine.

3. Failure (F(t)): Number of individuals who represent the fraction of susceptible vaccinates who fail to be immunized (that is, not completely protected after vaccination) and can acquire infection with the same rate λ(t) as those non-vaccinated susceptible.

4. Protected (P(t)): Number of individuals who received the vaccine and got protected against the infection.

5. Placebo (Pl(t)): Number of individuals who received the placebo shot and are subject to the same risk of infection as the susceptible individuals.

6. Infective (I(t)): Number of individuals infected before receiving either the vaccine or the placebo.

7. Infected placebo (I_p(t)): Number of individuals who received the placebo and got the infection with the same rate λ(t) as the other infected individuals.

8. Infected vaccinated (I_v(t)): Number of individuals who received the vaccine but were not immunized and got the infection with the same rate λ(t) as the other infected.

Fig 1 shows a diagram with the model’s compartments and transitions.

[Figure omitted. See PDF.]

Notice that the three infected compartments (I, I_p and I_v) are independent of each other. In the case of compartment I, we have infected individuals before receiving placebo or vaccine. Thus, they are not able to participate in the clinical trial and for this reason this compartment is not part of most of the analyses.

It is worth noticing that the model does not consider demography because the cohort is fixed in size and because the duration of the clinical trial is too short relative to the life-span of its components.

The model is described by the following equations:(7) (8) (9) (10) (11) (12) (13) (14)where λ is the force-of-infection (described below), q is the daily proportion of susceptible individuals who receive the vaccine, ν is the daily administration rate, either of vaccine or placebo, Θ_v(t − t_v) and Θ_p(t − t_p) are Heaviside functions that determines when the vaccine and the placebo administration start, σ is the rate with which vaccinated individuals get fully protected and ω is the true efficacy of the vaccine and represents the probability of being protected, given that the individual received the vaccine. In the numerical examples, we used two different functions to determine the starting date of administration, namely, Gaussian functions.

The force of infection λ was modeled based in a real case of time-dependent incidence of a COVID-19 outbreak in the city of Santos in the Southeast Region of Brazil in 2020, reproduced in Fig 2.

[Figure omitted. See PDF.]

Considering that the outbreak can be approximated by a Gaussian function, we assume that the force of infection λ is proportional to the following:(15)where χ is the intensity of the outbreak, τ is the time delay from the CT launch to the outbreak peak, and ς is half of its duration. The values of the parameters in Eq (15) will be given later. Thus, the force of infection of the outbreak λ is defined as the per capita incidence, in other words,(16)where S_Santos is the population of the city whose outbreak is shown in Fig 2.

As explained above, the vaccine efficacy, VE, is calculated using Eq (1) and the experimental results represented by the quantities in Table 1. Such quantities are translated into the model variables as follows,

Therefore, in terms of the model’s variables we have:(17)and(18)

Since the incidence can be written using Eqs (15) and (16), its free parameters assume the following values:

In the following calculations, we shall vary the values of λ(t) with respect to the beginning of the trial by changing the position of the peak of the outbreak with respect to the initial time t = 0. This procedure is illustrated in Fig 3, where the area below the curve represents the burden of infection that the cohort is submitted during the clinical trial at the time t = 0. In this figure, the outbreak reaches its peak at time t = 25, that is 25 days before the beginning of the trial.

[Figure omitted. See PDF.]

Results

The Clinical Trials (CT) ends after the outbreak

We simulated the model in Eqs (7)–(14) for 3 years. We calculate the vaccine efficacy when the trials start at arbitrary times, t_v and t_p. We also assume that the position of the outbreak peak varies with respect to the beginning of the trial. The value of VE is calculated at the end of each simulation.

The parameters in these simulated scenarios were set to q = 0.5, ν = 0.028days⁻¹, , , and τ assumes different values.

Let us consider a set of simulations. In these simulations the position of the peak of the outbreak (given by the value of τ) varies with respect to t = 0. This means that the burden of incidence to which the trial cohort is subjected may vary from simulation to simulation. In the first simulation we assume that t_v = t_p = 0. The result can be seen in Fig 4.

[Figure omitted. See PDF.]

It can be noted that the value of the vaccine efficacy (VE) varies significantly with the burden of infection suffered by the trial population.

In Fig 5, which represents the scenario t_v ≤ t_p, we show the effect on the calculated VE due to the fact that the placebo administered begins after the vaccine administered, we set t_v = 0 and t_p = 5, 10, and 15 days. The simulation with t_p = 0 is included for comparison.

[Figure omitted. See PDF.]

The quantity τ is the time delay in days from the clinical trial launch (which is set to t = 0) to the outbreak peak.

It is remarkable that the VE is less than the nominal value that was set at ω = 0.7 (70%). This is because the value of ARV is reduced by the delay in injecting the placebo.

In Fig 6 the results shown were obtained by setting t_v = 10 days and t_p = 0, 5, 10 and 15 days.

[Figure omitted. See PDF.]

The quantity τ is the time delay in days from the clinical trial launch (which is set to t = 0) to the outbreak peak.

In this case the placebo administered began before the vaccine administered in the first two simulations (with t_p = 0, 5). In this case the calculated VE is generally greater that the nominal value. This is because the value of ARU is reduced by the delay in the administered of the vaccine.

In Fig 7 the results were obtained when we set t_v = 15 and t_p = 10, 15, 20 and 25 days.

[Figure omitted. See PDF.]

The quantity τ is the time delay in days from the clinical trial launch (which is set to t = 0) to the outbreak peak.

Note that when t_p < t_v the calculated VE is smaller than the nominal value. It is also interesting that when t_p = 20 the calculated VE has a maximum value.

Clinical trial interrupted when infections reach 200 individuals

In practice, the trial is not as long as assumed in the first set of examples. In the simulations described below the study is designed to be interrupted when the total number of infections attains 200.

The idea of interrupting the trial at a certain point is to comply with the common practice in actual clinical trials to obtain the necessary power of test [24]. However, the nominal value of the vaccine efficacy, ω, is only reached if at the end of the trial only the compartments P, I_v and I_p are different from zero. To do this we run the simulation integrating the incidence curve up to the stopping point show in Fig 8. In other words, the study ends at the first day when the total number of infected individuals inoculated with the vaccine and the placebo reached at least 200.

[Figure omitted. See PDF.]

In Fig 9 we show the evolution of the VE as a functions of the outbreak peak, when t_v = t_p = 0.

[Figure omitted. See PDF.]

The clinical trial ends at the first day when the total number of infected individuals inoculated with the vaccine and the placebo reached at least 200.

Note that the estimated efficacy is different from the nominal efficacy, w = 0.7. It is reached when all the compartments of the model but the final ones, are zero. It can be noted that the nominal efficacy is only achieved if the fraction of the burden is sufficient large.

We recall that infected individuals in this study come from three different groups. The ones that were not inoculated (I), the inoculated by the placebo (I_p) and those inoculated by the vaccine (I_v). One natural question is how the non-inoculated perform vis a vis those that participated in the study and were inoculated. In particular, what happens to the other compartments when the inoculated individuals reached the value of 200. This is addressed in Table 2 which presents the accumulated numbers of individuals in each compartment of the model in Eqs (7)–(14) after 60 days, 3 years, and at the first day when the total number of infected individuals inoculated with the vaccine and the placebo reached at least 200. In this example, we assume that the vaccine and the placebo inoculation start at t_v = 0, t_p = 0 and the outbreak peak occurs at τ = 60.

[Figure omitted. See PDF.]

In this example, the vaccine and the placebo inoculation start at t_v = 0, t_p = 0 and the outbreak peak occurs at τ = 60.

Vaccination and placebo arms as gaussian distributions

In the first two scenarios we assume that number of injected individuals daily, with the vaccine and the placebo, follows a Gaussian density function, with its mean representing the peak of the trial, and standard deviation representing half of the time duration of the trial. The peak of placebo and vaccine administered does not need to coincide, and using a Gaussian density allows to consider residual administered before and after the trial. Thus, Θ_v(t − T_v) and Θ_p(t − T_p) are replaced by a function Θ_p(s)G(s, L), where(19)and s = t − T_v or s = t − T_p.

We consider three different scenarios:

1. T_v = 0, T_p = 0, and L = 20.

2. T_v = 0, T_p = 10, and L = 20.

3. T_v = 10, T_p = 0, and L = 20.

We evaluate the VE at the first day when the total number of infected individuals inoculated with the vaccine and the placebo reached at least 200.

Fig 10 shows how the VE varies as the outbreak peak changes when the evolution of the Vaccine and the Placebo administered follow the Gaussian density in Eq (19).

[Figure omitted. See PDF.]

The placebo and vaccine daily inoculation follows the Gaussian function in Eq (19).

Fig 11 shows the case when the vaccination administered evolution peak is T_v = 0 and the placebo administered peak is T_p = 10.

[Figure omitted. See PDF.]

The placebo and vaccine daily inoculation follows the Gaussian function in Eq (19).

Finally, Fig 12 shows the case when the vaccination administered peak is T_v = 10 and the placebo administered peak is T_p = 0.

[Figure omitted. See PDF.]

The placebo and vaccine daily inoculation follows the Gaussian function in Eq (19).

Sensitivity analysis

In what follows, we shall evaluate the sensitivity of an arbitrary dependent variable Q with respect to the independent (arbitrary) variable p. See [25]. This is done through the calculation of the elasticity defined as(20)

We are going to set t_v = t_p = 0, and since the model is linear, the analytic solution of its states is straightforward. So,(21) (22) (23) (24) (25) (26) (27) (28)

In Table 3, we present the numerical values of the sensitivity ϵ_p using Eqs (21)–(28). For these calculations we take q = 0.5, w = 0.7, ν = 0.028, σ = 0.1 and τ = 100.

[Figure omitted. See PDF.]

More details can be found in the S1 File.

Illustrating the model with the case of CORONAVAC ^® in Brazil

In order to illustrate the model we examine the case of the CT of the vaccine made by firm SINOVAC SARS-CoV-2, called CORONAVAC in Brazil.

The trial was carried out in the second semester of 2020, when there was an outbreak of COVID-19, that begun in March, reaching a peak in May, subsiding from there onward until a second wave begun in late November.

We fitted the actual outbreak data with a continuous curve with the same shape as in Eq (15). The result is shown in Fig 13.

[Figure omitted. See PDF.]

The solid line represents a continuous function fitted to the data.

The trial began in late July and ended in December. As can be seen from the figure, the trial was conducted in the last quarter of the outbreak. The purpose of the calculations below is to investigate the influence of the period of time elapsed between the peak in the outbreak and the average time the actual trial begun on the estimated efficacy of the vaccine. This period of time was on average equal to 60 days, with variance equal to 30 days. We then simulated the model, varying the value of the proportion w (the nominal efficacy) until the calculated efficacy reached the actual value of efficacy founded in the trial, that is, 50.3% against infection. The efficacy obtained should the trial begun at the peak of the outbreak (that is, 60 days before, on average) was 58.8%. This result does not depend on whether the trial lasts 3 years or if it is terminated when total infections reach 200, as was the case of the CORONAVAC trail.

This illustrate in practice the time-dependence of the calculated vaccine efficacy in settings where the force of infection varies with time (an outbreak). Therefore, we must consider this time-dependence when estimating vaccine efficacy in CTs of vaccines against infections that show a substantial time variation or seasonality.

Discussion

In this paper we tested the hypothesis that the time difference between the moment the CT begins and the peak in the outbreak intensity results in substantially different values for VE. We tested this hypothesis with a mathematical model and exemplify the method with the case of the VE efficacy estimation for one of the vaccines against the new coronavirus SARS-CoV-2. The time-dependence of VE on the force of infection has been previously explored by Kaslov [15] who demonstrated that when the force of infection varies with time, this affects the estimation of VE. This author proposed three different shapes for VE depending on whether the force of infection is constant, or varies linearly with time or varies as a logarithmic fashion with time.

The advent of SARS-CoV-2, however, brought new challenges for the estimation of VE [5, 23], the most important of which, is to estimate VE in the middle of an outbreak. We have shown that, depending on the time difference the peak of the outbreak and the time the CT begins VE can result in substantially different values. Furthermore, this time-dependence of the VE estimation is not linear but assume distinct shapes depending on several factors, in particular if the trial begins before or after the peak of the outbreak. Moreover, VE has shown to be strongly dependent on the duration of the trial. As shown in the sensitivity analysis, VE shown a variable sensitivity to each of the parameters that remarkably varies with the duration of the trial. This explains the strange shapes of the figures that relate VE with the burden of infection. As we can note in Table 3, VE is much more sensitive to the period between the peak of the outbreak and the beginning of the trial, expressed in the value of the parameter τ. Sensitivity of VE is 5 times more influenced by τ when the trial lasts only 50 days than when the trial lasts 500 days. Moreover, the relative sensitivity of VE to each of the parameters also varies with the duration of the trial. For instance, VE is much more sensitive to the rate of vaccination ν than to the nominal efficacy w when the trial lasts 50 days but it reverses when the trial lasts more than 100 days. In addition, it is noteworthy that VE is not sensitive at all to the force of infection λ when the trial lasts more than 300 days. However, one should be aware that this sensitivity is related to the magnitude of the force of infection, not to its time-dependence.

Some important limitations of the model should me mentioned here. First, choosing the definition of vaccine efficacy based on risk rather than rate may not be a good option. The risk ratio, as it aggregates over time, cannot discriminate different temporal trajectories that lead to the same final risk. that is, there is an infinite number of possibilities to obtain the same value of vaccine efficacy, when measured from the risk ratio, which could be discriminated if they were measured as a ratio of rates or comparison of survival curves [26].

In fact, the vast majority of vaccine efficacy studies use Poisson regression or survival analysis as statistical models. These approaches make it possible to express efficiency from the ratio of rates, which can be a time-dependent quantity. However, these techniques are more appropriate to estimate post-license efficiency rather than efficacy as is the case of our model. The two approaches above (rate and risk) may still be unsatisfactory because they do not allow the logically correct comparison to be made. Ideally, the comparison that makes sense to is between an individual vaccinated and challenged with a standard inoculum and an individual who has taken the placebo and is challenged with the same inoculum. The above measures do not differentiate the number of challenges vaccinated and unvaccinated have been exposed to. some of these individuals, the majority, do not receive any inoculum and are therefore uninformative. other individuals receive 2 or more inocula and this will complicate the interpretation of the vaccine efficacies defined above [27]. Finally, it should be mentioned that we did not take into account the fact pointed by Hay et al. [28], who showed that during the growing phase of the outbreak there could be a higher proportion of individuals with higher viral loads, whereas during the declining phase of the outbreak there would be more individuals with lower viral loads. This would probably influence our results and in a future work we will address this issue.

Supporting information

S1 File. This file contains the sensitivity analysis of VE with respect to the model parameters, a list of symbols, and the VE accounting for a time-dependent force of infection.

https://doi.org/10.1371/journal.pone.0285466.s001

(PDF)

Citation: Loria J, Albani VVL, Coutinho FAB, Covas DT, Struchiner CJ, Zubelli JP, et al. (2023) Time-dependent vaccine efficacy estimation quantified by a mathematical model. PLoS ONE 18(5): e0285466. https://doi.org/10.1371/journal.pone.0285466

About the Authors:

Jennifer Loria

Roles: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Software, Validation, Visualization, Writing – original draft, Writing – review & editing

Affiliations: Instituto de Matemática Pura e Aplicada, Rio de Janeiro, Brazil, School of Mathematics, Universidad de Costa Rica, San José, Costa Rica

Vinicius V. L. Albani

Roles: Conceptualization, Formal analysis, Investigation, Methodology, Software, Validation, Visualization, Writing – original draft, Writing – review & editing

Affiliation: LAMMCA, Department of Mathematics, Federal University of Santa Catarina, Florianopolis, Brazil

Francisco A. B. Coutinho

Roles: Conceptualization, Formal analysis, Investigation, Methodology, Resources, Supervision, Validation, Writing – original draft

Affiliation: Department of Pathology, University of São Paulo, São Paulo, Brazil

Dimas T. Covas

Roles: Conceptualization, Data curation, Funding acquisition

Affiliation: Instituto Butantan, São Paulo, Brazil

Claudio J. Struchiner

Roles: Conceptualization, Investigation, Methodology

Affiliation: School of Applied Mathematics, Fundação Getúlio Vargas, Rio de Janeiro, Brazil

Jorge P. Zubelli

Roles: Formal analysis, Funding acquisition, Investigation, Methodology, Supervision, Validation, Visualization, Writing – original draft, Writing – review & editing

E-mail: [email protected]

Affiliation: Mathematics Department, Khalifa University, Abu Dhabi, UAE

ORICD: https://orcid.org/0000-0003-1581-9505

Eduardo Massad

Roles: Conceptualization, Formal analysis, Funding acquisition, Investigation, Methodology, Supervision, Validation, Writing – original draft, Writing – review & editing

Affiliations: School of Applied Mathematics, Fundação Getúlio Vargas, Rio de Janeiro, Brazil, School of Medicine, University of São Paulo and LIM01-HCFMUSP, São Paulo, Brazil

References

1. Scheppler L, De Clercq N, McGoldrick M, Dias J. Regulatory Harmonization and Streamlining of Clinical Trial Applications globally should lead to faster clinical development and earlier access to life-saving vaccines. Vaccine. 2021;.

2. Bellomo N, Bingham R, Chaplain MA, Dosi G, Forni G, Knopoff DA, et al. A multiscale model of virus pandemic: Heterogeneous interactive entities in a globally connected world. Mathematical Models and Methods in Applied Sciences. 2020;30(08):1591–1651. pmid:35309741

3. Bellomo N, Burini D, Dosi G, Gibelli L, Knopoff D, Outada N, et al. What is life? A perspective of the mathematical kinetic theory of active particles. Mathematical Models and Methods in Applied Sciences. 2021;31(09):1821–1866.

4. Delany I, Rappuoli R, De Gregorio E. Vaccines for the 21st century. EMBO molecular medicine. 2014;6(6):708–720. pmid:24803000

5. World Health Organization, Global Immunization Vision and Strategy 2006–2015; 2016.

6. Orenstein WA, Bernier RH, Dondero TJ, Hinman AR, Marks JS, Bart KJ, et al. Field evaluation of vaccine efficacy. Bulletin of the World Health Organization. 1985;63(6):1055. pmid:3879673

7. Mao HH, Chao S. Advances in vaccines. Current Applications of Pharmaceutical Biotechnology. 2019; p. 155–188.

8. Garon JR, Orenstein WA. Understanding the host–pathogen interaction saves lives: lessons from vaccines and vaccinations. Current Opinion in Immunology. 2015;36:8–13. pmid:25974089

9. Lal H, Cunningham AL, Godeaux O, Chlibek R, Diez-Domingo J, Hwang SJ, et al. Efficacy of an adjuvanted herpes zoster subunit vaccine in older adults. New England Journal of Medicine. 2015;372(22):2087–2096. pmid:25916341

10. Malkin EM, Durbin AP, Diemert DJ, Sattabongkot J, Wu Y, Miura K, et al. Phase 1 vaccine trial of Pvs25H: a transmission blocking vaccine for Plasmodium vivax malaria. Vaccine. 2005;23(24):3131–3138. pmid:15837212

11. Villa LL, Costa RL, Petta CA, Andrade RP, Ault KA, Giuliano AR, et al. Prophylactic quadrivalent human papillomavirus (types 6, 11, 16, and 18) L1 virus-like particle vaccine in young women: a randomised double-blind placebo-controlled multicentre phase II efficacy trial. The lancet oncology. 2005;6(5):271–278. pmid:15863374

12. Perrett K, Winter A, Kibwana E, Jin C, John T, Yu L, et al. Antibody persistence after serogroup C meningococcal conjugate immunization of United Kingdom primary-school children in 1999–2000 and response to a booster: a phase 4 clinical trial. Clinical Infectious Diseases. 2010;50(12):1601–1610. pmid:20459323

13. Halloran ME, Struchiner CJ, Longini IM Jr. Study designs for evaluating different efficacy and effectiveness aspects of vaccines. American journal of epidemiology. 1997;146(10):789–803. pmid:9384199

14. Halloran ME, Longini IM, Struchiner CJ, Longini IM. Design and analysis of vaccine studies. vol. 18. Springer; 2010.

15. Kaslow DC. Force of infection: a determinant of vaccine efficacy? NPJ Vaccines. 2021;6(1):1–7.

16. Vynnycky E, White R. An introduction to infectious disease modelling. Oxford University Press; 2010.

17. Massad E, Coutinho F, Burattini M, Amaku M. Estimation of R0 from the initial phase of an outbreak of a vector-borne infection. Tropical Medicine & International Health. 2010;15(1):120–126. pmid:19891761

18. Albani VVL, Velho RM, Zubelli JP. Estimating, Monitoring, and Forecasting the Covid-19 Epidemics: A Spatio-Temporal Approach Applied to NYC Data. Scientific Reports. 2021; p. 9089. pmid:33907222

19. Albani VVL, Loria J, Massad E, Zubelli JP. COVID-19 Underreporting and its Impact on Vaccination Strategies. BMC Infectious Diseases. 2021;21:1111. pmid:34711190

20. Albani V, Loria J, Massad E, Zubelli JP. The Impact of COVID-19 Vaccination Delay: A Data-Driven Modelling Analysis for Chicago and New York City. Vaccine. 2021;39(41):6088–6094. pmid:34507859

21. Amaku M, Covas DT, Coutinho FAB, Azevedo RS, Massad E. Modelling the impact of delaying vaccination against SARS-CoV-2 assuming unlimited vaccine supply. Theoretical Biology and Medical Modelling. 2021;18(1):1–11. pmid:34325717

22. Campos EL, Cysne RP, Madureira AL, Mendes GL. Multi-generational SIR modeling: Determination of parameters, epidemiological forecasting and age-dependent vaccination policies. Infectious Disease Modelling. 2021;6:751–765. pmid:34127952

23. Hodgson SH, Mansatta K, Mallett G, Harris V, Emary KR, Pollard AJ. What defines an efficacious COVID-19 vaccine? A review of the challenges assessing the clinical efficacy of vaccines against SARS-CoV-2. The Lancet Infectious Diseases. 2021;21(2):e26–e35. pmid:33125914

24. Blackwelder WC. Sample size and power for prospective analysis of relative risk. Statistics in Medicine. 1993;12(7):691–698. pmid:8511445

25. Martcheva M. An introduction to mathematical epidemiology. vol. 61. Springer; 2015.

26. Struchiner CJ, Halloran ME, Brunet RC, Ribeiro J, Massad E. Malaria vaccines: lessons from field trials. Cadernos de Saúde Pública. 1994;10:S310–S326. pmid:15042221

27. Struchiner C, Halloran M. Randomization and baseline transmission in vaccine field trials. Epidemiology & Infection. 2007;135(2):181–194. pmid:17291359

28. Hay JA, Kennedy-Shaffer L, Kanjilal S, Lennon NJ, Gabriel SB, Lipsitch M, et al. Estimating epidemiologic dynamics from cross-sectional viral load distributions. Science. 2021;373(6552):eabh0635. pmid:34083451