A Scoping Review of Three Dimensions for Long-Term COVID-19 Vaccination Models: Hybrid Immunity, Individual Drivers of Vaccinal Choice, and Human Errors

Abstract

The virus that causes COVID-19 changes over time, occasionally leading to Variants of Interest (VOIs) and Variants of Concern (VOCs) that can behave differently with respect to detection kits, treatments, or vaccines. For instance, two vaccination doses were 61% effective against the BA.1 predominant variant, but only 24% effective when BA.2 became predominant. While doses still confer protection against severe disease outcomes, the BA.5 variant demonstrates the possibility that individuals who have received a few doses built for previous variants can still be infected with newer variants. As previous vaccines become less effective, new ones will be released to target specific variants and the whole process of vaccinating the population will restart. While previous models have detailed logistical aspects and disease progression, there are three additional key elements to model COVID-19 vaccination coverage in the long term. First, the willingness of the population to participate in regular vaccination campaigns is essential for long-term effective COVID-19 vaccination coverage. Previous research has shown that several categories of variables drive vaccination status: sociodemographic, health-related, psychological, and information-related constructs. However, the inclusion of these categories in future models raises questions about the identification of specific factors (e.g., which sociodemographic aspects?) and their operationalization (e.g., how to initialize agents with a plausible combination of factors?). While previous models separately accounted for natural- and vaccine-induced immunity, the reality is that a significant fraction of individuals will be both vaccinated and infected over the coming years. Modeling the decay in immunity with respect to new VOCs will thus need to account for hybrid immunity. Finally, models rarely assume that individuals make mistakes, even though this over-reliance on perfectly rational individuals can miss essential dynamics. Using the U.S. as a guiding example, our scoping review summarizes these aspects (vaccinal choice, immunity, and errors) through ten recommendations to support the modeling community in developing long-term COVID-19 vaccination models.

1. Introduction

Most Americans have come to agree that “we will never be rid of COVID-19 in our lifetime” [1]. Although vaccines and the ongoing development of antiviral drugs are great achievements [2,3], COVID-19 has continued to wreak havoc in several ways. During the first half of 2022, more than 170,000 individuals died from COVID-19 in the U.S., amounting to over a million deaths since the emergence of the virus [4]. In addition, COVID-19 continues to exert a significant toll on the economy through disruptions in the workforce. As COVID-19 has exacerbated existing systemic challenges, the temporary matter of sickness-related absences in the early days of COVID-19 [5] is now amplified by broader issues manifested through waves of resignation. For example, many nurses or teachers are leaving the profession [6,7], with departures being even more pronounced in places that already experienced staffing shortages before COVID-19 [7]. This phenomenon, “The Great Resignation”, is not limited to healthcare or academia, and is the theme of dozens of articles that document labor shortage across occupations [8,9,10]. As a result, several efforts are underway to transition to a post-pandemic future, such as shifting work arrangements with a growing part of remote work [11].

Despite our logistical preparedness, there are reasons to be concerned when we account for two phenomena. First, and most importantly, the effectiveness of our vaccines is decreasing against new strains, which leads to (re)infections. Indeed, many Americans have already had COVID-19 multiple times [1] and the spread of immune-evasive subvariants is fueling the growth of reinfections. For instance, California went from 1 in 20 reinfections cases at the beginning of 2022 to 1 in 7 by July [12]. Currently available COVID-19 vaccines were developed for a prototype strain, but variants such as Omicron have more than 30 mutations in the spike protein of the prototype strain, which is essential for infection as it initiates the host cell entry [13]. Consequently, “available FDA-approved and -authorized COVID-19 vaccines are less effective against currently circulating virus variants than against previously circulating strains of virus” [14]. This effect is demonstrated since most infections in the U.S. as of June 2022 are attributed to two Omicron sublineages (BA.4 and BA.5) [14]. Second, the problem is compounded by our shifting responses and attitudes to the pandemic. A survey in June 2022 reported that half of U.S. adults wear a face mask “when away from home, the lowest since the start of the pandemic” [15]. Another survey showed that the proportion of people bothered when others do not mask in public fell to an all-time low, at 18% among Republicans and 52% among Democrats [16]. On the one hand, we see a diminishing use of face masks, lower vaccine effectiveness, and a lower intended receipt of COVID boosters [17]. On the other hand, we likely need annual vaccine composition updates [18]. Surveys in Jordan show that less than 1 in 5 persons would take an annual booster [19], while about 2 in 3 Canadians [20] and more than 4 in 5 German-speaking adults would be willing [21]. Results in the U.S. are mixed, as surveys were conducted across sub-populations with different profiles [22].

The Modeling & Simulation (M&S) approach of Agent-Based Modeling (ABM) has been used to model COVID-19 since the second half of 2020, because of the approach’s ability to track individual agents who can have different attributes, different decision-making processes, and be exposed to different environments [23]. This ability to implement heterogeneity is indeed essential for COVID-19 since it is found in individual risk factors (e.g., age, hypertension, diabetes), contact patterns (e.g., social networks in the community or work settings), behaviors (e.g., willingness to be vaccinated), and spatial aspects (e.g., access to healthcare) [24,25,26]. Although ABMs for COVID-19 have been extremely varied in purpose and design [27], their design broadly followed three stages in the pandemic [28]: ABMs started with a handful of stages (e.g., susceptible, exposed, infected, recovered) and examined which non-pharmaceutical interventions would have the strongest effect [29,30,31], then ABMs were created to support vaccine-related studies (e.g., who should be vaccinated, where to place the centers and how to staff them) [32,33,34] and lastly, the current wave of studies where repeated boosters account for waning immunity [35,36].

Truszkowska and colleagues provided one of the first models in which immunity was totally and gradually lost over the course of six months [35]. This approach included vaccine hesitancy, which was captured by assuming that only a percentage of the population would get vaccinated [35]. More recently, Kelly et al. proposed a comprehensive model that accounts for the waning vaccine induced immunity and naturally acquired immunity, as well as seasonal patterns in infection, and the emergence of annual or biannual variants [36]. Again, vaccine hesitancy was modeled population-wide by assuming either a high booster intake (85% to 98%) of individuals 50 years of age or higher with comorbidities, or modeled with a reduced intake of 50%. These models concluded that a high adherence to frequent booster doses is necessary to avoid future outbreaks and prevent a burden on the healthcare system—a conclusion shared by studies using differential equations [37,38]. Since adherence to frequent boosters is the key but it was only modeled by assuming a percentage across wide sub-populations, it is important for future ABMs to address this research gap by detailing how individuals decide to get vaccinated as a function of sociodemographic, health, psychological, and information-related variables.

Several open-source COVID-19 ABMs can be reused and extended by the research community, including COVASIM [39], OpenABM-COVID19 [40], and COMOKIT [41]. These packages and other individual-based epidemiology simulation kits [42,43] allow us to create a virtual population that reflects several demographic indicators (e.g., age, sex) and their impact on disease spread (e.g., age-linked disease progression and mortality). Packages such as COVASIM also embed each individual in several networks (home, community, work, school) based on age (e.g., children go to school, adults to work). Through revisions, the current version (3.1.2 from 16 January 2022) also captures “co-circulating variants, imperfect immune memory, and multiple vaccines” [44], tracks the properties of variants (relative transmissibility compared to wild type, relative severity, relative immunity, variant-specific protection of vaccines), and allows users to add custom variants. The popularity and reuse of such packages is demonstrated by their applications to multiple settings such as Australia [45,46,47], the UK and/or US [48,49,50], Vietnam [51], or Italy [52], at both the country- and city-scale [53]. Our scoping review thus emphasizes elements that are commonly missing across these packages in order to have a broad representation of the current technical capabilities in the field.

The main contribution of our scoping review is to support the development of the next generation of ABMs by examining three aspects: vaccine hesitancy, joint occurrence of natural- and vaccine-induced immunity, and the inclusion of human errors in decision-making. Specifically, we synthesize the current evidence base and identify research gaps, leading to 10 recommendations on aspects to include in future models, alongside their operationalization. The recommendations are featured prominently (as R1, R2, etc.) throughout the sections.

The review is organized as follows. Section 2 starts with the concept of immunity and its representation in models. Then, Section 3 summarizes the evidence base regarding strong drivers of vaccinal choice. We cover the last dimension in Section 4 by explaining how individuals are not perfect and can engage in several types of errors during their decision-making processes. The last two aspects have received less attention in modeling packages, hence these two sections include detailed suggestions for inclusion in future models. Finally, Section 4 discusses a list of limitations inherent to the modeling process and limited current knowledge on variants and individual choices.

2. Immunity: Variants, Waning Effect, and Hybrid Cases

Early models of COVID-19 that included vaccines were developed to help plan ahead. For instance, models would determine the most effective locations to setup vaccination centers or prioritize certain sub-populations that should be first administered vaccinations. In the simulated timespans of these models, it was adequate to consider that individuals would have a constant level of immunity once recovered or vaccinated, and that the dominant strain at the time was the only one. As we transition into a more long-term perspective, we need to account for changes in the level of immunity, including the cumulative effect of vaccination and recovery (particularly since most Americans have had COVID-19 [54]), as well as the ongoing emergence of variants.

As shown in Table 1, recent models have handled immunity differently. Most models use a gradual decay, with arguments in favor of a Gamma distribution rather than an exponential one (to reflect that the initial immunity remains at a high level for several weeks instead of declining immediately). This is represented either by the direct use of a fitted Gamma distribution, or by the ‘linear chain trick’ which consists of use a sequence of exponentially distributed decays (e.g., agents go through stages V₁, …, V₅ back-to-back) [55,56].

The models either did not account for the synergistic effect of natural infections together with vaccinations, or created a dedicated stage without specifying the corresponding transitions. For example, a model considered that a vaccinated person who became infected lost the vaccine-induced immunity and only gained natural immunity upon recovery, hence making the two forms of immunity mutually exclusive [58]. However, the hybrid case resulting from natural infection and vaccination confers the most robust and durable immunity [60,61,62]. Studies on immunology on different variants and various countries have found that hybrid immunity was protective against reinfection and severe disease outcomes [63,64,65,66]. As summarized by Bates and colleagues, “the additional antigen exposure from natural infection substantially boosts the quantity, quality, and breadth of humoral immune response regardless of whether it occurs before or after vaccination” [67]. Newer models are starting to include this effect, for example by assuming that a prior infection counts as a single vaccination, hence the ‘actual’ vaccination has the effect of a booster [68]. A recent retrospective cohort study in Sweden [69] confirmed several of the numbers in the OpenCOVID model and sheds light on the effect of the synergistic effect of hybrid immunity. The study showed that natural immunity conferred a 95% lower risk of infection and 87% lower risk of hospitalization for up to 620 days, which closely aligns with OpenCOVID. The study further established that hybrid immunity provided an additional 58% (for one dose) or 66% (for two doses) reduction in infection compared to natural immunity, and lasted up to 9 months with mixed findings on attenuation. Although numerous knowledge gaps remain [70], findings are starting to emerge regarding mediating factors in the immune response created by receiving a vaccine after a natural infection. In particular, a study using the U.S. Military Health System further suggested that the timing between a prior infection and vaccination was highly predictive of immune response, while prior disease severity did not play a role [62].

The prevalence of hybrid immunity is rising quickly; a cross-sectional study of blood donors in the U.S. found an increase from 0.7% of blood samples with hybrid immunity in January 2021 to 18.9% by December [71]. A review by Bhiman and Moore listed several scenarios as potential consequences of this ongoing increase in the hybrid immune population [72]; the inclusion of such scenarios in future computational models may improve our ability at forecasting population-wide trends.

Few models examined the continuous emergence of new variants. For example, in the OpenCOVID model [36], each new variant retained the same severity and was assumed to be 25% more infective as well as 25% more able to evade immunity than its predecessor. Another model considered that new variants would be 3.5-fold as virulent compared to the wild-type [55]. Although models assumed a regular emergence of new variants, illustrative examples [73] explain that the emergence depends on the number of circulating cases (I) and susceptible individuals (S) as well as the rate of infection (β) and the chance of each infection to create an escape variant by mutation (μ). Thus, the time until the first escape mutation appears is approximated as an exponential of mean

(1)$\frac{1}{SI\mathsf{\beta }\mathsf{\mu }}$

Based on sensitivity analyses on the models, scholars determined that the primary effect of vaccines was to block infections (80%) and secondarily of avoiding severe disease outcomes (the remaining 5%) [36]. Simulation results from COVASIM [55] on vaccine design further suggest that, going forward, creating a broadly neutralizing vaccine would be more beneficial (40% reduction in death) than a more durable vaccine (10% reduction in death). However, the effect would be highly reliant on releasing the vaccine at the right time, hence the preferred solution would be a vaccine that is both durable and broadly neutralizing (65% reduction).

Given the evidence base, we recommend that future models include:

(R1) a gradual decay in immunity by using a Gamma distribution, for instance through the creation of intermediate states (i.e., the ‘linear chain trick’).

(R2) hybrid immunity, which confers the most robust and durable immunity; this may be approximated by counting a prior infection as having the effect of a booster dose.

(R3) the ongoing emergence new variants that partially evade immunity, with a timing depending on circulating cases rather than at fixed time intervals.

3. Vaccinal Choice

3.1. Drivers of Vaccinal Choice

Numerous studies have been conducted to identify socio-demographic factors related to an increased or decreased willingness to take vaccines. Using the U.S. population as a guiding example, sample studies relevant to this context are listed in Table 2. Note that models of COVID-19 have not yet accounted for the many drivers of individual vaccination choices, instead opting to either use a population-wide percentage of vaccinated individuals [35] or account for age [33,52]. This occurred even in highly configurable frameworks where agents execute detailed plans based on their health, demographic, and organizational roles [74]. Two models accounted for the level of caution (rising with case numbers), sense of safety (rising with vaccination), or perceived vaccination risk, but at the population-level rather than via individuals [75,76]. Given Table 2, we recommend that future models include:

(R4)multiple socio-demographic factors (e.g., age, sex, race and ethnicity, annual income, college degree, key comorbidities, political party) that were found to strongly predict vaccinal choices.

3.2. Capturing Drivers in a Model: The Role of Data and Sequential Agent Initialization

When agents possess multiple traits, the values may not be initialized independently, one at a time (Figure 1a). Indeed, values can be correlated: for instance, comorbidities tend to raise with age. Multiple correlations can exist, as is the case with the dependency of annual income both on employment and on educational attainments. Considering that every factor may depend on all other factors would be infeasible, as it leads to circular dependencies. Even a scheme where each factor depends on all those previously initialized can be practically infeasible, since it would be arduous to find a nationally representative dataset that covers all factors (Figure 1b). Consequently, an initialization scheme will have to retain some of the dependencies based on the data available [83] (Figure 1c). In this subsection, we illustrate how these considerations can be addressed by using U.S. data.

Consider that we aim to create a virtual population of agents with multiple target socio-demographic factors. To start, we identify a data source that contains as many as possible of the important attributes together. In our case, the US Census Current Population Survey (CPS) 2019 [84] can serve to create a pool of agents who are jointly assigned an age, income, race and ethnicity, and sex. That is, all four attributes can be assigned simultaneously, thus capturing their interdependencies in the population. Then, additional data sources will need to be found to cover the remaining aspects. Note that these sources need to also include some of the previously initialized attributes, otherwise dependencies cannot be adequately reflected. For example, if a dataset contains the prevalence of prior COVID-19 infection based only on data per county, then there would be no way to link the data to individuals characterized by age, race and ethnicity, annual income, or sex. In the case of the U.S., several data sources can be used as shown in Table 3.

Finally, there can be a gap between the factors used in studies on social determinants of health and the content of national surveillance systems. For example, studies reported that comorbidities were a strong determinant of vaccinal choice (Table 2), but different surveillance systems may track different types of specific comorbidities, hence there will be a need to translate a high-level concept into specific conditions and work across additional sources. A pitfall for modelers is to misunderstand the notion of comorbidities. Since comorbidities refer to other diseases present in a patient, death reporting systems may show pneumonia as a prevalent comorbidity, hence modelers could (erroneously) use pneumonia as an agent’s attribute for comorbidity. However, pneumonia is a complication of COVID-19, rather than an underlying medical condition that increases the risk profile of a person if they get infected. It is thus important to carefully translate high-level constructs into specific ones given the context. In this example, comorbidities as risk factors may include hypertension and diabetes since they are commonly found in patients hospitalized with COVID-19 worldwide [85,86], while noting that the evidence-base is weaker for hypertension once we control for other risk factors such as age [87]. Dependencies matter across comorbidities as well, since diabetic agents have an elevated risk for hypertension; Table 3 thus initializes diabetes and then hypertension.

In sum, a satisfactory initialization of several attributes in the agents will require:

(R5) the identification of several data sources that share factors to allow data linkages.

(R6) an initialization scheme that satisfies multiple core dependencies between factors by initializing them in a specific batch order.

(R7) a careful contextual translation from high-level constructs onto specific variables offered in different surveillance systems and reports.

Table 3

Creation of a large pool of agents by first initializing four attributes together, then determining the remaining five by accounting for several interdependencies.

			1st Wave of Initialization of Four Categorical Features Jointly [80]
	Ref	Factors	Age	Income	Race and Ethnicity	Sex
2nd wave of initialization	[88,89]	Bachelor’s degree	✓		✓	✓
	[90,91]	Political party 1			✓	✓
	[92]	Diabetes	✓		✓	✓
	[93,94]	Hypertension 2	✓			✓
	[95]	≥1 dose of vaccine	✓		✓

¹ Two steps are involved: first we apply a population-wide probability of voting (agents who do not vote are ‘undecided’), then we subdivide agents into Democrats and Republicans based on sex, race and ethnicity. ² Hypertension is created after agents have been assigned a level of diabetes, to ensure that diabetic agents have twice the risk for hypertension [93].

Note that a model is necessarily a simplification and data availability may govern some of these simplifications. For example, a modeling team may not have access to data for some of the strong drivers of vaccinal choice (Table 2), hence agents may not possess the corresponding attributes. For instance, sociodemographic information may not include occupations, hence a model may lack the notion that certain professions are at higher risk (e.g., medical professionals) or face specific obligations (e.g., annual vaccine).

3.3. Extending an Existing Package: Example in COVASIM

Although COVID-19 modeling packages are often open source, it may not be desirable to alter their initialization of agents by changing the code to add several factors and dependencies. Indeed, editing the source code of a package (known as ‘forking’) results in creating a new local version for a modeling team, which complicates the possibility of benefiting from updates in future package releases. Consequently, we recommend:

(R8) to build on top of existing modeling packages whenever possible, so that future releases (e.g., an optimized COVASIM) can be conveniently used by the team

Expanding the characteristics of agents without altering a package’s initialization process can be achieved by matching, illustrated in Figure 2 for COVASIM. First, the modeling package would follow its normal procedure to initialize a set S of agents of the desired size. In parallel, modelers would create a larger pool P of agents with their required characteristics and dependencies (Section 3.2). Second, we augment the characteristics of each agent in S by finding a similar agent (i.e., a ‘match’) in P. This requires efficient data structures, as poor implementations could significantly slow down the process of agent creation. For example, an inefficient approach for an agent in S would be to scan the entire population P, each time computing the distance with an agent in P, then finally selecting the most similar one. Once a match in P has been found, it must be removed, which should also be done carefully to avoid triggering massive data movements. Research in discrete simulations has often used hierarchical data structures (e.g., two or three tiers) to optimize operations [96]. In this case, we need a structure that allows fast lookups to identify a match, hence the attributes offered by the package (e.g., age, sex) can be used as keys in a dictionary (Figure 2). We also need a structure that supports quick removals of an individual to move onto the next one, hence a queue.

4. Human Errors in Decision-Making

4.1. Limitations of Observations and Reflections

There is a broad tendency to create ABMs with highly rational agents [97], which is even more the case in COVID-19 research as models are often grounded in compartmental techniques designed for epidemiology. However, humans are neither mechanical objects nor omniscient, hence individuals neither form homogeneous responses to vaccines (e.g., with a set percentage of the entire population taking vaccines) nor engage in a comprehensive inventory of all relevant parameters to achieve an optimum (e.g., by maximizing a utility pay-off). Reusing the analogy of Giubilini and Savulescu, refusing vaccination is similar to refusing the use of seat belts when driving [98]: given the balance of extremely rare side effects and the cost of delayed or absent vaccination [99], this decision would not happen if individuals used the evidence-base to maximize their own benefits. Indeed, cognitive sciences show that humans only capture a small portion of the available information [100], sometimes making errors in storing this information, and ultimately interpreting it based on their own heterogeneous beliefs [101]. Prior works in modeling COVID-19 [28,29,33] and perceptual uncertainties in ABMs [102] have already accounted for some of these aspects when creating cognitive social simulations [103] that reflect the imperfect, heterogeneous decisions that individuals make on vaccination. While some works introduce the notion of imperfect decisions in COVID-19 by accounting for uncertainty about individual health states [104], research shows that there are at least three sources of individual errors to reflect that individuals use social information sub-optimally [105] when observing their peers (with respect to infection, vaccine choices, or death). That is, an agent may only observe some of its peers (insufficient samples) and/or pay attention to only some of their information (superficial observations) and/or does not sufficiently reflect on the evidence collected (limited ability). Consequently, we recommend to:

(R9) shift from purely rationale/mechanistic COVID-19 models of human decision-making onto cognitive social simulations that account for imperfection in information capture, its storage, and its use for behavior change.

4.2. Operationalizing Human Errors in a Model: The Role of Machine Learning as a Filter

At each simulation tick, each agent can observe the behavior of its peers (e.g., whether to wear a mask, engage in social distancing, wash hands, vaccinate) along with their characteristics. Conceptually, the set of observations can simply be reduced to account for insufficient samples and superficial observations, as shown in Figure 3. However, the main technical difficulty is to ensure that an agent can change based on these observations. The development of such adaptive skills requires a learning mechanism. As we recently discussed [106], hybrid agent-based models can deal with adoption behavior in different ways, such as by giving their observations as input to a machine learning model. That is, an agent observes by gathering a data table, reflects by deriving a machine learning model (e.g., why do my friends wear masks?), and acts by applying the model to its own characteristics (e.g., my friends with comorbidities wear a mask and I have comorbidities hence I will wear masks). We thus recommend to:

(R10) enable agents to change behavior not only in response to policies (e.g., lockdown) or instincts (e.g., disease avoidance), but also by learning from their individual contexts.

5. Discussion

5.1. Overview

In the absence of global vaccination, simulation experts have predicted continuous waves of COVID-19 infections [107]. This is now a reality, and most Americans understand that COVID-19 is here to stay [1]. As mutations and natural selection continue to occur, new variants emerge and vaccines need to be continuously redeveloped [14] and adopted by the population [17,18,19,20,21,22]. In the U.S., the federal government has purchased 66 million doses of bivalent boosters that target BA.4 and BA.5 Omicron subvariants [108]. Although the Omicron subvariant BA.5 has been called ‘the worst version’, Bruce Y. Lee explained in a popular piece that “it’s the worst version of what’s been getting progressively worse, and you never know when another even worse version will emerge” (emphasis added). [109] Since COVID-19 precautions are going away [15,16], the virus will continue to spread and subvariants will emerge. Expecting the problem to just disappear by believing variants will get gradually weaker may amount to wishful thinking, “like expecting different animal and plant species to get weaker over time.” [108]

Since Agent-Based Modeling has been the dominating approach for COVID-19 in recent times [110], our scoping review sought to guide the development of future ABMs by examining three dimensions that are essential to effectively pivot into a long-term view. We covered emerging considerations on immunity (e.g., decays, hybrid cases), the growing evidence base on determinants of vaccinal choice and the ability to use them in simulation, and the possibility of shifting towards cognitive social simulations to better capture human decision-making. While recent reviews and recommendations have targeted national health systems and governments [111,112], our ten recommendations are directed at the community on modeling and simulation.

However, ongoing discussions in the scientific community and changes in the evidence base may lead to additional considerations, as discussed below.

5.2. Limitations

Our scoping review stressed the importance of psychological and information variables, suggesting that they can be included through a sequential initialization process or by engaging in hybrid modeling (e.g., by using machine learning to create adaptive agents). However, COVID-19 models may ultimately become too complex, given the pursuit to “covering the full behavioral and social complexity of societies under pandemic crisis” [113] through an ever-expanding set of rules and states. The design of COVID-19 models going forward may thus benefit from a shift from a monolithic piece onto a set of integrated sub-models. This shift would come with its own challenges, as risks for model integration have been discussed extensively elsewhere [114]. However, it may be simpler to develop and reuse dedicated modules, since many COVID-19 models have significant overlaps [115]. For example, this would facilitate the integration of a sub-model dedicated to capturing the perceived utility of certain actions (e.g., getting a vaccine) for an agent, such as modeling vaccination as a function of personal beliefs (e.g., via the Theory of Planned Behavior [116]) or through an emphasis on government accountability [117].

This paper focused on improving models of COVID-19. However, the actions taken to prevent COVID-19 will also impact other diseases. For instance, facemasks and social distancing can reduce other airborne diseases, while the prioritization of hospital resources for COVID-19 impacts other operations [118]. In addition, there is a growing interest in coinfection, particularly for coinfections of COVID-19 and bacteria [119], such as tuberculosis [120]. Consequently, COVID-19 cannot always be designed or expected to be utilized in isolation. Emerging models have started to include COVID-19 alongside another disease [121]. Future COVID-19 models may thus have to be integrated alongside modules for other diseases. Given this requirement, there may be a tradeoff between the sophistication of a COVID-19 model and the ability to integrate it alongside other disease models.

Author Contributions

P.J.G. wrote the first draft of the manuscript, with input from J.T.B. and G.G.B., who jointly performed the review in Section 3 and identified data sources. All authors have read and agreed to the published version of the manuscript.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Not applicable.

Conflicts of Interest

The authors declare no conflict of interest.

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Figures and Tables

Figure 1. A simple initialization scheme (a) would ignore several important dependencies in the data, but capturing a large number of dependencies (b) may be infeasible given data limitations. A practical approach is thus to initialize the agents’ attributes in consecutive waves (c).

Figure 2. The attributes of COVASIM agents can be expanded by matching them with virtual individuals from a large pool generated from nationally representative U.S. datasets (Table 3). The pool should be larger than the COVASIM population to ensure that each retrieval request for a matching profile can be satisfied. Icons from User Insights and Domp Icon at icon-icons.com, CC Attribution license (CC BY 4.0).

Figure 3. Each agent observes the vaccination decision and attributes of its peers. The initially perfect observations are altered by perceptual limitations and errors, thus producing an observation set with fewer peers and/or fewer attributes. Icons from User Insights and Coronavirus at icon-icons.com, CC Attribution license (CC BY 4.0).

Table 1

Handling of variants and (temporary) immunity in recent COVID-19 models.

		Handling of Immunity
Ref	Variants	Natural	Vaccine	Hybrid
[35]	Delta variant only	Lasts 180 days	Impact on 5 parameters (e.g., death, infection, hospitalization, transmission, asymptomatic) defined via piecewise linear functions. Peak reached 2 weeks after one shot, remains constant for 8 months, then linear decay for 6 months. The level of immunity for each linear segment depended on the vaccine used. Booster restores peak vaccination benefit in 1 day.	Exists but unspecified
[36]	Emerge ahead of the winter, every 6 or 12 months	Initial peak at 95%, exponential decay to 20% in 600 days	Initial peak at 85%, exponential decay to 15%, half-life 105 days. Higher peak after a booster, but same decay.	None
[57]	Randomly appear 4/6/10 months after past variant	2-part exponential decay, with half-life and duration parameters fit to data. Neutralization level depends on variant and vaccine.		None
[58]	Omicron variant only	Gamma distribution set either to 9 months (shape 7 and scale 39.11) or one year (shape 3.7 and 98.65).		None
[59]	Omicron variant only	Protection against same variant has exponential duration of mean 1/900, then no protection	After an average of 6 months since the second dose or a booster, individuals transition into a ‘waned vaccine effectiveness’ status.	Exists but unspecified
[55]	Unspecified (2021 data)	Delayed gamma-distributed temporary immunity with mean 350 (for vaccines) and 242 days (for recovery)		None
[56]	Delta variant only	Full immunity for entire duration of simulation (March 2020–Nov. 2021)	10% of individuals do not lose immunity, 10% have little protection, the remaining 80% get temporary protection by moving through a series of compartments instead of a single one (which would cause an exponential decay) hence using a Gamma distribution with peak efficacy of 92% and decay to 35% over 6 months	None

Table 2

Propensity of U.S. individuals in specific socio-demographic categories to get vaccinated or not, and strength of the effect as reported in the references. We counted a reduction in refusal as being equivalent to an increase in acceptance.

Determinant	Category	Get Vaccine	Effect Strength	References
Age	18–29	No	Strong	[77,78]
	30–49	Yes		[79]
	50–64	Yes
	65+	Yes
Sex	Male	No		[77]
Ethnicity	Non-Hispanic White	Yes		[79]
	Non-Hispanic Black	No		[77,80,81]
	Hispanic	No
	Asian/Pacific Islander	Yes		[80]
	Other	No		[79,80]
Employment	Unemployed	Yes	Weak	[77]
	Employed (full or part time)	No	Strong
Education level	Bachelor’s degree or more	Yes		[79,80,82]
	No college degree	No		[80,82]
Annual income	<25,000	No		[81]
	≥70,000	Yes		[80]
Comorbidities	Yes	Yes
Political party	Republican	No		[79]
	Democrat	Yes
Prior COVID-19 infection	Yes	Yes		[77]
Know someone who died	Yes	Yes
Religion	Catholic	No	Weak

References

1. Axios/Ipsos. Most Americans Agree We Will Never Fully Be Rid of COVID-19. Last Updated: 2022-07-19. Available online: https://www.ipsos.com/en-us/news-polls/axios-ipsos-coronavirus-index (accessed on 2 August 2022).

2. Wen, W.; Chen, C.; Tang, J.; Wang, C.; Zhou, M.; Cheng, Y.; Zhou, X.; Wu, Q.; Zhang, X.; Feng, Z. et al. Efficacy and safety of three new oral antiviral treatment (molnupiravir, fluvoxamine and Paxlovid) for COVID-19: A meta-analysis. Ann. Med.; 2022; 54, pp. 516-523. [DOI: https://dx.doi.org/10.1080/07853890.2022.2034936] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/35118917]

3. Liu, Y.; Ye, Q. Safety and Efficacy of the Common Vaccines against COVID-19. Vaccines; 2022; 10, 513. [DOI: https://dx.doi.org/10.3390/vaccines10040513] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/35455262]

4. National Center for Health Statistics. Daily Updates of Totals by Week and State. Last Updated: 2022-08-02. Available online: https://www.cdc.gov/nchs/nvss/vsrr/covid19/index.htm (accessed on 2 August 2022).

5. Lyttelton, T.; Zang, E. Occupations and Sickness-Related Absences during the COVID-19 Pandemic. J. Health Soc. Behav.; 2022; 63, pp. 19-36. [DOI: https://dx.doi.org/10.1177/00221465211053615] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/35100514]

6. Thurman, W.; Heitkemper, E.; Johnson, K.E. Challenges Facing Public Health Nursing Faculty in the United States: COVID-19 as a Catalyst for Change. Am. J. Public Health; 2022; 112, pp. S314-S320. [DOI: https://dx.doi.org/10.2105/AJPH.2022.306819] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/35679541]

7. Carver-Thomas, D.; Burns, D.; Leung, M.; Ondrasek, N. Teacher Shortages During the Pandemic: How California Districts Are Responding; Learning Policy Institute: Palo Alto, CA, USA, 2022; [DOI: https://dx.doi.org/10.54300/899.809]

8. Jiskrova, G.K. Impact of COVID-19 pandemic on the workforce: From psychological distress to the Great Resignation. J. Epidemiol. Community Health; 2022; 76, pp. 525-526. [DOI: https://dx.doi.org/10.1136/jech-2022-218826]

9. Schmid, S.R.; Melkote, S.N. Manufacturing and the Great Resignation. Mech. Eng.; 2022; 144, pp. 38-43. [DOI: https://dx.doi.org/10.1115/1.2022-MAY3]

10. Gavin, M.; Poorhosseinzadeh, M.; Arrowsmith, J. The transformation of work and employment relations: COVID-19 and beyond. Labour Ind.; 2022; 32, pp. 1-9. [DOI: https://dx.doi.org/10.1080/10301763.2022.2038353]

11. Pronk, N.P. Addressing COVID-19 Disruptions at the Workplace While Preparing for a Postpandemic Future. ACSM’S Health Fit. J.; 2022; 26, pp. 52-55. [DOI: https://dx.doi.org/10.1249/FIT.0000000000000764]

12. Varizi, A.; Beamish, R. COVID in California: Reinfections account for 1 in 7 coronavirus cases in state. San Francisco Chronicles, 1 August 2022. Available online: https://www.sfchronicle.com/bayarea/article/COVID-in-California-Latest-updates-about-the-17342154.php (accessed on 2 August 2022).

13. Behl, T.; Kaur, I.; Sehgal, A.; Singh, S.; Sharma, N.; Anwer, K.; Makeen, H.A.; Albratty, M.; Alhazmi, H.A.; Bhatia, S. et al. There is nothing exempt from the peril of mutation—The Omicron spike. Biomed. Pharmacother.; 2022; 148, 112756. [DOI: https://dx.doi.org/10.1016/j.biopha.2022.112756] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/35228064]

14. U.S. Food & Drug Administration (FDA)Office of Vaccines Research and Review. Fall 2022 COVID-19 Vaccine Strain Composition Selection Recommendation. Memorandum. 30 June 2022. Available online: https://www.fda.gov/media/159597/download (accessed on 3 August 2022).

15. Saad, L. Roundup of Gallup COVID-19 Coverage. Updated 06/06/22. Available online: https://news.gallup.com/opinion/gallup/308126/roundup-gallup-covid-coverage.aspx (accessed on 3 August 2022).

16. Tyson, A. 57% of Americans Say Masks Should Be Required on Airplanes and Public Transportation. Pew Research Center. Last updated: 05/11/22. Available online: https://www.pewresearch.org/fact-tank/2022/05/11/57-of-americans-say-masks-should-be-required-on-airplanes-and-public-transportation/ (accessed on 3 August 2022).

17. Lennon, R.P.; Block, R.; Schneider, E.C.; Zephrin, L.; Shah, A. The African American Research Collaborative 2021 COVID Group. Underserved population acceptance of combination influenza-COVID-19 booster vaccines. Vaccine; 2021; 40, pp. 562-567. [DOI: https://dx.doi.org/10.1016/j.vaccine.2021.11.097] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/34903376]

18. Marks, P.; Woodcock, J.; Califf, R. COVID-19 Vaccination—Becoming Part of the New Normal. JAMA; 2022; 327, 1863. [DOI: https://dx.doi.org/10.1001/jama.2022.7469]

19. Abuhammad, S.; Khabour, O.F.; Alzoubi, K.H.; Hamaideh, S.; Alzoubi, B.A.; Telfah, W.S.; El-Zubi, F.K. The public’s attitude to and acceptance of periodic doses of the COVID-19 vaccine: A survey from Jordan. PLoS ONE; 2022; 17, e0271625. [DOI: https://dx.doi.org/10.1371/journal.pone.0271625]

20. Reifferscheid, L.; Lee, J.S.W.; MacDonald, N.E.; Sadarangani, M.; Assi, A.; Lemaire-Paquette, S.; MacDonald, S.E. Transition to endemic: Acceptance of additional COVID-19 vaccine doses among Canadian adults in a national cross-sectional survey. medRxiv; 2022; [DOI: https://dx.doi.org/10.1186/s12889-022-14025-8] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/36104675]

21. Weitzer, J.; Birmann, B.M.; Steffelbauer, I.; Bertau, M.; Zenk, L.; Caniglia, G.; Laubichler, M.D.; Steiner, G.; Schernhammer, E.S. Willingness to receive an annual COVID-19 booster vaccine in the German-speaking DA-CH region in Europe: A cross-sectional study. Lancet Reg. Health-Eur.; 2022; 18, 100414. [DOI: https://dx.doi.org/10.1016/j.lanepe.2022.100414]

22. Lee, R.C.; Hu, H.; Kawaguchi, E.S.; Kim, A.E.; Soto, D.W.; Shanker, K.; Klausner, J.D.; Van Orman, S.; Unger, J.B. COVID-19 booster vaccine attitudes and behaviors among university students and staff in the United States: The USC Trojan pandemic research Initiative. Prev. Med. Rep.; 2022; 28, 101866. [DOI: https://dx.doi.org/10.1016/j.pmedr.2022.101866] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/35785408]

23. Tao, Y.; Yang, C.; Wang, T.; Coltey, E.; Jin, Y.; Liu, Y.; Jiang, R.; Fan, Z.; Song, X.; Shibasaki, R. et al. A Survey on Data-Driven COVID-19 and Future Pandemic Management. ACM Comput. Surv.; 2022; [DOI: https://dx.doi.org/10.1145/3542818]

24. Bilal, U.; Mullachery, P.H.; Schnake-Mahl, A.; Rollins, H.; McCulley, E.; Kolker, J.; Barber, S.; Roux, A.V.D. Heterogeneity in Spatial Inequities in COVID-19 Vaccination Across 16 Large US Cities. Am. J. Epidemiol.; 2022; 191, pp. 1546-1556. [DOI: https://dx.doi.org/10.1093/aje/kwac076]

25. Masters, N.B.; Zhou, T.; Meng, L.; Lu, P.-J.; Kriss, J.L.; Black, C.; Omari, A.; Boone, K.; Weiss, D.; Carter, R.J. et al. Geographic Heterogeneity in Behavioral and Social Drivers of COVID-19 Vaccination. Am. J. Prev. Med.; 2022; [DOI: https://dx.doi.org/10.1016/j.amepre.2022.06.016]

26. Saldaña, J.; Scoglio, C. Influence of heterogeneous age-group contact patterns on critical vaccination rates for herd immunity to SARS-CoV-2. Sci. Rep.; 2022; 12, 2640. [DOI: https://dx.doi.org/10.1038/s41598-022-06477-0] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/35173229]

27. Lorig, F.; Johansson, E.; Davidsson, P. Agent-Based Social Simulation of the Covid-19 Pandemic: A Systematic Review. J. Artif. Soc. Soc. Simul.; 2021; 24, 5. [DOI: https://dx.doi.org/10.18564/jasss.4601]

28. Giabbanelli, P.J.; Badham, J.; Castellani, B.; Kavak, H.; Mago, V.; Negahban, A.; Swarup, S. Opportunities and Challenges in Developing COVID-19 Simulation Models: Lessons from Six Funded Projects. Proceedings of the 2021 Annual Modeling and Simulation Conference (ANNSIM); Fairfax, VA, USA, 19–22 July 2021; [DOI: https://dx.doi.org/10.23919/annsim52504.2021.9552089]

29. Li, J.; Giabbanelli, P.J. Identifying Synergistic Interventions to Address COVID-19 Using a Large Scale Agent-Based Model. Proceedings of the International Conference on Computational Science; Krakow, Poland, 16–18 June 2021; pp. 655-662. [DOI: https://dx.doi.org/10.1007/978-3-030-77967-2_54]

30. Colomer, M.À.; Margalida, A.; Alòs, F.; Oliva-Vidal, P.; Vilella, A.; Fraile, L. Modelling the SARS-CoV-2 outbreak: Assessing the usefulness of protective measures to reduce the pandemic at population level. Sci. Total Environ.; 2021; 789, 147816. [DOI: https://dx.doi.org/10.1016/j.scitotenv.2021.147816]

31. Ghaffarzadegan, N.; Childs, L.M.; Täuber, U.C. Diverse Computer Simulation Models Provide Unified Lessons on University Operation during a Pandemic. BioScience; 2020; 71, pp. 113-114. [DOI: https://dx.doi.org/10.1093/biosci/biaa122]

32. Zhou, S.; Zhou, S.; Zheng, Z.; Lu, J. Optimizing spatial allocation of COVID-19 vaccine by agent-based spatiotemporal simulations. GeoHealth; 2021; 5, e2021GH000427. [DOI: https://dx.doi.org/10.1029/2021GH000427] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/34179672]

33. Li, J.; Giabbanelli, P. Returning to a Normal Life via COVID-19 Vaccines in the United States: A Large-scale Agent-Based Simulation Study. JMIR Med. Inform.; 2021; 9, e27419. [DOI: https://dx.doi.org/10.2196/27419] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/33872188]

34. Romero-Brufau, S.; Chopra, A.; Ryu, A.J.; Gel, E.; Raskar, R.; Kremers, W.; Anderson, K.S.; Subramanian, J.; Krishnamurthy, B.; Singh, A. et al. Public health impact of delaying second dose of BNT162b2 or mRNA-1273 covid-19 vaccine: Simulation agent based modeling study. BMJ; 2021; 373, n1087. [DOI: https://dx.doi.org/10.1136/bmj.n1087] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/33980718]

35. Truszkowska, A.; Zino, L.; Butail, S.; Caroppo, E.; Jiang, Z.P.; Rizzo, A.; Porfiri, M. Predicting the Effects of Waning Vaccine Immunity Against COVID-19 through High-Resolution Agent-Based Modeling. Adv. Theory Simul.; 2022; 5, 2100521. [DOI: https://dx.doi.org/10.1002/adts.202100521] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/35540703]

36. Kelly, S.L.; Le Rutte, E.A.; Richter, M.; Penny, M.A.; Shattock, A.J. COVID-19 vaccine booster strategies in light of emerging viral variants: Frequency, timing, and target groups. medRxiv; 2022; [DOI: https://dx.doi.org/10.1007/s40121-022-00683-z] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/36094720]

37. Johnston, M.D.; Pell, B.; Nelson, P. A Mathematical Study of COVID-19 Spread by Vaccination Status in Virginia. Appl. Sci.; 2022; 12, 1723. [DOI: https://dx.doi.org/10.3390/app12031723]

38. Caetano, C.; Morgado, M.L.; Patricio, P.; Leite, A.; Machado, A.; Torres, A.; Pereira, J.F.; Namorado, S.; Sottomayor, A.; Peralta, A. et al. Measuring the impact of COVID-19 vaccination and immunity waning: A modelling study for Portugal. medRxiv; 2021; [DOI: https://dx.doi.org/10.1101/2021.12.10.21267618]

39. Kerr, C.C.; Stuart, R.M.; Mistry, D.; Abeysuriya, R.G.; Rosenfeld, K.; Hart, G.R.; Núñez, R.C.; Cohen, J.A.; Selvaraj, P.; Hagedorn, B. et al. Covasim: An agent-based model of COVID-19 dynamics and interventions. PLoS Comput. Biol.; 2021; 17, e1009149. [DOI: https://dx.doi.org/10.1371/journal.pcbi.1009149]

40. Hinch, R.; Probert, W.J.; Nurtay, A.; Kendall, M.; Wymant, C.; Hall, M.; Lythgoe, K.; Bulas Cruz, A.; Zhao, L.; Stewart, A. et al. OpenABM-Covid19—An agent-based model for non-pharmaceutical interventions against COVID-19 including contact tracing. PLoS Comput. Biol.; 2021; 17, e1009146. [DOI: https://dx.doi.org/10.1371/journal.pcbi.1009146]

41. Gaudou, B.; Huynh, N.Q.; Philippon, D.; Brugière, A.; Chapuis, K.; Taillandier, P.; Larmande, P.; Drogoul, A. COMOKIT: A Modeling Kit to Understand, Analyze, and Compare the Impacts of Mitigation Policies Against the COVID-19 Epidemic at the Scale of a City. Front. Public Health; 2020; 8, 563247. [DOI: https://dx.doi.org/10.3389/fpubh.2020.563247] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/33072700]

42. Aylett-Bullock, J.; Cuesta-Lazaro, C.; Quera-Bofarull, A.; Icaza-Lizaola, M.; Sedgewick, A.; Truong, H.; Curran, A.; Elliott, E.; Caulfield, T.; Fong, K. et al. June: Open-source individual-based epidemiology simulation. R. Soc. Open Sci.; 2021; 8, 210506. [DOI: https://dx.doi.org/10.1098/rsos.210506] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/34295529]

43. Reguly, I.Z.; Csercsik, D.; Juhász, J.; Tornai, K.; Bujtár, Z.; Horváth, G.; Keömley-Horváth, B.; Kós, T.; Cserey, G.; Iván, K. et al. Microsimulation based quantitative analysis of COVID-19 management strategies. PLoS Comput. Biol.; 2022; 18, e1009693. [DOI: https://dx.doi.org/10.1371/journal.pcbi.1009693]

44. Kerr, C.C.; Mistry, D.; Stuart, R.M.; Rosenfeld, K.; Hart, G.R.; Núñez, R.C.; Cohen, J.A.; Selvaraj, P.; Abeysuriya, R.G.; Jastrzębski, M. et al. Controlling COVID-19 via test-trace-quarantine. Nat. Commun.; 2021; 12, 2993. [DOI: https://dx.doi.org/10.1038/s41467-021-23276-9]

45. Cohen, J.A.; Stuart, R.M.; Rosenfeld, K.; Lyons, H.; White, M.; Kerr, C.; Klein, D.J.; Famulare, M. Mechanistic modeling of SARS-CoV-2 immune memory, variants, and vaccines. medRxiv; 2021; preprint

46. Scott, N.; Palmer, A.; Delport, D.; Abeysuriya, R.; Stuart, R.M.; Kerr, C.C.; Mistry, D.; Klein, D.J.; Sacks-Davis, R.; Heath, K. et al. Modelling the impact of relaxing COVID-19 control measures during a period of low viral transmission. Med. J. Aust.; 2020; 214, pp. 79-83. [DOI: https://dx.doi.org/10.5694/mja2.50845]

47. Abeysuriya, R.G.; Delport, D.; Stuart, R.M.; Sacks-Davis, R.; Kerr, C.C.; Mistry, D.; Klein, D.J.; Hellard, M.; Scott, N. Preventing a cluster from becoming a new wave in settings with zero community COVID-19 cases. BMC Infect. Dis.; 2022; 22, 232. [DOI: https://dx.doi.org/10.1186/s12879-022-07180-1]

48. Sanz-Leon, P.; Stevenson, N.J.; Stuart, R.M.; Abeysuriya, R.G.; Pang, J.C.; Lambert, S.B.; Kerr, C.C.; Roberts, J.A. Risk of sustained SARS-CoV-2 transmission in Queensland, Australia. Sci. Rep.; 2022; 12, 6309. [DOI: https://dx.doi.org/10.1038/s41598-022-10349-y]

49. Hamer, D.H.; White, L.F.; Jenkins, H.E.; Gill, C.J.; Landsberg, H.E.; Klapperich, C.; Bulekova, K.; Platt, J.; Decarie, L.; Gilmore, W. et al. Assessment of a COVID-19 Control Plan on an Urban University Campus During a Second Wave of the Pandemic. JAMA Netw. Open; 2021; 4, e2116425. [DOI: https://dx.doi.org/10.1001/jamanetworkopen.2021.16425]

50. Panovska-Griffiths, J.; Kerr, C.C.; Waites, W.; Stuart, R.M.; Mistry, D.; Foster, D.; Klein, D.J.; Viner, R.M.; Bonell, C. Modelling the potential impact of mask use in schools and society on COVID-19 control in the UK. Sci. Rep.; 2021; 11, 8747. [DOI: https://dx.doi.org/10.1038/s41598-021-88075-0]

51. Krivorotko, O.; Sosnovskaia, M.; Vashchenko, I.; Kerr, C.; Lesnic, D. Agent-based modeling of COVID-19 outbreaks for New York state and UK: Parameter identification algorithm. Infect. Dis. Model.; 2021; 7, pp. 30-44. [DOI: https://dx.doi.org/10.1016/j.idm.2021.11.004] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/34869960]

52. Pham, Q.D.; Stuart, R.M.; Nguyen, T.V.; Luong, Q.C.; Tran, Q.D.; Pham, T.Q.; Phan, L.T.; Dang, T.Q.; Tran, D.N.; Do, H.T. et al. Estimating and mitigating the risk of COVID-19 epidemic rebound associated with reopening of international borders in Vietnam: A modelling study. Lancet Glob. Health; 2021; 9, pp. e916-e924. [DOI: https://dx.doi.org/10.1016/S2214-109X(21)00103-0]

53. Cattaneo, A.; Vitali, A.; Mazzoleni, M.; Previdi, F. An agent-based model to assess large-scale COVID-19 vaccination campaigns for the Italian territory: The case study of Lombardy region. Comput. Methods Programs Biomed.; 2022; 224, 107029. [DOI: https://dx.doi.org/10.1016/j.cmpb.2022.107029] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/35908330]

54. Lutz, C.B.; Giabbanelli, P.J. When Do We Need Massive Computations to Perform Detailed COVID-19 Simulations?. Adv. Theory Simul.; 2021; 5, 2100343. [DOI: https://dx.doi.org/10.1002/adts.202100343] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/35441122]

55. Cai, J.; Deng, X.; Yang, J.; Sun, K.; Liu, H.; Chen, Z.; Peng, C.; Chen, X.; Wu, Q.; Zou, J. et al. Modeling transmission of SARS-CoV-2 omicron in China. Nat. Med.; 2022; 28, pp. 1468-1475. [DOI: https://dx.doi.org/10.1038/s41591-022-01855-7]

56. Pell, B.; Johnston, M.D.; Nelson, P. A data-validated temporary immunity model of COVID-19 spread in Michigan. Math. Biosci. Eng.; 2022; 19, pp. 10122-10142. [DOI: https://dx.doi.org/10.3934/mbe.2022474]

57. Clarke, K.E.; Jones, J.M.; Deng, Y.; Nycz, E.; Lee, A.; Iachan, R.; Gundlapalli, A.V.; Hall, A.J.; MacNeil, A. Seroprevalence of Infection-Induced SARS-CoV-2 Antibodies—United States, September 2021–February 2022. Morb. Mortal. Wkly. Rep.; 2022; 71, 606. [DOI: https://dx.doi.org/10.15585/mmwr.mm7117e3]

58. Cohen, J.A.; Stuart, R.M.; Panovska-Griffiths, J.; Mudimu, E.; Abeysuriya, R.G.; Kerr, C.C.; Famulare, M.; Klein, D.J. The changing impact of vaccines in the COVID-19 pandemic. medRxiv; 2022; [DOI: https://dx.doi.org/10.1101/2022.03.10.22272222]

59. Lee, S.; Zabinsky, Z.B.; Wasserheit, J.N.; Ross, J.M.; Chen, S.; Liu, S. COVID-19 Endemic Plan: Impact of Vaccination and Non-pharmaceutical Interventions with Viral Variants and Waning Immunity Using an Agent-Based Simulation. medRxiv; 2022; [DOI: https://dx.doi.org/10.1101/2022.06.03.22275834]

60. Feng, A.; Obolski, U.; Stone, L.; He, D. Modelling COVID-19 Vaccine Breakthrough Infections in Highly Vaccinated Israel-the effects of waning immunity and third vaccination dose. medRxiv; 2022; [DOI: https://dx.doi.org/10.1101/2022.01.08.22268950]

61. Pepper, M. Immunity to SARS-CoV-2 infection. Immunol. Rev.; 2022; 309, 5. [DOI: https://dx.doi.org/10.1111/imr.13117]

62. Epsi, N.J.; Richard, S.A.; Lindholm, D.A.; Mende, K.; Ganesan, A.; Huprikar, N.; Lalani, T.; Fries, A.C.; Maves, R.C.; Colombo, R.E. et al. Understanding ‘hybrid immunity’: Comparison and predictors of humoral immune responses to SARS-CoV-2 infection and COVID-19 vaccines. Clin. Infect. Dis.; 2022; ciac392. [DOI: https://dx.doi.org/10.1093/cid/ciac392]

63. Altarawneh, H.N.; Chemaitelly, H.; Ayoub, H.H.; Tang, P.; Hasan, M.R.; Yassine, H.M.; Al-Khatib, H.A.; Smatti, M.K.; Coyle, P.; Al-Kanaani, Z. et al. Effects of Previous Infection and Vaccination on Symptomatic Omicron Infections. N. Engl. J. Med.; 2022; 387, pp. 21-34. [DOI: https://dx.doi.org/10.1056/NEJMoa2203965]

64. Nguyen, D.C.; Lamothe, P.A.; Woodruff, M.C.; Saini, A.S.; Faliti, C.E.; Sanz, I.; Lee, F.E.H. COVID-19 and plasma cells: Is there long-lived protection?. Immunol. Rev.; 2022; 309, pp. 40-63. [DOI: https://dx.doi.org/10.1111/imr.13115]

65. Bhattacharya, M.; Sharma, A.R.; Dhama, K.; Agoramoorthy, G.; Chakraborty, C. Hybrid immunity against COVID-19 in different countries with a special emphasis on the Indian scenario during the Omicron period. Int. Immunopharmacol.; 2022; 108, 108766. [DOI: https://dx.doi.org/10.1016/j.intimp.2022.108766] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/35413676]

66. Sette, A.; Crotty, S. Immunological memory to SARS-CoV-2 infection and COVID-19 vaccines. Immunol. Rev.; 2022; 310, pp. 27-46. [DOI: https://dx.doi.org/10.1111/imr.13089]

67. Bates, T.A.; McBride, S.K.; Leier, H.C.; Guzman, G.; Lyski, Z.L.; Schoen, D.; Winders, B.; Lee, J.Y.; Lee, D.X.; Messer, W.B. et al. Vaccination before or after SARS-CoV-2 infection leads to robust humoral response and antibodies that effectively neutralize variants. Sci. Immunol.; 2022; 7, eabn8014. [DOI: https://dx.doi.org/10.1126/sciimmunol.abn8014]

68. Mozaffer, F.; Cherian, P.; Krishna, S.; Wahl, B.; Menon, G.I. Effect of hybrid immunity, school reopening, and the Omicron variant on trajectory of COVID-19 epidemic in India: A modelling study. medRxiv; 2022; [DOI: https://dx.doi.org/10.1016/j.lansea.2022.100095]

69. Nordström, P.; Ballin, M.; Nordström, A. Risk of SARS-CoV-2 reinfection and COVID-19 hospitalisation in individuals with natural and hybrid immunity: A retrospective, total population cohort study in Sweden. Lancet Infect. Dis.; 2022; 22, pp. 781-790. [DOI: https://dx.doi.org/10.1016/S1473-3099(22)00143-8]

70. Pilz, S.; Theiler-Schwetz, V.; Trummer, C.; Krause, R.; Ioannidis, J.P. SARS-CoV-2 reinfections: Overview of efficacy and duration of natural and hybrid immunity. Environ. Res.; 2022; 209, 112911. [DOI: https://dx.doi.org/10.1016/j.envres.2022.112911]

71. Busch, M.P.; Stramer, S.L.; Stone, M.; Yu, E.A.; Grebe, E.; Notari, E.; Saa, P.; Ferg, R.; Manrique, I.M.; Weil, N. et al. Population-weighted seroprevalence from SARS-CoV-2 infection, vaccination, and hybrid immunity among US blood donations from January–December 2021. Clin. Infect. Dis.; 2022.

72. Bhiman, J.N.; Moore, P.L. Leveraging South African HIV research to define SARS-CoV-2 immunity triggered by sequential variants of concern. Immunol. Rev.; 2022; 310, pp. 61-75. [DOI: https://dx.doi.org/10.1111/imr.13086] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/35599324]

73. Otto, S.P.; Day, T.; Arino, J.; Colijn, C.; Dushoff, J.; Li, M.; Mechai, S.; Van Domselaar, G.; Wu, J.; Earn, D.J. et al. The origins and potential future of SARS-CoV-2 variants of concern in the evolving COVID-19 pandemic. Curr. Biol.; 2021; 31, pp. R918-R929. [DOI: https://dx.doi.org/10.1016/j.cub.2021.06.049] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/34314723]

74. Ciunkiewicz, P.; Brooke, W.; Rogers, M.; Yanushkevich, S. Agent-based epidemiological modeling of COVID-19 in localized environments. Comput. Biol. Med.; 2022; 144, 105396. [DOI: https://dx.doi.org/10.1016/j.compbiomed.2022.105396] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/35299041]

75. Usherwood, T.; LaJoie, Z.; Srivastava, V. A model and predictions for COVID-19 considering population behavior and vaccination. Sci. Rep.; 2021; 11, 12051. [DOI: https://dx.doi.org/10.1038/s41598-021-91514-7] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/34103618]

76. Liu, Y.; Wu, B. Co-evolution of Vaccination Behavior and Perceived Vaccination Risk can lead to a Stag-Hunt like Game. arXiv; 2022; arXiv: 2207.05964

77. Piltch-Loeb, R.; Silver, D.R.; Kim, Y.; Norris, H.; McNeill, E.; Abramson, D.M. Determinants of the COVID-19 vaccine hesitancy spectrum. PLoS ONE; 2022; 17, e0267734. [DOI: https://dx.doi.org/10.1371/journal.pone.0267734]

78. Moore, J.; Gilbert, K.; Lively, K.; Laurent, C.; Chawla, R.; Li, C.; Johnson, R.; Petcu, R.; Mehra, M.; Spooner, A. et al. Correlates of COVID-19 Vaccine Hesitancy among a Community Sample of African Americans Living in the Southern United States. Vaccines; 2021; 9, 879. [DOI: https://dx.doi.org/10.3390/vaccines9080879]

79. Wang, Y.; Liu, Y. Multilevel determinants of COVID-19 vaccination hesitancy in the United States: A rapid systematic review. Prev. Med. Rep.; 2021; 25, 101673. [DOI: https://dx.doi.org/10.1016/j.pmedr.2021.101673]

80. Rane, M.S.; Kochhar, S.; Poehlein, E.; You, W.; Robertson, M.M.; Zimba, R.; A Westmoreland, D.; Romo, M.L.; Kulkarni, S.G.; Chang, M. et al. Determinants and Trends of COVID-19 Vaccine Hesitancy and Vaccine Uptake in a National Cohort of US Adults: A Longitudinal Study. Am. J. Epidemiol.; 2022; 191, pp. 570-583. [DOI: https://dx.doi.org/10.1093/aje/kwab293]

81. Saluja, S.; Lam, C.N.; Wishart, D.; McMorris, A.; Cousineau, M.R.; Kaplan, C.M. Disparities in COVID-19 vaccine hesitancy among Los Angeles County adults after vaccine authorization. Prev. Med. Rep.; 2021; 24, 101544. [DOI: https://dx.doi.org/10.1016/j.pmedr.2021.101544]

82. Halbrook, M.; Gadoth, A.; Martin-Blais, R.; Gray, A.N.; Kashani, S.; Kazan, C.; Kane, B.; Tobin, N.H.; Ferbas, K.G.; Aldrovandi, G.M. et al. Longitudinal Assessment of Coronavirus Disease 2019 Vaccine Acceptance and Uptake Among Frontline Medical Workers in Los Angeles, California. Clin. Infect. Dis.; 2021; 74, pp. 1166-1173. [DOI: https://dx.doi.org/10.1093/cid/ciab614]

83. Freund, A.J.; Giabbanelli, P.J. The Necessity and Difficulty of Navigating Uncertainty to Develop an Individual-Level Computational Model. Proceedings of the International Conference on Computational Science; Krakow, Poland, 16–18 June 2021; pp. 407-421. [DOI: https://dx.doi.org/10.1007/978-3-030-77980-1_31]

84. United States Census Bureau and Bureau of Labor Statistics. Current Population Survey (CPS) Annual Social and Economic (ASEC) Supplement. PINC-01. Selected Characteristics of People 15 Years and over, by Total Money Income, Work Experience, Race, Hispanic Origin, and Sex. Last Revised: 10/08/21. Available online: https://www.census.gov/data/tables/time-series/demo/income-poverty/cps-pinc/pinc-01.2019.html (accessed on 12 August 2022).

85. Petrakis, V.; Panagopoulos, P.; Papazoglou, D.; Papanas, N. Diabetes Mellitus and Hypertension as Major Risk Factors of Mortality from Covid-19 Pneumonia. Exp. Clin. Endocrinol. Diabetes; 2020; 130, pp. 205-206. [DOI: https://dx.doi.org/10.1055/a-1325-0381]

86. Merga, B.T.; Ayana, G.M.; Raru, T.B.; Alemu, A.; Negash, B.; Bekana, M.; Birhanu, A.; Dessie, Y. Association of Pre-Existing Comorbidities with Disease Severity Among COVID-19 Patients in Eastern Ethiopia. Infect. Drug Resist.; 2022; 15, 2825. [DOI: https://dx.doi.org/10.2147/IDR.S362140] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/35673546]

87. McFarlane, E.; Linschoten, M.; Asselbergs, F.W.; Lacy, P.S.; Jedrzejewski, D.; Williams, B. The impact of pre-existing hypertension and its treatment on outcomes in patients admitted to hospital with COVID-19. Hypertens. Res.; 2022; 45, pp. 834-845. [DOI: https://dx.doi.org/10.1038/s41440-022-00893-5] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/35352027]

88. United States Census Bureau. Educational Attainment in the United States: 2019. Table 1. Educational Attainment of the Population 18 Years and over, by Age, Sex, Race, and Hispanic Origin: 2019. Last Revised: 10/08/21. Available online: https://www.census.gov/data/tables/2019/demo/educational-attainment/cps-detailed-tables.html (accessed on 12 August 2022).

89. United States Census Bureau. Annual Estimates of the Resident Population for Selected Age Groups by Sex for the United States: April 1, 2010 to July 1, 2019. Available online: https://www2.census.gov/programs-surveys/popest/tables/2010-2019/national/asrh/nc-est2019-asr5h.xlsx (accessed on 12 August 2022).

90. Igielnick, R.; Keeter, S.; Hartig, H. Behind Biden’s 2020 Victory: An examination of the 2020 Electorate, Based on Validated Voters. Pew Research Center: Report. 30 June 2021. Available online: https://www.pewresearch.org/politics/2021/06/30/behind-bidens-2020-victory/ (accessed on 12 August 2022).

91. United States Census Bureau. Voting and Registration in the Election of November 2020. Last Revised: 10/08/21. Available online: https://www.census.gov/data/tables/time-series/demo/voting-and-registration/p20-585.html (accessed on 16 August 2022).

92. Centers for Disease Prevention and Control. National Diabetes Statistics Report. 2020; Available online: https://www.cdc.gov/diabetes/pdfs/data/statistics/national-diabetes-statistics-report.pdf (accessed on 12 October 2022).

93. National Center for Health Statistics. Table 022: Hypertension among Adults Aged 20 and over, by Selected Characteristics: United States, Selected Years 1988–1994 through 2015–2018. Available online: https://ftp.cdc.gov/pub/Health_Statistics/NCHS/Publications/Health_US/hus19tables/table022.xlsx (accessed on 12 August 2022).

94. Stierman, B.; Afful, J.; Carroll, M.D.; Chen, T.C.; Davy, O.; Fink, S.; Fryar, C.D.; Gu, Q.; Hales, C.M.; Hughes, J.P. et al. National Health and Nutrition Examination Survey 2017–March 2020 Prepandemic Data Files Development of Files and Prevalence Estimates for Selected Health Outcomes. National Health Statistics Reports 158. 2021; Available online: https://www.cdc.gov/nchs/data/nhsr/nhsr158-508.pdf (accessed on 16 August 2022).

95. Kriss, J.L.; Hung, M.C.; Srivastav, A.; Black, C.L.; Lindley, M.C.; Lee, J.T.; Koppaka, R.; Tsai, Y.; Lu, P.J.; Yankey, D. et al. COVID-19 Vaccination Coverage, by Race and Ethnicity—National Immunization Survey Adult COVID Module, United States, December 2020–November 2021. Morb. Mortal. Wkly. Rep.; 2022; 71, 757. [DOI: https://dx.doi.org/10.15585/mmwr.mm7123a2] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/35679179]

96. Rao, D.M.; Higiro, J.D. Managing Pending Events in Sequential and Parallel Simulations Using Three-tier Heap and Two-tier Ladder Queue. ACM Trans. Model. Comput. Simul.; 2019; 29, pp. 1-28. [DOI: https://dx.doi.org/10.1145/3265750]

97. Dignum, V.; Dignum, F. Agents are dead. Long live agents!. Proceedings of the 19th International Conference on Autonomous Agents and MultiAgent Systems; Auckland, New Zealand, 9–13 May 2020; pp. 1701-1705.

98. Giubilini, A.; Savulescu, J. Vaccination, Risks, and Freedom: The Seat Belt Analogy. Public Health Ethic; 2019; 12, pp. 237-249. [DOI: https://dx.doi.org/10.1093/phe/phz014] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/32082418]

99. Shaw, D. The Side Effects of Not Being Vaccinated: Individual Risk and Vaccine Hesitancy Nationalism. J. Bioethical Inq.; 2021; 19, pp. 7-10. [DOI: https://dx.doi.org/10.1007/s11673-021-10141-z]

100. Burden, D.; Savin-Baden, M. Virtual Humans: Today and Tomorrow; Chapman and Hall/CRC: Boca Raton, FL, USA, 2019.

101. Nyblade, B.; O’Mahony, A.; Sieck, K. Building on Social Science: Theoretic Foundations for Modelers. Soc.-Behav. Model. Complex Syst.; 2019; pp. 63-99. [DOI: https://dx.doi.org/10.1002/9781119485001.ch4]

102. Grantham, E.O.; Giabbanelli, P.J. Creating perceptual uncertainty in agent-based models with social interactions. Proceedings of the 2020 Spring Simulation Conference (SpringSim); Fairfax, VA, USA, 18–21 May 2020; pp. 1-12.

103. Sun, R. Cognitive social simulation for policy making. Policy Insights Behav. Brain Sci.; 2018; 5, pp. 240-246. [DOI: https://dx.doi.org/10.1177/2372732218785925]

104. Espinoza, B.; Swarup, S.; Barrett, C.L.; Marathe, M. Heterogeneous adaptive behavioral responses may increase epidemic burden. Sci. Rep.; 2022; 12, 11276. [DOI: https://dx.doi.org/10.1038/s41598-022-15444-8] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/35788663]

105. Morin, O.; Jacquet, P.O.; Vaesen, K.; Acerbi, A. Social information use and social information waste. Philos. Trans. R. Soc. B; 2021; 376, 20200052. [DOI: https://dx.doi.org/10.1098/rstb.2020.0052]

106. Negahban, A.; Giabbanelli, P.J. Hybrid Agent-Based Simulation of Adoption Behavior and Social Interactions: Alternatives, Opportunities, and Pitfalls. IEEE Trans. Comput. Soc. Syst.; 2021; 9, pp. 770-780. [DOI: https://dx.doi.org/10.1109/TCSS.2021.3101794]

107. Castro e Silva, A.; Bernardes, A.T.; Barbosa, E.A.; Chagas, I.A.; Dáttilo, W.; Reis, A.B.; Ribeiro, S.P. Successive Pandemic Waves with Different Virulent Strains and the Effects of Vaccination for SARS-CoV-2. Vaccines; 2022; 10, 343. [DOI: https://dx.doi.org/10.3390/vaccines10030343]

108. U.S. Department of Health & Human Services (HHS). Biden-Harris Administration Secures 66 Million Doses of Moderna’s Variant-Specific COVID-19 Vaccine Booster for Potential Use in Fall and Winter 2022. Released 08/29/22. Available online: https://www.hhs.gov/about/news/2022/07/29/biden-harris-administration-secures-66-million-doses-modernas-variant-specific-covid-19-vaccine-booster-for-potential-use-in-fall-winter-2022.html (accessed on 8 August 2022).

109. Lee, B.Y. Is Omicron BA.5 ‘Worst Version’ of Covid-19 Coronavirus Seen? Forbes. 07/03/22. Available online: https://www.forbes.com/sites/brucelee/2022/07/03/is-omicron-ba5-worst-version-of-covid-19-coronavirus-seen (accessed on 9 August 2022).

110. Zhang, W.; Liu, S.; Osgood, N.; Zhu, H.; Qian, Y.; Jia, P. Using simulation modelling and systems science to help contain COVID-19: A systematic review. Syst. Res. Behav. Sci.; 2022; [DOI: https://dx.doi.org/10.1002/sres.2897]

111. Sachs, J.D.; Karim, S.S.A.; Aknin, L.; Allen, J.; Brosbøl, K.; Colombo, F.; Barron, G.C.; Espinosa, M.F.; Gaspar, V.; Gaviria, A. et al. The Lancet Commission on lessons for the future from the COVID-19 pandemic. Lancet; 2022; [DOI: https://dx.doi.org/10.1016/S0140-6736(22)01585-9]

112. Mansouri, M.A.; Garcia, L.; Kee, F.; Bradley, D.T. Systems-oriented modelling methods in preventing and controlling emerging infectious diseases in the context of healthcare policy. Systems; 2022; in print

113. Squazzoni, F.; Polhill, J.G.; Edmonds, B.; Ahrweiler, P.; Antosz, P.; Scholz, G.; Chappin, É.; Borit, M.; Verhagen, H.; Giardini, F. et al. Computational Models That Matter During a Global Pandemic Outbreak: A Call to Action. J. Artif. Soc. Soc. Simul.; 2020; 23, [DOI: https://dx.doi.org/10.18564/jasss.4298]

114. Voinov, A.; Shugart, H.H. ‘Integronsters’, integral and integrated modeling. Environ. Model. Softw.; 2013; 39, pp. 149-158. [DOI: https://dx.doi.org/10.1016/j.envsoft.2012.05.014]

115. Shroeder, S.A.; Vendome, C.; Giabbanelli, P.J.; Montfort, A.M. Towards reusable building blocks to develop COVID-19 simulation models. Proceedings of the 2022 Winter Simulation Conference; Singapore, 11–14 December 2022.

116. Dou, K.; Yang, J.; Wang, L.-X.; Li, J.-B. Theory of planned behavior explains males’ and females’ intention to receive COVID-19 vaccines differently. Hum. Vaccines Immunother.; 2022; 2086393. [DOI: https://dx.doi.org/10.1080/21645515.2022.2086393]

117. Jelnov, A.; Jelnov, P. Vaccination policy and trust. Econ. Model.; 2022; 108, 105773. [DOI: https://dx.doi.org/10.1016/j.econmod.2022.105773]

118. Lastrucci, V.; Collini, F.; Forni, S.; D’Arienzo, S.; Di Fabrizio, V.; Buscemi, P.; Lorini, C.; Gemmi, F.; Bonaccorsi, G. The indirect impact of COVID-19 pandemic on the utilization of the emergency medical services during the first pandemic wave: A system-wide study of Tuscany Region, Italy. PLoS ONE; 2022; 17, e0264806. [DOI: https://dx.doi.org/10.1371/journal.pone.0264806]

119. Bazaid, A.S.; Barnawi, H.; Qanash, H.; Alsaif, G.; Aldarhami, A.; Gattan, H.; Alharbi, B.; Alrashidi, A.; Abu Al-Soud, W.; Moussa, S. et al. Bacterial Coinfection and Antibiotic Resistance Profiles among Hospitalised COVID-19 Patients. Microorganisms; 2022; 10, 495. [DOI: https://dx.doi.org/10.3390/microorganisms10030495] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/35336071]

120. Wang, Q.; Guo, S.; Wei, X.; Dong, Q.; Xu, N.; Li, H.; Zhao, J.; Sun, Q. Global prevalence, treatment and outcome of tuberculosis and COVID-19 coinfection: A systematic review and meta-analysis (from November 2019 to March 2021). BMJ Open; 2022; 12, e059396. [DOI: https://dx.doi.org/10.1136/bmjopen-2021-059396] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/35725250]

121. Ojo, M.M.; Benson, T.O.; Peter, O.J.; Goufo, E.F.D. Nonlinear optimal control strategies for a mathematical model of COVID-19 and influenza co-infection. Phys. A Stat. Mech. Its Appl.; 2022; 607, 128173. [DOI: https://dx.doi.org/10.1016/j.physa.2022.128173] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/36106051]