Megastudies: A New Approach to Reducing Vaccine Hesitation Worldwide

Abstract

Vaccine hesitancy is a considerable obstacle to achieving vaccine protection worldwide. There needs to be more evidence-based research for interventions for vaccine hesitancy. Existing effectiveness evaluations are limited to one particular hypothesis, and no studies have compared the effectiveness of different interventions. A megastudy takes a large-scale, multi-intervention, uniform participant and the same evaluation criteria approach to evaluate many interventions simultaneously and find the most effective ones. Therefore, megastudies can help us find the most effective interventions for vaccine hesitancy. Additionally, considering the complex causes of vaccine hesitancy, we design interventions that involve social factors in megastudies. Lastly, quality control and justice are critical issues for megastudies in the future.

1. Introduction

The World Health Organization defines vaccine hesitancy as a “delay in acceptance or refusal of safe vaccines despite the availability of vaccine” [1]. Infectious diseases are an ever-present threat to humans, while vaccination prevents over 20 life-threatening diseases, helping people of all ages live longer, healthier lives. Immunization currently prevents 3.5–5 million deaths yearly from diseases such as diphtheria, tetanus, pertussis, influenza, and measles [2]. According to research on the relationship between vaccine refusal and vaccine-preventable disease outbreaks in the United States, 70.6% of measles-infected individuals were unvaccinated in measles outbreaks from 2000 to 2015 [3]. Additionally, vaccination works as individual immunity and achieves the protective effect with a significantly vaccinated population; the higher the coverage and the faster coverage is achieved, the better the vaccine’s protection [4,5].

The COVID-19 (coronavirus disease 2019) global pandemic is accompanied by ultra-contagiousness, a rapid mutation rate, and a lack of specific drugs, making vaccines a key measure for reducing its damage. According to a target population size for COVID-19 vaccination, the expected global vaccination population for COVID-19 is estimated to be 5.8 billion, representing 74.4% of the worldwide population [6]. However, vaccine hesitation is a severe obstacle to this goal [7,8]. According to an Oxford University research institute, by 27 October 2022, only 68.4% of the world population had received at least one dose of a COVID-19 vaccine [9]. In January 2019, the WHO (World Health Organization) declared vaccine hesitancy among the top 10 threats to global health [1].

Vaccine hesitancy occurs in response to all sorts of vaccines and exists to varying degrees around the world [10,11,12]. SAGE (World Health Organization Strategic Advisory Group of Experts on Immunization) defined vaccine hesitancy as “complex and context-specific,” “varying over time and place,” and “vaccine specific” [13]. Worse, fragile groups, such as people with chronic diseases, the elderly, and pregnant women, usually hesitate to take vaccines [14,15,16]. In addition, marginal groups in deep need of vaccine protection (e.g., minority groups, migrants, or refugees) may be more vaccine-hesitant, which is worrying [17,18,19]. Multiple researchers have discussed vaccine hesitancy’s causes and influencing factors [20]. They revealed that causes of vaccine hesitancy, including worrying about vaccines’ safety [21]; inconvenience in accessing vaccines [1]; mistrust of the medical system, government, or a new vaccine [22]; religious belief [23]; personality [24]; complacency (underestimated risk of infection) [25]; conspiracy thinking [26]; negative information about vaccines spread on social media [27]; and so on.

Interventions to reduce vaccine hesitancy are urgently needed. Studies on the causes and influencing factors can guide us to strategies and skills to reduce vaccine hesitancy. Many studies have brought up suggestions based on vaccine hesitancy cause analysis [28]. For example, a meta-analysis on measles and HPV (human papillomavirus) vaccines introduced nine vaccine promotion intervention domains. They include education (providing education on vaccination, disease, and how vaccines work); on-site vaccination (providing vaccines at the workplace or places of worship); financial incentives; free vaccination; institutional recommendation (a recommendation made by the institution that person works at especially for healthcare providers); healthcare provider recommendation; a reminder and recall; gain vs. loss framing of the vaccine; and vaccine champions (an institutionally appointed champion to encourage vaccination) [29].

In the research on different vaccines, researchers believe we need strategies on the individual, provider, health system, and national levels [5,30]. Some researchers find specific theoretical models (interventions based on the health belief model, the theory of planned behavior, and the 3C model (complacency, convenience, and confidence model)) could provide us with excellent guidance to reduce vaccine hesitancy in pregnant women [31]. The idea of designing community engagement strategies aiming to build public trust was brought up [22]. In addition, many researchers have evaluated the effects of interventions in rigorously controlled studies, and many interventions proved effective, such as individually education, sending reminder messages, and financial incentives [28,32,33].

We already know so many inventions for vaccine hesitancy, but which intervention or mix-program is most effective in promoting vaccination? The cost of vaccine hesitancy to global health and economies is so great that we need more than a random practice of interventions or the selection of interventions based only on the evidence of scattered studies. Instead, we need to find the most effective interventions for vaccine hesitancy based on more rigorous evidence.

Nonetheless, RCTs (randomized controlled trials) on vaccine hesitancy have only examined the effects of one intervention limited to a single location and a tiny population and are highly specific. Given the different research designs and participants, we cannot compare the effectiveness of inventions in different studies. Additionally, obtaining information particular interventions one by one would take considerable time.

Therefore, we have an urgent task: compare various interventions’ efficacies to find the most effective ones.

2. Emerging Megastudies in Behavioral Study

In response to these problems, a megastudy offers a new and attractive option. Megastudies have been applied in psychology, linguistics, and computer science for years [34,35,36,37]. At its core, it is about different researchers using the same dataset and searching for the optimal outcome to solve a common problem. It is currently applied in behavioral science.

Insufficient physical exercise is a substantial public health issue. In this field, researchers test different intervention ideas in different samples using different methods over different time intervals. The problem is that comparing those conclusions from individual investigations is difficult. A study published in Nature in 2021 described megastudies on exercise behavior. Researchers realized that the “one-apple-at-time” approach is inefficient in advancing behavioral science. “Typically, individual research studies are designed to establish the validity of a single idea rather than to assess its efficacy relative to other theoretically informed approaches in a particular policy context. We propose that the megastudy approach surmounts this and many other obstacles to developing optimal behaviorally informed policy interventions” [38].

With 61,293 participants, small independent teams of 30 scientists from 15 different US universities designed and implemented 54 interventions (designed by different independent teams) to encourage exercise for four weeks and then identified the best intervention with the same standard. The interventions practiced in this megastudy include a bonus for returning after missed workouts, exercising social norms shared (high and increasing), free audiobook provided, planning fallacy described and planning revision encouraged, rigidity reward, and others in a total of 54 interventions.

This megastudy on exercise included a placebo control group in which participants received a little reward to enter an experimental condition and no other incentives. Researchers set the planning, reminders, and micro-incentives as the baseline, then tested the other 53 experimental conditions. “Where a typical RCT study would develop an intervention based on one hypothesis, megastudies trying many interventions at once—as was done in the megastudy—could speed up scientific discovery” [39]. However, the most significant increase (27% increase in exercise and 16% increase compared to the baseline) occurred in the condition of “a bonus for returning to the gym after a missed workout,” which differs greatly from the prediction [38].

Is this research approach suitable for the pressing question of vaccine hesitancy? After the exercise behavior megastudy, Milkman conducted a study on vaccine hesitancy in 2020 [40]. Studies have shown that only 79% of people who intended to receive the influenza vaccine did so [41]. Follow-up failure results from forgetfulness, failure to expect and plan barriers, and lack of motivation [42]. Psychological reminders have the potential to bridge these “intention-action gaps” [43]. Hence, researchers randomly assigned 689,693 Walmart pharmacy patients to receive one of 22 text reminders using various behavioral science principles to nudge flu vaccination or to a business-as-usual control condition where patients received no messages. The most effective messages reminded patients that a flu shot was waiting for them, and they delivered reminders on multiple days. Like the megastudy on exercise, neither experts nor lay people expected the best-performing treatment, underscoring the value of simultaneously testing many nudges in a highly powered megastudy [40].

In October 2022, Duckworth and Milkman summarized the megastudy approach in A Guide to Megastudies [44]. They described its origins, implementation steps, strengths, conditions, limitations, and future. These presentations allow us to fully understand this approach and provide a desirable option for reducing vaccine hesitancy. Duckworth and Milkman define megastudies in behavior studies as “a massive field experiment in which many different treatments are tested synchronously in one large sample using a common, objectively measured outcome” [37,44]. It typically takes the form of independent research teams developing sets of treatment(s) and control conditions (“sub-studies”), with participants randomly assigned across all of them, such as what Duckworth and Milkman illustrate in the following Figure 1 [44].

Duckworth and Milkman briefly discussed the advantages and disadvantages of megastudies. Next, we must discuss the advantages and disadvantages of megastudies and the challenges and limitations of applying megastudies to address vaccine hesitancy.

3. Strengths and Weakness

Faced with vaccine hesitancy, a significant global public health problem, we need to adopt random interventions that have only been proven “effective”. Traditional single RCT studies can provide more valid evidence than other studies (e.g., cross-sectional studies), but their results only validate one particular hypothesis in the research. Even though researchers have carried out corresponding studies around the same topic, the results are difficult to compare because of their different study designs and participants. In contrast, in megastudies, participants for each intervention come from a shared pool. Each intervention takes the same period and is assessed with the same set of criteria, which allows us to draw conclusions about which behavioral interventions are most reliable.

The strengths of megastudies over traditional individual RCTs include the following:

• Optimal Choice: compared to traditional single RCTs, megastudies can find the most effective intervention rather than one effective intervention with direct evidence because the researcher applies many interventions within the same population, time, and condition.

• Less Time: Compared to many single RCTs applied by different teams in different periods, megastudies can find the most effective interventions to increase vaccination coverage relatively quickly. This is vital when a pandemic breaks out.

• Broad Vision: Compared to traditional single RCTs, megastudies may find unexpected outstanding interventions. Because traditional RCT studies usually test hypotheses, researchers suppose the most likely option and do not try those in which the researcher has little confidence. Hence, they may miss interventions that appear less promising but are, in fact, highly effective [38,40]. In megastudies, different teams present different interventions and may apply interventions of which they are less confident [44].

• Complete Results: Megastudies compare the efficacy of different interventions and find the optimal approach so researchers do not have to worry about obtaining invalid results for specific measures and publishing invalid results, which is also essential for our guidance. In independent studies, researchers may not publish invalid results that cause misinformation [44].

• Inclusion: Megastudies can involve more researchers. Different teams from different disciplines and schools of thought can make the study more diverse [44]. However, if the selection criteria and procedures are inappropriate, it may also cause inclusion problems. Who will be invited, and who will not? Therefore, we also need to design open and scientific procedures to recruit researchers.

Nonetheless, compared with traditional RCTs, some disadvantages appear in megastudies.

• Cost: The most apparent burden of megastudies may be the considerable cost. For example, the exercise study by Milkman et al. costs $2.6 million to implement the intervention. However, given the enormous health and economic losses to humans caused by vaccine hesitancy, megastudies in the vaccine hesitancy field are urgent. Secondly, megastudies are costly compared to one traditional RCT but may save much money compared with numerous independent studies. Duckworth and Milkman mentioned that high costs might make it challenging to conduct megastudies and difficult to replicate [44].

• Applicability: Conclusions from megastudies cannot be extrapolated. The evidence level from megastudies is not higher than RCTs: we should know that a megastudy remains an RCT. Megastudies differ from a traditional single RCT in evidence power and research purposes. Megastudies help us compare the effectiveness of many interventions, finding the best one from many interventions (there are both baseline and placebo groups inside) rather than finding the effective one in two options (the placebo group).

Despite these disadvantages, megastudies are still a desirable option in the face of the urgent need to reduce vaccine hesitation. However, there are some critical issues if we want to reduce vaccine hesitation in megastudies.

4. Challenges and Solutions

Vaccine hesitancy is not only a behavioral problem and complex social phenomenon [45]. The interventions in physical exercise and influenza megastudies focus on the gap between intention and action [29]. Interventions in two megastudies for behavior change were mainly “Nudges”: planning, bonus, exercise commitment contracts, sharing social values or norms in the community, fitness questionnaires, and different urging messages [38,40]. However, many people still have not made a firm commitment to vaccination. Can megastudies be conducted on these populations? Let us examine different causes that influence vaccination intentions.

• Safety and Effectiveness: Worrying about the safety and effectiveness of vaccines is a common factor of vaccine hesitancy [20.21]. Many people worry about the damage caused by vaccines, especially when adverse events are reported [46,47]. Among those concerned about vaccine safety, some are concerned about additional risks because of their clinical conditions, such as pregnant women [16,30], the elderly [16,48], and patients with chronic diseases [49,50]. In recent years, vaccine hesitancy has closely been related to widespread rumors about infertility or autism, which proved totally false [29]. In addition, suspicion of vaccine effectiveness matters in vaccine hesitation, especially facing a new vaccine, the vaccine does not work as expected or few studies have been conducted to demonstrate its efficacy and safety in specific populations at risk [46,51,52,53].

• Social Factors: Except for the clinical factors, many social factors are tangled with the worry about vaccines. Sometimes, vaccine hesitancy is provoked by racism, inequality, social exclusion, and religious ideas [17,18,19,23]. Education, power structures and gender inequalities in the family may also trigger vaccine hesitancy. For example, the survey revealed that fathers with limited education might have a more significant influence on preventing their children from being vaccinated than mothers with more education. The blocking effect of maternal grandfathers was more pronounced [54]. Additionally, social media drives the widespread dissemination of rumors about vaccines [27,55,56,57]. Finally, some suspect vaccines are a conspiracy of other countries, governments, or big companies [26,54]. In some cases, resistance reflected not specific concerns about the vaccine but rather a convergence of broader social factors [45].

• Diversity: The reasons for vaccine hesitancy are diverse. Vaccine hesitancy is related to people’s characteristics, such as race, health conditions, age, religion, education, political and economic status, region, and so on [20,51]. Studies have shown that differences in vaccination rates between China and the United States is slight. However, the reasons for vaccination and concerns about vaccines are very different. Therefore, it is necessary to develop different comprehensive intervention strategies for different countries [58]. Therefore, causes for vaccine hesitancy may differ for populations, and the effective interventions for them may differ.

These problems cannot be solved only by nudges. If there are questions about the safety, efficacy, and accessibility of the vaccine, we should solve them in the development, production, and distribution process [59,60]. When discussing vaccine hesitancy, it is supposed to be safe and accessible [1]. However, even if vaccines are safe and effective, people may not believe it; even if vaccines are accessible, people may not know. We can intervene in these issues.

We already have many recommendations to address vaccine hesitation provoked by various causes. Some suggestions are about policies and institutional change, such as introducing more transparency into policy decision-making before immunization programs, more public participation in vaccine programs, providing up-to-date information to the public and health providers about the rigorous procedures undertaken before the introduction of new vaccines, and through diversified post-marketing surveillance of vaccine-related events [61]. Although many studies have shown that negative messages on social media increase vaccine hesitation, some studies show that open media use can increase new crown vaccinations. Therefore, we can make individualized and targeted strategies for COVID-19 vaccination and disseminate the vaccination information to different media use groups [62]. Nevertheless, many suggestions are easy to apply and proved effective, such as individual education, a loss–benefit framework, a recommendation from health service providers, and so on [28,29]. Then, we need to know which is most effective for a particular group or cause. In addition, we can design new intervention programs according to existing research and use megastudies to find the most effective one.

For example, scholars have suggested several plans for the conflict between fasting and vaccination for Muslims: staggering the time of vaccination with the time of fasting (e.g., providing vaccine at night, during special Ramadan nightly prayers, Taraweeh), having religious leaders explain that there is no contradiction between fasting and vaccination, and setting up vaccination sites outside mosques [23]. How effective are these methods? Which one, or what combination, will be most effective? We can only know with megastudies. The answer may differ from what we might expect [63].

There are many long-standing, systemic social problems behind vaccine hesitation, which are difficult to change in the short term. However, we can still promote vaccination before completely solving the related social problems. For example, immigrants may distrust local governments because of social exclusion. Hence, they may understand vaccines as a conspiracy. We cannot change discrimination against immigrants in the short term, nor can we change immigrants’ attitudes toward the government in the short term. However, we may change immigrants’ perceptions of vaccines via transparent information or change their perceptions of vaccines via key opinion leaders in the migrant community. In addition, there are many ways to address rumors and conspiracy theories on social media that may reverse false beliefs, such as social media postings to dispel rumors, celebrities promoting the safety of vaccines, and medical professionals explaining the safety of vaccines.

The core strength of megastudies is that they simultaneously compare multiple interventions in the same population. We can apply it with different interventions. Duckworth and Milkman have talked about the future of megastudies. They believe that megastudies could test more social and experiential interventions and aim to treat participants for extended time horizons [43]. However, they did not talk in detail. In the future, to reduce vaccine hesitancy, the involvement of social science scholars may be essential in megastudies. There is no conflict between the social sciences and megastudies: sociological and political science researchers have used megastudies to reduce people’s partisan hatred [64].

In response to the heterogeneity behind vaccine hesitancy, we propose to find optimal interventions for specific populations and causes rather than searching for a universal plan. We could, for example, conduct more thorough studies targeting groups such as the elderly, the chronically ill, immigrants, and minorities. In addition, we can conduct megastudies that target a particular cause of vaccine hesitation, such as addressing concerns about safety or social media rumors, to find the optimal intervention for a particular worry.

Still, we must admit that not all strategies that may reduce vaccine hesitancy can be tested in megastudies. Duckworth and Milkman admit that policy solutions are amenable to examination by field experiment or megastudy, and, of course, many of the most efficacious solutions may not be [44]. For example, establishing a transparent vaccine research mechanism and a rigorous safety system is crucial to reducing vaccine hesitation, but we may not verify its effectiveness in megastudies.

5. Further Challenges for Megastudies in Future

We should involve more social factors in megastudies on vaccine hesitancy. Meanwhile, it is necessary to take megastudies in different populations and social contexts or focus on vaccine hesitancy provoked by different causes. However, new challenges may then appear if we do so. For example, interventions in studies may be more complex, labor-intensive, and longer processes. Moreover, it may be challenging to access marginal groups, such as the elderly who do not use a smartphone, migrants who do not trust the medical system, and women who cannot read. Therefore, we must discuss the following significant issues:

Quality Control: Megastudies contain complex procedures and massive data. How can we manage it? Would falsification happen?

No study can eliminate errors, nor can we eliminate the possibility of falsification. However, by analyzing the study design, people may have a weaker incentive to falsify megastudies. In a single RCT study, people want to verify that an intervention is effective. If the results are negative, it may mean that the study has limited value or even no possibility of publication. Thus, researchers may be more motivated to falsify data and get valid results. In megastudies, as mentioned earlier, invalid results are not terrible. Therefore, even if a team designs an intervention that is ineffective or not the most effective, their work is a significant part of the overall study.

However, we recognize participants may cut corners. Sometimes, it could result from irregular training, and sometimes it may result from an overloading job. Therefore, it would be better to have stable sites or platforms. For example, gyms and pharmacies are excellent places, and hospitals, community health centers, or schools may be suitable. In addition, applying interventions via digital platforms may save labor and prevent falsification.

Ethical Issues: In many cases, vaccine hesitancy is related to a vulnerable position. Therefore, justice is a crucial ethical issue in the hesitation toward vaccines, and there would be a necessary concern about justice in megastudies on vaccine hesitancy.

Achieving adequate vaccine protection requires the participation of everyone. However, some groups may lack understanding about vaccines because of their marginal status (e.g., the elderly or disabled, or illegal immigrants). Similarly, because of their marginal status, it may be challenging for socioeconomically disadvantaged populations to take part in megastudies.

Even if one person from a marginal group participates in the study, it could be challenging for him or her to complete the interventions, some of which may require participants to have specific competencies (e.g., literacy, smartphone use, access to the Internet). In addition, factors such as time, financial, technical, or cognitive thresholds may exclude some vulnerable populations. We may not involve these participants in the study and thus cannot find the best way to intervene with vaccine hesitancy for marginalized populations or come up with the wrong way to intervene. Injustice could cause unaddressed vaccine hesitancy in marginalized populations and create new health inequities [65]. Without consideration of these factors, the strengths of megastudies may decrease, even causing health inequalities in the long term. Therefore, some details of the study design—such as simplifying the interventions, flexible time, and accessibility—could be crucial for a megastudy.

6. Conclusions

Vaccines have saved millions of lives against infectious diseases, and vaccine hesitancy is one of the most significant impediments to the protective effect of vaccines. By establishing which interventions are the most effective in reducing vaccine hesitancy, megastudies provide an irreplaceable approach to addressing this urgent problem. In the future, we must continue to improve the design of megastudies, expand the areas of application of megastudies, improve the procedures for administering megastudies, and develop ethical norms for megastudies.

Nevertheless, megastudies may not eliminate vaccine hesitancy, much less eliminate all the problems that prevent vaccination. Sometimes, the safety, efficacy, and accessibility of vaccines are not resolved [51,59], and sometimes we need laws and policies or systematical change [5,30,66]. Last but not least, it is necessary to reflect on the relationship between science and society [46]. However, even if vaccine hesitancy cannot be eliminated, megastudies may still select the most effective interventions to reduce vaccine hesitancy as much as possible. Although not the complete picture, a megastudy will be a crucial and irreplaceable piece in reducing vaccine hesitancy.

Author Contributions

L.Y.: wrote and drafted the manuscript; J.Q.: participated in the sources and references; W.-K.M.: revised the manuscript and is co-corresponding author for this paper; Y.W.: carried out the design of the commentary paper and is co-corresponding author for this paper. All authors have read and agreed to the published version of the manuscript.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Not applicable.

Conflicts of Interest

The authors declare no conflict of interest.

Footnotes

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Figure

Figure 1. By Duckworth and Milkman, in A Guide to Megastudies. Left panel: traditional field experiments randomly assign participants to multiple conditions (e.g., conditions A and B), testing a limited number of related hypotheses. Right panel: megastudies randomly assign participants to a larger set of treatments often clustered by sub-study (different colors indicate different sub-studies), each testing potentially unrelated hypotheses (e.g., conditions A, B, C, D, etc.) [44].

References

1. World Health Organization. Ten-Threats-to-Global-Health-in-2019. 2019; Available online: https://www.who.int/news-room/spotlight/ten-threats-to-global-health-in-2019. (accessed on 16 October 2022).

2. World Health Organization. Vaccines and Immunization. 2022; Available online: https://www.who.int/health-topics/vaccines-and-immunization#tab=tab_1. (accessed on 23 November 2022).

3. Phadke, V.K.; Bednarczyk, R.A.; Salmon, D.A.; Omer, S.B. Association Between Vaccine Refusal and Vaccine-Preventable Diseases in the United States: A Review of Measles and Pertussis. JAMA; 2016; 315, pp. 1149-1158. [DOI: https://dx.doi.org/10.1001/jama.2016.1353] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/26978210]

4. Salmon, D.A.; Dudley, M.Z.; Glanz, J.M.; Omer, S.B. Vaccine hesitancy: Causes, consequences, and a call to action. Vaccine; 2015; 33, pp. D66-D71. [DOI: https://dx.doi.org/10.1016/j.vaccine.2015.09.035] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/26615171]

5. Leshem, E.; Lopman, B.A. Population immunity and vaccine protection against infection. Lancet; 2021; 397, pp. 1685-1687. [DOI: https://dx.doi.org/10.1016/S0140-6736(21)00870-9] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/33901422]

6. Wang, W.; Wu, Q.; Yang, J.; Dong, K.; Chen, X.; Bai, X.; Chen, X.; Chen, Z.; Viboud, C.; Ajelli, M. et al. Global, regional, and national estimates of target population sizes for COVID-19 vaccination: Descriptive study. BMJ; 2020; 371, m4704. [DOI: https://dx.doi.org/10.1136/bmj.m4704]

7. Dror, A.A.; Eisenbach, N.; Taiber, S.; Morozov, N.G.; Mizrachi, M.; Zigron, A.; Srouji, S.; Sela, E. Vaccine hesitancy: The next challenge in the fight against COVID-19. Eur. J. Epidemiol.; 2020; 35, pp. 775-779. [DOI: https://dx.doi.org/10.1007/s10654-020-00671-y]

8. Troiano, G.; Nardi, A. Vaccine hesitancy in the era of COVID-19. Public Health; 2021; 194, pp. 245-251. [DOI: https://dx.doi.org/10.1016/j.puhe.2021.02.025]

9. Our World in Data. Coronavirus (COVID-19) Vaccinations. 2022; Available online: https://ourworldindata.org/COVID-vaccinations (accessed on 27 October 2022).

10. Jarrett, C.; Wilson, R.; O’Leary, M.; Eckersberger, E.; Larson, H.J. Strategies for addressing vaccine hesitancy–A systematic review. Vaccine; 2015; 33, pp. 4180-4190. [DOI: https://dx.doi.org/10.1016/j.vaccine.2015.04.040]

11. Lane, S.; MacDonald, N.E.; Marti, M.; Dumolard, L. Vaccine hesitancy around the globe: Analysis of three years of WHO/UNICEF Joint Reporting Form data-2015–2017. Vaccine; 2018; 36, pp. 3861-3867. [DOI: https://dx.doi.org/10.1016/j.vaccine.2018.03.063]

12. Sallam, M. COVID-19 vaccine hesitancy worldwide: A concise systematic review of vaccine acceptance rates. Vaccines; 2021; 9, 160. [DOI: https://dx.doi.org/10.3390/vaccines9020160]

13. MacDonald, N.E. SAGE Working Group on Vaccine Hesitancy. Vaccine hesitancy: Definition, scope and determinants. Vaccines; 2015; 14, pp. 4161-4164. [DOI: https://dx.doi.org/10.1016/j.vaccine.2015.04.036]

14. Sripongpun, P.; Pinpathomrat, N.; Bruminhent, J.; Kaewdech, A. Coronavirus Disease 2019 Vaccinations in Patients with Chronic Liver Disease and Liver Transplant Recipients: An Update. Front. Med.; 2022; 9, 924454. [DOI: https://dx.doi.org/10.3389/fmed.2022.924454]

15. Nicholls, L.A.B.; Gallant, A.J.; Cogan, N.; Rasmussen, S.; Young, D.; Williams, L. Older adults’ vaccine hesitancy: Psychosocial factors associated with influenza, pneumococcal, and shingles vaccine uptake. Vaccine; 2021; 39, pp. 3520-3527. [DOI: https://dx.doi.org/10.1016/j.vaccine.2021.04.062]

16. Afolabi, I.B.; Aremu, A.B.; Maidoki, L.A.; Atulomah, N.O. Predictors of Hepatitis B Virus Infection Vaccine Hesitancy Among Pregnant Women Attending Antenatal Care at Lubaga Hospital, Kampala, Uganda. Int. J. Women’s Health; 2022; 14, pp. 1093-1104. [DOI: https://dx.doi.org/10.2147/IJWH.S378000] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/35999868]

17. Quinn, S.C.; Andrasik, M.P. Addressing vaccine hesitancy in BIPOC communities—Toward trustworthiness, partnership, and reciprocity. N. Engl. J. Med.; 2021; 385, pp. 97-100. [DOI: https://dx.doi.org/10.1056/NEJMp2103104] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/33789007]

18. Tankwanchi, A.S.; Bowman, B.; Garrison, M.; Larson, H.; Wiysonge, C.S. Vaccine hesitancy in migrant communities: A rapid review of latest evidence. Curr. Opin. Immunol.; 2021; 71, pp. 62-68. [DOI: https://dx.doi.org/10.1016/j.coi.2021.05.009]

19. Cénat, J.M.; Noorishad, P.-G.; Bakombo, S.M.; Onesi, O.; Mesbahi, A.; Darius, W.P.; Caulley, L.; Yaya, S.; Chomienne, M.-H.; Etowa, J. A systematic review on vaccine hesitancy in Black communities in Canada: Critical issues and research failures. Vaccines; 2022; 10, 1937. [DOI: https://dx.doi.org/10.3390/vaccines10111937]

20. Pires, C. Global predictors of COVID-19 vaccine hesitancy: A systematic review. Vaccines; 2022; 10, 1349. [DOI: https://dx.doi.org/10.3390/vaccines10081349]

21. Kumar, S.; Shah, Z.; Garfield, S. Causes of Vaccine Hesitancy in Adults for the Influenza and COVID-19 Vaccines: A Systematic Literature Review. Vaccines; 2022; 10, 1518. [DOI: https://dx.doi.org/10.3390/vaccines10091518]

22. Karafillakis, E.; Simas, C.; Jarrett, C.; Verger, P.; Peretti-Watel, P.; Dib, F.; De Angelis, S.; Takacs, J.; Ali, K.A.; Celentano, L.P. HPV vaccination in a context of public mistrust and uncertainty: A systematic literature review of determinants of HPV vaccine hesitancy in Europe. Hum. Vaccines Immunother.; 2019; 15, pp. 1615-1627. [DOI: https://dx.doi.org/10.1080/21645515.2018.1564436]

23. Ali, S.N.; Hanif, W.; Patel, K.; Khunti, K. Ramadan and COVID-19 vaccine hesitancy—A call for action. Lancet; 2021; 397, pp. 1443-1444. [DOI: https://dx.doi.org/10.1016/S0140-6736(21)00779-0]

24. Zhang, J.Z.; Ge, P.; Li, X.L.; Yin, M.; Wang, Y.J.; Ming, W.K.; Li, J.H.; Li, P.; Sun, X.Y.; Wu, Y.B. Personality Differences of Public Preference for the COVID-19 Vaccination: Discrete choice Experiment and Latent Profile Analysis Study. Int. J. Environ. Res. Public Health; 2022; 19, 4842. [DOI: https://dx.doi.org/10.3390/ijerph19084842] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/35457708]

25. Santhanes, D.; Wong, C.P.; Yap, Y.Y.; San, S.P.; Chaiyakunapruk, N.; Khan, T.M. Factors involved in human papillomavirus (HPV) vaccine hesitancy among women in the South-East Asian Region (SEAR) and Western Pacific Region (WPR): A scoping review. Hum. Vaccines Immunother.; 2018; 14, pp. 124-128. [DOI: https://dx.doi.org/10.1080/21645515.2017.1381811] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/28933635]

26. Nioi, M.; Napoli, P.E. The Waiver of Patent Protections for COVID-19 Vaccines during the ongoing Pandemic and the Conspiracy Theories: Lights and Shadows of an Issue on the Ground. Front. Med.; 2021; 8, 756623. [DOI: https://dx.doi.org/10.3389/fmed.2021.756623] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/34805221]

27. Pierri, F.; Perry, B.L.; DeVerna, M.R.; Yang, K.-C.; Flammini, A.; Menczer, F.; Bryden, J. Online misinformation is linked to early COVID-19 vaccination hesitancy and refusal. Sci. Rep.; 2022; 12, pp. 1-7. [DOI: https://dx.doi.org/10.1038/s41598-022-10070-w]

28. ECDC. Catalogue of Interventions Addressing Vaccine Hesitancy. 2017; Available online: https://www.ecdc.europa.eu/en/publications-data/catalogue-interventions-addressing-vaccine-hesitancy. (accessed on 25 April 2017).

29. Aguolu, O.G.; Malik, A.A.; Ahmed, N.; Omer, S.B. Overcoming Vaccine Hesitancy for Future COVID-19 and HIV Vaccines: Lessons from Measles and HPV Vaccines. Curr. HIV/AIDS Rep.; 2022; 19, pp. 328-343. [DOI: https://dx.doi.org/10.1007/s11904-022-00622-0]

30. Larson, H.J.; Gakidou, E.; Murray, C.J.L. The Vaccine-Hesitant Moment. N. Engl. J. Med.; 2022; 387, pp. 58-65. [DOI: https://dx.doi.org/10.1056/NEJMra2106441]

31. Afolabi, I.B.; Aremu, A.B.; Maidoki, L.A.; Atulomah, N.O. Dynamics of Hepatitis B infection prevention practices among pregnant women attending antenatal care at Lubaga Hospital Kampala, Uganda using the constructs of information-motivation-behavioural skills model. BMC Public Health; 2022; 22, 2243. [DOI: https://dx.doi.org/10.1186/s12889-022-14723-3]

32. Jiang, C.; Whitmore-Sisco, L.; Gaur, A.H.; Adderson, E.E.; Group, T.W. A quality improvement initiative to increase Tdap (tetanus, diphtheria, acellular pertussis) vaccination coverage among direct health care providers at a children’s hospital. Vaccine; 2018; 36, pp. 214-219. [DOI: https://dx.doi.org/10.1016/j.vaccine.2017.11.071]

33. Walkey, A.J.; Law, A.; Bosch, N.A. Lottery-Based Incentive in Ohio and COVID-19 Vaccination Rates. JAMA; 2021; 326, pp. 766-767. [DOI: https://dx.doi.org/10.1001/jama.2021.11048]

34. Kuperman, V. Virtual experiments in megastudies: A case study of language and emotion. Q. J. Exp. Psychol.; 2015; 68, pp. 1693-1710. [DOI: https://dx.doi.org/10.1080/17470218.2014.989865]

35. Keuleers, E.; Balot, D.A. Megastudies, crowdsourcing, and large datasets in psycholinguistics: An overview of recent developments. J. Exp. Psychol.; 2015; 68, pp. 1457-1468. [DOI: https://dx.doi.org/10.1080/17470218.2015.1051065] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/25975773]

36. Donoho, D. 50 years of data science. J. Comput. Graph Stat.; 2017; 26, pp. 745-766. [DOI: https://dx.doi.org/10.1080/10618600.2017.1384734]

37. Tsang, Y.K.; Zou, Y. An ERP megastudy of Chinese word recognition. Psychophysiology; 2022; 59, e1411. [DOI: https://dx.doi.org/10.1111/psyp.14111] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/35609148]

38. Milkman, K.L.; Gromet, D.; Ho, H.; Kay, J.S.; Lee, T.W.; Pandiloski, P.; Park, Y.; Rai, A.; Bazerman, M.; Beshears, J. Megastudies improve the impact of applied behavioural science. Nature; 2021; 600, pp. 478-483. [DOI: https://dx.doi.org/10.1038/s41586-021-04128-4]

39. Royer, H. Benefits of megastudies for testing behavioural interventions. Nature; 2021; 600, pp. 392-393. [DOI: https://dx.doi.org/10.1038/d41586-021-03400-x]

40. Milkman, K.L.; Gandhi, L.; Patel, M.S.; Graci, H.N.; Gromet, D.M.; Ho, H.; Kay, J.S.; Lee, T.W.; Rothschild, J.; Bogard, J.E. A 680,000-person megastudy of nudges to encourage vaccination in pharmacies. Proc. Natl. Acad. Sci. USA; 2022; 119, e2115126119. [DOI: https://dx.doi.org/10.1073/pnas.2115126119]

41. DiBonaventura, M.D.; Chapman, G.B. Moderators of the intention–behavior relationship in influenza vaccinations: Intention stability and unforeseen barriers. Psychol. Health; 2005; 20, pp. 761-774. [DOI: https://dx.doi.org/10.1080/14768320500183368]

42. Bronchetti, E.T.; Huffman, D.B.; Magenheim, E. Attention, intentions, and follow-through in preventive health behavior: Field experimental evidence on flu vaccination. J. Econ. Behav. Organ.; 2015; 116, pp. 270-291. [DOI: https://dx.doi.org/10.1016/j.jebo.2015.04.003]

43. Sheeran, P. Intention—Behavior Relations: A Conceptual and Empirical Review. Eur. Rev. Soc. Psychol.; 2002; 12, pp. 1-36. [DOI: https://dx.doi.org/10.1080/14792772143000003]

44. Duckworth, A.L.; Milkman, K.L. A guide to megastudies. PNAS Nexus; 2022; 1, 214. [DOI: https://dx.doi.org/10.1093/pnasnexus/pgac214]

45. Cooper, S.; Rooyen, H.V.; Wiysonge, C.S. COVID-19 vaccine hesitancy in South Africa: A complex social phenomenon. S. Afr. Med. J.; 2021; 8, pp. 702-703. [DOI: https://dx.doi.org/10.7196/SAMJ.2021.v111i8.15800] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/35227348]

46. Rosenbaum, L. Escaping catch-22—Overcoming COVID vaccine hesitancy. N. Engl. J. Med.; 2021; 384, pp. 1367-1371. [DOI: https://dx.doi.org/10.1056/NEJMms2101220] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/33577150]

47. Nioi, M.; d’Aloja, E.; Fossarello, M.; Napoli, P.E. Dual corneal-graft rejection after mrna vaccine (Bnt162b2) for covid-19 during the first six months of follow-up: Case report, state of the art and ethical concerns. Vaccines; 2021; 9, 1274. [DOI: https://dx.doi.org/10.3390/vaccines9111274]

48. Zhang, D.; Zhou, W.; Poon, P.K.-M.; Kwok, K.O.; Chui, T.W.-S.; Hung, P.H.Y.; Ting, B.Y.T.; Chan, D.C.-C.; Wong, S.Y.-S. Vaccine Resistance and Hesitancy among Older Adults Who Live Alone or Only with an Older Partner in Community in the Early Stage of the Fifth Wave of COVID-19 in Hong Kong. Vaccines; 2022; 10, 1118. [DOI: https://dx.doi.org/10.3390/vaccines10071118]

49. Vinograd, I.; Baslo, R.; Eliakim-Raz, N.; Farbman, L.; Taha, A.; Sakhnini, A.; Lador, A.; Stemmer, S.; Gafter-Gvili, A.; Fraser, D. Factors associated with influenza vaccination among adult cancer patients: A case–control study. Clin. Microbiol. Infect.; 2014; 20, pp. 899-905. [DOI: https://dx.doi.org/10.1111/1469-0691.12625]

50. Vallée, A.; Fourn, E.; Majerholc, C.; Touche, P.; Zucman, D. COVID-19 vaccine hesitancy among French people living with HIV. Vaccines; 2021; 9, 302. [DOI: https://dx.doi.org/10.3390/vaccines9040302]

51. Biswas, M.R.; Alzubaidi, M.S.; Shah, U.; Abd-Alrazaq, A.A.; Shah, Z. A scoping review to find out worldwide COVID-19 vaccine hesitancy and its underlying determinants. Vaccines; 2021; 9, 1243. [DOI: https://dx.doi.org/10.3390/vaccines9111243] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/34835174]

52. Zizza, A.; Banchelli, F.; Guido, M.; Marotta, C.; Di Gennaro, F.; Mazzucco, W.; Pistotti, V.; D’Amico, R. Efficacy and safety of human papillomavirus vaccination in HIV-infected patients: A systematic review and meta-analysis. Sci. Rep.; 2021; 11, 4954. [DOI: https://dx.doi.org/10.1038/s41598-021-83727-7]

53. Andrews, N.; Stowe, J.; Kirsebom, F.; Toffa, S.; Rickeard, T.; Gallagher, E.; Gower, C.; Kall, M.; Groves, N.; O’Connell, A.-M. Covid-19 vaccine effectiveness against the Omicron (B. 1.1. 529) variant. N. Engl. J. Med.; 2022; 386, pp. 1532-1546. [DOI: https://dx.doi.org/10.1056/NEJMoa2119451]

54. Nayar, R.K.; Nair, A.T.; Shaffi, M.; Swarnam, K.; Kumar, A.; Abraham, M.; Lordson, J.; Grace, C. Methods to overcome vaccine hesitancy. Lancet; 2019; 393, pp. 1203-1204. [DOI: https://dx.doi.org/10.1016/S0140-6736(19)30218-1]

55. Wilson, S.L. Wiysonge. C. Social media and vaccine hesitancy. BMJ Glob. Health; 2020; 5, e004206. [DOI: https://dx.doi.org/10.1136/bmjgh-2020-004206] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/33097547]

56. Loomba, S.; Figueiredo, A.D.; Piatek, S.J.L.; Graaf, K.D.; Larson, H.J. Measuring the impact of COVID-19 vaccine misinformation on vaccination intent in the UK and USA. Nat. Hum. Behav.; 2021; 5, pp. 337-348. [DOI: https://dx.doi.org/10.1038/s41562-021-01056-1] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/33547453]

57. Ahmed, S.; Rasul, M.E.; Cho, J. Social Media News Use Induces COVID-19 Vaccine Hesitancy Through Skepticism Regarding Its Efficacy: A Longitudinal Study From the United States. Front. Psychol.; 2022; 13, 900386. [DOI: https://dx.doi.org/10.3389/fpsyg.2022.900386] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/35756213]

58. Liu, T.; He, Z.; Huang, J.; Yan, N.; Chen, Q.; Huang, F.; Zhang, Y.; Akinwunmi, O.M.; Akinwunmi, B.O.; Zhang, C.J.P. et al. A Comparison of Vaccine Hesitancy of COVID-19 Vaccination in China and the United States. Vaccines; 2021; 9, 649. [DOI: https://dx.doi.org/10.3390/vaccines9060649]

59. Wouters, O.J.; Shadlen, K.C.; Salcher-Konrad, M.; Pollard, A.J.; Larson, H.J.; Teerawattananon, Y.; Jit, M. Challenges in ensuring global access to COVID-19 vaccines: Production, affordability, allocation, and deployment. Lancet; 2021; 397, pp. 1023-1034. [DOI: https://dx.doi.org/10.1016/S0140-6736(21)00306-8]

60. Berdzuli, N.; Datta, S.S. How to tackle inequitable access, vaccine hesitancy, and other barriers to achieve high vaccine uptake. BMJ; 2022; 377, o1094. [DOI: https://dx.doi.org/10.1136/bmj.o1094]

61. Kumar, D.; Chandra, R.; Mathur, M.; Samdariya, S.; Kapooret, N. Vaccine hesitancy: Understanding better to address better. Isr. J. Health Policy Res.; 2016; 5, 2. [DOI: https://dx.doi.org/10.1186/s13584-016-0062-y]

62. Gong, F.M.; Gong, Z.L.; Li, Z.; Min, H.W.; Zhang, J.Z.; Li, X.L.; Fu, T.T.; Fu, X.M.; He, J.B.; Wang, Z. et al. Impact of Media Use on Public Behavior towards Vaccination with the Covid-19 Vaccine: A Latent Profile Analysis. Vaccines; 2022; 19, 1737. [DOI: https://dx.doi.org/10.3390/vaccines10101737]

63. Kasstan, B. “If a rabbi did say ‘you have to vaccinate,’ we wouldn’t”: Unveiling the secular logics of religious exemption and opposition to vaccination”. Soc. Sci. Med.; 2021; 280, 114052. [DOI: https://dx.doi.org/10.1016/j.socscimed.2021.114052]

64. Voelkel, J.; Stagnaro, M.; Chu, J.; Pink, S.; Mernyk, J.; Redekopp, C.; Cashman, M.; Submitters, Q.; Druckman, J.; Rand, D. Megastudy Identifying Successful Interventions to Strengthen Americans’ Democratic Attitudes; Northwestern University: Evanston, IL, USA, 2022.

65. Flor, L.S.; Friedman, J.; Spencer, C.N.; Cagney, J.; Arrieta, A.; Herbert, M.E.; Stein, C.; Mullany, E.C.; Hon, J.; Patwardhan, V. et al. Quantifying the effects of the COVID-19 pandemic on gender equality on health, social, and economic indicators: A comprehensive review of data from March, 2020, to September, 2021. Lancet; 2022; 399, pp. 2381-2397. [DOI: https://dx.doi.org/10.1016/S0140-6736(22)00008-3]

66. Pingali, S.C.; Delamater, P.L.; Buttenheim, A.M.; Salmon, D.A.; Klein, N.P.; Omer, S.B. Associations of Statewide Legislative and Administrative Interventions With Vaccination Status Among Kindergartners in California. JAMA; 2019; 322, pp. 49-56. [DOI: https://dx.doi.org/10.1001/jama.2019.7924] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/31265099]