Impact of COVID-19-related knowledge on protective behaviors: The moderating role of primary sources of information

Abstract

This study assessed the modifying role of primary source of COVID-19 information in the association between knowledge and protective behaviors related to COVID-19 among adults living in the United States (US). Data was collected from 6,518 US adults through an online cross-sectional self-administered survey via social media platforms in April 2020. Linear regression was performed on COVID-19 knowledge and behavior scores, adjusted for sociodemographic factors. An interaction term between knowledge score and primary information source was included to observe effect modification by primary information source. Higher levels of knowledge were associated with increased self-reported engagement with protective behaviors against COVID-19. The primary information source significantly moderated the association between knowledge and behavior, and analyses of simple slopes revealed significant differences by primary information source. This study shows the important role of COVID-19 information sources in affecting people’s engagement in recommended protective behaviors. Governments and health agencies should monitor the use of various information sources to effectively engage the public and translate knowledge into behavior change during an evolving public health crisis like COVID-19.

About the Authors:

Sooyoung Kim

Roles Conceptualization, Formal analysis, Investigation, Methodology, Validation, Visualization, Writing – original draft, Writing – review & editing

Affiliation: Department of Public Health Policy and Management, New York University School of Global Public Health, New York, New York, United States of America

Ariadna Capasso

Roles Data curation, Investigation, Validation, Writing – review & editing

Affiliation: Department of Social and Behavioral Sciences, New York University School of Global Public Health, New York, New York, United States of America

ORCID logo https://orcid.org/0000-0001-5462-7800

Stephanie H. Cook

Roles Methodology, Validation, Writing – review & editing

Affiliations Department of Social and Behavioral Sciences, New York University School of Global Public Health, New York, New York, United States of America, Department of Biostatistics, New York University School of Global Public Health, New York, New York, United States of America

Shahmir H. Ali

Roles Data curation, Validation, Writing – review & editing

Affiliation: Department of Social and Behavioral Sciences, New York University School of Global Public Health, New York, New York, United States of America

ORCID logo https://orcid.org/0000-0002-0360-3507

Abbey M. Jones

Roles Data curation, Validation, Writing – review & editing

Affiliation: Department of Epidemiology, New York University School of Global Public Health, New York, New York, United States of America

Joshua Foreman

Roles Data curation, Validation, Writing – review & editing

Affiliation: Department of Social and Behavioral Sciences, New York University School of Global Public Health, New York, New York, United States of America

Ralph J. DiClemente

Roles Data curation, Supervision, Writing – review & editing

Affiliation: Department of Social and Behavioral Sciences, New York University School of Global Public Health, New York, New York, United States of America

Yesim Tozan

Roles Conceptualization, Data curation, Supervision, Validation, Writing – review & editing

* E-mail: [email protected]

Affiliations Department of Public Health Policy and Management, New York University School of Global Public Health, New York, New York, United States of America, Global and Environmental Public Health Program, New York University School of Global Public Health, New York, New York, United States of America

ORCID logo https://orcid.org/0000-0001-8997-8897

Introduction

The World Health Organization (WHO) declared COVID-19 a Public Health Emergency of International Concern (PHEIC) on January 30, 2020 [1]. The COVID-19 pandemic continues to cause significant global disruption across sectors ranging from healthcare to education to the economy. Millions of people around the world have been subjected to mitigation strategies, including stay-at-home restrictions, physical and social distancing, and mask wearing, all of which have necessitated substantial changes in individual behaviors for the collective good of the community. Despite recommendations by public health authorities, people have had varying levels of engagement with protective behaviors against COVID-19, and this has presented a major obstacle to the success of measures to control the pandemic [2–5].

Knowledge is shown to be positively associated with health protective behaviors [6–8]. During the current pandemic, an associated infodemic, defined as “an overabundance of information—some accurate and some not—”has made it challenging for people to find reliable and credible sources to acquire knowledge when they need it [9]. The start of COVID-19 as pneumonia of unknown aetiology [10] allowed for extensive speculation of the origins of the disease and the limited information at the time [11]. This led to delays in communication of the scientific knowledge about the disease by public health authorities. In addition, the rise of online communication mediums, such as social media, blogs, and podcasts, has resulted in user-generated content that is often disseminated without verification of its veracity and consumed at an unprecedented speed and scale by the public. A study conducted in six developed countries in April 2020 showed that while the majority of people used official news organizations as their primary source of information, about half of the participants reported also using Google or other online search and social media platforms for COVID-19-related information. Specifically, 25–53% of the participants across six countries reported using Facebook to obtain information on COVID-19 at least once over the past week, while 15–46% of the participants used YouTube for the same purpose [12].

The current challenge for public health authorities is, therefore, to strategize the dissemination of COVID-19-related information to counter the misinformation emanating from these easily accessible and often unregulated online sources and to deliver timely and correct information to the public supported by scientific evidence. During this pandemic, evidence has shown that people use a vast range of sources to get COVID-19-related information [13], and their choice of primary source reflects their trust in the legitimacy of these sources and affects their attitudes and vaccine uptake, as also supported by past research on vaccine hesitancy in general [14, 15]. Despite the growing volume of research on the COVID-19 infodemic, evidence is lacking on how the source from which people acquire COVID-19-related knowledge influences the association between knowledge and protective behaviors. Prior to COVID-19, the use of social media, including Wikipedia, blogs, and social networking services platforms, as a source of health-related information to affect behavior change was found effective but at the same time risky because of the trustworthiness of information depending on the information-seeking context and source [16–18]. Furthermore, no study to date has examined the role of different information sources in moderating the association between knowledge and behaviors while controlling for other factors.

In this study, we tested the hypothesis that different primary sources of COVID-19 information will act as an effect modifier of the relationship between levels of knowledge and self-reported engagement in protective behaviors. We focused on the primary source of information based on the idea that people’s choice of primary information source is a manifestation of their health consciousness and motivation for health-oriented behaviors [19]. The findings of this study have the potential to improve the targeting and effectiveness of risk communication strategies seeking to achieve behavior change.

Methods

Study participants and design

This study used data from an online survey conducted in April 2020. Details of the survey design and administration are reported elsewhere [20]. Briefly, the sample was recruited among social media users through an online advertisement campaign within Facebook and its affiliated platforms. The eligibility was limited to English-speaking adults aged 18 years and over residing in the United States (US). Participation was voluntary, and participants did not receive any compensation. For this study, we only included participants who provided written informed consent and responded to all behavior- and knowledge-related questions. As a result, a total of 6,518 responses were included in the analysis. The study protocol was reviewed and deemed exempt by the affiliated institution’s Institutional Review Board.

Questionnaires and variables of interest

The questionnaire was designed based on the Health Belief Model (HBM) [21] and the World Health Organization (WHO) survey tool for behavioral insights on COVID-19 [22].

Outcome variable.

The outcome of interest was the degree of self-reported engagement with recommended protective behaviors against COVID-19. For this, an index variable was derived from participants’ answers to a set of 13 binary questions. Answers were assessed based on the recommendations on protective behaviors provided by the Centers for Disease Control and Prevention (CDC). While correct answers were assigned a score of 1, answers that do not comply with the CDC recommendations at the time of the survey were scored 0. The sum of scores was used as the overall behavior score (Range: 0–13). Cronbach’s alpha coefficient was used to assess the internal consistency of the summed scores and was 0.7 and was deemed acceptable [23]. All the questions used to construct the outcome variable are provided in the S1 Table.

Predictor variables.

The main predictor was the level of knowledge on COVID-19. Similar to the behavior score, we created an index variable for the knowledge score using participants’ answers to a set of 21 binary questions. An overall knowledge score (Range: 0–21) was calculated by assigning 1 for the correct answers and 0 for the incorrect answers. We tested the internal consistency of the questions used to derive the sum of scores and Cronbach’s alpha coefficient was 0.60 for the knowledge score and was deemed acceptable [23]. All the questions used to construct the predictor variable are provided in the S1 Table.

Moderator variables.

Participant’s primary source of COVID-19-related information was used as a moderator. The questionnaire asked participants the primary source they used to acquire COVID-19-related information, with the ability to choose only one answer from a list of choices. Primary information sources were categorized into six mutually exclusive categories: (1) family, friends, and colleagues; (2) doctor or medical provider; (3) government or other official sources (e.g., CDC or WHO); (4) traditional media (e.g., newspapers, TV); (5) new media (e.g., social media, web surfing on non-official sources, podcasts); and (6) religious leaders.

Statistical analysis

All statistical analyses were conducted in R (version 3.6.3). The minimal dataset to replicate the analysis is provided in the S1 Dataset. First, we stratified the participants into groups by their primary source of information. We then used chi-square test to evaluate differences in demographic characteristics by group. Pairwise Wilcoxon rank-sum test—a nonparametric alternative to the t test—was used, given the left-skewedness of the scores’ distribution, to compare the distribution of knowledge and behavior scores between each group.

In order to test the hypothesis, ordinary least squares (OLS) linear regression was performed with behavior score as dependent variable, knowledge score as independent variable, and with an interaction term between knowledge score and primary source of information. Sociodemographic factors, including age, sex, employment status, education level, number of sources to acquire COVID-19-related information, and political affiliation, which may affect self-reported engagement with recommended protective behaviors, represented by vector Z in the equation below, were included as covariates [13, 24]. We reported adjusted regression coefficients and corresponding p-values and 95% confidence intervals (CIs).

(1)

Results

Description of sample

A total of 6,518 people participated in the survey from April 16 to 21, 2020. Of which, 1,984 (36.8%) participants indicated government or other official sources as their primary source of COVID-19-related information, followed by 1,792 (33.9%) who reported doctor or medical provider (Table 1). Traditional media channels were preferred by 721 (13.6%), and new media by 519 (9.8%) participants, whereas a small fraction (309; 5.8%) cited family, friends, or colleagues. Only three participants named religious leaders as their primary information source. Chi-square test showed almost all demographic variables, including age, sex, race, employment status, and political affiliation (p-value <0.01) were significantly differently distributed between each group, as shown in Table 1. Participants in older age groups preferred doctors or traditional media, whereas those who self-identified as non-white (p-value <0.001), Republican (p-value <0.001), or residing in a rural area (p-value = 0.011) were over-represented among those who indicated social media or family, friends, and colleagues as their primary information source. Standardized residuals of the chi-square test are reported in the S2 Table.

[Figure omitted. See PDF.]

Table 1. Demographics of the study participants (n = 6,518).

https://doi.org/10.1371/journal.pone.0260643.t001

The highest knowledge score was observed for participants who used traditional media (Median = 20, IQR 19–21), government or official sources (Median = 20, IQR 19–21), and doctors or medical providers as primary information source (Median = 20, IQR 19–20), followed by those who preferred new media (Median = 19, IQR 18–20) and family, friends, and colleagues (Median = 19, IQR 17–20). A similar trend was observed for the behavior score with less variability across primary information sources.

Regression analysis on the association between level of COVID-19 knowledge and degree of protective behaviors

The main effect model without the interaction term (adjusted R-square = 0.212), and the fully adjusted regression model with the moderator are reported in Table 2 (adjusted R-square = 0.224). In the main effect model, higher level of knowledge score was positively associated with higher degrees of engagement with protective behaviors against COVID-19. When controlled for all covariates, a unit increase in the knowledge score was associated with a 0.273 increase in the behavior score (95% CI: 0.241–0.305, p-value<0.01).

[Figure omitted. See PDF.]

Table 2. Main effect model and the full linear regression model between the COVID-19 knowledge score and the protective behavior score with covariates and the interaction term (n = 3,663).

https://doi.org/10.1371/journal.pone.0260643.t002

In the fully adjusted model, all interaction terms between the knowledge score and the primary source of information were significantly associated with the changes in the behavioral score (excluding religious leaders because a coefficient could not be derived due to the limited sample size (n = 3)). In summary, while the behavior score was positively associated with the knowledge score (adjusted coefficient 0.275, p-value <0.01), when all covariates were held constant, the association was significantly stronger when the primary source of information was social media, podcasts or unofficial websites (interaction term coefficient 0.1, p-value = 0.031), or family, friends and colleagues (interaction term coefficient 0.158, p-value <0.01), in comparison to when the primary source was through doctor or medical staff (reference category). On the contrary, the association was significantly weaker when the primary source of information was traditional media (interaction term coefficient -0.109, p-value = 0.044), or the government or other official sources (interaction term coefficient -0.096, p-value = 0.018). For all primary information sources, the increase in the behavioral score was larger with the increasing number of sources used (adjusted coefficient 0.156, 95% CI: 0.132–0.179, p-value<0.01). The F-test showed that model with interaction term fitted the data significantly better than the main effect model (p-value <0.01).

The visualization of this result as a fitted linear plot of the association between COVID-19 knowledge and behavior with the modifying effect of primary information source is shown in Fig 1. In this figure, the slope represents the association between knowledge and behavior. The varying slopes by different primary information source, as represented by different colors in the legend, corresponds to the effect modification as shown by the significant coefficients for interaction terms in Table 2.

[Figure omitted. See PDF.]

Fig 1. Fitted linear model for the association between COVID-19 knowledge (x-axis) and protective behaviors (y-axis) by primary information source of COVID-19.

https://doi.org/10.1371/journal.pone.0260643.g001

Discussion

Our study provides one of the first empirical evidence on effect modification by primary source of information on the association between knowledge and engagement with protective behaviors against COVID-19. The findings can be summarized in three points: First, the primary source of COVID-19 knowledge differs across sociodemographic subgroups, which may result in varying levels of knowledge related to COVID-19. Distinctively, participants with the lowest level of knowledge preferred informal sources, such as social media and family, friends and colleagues, as their primary source of information. Second, among those with high levels of knowledge, the primary source of information did not predict their protective behaviors. Lastly, the primary source of information significantly moderated the association between knowledge and behavior, and analyses of simple slopes revealed significant differences by primary information source (Fig 1). While our study design is cross-sectional and therefore does not allow us to draw any inference on causality, the results suggest that different communication media may deliver the same information in distinctive ways, which may lead to differing levels of knowledge across individuals and translate into different levels of engagement with behaviors.

This study brings particular attention to two sources of information: 1) online media sources, including social media and websites other than those of official governments or international organizations, and 2) informal communication between family, friends, and colleagues. In contrast to other information sources, acquisition of knowledge from both of these sources occurs interactively and informally. Not only were both a preferred source of COVID-19-related information among the study participants at the bottom quartile of the knowledge score distribution, but also these sources were associated with a higher increase in the level of engagement with protective behaviors given the same unit increase in the knowledge score, as demonstrated by steeper slopes in Fig 1. These two contrasting findings from our study suggest that these sources, when leveraged well, hold potential to empower people. By the same token, when misused, these information sources may present a health threat. The mechanism of how information presented through these informal sources can potentially be associated with behavior change could be an area of future research. Due to the limitation of the cross-sectional study design, it is not within the scope of the current study to test whether these two sources had indeed diminished the level of both correct knowledge and engagement with protective behaviors. Regardless, the results suggest that, when these sources are leveraged appropriately to improve knowledge, the translation of knowledge to behavior among the participants who primarily uses these sources could be effective.

In summary, primary sources of information may be partially accountable for varying levels of COVID-19-related knowledge, reflecting different sociodemographic characteristics of the main audience of each source, and its heterogeneous associations with individuals’ engagement with protective behaviors against COVID-19. Our results suggest that the primary source of information may act as a moderator in the pathway from knowledge to behavior, and sources of information and the manner in which each source conveys information to the public could serve as the tangible target of intervention for improved risk communication.

The study has a number of limitations that leave room for further research. First, the study design introduces a number of biases. Despite the large sample size, the study sample, drawn from nonprobability convenience sampling via social media platforms affiliated with Facebook, is not representative of the US population [20]. While our sample showed a balanced distribution of participants from every US state, age group, and type of residence, certain key subpopulations are underrepresented in the study sample. For example, the survey did not include the people without access to Internet or social media account affiliated with Facebook. While around 70% of the US population are estimated to have Facebook accounts, and among them, 75% use Facebook on a daily basis [25], it is also estimated that about 20% of the US households do not have Internet at home [26]. In addition, more than 40 million adults in the US are of foreign origin, and almost one third of them do not speak English well, thus would be unable to participate in our study [27]. Our sample of participants was also overwhelmingly non-Hispanic white. Overall, high scores of COVID-19-related knowledge observed in this study could have been due to the sampling strategy, which most likely attracted people who were more interested in COVID-19. Meanwhile, the small sample size of those who reported seeking COVID-19-related information primarily from religious leaders could be due to the lack of engagement of this specific group in social media platforms in general [28]. To partially overcome this limitation, we made substantial efforts to oversample from potentially under-represented groups, including men and racial and ethnic minorities [20]. However, it is important to note that, due to all of the aforementioned factors, the findings from our study are not generalizable to the US population as a whole. We still strongly believe that our findings hold important implications for designing risk communication strategies, targeting particularly those who use social media platforms as their primary source of information.

Second, level of engagement with protective behaviors was based on self-report, which is subject to response bias. Although presented in random order, the questions on engagement with protective behaviors were relatively simple and dichotomized, and it was straightforward to guess the correct answers; this may have resulted in over-estimation of the level of engagement in specific subgroups. Moreover, given the nature of the observational design, our study is subject to many known and unknown confounders, which should be addressed by quasi-experimental or experimental studies.

Additionally, measurements of the level of knowledge and behavioral engagement were done via questions developed and used in previous surveys [20], but we used unvalidated scales based on the sum of correct answers to the survey questions without being able to test their validity. Internal consistency of the questions was tested and shown to be reliable for the behavior score, but less reliable for the knowledge score. Future studies using validated scales to measure behaviors and knowledge are needed.

Lastly, overall high knowledge scores among the study participants and the resulting lack of variability in the independent variable may be regarded as a limitation to the analysis. A supplementary qualitative study of the participants could facilitate a deeper understanding of the association between knowledge and behaviors in order to better inform risk communication strategies.

Conclusions

As the COVID-19 pandemic persists over a year since its first detection, more and more people are feeling fatigued about continuous engagement with protective measures [29, 30]. However, it is crucial to maintain public vigilance and a collectively high level of engagement with protective behaviors in order to prevent further spread of the virus, particularly because of the slow progress with COVID-19 vaccine implementation and the emergence of several variants of the COVID-19 virus [31]. Even small deficiencies in protective behavior compliance can result in significant community spread, as evidenced by the second and third waves of COVID-19 cases in several countries, including the US [32, 33]. With the concurrent infodemic severely hindering the ability of health authorities to convey timely and accurate information, customizing risk communications with appropriate prioritization of high-risk populations and information platforms with stronger capacity to promote behavior change (such as online or interpersonal information sources) is of absolute importance. In order to effectively promote community-level protective behaviors, it is necessary to step out of the traditional way of mass-scale and didactic communication and proactively reach out and engage through the channels where people are more willing to seek information.

Supporting information

S1 Table. Questionnaire used in the survey.

https://doi.org/10.1371/journal.pone.0260643.s001

(DOCX)

S2 Table. Standardized residuals of chi-square test performed on the demographics of the study participants (n = 6,518).

https://doi.org/10.1371/journal.pone.0260643.s002

(DOCX)

S1 Dataset. Study dataset.

https://doi.org/10.1371/journal.pone.0260643.s003

(CSV)

Citation: Kim S, Capasso A, Cook SH, Ali SH, Jones AM, Foreman J, et al. (2021) Impact of COVID-19-related knowledge on protective behaviors: The moderating role of primary sources of information. PLoS ONE 16(11): e0260643. https://doi.org/10.1371/journal.pone.0260643

References

1. World Health Organization. COVID 19 Public Health Emergency of International Concern (PHEIC) Global research and innovation forum: towards a research roadmap2020 19 October 2020. Available from: https://www.who.int/publications/m/item/covid-19-public-health-emergency-of-international-concern-(pheic)-global-research-and-innovation-forum.

2. Chernozhukov V, Kasahara H, Schrimpf P. Causal impact of masks, policies, behavior on early covid-19 pandemic in the U.S. J Econom. 2021;220(1):23–62. Epub 2020/10/27. pmid:33100476; PubMed Central PMCID: PMC7568194.

3. Zajenkowski M, Jonason PK, Leniarska M, Kozakiewicz Z. Who complies with the restrictions to reduce the spread of COVID-19?: Personality and perceptions of the COVID-19 situation. Pers Individ Dif. 2020;166:110199. Epub 2020/06/23. pmid:32565591; PubMed Central PMCID: PMC7296320.

4. Dzisi EKJ, Dei OA. Adherence to social distancing and wearing of masks within public transportation during the COVID 19 pandemic. Transportation Research Interdisciplinary Perspectives. 2020;7. pmid:34173465

5. Howard J, Huang A, Li Z, Tufekci Z, Zdimal V, van der Westhuizen HM, et al. An evidence review of face masks against COVID-19. Proc Natl Acad Sci U S A. 2021;118(4). Epub 2021/01/13. pmid:33431650; PubMed Central PMCID: PMC7848583.

6. Wee EG, Giri MS, Sundram TK, Venudran CV. COVID-19: Knowledge, Attitude and Preventive Behaviours of Medical and Dental Students International Journal of Biomedical and Clinical Sciences. 2020;5(3):236–56. Epub 24 Sep 2020.

7. Lee M, Kang B-A, You M. Association Between Knowledge, Attitudes and Practices (KAP) Towards The COVID-19: A Cross-Sectional Study in South Korea. BMC Public Health2020.

8. Hua F, Qin D, Yan J, Zhao T, He H. COVID-19 Related Experience, Knowledge, Attitude, and Behaviors Among 2,669 Orthodontists, Orthodontic Residents, and Nurses in China: A Cross-Sectional Survey. Front Med (Lausanne). 2020;7:481. Epub 2020/08/28. PubMed Central PMCID: PMC7427309. pmid:32850923

9. World Health Organization. Infodemic Management—Infodemiology Geneva, Switzerland2020 [cited 2020 16 October 2020]. Available from: https://www.who.int/teams/risk-communication/infodemic-management.

10. Pneumonia of unknown cause–China [Internet]. 2020. DIsease Outbreak News; January 5, 2020. Available from: https://www.who.int/emergencies/disease-outbreak-news/item/2020-DON229

11. Hou Z, Du F, Zhou X, Jiang H, Martin S, Larson H, et al. Cross-Country Comparison of Public Awareness, Rumors, and Behavioral Responses to the COVID-19 Epidemic: Infodemiology Study. J Med Internet Res. 2020;22(8):e21143. Epub 2020/07/24. pmid:32701460; PubMed Central PMCID: PMC7402643.

12. Nielsen RK, Fletcher R, Newman N, Brennen JS, Howard PN. Navigating the ‘Infodemic’: How People in Six Countries Access and Rate News and Information about Coronavirus. the Reuters Institute for the Study of Journalism 2020 15 April 2020. Report No.

13. Ali SH, Foreman J, Tozan Y, Capasso A, Jones AM, DiClemente RJ. Trends and Predictors of COVID-19 Information Sources and Their Relationship With Knowledge and Beliefs Related to the Pandemic: Nationwide Cross-Sectional Study. JMIR Public Health Surveill. 2020;6(4):e21071. Epub 2020/09/17. pmid:32936775.

14. Yaqub O, Castle-Clarke S, Sevdalis N, Chataway J. Attitudes to vaccination: a critical review. Soc Sci Med. 2014;112:1–11. Epub 2014/05/03. pmid:24788111.

15. Park S, Massey PM, Stimpson JP. Primary Source of Information About COVID-19 as a Determinant of Perception of COVID-19 Severity and Vaccine Uptake: Source of Information and COVID-19. J Gen Intern Med. 2021;36(10):3088–95. Epub 2021/08/12. pmid:34378115; PubMed Central PMCID: PMC8354304.

16. Park M, Sun Y, McLaughlin ML. Social Media Propagation of Content Promoting Risky Health Behavior. Cyberpsychol Behav Soc Netw. 2017;20(5):278–85. Epub 2017/05/13. pmid:28498046.

17. Kim K-S, Yoo-Lee E, Joanna Sin S-C. Social media as information source: Undergraduates’ use and evaluation behavior. Proceedings of the American Society for Information Science and Technology. 2011;48(1):1–3. https://doi.org/10.1002/meet.2011.14504801283

18. Maher CA, Lewis LK, Ferrar K, Marshall S, De Bourdeaudhuij I, Vandelanotte C. Are health behavior change interventions that use online social networks effective? A systematic review. J Med Internet Res. 2014;16(2):e40. Epub 2014/02/20. pmid:24550083; PubMed Central PMCID: PMC3936265.

19. Dutta-Bergman MJ. Primary sources of health information: comparisons in the domain of health attitudes, health cognitions, and health behaviors. Health Commun. 2004;16(3):273–88. Epub 2004/07/22. pmid:15265751.

20. Ali SH, Foreman J, Capasso A, Jones AM, Tozan Y, DiClemente RJ. Social media as a recruitment platform for a nationwide online survey of COVID-19 knowledge, beliefs, and practices in the United States: methodology and feasibility analysis. BMC Med Res Methodol. 2020;20(1):116. Epub 2020/05/15. pmid:32404050; PubMed Central PMCID: PMC7220591.

21. Rosenstock IM. The Health Belief Model and Preventive Health Behavior. Health Education Monographs. 1974;2(4):354–86.

22. World Health Organization. Regional Office for Europe. Survey tool and guidance: rapid, simple, flexible behavioural insights on COVID-19. Copenhagen, Denmark: 2020 29 July 2020. Report No.

23. Hulin C, Netemeyer R, Cudeck R. Can a Reliability Coefficient Be Too High? Journal of Consumer Psychology. 2001;10(1/2):55–8.

24. Painter M, Qiu T. Political beliefs affect compliance with covid-19 social distancing orders. Available at SSRN 3569098. 2020.

25. Smith A, Anderson M. Social Media Use in 2018 Washington, D.C.: Pew Research Center; 2018 [updated March 1, 2018; cited 2020 March 29]. Available from: https://www.pewresearch.org/internet/2018/03/01/social-media-use-in-2018/.

26. U.S. Census Bureau. Quick Facts 2019 [cited 2020 November 3]. Available from: https://www.census.gov/quickfacts/fact/table/US/AGE135219#AGE135219.

27. Gambino CP, Acosta YD, Grieco EM. English-Speaking Ability of the Foreign-Born Population in the United States: 2012. Washinton, D.C.: U.S. Census Bureau, 2014.

28. Jansen J. Part 3: Technology and religious group members. Pew Research Group, 2011 December 23, 2011. Report No.

29. Michie S, West R, Harvey N. The concept of “fatigue” in tackling covid-19. bmj. 2020;371.

30. Islam AN, Laato S, Talukder S, Sutinen E. Misinformation sharing and social media fatigue during COVID-19: An affordance and cognitive load perspective. Technological Forecasting and Social Change. 2020;159:120201. pmid:32834137

31. Dooling K, Marin M, Wallace M, McClung N, Chamberland M, Lee GM, et al. The Advisory Committee on Immunization Practices’ Updated Interim Recommendation for Allocation of COVID-19 Vaccine—United States, December 2020. Morbidity and Mortality Weekly Report (MMWR). 2021;69(5152):1657–60. pmid:33382671

32. Dyke MEV, Rogers TM, Pevzner E, Satterwhite CL, Shah HB, Beckman WJ, et al. Trends in County-Level COVID-19 Incidence in Counties With and Without a Mask Mandate—Kansas, June 1–August 23, 2020. Morbidity and Mortality Weekly Report (MMWR). 2020;69(47):1777–81. https://www.cdc.gov/mmwr/volumes/69/wr/mm6947e2.htm?s_cid=mm6947e2_w.

33. Anderson RM, Heesterbeek H, Klinkenberg D, Hollingsworth TD. How will country-based mitigation measures influence the course of the COVID-19 epidemic? The Lancet. 2020;395(10228):931–4. pmid:32164834