Comparison of hospitalist morale in a COVID-19 alternate care site (ACS) to hospitalist morale in conventional hospitals in Maryland

Abstract

Background

Morale and burnout were concerns for hospitalists prior to the COVID-19 pandemic; these concerns were amplified as COVID-19 spread and hospitals experienced unprecedented stress. In contrast to prior literature, our study assesses both satisfaction and the importance of various factors. This study examines morale of hospitalists early in the COVID-19 pandemic in two settings: conventional hospitals and a COVID-19 Alternate Care site (ACS) in the same geographic region in Maryland. Multiple studies published early in the pandemic show low morale in COVID-19 hospitals.

Methods

In a cross-sectional survey study, we analyze data from the Hospitalist Morale Index (HMI) administered between September 2020 and March 2021 to determine the pandemic’s impact on hospitalist morale.

Results

Surprisingly, our study found morale in the ACS was better than morale at the conventional hospitals. ACS hospitalists and conventional hospitalists were demographically similar. Our results show that a significantly higher proportion of conventional hospitalists reported burnout compared to the ACS hospitalists. General quality of life was rated significantly higher in the ACS group than the conventional group. Significantly more ACS hospitalists were invested in making their group outstanding. Five main HMI domains were examined with questions on a 5-point rating scale: Clinical Factors, Workload, Material Rewards, Leadership, and Appreciation/Acknowledgement. ACS hospitalists rated most measures higher than conventional hospitalists; largest differences were observed in Clinical Factors and Appreciation/Acknowledgement domains. Narrative comments from ACS hospitalists revealed strong identification with the mission of the ACS and pride in contributing during a crisis. One key difference between the two groups explains these findings: provider autonomy. The ACS staff chose the position and the assignment, while conventional hospitalists caring for COVID-19 patients could not readily opt out of this work.

Conclusion

Our data suggest that autonomy in assignments with risk has implications for morale and burnout.

Introduction

Healthcare worker burnout was high prior to the COVID-19 pandemic [1], but the unique confluence of circumstances presented by COVID-19, such as high contagiousness with risk to caregivers and the duration of the pandemic, exacerbated this crisis. The early pandemic stressed healthcare capacity globally with high occupancy of Intensive Care Unit (ICU) beds, and in some locations, adoption of crisis standards of care [2]. The Healthcare Worker Exposure Response and Outcomes (HERO) registry, created to study the impact of the pandemic, surveyed over 55,000 U.S. healthcare workers including physicians [3]. Using HERO registry data, Nieuwsma [4] found rates of potential moral injury, witnessing immoral acts by others, to be similar in COVID-19 healthcare workers and post 9/11 combat veterans. A provider focus-group study captured the breadth and depth of what providers felt they needed from their healthcare institutions: "Hear me," "Protect me," "Support me," "Prepare me," and "Care for me," [5].

Alternate Care Sites (ACSs) were created or re-purposed in some locations to accommodate surges of COVID-19 patients [6]. Some ACSs were temporary hospitals that already existed prior to the pandemic and were not ideally suited to manage patients with contagious respiratory infections [7], some were created at the beginning of the pandemic when personal protective equipment supplies were insufficient [8]. Morale in ACSs has been threatened by long hours [7], fears of infection or infecting loved ones [9], forced separation from family [10], lack of sufficient personal protective equipment, and illness and death of colleagues [9]. Data from COVID-19 hospitals and ACSs show low morale and high rates of insomnia, anxiety and depression in hospitalists and nurses [7, 11, 12]. Studies of morale in healthcare workers during COVID-19 are a fresh area of inquiry. There is a paucity of data comparing morale in different settings and our study adds findings that suggest a given setting’s policies affect clinician morale. We compared morale at the Baltimore Convention Center Field Hospital (BCCFH), a COVID-19 ACS, to regional conventional hospitals using the validated Hospitalist Morale Index (HMI) [13] between September 2020 and March 2021. The HMI is an instrument designed to assess morale, well-being, and burnout. The objective of the present study was to compare hospitalist morale at a COVID-19 ACS to hospitalist morale in conventional hospitals in the same region during the COVID-19 pandemic and identify areas for potential interventions.

Methods

Setting

The BCCFH was a Maryland Department of Health COVID-19 ACS jointly operated by Johns Hopkins Medicine and the University of Maryland. It was in continuous operation for inpatient care from April 2020-June 2021, cared for 1,495 COVID-19 admissions, and served as a free public mass testing and vaccination site. It continues in operation today providing monoclonal antibodies and test-to-treat care.

Participants

We completed a cross-sectional survey of two groups of hospitalists via e-mail link in September 2020 (5 conventional hospitalist programs within a regional healthcare network, n = 183) as scheduled and in March 2021 (BCCFH hospitalists at a single ACS, n = 69) after state and additional approvals. Those working in the BCCFH cared exclusively for patients with COVID-19, admitted directly from emergency departments or from other hospitals, who met specific requirements such as no more frequently than daily blood work, oxygen requirements of ≤4L by nasal cannula, and intermittent, not continuous, intravenous medications. Total assistance/bedbound and patients lacking capacity were excluded on referral. BCCFH clinicians who had performed clinical shifts within the past 3 months were included in this survey. A detailed description of the recruitment and hiring process for the BCCFH has been described elsewhere [14]. Briefly, clinicians were licensed physicians and experienced (minimum two years) advanced practice providers with internal medicine, family practice, emergency medicine, or surgical training. The conventional hospitalists were employed at academic medical centers and community hospitals in the Baltimore region and rotated on both inpatient non-intensive care COVID-19 and non-COVID-19 units. Moonlighters, including those in fellowship or who worked at multiple institutions, were excluded from the survey. Consent to complete the survey was completed online prior to entering the survey. This study was reviewed and approved by the Johns Hopkins University Institution Review Board (IRB00249880) on June 25, 2020.

Survey instrument

The survey instrument was the validated Hospitalist Morale Index (HMI), which assesses self-reported quality of life, burnout (emotional exhaustion and/or depersonalization) [15], commitment to group excellence and thoughts of leaving the current group. The HMI is comprised of 5 main domains (Clinical factors, Workload, Material Rewards, Leadership, and Appreciation/Acknowledgement), each with several subdomains, and 5 single item indicators (Overall culture, Job security, Autonomy/Independence, Work/life balance and Professional growth opportunities). Overall, domain and single item indicator scores are weighted means of items based on importance and satisfaction ratings, ranging from 0 (low) to 5 (high). Demographic factors surveyed included sex, race/ethnicity, and age. Clinical factors included provider academic rank or position and years as a hospitalist. The survey also included an opportunity to provide narrative comments.

Using a convenience sample, the HMI was administered over a 3-week period, with reminders two times a week with alternating weekend days targeted to those who did not respond. The confidential survey with personalized links and reminders was distributed via email using Qualtrics^XM® software. Personalized links can only be completed once to prevent multiple responses. Once the survey was closed, any identifiers were removed before analysis.

Statistical analysis

Statistical analyses employing t-tests, chi-squared tests, and ANOVA examined differences in continuous and categorical factors, and HMI respectively, between the conventional and BCCFH hospitalist groups (STATA^®13).

Results

Of the 69 BCCFH hospitalists, 55 (80%) responded to the survey; of the 183 conventional hospitalists, 141 (77%) responded. Demographically, the hospitalist groups were similar, apart from ethnicity (Table 1), which varied significantly. More BCCFH than conventional hospitalists were African American (30% versus 5% respectively, p<0.01). Fewer BCCFH than conventional hospitalists were Asian (21% versus 42% respectively, p<0.01). Caucasian and Other categories were similar at both sites. Provider academic rank or position differed significantly, with fewer faculty physicians and more nurse practitioners/physician assistants at the BCCFH. Forty-five percent of BCCFH hospitalists were nonfaculty physicians, 9% faculty physicians and 45% NPPAs, versus 51% nonfaculty physicians, 38% faculty physicians and 11% nurse practitioner/physician assistants for conventional hospitalists (p<0.001).

[Figure omitted. See PDF.]

Baltimore Convention Center Field Hospital (BCCFH) Versus Conventional Hospitalists.

Measurements of self-reported burnout, with subcategories of emotional exhaustion and depersonalization, quality of life, morale as a hospitalist in current group, thoughts of leaving the current group and commitment to making the group outstanding differed significantly between the two study populations (Table 2). Four percent of BCCFH hospitalists versus 23% of conventional hospitalists endorsed burnout (p<0.01), which was defined as emotional exhaustion and/or depersonalization occurring once a week or more. Four percent of BCCFH hospitalists experienced emotional exhaustion weekly or more often, compared to 20% of conventional hospitalists (p<0.01). Depersonalization experienced weekly or more often was endorsed by none of BCCFH hospitalists versus 10% of the conventional hospitalists (p = 0.01). Significantly more BCCFH than conventional hospitalists reported good quality of life, 91% versus 70% (p<0.01). Quality of life in the HMI was defined: as good as it can be, or good, versus okay, somewhat bad, or as bad as it can be. Self-reported “morale as a hospitalist in my current group,” the top three quintiles (excellent, very good or good versus fair or poor), was endorsed by 95% of BCCFH hospitalists versus 74% of conventional hospitalists (p<0.01). “Morale of my current group,” defined as excellent, very good or good versus fair or poor, was rated higher at BCCFH than at conventional hospitals but did not reach statistical significance (85% versus 74%, p = 0.09). Likewise, for the category “recommend my current group as great to work with,” the top two quartiles (absolutely or yes versus no or definitely no) were chosen by 98% of BCCFH hospitalists versus 92% of conventional hospitalists (p = 0.19). Significantly fewer BCCFH than conventional hospitalists had serious thoughts of leaving their group, defined as absolutely or yes versus no or definitely no because they were unhappy (13% versus 28%, p = 0.04). “Commitment to making my hospitalist group outstanding,” was highly rated, defined as tremendously or quite a lot versus somewhat, a little or not at all, by 91% of BCCFH hospitalists, versus 67% of conventional hospitalists (p<0.01).

[Figure omitted. See PDF.]

BCCFH Versus Conventional Hospitalists.

As part of the HMI score, 5 main domains and 5 single item indicators were examined on a 5-point scale (Table 3). In the Clinical Factors domain, BCCFH hospitalists rated ratio of face time to documentation significantly higher than conventional hospitalists (3.10 versus 2.36, p<0.001), but rated relationship with consultants significantly lower (2.16 versus 2.61, p = 0.03). In the Workload domain, BCCFH hospitalists rated percentage of night shifts, daily patient census and total shifts per schedule block more negatively than conventional hospitalists (1.15 versus 2.63, p<0.001; 2.01 versus 3.14, p<0.001; 2.06 versus 3.09, p<0.001, respectively). The comments show that BCCFH survey respondents wanted more total shifts and more night shifts, given the compensation model, rather than concerns for excessive workload. In the Material Rewards domain, BCCFH hospitalists ranked pay better than conventional hospitalists (3.17 versus 2.18, p<0.001), and benefits worse (2.04 versus 2.45, p = 0.04). In the Leadership domain subcategories of leadership fairness, effectiveness, approachability, and availability, amongst others, BCCFH and conventional hospitalists ranked their leadership similarly, although BCCFH hospitalist rankings were slightly higher, but not statistically significantly different. In the Appreciation/Acknowledgement domain, BCCFH again outranked conventional hospitalists, most clearly in the subdomain of “feeling valued in your organization” (3.25 versus 2.61, respectively, p = <0.01). For single item indicators, the only statistically significant difference was lower job security for BCCFH compared with conventional hospitalists (2.33 versus 3.41 respectively, p<0.001). Narrative comments from both sites directly relating to morale are shown in Table 4 and show strong site clinician identification with the mission of the BCCFH.

[Figure omitted. See PDF.]

BCCFH Versus Conventional Hospitalists.

[Figure omitted. See PDF.]

Related to Morale in the HMI Survey.

Discussion

Our study compares morale in a COVID-19 ACS to that in conventional hospitals and reports higher morale among providers at the BCCFH ACS compared to hospitalists at conventional hospitals. This contrasts with other studies which have reported low morale at other COVID-19 ACSs [9]. Other studies have shown high rates of burnout [12] anxiety [11] and insomnia [9] in many providers caring for COVID-19 patients in ACSs and elsewhere early in the pandemic. Given the low morale widely reported in other COVID-19 ACSs, our finding of significantly higher morale in the BCCFH, compared with local conventional hospitals, is unexpected. Morale is a concept that captures not only job satisfaction, but also contentment and happiness [13]. In contrast to prior literature, our study assesses both satisfaction and the importance of each of these domains. Future studies should explore how to increase clinicians’ feelings of being valued in an organization, increase face time with patients, reduce administrative burden, and engender autonomy, which have all been shown to be improve morale. Specific measures that have been used in our hospital medicine group to build morale during the pandemic include frequent update meetings, respite stations in COVID-19 units, remote social activities and “shoutouts” to highlight individual accomplishments. At the BCCFH, the sense of mission was important. The field hospital gave providers a way to respond to a crisis in a hands-on way. They were excited to build new things, such as new service lines (M. E. Kantsiper, personal communication April 11, 2022). We believe high morale resulted from the staff’s strong identification with the mission of the BCCFH, as shown in many of the open-ended survey comments, and from their being a self-selected group from a pool of local providers. Whereas conventional hospitals were struggling to increase their capacity and asking their hospitalists to do more and different activities from their usual work, this ACS was specifically set up to treat COVID-19 patients.

A key difference between BCCFH and conventional hospitalists was in freedom of choice to care for COVID-19 patients. The COVID-19 ACSs described in earlier studies were staffed by individuals that may not have chosen the assignment [10] and similarly, for conventional hospitalists, there was no choice as to whether to care for COVID-19 patients. Significant differences between the two groups were observed in the Clinical, Workload, and Appreciation/Acknowledgement domains, and in self-rated measures of burnout, professional satisfaction, and quality of life. Notably, the only employment benefit for BCCFH hospitalists was paid leave in case of infection with COVID-19, yet the respondents rated benefits as nearly as highly as conventional hospitalists who had full employment benefits. The unique structure and environment of the BCCFH led to a higher ratio of face time to documentation and to more hospitalists feeling valued in their organization. Future studies should explore how to increase clinicians’ sense of being valued by their organization and how to increase face time with patients which have been shown to be indicators of hospitalist morale, in addition to exploring ways to minimize administrative burden on clinicians.

Potential study limitations include that it is a single ACS site survey and that the conventional hospitalists were surveyed in the fall of 2020 when a vaccine was not yet available, and the ACS hospitalists were surveyed just after. While vaccination timing may have boosted overall morale in the ACS group, we do not believe it explains all the differences between the two groups in all the domains surveyed, and no one commented on vaccination in the open-ended comments. Also, studies have shown providers have experienced significant mental health symptoms during the pandemic in conventional hospitals, and this may have mitigated any potential differences [4, 16]. Differences in the patient populations of the sites may have played a small role in morale, however patient acuity was not mentioned in providers’ comments from either site. None of the hospitalists provided ICU level of care. The response rates for BCCFH and conventional hospitalists were 80% and 77% respectively, exceeding the suggested threshold of 60%, and minimizing concerns for non-response bias [17]; review of respondent demographics with site leadership found the data to be representative of the individual sites.

This study shows that caring for COVID-19 patients per se does not necessarily produce low morale; even in a COVID-19 ACS, hospitalists were highly engaged in this mission and few reported burnout. This survey demonstrates the impact of face time with patients, feeling valued in one’s organization, and compensation on hospitalist morale. Because autonomy and freedom of choice are drivers of provider morale and satisfaction, future policies in hospitalist staffing should consider individual clinicians’ desire to contribute to new initiatives and prioritize autonomy in assignments that involve risk.

Acknowledgments

Bayview Education and Academic Research (BEAR) Core Consortium members

*Henry Michtalik MD, MPH, MHS, Division of Hospital Medicine, Johns Hopkins Bayview Medical Center, Johns Hopkins University, Baltimore, Maryland, USA [email protected]

Shaker Eid MBA, MD, Division of Hospital Medicine, Johns Hopkins Bayview Medical Center, Johns Hopkins University, Baltimore, Maryland, USA

Neal Shah MD, Division of Hospital Medicine, Johns Hopkins Bayview Medical Center, Johns Hopkins University, Baltimore, Maryland, USA

Regina Kauffman BA, Division of Hospital Medicine, Johns Hopkins Bayview Medical Center, Johns Hopkins University, Baltimore, Maryland, USA

Flora Kisuule MD, Division of Hospital Medicine, Johns Hopkins Bayview Medical Center, Johns Hopkins University, Baltimore, Maryland, USA

Venkat Gundareddy MD, MPH, Division of Hospital Medicine, Johns Hopkins Bayview Medical Center, Johns Hopkins University, Baltimore, Maryland, USA

Scott Wright MD, Division of Hospital Medicine, Johns Hopkins Bayview Medical Center, Johns Hopkins University, Baltimore, Maryland, USA

Che Harris MD, MS, Division of Hospital Medicine, Johns Hopkins Bayview Medical Center, Johns Hopkins University, Baltimore, Maryland, USA

Ivonne Pena MD, Division of Hospital Medicine, Johns Hopkins Bayview Medical Center, Johns Hopkins University, Baltimore, Maryland, USA

Amteshwar Singh MD, Division of Hospital Medicine, Johns Hopkins Bayview Medical Center, Johns Hopkins University, Baltimore, Maryland, USA

Susrutha Kotwal MD, Division of Hospital Medicine, Johns Hopkins Bayview Medical Center, Johns Hopkins University, Baltimore, Maryland, USA

*Lead author

Citation: Washburn C, Kantsiper ME, Esteve R, Gupta I, Memon G, Michtalik HJ, et al. (2023) Comparison of hospitalist morale in a COVID-19 alternate care site (ACS) to hospitalist morale in conventional hospitals in Maryland. PLoS ONE 18(8): e0288981. https://doi.org/10.1371/journal.pone.0288981

About the Authors:

Catherine Washburn

Contributed equally to this work with: Catherine Washburn, Henry J. Michtalik

Roles: Conceptualization, Writing – original draft, Writing – review & editing

E-mail: [email protected]

Affiliation: Division of Hospital Medicine, Johns Hopkins Bayview Medical Center, Johns Hopkins University, Baltimore, Maryland, United States of America

ORICD: https://orcid.org/0000-0003-2703-3243

Melinda E. Kantsiper

Roles: Formal analysis, Project administration, Writing – review & editing

¶‡ MEK, RE, IG and GM also contributed equally to this work.

Affiliations: Division of Hospital Medicine, Johns Hopkins Bayview Medical Center, Johns Hopkins University, Baltimore, Maryland, United States of America, Baltimore Convention Center Field Hospital, Baltimore, Maryland, United States of America

Rogette Esteve

Roles: Conceptualization, Writing – review & editing

Current address: Concentra Department of Occupational Medicine, Baltimore, Maryland, United States of America

¶‡ MEK, RE, IG and GM also contributed equally to this work.

Affiliation: Baltimore Convention Center Field Hospital, Baltimore, Maryland, United States of America

ORICD: https://orcid.org/0000-0003-2567-2121

Ishaan Gupta

Roles: Conceptualization, Formal analysis, Writing – review & editing

¶‡ MEK, RE, IG and GM also contributed equally to this work.

Affiliations: Division of Hospital Medicine, Johns Hopkins Bayview Medical Center, Johns Hopkins University, Baltimore, Maryland, United States of America, Baltimore Convention Center Field Hospital, Baltimore, Maryland, United States of America

Gulzeb Memon

Roles: Conceptualization, Writing – review & editing

Current address: Department of Medicine, Anne Arundel Medical Center, Annapolis, Maryland, United States of America

¶‡ MEK, RE, IG and GM also contributed equally to this work.

Affiliation: Baltimore Convention Center Field Hospital, Baltimore, Maryland, United States of America

Henry J. Michtalik

Contributed equally to this work with: Catherine Washburn, Henry J. Michtalik

Roles: Conceptualization, Data curation, Formal analysis, Investigation, Supervision, Writing – original draft, Writing – review & editing

Affiliation: Division of Hospital Medicine, Johns Hopkins Bayview Medical Center, Johns Hopkins University, Baltimore, Maryland, United States of America

Bayview Educational and Academic Research (BEAR) Core Consortium

¶Membership of the BEAR Core Consortium is provided in the Acknowledgments.

Affiliation: Division of Hospital Medicine, Johns Hopkins Bayview Medical Center, Johns Hopkins University, Baltimore, Maryland, United States of America

References

1. Shanafelt TD, Hasan O, Dyrbye LN, Sinsky C, Satele D, Sloan J, et al. Changes in burnout and satisfaction with work-life balance in physicians and the general US working population between 2011 and 2014. Mayo Clin Proc. 2015 Dec; 90(12): 1600–1613. pmid:26653297

2. French G, Hulse M, Nguyen D, Sobotka K, Webster K, Corman J, et al. (Morbidity and Mortality Weekly Report). Impact of hospital strain on excess deaths during the COVID-19 pandemic-United States, July 2020-July 2021 [Internet]. Atlanta [GA]: Centers for Disease Control and Prevention: Nov 2021 [cited Jan 2022].1686 p. Report.: 70. Contract.: 46. https://www.cdc.gov/mmwr/volumes/70/wr/mm7046a5.htm

3. Forrest CB, Xu H, Thomas LE, Webb LE, Cohen LW, Carey TS, et al. & the members of the HERO Registry Research Group. Impact of the early phase of the COVID-19 pandemic on US healthcare workers: results from the HERO registry. J Gen Intern Med. 2021 March; 36: 1319–1326. pmid:33694071

4. Nieuwsma JA, O’Brien EC, Xu H, Smigelsky MA, Meador KG; VISN 6 MIRECC Workgroup & HERO Research Program. Patterns of potential moral injury in post-9/11 combat veterans and COVID-19 healthcare workers. J Gen Intern Med. 2022 Apr; 37: 2033–2040. pmid:35381899

5. Shanafelt T, Ripp J, & Trockel M. Understanding and addressing sources of anxiety among health care professionals during the COVID-19 pandemic. JAMA. 2020 June; 323(21): 2133–2134. pmid:32259193

6. Gupta I, Siddiqui ZK, Phillips M D, Singh A, Eid S M, Wortman L et al & members of CONQUER COVID Consortium. Recruitment, readiness, and retention of providers at a field hospital during the pandemic. Disaster Med Public Health Prep. 2022 Jan 10;17: e102. pmid:35000667

7. Zhang Y, Xiang D, & Alejok N. Coping with COVID-19 in United Nations peacekeeping field hospitals: increased workload and mental stress for military healthcare providers. BMJ Mil Health. 2021 Aug;167(4):229–233. pmid:33177148

8. Sampathkumar P, Beam E, Breeher LE, & O’Horo JC. Precautions, utilization of personal protective equipment, and conservation strategies during the COVID-19 pandemic. Mayo Clin Proc. 2020 Sept 95(9S), S11–S13. pmid:32807520

9. Que J, Shi L, Deng J, Liu J, Zhang L, Wu S et al. Psychological impact of the COVID-19 pandemic on healthcare workers: A cross-sectional study in China. Gen Psychiatr. 2020 June 14; 33(3), Article e100259. pmid:32596640

10. Pan X, Xiao Y, Ren D, Xu ZM, Zhang Q, Yang LY, et al. Prevalence of mental health problems and associated risk factors among military healthcare workers in specialized COVID-19 hospitals in Wuhan, China: A cross-sectional survey. Asia Pac Psychiatry. 2022 March; 14(1) e12427. pmid:33089622

11. Zhang XB, Xiao W, Lei J, Li MX, Wang X, Hong YJ, et al. Prevalence and influencing factors of anxiety and depression symptoms among the first-line medical staff in Wuhan mobile cabin hospital during the COVID-19 epidemic: A cross-sectional survey. Medicine (Baltimore). 2021 May 28; 100(21), e25945. pmid:34032703

12. Silva-Gomes RN, & Silva-Gomes VT. COVID-19 pandemic: burnout syndrome in healthcare professionals working in field hospitals in Brazil [Letter to the editor]. Enferm Clin (Engl Ed). 2021 Mar-Apr; 31(2), 128–129. pmid:33199160

13. Chandra S, Wright SM, Ghazarian S, Kargul GM, & Howell EE. Introducing the Hospitalist Morale Index: A new tool that may be relevant for improving provider retention. J Hosp Med. 2016 June; 11(6), 425–31. https://shmpublications.onlinelibrary.wiley.com/doi/10.1002/jhm.2543 pmid:26969890

14. Gupta I, Siddiqui Z, Phillips M, Singh A, Eid S, Wortman L, et al. Recruitment, readiness and retention of providers at a field hospital during the pandemic. Disaster Med Public Health Prep. 2022 Jan 10; 17: e102. pmid:35000667

15. Li-Sauerwine S, Rebillot K, Melamed M, Addo N, Lin M. A 2-question summative score correlates with the Maslach Burnout Inventory. West J Emerg Med. 2020 Apr 21; 21(3), 610–617. pmid:32421508

16. Van Wert M, Gandhi S, Gupta I, Singh A, Eid S, Burhanullah H, et al. Healthcare worker mental health after the initial peak of the COVID-19 pandemic: A US medical center cross-sectional survey. J Gen Intern Med. 2022 Apr; 37(5): 1169–1176. pmid:34993856

17. Johnson T, Wislar J. Response Rates and Nonresponse Errors in Surveys. JAMA. 2012; 307(17):1805–6. pmid:22550194