An adaptive simulation intervention decreases emergency physician physiologic stress while caring for patients during COVID-19: A randomized clinical trial

Abstract

Background

Stressful work environments and burnout in emergency medicine (EM) physicians adversely impact patient care quality. The future EM workforce will need to prioritize clinician well-being to ensure optimal patient care.

Methods

This prospective, randomized, controlled study aimed to determine whether an adaptive simulation intervention, COVID-19 Responsive Intervention: Systems Improvement Simulations (CRI:SIS), decreased physiologic stress as measured by heart rate variability (HRV) in front-line EM physicians during the COVID-19 pandemic. HRV was measured with smart shirts and self-reported State-Trait Anxiety Inventory (STAI) were collected at baseline and during four 8-hour clinical shifts for all participants. The intervention group (n = 40) received a 3-hour virtual educational simulation intervention consisting of four simulation scenarios (CRI:SIS). The control group (n = 41) received no simulation intervention.

Results

There were no significant differences in demographics between groups. HRV data collected from 81 physicians across a total of 324 clinical shifts showed an increase in HRV (decrease in physiologic stress) in shifts immediately following CRI:SIS in the intervention group as measured by a root mean square standard deviation (RMSSD) difference of 11.55 ms (95% CI, −19.90 to −3.20; P = 0.007) compared to the control group. Post-intervention STAI did not significantly differ between intervention and control.

Conclusion

An adaptive simulation-based educational intervention led to decreased physiologic stress (increased HRV) among emergency physicians who received a simulation education intervention. Reduced physiologic stress generated by adaptive simulation interventions may improve both patient safety and clinician well-being.

Introduction

The extensive and long duration of response to the coronavirus 2019 (COVID-19) pandemic increased clinician work burden while creating an environment of uncertainty, raising concerns about increasing burnout and attrition of front-line health workers who were directly diagnosing and managing critically ill patients. [1–4] The relationship between COVID-19-related stress and work intentions showed that approximately 1 in 3 physicians and nurses surveyed intended to reduce work hours. One in 5 physicians and 2 in 5 nurses intended to leave their practice altogether. [5] Of those surveyed, 74.7% of emergency physicians reporting burnout symptoms since the start of the pandemic. [6] Reducing burnout and improving a sense of feeling valued may allow healthcare organizations to better maintain their workforce post-pandemic. [7–9] These work trends are costly; turnover and reduced clinical hours due to physician burnout in the United States cost $4.6 billion annually. [10] Physician burnout is also linked to increased medical errors, increased risk of patient safety incidents, extended patient waiting times, and decreased patient satisfaction. [11–13]

Due to the increasing prevalence of burnout in the healthcare sector, recent research has increasingly focused on methods to measure stress in both simulated and in situ clinical environments. The emergency care workforce has one of the highest rates of burnout and attrition with the highest negative impact from the pandemic. [14] Our recent work outlining the impact of COVID-19 on Emergency Department (ED) clinicians highlights the importance of supporting clinicians along a range of psychological needs. If these needs are unmet, ED healthcare workers may be at an increased risk of infection, anxiety, and burnout. [4,15] Experts have advocated for more accurate and early detection of stress amongst healthcare personnel to develop and intervene prior to development of severe burnout symptoms. [16–20]

During acute stress events, healthcare workers may experience activation of the sympathetic nervous system resulting in key physiological changes. [21] Established markers of this stress response include a decrease in heart rate variability (HRV). [22] Evidence suggests links between physiologic measures of acute stress, particularly HRV, and the emotional exhaustion subscale of burnout. [23,24] HRV is the measure between consecutive heartbeats and is controlled by the autonomic nervous system. Increased parasympathetic activity decreases heart rate via the vagus nerve. Therefore, increased vagal tone leads to increased HRV. An increase in HRV correlates with decreased stress. [25] Low HRV has been observed in individuals presenting with burnout resulting from repeated or continuous stress exposure. [26] For workers presenting with clinical burnout, measures of HRV have been shown to be lower than both workers with non-clinical burnout and healthy individuals with no burnout symptomology. [26] Such low levels of HRV suggest sympathetic predominance which may contribute to the adverse health effects associated with clinical burnout. [23,26,27]

In March of 2020, EDs began seeing increased volumes of COVID-19-positive patients. Rapidly changing hospital protocols, as well as the intense, challenging work of caring for an influx of severely ill patients, created new stressors including the necessity to adopt rapidly changing clinical guidelines. The challenges facing healthcare clinicians were exacerbated by the novelty of COVID-19 and the constantly evolving clinical understanding of the disease. [1,2] Within our own institution, daily and weekly emails from both the hospital and ED administration provided rapidly updating guidance on evolving COVID-19 protocols. These communications occurred as frequently as 200 per month.

Context and goal of this study

Healthcare simulation, a growing field that uses technology to improve safety and care quality, has shown increasing promise in improving hospital and clinician preparedness during rapidly evolving pandemics like COVID-19. [28–30] However, evaluation of its impact on clinician stress and burnout has thus far been limited. [31] Accordingly, we developed and piloted a new rapidly adaptive simulation intervention to increase guideline adoption and preparedness for COVID-19 care delivery. The objective of this study was to evaluate the effect of simulation–based training and quality improvement intervention (COVID-19 Responsive Intervention: Systems Improvement Simulations – CRI:SIS) on emergency physician physiologic stress. To our knowledge, this was the first clinical trial examining the effect of a simulation intervention on physiologic stress amongst emergency physicians.

Methods

Setting and study design

This prospective, randomized controlled trial (RCT) of emergency medicine (EM) resident and faculty physicians occurred at two sites of a teaching hospital system located in an urban city within the Northeast United States: (1) a tertiary care referral center with an average annual adult volume of 100,000 visits; and (2) an urban community hospital with an average annual adult volume of 65,000 visits. Eligible participants were enrolled from January 1 to December 31 2021. This trial period encompassed the second wave, Delta surge, and third wave of the COVID-19 pandemic.

All participants wore Hexoskin Smart Shirts (Carré Technologies Inc.) with embedded textile sensors that allow for continuous 1-lead ECG recording to measure HRV. A baseline session established rest measurements, baseline State-Trait Anxiety Inventory assessment, and collection of demographic information as well as potential confounding factors related to HRV measurement (e.g., caffeine intake, Body Mass Index, etc.). The intervention group wore smart shirts during two pre-CRI:SIS and two post-CRI:SIS ED clinical shifts. The control group wore the shirts during four clinical shifts with no simulation intervention. Data collection concluded after all enrolled participants completed the baseline measurement and four data collections. Our institutional review board approved this study and we obtained written consent for each participant. We followed the CONSORT reporting guidelines for randomized control trials. [32]

Ethics approval

The study protocol was approved by the Yale University Institutional Review Board (HIC# 2000029372). All recruited physicians were fully briefed and given the opportunity to ask questions regarding the details of the study before either agreeing or declining to participate. Each participant submitted signed written statements of informed consent that outlined the study’s objectives, the safety risks of participating, the requirements of the participants, and permission to publish their deidentified biometric data before commencing involvement in the study.

Participants and recruitment

The target participant demographics were frontline EM physicians, consisting of resident and attending physicians who were actively treating acutely ill COVID-19 patients in the ED at either of the two sites. Physicians were recruited via email or in person. Trial exclusion criteria included the use of beta-blockers and antiarrhythmic medication, active thyroid dysfunction, and pregnancy. We ended recruitment when we reached our target sample size plus an additional two participants per arm to accommodate for potential loss to follow-up.

Randomization

After completing two clinical shifts, participants were randomized 1:1 to either the control or intervention group using a computer-generated randomization sequence. [33] Permuted block randomization was performed and randomization was stratified by clinical experience level: Junior (PGY 1–3) or Senior (PGY 4, attendings). Randomization was conducted using the sequentially numbered, opaque, sealed envelope (SNOSE) technique prepared by a statistician prior to the start of the clinical trial. A research associate not involved with delivering the intervention opened a corresponding envelope designating the participant to either the control or intervention arm. The randomization list was kept in a secure cloud-based electronic file and the randomization sequence was blinded from investigators and participants. All participants were randomized within 12 hours of completing their baseline data collection.

Intervention

The intervention group completed a 3-hour virtual training simulation (CRI:SIS), using Laerdal LLEAP simulation software, between 1–10 days prior to their next scheduled clinical shift. The control group wore smart shirts during four clinical shifts and received no simulation intervention. All participants had similar access to COVID-19 Task Force updates, guidelines, weekly town hall meetings, and any in-service support available to clinical staff. After the study had been completed, the control group was offered the simulation intervention for its educational benefit.

The intervention consisted of a carefully tailored group of four intervention scenarios (S1 Table) with varying severity of COVID-19 presentations requiring appropriate communication, clinical management, and disposition of patients. Specific critical actions were adapted over the course of the study as guidelines and recommendations evolved in consultation with administrative leadership. The scenarios were administered by one of two trained co-investigators with expertise in healthcare simulation and board-certified in emergency medicine. The study principal investigator, who was also a board-certified emergency physician and a healthcare simulation expert, was present for every scenario to ensure consistency in the intervention delivery. The scenarios were conducted in the same sequence for every participant and focused on the four following objectives:

1. “CMO” (comfort measures only): Decision-making regarding goals of care and potential withdrawal of care in the ED in a patient presenting with severe illness and poor prognosis with extremely low chances of survival in the next 48 hours

2. Mild COVID-19: Understanding discharge criteria, adapted to address vaccine hesitancy

3. Moderate COVID-19: Familiarity with COVID-19 Severity Index [34] to determine the level of care required, disposition, and potential for clinical deterioration

4. Severe COVID-19: Management of complications associated with severe hypoxemia including ventilator management and treatment of pneumothorax

Each intervention session began with a prebriefing to discuss recent changes in clinical guidelines and was followed by a structured debriefing session led by a member of the research team with clinical expertise in emergency medicine and advanced debriefing skills. Over the course of the study, critical actions were updated, and the scenarios were adapted based on new COVID-19 guidelines at our institution.

Intervention Adaptations

CRI:SIS successfully made iterative updates throughout the study as guidelines and best practices evolved. The Yale New Haven Hospital (YNHH) COVID-19 Task Force made 149 guideline changes from January through December 2021. We made a total of 22 iterative changes to the intervention over the course of 12 months (30 sessions). S1 Fig 1 shows a timeline of highlighted changes made to our adaptive simulation relative to current events and significant YNHH guideline changes.

Outcomes measures

Our primary outcomes consisted of physician HRV while on shift. Consistent with prior work on physiologic stress, HRV was assessed as the time-domain measure of root mean square standard deviation (RMSSD) of sequential R-R intervals. [24,35] RMSSD was selected because it was more resistant to respiratory artifacts than other HRV measures, such as the standard deviation of all normal-to-normal intervals (SDNN). [24]

We also included a co-primary outcome of the self-reported State-Trait Anxiety Inventory (STAI). [36,37] STAI is a psychological assessment tool used to measure both state anxiety (how a person feels at a particular moment) and trait anxiety (individual differences in anxiety proneness). It helps to identify and quantify the presence and severity of anxiety symptoms. It can also be useful in monitoring changes in anxiety levels over time and evaluating the effectiveness of treatment interventions. [38] During the baseline session, all participants completed the 40-item self-reported STAI, consisting of 20-item trait anxiety (TA) questions and 20-item state anxiety (SA) questions. A survey including the 20-item SA questions was administered immediately following shift completion. Inventory items were scored on a 4-point ranking scale and summed up to a continuous variable sum score. The survey included a self-reported field indicating the number of resuscitations performed per shift (S2 Table).

We collected basic demographic information from self-reported surveys of participants during the baseline session, consisting of sex, age, race and ethnicity, years of experience in emergency medicine practice at time of enrollment, Body Mass Index, average weekly exercise time in minutes, and average daily caffeine consumption in mg.

Statistical analysis and sample size calculation

We presented demographic descriptive data by randomized groups as means and standard deviations as appropriate. Analyses of primary outcomes were performed on the intent to treat population using all available outcome data. We used linear repeated measures mixed effects models to compare the primary outcomes of HRV and state anxiety questions of the STAI. A random effect was included for the subject to accommodate correlation between repeated assessments. Models were adjusted for participant activity level (number of steps). Least squares means (LSMeans) were used to describe outcomes pre-intervention (i.e., first two shifts) and post-intervention (i.e., second two shifts). Linear contrasts were used to compare the outcomes between pre-intervention and post-intervention shifts. Comparisons are presented as differences between LSMeans with 95% Confidence Intervals (CI). We used a Bonferroni correction to account for multiple comparisons from the two primary outcomes. All analyses were performed using SAS 9.4 statistical software. All tests were 2-tailed, and P < 0.025 was considered significant.

In order to automate the data derived from wearable sensors, we previously created a tool to both extract meaningful metrics and introduce methodologies for noise filtration and data quality assurance. [39] Using this tool, ECG recordings were algorithmically converted, filtered, and analyzed to calculate RMSSD. Representative 5-minute windows were selected from each recording as this timeframe is considered the conventional recording period to measure short-term HRV. [24] To standardize our window selection for HRV analysis, we selected time windows within a 30-minute period when the participant was involved in a shift sign-out. Our clinical experts determined that end-of-shift sign-out was the optimal choice because it met two essential criteria: 1) participants were not actively exerting themselves and would likely be sitting for at least ten minutes, allowing their HR to return to its resting state, and 2) it was a consistent period at the end of every participant’s shift that would represent cumulative stress. [40] Selected windows met strict quality level criteria (>85% clear R-R intervals) and activity level (<20 steps/5-min) thresholds, captured from the smart garment ECG and accelerometry sensors. Quality thresholds for our raw ECG data were determined by existing guidelines in line with previous literature. [41]

Given an SD of 15ms in HRV, we targeted a sample size of 38 per group providing 80% power at the two-sided 0.025 significance level to detect differences of 10.8ms, an effect reflecting clinically meaningful changes to stress in prior HRV studies. [15,29,30] In addition, a sample size of 38 per group would provide 80% power to detect an estimated STAI score difference of 5.8 with SD = 8 (effect size = 0.72). [42] We planned to include 40 subjects per group to accommodate a potential 5% lost to follow-up.

Results

A total of 81 participants were enrolled and randomized in the study (mean [SD] age, 35.5 [10] years; 45 [55.6%] women and 36 [44.4%] men). One participant left the study prior to formal initiation of data collection and randomization. None were lost to follow-up. In terms of race and ethnicity, 49 [60.5%] participants identified as Caucasian/White, non-Hispanic; 14 (17.3%) Asian/Pacific Islander; 7 (8.6%) Black or African American, 5 (6.2%) multiple ethnicities or other, 2 (2.5%) Hispanic, and 4 (6.2%) did not disclose their ethnicity. Emergency medicine clinical experience was a mean (SD) of 6 (8) years. A total of 71 (85.2%) participants had a BMI of less than 30, with 11 (14.8%) having a BMI of 30 or greater. The mean physical activity per week was 127.5 (120) minutes. The mean (SD) caffeine intake was approximately 188.5 (122.5) mg per week. At baseline, the control group reported mean (SD) RMSSD of 29.75 (13.26) ms, compared to the intervention group’s 35.13 (23.60) ms. The control group reported a mean (SD) Trait Anxiety (TA) score of 37.90 (10.13) points and a State Anxiety (SA) score of 32.68 (8.31) points, while the intervention group reported a mean (SD) Trait Anxiety (TA) score of 38.83 (11.47) points and a State Anxiety (SA) score of 34.45 (9.83) points. No significant differences were found in any demographic factors, health-related characteristics, and baseline RMSSD and STAI between the control and intervention groups (Table 1). The control group performed a mean (SD) of 2.45 (3.22) resuscitations per shift vs. the intervention group’s 3.22 (7.06) resuscitations per shift with no statistically significant difference between the two groups.

[Figure omitted. See PDF.]

Of the 81 study participants, 41 were randomized to the control arm (20 Juniors and 21 Seniors), and 40 were randomized to the intervention arm (20 Juniors and 20 Seniors). Please refer to the CONSORT diagram (Fig 1) for participant flow through the trial.

[Figure omitted. See PDF.]

The diagram outlines the flow of participants through each stage of the randomized controlled trial, from enrollment to analysis. A total of 81 participants enrolled and completed the clinical trial. CONSORT indicates consolidated standards of reporting trials; CRI:SIS, COVID-19 Responsive Intervention: Systems Improvement Simulations.

Main outcomes

Heart rate variability.

The HRV results of 81 participants (41 control and 40 intervention) were evaluated. The post-CRI:SIS clinical shifts were collected on average within three days (range 0–8 days) after the intervention.

Pre-CRI:SIS, the mean RMSSD for the control group was 38.96 ms (SE, 3.96), and the mean RMSSD for the intervention group was 38.29 ms (SE, 4.11). A linear mixed model for repeated measurements, adjusted for activity level (steps), showed that the mean difference in pre-CRI:SIS RMSSD for the intervention group compared to the control group was −0.67 ms (95% CI, −10.678ms to 12.026 ms; P = 0.91), indicating no difference in stress between groups. Post-CRI:SIS, the mean RMSSD for the control group was 31.77 ms (2.91), and the mean RMSSD of the intervention group was 43.32 ms (SE, 3.03). In a mixed model, a mean difference in RMSSD for the intervention group compared to the control group was 11.55 ms (95% CI, −19.90 ms to −3.20 ms; P = 0.007), showing significantly higher HRV, and therefore lower physiological stress in the intervention group (Table 2) as compared to the control group.

[Figure omitted. See PDF.]

STAI.

We received 317 (97.8%) post-shift survey responses across our 324 clinical shift data collections among 81 participants (41 control and 40 intervention). Pre-CRI:SIS, the reported mean SA score of 34.3 points (SE, 1.0) for the control group, and a mean SA score of 36.9 points (SE, 1.1) for the intervention group. Post-CRI:SIS, the reported mean SA score was 34.2 points (SE, 1.0) for the control group, and 35.8 points (SE 1.1) for the intervention group. Pre-CRI:SIS, there was no significant mean difference between the control and the intervention groups (−2.57 points, 95% CI, −5.52 to 0.38; P = 0.09). Post-CRI:SIS, there was no significant mean difference between the groups (−1.64 points, 95% CI, −4.58 to 1.30; P = 0.27).

Discussion

To our knowledge, this is the first randomized control trial to evaluate the use of a rapidly adaptive simulation intervention on physiologic measures of stress among emergency physicians caring for acutely ill patients. This trial demonstrated the effectiveness of an adaptive COVID simulation intervention (CRI:SIS) on physician stress as measured by HRV during clinical shifts and compared this approach to clinicians who received no simulation intervention. The findings indicate a significant increase in HRV occurred in clinical shifts immediately following the completion of CRI:SIS in the intervention group compared to the control group as measured by RMSSD, correlating with increased parasympathetic tone (decreased stress) for the participants that received the intervention. To our knowledge, we are the first prospective clinical trial to demonstrate efficacy of a simulation intervention on physiologic stress in emergency physicians. In a systematic review of HRV as a measure of stress, [35] of the 16 articles reviewed, only one study used a randomized control trial experimental design but failed to demonstrate significant improvement in stress. [43–45] We believe these findings could be instrumental in reducing stress and burnout among physicians, particularly in the setting of a future pandemic, disasters, or even more chronically observed surges in healthcare worker stress from hospital crowding [12,43,46–49]

Our findings that CRI:SIS as a simulation intervention was able to decrease levels of stress in physician participants may reflect an increased level of confidence in the intervention group. During debriefing, participants had the opportunity to reflect on their own clinical experiences related to COVID-19 care. The debriefer for every intervention session was not only a simulation faculty member trained in debriefing skills, but also an attending physician in emergency medicine with the shared experience of caring for acutely ill patients with COVID-19. The intervention group had the opportunity to reflect on their performance in terms of what went well and what could be improved, receiving constructive feedback highlighting both positive actions and areas for improvement in a safe and supportive environment by a seasoned clinician. Participants also had the opportunity to process any emotional reactions they experienced during the simulation. These responses were most notable for the scenarios involving managing vaccine hesitancy and transitioning a patient to comfort measures only (CMO). By actively engaging in the CRIS:IS debriefing sessions, participants may have been able to leverage the experience gained from the simulations to build competence, resilience, and confidence in their abilities. While specific references on the direct impact of simulation debriefing on confidence may be limited, there are numerous studies that highlight the effectiveness of simulation-based training in enhancing learning outcomes and performance improvement. [50–52]

We did not find a statistically significant difference in the State-Trait Anxiety Index between the control and intervention groups. Trait anxiety refers to a trait or personality. In contrast, state anxiety reflects the psychological and physiological transient reactions directly related to adverse situations in a specific moment. While our HRV results indicate that a simulation intervention mitigates physiologic stress, we did not observe a difference in the STAI survey results. Akbar et al. reported similar results in their study of physician stress during electronic health record in-box work. [53] Several reasons may explain these findings. Firstly, STAI results were derived from self-report of stress and anxiety by participants. One of the main limitations of previous studies measuring stress is the reliance on self-reported measures of well-being. [54,55] In addition to not directly measuring stress per se, self-report approaches have several limitations for stress monitoring in the workplace. When people subjectively report how they feel, their evaluation could be affected by memory bias and emotion recognition, regulation, and expression biases. [56–59] Administering surveys for self-reports at the end of a clinical shift may also be less accurate since they require the full cognitive attention of the participant at a time when they may be eager to leave the hospital. Qualitative interviews with emergency physicians described sentiments of pride and responsibility to “rise up” to the challenge during the pandemic as a professional obligation and duty to their patients, which may potentially mask or bias their perceptions of stress or anxiety. [6,60,61] Other explanations may include physicians’ ability to mount physiological adaptations (e.g., improved autonomic regulation) or habituation of physiological responses that may occur independently of anxiety reduction. [62] Finally, since HRV reflects physiologic responses to stress, it may be able to detect more subtle or earlier signs of stress response or anxiety. This suggests that our intervention may be useful in mitigating negative effects on clinicians before it rises to the level of conscious awareness in our participants.

Limitations

Our study included some limitations. This study was conducted at two hospitals at a single site, and the study only enrolled physicians and no other members of the health care team. Although physicians are an important target group for measuring stress in the setting of a pandemic, it may limit the generalizability of our findings. Future studies should consider recruiting participants at multiple sites with a more diverse sample population including other clinicians and nurses. Due to the nature of the simulation intervention, participants were not blinded to the study. We feel this is not a limitation because while lack of blinding could impact the self-reported STAI results post-CRI:SIS, we do not believe this impacted the post-CRI:SIS physiologic data collected in the intervention vs. control group. A lack of detectable difference in STAI results may be a potential limitation in this study’s ability to demonstrate differences in self-reported anxiety due to our intervention. We were unable to control for the number of COVID-19 patients and resuscitations evaluated during individual clinical shifts. However, based on our post-shift survey results, we determined there was no significant difference in the number of resuscitations between the control and intervention groups. The control arm did not receive any educational interactions or interactions with the instructional team; further studies may be needed to determine whether the observed effects were attributable to the simulation format itself or to other elements such as structured human interaction. Future work may also include measurements of clinical patient outcomes to determine potential impact on care quality and delivery as it relates to changes in clinician stress. Additional biomarkers of stress (e.g., blood glucose levels, hormones such as cortisol and epinephrine, body temperature, electrodermal activity) can also be measured to correlate with HRV results. [63] Finally, the effects of healthcare simulation on other domains of stress beyond the physiological, including emotional, cognitive, and social dimensions, may be explored to characterize its impact on overall stress in physicians. [64]

Conclusions

Our findings indicate that emergency physicians who participated in a rapidly adaptive simulation intervention targeting COVID-19 care, CRI:SIS, demonstrated reduced physiologic stress as measured by increased heart rate variability. This suggests that an adaptive simulation technology focusing on quality and safety for a serious public health crisis may be a unique approach to concurrently prevent burnout and improve evidence-based knowledge translation for front-line emergency physicians. Given the stress on our emergency care system and its challenges in keeping up with the rising demand for acute care needs, healthcare administrators and policymakers will need to prioritize emergency physician well-being to ensure the highest quality patient care in the future.

Supporting information

S1 Table. Final Iteration of Intervention Scenarios.

Summary of the intervention scenarios used in the study, detailing their design, objectives, and critical actions.

https://doi.org/10.1371/journal.pone.0331488.s001

(DOCX)

S2 Table. Post-Clinical Shift Survey.

https://doi.org/10.1371/journal.pone.0331488.s002

(DOCX)

S1 Fig. Timeline of CRI:SIS adaptations in response to changes in best practices during the COVID-19 pandemic.

An illustrative overview of key COVID-19 pandemic developments, including changes in Yale-New Haven Hospital COVID-19 Emergency Department Task Force protocols and clinical scenarios. Selected protocol updates (top) and simulation scenario modifications (bottom) are examples of the rapidly adaptive nature of CRI:SIS over the course of the clinical trial. Key events (flags) provide a national context for these changes.

https://doi.org/10.1371/journal.pone.0331488.s003

(DOCX)

S1 checklist. CRISIS CONSORT Checklist.

https://doi.org/10.1371/journal.pone.0331488.s004

Acknowledgments

August Alloco, BS, contributed to the ECG data cleaning.

Hugo He, MS, contributed to statistical analyses.

Winslow Burleson, PhD, contributed to the design.

References

1. 1. Adams JG, Walls RM. Supporting the Health Care Workforce During the COVID-19 Global Epidemic. JAMA. 2020;323(15):1439–40. pmid:32163102

* View Article

* PubMed/NCBI

* Google Scholar

2. 2. Gan WH, Lim JW, Koh D. Preventing Intra-hospital Infection and Transmission of Coronavirus Disease 2019 in Health-care Workers. Saf Health Work. 2020;11(2):241–3. pmid:32292622

* View Article

* PubMed/NCBI

* Google Scholar

3. 3. Hendin A, La Rivière CG, Williscroft DM, O’Connor E, Hughes J, Fischer LM. End-of-life care in the emergency department for the patient imminently dying of a highly transmissible acute respiratory infection (such as COVID-19). CJEM. 2020;22(4):414–7. pmid:32213224

* View Article

* PubMed/NCBI

* Google Scholar

4. 4. Lai J, Ma S, Wang Y, Cai Z, Hu J, Wei N, et al. Factors Associated With Mental Health Outcomes Among Health Care Workers Exposed to Coronavirus Disease 2019. JAMA Netw Open. 2020;3(3):e203976. pmid:32202646

* View Article

* PubMed/NCBI

* Google Scholar

5. 5. Sinsky CA, Brown RL, Stillman MJ, Linzer M. COVID-Related Stress and Work Intentions in a Sample of US Health Care Workers. Mayo Clin Proc Innov Qual Outcomes. 2021;5(6):1165–73. pmid:34901752

* View Article

* PubMed/NCBI

* Google Scholar

6. 6. Nguyen J, Liu A, McKenney M, Liu H, Ang D, Elkbuli A. Impacts and challenges of the COVID-19 pandemic on emergency medicine physicians in the United States. Am J Emerg Med. 2021;48:38–47. pmid:33836387

* View Article

* PubMed/NCBI

* Google Scholar

7. 7. Hamidi MS, Bohman B, Sandborg C, Smith-Coggins R, de Vries P, Albert MS, et al. Estimating institutional physician turnover attributable to self-reported burnout and associated financial burden: a case study. BMC Health Serv Res. 2018;18(1):851. pmid:30477483

* View Article

* PubMed/NCBI

* Google Scholar

8. 8. Shanafelt TD, Mungo M, Schmitgen J, Storz KA, Reeves D, Hayes SN, et al. Longitudinal Study Evaluating the Association Between Physician Burnout and Changes in Professional Work Effort. Mayo Clin Proc. 2016;91(4):422–31. pmid:27046522

* View Article

* PubMed/NCBI

* Google Scholar

9. 9. Sinsky CA, Dyrbye LN, West CP, Satele D, Tutty M, Shanafelt TD. Professional Satisfaction and the Career Plans of US Physicians. Mayo Clin Proc. 2017;92(11):1625–35. pmid:29101932

* View Article

* PubMed/NCBI

* Google Scholar

10. 10. Han S, Shanafelt TD, Sinsky CA, Awad KM, Dyrbye LN, Fiscus LC, et al. Estimating the Attributable Cost of Physician Burnout in the United States. Ann Intern Med. 2019;170(11):784–90. pmid:31132791

* View Article

* PubMed/NCBI

* Google Scholar

11. 11. Bragard I, Dupuis G, Fleet R. Quality of work life, burnout, and stress in emergency department physicians: a qualitative review. Eur J Emerg Med. 2015;22(4):227–34. pmid:25093897

* View Article

* PubMed/NCBI

* Google Scholar

12. 12. Schneider A, Weigl M. Associations between psychosocial work factors and provider mental well-being in emergency departments: A systematic review. PLoS One. 2018;13(6):e0197375. pmid:29864128

* View Article

* PubMed/NCBI

* Google Scholar

13. 13. Basu S, Qayyum H, Mason S. Occupational stress in the ED: a systematic literature review. Emerg Med J. 2017;34(7):441–7. pmid:27729392

* View Article

* PubMed/NCBI

* Google Scholar

14. 14. Petrino R, Riesgo LG-C, Yilmaz B. Burnout in emergency medicine professionals after 2 years of the COVID-19 pandemic: a threat to the healthcare system?. Eur J Emerg Med. 2022;29(4):279–84. pmid:35620812

* View Article

* PubMed/NCBI

* Google Scholar

15. 15. Wong AH, Pacella-LaBarbara ML, Ray JM, Ranney ML, Chang BP. Healing the Healer: Protecting Emergency Health Care Workers’ Mental Health During COVID-19. Ann Emerg Med. 2020;76(4):379–84. pmid:32534830

* View Article

* PubMed/NCBI

* Google Scholar

16. 16. Shanafelt TD, Hasan O, Dyrbye LN, Sinsky C, Satele D, Sloan J, et al. Changes in Burnout and Satisfaction With Work-Life Balance in Physicians and the General US Working Population Between 2011 and 2014. Mayo Clin Proc. 2015;90(12):1600–13. pmid:26653297

* View Article

* PubMed/NCBI

* Google Scholar

17. 17. West CP, Dyrbye LN, Shanafelt TD. Physician burnout: contributors, consequences and solutions. J Intern Med. 2018;283(6):516–29. pmid:29505159

* View Article

* PubMed/NCBI

* Google Scholar

18. 18. Panagioti M, Panagopoulou E, Bower P, Lewith G, Kontopantelis E, Chew-Graham C, et al. Controlled Interventions to Reduce Burnout in Physicians: A Systematic Review and Meta-analysis. JAMA Intern Med. 2017;177(2):195–205. pmid:27918798

* View Article

* PubMed/NCBI

* Google Scholar

19. 19. van Vendeloo SN, Brand PLP, Verheyen CCPM. Burnout and quality of life among orthopaedic trainees in a modern educational programme: importance of the learning climate. Bone Joint J. 2014;96-B(8):1133–8. pmid:25086133

* View Article

* PubMed/NCBI

* Google Scholar

20. 20. Merlo G, Rippe J. Physician Burnout: A Lifestyle Medicine Perspective. Am J Lifestyle Med. 2020;15(2):148–57. pmid:33790702

* View Article

* PubMed/NCBI

* Google Scholar

21. 21. Morris MC, Rao U. Psychobiology of PTSD in the acute aftermath of trauma: Integrating research on coping, HPA function and sympathetic nervous system activity. Asian J Psychiatr. 2013;6(1):3–21. pmid:23380312

* View Article

* PubMed/NCBI

* Google Scholar

22. 22. Dishman RK, Nakamura Y, Garcia ME, Thompson RW, Dunn AL, Blair SN. Heart rate variability, trait anxiety, and perceived stress among physically fit men and women. Int J Psychophysiol. 2000;37(2):121–33. pmid:10831999

* View Article

* PubMed/NCBI

* Google Scholar

23. 23. Kanthak MK, Stalder T, Hill LK, Thayer JF, Penz M, Kirschbaum C. Autonomic dysregulation in burnout and depression: evidence for the central role of exhaustion. Scand J Work Environ Health. 2017;43(5):475–84. pmid:28514792

* View Article

* PubMed/NCBI

* Google Scholar

24. 24. Shaffer F, Ginsberg JP. An Overview of Heart Rate Variability Metrics and Norms. Front Public Health. 2017;5:258. pmid:29034226

* View Article

* PubMed/NCBI

* Google Scholar

25. 25. Thayer JF, Lane RD. Claude Bernard and the heart-brain connection: further elaboration of a model of neurovisceral integration. Neurosci Biobehav Rev. 2009;33(2):81–8. pmid:18771686

* View Article

* PubMed/NCBI

* Google Scholar

26. 26. Lennartsson A-K, Jonsdottir I, Sjörs A. Low heart rate variability in patients with clinical burnout. Int J Psychophysiol. 2016;110:171–8. pmid:27535344

* View Article

* PubMed/NCBI

* Google Scholar

27. 27. de Vente W, van Amsterdam JGC, Olff M, Kamphuis JH, Emmelkamp PMG. Burnout Is Associated with Reduced Parasympathetic Activity and Reduced HPA Axis Responsiveness, Predominantly in Males. Biomed Res Int. 2015;2015:431725. pmid:26557670

* View Article

* PubMed/NCBI

* Google Scholar

28. 28. Garcia-Vicuña D, Esparza L, Mallor F. Hospital preparedness during epidemics using simulation: the case of COVID-19. Cent Eur J Oper Res. 2022;30(1):213–49. pmid:34602855

* View Article

* PubMed/NCBI

* Google Scholar

29. 29. Redmond B, Joseph M, Ray J, Shabanova V, Gross IT, Bruno C, et al. Stress as tool or toxin: physiologic markers and subjective report in neonatal simulation. Pediatr Res. 2020;88(5):784–91. pmid:32045934

* View Article

* PubMed/NCBI

* Google Scholar

30. 30. Brydges R, Campbell DM, Beavers L, Khodadoust N, Iantomasi P, Sampson K, et al. Lessons learned in preparing for and responding to the early stages of the COVID-19 pandemic: one simulation’s program experience adapting to the new normal. Adv Simul (Lond). 2020;5:8. pmid:32514385

* View Article

* PubMed/NCBI

* Google Scholar

31. 31. Schneider KA, Ngwa GA, Schwehm M, Eichner L, Eichner M. The COVID-19 pandemic preparedness simulation tool: CovidSIM. BMC Infect Dis. 2020;20(1):859. pmid:33213360

* View Article

* PubMed/NCBI

* Google Scholar

32. 32. Turner L, Shamseer L, Altman DG, Weeks L, Peters J, Kober T, et al. Consolidated standards of reporting trials (CONSORT) and the completeness of reporting of randomised controlled trials (RCTs) published in medical journals. Cochrane Database Syst Rev. 2012;11(11):MR000030. pmid:23152285

* View Article

* PubMed/NCBI

* Google Scholar

33. 33. Evans LV, Ray JM, Bonz JW, Joseph M, Gerwin JN, Dziura JD, et al. Improving patient and clinician safety during COVID-19 through rapidly adaptive simulation and a randomised controlled trial: a study protocol. BMJ Open. 2022;12(5):e058980. pmid:35589358

* View Article

* PubMed/NCBI

* Google Scholar

34. 34. Haimovich AD, Ravindra NG, Stoytchev S, Young HP, Wilson FP, van Dijk D, et al. Development and Validation of the Quick COVID-19 Severity Index: A Prognostic Tool for Early Clinical Decompensation. Ann Emerg Med. 2020;76(4):442–53. pmid:33012378

* View Article

* PubMed/NCBI

* Google Scholar

35. 35. Peabody JE, Ryznar R, Ziesmann MT, Gillman L. A Systematic Review of Heart Rate Variability as a Measure of Stress in Medical Professionals. Cureus. 2023;15(1):e34345. pmid:36865953

* View Article

* PubMed/NCBI

* Google Scholar

36. 36. Jones KI, Amawi F, Bhalla A, Peacock O, Williams JP, Lund JN. Assessing surgeon stress when operating using heart rate variability and the State Trait Anxiety Inventory: will surgery be the death of us?. Colorectal Dis. 2015;17(4):335–41. pmid:25406932

* View Article

* PubMed/NCBI

* Google Scholar

37. 37. Spielberger C, Gorsuch R, Lushene R, Vagg PR, Jacobs G. Manual for the State-Trait Anxiety Inventory. 1983.

38. 38. Ayed A, Khalaf IA, Fashafsheh I, Saleh A, Bawadi H, Abuidhail J, et al. Effect of High-Fidelity Simulation on Clinical Judgment Among Nursing Students. Inquiry. 2022;59:469580221081997. pmid:35285302

* View Article

* PubMed/NCBI

* Google Scholar

39. 39. Gerwin JN, de Oliveira Almeida G, Boyce MW, Joseph M, Wong AH, Burleson W, et al. HRVEST: a novel data solution for using wearable smart technology to measure physiologic stress variables during a randomized clinical trial. Front Comput Sci. 2024;6.

* View Article

* Google Scholar

40. 40. Chang TP, Azen C, Sherman JM. Physiological Stress Markers Following Resuscitations Remain Elevated Throughout Physician Shift Hours. Acad Emerg Med. 2020;27(6):510–3. pmid:32282983

* View Article

* PubMed/NCBI

* Google Scholar

41. 41. Pham T, Lau ZJ, Chen SHA, Makowski D. Heart Rate Variability in Psychology: A Review of HRV Indices and an Analysis Tutorial. Sensors (Basel). 2021;21(12):3998. pmid:34207927

* View Article

* PubMed/NCBI

* Google Scholar

42. 42. Megel ME, Black J, Clark L, Carstens P, Jenkins LD, Promes J, et al. Effect of High-Fidelity Simulation on Pediatric Nursing Students’ Anxiety. Clinical Simulation in Nursing. 2012;8(9):e419–28.

* View Article

* Google Scholar

43. 43. Stein C. The effect of clinical simulation assessment on stress and anxiety measures in emergency care students. Afr J Emerg Med. 2020;10(1):35–9. pmid:32161710

* View Article

* PubMed/NCBI

* Google Scholar

44. 44. Wetzel CM, Black SA, Hanna GB, Athanasiou T, Kneebone RL, Nestel D, et al. The effects of stress and coping on surgical performance during simulations. Ann Surg. 2010;251(1):171–6. pmid:20032721

* View Article

* PubMed/NCBI

* Google Scholar

45. 45. Oldenburg M, Wilken D, Wegner R, Poschadel B, Baur X. Job-related stress and work ability of dispatchers in a metropolitan fire department. J Occup Med Toxicol. 2014;9(1).

* View Article

* Google Scholar

46. 46. Petrowski K, Herhaus B, Schöniger C, Frank M, Pyrc J. Stress load of emergency service: effects on the CAR and HRV of HEMS emergency physicians on different working days (N = 20). Int Arch Occup Environ Health. 2019;92(2):155–64. pmid:30315367

* View Article

* PubMed/NCBI

* Google Scholar

47. 47. Mefford JM, Kahle S, Gupta S, Tancredi D, Danielson AR, Clarke SO. Heart Rate Variability and Acute Stress Among Novice Airway Managers. AEM Educ Train. 2019;3(3):291–4. pmid:31360823

* View Article

* PubMed/NCBI

* Google Scholar

48. 48. Virtanen M, Pentti J, Vahtera J, Ferrie JE, Stansfeld SA, Helenius H, et al. Overcrowding in hospital wards as a predictor of antidepressant treatment among hospital staff. Am J Psychiatry. 2008;165(11):1482–6. pmid:18676590

* View Article

* PubMed/NCBI

* Google Scholar

49. 49. Virtanen M, Batty GD, Pentti J, Vahtera J, Oksanen T, Tuisku K, et al. Patient overcrowding in hospital wards as a predictor of diagnosis-specific mental disorders among staff: a 2-year prospective cohort study. J Clin Psychiatry. 2010;71(10):1308–12. pmid:20492841

* View Article

* PubMed/NCBI

* Google Scholar

50. 50. Rudolph JW, Simon R, Raemer DB, Eppich WJ. Debriefing as formative assessment: closing performance gaps in medical education. Acad Emerg Med. 2008;15(11):1010–6. pmid:18945231

* View Article

* PubMed/NCBI

* Google Scholar

51. 51. Eppich W, Cheng A. Promoting Excellence and Reflective Learning in Simulation (PEARLS): development and rationale for a blended approach to health care simulation debriefing. Simul Healthc. 2015;10(2):106–15. pmid:25710312

* View Article

* PubMed/NCBI

* Google Scholar

52. 52. Sawyer T, Eppich W, Brett-Fleegler M, Grant V, Cheng A. More Than One Way to Debrief: A Critical Review of Healthcare Simulation Debriefing Methods. Simul Healthc. 2016;11(3):209–17. pmid:27254527

* View Article

* PubMed/NCBI

* Google Scholar

53. 53. Akbar F, Mark G, Prausnitz S, Warton EM, East JA, Moeller MF, et al. Physician Stress During Electronic Health Record Inbox Work: In Situ Measurement With Wearable Sensors. JMIR Med Inform. 2021;9(4):e24014. pmid:33908888

* View Article

* PubMed/NCBI

* Google Scholar

54. 54. Gregory ME, Russo E, Singh H. Electronic Health Record Alert-Related Workload as a Predictor of Burnout in Primary Care Providers. Appl Clin Inform. 2017;8(3):686–97. pmid:28678892

* View Article

* PubMed/NCBI

* Google Scholar

55. 55. Heponiemi T, Kujala S, Vainiomäki S, Vehko T, Lääveri T, Vänskä J, et al. Usability Factors Associated With Physicians’ Distress and Information System-Related Stress: Cross-Sectional Survey. JMIR Med Inform. 2019;7(4):e13466. pmid:31687938

* View Article

* PubMed/NCBI

* Google Scholar

56. 56. Gross JJ, John OP. Individual differences in two emotion regulation processes: implications for affect, relationships, and well-being. J Pers Soc Psychol. 2003;85(2):348–62. pmid:12916575

* View Article

* PubMed/NCBI

* Google Scholar

57. 57. Sato H, Kawahara J. Selective bias in retrospective self-reports of negative mood states. Anxiety Stress Coping. 2011;24(4):359–67. pmid:21253957

* View Article

* PubMed/NCBI

* Google Scholar

58. 58. Ray RD, McRae K, Ochsner KN, Gross JJ. Cognitive reappraisal of negative affect: converging evidence from EMG and self-report. Emotion. 2010;10(4):587–92. pmid:20677875

* View Article

* PubMed/NCBI

* Google Scholar

59. 59. Stone AA, Turkkan JS, Bachrach CA, Jobe JB, Kurtzman HS, Cain VS. The science of self-report: Implications for research and practice. Mahwah, NJ, US: Lawrence Erlbaum Associates Publishers. 2000.

60. 60. Kelker H, Yoder K, Musey P Jr, Harris M, Johnson O, Sarmiento E, et al. Prospective study of emergency medicine provider wellness across ten academic and community hospitals during the initial surge of the COVID-19 pandemic. BMC Emerg Med. 2021;21(1):36. pmid:33761876

* View Article

* PubMed/NCBI

* Google Scholar

61. 61. Marco CA, Larkin GL, Feeser VR, Monti JE, Vearrier L, ACEP Ethics Committee. Post-traumatic stress and stress disorders during the COVID-19 pandemic: Survey of emergency physicians. J Am Coll Emerg Physicians Open. 2020;1(6):1594–601. pmid:33392568

* View Article

* PubMed/NCBI

* Google Scholar

62. 62. Coffman JA. Chronic Stress, Physiological Adaptation and Developmental Programming of the Neuroendocrine Stress System. Future Neurol. 2020;15(1).

* View Article

* Google Scholar

63. 63. Dorsey A, Scherer EM, Eckhoff R, Furberg R. Measurement of human stress: A multidimensional approach. Research Triangle Park, NC: RTI Press. 2022.

64. 64. Yates SW. Physician Stress and Burnout. Am J Med. 2020;133(2):160–4. pmid:31520624

* View Article

* PubMed/NCBI

* Google Scholar

Citation: Evans LV, Bonz JW, Buck S, Gerwin JN, Bonner S, Ikejiani S, et al. (2025) An adaptive simulation intervention decreases emergency physician physiologic stress while caring for patients during COVID-19: A randomized clinical trial. PLoS One 20(9): e0331488. https://doi.org/10.1371/journal.pone.0331488

About the Authors:

Leigh V. Evans

Roles: Conceptualization, Funding acquisition, Investigation, Project administration, Supervision, Writing – original draft, Writing – review & editing

Affiliation: Department of Emergency Medicine, Yale School of Medicine, New Haven, Connecticut, United States of America

James W. Bonz

Roles: Data curation, Investigation, Supervision, Writing – review & editing

Affiliation: Department of Emergency Medicine, Yale School of Medicine, New Haven, Connecticut, United States of America

Samuel Buck

Roles: Data curation, Investigation, Writing – review & editing

Affiliation: Department of Emergency Medicine, Yale School of Medicine, New Haven, Connecticut, United States of America

Jeffrey N. Gerwin

Roles: Formal analysis, Methodology, Project administration, Validation, Writing – original draft, Writing – review & editing

Affiliation: Department of Emergency Medicine, Yale School of Medicine, New Haven, Connecticut, United States of America

Shacelles Bonner

Roles: Data curation, Formal analysis, Investigation, Project administration, Writing – review & editing

Affiliations: Department of Emergency Medicine, Yale School of Medicine, New Haven, Connecticut, United States of America, Department of Emergency Medicine, NYU Grossman School of Medicine, New York, New York, United States of America

Suzette Ikejiani

Roles: Data curation, Formal analysis, Investigation, Project administration, Writing – review & editing

Affiliations: Department of Emergency Medicine, Yale School of Medicine, New Haven, Connecticut, United States of America, Department of Emergency Medicine, The Johns Hopkins University School of Medicine, Baltimore, Maryland, United States of America

Tatiana Moylan

Roles: Data curation, Investigation, Project administration, Resources, Writing – review & editing

Affiliation: Department of Emergency Medicine, Yale School of Medicine, New Haven, Connecticut, United States of America

Melissa Joseph

Roles: Formal analysis, Investigation, Methodology, Project administration, Resources, Writing – review & editing, Data curation

Affiliations: Department of Emergency Medicine, Yale School of Medicine, New Haven, Connecticut, United States of America, Langley Memorial Hospital, Langley, British Columbia, Canada, University of British Columbia, Vancouver, British Columbia, Canada,

Gustavo de Oliveira Almeida

Roles: Formal analysis, Investigation, Software, Validation, Writing – review & editing

Affiliation: University of Arizona Health Sciences, Phoenix, Arizona, United States of America

Jessica M. Ray

Roles: Funding acquisition, Investigation, Methodology, Project administration, Writing – review & editing, Conceptualization

Affiliations: Department of Health Outcomes & Biomedical Informatics, University of Florida College of Medicine, Gainesville, Florida, United States of America, Office of Quality and Patient Safety, St. Jude Children’s Research Hospital, Memphis, Tennessee, United States of America,

James D. Dziura

Roles: Formal analysis, Investigation, Methodology, Supervision, Writing – review & editing

Affiliations: Department of Emergency Medicine, Yale School of Medicine, New Haven, Connecticut, United States of America, Yale School of Public Health, New Haven, Connecticut, United States of America,

Arjun K. Venkatesh

Roles: Investigation, Methodology, Supervision, Writing – review & editing

Affiliation: Department of Emergency Medicine, Yale School of Medicine, New Haven, Connecticut, United States of America

ORICD: https://orcid.org/0000-0002-8248-0567

Ambrose H. Wong

Roles: Data curation, Funding acquisition, Investigation, Methodology, Project administration, Resources, Supervision, Writing – original draft, Conceptualization

E-mail: [email protected]

Affiliations: Department of Emergency Medicine, Yale School of Medicine, New Haven, Connecticut, United States of America, Department of Emergency Medicine, NYU Grossman School of Medicine, New York, New York, United States of America, Department of Population Health, NYU Grossman School of Medicine, New York, New York, United States of America

ORICD: https://orcid.org/0000-0001-7471-1647

[/RAW_REF_TEXT]

[/RAW_REF_TEXT]

[/RAW_REF_TEXT]

References

1. Adams JG, Walls RM. Supporting the Health Care Workforce During the COVID-19 Global Epidemic. JAMA. 2020;323(15):1439–40. pmid:32163102

2. Gan WH, Lim JW, Koh D. Preventing Intra-hospital Infection and Transmission of Coronavirus Disease 2019 in Health-care Workers. Saf Health Work. 2020;11(2):241–3. pmid:32292622

3. Hendin A, La Rivière CG, Williscroft DM, O’Connor E, Hughes J, Fischer LM. End-of-life care in the emergency department for the patient imminently dying of a highly transmissible acute respiratory infection (such as COVID-19). CJEM. 2020;22(4):414–7. pmid:32213224

4. Lai J, Ma S, Wang Y, Cai Z, Hu J, Wei N, et al. Factors Associated With Mental Health Outcomes Among Health Care Workers Exposed to Coronavirus Disease 2019. JAMA Netw Open. 2020;3(3):e203976. pmid:32202646

5. Sinsky CA, Brown RL, Stillman MJ, Linzer M. COVID-Related Stress and Work Intentions in a Sample of US Health Care Workers. Mayo Clin Proc Innov Qual Outcomes. 2021;5(6):1165–73. pmid:34901752

6. Nguyen J, Liu A, McKenney M, Liu H, Ang D, Elkbuli A. Impacts and challenges of the COVID-19 pandemic on emergency medicine physicians in the United States. Am J Emerg Med. 2021;48:38–47. pmid:33836387

7. Hamidi MS, Bohman B, Sandborg C, Smith-Coggins R, de Vries P, Albert MS, et al. Estimating institutional physician turnover attributable to self-reported burnout and associated financial burden: a case study. BMC Health Serv Res. 2018;18(1):851. pmid:30477483

8. Shanafelt TD, Mungo M, Schmitgen J, Storz KA, Reeves D, Hayes SN, et al. Longitudinal Study Evaluating the Association Between Physician Burnout and Changes in Professional Work Effort. Mayo Clin Proc. 2016;91(4):422–31. pmid:27046522

9. Sinsky CA, Dyrbye LN, West CP, Satele D, Tutty M, Shanafelt TD. Professional Satisfaction and the Career Plans of US Physicians. Mayo Clin Proc. 2017;92(11):1625–35. pmid:29101932

10. Han S, Shanafelt TD, Sinsky CA, Awad KM, Dyrbye LN, Fiscus LC, et al. Estimating the Attributable Cost of Physician Burnout in the United States. Ann Intern Med. 2019;170(11):784–90. pmid:31132791

11. Bragard I, Dupuis G, Fleet R. Quality of work life, burnout, and stress in emergency department physicians: a qualitative review. Eur J Emerg Med. 2015;22(4):227–34. pmid:25093897

12. Schneider A, Weigl M. Associations between psychosocial work factors and provider mental well-being in emergency departments: A systematic review. PLoS One. 2018;13(6):e0197375. pmid:29864128

13. Basu S, Qayyum H, Mason S. Occupational stress in the ED: a systematic literature review. Emerg Med J. 2017;34(7):441–7. pmid:27729392

14. Petrino R, Riesgo LG-C, Yilmaz B. Burnout in emergency medicine professionals after 2 years of the COVID-19 pandemic: a threat to the healthcare system?. Eur J Emerg Med. 2022;29(4):279–84. pmid:35620812

15. Wong AH, Pacella-LaBarbara ML, Ray JM, Ranney ML, Chang BP. Healing the Healer: Protecting Emergency Health Care Workers’ Mental Health During COVID-19. Ann Emerg Med. 2020;76(4):379–84. pmid:32534830

16. Shanafelt TD, Hasan O, Dyrbye LN, Sinsky C, Satele D, Sloan J, et al. Changes in Burnout and Satisfaction With Work-Life Balance in Physicians and the General US Working Population Between 2011 and 2014. Mayo Clin Proc. 2015;90(12):1600–13. pmid:26653297

17. West CP, Dyrbye LN, Shanafelt TD. Physician burnout: contributors, consequences and solutions. J Intern Med. 2018;283(6):516–29. pmid:29505159

18. Panagioti M, Panagopoulou E, Bower P, Lewith G, Kontopantelis E, Chew-Graham C, et al. Controlled Interventions to Reduce Burnout in Physicians: A Systematic Review and Meta-analysis. JAMA Intern Med. 2017;177(2):195–205. pmid:27918798

19. van Vendeloo SN, Brand PLP, Verheyen CCPM. Burnout and quality of life among orthopaedic trainees in a modern educational programme: importance of the learning climate. Bone Joint J. 2014;96-B(8):1133–8. pmid:25086133

20. Merlo G, Rippe J. Physician Burnout: A Lifestyle Medicine Perspective. Am J Lifestyle Med. 2020;15(2):148–57. pmid:33790702

21. Morris MC, Rao U. Psychobiology of PTSD in the acute aftermath of trauma: Integrating research on coping, HPA function and sympathetic nervous system activity. Asian J Psychiatr. 2013;6(1):3–21. pmid:23380312

22. Dishman RK, Nakamura Y, Garcia ME, Thompson RW, Dunn AL, Blair SN. Heart rate variability, trait anxiety, and perceived stress among physically fit men and women. Int J Psychophysiol. 2000;37(2):121–33. pmid:10831999

23. Kanthak MK, Stalder T, Hill LK, Thayer JF, Penz M, Kirschbaum C. Autonomic dysregulation in burnout and depression: evidence for the central role of exhaustion. Scand J Work Environ Health. 2017;43(5):475–84. pmid:28514792

24. Shaffer F, Ginsberg JP. An Overview of Heart Rate Variability Metrics and Norms. Front Public Health. 2017;5:258. pmid:29034226

25. Thayer JF, Lane RD. Claude Bernard and the heart-brain connection: further elaboration of a model of neurovisceral integration. Neurosci Biobehav Rev. 2009;33(2):81–8. pmid:18771686

26. Lennartsson A-K, Jonsdottir I, Sjörs A. Low heart rate variability in patients with clinical burnout. Int J Psychophysiol. 2016;110:171–8. pmid:27535344

27. de Vente W, van Amsterdam JGC, Olff M, Kamphuis JH, Emmelkamp PMG. Burnout Is Associated with Reduced Parasympathetic Activity and Reduced HPA Axis Responsiveness, Predominantly in Males. Biomed Res Int. 2015;2015:431725. pmid:26557670

28. Garcia-Vicuña D, Esparza L, Mallor F. Hospital preparedness during epidemics using simulation: the case of COVID-19. Cent Eur J Oper Res. 2022;30(1):213–49. pmid:34602855

29. Redmond B, Joseph M, Ray J, Shabanova V, Gross IT, Bruno C, et al. Stress as tool or toxin: physiologic markers and subjective report in neonatal simulation. Pediatr Res. 2020;88(5):784–91. pmid:32045934

30. Brydges R, Campbell DM, Beavers L, Khodadoust N, Iantomasi P, Sampson K, et al. Lessons learned in preparing for and responding to the early stages of the COVID-19 pandemic: one simulation’s program experience adapting to the new normal. Adv Simul (Lond). 2020;5:8. pmid:32514385

31. Schneider KA, Ngwa GA, Schwehm M, Eichner L, Eichner M. The COVID-19 pandemic preparedness simulation tool: CovidSIM. BMC Infect Dis. 2020;20(1):859. pmid:33213360

32. Turner L, Shamseer L, Altman DG, Weeks L, Peters J, Kober T, et al. Consolidated standards of reporting trials (CONSORT) and the completeness of reporting of randomised controlled trials (RCTs) published in medical journals. Cochrane Database Syst Rev. 2012;11(11):MR000030. pmid:23152285

33. Evans LV, Ray JM, Bonz JW, Joseph M, Gerwin JN, Dziura JD, et al. Improving patient and clinician safety during COVID-19 through rapidly adaptive simulation and a randomised controlled trial: a study protocol. BMJ Open. 2022;12(5):e058980. pmid:35589358

34. Haimovich AD, Ravindra NG, Stoytchev S, Young HP, Wilson FP, van Dijk D, et al. Development and Validation of the Quick COVID-19 Severity Index: A Prognostic Tool for Early Clinical Decompensation. Ann Emerg Med. 2020;76(4):442–53. pmid:33012378

35. Peabody JE, Ryznar R, Ziesmann MT, Gillman L. A Systematic Review of Heart Rate Variability as a Measure of Stress in Medical Professionals. Cureus. 2023;15(1):e34345. pmid:36865953

36. Jones KI, Amawi F, Bhalla A, Peacock O, Williams JP, Lund JN. Assessing surgeon stress when operating using heart rate variability and the State Trait Anxiety Inventory: will surgery be the death of us?. Colorectal Dis. 2015;17(4):335–41. pmid:25406932

37. Spielberger C, Gorsuch R, Lushene R, Vagg PR, Jacobs G. Manual for the State-Trait Anxiety Inventory. 1983.

38. Ayed A, Khalaf IA, Fashafsheh I, Saleh A, Bawadi H, Abuidhail J, et al. Effect of High-Fidelity Simulation on Clinical Judgment Among Nursing Students. Inquiry. 2022;59:469580221081997. pmid:35285302

39. Gerwin JN, de Oliveira Almeida G, Boyce MW, Joseph M, Wong AH, Burleson W, et al. HRVEST: a novel data solution for using wearable smart technology to measure physiologic stress variables during a randomized clinical trial. Front Comput Sci. 2024;6.

40. Chang TP, Azen C, Sherman JM. Physiological Stress Markers Following Resuscitations Remain Elevated Throughout Physician Shift Hours. Acad Emerg Med. 2020;27(6):510–3. pmid:32282983

41. Pham T, Lau ZJ, Chen SHA, Makowski D. Heart Rate Variability in Psychology: A Review of HRV Indices and an Analysis Tutorial. Sensors (Basel). 2021;21(12):3998. pmid:34207927

42. Megel ME, Black J, Clark L, Carstens P, Jenkins LD, Promes J, et al. Effect of High-Fidelity Simulation on Pediatric Nursing Students’ Anxiety. Clinical Simulation in Nursing. 2012;8(9):e419–28.

43. Stein C. The effect of clinical simulation assessment on stress and anxiety measures in emergency care students. Afr J Emerg Med. 2020;10(1):35–9. pmid:32161710

44. Wetzel CM, Black SA, Hanna GB, Athanasiou T, Kneebone RL, Nestel D, et al. The effects of stress and coping on surgical performance during simulations. Ann Surg. 2010;251(1):171–6. pmid:20032721

45. Oldenburg M, Wilken D, Wegner R, Poschadel B, Baur X. Job-related stress and work ability of dispatchers in a metropolitan fire department. J Occup Med Toxicol. 2014;9(1).

46. Petrowski K, Herhaus B, Schöniger C, Frank M, Pyrc J. Stress load of emergency service: effects on the CAR and HRV of HEMS emergency physicians on different working days (N = 20). Int Arch Occup Environ Health. 2019;92(2):155–64. pmid:30315367

47. Mefford JM, Kahle S, Gupta S, Tancredi D, Danielson AR, Clarke SO. Heart Rate Variability and Acute Stress Among Novice Airway Managers. AEM Educ Train. 2019;3(3):291–4. pmid:31360823

48. Virtanen M, Pentti J, Vahtera J, Ferrie JE, Stansfeld SA, Helenius H, et al. Overcrowding in hospital wards as a predictor of antidepressant treatment among hospital staff. Am J Psychiatry. 2008;165(11):1482–6. pmid:18676590

49. Virtanen M, Batty GD, Pentti J, Vahtera J, Oksanen T, Tuisku K, et al. Patient overcrowding in hospital wards as a predictor of diagnosis-specific mental disorders among staff: a 2-year prospective cohort study. J Clin Psychiatry. 2010;71(10):1308–12. pmid:20492841

50. Rudolph JW, Simon R, Raemer DB, Eppich WJ. Debriefing as formative assessment: closing performance gaps in medical education. Acad Emerg Med. 2008;15(11):1010–6. pmid:18945231

51. Eppich W, Cheng A. Promoting Excellence and Reflective Learning in Simulation (PEARLS): development and rationale for a blended approach to health care simulation debriefing. Simul Healthc. 2015;10(2):106–15. pmid:25710312

52. Sawyer T, Eppich W, Brett-Fleegler M, Grant V, Cheng A. More Than One Way to Debrief: A Critical Review of Healthcare Simulation Debriefing Methods. Simul Healthc. 2016;11(3):209–17. pmid:27254527

53. Akbar F, Mark G, Prausnitz S, Warton EM, East JA, Moeller MF, et al. Physician Stress During Electronic Health Record Inbox Work: In Situ Measurement With Wearable Sensors. JMIR Med Inform. 2021;9(4):e24014. pmid:33908888

54. Gregory ME, Russo E, Singh H. Electronic Health Record Alert-Related Workload as a Predictor of Burnout in Primary Care Providers. Appl Clin Inform. 2017;8(3):686–97. pmid:28678892

55. Heponiemi T, Kujala S, Vainiomäki S, Vehko T, Lääveri T, Vänskä J, et al. Usability Factors Associated With Physicians’ Distress and Information System-Related Stress: Cross-Sectional Survey. JMIR Med Inform. 2019;7(4):e13466. pmid:31687938

56. Gross JJ, John OP. Individual differences in two emotion regulation processes: implications for affect, relationships, and well-being. J Pers Soc Psychol. 2003;85(2):348–62. pmid:12916575

57. Sato H, Kawahara J. Selective bias in retrospective self-reports of negative mood states. Anxiety Stress Coping. 2011;24(4):359–67. pmid:21253957

58. Ray RD, McRae K, Ochsner KN, Gross JJ. Cognitive reappraisal of negative affect: converging evidence from EMG and self-report. Emotion. 2010;10(4):587–92. pmid:20677875

59. Stone AA, Turkkan JS, Bachrach CA, Jobe JB, Kurtzman HS, Cain VS. The science of self-report: Implications for research and practice. Mahwah, NJ, US: Lawrence Erlbaum Associates Publishers. 2000.

60. Kelker H, Yoder K, Musey P Jr, Harris M, Johnson O, Sarmiento E, et al. Prospective study of emergency medicine provider wellness across ten academic and community hospitals during the initial surge of the COVID-19 pandemic. BMC Emerg Med. 2021;21(1):36. pmid:33761876

61. Marco CA, Larkin GL, Feeser VR, Monti JE, Vearrier L, ACEP Ethics Committee. Post-traumatic stress and stress disorders during the COVID-19 pandemic: Survey of emergency physicians. J Am Coll Emerg Physicians Open. 2020;1(6):1594–601. pmid:33392568

62. Coffman JA. Chronic Stress, Physiological Adaptation and Developmental Programming of the Neuroendocrine Stress System. Future Neurol. 2020;15(1).

63. Dorsey A, Scherer EM, Eckhoff R, Furberg R. Measurement of human stress: A multidimensional approach. Research Triangle Park, NC: RTI Press. 2022.

64. Yates SW. Physician Stress and Burnout. Am J Med. 2020;133(2):160–4. pmid:31520624