The Otolaryngology boot camp: a scoping review evaluating commonalities and appraisal for curriculum design and delivery

Abstract

Objective

Surgical boot camps are becoming increasingly popular in Otolaryngology–Head and Neck Surgery (OHNS) residency programs. Despite pioneering virtual reality and simulation-based surgical education, these boot camps have lacked critical appraisal. The objective of this article was to examine the adoption and utility of surgical boot camps in OHNS residency training programs around the world.

Data Sources

Ovid Medline and PubMed databases were systematically searched in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines for scoping reviews. Additionally, a grey literature search was performed.

Review Methods

Inclusion criteria were peer-reviewed publications and grey literature sources that reported on OHNS boot camps for the novice learner. The search was restricted to human studies published in English. Studies were excluded if they were not examining junior trainees.

Results

A total of 551 articles were identified. Following removal of duplicates, screening, and full text review, 16 articles were included for analysis. Seven major boot camps were identified across various academic sites in the world. Most boot camps were one-day intensive camps incorporating a mixture of didactic, skill specific, and simulation sessions using an array of task trainers and high-fidelity simulators focusing on OHNS emergencies. Studies measuring trainee outcomes demonstrated improvement in trainee confidence, immediate knowledge, and skill acquisition.

Conclusion

Surgical boot camps appear to be an effective tool for short term knowledge and skill acquisition. Further studies should examine retention of skill and maintenance of confidence over longer intervals, as little is known about these lasting effects.

Introduction

Upon completing medical school, junior trainees enter post-graduate training programs with dramatically increased responsibilities. To address the concern regarding trainee skill inadequacy, surgical boot camps were developed to help develop skillsets from interpreting diagnostic imaging to performing surgical procedures [1].

The educational design of most surgical boot camps is a combination of didactic learning and small group simulation sessions. Both governing medical educational bodies of Canada (Royal College of Physicians and Surgeons of Canada) and the United States (Accreditation Council for Graduate Medical Education) have embraced competency-based educational frameworks for post graduate medical education (PGME) [2]. These frameworks are an outcomes-based approach to curriculum design where trainee advancement is dependent on mastering entrustable professional activities (EPA’s) [1]. With this shift, simulation training is integral in allowing trainees to practice clinical and procedural skills in areas specifically identified as key competencies or milestones before encountering real patient scenarios [1].

Literature examining the role of surgical boot camps has been extensively covered over the past decade. The majority of studies have examined the following outcomes: knowledge and technical skills acquisition, team communication skill development, and individual confidence improvement [3,4,5,6]. Moreover, surgical boot camps allow for social and cultural welcoming [7, 8]. Despite widespread adoption by various surgical specialties, including cardiac, general, neuro, orthopedic, trauma, and vascular surgery, few surgical boot camps have been reported on in Otolaryngology–Head and Neck Surgery (OHNS) [9,10,11,12,13,14,15,16]. Furthermore, OHNS boot camps lack critical appraisal despite being one of the leaders in virtual reality and simulation-based surgical education [17, 18]. The goal of this scoping review was to examine the utility of PGME surgical boot camps in OHNS around the world. To achieve this goal, this manuscript will address four fundamental objectives. (1) Thoroughly summarize existing OHNS boot camps around the world. (2) Determine overlap in curriculum design and delivery, resources, and simulation. (3) Examine pros and cons of existing boot camp formats. (4) Suggest an optimal boot camp design for junior residents in OHNS.

Methods

A scoping review based on the Preferred Reporting Items for Systematic Reviews and Meta-Analysis scoping review (PRISMA-ScR) guidelines was performed in February 2021 [19]. The research databases included were Ovid Medline (September 1946 – February 2021) and PubMed (January 1946 –February 2021). The search terms included [(otolaryngology/otorhinolaryngology/ear nose throat/ENT/ORL/head and neck surgery) AND (“boot camp/bootcamp/training course)]. Inclusion criteria were peer-reviewed publications comparing pre- and post-course quantitative and qualitative data in skill performance or knowledge acquisition. The search was restricted to human studies published in English. In addition to the peer-reviewed search, an online grey literature search was utilized, specifically looking at conference proceedings and published information from medical educational and department websites. Excluded studies were non-English publications and studies not examining OHNS interns or junior residents (PGY-1 and PGY-2). Due to boot camps typically being introductory camps, the search was limited to junior residents. All other articles including opinion pieces and editorials were included for qualitative analysis.

Four reviewers (A.B-S., C.J.L., M.Y.H., & J.T.L.,) independently screened all abstracts to identify studies that fulfilled the predetermined eligibility criteria. Any disagreement between the reviewers was resolved by consensus. Qualitative data from each included study was extracted using standardized data forms including the study’s title, author(s), year of publication, education themes, and outcomes assessed.

Results

A total of 21 articles were identified by Ovid Medline, 527 articles by PubMed, and 3 articles from a grey literature search. Following the removal of the duplicate records, 530 abstracts were screened (Fig. 1) [20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94]. Of the 79 articles that underwent full-text review, 63 articles were excluded, and the remaining 16 articles underwent complete qualitative analysis with the data being summarized in Tables 1, 2, 3, 4 and 5.

[IMAGE OMITTED: SEE PDF]

[IMAGE OMITTED: SEE PDF]

[IMAGE OMITTED: SEE PDF]

[IMAGE OMITTED: SEE PDF]

[IMAGE OMITTED: SEE PDF]

[IMAGE OMITTED: SEE PDF]

Boot camps were analyzed for their course objectives, outcomes assessed, and overall study conclusions (Table 1). The earliest boot camp identified was in 2011, where Georgetown University (Washington, DC, United States of America) hosted the inaugural training course for junior trainees. This program established the standard to which subsequent boot camps developed their curricula [87]. Thirteen of the sixteen studies described one-day courses, while the remaining three were longitudinal in design, taking place over one- to six-months. All camps incorporated technical skills stations, simulation sessions, and didactic teaching surrounding common OHNS emergencies and consultation requests. Overall, these sixteen studies could more easily be organized into seven international boot camps with their associated academic centres (Tables 2, 4 and 5). Boot camps were based at the University of Georgetown (USA), New York University (USA), Albert Einstein College of Medicine (USA), University of California, Davis (USA), Western University (Canada), University of Cambridge (United Kingdom), and Hospital Un Canto a la Vida (Ecuador).

Most boot camps specifically stated their objectives. Common themes included recognizing and triaging common OHNS emergencies, performing critical basic procedural skills, communicating within a team, and knowing when to call for help. Several task trainers and simulators used for the development of specific procedural skills are listed in Table 2 and categorized by subspecialty in Table 3. The most common simulation scenarios included management of post-surgical and oropharyngeal bleeding (57%), acute airway obstruction from angioedema (43%), and facial/neck trauma (29%). The most common task trainers were surgical airway (71%), epistaxis (57%), peritonsillar abscess drainage (43%), and bag mask ventilation with tracheal intubation (29%). Taking this together, skills stations could be categorized into either 1) basic airway control or 2) special skills. Basic airway control stations include bag mask ventilation, intubation, and surgical airway simulation. Special skill stations include bronchoscopy, peritonsillar abscess drainage, epistaxis and post-tonsillectomy hemorrhage control. Using this terminology allows boot camps to develop goal-oriented simulation stations with thoughtful and explicitly stated objectives.

With respect to each boot camp’s educational frameworks, all courses incorporated some elements of didactic and simulation sessions (Table 4). Didactic sessions involved common OHNS on-call scenarios, emergency situations, operative skills, and perioperative care of the post-surgical patient. Simulation sessions were predominantly focused on acute and subacute OHNS presentations including airway obstruction, epistaxis, and trauma. OHNS simulation resources can be subdivided into physical task trainers to virtual reality platforms [17]. Physical task trainers including mannequin, animal, and cadaveric simulators are often employed (Table 5).

Our synthesis of the data demonstrated that participation in introductory boot camps appears to improve trainee confidence [16, 84, 87, 93], immediate knowledge acquisition [82, 85, 92, 93], and immediate improvement in procedural skills [83, 91] (Table 1). Studies utilizing prospective cohorts and randomized controlled trials (RCTs) revealed an improvement in immediate didactic knowledge (as demonstrated by multiple choice examination), technical skills (based on blinded faculty assessment), and self-perceived confidence which was maintained up to 6 months [82, 87, 91, 93]. In a head-to-head RCT comparing simulation versus traditional didactic learning methods, junior trainees randomized to the simulation arm performed significantly better in both epistaxis and epiglottitis scenarios scored individually by a blinded expert surgeon. Additionally, participants randomized to the simulation group had an improved perception of education and were more likely to make positive recommendations to their colleagues [91].

This study is the first scoping review in OHNS boot camps for junior resident learners. Through our analysis, we have gained valuable insight into the variability of practices around the world. In Table 6 and Table 7, we have summarized our interpreted pros and cons of various boot camps features and developed suggestions for successfully implementing an OHNS surgical boot camp for junior residents.

[IMAGE OMITTED: SEE PDF]

[IMAGE OMITTED: SEE PDF]

One-to-seven-day camps for junior learners provide an optimal balance of relative ease in camp set up and execution with less time away from clinical activities for learners. Multidisciplinary staff including faculty from anesthesia, emergency medicine, thoracic surgery along with OHNS may provide added expertise and allow for more focus on interdisciplinary teamwork which is integral for trainee development. Didactic-based curriculum leads to improvements in knowledge retention and comprehension post course [82] while simulation improves confidence, competence, skill performance, and adds value to the learners’ overall experience with specific emphasis on teamwork / collaboration [1, 82, 85, 87, 91, 94]. Therefore, a curriculum with both didactic and simulation-based learning is advised. Learner feedback should be facilitated in a safe learning environment with emphasis on resident experience with combination of structured written and oral debriefing sessions after simulation (Tables 6 and 7).

Discussion

Intensive crash courses for residents and fellows have existed in OHNS for numerous years employing simulation to enhance specific aspects of training such as functional endoscopic sinus surgery, removal of foreign bodies, or management of facial trauma [26, 32, 77, 78]. Contrastingly, the concept of an introductory “boot camp” style training course for incoming OHNS trainees emphasizing fundamental skills is a recent occurrence.

As the first published modern-day boot camp for junior OHNS trainees, the Georgetown University boot camp began as a simple, simulation-based one-day emergency course. This has become popularized across the world since its inception in 2011 [87]. Many institutions have adopted similar boot camp style courses for junior trainees with mirroring objectives and content throughout the United States, Canada, and the United Kingdom. Several themes of the modern-day boot camp include the use of simulation, interdisciplinary faculty and trainees (anaesthesia, emergency medicine, family medicine, and pediatrics), and the use of validated educational frameworks for curriculum design (Kolb learning style theory and needs assessment models).

Simulation is an educational approach that enables learners to encounter components of the clinical interactions while enabling educators to provide education and simultaneous assessment in a standardized environment [17, 95]. Widely adopted across various industries, simulation as a training adjunct has become a staple in aerospace and military training, whereas its adoption in medical education has been comparatively slow [96]. In 2012, the Accreditation Council for Graduate Medical Education (ACGME) recognized simulation as a means of evaluating resident performance for various “educational milestones,” in its shift towards competency-based medical education [97, 98]. In OHNS, trainee knowledge and procedural skill were evaluated via cadaveric dissection, temporal bone drilling, and surgical simulator labs [99]. A recent national survey of American OHNS residency programs demonstrated that nearly two-thirds of programs incorporated simulation modalities into curricula [100]. When assessing the Canadian landscape in OHNS programs, 30.8% actively use some form of VR training simulator that 90.9% of program directors felt would be a fair and effective means for evaluation [101]. Given the importance of simulation training in OHNS, many boot camps utilize this method to help junior trainees develop critical skills in a controlled environment.

In this scoping review, all seven boot camps used simulation as the curriculum core through simulation scenarios and specific task trainers. The most common simulation scenarios included management of post-surgical and oropharyngeal bleeding (57%), acute airway obstruction from angioedema (43%), and facial/neck trauma (29%). The most common task trainers were surgical airway (71%), epistaxis (57%), peritonsillar abscess drainage (43%), and bag mask ventilation with tracheal intubation (29%). High fidelity cadaveric and mannequin-based task trainers for task specific procedures appear to be the current trend. All studies that used high fidelity simulation scenarios used the Laerdal (Wappinger Falls, NY) SimMan® adult simulator. SimMan® offers a highly realistic training model with real time neurological and physiological function.

Despite some of the diversity in task trainers and simulations used across the world, the principal theme in all boot camp curricula appeared to be management of emergency situations and on-call scenarios. The goal was to have junior trainees leave the camp equipped with the skillset to identify and triage acute emergencies, perform basic minor airway procedures, and communicate and activate emergency protocols. We noted that trainee participation in introductory boot camps appears to improve their confidence, immediate knowledge acquisition, and immediate improvement in procedural skills in comparison to traditional didactic methods of learning [82, 87, 91, 93]. Simulation learning also improved performance significantly in epistaxis and epiglottis scenarios, improved perception of education and increased the likelihood of making positive recommendations to colleagues when compared to traditional didactic learning methods [91]. The large heterogeneity of the studies included in this review precludes meta-analysis. However, the role of this scoping review was to examine OHNS boot camps more descriptively around the world. Here we have identified a trend in the literature suggesting positive outcomes for trainees that participate in introductory boot camps for their overall clinical and psychosocial development as an early trainee.

Despite strongly positive outcomes from boot camps and simulation training, criticisms of the lack of evidence to suggest long-term retention exist [31, 67]. Three studies demonstrated that perceived confidence in procedural tasks and knowledge lasted up to 2- 6 months [87, 93, 102]. However, neither long-term knowledge retention nor procedural competency has been assessed among OHNS trainees. Also, according to a survey of OHNS residency program directors in the United States and Puerto Rico, there are several barriers that exist which prevent participation in boot camps and simulation training [67]. Some of these include cost, lack of local access, lack of interest, and scheduling difficulties [67]. This suggests making boot camp programs more widely available, having partially subsidized costs, and more data on their short- and long-term benefits could address the hesitancy that some program directors have.

Although boot camps are typically delivered at the beginning of OHNS programs because they are introductory, consensus on when they should be offered is lacking. When surveying American OHNS program directors, a slight majority felt boot camps should be offered within the first few months of residency [67]. Interestingly, simulation training programs have been shown to be effective in all postgraduate years, with knowledge and skills acquisition demonstrated across all training levels [31]. Several other studies have evaluated the effectiveness of OHNS boot camps for medical students and suggest that boot camps may aid with the transition to residency as they all reported improved knowledge, confidence, and clinical performance after completion of the course [27, 33, 38, 62]. Taking these pieces of evidence together, it seems that the surgical boot camp style of education delivery at any level is beneficial in the short-term of less than six months. The lasting effects, however, remain uncertain and future investigations should examine the long-term retention of knowledge, confidence, and technical skill.

Conclusion

Boot camp style training programs for junior OHNS are becoming widely adopted across the world. Fuelled by the utilization of simulation technology to deliver time-effective education for common OHNS emergencies, these programs embrace the educational shift towards competency-based accreditation standards for residency programs. A number of studies have justified this form of education to improve trainee’s performance, confidence, and skill in the short term. However, current literature has failed to examine a number of important long-term outcomes. Future studies that examine the effect of OHNS boot camps on long term outcomes will play a critical role justifying widespread adoption of boot camps for resident education.

Availability of data and materials

Not applicable. No datasets were generated or analyzed during this study.

References

Blackmore C, Austin J, Lopushinsky SR, Donnon T. Effects of postgraduate medical education boot camps on clinical skills, knowledge, and confidence: a meta-analysis. J Grad Med Educ. 2014;6(4):643–52. https://doi.org/10.4300/jgme-d-13-00373.1.

Srinivasan M, Li ST, Meyers FJ, Pratt DD, Collins JB, Braddock C, et al. Teaching as a competency: competencies for medical educators. Acad med J Assoc Am Med Coll. 2011;86(10):1211–20. https://doi.org/10.1097/ACM.0b013e31822c5b9a.

Seddon J. General practitioners teaching new surgical trainees about clinic letter writing. Clin Teach. 2018;15(1):44–7. https://doi.org/10.1111/tct.12633.

Heskin L, Mansour E, Lane B, Kavanagh D, Dicker P, Ryan D, et al. The impact of a surgical boot camp on early acquisition of technical and nontechnical skills by novice surgical trainees. Am J Surg. 2015;210(3):570–7. https://doi.org/10.1016/j.amjsurg.2014.12.046.

Fernandez GL, Page DW, Coe NP, Lee PC, Patterson LA, Skylizard L, et al. Boot cAMP: educational outcomes after 4 successive years of preparatory simulation-based training at onset of internship. J Surg Educ. 2012;69(2):242–8. https://doi.org/10.1016/j.jsurg.2011.08.007.

Krajewski A, Filippa D, Staff I, Singh R, Kirton OC. Implementation of an intern boot camp curriculum to address clinical competencies under the new accreditation council for graduate medical education supervision requirements and duty hour restrictions. JAMA Surg. 2013;148(8):727–32. https://doi.org/10.1001/jamasurg.2013.2350.

Bhatt NR, Doherty EM, Mansour E, Traynor O, Ridgway PF. Impact of a clinical decision making module on the attitudes and perceptions of surgical trainees. ANZ J Surg. 2016;86(9):660–4. https://doi.org/10.1111/ans.13448.

Cleland J, Walker KG, Gale M, Nicol LG. Simulation-based education: understanding the socio-cultural complexity of a surgical training “boot camp.” Med Educ. 2016;50(8):829–41. https://doi.org/10.1111/medu.13064.

Ortiz Figueroa F, Moftakhar Y, Dobbins Iv AL, Khan R, Dasgupta R, Blanda R, et al. Trauma boot camp: a simulation-based pilot study. Cureus. 2016;8(1):e463. https://doi.org/10.7759/cureus.463.

Seeley MA, Kazarian E, King B, Biermann JS, Carpenter JE, Caird MS, et al. Core concepts: orthopedic intern curriculum boot camp. Orthopedics. 2016;39(1):e62–7. https://doi.org/10.3928/01477447-20151228-03.

Sheahan MG, Duran C, Bismuth J. National simulation-based training of fellows: the vascular surgery example. Surg clin North Am. 2015;95(4):781–90. https://doi.org/10.1016/j.suc.2015.04.008.

Bismuth J, Duran C, Donovan M, Davies MG, Lumsden AB. The cardiovascular fellows bootcamp. J Vasc Surg. 2012;56(4):1155-61.e1. https://doi.org/10.1016/j.jvs.2012.05.108.

Selden NR, Anderson VC, McCartney S, Origitano TC, Burchiel KJ, Barbaro NM. Society of neurological surgeons boot camp courses: knowledge retention and relevance of hands-on learning after 6 months of postgraduate year 1 training. J Neurosurg. 2013;119(3):796–802. https://doi.org/10.3171/2013.3.jns122114.

Yeh DH, Fung K, Malekzadeh S. Boot camps: preparing for residency. Otolaryngol Clin North Am. 2017;50(5):1003–13. https://doi.org/10.1016/j.otc.2017.05.010.

Chin CJ, Roth K, Rotenberg BW, Fung K. Emergencies in otolaryngology-head and neck surgery bootcamp: a novel Canadian experience. Laryngoscope. 2014;124(10):2275–80. https://doi.org/10.1002/lary.24754.

Chin CJ, Chin CA, Roth K, Rotenberg BW, Fung K. Simulation-based otolaryngology—head and neck surgery boot camp: “how I do it.” J Laryngol Otol. 2016;130(3):284–90. https://doi.org/10.1017/S0022215115003485.

Deutsch ES. Simulation in otolaryngology: smart dummies and more. Otolaryngol Head Neck Surg. 2011;145(6):899–903. https://doi.org/10.1177/0194599811424862.

Deutsch ES, Malloy KM, Malekzadeh S. Simulation-based otorhinolaryngology emergencies boot camp: Part 3: Complex teamwork scenarios and conclusions. Laryngoscope. 2014;124(7):1570–2. https://doi.org/10.1002/lary.24570.

Moher D, Liberati A, Tetzlaff J, Altman DG, The PG. Preferred reporting items for systematic reviews and meta-analyses: the prisma statement. PLoS Med. 2009;6(7):e1000097. https://doi.org/10.1371/journal.pmed.1000097.

Bakker NH, Fokkens WJ, Grimbergen CA. Investigation of training needs for functional endoscopic sinus surgery (FESS). Rhinology. 2005;43(2):104–8.

Bock A, Modabber A, Hölzle F, Prescher A, Classen-Linke I. Improvement of anatomical knowledge and surgical skills in head and neck region - An interdisciplinary hands-on course for clinical students. Annal anat Anat Anz off organ of the Anatomische Gesellschaft. 2019;224:97–101. https://doi.org/10.1016/j.aanat.2019.03.011.

Bouhabel S, Kay-Rivest E, Nhan C, Bank I, Nugus P, Fisher R, et al. Error Detection-Based Model to Assess Educational Outcomes in Crisis Resource Management Training: A Pilot Study. Otolaryngol Head Neck Surg. 2017;156(6):1080–3. https://doi.org/10.1177/0194599817697946.

Bur AM, Gomez ED, Newman JG, Weinstein GS, O’Malley BW Jr, Rassekh CH, et al. Evaluation of high-fidelity simulation as a training tool in transoral robotic surgery. Laryngoscope. 2017;127(12):2790–5. https://doi.org/10.1002/lary.26733.

Chambers KJ, Aswani J, Patel A, Fundakowski C, Mannion K, Lin DT, et al. The value of a collaborative course for advanced head and neck surgery in East Africa. Laryngoscope. 2015;125(4):883–7. https://doi.org/10.1002/lary.25028.

Chang B, Powell A, Ellsperman S, Wehrmann D, Landry A, Jabbour N, et al. Multicenter advanced pediatric otolaryngology fellowship prep surgical simulation course with 3d printed high-fidelity models. Otolaryngol Head Neck Surg. 2020;162(5):658–65. https://doi.org/10.1177/0194599820913003.

Christophel JJ, Park SS, Nogan SJ, Essig GF Jr. A Facial Trauma simulation course for evaluation and treatment of facial fractures. JAMA facial plast surg. 2017;19(6):464–7. https://doi.org/10.1001/jamafacial.2017.0313.

Dermody SM, Masciello M, Malekzadeh S. A multispecialty critical airway simulation course for medical students. Laryngoscope. 2020. https://doi.org/10.1002/lary.29264.

Fortes B, Balsalobre L, Weber R, Stamm R, Stamm A, Oto F, et al. Endoscopic sinus surgery dissection courses using a real simulator: the benefits of this training. Braz J Otorhinolaryngol. 2016;82(1):26–32. https://doi.org/10.1016/j.bjorl.2015.02.003.

Griffin GR, Hoesli R, Thorne MC. Validity and efficacy of a pediatric airway foreign body training course in resident education. Ann Otol Rhinol Laryngol. 2011;120(10):635–40. https://doi.org/10.1177/000348941112001002.

Hinchcliff M, Kao M, Johnson K. The importance of technical skills assessment during an airway foreign body removal course. Int J Pediatr Otorhinolaryngol. 2019;117:1–5. https://doi.org/10.1016/j.ijporl.2018.11.007.

Hogg ES, Kinshuck AJ, Littley N, Lau A, Tandon S, Lancaster J. A high-fidelity, fully immersive simulation course to replicate ENT and head and neck emergencies. J Laryngol Otol. 2019;133(2):115–8. https://doi.org/10.1017/s0022215118002347.

Jabbour N, Reihsen T, Sweet RM, Sidman JD. Psychomotor skills training in pediatric airway endoscopy simulation. Otolaryngol Head Neck Surg. 2011;145(1):43–50. https://doi.org/10.1177/0194599811403379.

Keilin CA, Farlow JL, Malloy KM, Bohm LA. Otolaryngology curriculum during residency preparation course improves preparedness for internship. Laryngoscope. 2021. https://doi.org/10.1002/lary.29443.

Kovatch KJ, Wertz AP, Carle TR, Harvey RS, Bohm LA, Thorne MC, et al. Optimal timing of entry-level otolaryngology simulation. OTO open. 2019;3(2):2473974x19845851. https://doi.org/10.1177/2473974x19845851.

Leeper WR, Haut ER, Pandian V, Nakka S, Dodd OJ, Bhatti N, et al. Multidisciplinary difficult airway course: an essential educational component of a hospital-wide difficult airway response program. J Surg Educ. 2018;75(5):1264–75. https://doi.org/10.1016/j.jsurg.2018.03.001.

Lind MM, Corridore M, Sheehan C, Moore-Clingenpeel M, Maa T. A multidisciplinary approach to a pediatric difficult airway simulation course. Otolaryngol Head Neck Surg. 2018;159(1):127–35. https://doi.org/10.1177/0194599818758993.

Shay SG, Chrin JD, Wang MB, Mendelsohn AH. Initial and Long-term retention of robotic technical skills in an otolaryngology residency program. Laryngoscope. 2019;129(6):1380–5. https://doi.org/10.1002/lary.27425.

Spiers H, Enayati H, Moussa R, Zargaran A, Thomas A, Murtaza A, et al. Augmenting ENT surgery outside the medical school curriculum: the role of a 1-day otolaryngology course. J Laryngol Otol. 2019;133(4):269–74. https://doi.org/10.1017/s0022215119000331.

A Vijendren, A Trinidade, A Ngu (2015) Is an Introduction to ENT course the answer for safe ENT care? European archives of oto-rhino-laryngology official journal of the European Federation of Oto-Rhino-Laryngological Societies (EUFOS) affiliated with the German Society for Oto-Rhino-Laryngology - Head and Neck Surgery. 272(4):1021–5, https://doi.org/10.1007/s00405-014-3362-2.

Volk MS, Ward J, Irias N, Navedo A, Pollart J, Weinstock PH. Using medical simulation to teach crisis resource management and decision-making skills to otolaryngology housestaff. Otolaryngol Head Neck Surg. 2011;145(1):35–42. https://doi.org/10.1177/0194599811400833.

Zirkle M, Blum R, Raemer DB, Healy G, Roberson DW. Teaching emergency airway management using medical simulation: a pilot program. Laryngoscope. 2005;115(3):495–500. https://doi.org/10.1097/01.mlg.0000157834.69121.b1, https://doi.org/10.2500/ajra.2009.23.3297.

Zuckerman JD, Wise SK, Rogers GA, Senior BA, Schlosser RJ, DelGaudio JM. The utility of cadaver dissection in endoscopic sinus surgery training courses. Am J Rhinol Allergy. 2009;23(2):218–24. https://doi.org/10.2500/ajra.2009.23.3297.

Al-Ramahi J, Luo H, Fang R, Chou A, Jiang J, Kille T. Development of an innovative 3D printed rigid bronchoscopy training model. Ann Otol Rhinol Laryngol. 2016;125(12):965–9. https://doi.org/10.1177/0003489416667742.

Amin M, Rosen CA, Simpson CB, Postma GN. Hands-on training methods for vocal fold injection education. Ann Otol Rhinol Laryngol. 2007;116(1):1–6. https://doi.org/10.1177/000348940711600101.

Andersen SA, Mikkelsen PT, Konge L, Caye-Thomasen P, Sorensen MS. Cognitive load in mastoidectomy skills training: virtual reality simulation and traditional dissection compared. J Surg Educ. 2016;73(1):45–50. https://doi.org/10.1016/j.jsurg.2015.09.010.

Braun T, Betz CS, Ledderose GJ, Havel M, Stelter K, Kuhnel T, et al. Endoscopic sinus surgery training courses: benefit and problems - a multicentre evaluation to systematically improve surgical training. Rhinology. 2012;50(3):246–54. https://doi.org/10.4193/Rhino11.266.

Cheng PC, Cho TY, Hsu WL, Lo WC, Wang CT, Cheng PW, et al. Training residents to perform tracheotomy using a live swine model. Ear Nose Throat J. 2019;98(7):E87-e91. https://doi.org/10.1177/0145561319840835.

Ishman SL, Brown DJ, Boss EF, Skinner ML, Tunkel DE, Stavinoha R, et al. Development and pilot testing of an operative competency assessment tool for pediatric direct laryngoscopy and rigid bronchoscopy. Laryngoscope. 2010;120(11):2294–300. https://doi.org/10.1002/lary.21067.

Lin SY, Laeeq K, Ishii M, Kim J, Lane AP, Reh D, et al. Development and pilot-testing of a feasible, reliable, and valid operative competency assessment tool for endoscopic sinus surgery. Am J Rhinol Allergy. 2009;23(3):354–9. https://doi.org/10.2500/ajra.2009.23.3275.

Mattioli F, Presutti L, Caversaccio M, Bonali M, Anschuetz L. Novel dissection station for endolaryngeal microsurgery and laser surgery: development and dissection course experience. Otolaryngol Head Neck Surg. 2017;156(6):1136–41. https://doi.org/10.1177/0194599816668324.

N Mehta, C Boynton, L Boss, H Morris, T Tatla Multidisciplinary difficult airway simulation training: two year evaluation and validation of a novel training approach at a District General Hospital based in the UK. European archives of oto-rhino-laryngology: official journal of the European Federation of Oto-Rhino-Laryngological Societies (EUFOS): affiliated with the German Society for Oto-Rhino-Laryngology - Head and Neck Surgery. 2013; 270(1):211–7. doi: https://doi.org/10.1007/s00405-012-2131-3.

Pacca P, Jhawar SS, Seclen DV, Wang E, Snyderman C, Gardner PA, et al. “Live cadaver” model for internal carotid artery injury simulation in endoscopic endonasal skull base surgery. Op Neurosurg (Hagerstown, Md). 2017;13(6):732–8. https://doi.org/10.1093/ons/opx035.

Svrakic M, Bent JP, 3rd. Individualized Learning Plan (ILP) Is an Effective Tool in Assessing Achievement of Otology-related Subcompetency Milestones. Otology and neurotology: official publication of the American Otological Society, American Neurotology Society [and] European Academy of Otology and Neurotology. 2018; 39(7):816–22. doi:https://doi.org/10.1097/mao.0000000000001855.

Verma SP, Dailey SH, McMurray JS, Jiang JJ, McCulloch TM. Implementation of a program for surgical education in laryngology. Laryngoscope. 2010;120(11):2241–6. https://doi.org/10.1002/lary.21099.

Wiebracht ND, Giliberto JP, Ct M, Casper K, Johnson KE. Pilot testing of a novel surgical simulator for endoscopic zenker’s diverticulotomy. Laryngoscope. 2017;127(3):592–6. https://doi.org/10.1002/lary.26129.

Akhlaghi MR, Vafamehr V, Dadgostarnia M, Dehghani A. Integrated surgical emergency training plan in the internship: A step toward improving the quality of training and emergency center management. J Educ Health Promot. 2013;2:60. https://doi.org/10.4103/2277-9531.120861.

Beer JI, Sieber DA, Scheuer JF, Greco TM. Three-dimensional facial anatomy: structure and function as it relates to injectable neuromodulators and soft tissue fillers. Plast Reconstruct Surg Glob Open. 2016;4:e1175. https://doi.org/10.1097/GOX.0000000000001175.

Calcagno HE, Lucke-Wold B, Noles M, Dillman D, Baskerville M, Spight D, et al. Integrated otolaryngology and anesthesia simulation model for crisis management of cavernous carotid artery injury. Archiv Neurol Neuro Disord. 2018;1(1):30–4100.

Chandran R, Kiew ASC, Zheng JX, Singh PA, Lim JKT, Koo SH, et al. Experiential learning in simulated parapharyngeal abscess in breathing cadavers. J Anesth. 2021;35(2):232–8. https://doi.org/10.1007/s00540-021-02904-0.

Green CA, Huang E, Zhao NW, O’Sullivan PS, Kim E, Chern H. Technical skill improvement with surgical preparatory courses: What advantages are reflected in residency? Am J Surg. 2017. https://doi.org/10.1016/j.amjsurg.2017.10.037.

Honeyman CS. Course review: north east microsurgery training course and workshop: a new two day microsurgery course for trainees in plastic surgery, otolaryngology and oral and maxillofacial surgery. Ann Plast Surg. 2016;77(3):262–3. https://doi.org/10.1097/sap.0000000000000733.

Kovatch KJ, Harvey RS, Schechtman SA, Healy DW, Malloy KM, Prince MEP, et al. Integrated otolaryngology-anesthesiology clinical skills and simulation rotation: A novel 1-month intern curriculum. Ann Otol Rhinol Laryngol. 2019;128(8):715–20. https://doi.org/10.1177/0003489419840682.

Mason KA, Theodorakopoulou E, Pafitanis G, Ghanem AM, Myers SR. Twelve tips for postgraduate or undergraduate medics building a basic microsurgery simulation training course. Med Teach. 2016;38(9):872–8. https://doi.org/10.3109/0142159x.2016.1150978.

Rivers CM. Course review: newcastle surgical training centre cadaveric speech and palate surgery course. Ann Plast Surg. 2019;83(5):e3–4. https://doi.org/10.1097/sap.0000000000001972.

Room HJ, Ji C, Kohli S, Choh CTP, Robinson P, Knight J, Dennis S. Core surgical field camps: a new deanery-based model for enhancing advanced skills in core surgical trainees through simulation. Br J Hosp Med. 2020;81(9):1–6. https://doi.org/10.12968/hmed.2020.0333.

Shulzhenko NO, Zeng W, Albano NJ, Lyon SM, Wieland AM, Mahajan AY, et al. Multispecialty microsurgical course utilizing the blue-blood chicken thigh model significantly improves resident comfort, confidence, and attitudes in multiple domains. J Reconstr Microsurg. 2020;36(2):142–50. https://doi.org/10.1055/s-0039-1700523.

Dean KM, DeMason CE, Choi SS, Malloy KM, Malekzadeh S. Otolaryngology boot camps: current landscape and future directions. Laryngoscope. 2019;129(12):2707–12. https://doi.org/10.1002/lary.27835.

Martins RH. Inserting otorhinolaryngology in the medical course. Braz J Otorhinolaryngol. 2006;72(5):578. https://doi.org/10.1016/S1808-8694(15)31011-9.

SAW Andersen, S Foghsgaard, P Cayé-Thomasen, MS Sørensen. The effect of a distributed virtual reality simulation training program on dissection mastoidectomy performance. Otology & neurotology: official publication of the American Otological Society, American Neurotology Society [and] European Academy of Otology and Neurotology. 2018; 39(10):1277–84. https://doi.org/10.1097/mao.0000000000002031.

Andersen SAW, Konge L, Sørensen MS. The effect of distributed virtual reality simulation training on cognitive load during subsequent dissection training. Med Teach. 2018;40(7):684–9. https://doi.org/10.1080/0142159x.2018.1465182.

Chen PG, Bassiouni A, Taylor CB, Psaltis AJ, Alrasheed A, Wrobel B, et al. Teaching residents frontal sinus anatomy using a novel 3-dimensional conceptualization planning software-based module. Am J Rhinol Allergy. 2018;32(6):526–32. https://doi.org/10.1177/1945892418801264.

Chen PG, McMains KC, Tewfik MA, Aden JK, Brown S, Weitzel EK. Teaching frontal sinus anatomy using the frontal sinus masterclass 3- D conceptualization model. Laryngoscope. 2018;128(6):1294–8. https://doi.org/10.1002/lary.26939.

Chin CJ, Clark A, Roth K, Fung K. Development of a novel simulation-based task trainer for management of retrobulbar hematoma. Int Forum Allergy Rhinol. 2020;10(3):412–8. https://doi.org/10.1002/alr.22494.

Hall C, Okhovat S, Milner TD, Montgomery J, Hitchings A, Kunanandam T, et al. Simulation training in laser safety education: the use of technical and non-technical skills simulation in a comprehensive laser safety course. J Laryngol Otol. 2019;133(8):700–3. https://doi.org/10.1017/s0022215119001506.

Jamal N, Bowe SN, Brenner MJ, Balakrishnan K, Bent JP. Impact of a Formal Patient Safety and Quality Improvement Curriculum: A Prospective. Controll Trial Laryngo. 2019;129(5):1100–6. https://doi.org/10.1002/lary.27527.

Kashat L, Carter B, Mosha M, Kavanagh KR. Mindfulness education for otolaryngology residents: a pilot study. OTO Open. 2020;4(3):2473974X2094527. https://doi.org/10.1177/2473974X20945277.

McFerran DJ, Grant HR, Ingrams DR, Fife DG. Endoscopic sinus surgery: are junior doctors being properly trained? Ann R Coll Surg Engl. 1998;80(5):359–63.

Paczona R. A cadaver larynx holder for teaching laryngomicrosurgery. J Laryngol Otol. 1997;111(1):56–7. https://doi.org/10.1017/s0022215100136424.

Patki A, Puscas L. A video-based module for teaching communication skills to otolaryngology residents. J Surg Educ. 2015;72(6):1090–4. https://doi.org/10.1016/j.jsurg.2015.07.008.

Sahovaler A, Eibling DE, Bruni I, Duvvuri U, MacNeil SD, Nichols AC, et al. Novel minimally invasive transoral surgery bleeding model implemented in a nationwide otolaryngology emergencies bootcamp. J Robot Surg. 2019;13(6):773–8. https://doi.org/10.1007/s11701-019-00920-7.

Wu K, Kim S, Rajasingham SJ, Bruni I, Fung K, Roth KE. Simulation of Urgent Airway Management in a Postthyroidectomy Hematoma. MedEdPORTAL J Teach Learn Resour. 2019;15:10802. https://doi.org/10.15766/mep_2374-8265.10802.

Amin MR, Friedmann DR. Simulation-based training in advanced airway skills in an otolaryngology residency program. Laryngoscope. 2013;123(3):629–34. https://doi.org/10.1002/lary.23855.

Bunting H, Wilson BM, Malloy KM, Malekzadeh S. A novel peritonsillar abscess simulator. Simul Healthc. 2015;10(5):320–5. https://doi.org/10.1097/sih.0000000000000104.

Cervenka BP, Hsieh TY, Lin S, Bewley A. Multi-institutional regional otolaryngology bootcamp. Ann Otol Rhinol Laryngol. 2020;129(6):605–10. https://doi.org/10.1177/0003489420903067.

Fuller JC, Justicz NS, Kim J, Cheney M, Castrillon R, Hadlock T. A facial plastic and reconstructive surgery training module using surgical simulation for capacity building. J Surg Educ. 2019;76(1):274–80. https://doi.org/10.1016/j.jsurg.2018.06.027.

Malekzadeh S, Deutsch ES, Malloy KM. Simulation-based otorhinolaryngology emergencies boot camp: Part 2: Special skills using task trainers. Laryngoscope. 2014;124(7):1566–9. https://doi.org/10.1002/lary.24571.

Malekzadeh S, Malloy KM, Chu EE, Tompkins J, Battista A, Deutsch ES. ORL emergencies boot camp: using simulation to onboard residents. Laryngoscope. 2011;121(10):2114–21. https://doi.org/10.1002/lary.22146.

Malloy KM, Malekzadeh S, Deutsch ES. Simulation-based otorhinolaryngology emergencies boot camp: Part 1: curriculum design and airway skills. Laryngoscope. 2014;124(7):1562–5. https://doi.org/10.1002/lary.24572.

Kiffel ER, Weinstock MS, Yang CJ, Rong E, Hollingsworth MA, Akbar N, et al. An Innovative otolaryngology resident boot camp. MedEdPublish. 2017;6:76. https://doi.org/10.15694/mep.2017.000076.

Scott GM, Fung K, Roth KE. Novel high-fidelity peritonsillar abscess simulator. Otolaryngol Head Neck Surg. 2016;154(4):634–7. https://doi.org/10.1177/0194599815625988.

Smith ME, Navaratnam A, Jablenska L, Dimitriadis PA, Sharma R. A randomized controlled trial of simulation-based training for ear, nose, and throat emergencies. Laryngoscope. 2015;125(8):1816–21. https://doi.org/10.1002/lary.25179.

Smith ME, Trinidade A, Tysome JR. The ENT boot camp: an effective training method for ENT induction. Clin Otolaryngol. 2016;41(4):421–4. https://doi.org/10.1111/coa.12533.

Swords C, Smith ME, Wasson JD, Qayyum A, Tysome JR. Validation of a new ENT emergencies course for first-on-call doctors. J Laryngol Otol. 2017;131(2):106–12. https://doi.org/10.1017/s002221511601001x.

Zapanta PE, Adler J, Constant J, Malekzadeh S, Deutsch ES, Fung K, et al. The Junior resident’s perspective of learning in a simulation-based otolaryngology boot camp. Otolaryngol Head Neck Surg. 2013;149(2 suppl):P154-P. https://doi.org/10.1177/0194599813496044a39.

Issenberg SB, Scalese RJ. Five tips for a successful submission on simulation-based medical education. J Grad Med Educ. 2014;6(4):623–5. https://doi.org/10.4300/JGME-D-14-00490.1.

Riley B. Using simulation-based medical education to meet the competency requirements for the single accreditation system. J Am Osteopath Assoc. 2015;115(8):504–8. https://doi.org/10.7556/jaoa.2015.104.

Griswold S, Fralliccardi A, Boulet J, Moadel T, Franzen D, Auerbach M, et al. Simulation-based education to ensure provider competency within the health care system. Acad Emerg Med Off J Soc Acad Emerg Med. 2017. https://doi.org/10.1111/acem.13322.

Hart D, Bond W, Siegelman JN, Miller D, Cassara M, Barker L, et al. Simulation for assessment of milestones in emergency medicine residents. Acad Emerg Med Off J Soc Acad Emerg Med. 2017. https://doi.org/10.1111/acem.13296.

The Otolaryngology Milestone Project. J Grad Med Educ. 2014;6(1 Suppl 1):166–79. https://doi.org/10.4300/JGME-06-01s1-22.

Kovatch KJ, Harvey RS, Prince MEP, Thorne MC. National trends in otolaryngology intern curricula following accreditation council for graduate medical education changes. Laryngoscope. 2017. https://doi.org/10.1002/lary.26960.

Lui JT, Compton ED, Ryu WHA, Hoy MY. Assessing the role of virtual reality training in Canadian otolaryngology-head & neck residency programs: a national survey of program directors and residents. J Otolaryngol Head Neck Surg. 2018;47(1):61. https://doi.org/10.1186/s40463-018-0309-4.

Maddry JK, Varney SM, Sessions D, Heard K, Thaxton RE, Ganem VJ, et al. A comparison of simulation-based education versus lecture-based instruction for toxicology training in emergency medicine residents. J Med Toxicol. 2014;10(4):364–8. https://doi.org/10.1007/s13181-014-0401-8.

Vogel L. Pre-residency boot camps calm medical graduate jitters. Can Med Assoc J. 2018;190(32):E967–8. https://doi.org/10.1503/cmaj.109-5642.