Night vision: a new blueprint for bedside ultrasound training in medical residency

Abstract

Background

Faculty expertise and support, resident scheduling, and cost of ultrasound machines are common barriers encountered when attempting to implement a new point-of-care ultrasound (POCUS) curriculum. Integration of a POCUS curriculum into existing night medicine rotations helps bypass these barriers by minimizing the amount of trained faculty required and harnessing clinical opportunities within a pre-existing curriculum.

Methods

37 PGY-1 residents participated in this pilot study within the inpatient setting of VCU Health hospital, primarily during their night medicine rotations. Faculty included four full-time nocturnists. Residents received didactics on pulmonary and abdominal imaging and completed online modules. Practice opportunities occurred primarily during night medicine rotations under the supervision of the nocturnists.

Results

Residents underwent pre- and post-curriculum surveys and knowledge/skills assessments. Comfort, knowledge, and skills significantly increased pre- to post-curriculum. Mean skills assessment scores increased from 13.4 to 23.6 points out of 34 (p < 0.001). Mean knowledge assessment scores increased from 54 to 61% (p < 0.001).

Conclusion

Integration of a POCUS curriculum into the existing night medicine rotation bypassed common barriers and resulted in statistically significant increases in resident comfort, knowledge, and skills.

Background

As of 2020, approximately 57% of medical schools reported having a formal ultrasound curriculum and 60% of internal medicine residency training programs have either a formal elective or required curriculum for diagnostic point-of-care ultrasound (POCUS) [1, 2]. The American College of Physicians formally acknowledged “the important role of point-of-care ultrasound (POCUS) in internal medicine.” The Alliance for Academic Internal Medicine “recognizes and supports the integration of POCUS across the longitudinal training environment of UME, GME, and CME.” [3] Beyond residency, POCUS has an important role among hospitalists, general internists, primary care, and within the subspecialties [4, 5]. The integration of POCUS into clinical practice allows providers to spend more time at the bedside, conveying results in real time, and increases patients’ satisfaction and understanding of the diagnostic process [6,7,8,9].

Though POCUS is a beneficial component of graduate medical education (GME), there are common barriers to implementing a curriculum. On a recent national survey of internal medicine program directors, the difficulties include faculty expertise and support, resident scheduling, and cost of ultrasound machines [2]. In the 2022–2023 academic year, a pilot longitudinal curriculum in diagnostic POCUS was implemented for residents in the internal medicine residency program at Virginia Commonwealth University (VCU) in Richmond, Virginia. This curriculum was integrated into the existing night medicine rotations to provide opportunities for direct observation and one-on-one training by a cadre of four teaching nocturnists trained in POCUS. The integration helped bypass barriers by limiting the number of faculty to train and harnessing clinical opportunities available within a pre-existing curriculum. Training in this environment can also be carried out with limited equipment, with the trained nocturnist utilizing a handheld ultrasound device available to the housestaff working at night.

Night rotations are a prime setting for learning opportunities, particularly when supervised by an on-site attending. Hanson et al. describe nighttime clinical care as “typically devoid of administrative demands, which affords more opportunities for faculty to observe direct patient contact and provide guidance and specific feedback for advanced learners.” [10] Residents have noted that the advantages of experiential learning at night include “evaluating undifferentiated patients, having more time to think, fewer competing demands, and monitoring patients over time.” [11] “Twelve tips for teaching” recommends a return to the bedside, as bedside teaching and patient-centered teaching points are particularly memorable and engaging for learners [12]. Dedicated overnight supervision by attendings has been well received by residents and improves educational and clinical value [12,13,14]. In contrast, a less robust nighttime education model can be detrimental to resident training [15].

Incorporation of POCUS into a night medicine structure can not only bypass the common barriers but also enhance resident nighttime education by providing additional supervised learning opportunities at the bedside, with a focus on patient-centered learning. The creation of a POCUS curriculum within GME is not a novel idea, but there is no description of a POCUS curriculum integrated into night-based rotations in the literature. Based on these key points about the existing landscape for this type of training, and the advantages that the night medicine setting offers, we developed a single-center feasibility pilot study integrating POCUS into nighttime teaching.

Methods

Setting and participants

Participants of this pilot curriculum included 37 PGY-1 residents in the internal medicine residency program at VCU in Richmond, VA. This came from a combination of 33 categorical internal medicine interns and 4 combined medicine-pediatrics interns. The primary learning environment was the inpatient setting of VCU Health hospital during night medicine rotations.

The primary faculty were four full-time teaching nocturnists who supervise the residents on night medicine rotations. Faculty included a hospitalist who created and directs the curriculum, and had completed the national Society of Hospital Medicine-Chest POCUS Certificate of Completion. Another faculty member received their training through an ACGME-accredited combined emergency medicine and internal medicine residency program, and is board certified in both fields. Two additional full-time nocturnists were pursuing ongoing training in point-of-care ultrasound at the time of the pilot year.

The curriculum director was granted 0.10 full-time equivalent hours during the pilot year. The protected time allowed the director to coordinate and provide core conference lectures, perform quality assurance on resident submitted images, conduct the pre- and post-curriculum assessments, and to provide additional hands-on scanning opportunities.

Program description

The purpose of the curriculum is to provide trainees with foundational skills to eventually become competent and confident POCUS users by learning indications for use and skills for image acquisition and interpretation and having the opportunity to practice and integrate it into medical decision-making. The curriculum provides trainees with an understanding of ultrasound physics and probe and machine manipulation to facilitate optimal image acquisition. Didactic components include brief indication and integration components with focus on POCUS physics and image acquisition. Bedside teaching and quality reviews facilitate honing image acquisition skills. Residents accumulate an image portfolio throughout residency. Images for consideration toward the portfolio are submitted to the curriculum director for quality assurance review, with feedback provided in a timely manner upon submission.

The PGY-1 curriculum focused on pulmonary and abdominal systems. Additional systems were excluded for the pilot year to maximize feasibility. Educational sessions were during required core conference didactics. Sessions included an introduction to POCUS basics and physics, and lectures on pulmonary and abdominal scanning techniques. Residents completed online learning modules and also received training in utilization of Qpath. Residents underwent pre- and post-curriculum surveys, knowledge assessments, and skills assessments. All residents participated in night medicine during their PGY-1 year, during which they obtained hands-on experience integrating POCUS into assessment and management of patients, as well as practice opportunities. Additional hands-on scanning opportunities were also provided by the curriculum director during daytime inpatient ward settings.

Evaluation

Surveys, structured through Redcap, were developed for this study and assigned to the residents at the beginning and end of the year. The survey (see Methods Supplement S1) assesses prior training, confidence in using POCUS, likelihood of utilizing POCUS, and perceptions regarding the importance of POCUS. The survey tests knowledge with questions regarding ultrasound principles, physics, and case-based applications. Confidence and likelihood of utilizing POCUS were assessed on a 5-point Likert scale. Knowledge was measured on a continuous scale from 0 to max attainable value of 13 based on the questions correctly answered. Residents also participated in an 8-minute observed skills assessment utilizing standardized patients at the beginning and end of the year. A standardized grading rubric (see Methods Supplement S2) was created and used in the assessment. This was also measured on a continuous scale from 0 to max attainable value of 34 based on the checklist of items performed correctly. The survey and skills assessment, with its corresponding grading rubric, were created without the use of a previously validated tool. To ensure content validity, the two POCUS-trained nocturnists created the instruments and they were reviewed by additional content area experts.

The results of these assessments were then used to compare pre- and post-intervention outcomes for the dependent variables of knowledge, skills, and attitudes. The intervention is defined as one year of training in the formal POCUS curriculum. Data was collected from July through October for pre-intervention assessments. Data was then collected again at the end of the academic year, in May and June. This study was conducted in an educational setting and involved normal educational practices. It was approved as exempt research by the Institutional Review Board of VCU School of Medicine. All participants were thoroughly informed of the curricular components and associated research during an orientation session. Written informed consent to participate was obtained from all participants.

Outcomes

59% of categorical internal medicine and combined medicine-pediatrics interns (n = 22) completed both the pre- and post-curriculum skills tests, and 100% (n = 37) completed the knowledge assessments and were included in the analysis. The figure shows scores on the skills assessment comparing the score pre-curriculum (blue) to post-curriculum (orange). The table shows confidence scores for knowledge and comfort/likelihood of use. Notably, skills, knowledge and confidence showed a statistically significant increase from pre- to post-curriculum. Mean score on the skills assessment increased from 13.4 points to 23.6 points (p < 0.001). Mean score on the knowledge assessment increased from 54 to 61% (p < 0.003). Interns performed a total of 88 imaging studies (64 supervised by curriculum director) and submitted 69 for quality assurance. Of those, there were 121 accepted portfolio images.

Paired samples t-test was used to compare pre- and post-test scores. Data analysis was performed using R version 4.3.0 (R Core Team, 2023). Effect sizes were calculated using the Hedges Correction of Cohen’s d as implemented in the “rstatix” package in R (Kassambra, 2023) (Fig. 1) and (Table 1).

[IMAGE OMITTED: SEE PDF]

[IMAGE OMITTED: SEE PDF]

All residents in the program evaluated their general experience and perceptions of POCUS in the residency program annual survey in January 2023. The majority of respondents agreed that POCUS is important to their future career (61.4%, n = 93) and more than half (55.9%) have participated in point of-care ultrasound since beginning post-graduate training. Respondents reported night medicine (29.1%, n = 55) and intensive care units (30.9%) most frequently as settings in which they participated in POCUS. The most commonly reported barriers were competing clinical work (32.7%, n = 49), availability of trained attendings (28.6%), and lack of equipment (22.4%).

Discussion

After the pilot year of the longitudinal diagnostic POCUS curriculum, residents had a statistically significant increase in comfort with POCUS and objective knowledge and skills assessment scores. There was a statistically significant decrease in the perceived likelihood of use by the end of the year. This may be attributed to a differentiation towards different career interests throughout intern year. Responses in the residency program survey regarding whether or not POCUS is important preparation for their future career indicated that residents with certain career interests found POCUS less applicable but acknowledged its importance in other career paths.

The educational outcomes of this curriculum are comparable to those of daytime-focused curricula that have utilized pre- and post-curriculum surveys and tests. Ferre et al. [16] at the Indiana University School of medicine piloted a GME POCUS curriculum that utilized online modules and a hands-on simulation session each month for four months. Results showed an increase in knowledge, comfort, and utilization of POCUS. Brant et al. [17] describe implementation of a longitudinal curriculum at University of Colorado-affiliated hospitals, which involved three half-days of education over a three-month period. Results showed increased comfort and knowledge levels. Filler et al.l [18] analyzed the impact of a structured longitudinal curriculum for PGY-3 emergency medicine residents over 18 months that involved bi-monthly didactic sessions, a structured four-week rotation, and asynchronous online learning. Results showed an increase in trainee confidence, satisfaction, and perception of utilization of POCUS. They did not have a significant mean increase in quiz scores.

While the educational outcomes of this curriculum are comparable to those previously described, it also benefited from harnessing clinical opportunities within a pre-existing curriculum, limiting the number of faculty to train, and minimizing required ultrasound equipment. The incorporation of POCUS into an existing night medicine rotation bypassed these common barriers encountered when attempting to implement a POCUS curriculum. The utilization of dedicated nocturnists addresses the limitation of trained faculty available. Only four instructors (two fully trained faculty, two undergoing training), working mostly during their clinical time, were needed. A majority of the images were obtained under the supervision of the curriculum director, signifying the impact that a single trained nocturnist can have on the curriculum. A single handheld ultrasound, obtained at a fraction of the cost of cart-based machines, was made accessible to residents through the nocturnist. The need for resident scheduling and potential interference is also negated, as the curriculum takes advantage of a pre-existing rotation which interns complete multiple times throughout the year. Due to the higher number of patients covered by a single intern and the emphasis of clinical work on assessment and management of acute situations overnight, there are increased opportunities for supervised image acquisition and integration into clinical care. In addition to bypassing common barriers with a night medicine-based POCUS curriculum, the resident nighttime educational experience can be enhanced, with additional opportunities for supervised observation at the bedside and increased patient-centered instruction.

Reported barriers were not specific to POCUS on night medicine and were similar to those within daytime curricula. The lack of available trained faculty should improve with completion of faculty training following the pilot year. Access to ultrasounds was improved by focusing on acquisition of a less costly handheld device, primarily used at night, though the number of available ultrasounds overall remained limited.

There are some limitations to this pilot study. The number of participants is small due to the constraints of it being limited to one year within one residency program and a significant percentage being unable to complete the pre and post skills assessments due to scheduling conflicts. The pilot year focused on PGY-1 residents, to increase the feasibility, with plans to integrate subsequent PGY-1 classes into the curriculum. Logistical issues resulted in a delay in obtaining the handheld ultrasound until approximately halfway through the academic year, which limited opportunities for image acquisition. Another limitation is the variability of education time amongst the residents. Even though all residents spent a similar amount of time on nights, there likely was a large amount of variability in scanning opportunities due to demands of the shift and intrinsic motivation of the residents to attempt scans. In addition, opportunities to acquire images outside of the night medicine curriculum were not consistently taken advantage of by all participants. Lastly, proximity of night medicine rotations to knowledge and skills assessment sessions likely led to a small amount of recall bias.

In an effort to address these barriers and enhance the educational experience, steps are being taken for subsequent years of the curriculum. Additional resources, such as textbooks and ultrasound machines, are being acquired for resident use. Faculty training in point-of-care ultrasound is ongoing. A dedicated didactic and hands-on practice session is being built into intern orientation. Additional systems, including cardiac, are being incorporated in subsequent years. Efforts are underway to offer additional practice opportunities, including standardized simulation sessions during resident conference. A POCUS elective is also being created to offer residents a multidisciplinary experience and additional training opportunities.

Conclusion

With a majority of medical schools providing a formal POCUS curriculum [1] and the recognition of its importance in patient care in internal medicine [3, 4, 19], it is imperative to continue the education and training during graduate medical education in internal medicine. This pilot curriculum study showed that a POCUS curriculum concentrated during night medicine rotations is feasible and can overcome common hurdles in implementing a residency POCUS curriculum, while also enhancing nighttime education and providing outcomes comparable to traditional daytime-based curricula.

Data availability

The data that support the findings of this study are not openly available due to reasons of sensitivity and are available from the corresponding author upon reasonable request. Data is located in an institution-specific REDCap database.

Abbreviations

POCUS:

Point-Of-Care Ultrasound

GME:

Graduate Medical Education

VCU:

Virginia Commonwealth University

References

Russell FM, Zakeri B, Herbert A, Ferre RM, Leiser A, Wallach PM. The State of Point-of-Care Ultrasound Training in Undergraduate Medical Education: Findings From a National Survey. Acad Med. 2021 Nov 16. doi: 10.1097/ACM.0000000000004512. Epub ahead of print. https://doi.org/10.1016/j.amjmed.2021.11.002

LoPresti CM, Schnobrich D, Novak W, et al. Current point of care ultrasound use and training among internal medicine residency programs from the 2020 APDIM program director’s survey. Am J Med. 2022;135(3):397–404. Epub 2021 Dec 7. PMID: 34890562.

LoPresti CM, Jensen TP, Dversdal RK, Astiz DJ. Point-of-care ultrasound for internal medicine residency training: A position statement from the alliance of academic internal medicine. Am J Med, 132 (11), 1356–60. https://doi.org/10.1016/j.amjmed.2019.07.019

Soni NJ, Schnobrich D, Mathews BK, et al. Point-of-Care ultrasound for hospitalists: A position statement of the society of hospital medicine. J Hosp Med. 2019;14:E1–6. https://doi.org/10.12788/jhm.3079.

Bhagra A, Tierney DM, Sekiguchi H, Soni NJ. Point-of-Care Ultrasonography for Primary Care Physicians and General Internists. Mayo Clin Proc. 2016;91(12):1811–1827. https://doi.org/10.1016/j.mayocp.2016.08.023

Durston W, Carl ML, Guerra W. Patient satisfaction and diagnostic accuracy with ultrasound by emergency physicians. Am J Emerg Med. 1999;17(7):642–6. https://doi.org/10.1016/s0735-6757(99)90150-x.

Howard ZD, Noble VE, Marill KA, Sajed D, Rodrigues M, Bertuzzi B, Liteplo AS. Bedside ultrasound maximizes patient satisfaction. J Emerg Med. 2014;46(1):46–53. https://doi.org/10.1016/j.jemermed.2013.05.044.

Mathews BK, Miller PE, Olson APJ. Point-of-Care ultrasound improves shared diagnostic Understanding between patients and providers. South Med J. 2018;111(7):395–400. https://doi.org/10.14423/SMJ.0000000000000833.

Andersen CA, Brodersen J, Rudbæk TR, et al. Patients’ experiences of the use of point-of-care ultrasound in general practice– a cross-sectional study. BMC Fam Pract. 2021;22:116. https://doi.org/10.1186/s12875-021-01459-z.

Hanson JT, Pierce RG, Dhaliwal G. The new education frontier: clinical teaching at night. Acad Med. 2014;89(2):215-8. https://doi.org/10.1097/ACM.0000000000000096. PMID: 24362386.

Torwekar BL, Robinson M, Durham M, Cooper D, Wurster W, Bowen JL. What Residents Want: Perceptions of Learning During a Pediatrics Night Float Rotation. Hosp Pediatr. 2023;13(5):401–408. https://doi.org/10.1542/hpeds.2022-006954. PMID: 37070381.

Richards JB, Wilcox SR, Roberts DH, Schwartzstein RM. Twelve tips for overnight teaching. Med Teach. 2014;36(3):196–200. https://doi.org/10.3109/0142159X.2013.847911. Epub 2013 Oct 28. PMID: 24164552.

Greene JG. Neurohospitalists enhance resident perception of the educational and clinical value of a night float rotation. Neurohospitalist. 2013;3(4):179–84. https://doi.org/10.1177/1941874413495879. PMID: 24198898; PMCID: PMC3810834.

Al-Khafaji J, Konjeti VR, Call S. Midnight Report: A Novel Faculty-Guided Night Curriculum to Enhance Resident Nighttime Education. Southern Medical Journal. 2020;113(5):201–204. https://doi.org/10.14423/smj.0000000000001090. PMID: 32358612.

Bricker DA, Markert RJ. Night float teaching and learning: perceptions of residents and faculty. J Grad Med Educ. 2010;2(2):236–41. https://doi.org/10.4300/JGME-D-10-00005.1. PMID: 21975627; PMCID: PMC2941384.

Ferre RM, Russell FM, Peterson D, Zakeri B, Herbert A, Nti B, Goldman M, Wilcox JG, Wallach PM. Piloting a Graduate Medical Education Point-of-Care Ultrasound Curriculum. Cureus. 2022;14(7):e27173. https://doi.org/10.7759/cureus.27173. PMID: 36017274.

Brant JA, Orsborn J, Good R, Greenwald E, Mickley M, Toney AG. Evaluating a longitudinal point-of-care-ultrasound (POCUS) curriculum for pediatric residents. BMC Med Educ. 2021;21(1):64. https://doi.org/10.1186/s12909-021-02488-z. PMID: 33468138.

Filler L, Orosco D, Rigdon D, Mitchell C, Price J, Lotz S, Stowell JR. Evaluation of a novel curriculum on point-of-care ultrasound competency and confidence. Emerg Radiol. 2020;27(1):37–40. https://doi.org/10.1007/s10140-019-01726-8. Epub 2019 Sep 4. PMID: 31485848.

Qaseem A, Etxeandia-Ikobaltzeta I, Mustafa RA et al. Appropriate Use of Point-of-Care Ultrasonography in Patients With Acute Dyspnea in Emergency Department or Inpatient Settings: A Clinical Guideline From the American College of Physicians. Ann Intern Med. 2021;174(7):985–993. https://doi.org/10.7326/M20-7844. PMID: 33900792.