The usability of a virtual reality augmented training program to teach goniometry to occupational therapy students

Abstract

Background

Training occupational therapy students in manual skills such as goniometry typically requires intensive one on one student teacher interactions and repetitive practice to ensure competency. There is evidence that immersive virtual reality (IVR) supported embodied learning can improve confidence and performance of skills. Embodied learning refers to learner’s experience of viewing a simulated body and its properties as if they were the learner’s own biological body. The aim of this proof-of-concept study was to develop and examine the usability and establish preliminary efficacy of an Embodied Goniometric Occupational Therapy Training Program (EGOTTP) to teach occupational therapy students wrist goniometry.

Methods

Following the generation of written scripts, we used a 360-degree camera to film and create the program, with 3- levels of detailed education for six wrist goniometry measures, flexion-extension, radial-ulnar deviation and pronation-supination. Five students rated their experience using EGOTTP on measures of usability and efficacy.

Results

The EGOTTP was developed over a 1-month period, and the participants reported good to high levels of embodiment and engagement, and their ratings suggested that the EGOTTP was easy to use with the support of an educator.

Conclusion

IVR training programs such as this one hold promise for teaching manual skill acquisition such as goniometry. As IVR technologies become more affordable and widespread, this approach could complement other strategies used to teach goniometry skills to students.

Introduction

One key trend in the future of health profession education is a shift towards a competency-based education (CBE) model [1]. CBE is an approach to curriculum design which emphasizes acquisition of domains of profession specific skills to a pre-defined level of ability [2]. While there are many characteristics of CBE, two keystones are an emphasis on learners achieving competence in the skills and abilities required for their profession in ecologically valid situations and on educational programs providing alternative pathways for learners to achieve competence, supporting diverse learning abilities and experiences [2, 3].

Immersive virtual reality (IVR) is an emerging technology that can be used for health professional education and has the potential to be a powerful method to support competency building. IVR uses a head mounted display (HMD) that projects 3-dimensional images that enable the user to feel that they are fully present in the simulated environment. Types of learning that can occur in the IVR context vary, but include analysis and problem solving, developing soft skills such as communication, empathy and collaboration skills, learning declarative knowledge, and training in procedural-practical skills, such as learning how to perform procedures such as taking blood pressure or manual muscle testing [4]. A recent systematic review suggested that education using IVR has a significant but small to moderate edge over regular teaching methods, such as watching 2D videos, reading written material or watching presentations, in terms of learning content, and compelling evidence of the great advantage of IVR for important non-cognitive learning factors such as engagement, motivation and affect [5]. Because IVR is immersive and the user feels present in the environment, it is an ideal method to teach skills in ecologically valid simulations: users learning operating skills will feel like they are in the operating room, nurses can learn to take blood pressure while surrounded by a portrayal of a chaotic emergency room. Realistic simulations are considered one of the best ways for individuals to learn because they are methods for learners to experience situations that are challenging, emotionally charged and allow for mistakes and errors that have no true consequences, such as injuring or alienating a patient, thus allowing learners to trial different responses and reflect on their outcomes [6]. Thus, IVR has the potential to be used to train skills in a naturalistic context and provide alternative methods for learning that may better match different learning styles, key aspects of CBE.

One of the challenges of IVR is creating the virtual learning environments. IVR creation can be divided into two general categories: programmed 3D animated graphics and passive Spherical Video-based Virtual Reality (SVVR) which are real environments captured and rendered as 3D videos. The programmed 3D animated graphics are complex computer-generated graphic environments. The environments are interactive and programmed to react to users’ movements and actions, often through handheld controllers, thus providing feedback on performance. While potentially highly interactive, these IVRs can be cartoonish and have low quality images because of the expense and expertise required to program both the environment and interactions. While the ability to influence and interact with a simulation is a positive aspect of learning [4], the cost and expertise needed to develop high quality interactive IVR may place them out of the reach of educators [7], particularly if they have specific pedagogical goals for the simulation.

SVVRs provide a highly realistic experience of a physical environment, outperforming programmed 3D generated IVR for context and ecological validity. While less interactive than a programmed 3D IVRs SVVRs allow users to choose what area of the environment they want to look at allowing a level of active exploration not available in traditional 2D videos and allow repeated performance of tasks unlike live interactions [8]. Unlike programmed 3D IVRs, SVVRs are low cost and easy to produce, and can be created with easily accessible 360 cameras [8]. Research suggests that SVVR videos are well suited to observational learning, immersive and interactive theory, and teaching, and facilitating self and peer reflection [9]. SVVRs have also been used successfully to teach performance skills [10, 11].

One advantage of either of these IVRs is the possibility for embodiment. Embodiment refers to a subjective experience where a simulated body and its properties are experienced as if it was the user’s own biological body [12]. If an IVR is rendered from the first-person point of view, such that a user perceives that limbs are where their biological limbs would be located, that user will take mental ownership of the limbs and embody into them [12]. The phenomenon of embodiment has been documented repeatedly, most notably in the rubber hand illusion [12]. Individuals perceive that what happens to their embodied, virtual limbs is happening to them [13]. There is evidence that IVR supported embodied learning where the students experience the performance as if performed from their point of view improves performance of skills and confidence [10, 14,15,16,17,18,19,20]. Interestingly, while having agency (i.e., the ability to actively move the virtual limb) would appear to be important for learning performance skills, evidence suggests that with a high level of immersion, the need for agency is less important for embodied learning [21]. This suggests that a highly immersive SVVR video in which students feel embodied may be appropriate for learning performance skills.

Goniometry, the measurement of joint ranges, is a necessary assessment skill for occupational therapy (OT) students. It is a challenging psychomotor skill combining both accurate knowledge of kinesiology and the ability to precisely measure movement. In this proof-of-concept study we describe how we developed and tested a series of SVVRs designed to teach OT students goniometry for the wrist and forearm.

The purpose of this study was to develop and examine the usability and establish preliminary efficacy of an IVR embodiment augmented training program, the Embodied Goniometric Occupational Therapy Training Program (EGOTTP), as a means to teach OT students goniometry.

Methodology

Design

This proof-of-concept study employed an observational, pre-post design. Ethical approval for the research was provided by the Federation University Human Research Ethics Committee (2023 − 143), in accordance with the Declaration of Helsinki. All participants signed an ethics committee approved consent form.

Participants

OT students and nursing faculty at Federation University in Victoria Australia who had limited experience with goniometry, were invited to participate in the study via a flyer. First year OT Students who had not yet learned goniometry, and students in their final year who had missed hands-on goniometry training during 12 months of lockdown for the COVID-19 pandemic, were invited, totaling 37 students. Given this small potential pool of participants, nursing faculty were also invited as they are conveniently co-located on campus and have not learned goniometry. Participants were excluded if they had any conditions that would make it unsafe for them to use IVR (seizure disorder or other symptom linked to an epileptic condition in the last five years; implanted medical device such as a pacemaker, defibrillator, or hearing aid; frequent migraines or earaches; cardiac conditions; contagious disorder on the face such as conjunctivitis that could be transmitted).

Embodied goniometric occupational therapy training programs (EGOTTP)

We created the EGOTTP, a SVVR program, by filming a skilled, medium skin toned occupational therapist with an Insta360 One X2 (Insta360.com) 360-degree camera while she was performing goniometric measurements on a simulated patient. We developed three EGOTTPs one each for wrist flexion/extension, wrist ulnar/radial deviation and forearm supination pronation. Each EGOTTP was filmed from the first-person point of view so that participants appeared to be looking at their own arms performing the goniometry measurements (See Fig. 1). Each of these three EGOTTP had three sections of the goniometric measurements. The first section had a detailed voice over which provided the learner with clinical reasoning cues for each step of the goniometry process as though the participant was thinking through each step. This section and all other sections also included instructions to the simulated patient on how to position and move their arm. The clinical reasoning included a rationale for each step of the goniometric process, tips on how to successfully complete the task, and detailed instructions. on how to: check active range of motion; position the joint; identify and palpate the axis of motion; place the goniometer axis, moveable arm and stationary arm; complete the measurement process; and read the goniometer. The second section had very limited clinical reasoning cues, most of which were procedural reminders. These included: to check for active range of motion; where to place the goniometer axis and arms, the measurement process, and the results of the measurement. The final section included no clinical reasoning cues, and only gave the instructions to the simulated patient. See Table 1 for more details of each section of the EGOTTP.

[IMAGE OMITTED: SEE PDF]

When constructing the EGOTTP we included several elements to enhance the sense of embodiment [22] (See Table 1). We marked where the virtual hands in the EGOTTP would situate on the real table and placed the participant’s hands on those spots to develop a sense of co-location. At the beginning of each EGOTTP participants participated in a short viso-tactile stimulation sequence, a maneuver typically done to embody limbs [12, 13, 22]. This stimulation consisted of stroking the dorsal side of participants arms from elbow to fingertip in sync with the same movement in the EGOTTP. Thus, participants felt their real arms stroked and saw the virtual arms stroked at the same time. Multisensory stimulation enhances a sense of embodiment so we included an olfactory component by stroking the arms with scented hand sanitizer [22,23,24]. To further strengthen the sense of embodiment and presence in the system, participants were provided with actual goniometers to hold during the program which they were told they could move in response to what they saw in the EGOTTP if they wanted to. There was felt on the table in both the simulated and real environments, providing tactile congruity. Finally, we completed all EGOTTP testing in the same room and same configuration that we filmed the SVVR EGOTTP.

Equipment

All EGOTTP videos were viewed using Meta Quest 2 virtual reality system (meta.com). The Meta Quest 2 is a wireless, standalone device that uses a head mounted display (HMD) to project a simulated 360-degree environment from a first-person point of view. Through the HMD, participants are immersed into the environment such that they experience the simulated world as the real world. The hand controls for the system were not used for project as participants did not actually interact with the virtual world.

Instruments

In addition to providing simple demographic information (age, sex, experience with video games and IVR), participants completed one scale to measure efficacy pre and post using the EGOTTP, and five scales to measure usability after using the EGOTTP. All usability questionnaires have demonstrated good psychometrics, as presented in the related papers cited below.

Usability

User Engagement Scale (UES): The UES is a 12-item Likert questionnaire that assesses participant engagement with digital technology [23]. In this 5-point scale, anchored on Strongly Disagree to Strongly Agree, participants’ rate three items for each section: how well the system focuses attention (FA), is usable (PU), is aesthetically pleasing (AE) and is rewarding to use (RW). Total section score is the average of each section’s items and total score is the average of all twelve items.

System Usability Scale (SUS): The SUS is a 10-item Likert questionnaire scaled from 1 (strongly disagree) to 5 (strongly agree) [24]. It assesses participant perception of the usability of a system. The total sum of the raw score is processed to achieve a score between 0 and 100 and this score is matched to normed score to “grade” the overall usability of the system from “F” (score 0 to 60) to “A” (score 91–100).

NASA Task Load Index (NASA TLX): The NASA TLX provides participants’ subjective estimate of mental workload in six dimensions: mental demands, physical demands, temporal demands, effort, performance, and frustration level [25]. Each dimension is identified by a single question which asks participants to rate the workload from 1 (very low) to 20 (very high) except the performance dimension which anchors the scale on success (1) to failure (20). Typically, this questionnaire is weighted by having participants select which dimensions most contribute to the workload of the task. However, for this study we used the raw 1–20 rating score.

Peck’s Embodiment Scale (PES): The PES measures the extent to which participants achieved ownership of and embodied the virtual limb [26]. It is a 16-item Likert questionnaire with a 7-point scale (1 never to 7 always). The questionnaire is scored as four subscales consisting of appearance, response, ownership, and multi-sensory which are determined by calculating the mean of the items. The total instrument embodiment score is determined by calculating the mean of the sum of the four subscales.

IPG Presence Questionnaire (IPQ): The IPQ estimates the participants experience of being present in the virtual environment [27]. It is a 14-item, 7-point Likert scale which has various anchors to questions. The IPQ has three subscales, spatial presence, involvement, and experience realism which are determined by calculating the mean of the appropriate items and a total presence score determined by calculating the mean of the sum of the three sub scores.

Efficacy

Self-Confidence Questionnaire (SCQ): The SCQ measured participant self-confidence before and after completing the EGOTT programs using the 8-item Self-Confidence Questionnaire adapted from a study specific questionnaire proposed by Paige and colleagues [22]. This instrument was Likert scaled from 1 (not at all) to 5 (completely) and participants self-rated their ability to perform goniometry. The eighth question, which asked participants to think about using the EGOTTP and how well they prepared novice occupational therapists to perform goniometry in comparison to others who had not received the training, was only asked post EGOTT training.

Procedures

Prior to engaging in the EGOTTPs, participants completed the demographics questionnaire and the SCQ. They were given an information sheet that described the basics of goniometry to ensure that all participants had general theoretical knowledge about the process. Participants were provided with instruction on the EGOTTPs and the IVR system. The participants were instructed to try to feel as if the virtual occupational therapist’s hands were their own hands and to try to feel like they were interacting with simulated world as they viewed the techniques. They were also instructed that this was an active leaning experience and that they should try to mentally synchronize their thoughts with the virtual scenario, so that their thoughts and actions aligned with the virtual experience. All the EGOTTPs were presented to participants in the same order: radial/ulnar deviation, wrist flexion extension, forearm pronation/supination. Each program ran ~ 13 min and all three repetitions were completed for each measurement. Participants were given a short break between each EGOTTP in which they removed the HMD. After completing all three EGOTTPs, participants completed the efficacy and usability questionnaires.

Data analysis

Data were collected using REDcaptm and analyzed using SPSS (v. 28). Prior to analysis, data were cleaned, and subscale and total scores were calculated for each measure. Given our primary aim was to determine if the design of the EGOTTP was usable, analyses were primarily descriptive using the data from the questionnaires to graph and visually identify if the EGOTTP was both usable and achieved some level of embodiment/presence. To assess efficacy, we compared participants confidence in their ability to perform goniometer measures pre and post EGOTTP using paired t-tests. We also calculated the effect size using Cohen’s D with the Hedge correction given the small sample size.

Results

Four entry level OT students (one in first year and three in final year) and a novice nurse laboratory technician participated in this study. None had experience in goniometry. The average age of participants was 35.4 years (SD 10.2). Three of the five had participated in a 2-hr introduction to goniometry during COVID-19 pandemic with no practical skill component, two years prior to the study, and had not used the technique in clinical practice. All participants were fair to medium in skin tone. Two participants had used an IVR system previously.

Usability of the EGOTT

Figures 2, 3 and 4 display the participant and study level scores on the UES, SUS and NASA TLX. Overall participants reported high levels of engagement (UES) (Fig. 2) and visually there was similarity between participants in their ratings on the instruments. They indicated that the most taxing aspect of the EGOTTP was the mental workload (Fig. 3). Opinions varied on other measures of workload, but in general these were low. In particular, participants reported very little frustration. The overall grade for the EGOTTP on the SUS was “A” with 4 out of 5 participants score indicating this grade. In general people rated all usability scores at 4 or greater (Fig. 4).

[IMAGE OMITTED: SEE PDF]

[IMAGE OMITTED: SEE PDF]

[IMAGE OMITTED: SEE PDF]

The PES total score was 4.8 suggesting a better average frequency of experiencing aspects of embodiment during the EGOTTP (Fig. 5). Experiencing the EGOTTP as multisensory and having a sense of ownership of the limb were scored highly, a sense of responsiveness was lowest overall. Participants also indicated an average presence in IPQ with a total score of 4.8 (Fig. 6). Participants generally reported feeling spatially present and they had a sense of realism.

[IMAGE OMITTED: SEE PDF]

[IMAGE OMITTED: SEE PDF]

Efficacy of the EGOTTP

Participants experienced a very large, significant improvement in confidence for every item as well as the total of the SCQ except for how energized they were to perform goniometry (Fig. 7). The largest effects were seen for how vivid images of goniometry were in the participants mind (Cohen’s D = 4.4); the participants perception of how well they thought they could perform goniometric measures relative to others at their stage (Cohen’s D = 2.4); and how easily they could “see” themselves performing goniometry (Cohen’s D = 2.3). Participants reported that the EGOTTP was helpful in preparing them to perform goniometry compared to others who had not received the training (M = 4.6, SD = 4.8).

[IMAGE OMITTED: SEE PDF]

Discussion

Despite a relatively small sample of participants, this study was able to examine the usability and aspects of the efficacy of an immersive virtual reality embodiment augmented training program for goniometry, called the EGOTTP, as a means of teaching this skill to OT students. Although there have been many advances in delivering the skill-based components of OT education such as the use of self-paced 3-D computer programs in anatomy and physiology (e.g. Anatomage table), and simulation environments to train interviewing skills, there is relatively scant literature on teaching basic manual skills in virtual environments. This research presents one of the first attempts to determine the useability and efficacy of IVR embodiment augmented training. Our research supports the results of previous research using IVR to create a first person POV confidence [10, 14,15,16,17,18,19,20]. Our participants reported that they felt significantly more confident with their goniometry skills after participating in the EGOTTP. Further, our SVVR videos, as noted in other studies [10, 11]., were perceived as ecologically valid and engaging methods to take part in the training. Overall, this proof-of-concept study suggests that further research to develop these technologies is warranted, and that students believe this approach can help them to develop manual therapy skills, as part of a holistic education program.

Usability of the EGOTTP

Overall, as we anticipated, participants in the study rated the usability of the EGOTTP as very high. Participant scores on the UES, SUS and NASA TLX all indicated that they were able to immerse well, felt spatially present and were able to engage well in the task. Of the six wrist goniometry measures (flexion/extension, radial/ulnar deviation, and pronation/supination), we learned that filming the scenarios with enough space for the participant to move was important. For example, the filming angles for pronation/supination required the simulated patient and occupational therapist in the video to be quite close (and large) in the environment and the participants all commented that this had felt uncomfortable. In future care should be taken to film with enough space for participants to move towards, and away from the simulated patient in the VR. Anecdotally, participants also commented that holding and moving the goniometer, thus activating the sixth sense of kinesthesia, was also very valuable to maintain realism in the experience. Activating the motor circuits and creating a motor memory for the moments of goniometry is vital for students to achieve to practice goniometry effectively and is one of the main advantages of using an EGOTTP.

Efficacy of the EGOTT

Again, as expected, the participants’ pre-post questionnaire results on the SCQ indicated they experienced significant improvement in confidence in performing goniometry. Confidence in ability, also known as self-efficacy has demonstrated moderate positive associations with performance ability, which is why we used this as an indicator of efficacy for this proof of concept study [28]. The large increases confidence across all items supports our hypothesis that embodied goniometry can improve student performance. However, future studies should assess actual performance before and after EGOTTP to determine if the participants confidence translates into actual improvements in performance.

Implications for OT education

Occupational therapy practitioners often need to perform evaluation and treatment that require complex psychomotor skills combined with strong clinical reasoning [29]. The CBE model of teaching focuses on ensuring that OT students achieve competence in these psychomotor skills. Bloom’s taxonomy provides one way to organize and describe competence in psychomotor learning. In the taxonomy, psychometric learning consists of seven increasingly complex phases; perception, set, guided response, mechanism, complex overt response, adaptation, and origination [30]. These first three phases, perception, in which the learner observes and develops a basic awareness of the psychomotor task, set, in which the learner develops the mindset that allows them to act, and guided response, in which the learner begins trialing the physical skill, can be addressed through SVVR training. In the EGOTTP, an OT student can observe the goniometric process and hear a master clinician describe how to perform it (perception and set). It also provides OT students with the opportunity to do an initial “practice” of the skill through embodiment and mental rehearsal (guided response). Once the OT student has practiced the skill using the EGOTTP, they work on developing the skills into habitual responses (mechanism), thus achieving basic competence of the skill. As the OT student starts fieldwork and subsequently enters practice they can continue to refine and strengthen their goniometry skills. Competence in other OT psychomotor skills, such as manual muscle testing and upper limb splinting could also be taught using IVR. Thus, IVR and SVVR experiences hold great promise as one tool to promote CBE.

Limitations and areas for further research

This study included only five participants and was a proof-of-concept study which only focused on determining the useability and elements of the efficacy of the EGOTTP. As noted above, further research should be undertaken to determine how IVR experiences can be incorporated into a holistic education program in training technical skills such as goniometry, and with larger sample sizes. Research is also required to identify which skills might lend themselves to IVR training. For example, splinting, manual muscle testing, and sensory testing may all translate well to skill acquisition in a IVR environment, while learning to administer cognitive and perceptual assessments to patients may not. Research is also required to more comprehensively determine if the use of a EGOTTP alone, or in combination with traditional educational methods is superior for training goniometry skills compared to regular class-based education, through conducting a randomized controlled trial on the topic. If IVR is used as part of a comprehensive program, then the ordering of the materials also becomes important, for example, should students read through a text on the technique, try it face to face in a tutorial-style learning environment, and then rehearse using IVR, or the reverse? These elements could be included as aims of a larger RCT. Finally, as noted above, while we examined the confidence aspect of efficacy, we did not formally measure participant practical skill in goniometry pre and post participation in an EGOTTP. We recommend that a simple rating scale be developed for future research so that participant accuracy in undertaking each element of goniometry of the wrist can be recorded.

Conclusion

With the increased interest in CBE, educators seek creative approaches to engage learners and promote skill acquisition. The use of IVR and embodied experiences to educate students is an ecologically valid and attractive approach to train students in specific skill acquisition. This proof-of-concept study demonstrates some positive results and provides initial support for the use of IVR programs with embodied learning experiences as adjunct to traditional educational methods in manual skill training such as goniometry. While this study suggests that participants did engage in this learning approach and believe it supports skill acquisition, further research is required to support its use. Future research needs to establish how, where and when an approach like the EGOTTP should be used, and to empirically test if the use of IVR as part of, or in place of, traditional learning approaches is effective for students to gain skills.

[IMAGE OMITTED: SEE PDF]

Data availability

No datasets were generated or analysed during the current study.

References

Thibault GE. The future of health professions education: emerging trends in the United States. FASEB BioAdvances. 2020;2(12):685–94.

Hamed R, Banks TM, Mahoney D, Simon P, Timmerberg JF, Nilsen DM. A call to shift to competency-based education. Am J Occup Ther. 2023;77(6):1–4.

Frank JR, Snell LS, Cate OT, Holmboe ES, Carraccio C, Swing SR, et al. Competency-based medical education: Theory to practice. Med Teach. 2010;32(8):638–45.

Radianti J, Majchrzak TA, Fromm J, Wohlgenannt I. A systematic review of immersive virtual reality applications for higher education: design elements, lessons learned, and research agenda. Comput Educ. 2020;147:103778.

Wu B, Yu X, Gu X. Effectiveness of immersive virtual reality using head-mounted displays on learning performance: a meta‐analysis. Br J Edu Technol. 2020;51(6):1991–2005.

Zigmont JJ, Kappus LJ, Sudikoff SN. Theoretical foundations of learning through simulation. Semin Perinatol. 2011;35(2):47–51.

Huang H, Hwang G-J, Jong MS-Y. Technological solutions for promoting employees’ knowledge levels and practical skills: an SVVR-based blended learning approach for professional training. Comput Educ. 2022;189:104593.

Wu B, Chang X, Hu Y. A meta-analysis of the effects of spherical video-based virtual reality on cognitive and non-cognitive learning outcomes. Interact Learn Environ. 2023:1–18.

Rosendahl P, Wagner I. 360° videos in education – a systematic literature review on application areas and future potentials. Educ Inform Technol. 2024;29(2):1319–55.

Chang C-Y, Sung H-Y, Guo J-L, Chang B-Y, Kuo F-R. Effects of spherical video-based virtual reality on nursing students’ learning performance in childbirth education training. Interact Learn Environ. 2022;30(3):400–16.

Huang H, Hwang G-J, Chang S-C. Facilitating decision making in authentic contexts: an SVVR-based experiential flipped learning approach for professional training. Interact Learn Environ. 2023;31(8):5219–35.

Kilteni K, Groten R, Slater M. The Sense of Embodiment in Virtual Reality. Presence: Teleoperators & Virtual Environments. 2012;21(4):373 – 87.

Danry V, Pataranutaporn P, Mueller F, Maes P, Leigh S-w, editors. On Eliciting a sense of self when integrating with computers. Proceedings of the Augmented Humans International Conference 2022.2022:68–81.

Yu S, Liu Q, Johnson-Glenberg MC, Han M, Ma J, Ba S, et al. Promoting musical instrument learning in virtual reality environment: effects of embodiment and visual cues. Comput Educ. 2023;198:104764.

Gabbott B, Tennent D, Snelgrove H. Effect of mental rehearsal on team performance and non-technical skills in surgical teams: systematic review. BJS open. 2020;4(6):1062–71.

Yang C-H, Liu S-F, Lin C-Y, Liu C-F. Immersive virtual reality-based cardiopulmonary resuscitation interactive learning support system. IEEE Access. 2020;8:120870–80.

Choi J, Thompson CE, Choi J, Waddill CB, Choi S. Effectiveness of immersive virtual reality in nursing education: systematic review. Nurse Educ. 2022;47(3):E57–61.

Ros M, Neuwirth LS, Ng S, Debien B, Molinari N, Gatto F, et al. The effects of an immersive virtual reality application in first person point-of-view (IVRA-FPV) on the learning and generalized performance of a lumbar puncture medical procedure. Educational Technol Res Dev. 2021;69(3):1529–56.

Omodei MM, McLennan J. Studying complex decision making in natural settings: using head-mounted video camera to study. Percept Motor Skills. 1994;79(3):1411–25.

Unsworth CA. Using a head-mounted video camera to explore current conceptualizations of clinical reasoning in occupational therapy. Am J Occup Ther. 2005;59(1):31–40.

Petersen GB, Petkakis G, Makransky G. A study of how immersion and interactivity drive VR learning. Comput Educ. 2022;179:104429.

Paige JT, Yu Q, Hunt JP, Marr AB, Stuke LE. Thinking it through: mental rehearsal and performance on 2 types of laparoscopic cholecystectomy simulators. J Surg Educ. 2015;72(4):740–8.

O’Brien HL, Cairns P, Hall M. A practical approach to measuring user engagement with the refined user engagement scale (UES) and new UES short form. Int J Hum Comput Stud. 2018;112:28–39.

Brooke J. SUS: a retrospective. J Usability Stud. 2013;8(2):29–40.

Hart SG. Nasa-Task load index (NASA-TLX) 20 years later. Proc Hum Factors Ergon Soc Annual Meeting. 2006;50(9):904–8.

Peck TC, Gonzalez-Franco M. Avatar Embodiment. A standardized questionnaire. Front Virtual Real. 2021;1.

Schubert TW. The sense of presence in virtual environments: a three-component scale measuring spatial presence, involvement, and realness. Z für Medienpsychologie. 2003;15(2):69–71.

Lochbaum M, Sisneros C, Cooper S, Terry PC. Pre-event self-efficacy and sports performance: a systematic review with meta-analysis. Sports. 2023;11(11):1–22.

American Occupational Therapy Association. Occupational therapy practice framework: domain and process. Am J Occup Ther. 2020;74:7412410010.

Ruhl C. Bloom’s Taxonomy of learning 2024 [ https://www.simplypsychology.org/blooms-taxonomy.html