Exploring the Efficacy of Mixed Reality versus Traditional Methods in Higher Education: A Comparative Study

Abstract

“Immersive technology” is a broad and evolving term that encompasses various kinds of technologies and viewpoints, and has applications in various fields, such as education, healthcare, entertainment, the arts, and engineering. This research paper aims to compare the effectiveness of two teaching methods, namely the conventional method (using PowerPoint slides), and the immersive technology method (initiating a mixed reality with a HoloLens 2 device). The experiment involved two groups of students, aged between 19 and 52 years-of-age, who received different types of instruction: the first group viewed a PowerPoint slide with an image of the human muscular system, and the second group viewed a 3D hologram of the human body that displayed the same muscle groups as in the PowerPoint slide. The researchers wanted to examine if mixed reality devices could improve students’ cognitive abilities and explore if the age of the participants had any impact on the effectiveness of the instruction method. The main findings of this study are that the instruction method that used mixed reality technology, the Microsoft HoloLens 2 device, was more suitable for younger participants, and that traditional instruction methods, such as PowerPoint slides, are more appropriate for older students. While this research provides some valuable insights into the factors that influence student performance in anatomy tests, it has some limitations that should be considered.

1. Introduction

Immersive technology is a dynamic and emerging field that utilizes digital media to create authentic and interactive learning environments. Virtual reality (VR), augmented reality (AR), mixed reality (MR), and movie 360 are among the various technologies that fall under the umbrella of immersive technology, whose potential to enhance learning outcomes has been widely documented [1]. Immersive technology, a cross-disciplinary field, has been applied in areas like healthcare, education, and crisis management. It has also been used to develop engaging and lifelike situations that enhance learners’ cognitive, emotional, and behavioural abilities [2,3,4,5], and has been applied in a variety of sectors, including entertainment, the arts, and engineering [6,7,8,9].

While immersive technology has the potential to enhance learning outcomes and create engaging and immersive experiences for learners, it also has some drawbacks and limitations, such as technical issues and accessibility barriers, and also raises a number of ethical concerns [10,11,12].

This paper focuses on mixed reality (MR) technologies and considers their effects on learning and memory by comparing them against to traditional teaching methods, such as PowerPoint presentations. MR technologies create immersive environments that blend the real and virtual worlds, and can be experienced through devices such as the HoloLens. The paper also examines the influence of participants’ age on their learning outcomes by applying these different methods. This paper explores the potential of MR technologies in education, and thereby contributes to an ongoing discussion of how immersive technology can be effectively utilized in a way that enhances learning outcomes and improves educational practices.

1.1. Immersive Technologies in Education

“Immersive technology” is a term that describes the combination of virtual elements and the real world on terms that enable the user to interact naturally with both. The concept of immersive technology can be traced back to the early 1960s, when Sutherland [13] created the “Human Machine Graphic Communication System”, the first immersive prototype of human-computer interaction. Since then, many kinds of immersive technologies have been developed, including VR, AR, RM, holography, telepresence, digital twins, and FPV drone flight [1].

“Immersive technology” is a term that means different things to different researchers. Some researchers, like Slater [14], focus on the amount of sensory information delivered by the technology (to users), along with its level of success in doing so, and conclude that immersive technology can give users a large amount of high-quality sensory information. Other researchers [15] focus on how the technology can make users feel like they are in a different world, and conclude that immersive technology can blur the boundary between reality and virtuality, creating a feeling of immersion. A third group of researchers [16] consider both aspects of sensory information and immersion, and conclude that immersive technology is creating a realistic digital environment, which users feel they inhabit and interact with.

Immersive technologies can create realistic and engaging learning environments that support effective pedagogies and learning outcomes [1,8,17], and can foster constructivist and experiential learning, where learners construct their own knowledge, practice skills, develop creativity, and comprehend abstract concepts [18]. The Association for Medical Education in Europe argues that “Projects for effective medical e-learning must reflect the dynamics and details of real-world practice, as well as provide effective learning opportunities” [19,20], which suggests that immersive technologies can offer valuable benefits for medical education and training that can be transferred to other fields that require practical and problem-solving abilities [1,21].

Immersive technologies, such as AR, VR, and MR can enhance student learning outcomes in various educational contexts and broaden learning environments beyond primary and higher education settings by overcoming limitations in physical space, fostering collaboration and experiential learning, and offering personalised learning approaches that can support students, regardless of their knowledge level [22].

Immersive technologies has been increasingly adopted in education, and especially in health and science teaching [1,7,21,23], where they These enable students to explore complex topics and scenarios in a realistic and interactive way and, to a greater extent than conventional methods, improve their learning outcomes.

Hamilton et al. [6] reviewed 29 studies that compared immersive VR with head- mounted display and other teaching methods, and found that immersive VR usually improved learning outcomes, in opposition to some studies that did not show any difference, and two studies, which showed immersive VR had negative effects [24,25]. The review also pointed out some problems with current research, such as short duration, no memory tests, and poor evaluation methods, and also raised the focus on science as an issue [26,27].

Ryan et al. [21] analysed 29 trials (N = 2722 students) that compared the application traditional and immersive technologies (VR, AR, or MR) in medical and nursing education, and found that, while the knowledge of both groups was the same, immersive technologies produced a better learning experience. They also reported that immersive technologies increased student satisfaction, self-efficacy, and engagement, suggesting that they are ideal for education.

The effectiveness of VR/AR/MR in different domains and contexts is still being investigated, and so are the factors that influence their design and implementation. Medical education is one of the domains that has shown great potential for VR/AR/MR applications, as students have benefitted from realistic and safe simulations of anatomical structures, physiological processes, and clinical scenarios. For example, Odame and Tümler [28] found that using off-the-shelf VR software (namely Sharecare YOU VR) to teach the anatomy of the human heart was more effective and satisfying than a conventional teaching method. Similarly, Banjar et al. [29] reviewed 12 experimental studies of the use of mixed reality in higher education, and found that most focused on 3D manipulation, visualization, and understanding of 3D object layers and components, especially in the medical and health sciences. The use of VR/AR/MR also presents opportunities and some challenges for instructional design, including the fact these technologies may impose different cognitive demands and preferences on learners and instructors. Therefore, it is important to consider the principles and theories of human cognition and learning that can guide the development and evaluation of VR/AR/MR interventions. One of the most influential theories in this regard is cognitive load theory (CLT), which is based on the assumption that human cognitive architecture consists of a limited capacity working memory and an unlimited capacity long-term memory, and which also holds that instructional methods should reduce the unimportant cognitive load and facilitate the intrinsic and relevant cognitive load associated with learning complex information [30,31]. Another relevant theory is the learning style theory, which suggests that learners have different preferences in relation to different approaches to instruction, and that instructors should balance their teaching methods to accommodate the diversity of learning styles in their classes [32,33]. However, both CLT and learning style theory have received some criticisms, in the educational and psychological literature, and have also given rise to a number of controversies, so their applicability to, and validity for, VR/AR/MR contexts needs further investigation. Dowling et al. [34] performed empirical studies that examined the effects of VR/AR/MR on learning outcomes and cognitive load, along with the moderating role of learning styles and other variables.

Another domain that has been influenced by the use of VR/AR/MR is adult education, where students can access flexible and self-directed learning opportunities that suit their needs and preferences. However, the impact of different Web-based delivery formats, such as hybrid and online, on adult students’ self-directed learning readiness (SDLR) and affective learning outcomes, such as motivation, satisfaction, and attitude, is still unclear. Nikitenko [35] compared the hybrid and online groups of 563 students enrolled on courses at the University of San Francisco and found no significant differences or relationships between the delivery formats, SDLR, affective learning outcomes, prior e-learning experience, and age, before concluding that SDLR and related programming are important for enhancing adult students’ learning experiences and outcomes in Web-enhanced settings.

1.2. Network Visualization of the Literature on Augmented and Mixed Reality in Education

In order to provide a comprehensive and visual overview of the current state of research of the use of augmented and mixed reality in education, we utilized the VOSviewer 1.6.20 software tool [36]. By using this tool, we were able to create a network visualization that offers a focused overview of the key subjects and trends in the study of augmented mixed reality in education.

The visualizations are based on the keywords that most frequently occur in the literature. A threshold, which was determined on the basis of bibliographic data extracted from the Web of Science (WoS) and Scopus databases, needed to be met for a keyword to be included in the visualization. By setting this threshold, we were able to concentrate the visualization on the most frequently occurring, and potentially most significant, terms in the field of augmented mixed reality in education. This effectively filtered out less frequently-used terms, making the visualization less cluttered and more interpretable.

By using VOSviewer, we were able to generate network visualizations based on bibliographic data from the WoS (Figure 1a) and Scopus (Figure 1b) databases. The keywords used for these visualizations were “mixed reality”, “augmented reality”, and “education”. Our search yields of the SCOPUS database yielded 1068 documents, and the search of the WoS database yielded 1530 documents. Documents were then filtered so that only those with selected keywords in the title, abstract, or keywords are left.

The resulting visualizations offer a graphical representation of relationships and connections between keywords in the literature. Figure 1a, b presents a keyword cloud generated with VOSviewer, representing the interconnectedness and frequency of various keywords associated with augmented and mixed reality in education.

The network cloud provides an overview of the current trends, focus areas, and terminologies that are prevalent in the field of augmented and mixed reality in education. Prominent terms like “augmented reality” and “education”, which are central to the image, are in larger font sizes, indicating their frequency in, or significance to, the field. These central terms are surrounded by other related keywords, such as “engineering education”, “learning environments”, “human-computer interaction”, and “extended reality”, which clearly show the extensive range of topics associated with augmented and mixed reality in education.

The lines connecting some of the circles highlight relationships or connections between specific terms, illustrating how these concepts are interrelated within the wider context of augmented and mixed reality in education. This image visually summarises the current state of the field, and also shows key themes related to augmented and mixed reality in education. These images provide an overview of the main topics, and also show their interconnections in the wider field.

In addition, we analysed the relationships and connections between various terms related to augmented and mixed reality and the use of PowerPoint in education by using the SCOPUS database (Figure 2). After referring to the WoS database, we could only identify one article [37] that included all the selected keywords, which was insufficient to generate a network visualization.

Here are some key observations about the network cloud obtained from the SCOPUS database: The terms “augmented reality”, “virtual reality”, “e-learning”, and “education” are prominent, indicating they are key areas of focus for the research of augmented mixed reality in education.

Lines in different colours connect these terms, indicating various types of relationships or levels of connection.

Some terms, like “immersive virtual reality” and “safety training”, are connected but positioned on the periphery, possibly indicating specialized or niche areas within the broader context.

The term “students” is connected to “curricula”, “learning systems”, and “computer- aided instruction”, highlighting a focus on educational content.

One limitation of the scientific literature that compares mixed reality (MR) to traditional PowerPoint methods to assess their influence is the scarcity of attention given to the age factor, with the result that it is infrequently acknowledged as a significant variable. In our review of the existing research, we could not identify any studies that include age as a parameter in their assessment of the efficacy of these instructional approaches.

One factor that may affect the adoption, and effectiveness, of technology enhanced learning (TEL) in higher education is the age of the students, which may influence their use of, and attitudes towards, technology, as well as their learning outcomes and cognitive load. However, studies of the relationship between TEL and age are scarce and inconclusive, and the ones that do exist use different definitions, measures, and methods. Some studies have found that, while mature students (usually defined as those over 21 years-of-age) use fewer technologies less frequently than younger students, they tend to have similar, or even slightly more positive, attitudes towards technology [38,39]. Other studies found that age is not a significant predictor of technology use or attitudes, and identified other variables, such as computer experience, self-efficacy, and perceived usefulness, to be more important [40]. Moreover, some studies have suggested that age may interact with other factors, such as the type and quality of the TEL intervention, the domain and level of the course, the assessment method, and prior knowledge and experience of students, to influence students’ learning outcomes and cognitive load. For example, Ref. [41] found that younger students performed better and reported lower cognitive load than older students when using a smart classroom system that supported a novel learning flow; and that students performed better and reported lower cognitive load when using a conventional classroom system. These findings indicate that age is a complex and multifaceted variable that needs to be considered in the design, implementation, and evaluation of TEL interventions in higher education.

Oversight of this is a critical gap, given the diverse age groups present in higher educational settings. In recognizing this, we incorporate age as a key variable in the investigation, and suggest that it may play an important role in determining the effectiveness of instructional methods, particularly in the context of immersive technologies like MR. Learners across different age brackets possess distinct cognitive abilities, preferences, and learning styles, and it is essential to understand how age interacts with the choice of instructional methods when designing tailored and effective educational interventions.

By acknowledging and addressing this gap, our research does not not only contribute to the existing knowledge about the comparative effectiveness of MR and PowerPoint but also argues that the age-related nuances of these methodologies should be explored in more depth and detail.

The following sections present our method, findings, and analysis, and explore the complex interplay of age, instruction methods, and learning outcomes. Through this approach, we aim to provide an overview of the potential benefits and drawbacks of using immersive technologies in various educational contexts.

2. Materials and Methods

This study employed a quasi-experimental design to compare the effectiveness of mixed reality (MR) and PowerPoint (PPT) presentations, when used as teaching methods in the teaching of the human muscular system. The study also explored the moderating role of participant IQ in the learning method effects.

After analysing the state of research, we were able to formulate the following research question: How does the use of mixed reality technology (a HoloLens 2 device) affect the learning outcomes and cognitive load of students from different age groups, when compared to the use of PowerPoint slides? We consequently adopted the following hypothesis: The use of mixed reality technology (a HoloLens 2 device) will result in better learning outcomes than the use of PowerPoint slides.

Following the confirmation of the research question and the hypothesis, we defined the objectives of this study:

• Compare the effectiveness of two teaching methods, the conventional method (using PowerPoint slides) and the immersive technology method (using mixed reality and a HoloLens 2 device), in relation to learning outcomes and student cognitive load.

• Investigate the moderating role of age in the teaching methods—learning outcomes—student cognitive load relationship.

• Identify the implications and provide recommendations that will assist and guide the design and implementation of immersive technology interventions in education and training.

2.1. Participants

A total of 115 students, drawn from Lucian Blaga University of Sibiu’s Faculty of Engineering and Faculty of Social and Human Sciences (in Subiu, Romania) and College Logos Centar (in Mostar, Bosnia and Herzegovina), participated in this study. The participants (43 females and 72 males) were aged between 19 and 52 years-of-age, with a mean age of 23.32 years-of-age. They were randomly assigned to either the MR group or the PPT group, with 50 and 65 students in each group, respectively, and the two groups were then matched, on their IQ and prior knowledge of the human muscular system, by a pre-test.

2.2. Materials

The MR group used Microsoft HoloLens 2 devices to study a holographic representation of the human muscular system. The HoloLens 2 is a head-mounted display that projects 3D images onto the user’s field of view, creating a mixed reality environment. The device has a resolution of 2048 × 1080 pixels per eye, a field of view of 52 degrees, and a refresh rate of 60 Hz. The device also has a built-in speaker, microphone, and camera, allowing the user to interact with the hologram by using voice, gesture, and gaze commands. The hologram used in this experiment was developed by Case Western Reserve University, as a free app called HoloAnatomy, and displayed both different colours for muscle groups and labels for the most important muscles, as shown in Figure 3.

The PPT group used a laptop computer and a projector to view a PowerPoint presentation on the human muscular system. The presentation consisted of 1 slide, containing the front and rear views of the human muscular system, and text pointing to the major muscles in the body. The presentation was designed so that it covered the same content, and provided the same level of detail, as the hologram.

The multiple-choice tests were chosen to measure the learning outcomes because they have several advantages in our research context. In this case, the main advantage of the multiple-choice tests is that they are easy to administer and evaluate, especially when a digital platform such as Google Forms is used. This made it possible to collect and analyse the participants’ responses by efficiently using simple statistics. Second, multiple choice tests can cover a wide range of content and be used to assess different levels of understanding, extending from the recognition of facts to the application of concepts [42,43]. Questions were designed to reflect the learning objectives of the research and to establish if the learning method that uses the HoloLens 2 device is more efficient than the traditional method that uses PowerPoint presentations when the human muscular system is studied.

In designing the content, goals, and other aspects of the different implementations, we drew on the guidelines and recommendations of previous studies that used MR devices (such as HoloLens) to enhance the learning outcomes of students in various domains. We also consulted with experts in the field of human anatomy and physiology to ensure the accuracy and relevance of the content, and to establish the goals of the learning material on the human muscular system. To ensure the even-handed treatment of the two research groups (students who studied the human muscular system by using the HoloLens 2 device, and students who studied this with a traditional PowerPoint presentation) to the greatest possible extent, we made sure that there were no significant differences between their IQ level and prior knowledge of the topic. The same learning objectives, content, and duration were also applied to both groups, who did not receive additional instruction nor guidance. The only difference was the mode of delivery, with one group receiving instruction through MR (using the HoloLens 2 device), and the other through PowerPoint.

2.3. Procedure

The experiment was conducted in accordance with Figure 4, and involved the following steps:

• The participants were welcomed, informed about the purpose and procedure of the study, and then asked to sign a consent form, fill out a demographic questionnaire and undertake an IQ test (based on Raven’s progressive matrices method).

• The participants were given a pre-test that assessed their prior knowledge of the human muscular system. The pre-test consisted of an interview, with questions about the basic anatomy, physiology, and relative positioning of the muscles.

• The participants were randomly assigned to either the MR group or the PPT group and then directed to separate rooms. They were instructed to self-study the material for 10 min by using either the HoloLens device or the laptop and projector.

• The participants were given a post-test to measure their learning outcomes. The post-test had a time limit of 15 min, included 20 multiple choice questions drawn from similar topics to the pre-test, and was administered online by using Google Forms.

At the end, the participants were thanked and debriefed, and were asked to provide feedback about their learning experience and state their satisfaction (or dissatisfaction) with the material and device.

2.4. Data Analysis

The data collected from the online questionnaires were exported to Microsoft Excel and Minitab 20 software for analysis. The primary objective of the data analysis was to compare the learning outcomes of the two groups by conducting a 20-question anatomy test. In addition, the study also sought to explore the potential moderating role of participant age in learning method effects.

The following statistical tests were performed to answer the research questions:

A t-test was used to compare the mean anatomy test scores of the two groups The t-test was a two-tailed, independent samples test, with a significance level of 0.05. The null hypothesis stated there would be no difference between the mean anatomy test scores of the two groups, while the alternative hypothesis claimed there would be a difference.

A multiple regression analysis was used to examine the relationship between the learning outcomes and the age of the participants. The multiple regression analysis was a linear, multivariate test, with a significance level of 0.05. The dependent variable was the learning outcome, and was measured by the anatomy test score. The independent variables were the learning methods (which were measured by a dummy variable that indicated whether the participant was in the MR group or the PPT group), and the age of the participants. The null hypothesis stated there would be no relationship between the learning outcome and the independent variables, while the alternative hypothesis claimed a relationship would exist.

Before the statistical tests were conducted, data were checked for normality, homogeneity of variance, and outliers, before any violations of the assumptions or extreme values were reported and addressed accordingly.

3. Results

3.1. Descriptive Statistics

This study involved the voluntary participation of students from the Lucian Blaga University the College Logos Centar. The participants, who numbered 115 in total (see Figure 5), were randomly allocated to one of two research groups.

Table 1 shows the learning outcomes recorded for the two study groups, and displays the instruction method and gender of participants.

Analysis of the IQ scores of participants with Raven’s standard progressive matrices test showed that students who received instruction with the HoloLens achieved similar IQ scores to the group instructed with traditional teaching methods. Figure 6 provides a visual representation that supports the claim there were, taking participant IQ scores into account, no significant differences between the two groups.

3.2. Inferential Statistics

To test the hypothesis that the type of instruction would affect student learning outcomes, a series of independent-samples t-tests were conducted to compare the mean anatomy test scores of the two groups - this revealed differences between the two groups, with the PPT group scoring slightly higher than the MR group. The effect size was small (t = −0.723), indicating that the magnitude of the differences was modest.

As shown in Figure 7, the students who received PowerPoint instruction performed slightly better in the test than those who learned the material with HoloLens. The first group of students had a mean score of 9.123 and a standard deviation of 4.314, and the second group of students had a mean score of 8.4 and a standard deviation of 3.608. While PowerPoint instruction may have some learning outcome advantages (over HoloLens instruction), the difference is not very large.

Although the results indicate a slight performance advantage for the PowerPoint instruction group, it is important to consider other factors that may have influenced the outcome, which include factors such as individual learning styles, prior experience with the instructional technology, and the specific content being taught, which could all have contributed to the observed differences.

It is also worth noting that the standard deviations of both groups are relatively high, indicating a wide range of individual performance within each group. This suggests that there is considerable variability in how students engage with, and benefit from, different instructional approaches.

In order to investigate the reasons for the different levels of performance and identify the optimal instructional method for each situation, we conducted a multiple mathematical regression analysis of the anatomy test scores, participant age and the type of instruction received (HoloLens vs. PowerPoint). The results, displayed in Figure 8, reveal that both the age and the instruction method (HoloLens vs. PowerPoint) had a significant (p = 0.004) influence on the learning outcomes.

More specifically, the regression model showed that age was a significant predictor of test performance, indicating that older participants tended to perform better in the test than younger counterparts. In addition, the type of instruction received was also a significant predictor of test performance, suggesting that participants who received PowerPoint instruction tended to score higher on the test than counterparts who received HoloLens instruction (see Figure 9).

However, it is important to note that while the differences in test scores (between the two instruction methods) were statistically significant, the effect sizes were relatively small. These results provide some valuable insights into the factors that influence student performance in anatomy tests, but some study limitations should be acknowledged and considered, including the small size of the study sample, which consists exclusively of undergraduate students, limiting the potential for the results to be generalized to other populations. Second, the study does not account for individual differences in learning styles or prior experience with the instructional technology, which could also affect the outcomes.

Further research is needed to replicate and confirm these findings, and to explore additional factors that could influence the effectiveness of different instructional methods. For example, future studies could investigate the role of student engagement, motivation, and cognitive load in shaping learning outcomes in mixed reality environments.

4. Discussion

The purpose of this study was to explore the effectiveness of mixed reality (MR) technology (the Microsoft HoloLens 2), compared to traditional teaching methods that use PowerPoint presentations. The study also examined the moderating role of participant age in the effects of the instruction methods. This section summarizes the main findings and interprets them in relation to the existing literature; discusses practical implications; and provides recommendations for future research.

4.1. Summary of Findings

The study involved 115 students from Lucian Blaga University of Sibiu and College Logos Centar, who were randomly assigned to either the MR group or the PPT group. The participants completed a Raven’s standard progressive matrices test (that measured their IQ) and a second test (that measured learning outcomes from the teaching of the human muscular system).

The results showed that the PPT group performed slightly better than the MR group, indicating that PowerPoint instruction was more effective than HoloLens instruction for teaching the human muscular system. However, the effect sizes were small, suggesting that the differences were modest. The results also showed that both the instruction method and the age of the participants were significant predictors of the test scores, indicating that older participants who received PowerPoint instruction tended to perform better in the tests. Instruction method had a slightly stronger effect than age on test scores.

4.2. Interpretation of Findings

The finding that PowerPoint instruction was more effective than HoloLens instruction for teaching the human muscular system was contrary to our hypothesis, and to previous studies that suggested MR technology could enhance learning outcomes in various domains [28,29]. One possible explanation for this result is that the HoloLens device imposed a higher cognitive load on learners than the PowerPoint presentation, as they had to process more information and interact with more features in the MR environment. Cognitive load theory suggests that when the cognitive load exceeds the working memory capacity, learning performance and efficiency decrease [30,31]. The HoloLens device may therefore have distracted or overwhelmed learners, instead of facilitating their learning.

Another possible explanation for this result is that the learners had different preferences or styles for learning about the human muscular system, and the PowerPoint presentation matched their preferences or styles more closely than the HoloLens device. Learning styles theory asserts that learners have different ways of perceiving, processing, and retaining information, and that matching the instruction method to the learning style can improve learning outcomes [32,33]. Therefore, the PowerPoint presentation might have suited the learners’ styles more than the HoloLens device, which might have appeared novel or unfamiliar in comparison.

The finding that the instruction method and the age of the participants were significant predictors of the test scores was consistent with our hypothesis and with previous studies that indicated age could moderate the effects of instructional methods on learning outcomes [34,35]. One possible explanation for this finding is that age reflects the cognitive abilities or skills required for learning about the human muscular system, such as reasoning, problem-solving, and memory. Older participants may therefore have been able to process and retain information more effectively and efficiently, regardless of the instruction method that was applied; moreover, participants who received PowerPoint instruction may have benefited more from the clear and concise presentation of the information, which reduced cognitive load and enhanced learning outcomes.

4.3. Implications for Practice

The findings of this study have several implications for education practice, and especially for the teaching of anatomy or other complex and abstract topics. The findings suggest that PowerPoint instruction might be more effective than HoloLens instruction in the teaching the human muscular system, at least if we take the sample and the context of this study as our reference points. Educators who use (or plan to use) MR technology for teaching anatomy should be aware of the potential drawbacks and challenges of this instructional method, including high cognitive load, unfamiliarity, and the variability of learners. Educators should also consider the preferences and styles of learners, and provide them with alternative or complementary instruction methods, such as PowerPoint presentations, printed materials, or practical exercises.

The findings also suggest that age might moderate the effects of instruction methods on learning outcomes, indicating that learners of different ages might vary in terms of their response to different instruction methods. Educators should therefore tailor the instruction methods to the age, cognitive abilities and skills of students, and should also provide learners with appropriate feedback and support, while monitoring their learning progress and performance.

4.4. Recommendations for Future Research

The findings of this study provide some insights into the effectiveness of MR technology and PowerPoint presentations for teaching the human muscular system, and also into the moderating role of age in instruction method effects. However, there are some limitations and gaps that should be addressed by future research.

First, the sample size was relatively small and consisted only of undergraduate students from two educational institutions, which limits the generalizability and external validity of the findings. Future research should use larger and more diverse samples that incorporate different age groups, education levels, and cultural backgrounds, as this will increase the representativeness and robustness of the findings.

Second, the study did not account for individual differences in learning styles, preferences, and motivations, which may have influenced the learners’ engagement with different instruction methods, along with subsequent performance. Future research should measure and control for these factors, and examine how they interact with the instruction methods and the learning outcomes.

Third, the study used a single topic (the human muscular system) and a single type of assessment (multiple choice tests) to measure the learning outcomes, which may not capture the full range of knowledge and skills that learners acquired from the instruction methods. Future research should use multiple topics and types of assessments (such as open-ended questions, practical tasks, and/or case studies) to provide a more comprehensive and holistic evaluation of the learning outcomes.

5. Limitations

The limitations of this study, including the relatively small sample size and exclusive focus on undergraduate students, means that we must be cautious when seeking to generalize the results to broader populations or educational settings. Future research should establish larger and more diverse participant groups and consider individual learning styles, motivation, and cognitive capacities, as this will help to provide a more comprehensive understanding of the implications of instructional methods in mixed reality education.

We also acknowledge that multiple choice tests have some disadvantages, such as the possibility of guessing the correct answer, the tendency to focus on memorizing facts rather than deep understanding [43], and the difficulty of creating well-formulated questions and plausible distractors [44], and we therefore propose future studies could explore alternative methods of assessment that potentially involve more creativity and emphasise student problem-solving skills to a greater extent: both requirements could be met by self-assessment, peer feedback, end-of-course evaluations, surveys, discussion groups, exit interviews, concept maps, portfolios, oral presentations, projects and experiments. Using these methods could provide opportunities for students to demonstrate their knowledge and perform meaningful tasks, which are both clear learning objectives. This would also facilitate a more active and engaging learning, along with a more flexible and adaptable approach to assessment that acknowledges the individual needs and abilities of students.

Despite these limitations, this study contributes to an ongoing discourse that seeks to leverage immersive technologies in education. It also highlights the potential of MR technologies (such as HoloLens 2) to enhance learning experiences, especially for individuals with specific cognitive profiles. The further exploration and refinement of instructional methodologies in MR environments could potentially pave the way for more tailored, inclusive, and effective educational practices.

6. Conclusions

This study examined the comparative effectiveness of mixed reality technology by utilizing Microsoft HoloLens 2, and then directly compared it to traditional PowerPoint presentations in the teaching of human anatomy to students. The findings revealed differential impacts that varied with participant age, suggesting that the effectiveness of the instructional method might vary in accordance with the cognitive profile. MR technology demonstrated promise in enhancing learning outcomes for younger individuals, whole traditional PowerPoint presentations seemed to be marginally more effective for older students. Age therefore emerged as a factor that influenced learning performance, indicating a need for tailored instructional approaches across different age groups.

The study underscored the importance of considering diverse cognitive abilities and age-related differences when designing educational interventions that leverage immersive technologies. Future research should explore the multifaceted influences that act on learning outcomes by establishing larger and more varied participant groups, as this will help to advance the understanding of effective instructional methodologies in mixed reality education. In addition, future research should investigate optimal ways to integrate MR technology with other pedagogical strategies, such as feedback, scaffolding, and collaboration, as this will maximize its potential to enhance student engagement and motivation. In addition, researchers should also examine the long-term effects of MR technology on the retention and transfer of knowledge, and also give consideration to the possible ethical and social implications of using immersive technologies in educational settings.

This study contributes valuable insights to the evolving landscape of educational technology, and offers nuanced perspectives that take into account the potential benefits of immersive learning environments, along with associated considerations. By highlighting the complex interplay of cognitive and contextual factors that affect learning outcomes, this study brings out useful implications that will be of direct interest to educators, researchers, and policymakers currently seeking to leverage the power of MR technology, with the aim of transforming and enriching learning experiences.

Author Contributions

Conceptualization, R.E.P., V.G. and J.M.G.; methodology, R.E.P. and V.G.; formal analysis, R.E.P. and V.G.; investigation, R.E.P., V.G. and M.G.; data curation, R.E.P., V.G., D.M. and M.G.; writing—original draft preparation, R.E.P. and V.G.; writing—review and editing, R.E.P., V.G. and J.M.G.; visualisation, R.E.P.; supervision, V.G.; funding acquisition, V.G. All authors have read and agreed to the published version of the manuscript.

Institutional Review Board Statement

The experiments were conducted after being approved by the ULBS Ethical Committee (no.28/6 December 2023).

Informed Consent Statement

Informed consent was obtained from all study subjects.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors on request.

Acknowledgments

The authors would like to thank the 115 students who participated in the study.

Conflicts of Interest

The authors declare no conflicts of interest.

Footnotes

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Figures and Table

Figure 1. VOSviewer Network Visualisation for Web of Science (a) and Scopus (b).

Figure 1. VOSviewer Network Visualisation for Web of Science (a) and Scopus (b).

Figure 2. Network Visualization of Augmented and Mixed Reality and PowerPoint in Education.

Figure 3. Group 1 teaching material—holographic image of the human muscular system displayed using a HoloLens.

Figure 4. Overview of the research procedure.

Figure 5. Gender distribution of the participants in the two study groups (HoloLens and PowerPoint).

Figure 6. IQ score distribution of the two research groups.

Figure 7. Correct answers at the anatomy test, grouped by instruction method.

Figure 8. Multiple correlation between the test results, instruction method and participant age.

Figure 9. Interaction plots for correct answers and age and instruction method.

Table 1

Descriptive statistics of the two research groups.

Instruction Method	Variable	Sex	N	Mean	SE Mean	StDev	Minimum	Q1	Median	Q3	Maximum
Hololens	Correct answers	Female	16	6.813	0.672	2.689	3.000	4.250	6.000	9.000	12.000
		Male	34	9.147	0.647	3.775	2.000	6.750	9.500	12.000	16.000
	Variable	Sex	N	Mean	SE Mean	StDev	Minimum	Q1	Median	Q3	Maximum
PowerPoint	Correct answers	Female	27	8.370	0.851	4.422	2.000	5.000	9.000	12.000	17.000
		Male	38	9.658	0.683	4.212	2.000	7.000	10.000	12.000	20.000

References

1. Tang, Y.M.; Chau, K.Y.; Kwok, A.P.K.; Zhu, T.; Ma, X. A Systematic Review of Immersive Technology Applications for Medical Practice and Education—Trends, Application Areas, Recipients, Teaching Contents, Evaluation Methods, and Performance. Educ. Res. Rev.; 2022; 35, 100429. [DOI: https://dx.doi.org/10.1016/j.edurev.2021.100429]

2. Di Serio, Á.; Ibáñez, M.B.; Kloos, C.D. Impact of an Augmented Reality System on Students’ Motivation for a Visual Art Course. Comput. Educ.; 2013; 68, pp. 586-596. [DOI: https://dx.doi.org/10.1016/j.compedu.2012.03.002]

3. Zhao, M.; Ong, S.-K.; Nee, A.Y. An Augmented Reality-Assisted Therapeutic Healthcare Exercise System Based on Bare-Hand Interaction. Int. J. Hum.–Comput. Interact.; 2016; 32, pp. 708-721. [DOI: https://dx.doi.org/10.1080/10447318.2016.1191263]

4. Bacon, L.; MacKinnon, L.; Cesta, A.; Cortellessa, G. Developing a Smart Environment for Crisis Management Training. J. Ambient Intell. Humaniz. Comput.; 2013; 4, pp. 581-590. [DOI: https://dx.doi.org/10.1007/s12652-012-0124-0]

5. Sebillo, M.; Vitiello, G.; Paolino, L.; Ginige, A. Training Emergency Responders through Augmented Reality Mobile Interfaces. Multimed. Tools Appl.; 2016; 75, pp. 9609-9622. [DOI: https://dx.doi.org/10.1007/s11042-015-2955-0]

6. Hamilton, D.; McKechnie, J.; Edgerton, E.; Wilson, C. Immersive Virtual Reality as a Pedagogical Tool in Education: A Systematic Literature Review of Quantitative Learning Outcomes and Experimental Design. J. Comput. Educ.; 2021; 8, pp. 1-32. [DOI: https://dx.doi.org/10.1007/s40692-020-00169-2]

7. Kavanagh, S.; Luxton-Reilly, A.; Wuensche, B.; Plimmer, B. A Systematic Review of Virtual Reality in Education. Themes Sci. Technol. Educ.; 2017; 10, pp. 85-119.

8. Khan, M.N.R.; Lippert, K.J. Immersive Technologies in Healthcare Education. Intelligent Systems and Machine Learning for Industry; CRC Press: Boca Raton, FL, USA, 2022; pp. 115-138.

9. Petruse, R.E.; Grecu, V.; Chiliban, B.M. Augmented Reality Applications in the Transition towards the Sustainable Organization; Springer: Berlin/Heidelberg, Germany, 2016; pp. 428-442.

10. Van Krevelen, D.; Poelman, R. A Survey of Augmented Reality Technologies, Applications and Limitations. Int. J. Virtual Real.; 2010; 9, pp. 1-20. [DOI: https://dx.doi.org/10.20870/IJVR.2010.9.2.2767]

11. Grecu, V.; Deneș, C.; Ipiña, N. Creative Teaching Methods for Educating Engineers. Appl. Mech. Mater.; 2013; 371, pp. 764-768.

12. Mallam, S.C.; Nazir, S.; Renganayagalu, S.K. Rethinking Maritime Education, Training, and Operations in the Digital Era: Applications for Emerging Immersive Technologies. J. Mar. Sci. Eng.; 2019; 7, 428. [DOI: https://dx.doi.org/10.3390/jmse7120428]

13. Sutherland, I.E. Sketch Pad a Man-Machine Graphical Communication System. Proceedings of the SHARE Design Automation Workshop; Association for Computing Machinery: New York, NY, USA, 1964; pp. 6-329.

14. Slater, M. Place Illusion and Plausibility Can Lead to Realistic Behaviour in Immersive Virtual Environments. Philos. Trans. R. Soc. B Biol. Sci.; 2009; 364, pp. 3549-3557. [DOI: https://dx.doi.org/10.1098/rstb.2009.0138]

15. Lee, H.-G.; Chung, S.; Lee, W.-H. Presence in Virtual Golf Simulators: The Effects of Presence on Perceived Enjoyment, Perceived Value, and Behavioral Intention. New Media Soc.; 2013; 15, pp. 930-946. [DOI: https://dx.doi.org/10.1177/1461444812464033]

16. Díaz-López, L.; Tarango, J.; Contreras, C.-P. Strategies for Inclusive and Safe Education Using Virtual Reality: From the Digital Library Perspective. Digit. Libr. Perspect.; 2019; 35, pp. 216-226. [DOI: https://dx.doi.org/10.1108/DLP-08-2019-0034]

17. Butt, A.L.; Kardong-Edgren, S.; Ellertson, A. Using Game-Based Virtual Reality with Haptics for Skill Acquisition. Clin. Simul. Nurs.; 2018; 16, pp. 25-32. [DOI: https://dx.doi.org/10.1016/j.ecns.2017.09.010]

18. Salzman, M.C.; Dede, C.; Loftin, R.B.; Chen, J. A Model for Understanding How Virtual Reality Aids Complex Conceptual Learning. Presence Teleoperators Virtual Environ.; 1999; 8, pp. 293-316. [DOI: https://dx.doi.org/10.1162/105474699566242]

19. Ellaway, R.; Masters, K. AMEE Guide 32: E-Learning in Medical Education Part 1: Learning, Teaching and Assessment. Med. Teach.; 2008; 30, pp. 455-473. [DOI: https://dx.doi.org/10.1080/01421590802108331] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/18576185]

20. Harden, R.M.; Laidlaw, J.M. Effective Continuing Education: The CRISIS Criteria. Med. Educ.; 1992; 26, pp. 407-422. [DOI: https://dx.doi.org/10.1111/j.1365-2923.1992.tb00194.x]

21. Ryan, G.V.; Callaghan, S.; Rafferty, A.; Higgins, M.F.; Mangina, E.; McAuliffe, F. Learning Outcomes of Immersive Technologies in Health Care Student Education: Systematic Review of the Literature. J. Med. Internet Res.; 2022; 24, e30082. [DOI: https://dx.doi.org/10.2196/30082] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/35103607]

22. Fortman, J.; Quintana, R. Fostering Collaborative and Embodied Learning with Extended Reality: Special Issue Introduction. Int. J. Comput.-Support. Collab. Learn.; 2023; 18, pp. 145-152. [DOI: https://dx.doi.org/10.1007/s11412-023-09404-1]

23. Mathew, P.S.; Pillai, A.S. Role of Immersive (XR) Technologies in Improving Healthcare Competencies: A Review. Virtual and Augmented Reality in Education, Art, and Museums; IGI Global: Hershey, PA, USA, 2020; pp. 23-46.

24. Makransky, G.; Terkildsen, T.S.; Mayer, R.E. Adding Immersive Virtual Reality to a Science Lab Simulation Causes More Presence but Less Learning. Learn. Instr.; 2019; 60, pp. 225-236. [DOI: https://dx.doi.org/10.1016/j.learninstruc.2017.12.007]

25. Parong, J.; Mayer, R.E. Learning Science in Immersive Virtual Reality. J. Educ. Psychol.; 2018; 110, 785. [DOI: https://dx.doi.org/10.1037/edu0000241]

26. Jensen, L.; Konradsen, F. A Review of the Use of Virtual Reality Head-Mounted Displays in Education and Training. Educ. Inf. Technol.; 2018; 23, pp. 1515-1529. [DOI: https://dx.doi.org/10.1007/s10639-017-9676-0]

27. Reed, D.A.; Cook, D.A.; Beckman, T.J.; Levine, R.B.; Kern, D.E.; Wright, S.M. Association between Funding and Quality of Published Medical Education Research. JAMA; 2007; 298, pp. 1002-1009. [DOI: https://dx.doi.org/10.1001/jama.298.9.1002] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/17785645]

28. Odame, A.; Tümler, J. Is Off-the-Shelf VR Software Ready for Medical Teaching?. Proceedings of the Virtual, Augmented and Mixed Reality: Design and Development; Virtual, 16 June 2022; Chen, J.Y.C.; Fragomeni, G. Springer International Publishing: Cham, Switzerland, 2022; pp. 224-237.

29. Banjar, A.; Xu, X.; Iqbal, M.Z.; Campbell, A. A Systematic Review of the Experimental Studies on the Effectiveness of Mixed Reality in Higher Education between 2017 and 2021. Comput. Educ. X Real.; 2023; 3, 100034. [DOI: https://dx.doi.org/10.1016/j.cexr.2023.100034]

30. Sweller, J. Cognitive Load Theory. Psychology of Learning and Motivation; Elsevier: Amsterdam, The Netherlands, 2011; Volume 55, pp. 37-76. ISBN 0079-7421

31. Sweller, J. Cognitive Load Theory and Educational Technology. Educ. Technol. Res. Dev.; 2020; 68, pp. 1-16. [DOI: https://dx.doi.org/10.1007/s11423-019-09701-3]

32. Felder, R.M.; Silverman, L.K. Learning and Teaching Styles in Engineering Education. Eng. Educ.; 1988; 78, pp. 674-681.

33. Felder, R.M. Opinion: Uses, Misuses, and Validity of Learning Styles. Adv. Eng. Educ.; 2020; 8, pp. 1-16.

34. Dowling, C.; Godfrey, J.M.; Gyles, N. Do Hybrid Flexible Delivery Teaching Methods Improve Accounting Students’ Learning Outcomes?. Account. Educ.; 2003; 12, pp. 373-391. [DOI: https://dx.doi.org/10.1080/0963928032000154512]

35. Nikitenko, G. Analysis of Adult Students’ Self-Directed Learning Readiness, Affective Learning Outcomes, Prior e-Learning Experience, and Age in Hybrid and Online Courses; Association for the Advancement of Computing in Education (AACE): Chesapeake, VA, USA, 2011; pp. 2503-2513.

36. VOSviewer—Visualizing Scientific Landscapes. Available online: https://www.vosviewer.com// (accessed on 13 January 2024).

37. Zhang, J.; Yen, S.-H.; Liu, T.-C.; Sung, Y.-T.; Chang, K.-E. Studies on Learning Effects of AR-Assisted and PPT-Based Lectures. Asia-Pac. Educ. Res.; 2022; 31, pp. 1-10. [DOI: https://dx.doi.org/10.1007/s40299-020-00533-x]

38. Staddon, R.V. Bringing Technology to the Mature Classroom: Age Differences in Use and Attitudes. Int. J. Educ. Technol. High. Educ.; 2020; 17, 11. [DOI: https://dx.doi.org/10.1186/s41239-020-00184-4]

39. Lewis, I. The Student Experience of Higher Education; Routledge: London, UK, 2018; Volume 16, ISBN 0-429-82624-9

40. Garland, K.J.; Noyes, J.M. Computer Experience: A Poor Predictor of Computer Attitudes. Comput. Hum. Behav.; 2004; 20, pp. 823-840. [DOI: https://dx.doi.org/10.1016/j.chb.2003.11.010]

41. Kim, S.; Song, S.-M.; Yoon, Y.-I. Smart Learning Services Based on Smart Cloud Computing. Sensors; 2011; 11, pp. 7835-7850. [DOI: https://dx.doi.org/10.3390/s110807835] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/22164048]

42. Butler, A.C. Multiple-Choice Testing in Education: Are the Best Practices for Assessment also Good for Learning?. J. Appl. Res. Mem. Cogn.; 2018; 7, pp. 323-331. [DOI: https://dx.doi.org/10.1016/j.jarmac.2018.07.002]

43. Roediger, H.L.; Marsh, E.J. The Positive and Negative Consequences of Multiple-Choice Testing. J. Exp. Psychol. Learn. Mem. Cogn.; 2005; 31, pp. 1155-1159. [DOI: https://dx.doi.org/10.1037/0278-7393.31.5.1155]

44. Marsh, E.J.; Roediger, H.L.; Bjork, R.A.; Bjork, E.L. The Memorial Consequences of Multiple-Choice Testing. Psychon. Bull. Rev.; 2007; 14, pp. 194-199. [DOI: https://dx.doi.org/10.3758/BF03194051]