School of the Future: A Comprehensive Study on the Effectiveness of Augmented Reality as a Tool for Primary School Children’s Education

Abstract

With the emerging technologies of augmented reality (AR) and virtual reality (VR), the learning process in today’s classroom is much more effective and motivational. Overlaying virtual content into the real world makes learning methods attractive and entertaining for students while performing activities. AR techniques make the learning process easy, and fun as compared to traditional methods. These methods lack focused learning and interactivity between the educational content. To make learning effective, we propose to use handheld marker-based AR technology for primary school students. We developed a set of four applications based on students’ academic course of primary school level for learning purposes of the English alphabet, decimal numbers, animals and birds, and an AR Globe for knowing about different countries around the world. These applications can be played wherever and whenever a user wants without Internet connectivity, subject to the availability of a tablet or mobile device and the required target images. These applications have performance evaluation quizzes (PEQs) for testing students’ learning progress. Our study investigates the effectiveness of AR-based learning materials in terms of learning performance, motivation, attitude, and behavior towards different methods of learning. Our activity results favor AR-based learning techniques where students’ learning motivation and performance are enhanced compared to the non-AR learning methods.

1. Introduction

AR techniques are used extensively in various real world applications for education, training, entertainment, tourism, and cultural heritage [1]. Especially in the education field, AR techniques have completely changed the design and method of learning [2]. Similarly, in the fields of cultural heritage and tourism, the historical places and objects can be reconstructed with the help of AR and VR techniques [3]. The ancient buildings that are not present in normal conditions are possible to visit with AR techniques [4]. AR techniques combine real world scenes and computer-based virtual world scenes and present their combined view to the user [5]. AR technology adds 3D user interactive virtual objects into the physical world [6]. According to the 2010 Horizon Report [7], AR techniques are of two types, marker-based and markerless-based. In marker-based AR techniques, image targets are predefined to visualize virtual content at different places. Target images are scanned with an AR camera (mobile or webcam), and then the system renders 3D visuals on the surface of that image targets in the real world. On the other hand, in the case of markerless-based AR techniques, there are no predefined target images [8]. An AR camera is capable of tracking the surfaces of any physical environment and overlay virtual content. It also utilizes a compass and Global Positioning System (GPS) for location tracking. AR techniques are used in almost every education field [9], such as engineering, medical education, surgical training, arts, designing, and language learning. Using AR-based learning techniques, students are highly motivated for learning activities, and their learning capabilities are significantly enhanced [10]. Students at primary age probably learn from activities where they observe, listen, and explore different things. Students are interested in learning and motivated to attend places, such as school, where they perform physical activities [11]. According to the study [12], techniques utilizing AR results in a better understanding of the concepts. In traditional knowledge transfer methods, a single teacher manages all the teaching processes, and the learning activities are performed face to face with the tutor [13]. The traditional method of learning is quite tedious for students and teachers that have a burden of many students to teach in class and satisfy each student. In traditional methods, learning materials are pen, paper, charts, videos, and presentations. Using these tools, the learning is not so interactive to the users as the AR learning content [14,15,16]. In this era of technology, most of the teaching community and researchers are interested in modifying the education system to enhance both the learning and teaching understandings [17]. The learning and teaching methods are greatly developed and expanded, due to technological advancements in recent years [18]. Most studies are now in favor of AR learning and training techniques compared to traditional learning methods [19]. AR learning techniques have several positive impressions on the students in comparison with traditional methods of learning. AR techniques increase students’ motivation for learning, teamwork in group tasks, and understanding of the learning content for an extended period [20].

As suggested by the authors of [21], the learning experience becomes more meaningful when more senses are involved. From the children’s perspective, a fascinating way of using their senses is through visualization, which is a crucial way to improve and develop their capacity to understand things [22]. Thus, the visualization of 3D graphics is more effective in solving and understanding the problem using all the children’s senses. Based on the reasons above, we build a new learning system that utilizes AR technology to improve children’s learning skills. Using this system, children can interact with the book contents displayed virtually via smartphones. At earlier ages, engaging primary school students in learning activities is quite a difficult task. So, exerting an entertainment-based approach to learning helps students take an interest in learning and exploring. So, our main target is to engage students in learning. We target fundamental learning aspects for students at the primary level, for which we develop a set of four applications for introducing mobile-based AR technology to education. We use mobile-based AR techniques because these are comparatively easy to use as compared to the other AR Head-Mounted Displays (HMDs) for children [23]. The first one is “Nursery Rhyme” for learning English alphabet, which is one of the first tasks students do while start learning. The second is learning decimal numbers with the help of an AR application named “AR Numbers”. To learn about animals, we have an AR application named “AR Zoo” Next, the application of “AR Globe” is for understanding the earth’s surface and to learn about different countries, their flags, and locations on the globe surface (see Supplementary Materials). Learning English alphabet and decimal numbers are the very first step of children’s learning process [9,24]. Similarly, learning about animals and globe study is one of the introductory topics of education for children [25,26]. To makes the learning process fun, attractive, and interesting, we use AR technology for learning purposes of these topics with AR-based mobile applications. In this way, we can make the usage of smartphones positive and effective for children.

This research study investigates the effectiveness of the AR learning methods for primary school students compared to traditional methods of learning. We conduct learning activities using both traditional and advanced AR learning methods. We analyze the student’s behavior and interest during learning activities using different learning approaches. To evaluate the term “effectiveness” of the AR learning methods, the following research topics are discussed comprehensively in this study.

• The attitude and behavior of the students towards AR learning methods in contrast to traditional methods.

• The interest of students and motivation towards AR-based learning activities and traditional learning methods.

• The learning achievements of students through the advanced techniques of AR compared to traditional learning methods.

• Parents’ and teachers’ response towards the potentials of implementing AR techniques in student’s academics at early ages.

We use Unity 3D game engine and other 3D modeling software, such as Maya, for application development. For AR support, we use Vuforia AR SDK, and for coding, we use C# language with Visual Studio. AR technology can help us in designing a more meaningful and enjoyable learning environment for students with positive outcomes. Considering the importance of AR technology in the education field, we summarize this research’s contributions as follows.

• Mobile-based AR techniques are used to develop four applications for primary school students’ learning purposes. With the help of these applications, students can learn and enjoy their subjects through android phones or tablets.

• Proposed applications are based on the academic books of grade one that fulfills all the curriculum requirements of the targeted section. Students can observe the abstract concepts of their academic books through their eyes with mobile-based AR technology in the form of 3D virtual content.

• The PEQs are built in each application used to measure the learning efficiency of these AR-based learning methods. Students learning achievements are tested using these quizzes without conducting any theoretical physical exam.

• Three different types of learning activities are performed to measure the effectiveness of AR technology in students’ academic activities as compared to the traditional methods of learning, especially at early ages. The above-mentioned research questions are discussed in detail to evaluate the effectiveness of AR-based learning methods.

The remaining paper is arranged as follows. In Section 2, we discuss the related works. Section 3 presents the overall architecture of the proposed applications. Section 4 provides details of the experiments and results. We finally put forward our conclusion and future development directions in Section 5.

2. Related Work

This section of the paper provides a detailed overview of different applications and studies in the literature for effective learning using AR technology.

2.1. AR in Education

AR technologies are widely used in different fields of education for learning purposes, and their benefits are supported by many researchers [27,28]. Students learn from AR applications, interacting with the learning contents in real-time. These AR applications are exciting, and children are attracted to the learning activities [29]. While learning with AR techniques, the students are entertained and fully engaged in the learning process [24]. Students have high achievements and a positive attitude towards AR learning content [30]. AR brings entertainment to the children, while they learn with its potential for user interaction [31]. AR can make interactive 3D virtual visuals in the real world that develop the students’ skills and improve rehabilitation [32,33]. AR techniques effectively bring up motivation and frustration tolerance in special students with disabilities [34]. Chang et al. [35] revealed that AR learning techniques attract students’ attention during the learning process. They used AR technology to learn English vocabulary and found it compelling.

Over the past years, several researchers have incorporated AR technology for the learning purposes of different educational concepts. As relevant to our study, Safar et al. [36] used AR applications as a tool for teaching and learning of English alphabet to preschool students. They compared the AR learning applications method with the traditional classroom-based learning method. Their results were in favor of AR applications and recommended this technology for students in schools. Similarly, Rambli et al. [37] developed an AR alphabet book and conducted an observational study in order to investigate the effectiveness of AR technology for primary school students. They revealed that AR technology is an engaging and helpful tool for learning purposes. Further, Edwin Ciezaa and David Lujan [38] used AR mobile applications for the learning purposes of vowels and numbers to the early age students. Their data analysis showed an increase in the learning achievement of the students. For learning numbers, an AR technology-based magical playbook was introduced consist of AR applications using a physical book as image targets for preschool students in a study at reference [24]. The students’ feedback in their study motivates AR technology for learning purposes in schools. In the case of animals, Zarzuela et al. [26] created an AR technology-based serious game for children to learn about animals. Barreira et al. [19] presented an AR-based learning application named “MOW” (Matching Objects and Words), which is used for learning the names of animals in English. For evaluation purposes, they compared their proposed approach with traditional learning methods. Students showed interest in AR learning methods and had higher progress in learning than traditional learning methods. Nainggolan et al. [39] designed AR-based applications for learning information about animals. They used target images and showed the animated 3D models of animals to give visual information. For understanding earth surface, Schnürer et al. [40] developed AR technology-based applications using printed maps as target images and visualize 3D content. Midak et al. [25] developed an AR-based mobile application for understanding solar planets in primary schools. Similarly, Wiehr et al. [41] used paper maps and augmented 3D virtual terrain with the help of AR technology. AR technology has several positive effects on the behavior of students learning. In the following section, we discuss the behaviors of the learners that are polished with AR learning techniques.

2.2. Learning Content Understanding

Several publications favor AR compared to conventional ink and paper, computer systems, and video-based learning methods [28]. Lindgren et al. [42] conducted two different sessions to learn astronomy. In one session, they used a computer system. In the other session, they utilized AR technology with direct interaction with the user. Results of their study showed apparent differences between the participants of the two sessions. Similarly, Sin et al. [43] compared AR-based learning to traditional book-based learning for teaching solar systems. Their results proved that the AR technique effective as compared to book-based learning. Nischelwitzer et al. [44] taught human anatomy by visualizing 3D models of human body parts in the real world, making the study very interesting. They concluded that students have a better understanding of the AR learning environment as compared to textbooks. Quarles et al. [45] used AR technology for the understanding of medical devices and their functionality. They compared their results with VR based training approach and found the AR-based training more effective. Similarly, Freitas and Campos [46] trained the students about animals with both AR techniques and book-based methods. Students achieved higher scores from the learning session based on AR technology.

2.3. Improve Collaboration

AR techniques help in improving collaboration among group participants [28]. Freitas and Campos [46] confirmed that collaboration among students in a group is highly increased when they share AR learning experiences compared to non-AR learning methods. Similarly, Morrison et al. [47] performed navigations activity in two groups. One group used a mobile AR-based map, while the other group used a simple digital GPS-based mobile map. They concluded that AR techniques are more effective, where participants collaborated compared to the digital mobile map. Matcha and Rambli [48] used AR technology in a collaborative learning environment for investigation purposes. Tang et al. [33] performed object assembling tasks on four groups having different learning methods. Their learning methods were paper, computer system, VR HMDs, and AR HMDs based. Their study revealed that AR techniques help to improve task performance activity with a minimum number of errors compared to other learning activities. AR methods of learning helped students to relieve mental stress, while performing assembling of objects tasks. Similarly, Pathomaree and Charoenseang [49] found AR effective in performing the activity of assembling task completion. Their results supported AR techniques that provide collaborative learning and help in communication. Students showed a positive attitude towards AR technology. Lin et al. [50] studied the effects of collaborative AR techniques on the student’s understandings. Their results suggested that AR techniques help the students learn, and the students’ response was too good. Yoon et al. [51] used collaborative AR learning in groups of students for a science museum. Their study supported collaborative AR for conceptual learning. Students praised the collaborative learning and found it helpful.

2.4. Increase Student Motivation

As compared to the traditional methods of learning, students show higher satisfaction from AR learning methods. Kaufmann and Dünser [52] used an AR technique and compared it to a computer system-based method to learn geometry education. They used “Construct3D”; an AR application, and found it very helpful to gain students’ interest in learning. Further, learning in the AR environment entertained the students, and they were highly motivated for learning. Young students could not use the AR HMDs, but they were willing to perform the activity [53]. Similarly, the study of Serio et al. [54] showed a positive impact of AR learning on students’ motivation for learning activities of the middle level. They analyzed the effects of AR learning methods with traditional learning methods. Tomi et al. [55] conducted a study to examine the AR-based learning approach’s efficiency in enhancing students’ motivation and learning capabilities. The outcomes of their study proved the effectiveness of mobile-based AR learning. Students from the AR learning methods were highly motivated towards learning compared to traditional learning methods.

In conclusion, AR learning can be used to increase the potential of education and training purposes. Interaction with the user with learning content in the real world should be taken into account. To cross the barrier with AR, we have to build the AR learning techniques with proper curriculum and planning. To bring AR to the classroom, the developers must understand the student’s curriculum [28]. Augmentation of the educational contents with AR techniques will be more helpful for the students as compared to the other traditional methods of learning and presentation [56]. Similarly, developing intelligent AR systems is required where students’ performance of learning and knowledge gaining can be evaluated. With these AR learning systems, students can collaborate and learn in an effective and motivating way.

3. Methodology

In this section, we discuss the development of the proposed AR-based learning approach in detail. To build an application for educational purposes, we must consider all the student’s academic and emotional aspects. In the technological aspect, we select AR techniques for developing learning and teaching content. AR technology fulfills all the educational needs required by students for learning effectively. With the help of AR techniques, the students see the virtual learning content in the real world, and they can interact with them in real-time. In the case of traditional learning, the students can only imagine the learning content.

Different types of AR Software Development Kits (SDKs) are mentioned in the literature that has been used for the development of educational applications [57]. We use the Vuforia SDK, due to its extensive usage in the development of AR learning applications. Vuforia is an AR SDK that can track target images, objects, and ground surfaces with computer vision techniques. Using it, a 3D virtual environment can be augmented into the real world surface. The target’s position and direction are tracked with the help of an AR camera (mobile, webcam), and then the virtual 3D objects are rendered on the surface of the target. The virtual content is user interactive in real-time, and it seems the same as the part of real world physical life. Figure 1 shows the complete architecture of the Vuforia SDK [58].

We design and develop our applications using the popular game engine Unity 3D. Unity is a game engine that provides different functionalities for designing and developing user interactive virtual 3D environments. It also supports application development across multiple platforms. The Vuforia AR SDK’s free support makes the development of the AR-based applications straightforward in the Unity 3D game engine [59]. The visualization of the 3D graphics in Unity 3D is highly improved as well. Unity 3D supports different 3D model formats that make designing more manageable and attractive [60]. Therefore, the augmentation of these 3D virtual content into the physical world makes AR techniques more realistic and precise. Further, Unity 3D improves the communication between the game objects and the programming section that helps the developer in the logic implementation more quickly and accurately.

3.1. Design and Development Process

During system architecture, our main target is to make these learning AR applications users friendly and interactive for students. The developmental phase involved different steps, such as 3D modeling of the objects, finger touch interaction of the students with augmented 3D models, animations, and sound effects. Figure 2 shows the overall design and development phases of the applications.

We create the target databases with the Vuforia Target Management System (VTMS) in the first phase of the development. The image targets of the AR applications are uploaded to the web-based VTMS at the Vuforia developmental portal. The feature points in these image targets are detected by the VTMS and are used to recognize these target images in the real environment. VTMS assigns a rating to each target image-based on the feature points, as shown in Figure 3. These target images can be accessed from both the cloud and local device-based datasets. We use device-based target databases downloaded from the Vuforia development portal and add to the Unity 3D environment. For designing the 3D models and animations, we use Unity 3D free assets and 3D designing software, such as Autodesk Maya, 3D Max, Blender, and other free internet resources. For logic development, we use C# programming language using Visual Studio. All the AR application’s design and developmental content are integrated within Unity 3D.

3.2. Applications Workflow

These applications present the virtual educational content in the real world environment once the image targets are in the camera view of smartphones or tablets. Using this way, students can see and observe the virtual 3D content in the physical world and interact with the augmented environment associated with the image targets in their books or hand notes. The complete workflow of the applications is described in Figure 4.

Process:

• Using an android phone, IOS, or tablet, a target book page or animal image is scanned.

• Features of the target page are stored in the Vuforia database.

• Obtained features are compared with the pre-stored features of the target images in the Vuforia database.

• When the features match up with the database, the application augments virtual content on the surface of that image target.

• The AR application visualizes the learning content, graphics with the image captured by the smartphone.

As mentioned above, we introduce four AR educational applications for students. These applications are discussed in the following sections (Section 3.2.1, Section 3.2.2, Section 3.2.3, and Section 3.2.4).

3.2.1. Nursery Rhyme

Nursery Rhyme is an AR application for the learning purpose of the English alphabet. This application is based on an academic book named “Nursery Rhyme” for the students of grade one. Teaching the English alphabet to children at the age of five to eight, especially non-native children, is difficult. Letters from A to Z are taught utilizing attractive 3D visuals, animations, and melodic sounds. AR technology can enhance alphabet learning in students. We target a primary level academic book as AR markers for visualizing AR content on the screen. The user needs a smartphone or tablet to scan the target images and observe virtual content on the surface of those target images. The content of the “Nursery Rhyme” AR-based application has two sections of learning. The first section is the students learning section, in which students take a session of learning the alphabet. Pages of the book are scanned with an AR camera to provide animated 3D virtual learning content. Students learn with the vocabulary of words associated with the specific letter. For example, for the letter “A”, we have virtual models of the objects and animals that start with the letter “A”, such as Ant and Apple. Students also learn the pronunciation of each letter and associated words. Similarly, for the letter “B”, corresponding objects are augmented in the real world environment. Students interact with these models with the help of touch and observe from different aspects. Users can zoom in, zoom out, rotate, translate these models. Students can play the corresponding melodic rhyme of each letter. Students can take screenshots of the 3D models that are saved in the application gallery for viewing and sharing with friends.

The second section is a quiz that evaluates children’s performance. In this session, students are tested with image-based Multiple-Choice Questions (MCQs) given in the book. The quiz session consists of twenty-six questions, one question per letter. Based on the correct answers, the students are awarded stars. Based on these stars, students learning performance is evaluated. Figure 5 shows screenshots of the application.

3.2.2. AR Numbers

AR Numbers is also an AR mobile application that is designed for learning decimal numbers. Understanding numbers is the first challenge for students when he or she starts learning. In this application, images of numbers from zero to nine are used as image targets for visualizing virtual content. The user can simply download the image target from the link given in the application, print them, and use them for the application as AR camera target images. This application scans those printed images, and after matching features with the AR database, it augments the 3D characters, animations, and audios related to each number. The learning section of this application is user-friendly. Students can interact with the learning content of the application and learn numbers by singing and counting virtual objects according to the image target’s number. The AR application’s interactive design helps to entertain students, while using this application and motivates them to learn.

This application is designed for both learning and testing purposes. In the first AR scene, students perform a learning activity of numbers with the help of AR content. In the second scene, students perform a quiz to test their learning level. This quiz comprises a counting activity in which students have to count the number of animals and objects in given images. Students get the reward in the form of trophies displayed in the application’s trophy room. Different screenshots from the AR Numbers application are shown in Figure 6.

3.2.3. AR Zoo

AR Zoo is a mobile-based AR application that visualizes 3D animated animals in front of students. At the primary level, learning animal names and their information is one of the important lessons for students. It is unable for students to visit a zoo or any other place to learn about animals. This AR Zoo application will be helpful to the students so that they can visit animals in a virtual environment. This application is helpful for all the students and also those students that are handicapped [26]. It can be used by students having some type of physical disabilities who cannot visit the zoo. We use images of different animals and birds as image targets for creating virtual content in the real world scenario. Users can download the animal’s images from the link given in the application. Using this application, students scan the image targets with the AR camera. Upon feature matching, the AR application creates a virtual 3D model of the image target’s mentioned animal. This model has walking animations, still images, and some funny acts for students’ entertainment. These animals include elephants, lions, dogs, donkeys, zebras, horses, and several birds, including hens, pigeons, and sparrows. These 3D animal models are user-friendly that the students can interact with by physical interactions and play with them. Students can listen to their sounds and learn about animals’ names, groups, food, and much more.

During this application, the user can choose among different scenes related to the animals. In the AR scene, students go through the animals. The second scene is a quiz for students testing what they learned. A quiz evaluates up to which level students become familiar with animals after using this AR Zoo application. Figure 7 shows screenshots of the AR Zoo application.

3.2.4. AR Globe

The AR Globe application is designed for understanding the physical globe. Its different views are water/soil view, continent division, and globe view. In the water/soil view, the surface of the earth is shown with the percentage of water and land covering the earth’s surface that is also known as the satellite view of the earth. This application is enriched with AR technology that is far different from the traditional globe. This application aims to introduce a unique augmented environment where learners can observe different views of the earth’s surface with the help of 3D interactive models. Students can also learn about different countries of the world and find their location on the surface of the globe. Students can learn about the physical location of countries on the globe. As this application uses marker-based AR, we define different countries’ flags as target images for visualizing a unique virtual 3D globe. These image targets can be downloaded from the link given in the application. This application has two AR session scenes. In the AR Globe scene, the earth’s surface is studied and observed with 3D virtual interactive globes. Different countries and their boundaries, locations, areas, and all other information are presented in the second scene. Students can view the country area covered with the flag for the corresponding targeted flag. Students can interact with these 3D models of the globe, and they can rotate, scale, and translate in the scene view to observe from a different angle of view. There is also textual information for learning purposes. Scenes from “AR Globe” are shown in Figure 8.

4. Experimentation and Results

The objectives of our proposed approach are to evaluate the effect of learning content developed with AR-based techniques compared to traditional learning approaches on primary school students in learning basic concepts, such as English alphabet, numbers, animals, birds, and globe study. In this modern era, using such technology for training learning in education fields motivates students to learn more than traditional learning methods. AR learning techniques make it possible to educate students in an attractive, inspiring, and engaging way.

4.1. Experimental Procedure and Participants

Our experimental procedure and data analysis techniques are based on the methods followed by Khan et al. [61], Sahin et al. [30], Lu et al. [62], Redondo et al. [63], Chiang et al. [55], and Chen et al. [64]. We use a quantitative research method based on Quasi-Experimental (QE) design. In QE designs, groups of the participants are arranged on nonrandom criteria [65]. In our study, participants are divided into three different groups as Group-A, Group-B, and Group-C. Participants in each group have an equal level of learning skills and grades. The activity is performed in local schools in Peshawar, Pakistan, and the participants of the activity are selected from the primary sections of two different schools. Fifty students are assigned to each group having participants with the same level of academic achievements. This decision was made based on their previous academic grades provided by their affiliated schools.

Each group is assigned to a teacher and performed a learning activity with a different method of teaching. In Group-A, the students perform the learning with the help of our designed AR applications. A tablet is provided to each student of the group along with image targets and is guided by the teacher about the application. Similarly, the participants of Group-B performed the learning session of the same syllabus by watching videos and presentations. In Group-C, the learning activity is performed with the help of the traditional book-based method. The experimental procedure of the study is shown in Figure 9.

In this activity, a total of one fifty students participated from two different schools. All these students belong to the primary section, with an average age of seven years. These students are classified, considering their previous academic marks as pre-test marks. The Mean (M) value of Group-A grades is 73.5, with a Standard Deviation (SD) of 8.5. Similarly, Group-B participants have average grades with M = 75.1 and SD of 9.2. In Group-C, students have average marks with M = 74.3 and SD = 7.8. Table 1 shows the participant sampling in different groups.

4.2. Learning Materials

The learning materials of Group-A include a tablet having the designed AR applications. A teacher is responsible for guiding the students about all these application’s usage and presenting the demos. Students are trained about using the different image targets for visualizing the AR content. Demos are presented for all the applications with the help of a projector screen. After that, students practice on these applications. We select various video lectures and presentations for the same syllabus for Group-B as provided to Group-A via the tablet. These video lectures are played on a projector screen. One of our team members presents the learning materials and guides the participants. Similarly, in Group-C, based on the traditional learning method, the participants learn through paper-based notebooks. The teacher uses a whiteboard for presenting the students with abstract concepts that are particularly difficult to understand.

4.3. Results Analysis and Discussion

We perform three different types of learning activities based on different methods of teaching and learning. Students are divided into three different groups, as discussed in Section 4.1. In the section ahead, each learning method is evaluated and discussed with the help of statistical data from different questionnaires, quizzes, and interviews.

4.3.1. The Attitude of Students towards AR

In this study, we use an attitude measuring scale called the augmented reality applications attitude scale (ARAAS) introduced by Küçük et al. [66]. The ARAAS will help us determine the students’ attitude towards the AR applications developed in this study compared to other learning methods. The ARAAS model consists of three dimensions with fifteen items. These dimensions are willingness, satisfaction, and anxiety. The willingness aspects of the scale reflect the students’ desire to use the specific method in the future for learning purposes. The satisfaction aspect of the study determines whether the learning methods are useful and easy for learning purposes. Anxiety is the measure of the doubt in the student’s mind using the learning methods. A higher level of satisfaction and willingness to perform learning methods in the future presents a positive attitude towards the teaching and learning method. Similarly, the higher level of anxiety shows a negative attitude towards the learning methods. The ARAAS model is implemented on a five-point Likert scale; five corresponds to “strongly agree”, and one corresponds to “strongly disagree”.

We prepare an attitude measuring questionnaire for each group of students to analyze their attitude towards the learning method. These questionnaires are also conducted at the end of each learning activity. In order to determine a significant difference between the attitude of the students in different groups, One-way analysis of variance (ANOVA) test is conducted [67]. Table 2 shows the statistical testing findings of the participants of different groups. The results show a significant difference in the means of the groups as the p-values are significantly less than the alpha value (p < 0.05).

Students showed an inclination towards AR-based methods with higher average scores for the dimensions of willingness and satisfaction. Many of the students showed willingness (M = 4.36 and SD = 0.65) and satisfaction (M = 4.02 and SD = 0.78), while inconsiderable anxiety towards the AR course (M = 1.80 and SD = 0.74). This study examines a significant difference in the attitude of the students towards other methods of learning. Students in Group-B and Group-C showed lower positive attitude scale values of willingness and satisfaction towards the learning method. While on the other hand, the negative attitude scale of anxiety has higher values, as shown in Table 2.

The learning environment is provided with a friendly and exciting way in AR activity to the students. They are fully engaged in the learning environment where they are enjoying the process of learning. The 3D models and their animations of the AR applications related to the course material fascinated the students. One of the students says, “Look, I am playing with the lion; he is like my pet”. Another student shouts, “I have the power to move an elephant on my hand”. Students enjoy watching the 3D model of the earth on the table in front of them. They can locate a country’s location on the earth’s surface with the help of the country’s flag as a target image for an AR camera.

During the AR learning session, we observe the student’s behavior and their attitude towards learning with AR technology. Our observations show that students have a very positive attitude towards AR-learning techniques, and they do not show any negative expressions, while performing learning activities. Students are enjoying and playing with the 3D visuals on the surface of their books and the table. Students start learning very easily and quickly in a short time. They explore these applications and do not face any difficulties, while using AR-based applications. It can be concluded that student who learns via AR applications has better attitudes towards the course, and therefore, achieves higher scores at the end of the session. In short, we find the AR applications helpful for the learning purposes of the students. Students are also in favor of the AR techniques, and they achieve a higher number of scores compared to the previous traditional methods of learning based on papers and textbooks.

4.3.2. Learning Motivation

The ARCS Model of Motivation is used as a questionnaire to analyze participants’ motivation towards learning [68]. The ARCS model has four dimensions based on attention, relevance, confidence, and satisfaction. The ARCS model’s attention aspect was to measure the participant’s attention during different types of learning methods using different learning materials for the same syllabus. The relevance aspect of the model is to assess the relevance of the learning materials provided to them in different learning activities. The confidence hypothesis measures the confidence level of the students, while performing learning through different methods. Similarly, the hypothesis of satisfaction is the ARCS model of motivation measures the student’s satisfaction during learning in different teaching and learning methods. In questionnaires, we use a five-point Likert scale for the ease of the students and precision: Value 1—strongly disagree, value 2—disagree, value 3—an undecided statement, value 4—agree, and value 5—strongly agree. On a Likert scale, a higher average value indicates a higher motivation towards the learning process.

Questionnaires were conducted at the end of the learning activity in each group. Table 3 has a complete description of the dimensions used in the ARCS motivational model to evaluate motivation. Comparatively, Group-A has the highest average values for the ARCS model’s relative attention, relevance, confidence, and satisfaction aspects. The statistical test result shows a significant difference between the motivations for learning in participants of different groups (p < 0.05).

The students of Group-A show great attention to learning during the activity. They really enjoy the virtual academic materials that emerge on their textbooks and table. The students enjoy and are amused by playing with the animals and other virtual models. While performing learning through the AR methods, the students start using applications by themselves and explore all the learning content developed within the applications. Students show a high rating of relevance hypothesis in the AR learning group because the AR applications provide relevant and significant information toward syllabus based on real world phenomena. Similarly, in the confidence context, Group-A participants are quite confident because the AR technology provides them support in integrating the real and virtual world. It helps to face the challenges, while understanding abstract concepts, such as studying the earth, observing wild animals, and learning numbers. Overall, the satisfaction level of Group-A participants is high with using advanced techniques of learning.

Statistical data of the ARCS model of motivation shows that Group-B and Group-C participants show lower motivation and interest in learning. As in Group-B, the learning methods are based on watching videos and presentations presenting to them by a mentor. They cannot interact with the learning materials provided to them, which fails to gain the students’ attention. The students quickly lose interest; some are found sleeping, while others distractedly look at the charts on the walls. It is the same in the case of Group-C activity, the students do not show high motivation for learning. The teacher is also unable to gain the attention of all the students during the learning process. Moreover, he is unable to explain the topic to satisfy every student in detail.

4.3.3. Learning Achievements

We analyze the students’ learning achievements and performance in different study groups having different learning methods. For this purpose, we conduct different PEQs for each group at the end of the learning session. These PEQs are based on the syllabus presented to them in the learning activities.

For Group-A, proposed AR learning applications have quizzes that evaluate students learning performance from the corresponding application. These quizzes have image-based MCQs type questions about what they have learned from these applications. The quiz in “Nursery Rhyme” is based on the exercises that are given in the academic book. This quiz is based on the alphabet in which students have to match a letter with its particular object. For example, for the letter “A”, we have different images, such as ball, car, apple, and drum. Then, students must select the correct object whose name starts with the letter “A” to get a star. If the user selects the right answer, he or she will be awarded a star. Similarly, in “AR Numbers”, we have a quiz based on counting objects in images. An image is given containing objects, and the user has to select the right number for the objects in that image. In the quiz section of AR Zoo, the user has to guess the name of animals given in that image in the given question.

In Group-B and Group-C, these PEQs were conducted with pages with the same number and type of questions. Table 4 presents the statistical test result of PEQs performed by Group-A, Group-B, and Group-C. Results show a huge significant difference (p < 0.05) in the learning gains of students between different groups (F = 209.51, p < 0.05).

We show that AR-based learning methods are more helpful to the students to improve their learning skills and knowledge gain. Participants of Group-A ranked first in learning achievements (M = 90.92 and SD = 6.09), the Group-B candidates ranked at the second position (M = 70.02 and SD = 7.92). Students in Group-C show lower learning achievements in PEQs (M = 56.34 and SD = 10.62). Figure 10 shows the comparative graph of the learning achievements of the students in PEQs conducted in different learning activities.

In Group-A, the AR learning materials boost the students’ learning process. AR technology attracts the student’s attention and motivates them towards learning. Students in AR learning sessions are attentive towards the learning that helps in understanding the course content. Students in AR activities enjoy the learning process and tries to explore the magical appearance of the 3D models in their textbooks. The transformation of the book images into 3D realistic models amuse the students that are remarkable in gaining students’ attention. With the help of AR technology, students can see and interact with 3D models of the abstract concepts discussed in their syllabus that are more helpful than 2D videos and images. In AR applications, the audio mixing and animations into the 3D models cause a more realistic impression on the students’ observing behavior. Students participate in AR learning activities with an active mind and try to explore new things.

In short, the learning materials provided with AR technology enhanced the student learning gains more effectively than other traditional methods of learning. In traditional learning methods, such as 2D videos and textbooks, the students could not focus entirely on learning. They are involved in other activities, such as playing with books and drawing on their textbooks. Students are not entirely involved in the learning process that results in lower achievements in knowledge gain.

4.3.4. Response from Teachers and Parents

Teachers and parents of the participants observed the learning sessions performed during the activity. At the end of the activities, teachers and parents are asked to share their observations and views towards the proposed AR learning method compared to the other traditional learning methods presented in the activity. Parents and teachers of the students are interviewed and asked about their points of view and opinions about the different learning activities performed during the sessions. Parents and teachers share their views about the AR learning applications that are discussed as under.

The overall feedback of the parents and teachers is in favor and supports the AR technology for the purpose of teaching and learning. Both the parents and teachers present relatively positive and high acceptance towards the AR applications as a learning tool for their children. They find it beneficial in enhancing their kid’s skills and knowledge. Teachers find it helpful and attractive as a teaching tool to easily and effectively handle several students. Teachers will be able to gain the student’s attention towards learning and visually explain topics to them. Further, they state that as with technology advancement, the introduction of AR technology to education is a good and helpful step to the young generation. This technology is quite realistic and informative for learning purposes. Teachers like the independent learning capability of the students while learning through these AR applications. When the students are properly guided about the applications and their usage, the students could learn and explore the learning content by themselves.

The parents appreciate the idea of introducing AR technology in education. They show a positive response towards the methods and find them beneficial and motivating for the student’s learning purpose. Parents of the participants are quite satisfied after observing the AR learning activity results compared to other traditional methods of learning. Parents are pleased after seeing their students learning through their tablets instead of playing games. Parents say that this technology makes it possible for students to observe the static study materials as live 3D representations. Parents recommended these applications for their children’s learning purpose using smartphones under the supervision of the teachers. With this, their students can focus more on learning instead of being wasting time playing games or watching videos. Further, they state that schools should implement these types of technologies in classrooms to motivate our students to learn and enhance their learning skills. This technology’s main advantage is that children are completely engaged in learning activities that develop a learning motivation in students. Notably, it is tough for teachers to gain a student’s attention toward learning at this young age. Creating an enjoyable and interactive environment between the students and the course content helps enhance students’ motivation towards learning.

5. Conclusions and Future Directions

In this paper, we used handheld marker-based AR technology for the learning purposes of primary school students as an effective learning method. We developed a set of four applications for learning of English alphabet, decimal numbers, animals, and learning about the globe. We performed different types of learning activities with various learning methods and evaluated the students’ learning performances. Comparative analysis of these activities revealed a positive impact of AR-based learning methods on children. A supportive effect was noted on the student’s motivation towards the learning process during AR learning sessions. Students also showed a positive attitude and behavior towards the AR learning method. As compared to the traditional learning methods, AR learning techniques make the learning process easy, fast, and enjoyable. In traditional methods, students could not interact with the educational content, resulting in a lack of focused learning. Interaction in real-time enhances the children’s excitement to explore the learning materials more. Students were pleased in the AR learning session and did not show any signs of boredom while learning. Students in Group-A showed a higher level of motivation towards the learning process. They showed higher scales for the ARCS motivational model dimensions as compared to the students in other groups of traditional learning. The confidence and satisfaction level of Group-A participants was high using advanced AR applications, providing higher learning achievements. On the other hand, the traditional learning methods participants showed lower motivation and interest in learning. As in Group-B and Group-C, students could not interact with the learning materials provided to them and failed to enhance students’ motivation for learning. As a result, the comparative results from the PEQs of the different learning activities showed high variance. Further, the response from the teachers and parents was positive and praised the AR learning techniques. Both the teachers and parents favor using AR techniques for learning and teaching.

As we targeted only primary school students, future work makes a path for advanced learning for higher-level students, such as middle and high school students. In AR learning methods, the content should be interactive so that students will be able to explore and play with the content. AR technology can be incorporated into different courses related to abstract concepts that are particularly difficult to understand and teach in real-time. The AR applications should be developed under the nature and constraints of an educational institute so that these methods follow the same learning content that an institute follows. This will be more helpful if the same learning is followed in both the traditional and AR learning activities. Another main point for future work should be to determine what type of course or which activities should be adopted for learning through AR technology. In future research, it is important to notice what type of educational experience fits with this type of learning and teaching medium. AR applications should be designed with proper curriculum and pedagogy having intelligent evolution quizzes and learning games for students to monitor and test the students learning experience and progress. It will be helpful for higher-level students to augment their academic subjects into AR. In the case of tourism and cultural heritage, AR technology could be used as a tool for a better experience with the completely virtual environment of the ancient sites and buildings. In museums, AR technology can be efficiently incorporated for visualizing declined assets or animals for attracting visitors. For AR virtual visuals, we use marker-based AR as it will be easy to handle for the primary level. Similarly, for the latest mobile technologies, location-based and markerless-based AR techniques should be adopted. Next, these mobile-based applications can be assembled for AR HMDs devices that feel more realistic and user-friendly.

Author Contributions

Conceptualization, A. and N.K.; Data curation, K.M.; Formal analysis, A.S.I., M.-Y.L. and M.S.; Funding acquisition, A.S.I. and M.-Y.L.; Methodology, A. and K.M.; Project administration, A.S.I., M.-Y.L. and M.S.; Software, A. and N.K.; Supervision, M.S.; Validation, A. and N.K.; Visualization, K.M.; Writing—original draft, A. and N.K.; writing—review and editing, A. and N.K. All authors have read and agreed to the published version of the manuscript.

Funding

This research is supported by: (1) This work was carried out during the tenure of an ERCIM ‘Alain Benoussan’ Fellowship Programme under the Contract 2019–40, and (2) Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (2018R1D1A1B07043302).

Data Availability Statement

Not appliable.

Conflicts of Interest

The authors declare no conflict of interest.

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Figures and Tables

Figure 1. The complete architecture of the Vuforia software development kit. This Vuforia platform consists of a Vuforia engine and a web-based Vuforia target management system where the target images are uploaded. The cloud target databases are accessed directly using web services, while the device-based database is downloaded from the developer portal.

Figure 2. Designing and developmental phases of the proposed applications consist of a target management system where the target images database is prepared. All the application assets, 3D models, C# scripts, audio clips are prepared and merged into the Unity 3D engine. The final application is built as android or iOS.

Figure 3. Feature points in image targets that are used by the Vuforia engine to recognize predefined targets.

Figure 4. Working on AR-based mobile applications. Predefined target images are scanned with the AR camera using a smartphone. Upon target recognition, 3D visuals are augmented on the camera screen.

Figure 5. Screenshots of different scenes from the application “Nursery Rhyme” for learning of English alphabet.

Figure 6. Screenshot visuals of the application “AR Numbers” where students learn decimal numbers.

Figure 7. Images of different AR camera scenes and quiz performances from the application “AR Zoo”.

Figure 8. Different AR camera scenes of the AR Globe application where the 3D globe is augmented on the camera preview.

Figure 9. Pre-session arrangement of the groups based on the student’s previous academic records, and activities plan of the experimental study.

Figure 10. Student’s learning achievements in performance evaluation quizzes.

Table 1

Groups arrangement based record of the previous academic grades of the participants in their academic sessions.

Group	No. of Students	Boys:Girls	Mean	Standard Deviation
Group-A	50	33:17	73.5	8.5
Group-B	50	28:22	75.1	9.2
Group-C	50	30:20	74.3	7.8

Table 2

Record of the One-way ANOVA tests conducted for students’ attitudes in different groups.

Variable	Group	Mean	Standard Deviation	F-Value	p-Value
Willingness	Group-A	4.36	0.65	137.21	2.40802 × 10−34
	Group-B	2.54	0.75
	Group-C	2.04	0.77
Satisfaction	Group-A	4.02	0.78	164.67	2.95426 × 10−38
	Group-B	1.88	0.79
	Group-C	1.54	0.61
Anxiety	Group-A	1.80	0.74	122.32	5.2641 × 10−32
	Group-B	4.16	0.90
	Group-C	3.98	0.83

Table 3

One-way ANOVA test results for different variables of the ARCS motivational model.

Variable	Group	Mean	Standard Deviation	F-Value	p-Value
Attention	Group-A	4.38	0.62	146.43	1.03374 × 10−35
	Group-B	2.54	0.75
	Group-C	2.02	0.76
Relevance	Group-A	4.36	0.65	216.20	1.65491 × 10−44
	Group-B	2.16	0.83
	Group-C	1.54	0.61
Confidence	Group-A	4.24	0.73	119.33	1.62856 × 10−31
	Group-B	2.14	0.89
	Group-C	2.04	0.74
Satisfaction	Group-A	4.36	0.62	165.15	2.55212 × 10−38
	Group-B	2.24	0.76
	Group-C	1.88	0.79

Table 4

One-way ANOVA tests result for learning achievements during different activities.

Group	Mean	Standard Deviation	F-Value	p-Value
Group-A	90.92	6.09	209.51	9.21763 × 10−44
Group-B	70.02	7.92
Group-C	56.34	10.62

Supplementary Materials

The following supporting materials are available on the Internet: Demo video of the applications: https://bit.ly/2SRoGzK; Nursery Rhyme book: https://bit.ly/3pbKfay; AR Numbers target images: https://bit.ly/3g0Jv3W; The textbook followed for AR Numbers application: https://bit.ly/2SNDrDO; AR Zoo image targets: https://bit.ly/3yXusjZ; AR Globe image targets: https://bit.ly/3c7N2w7; Sample of PEQs of Group-B and Group-C: https://bit.ly/34DOFxx.

References

1. Poux, F.; Valembois, Q.; Mattes, C.; Kobbelt, L.; Billen, R. Initial User-Centered Design of a Virtual Reality Heritage System: Applications for Digital Tourism. Remote Sens.; 2020; 12, 2583. [DOI: https://dx.doi.org/10.3390/rs12162583]

2. Quintero, J.; Baldiris, S.; Rubira, R.; Cerón, J.; Velez, G. Augmented reality in educational inclusion. A systematic review on the last decade. Front. Psychol.; 2019; 10, 1835. [DOI: https://dx.doi.org/10.3389/fpsyg.2019.01835] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/31456716]

3. Čejka, J.; Bruno, F.; Skarlatos, D.; Liarokapis, F. Detecting square markers in underwater environments. Remote Sens.; 2019; 11, 459. [DOI: https://dx.doi.org/10.3390/rs11040459]

4. Alidoost, F.; Arefi, H.; Tombari, F. 2D image-to-3D model: Knowledge-based 3D building reconstruction (3DBR) using single aerial images and convolutional neural networks (CNNs). Remote Sens.; 2019; 11, 2219. [DOI: https://dx.doi.org/10.3390/rs11192219]

5. Azuma, R.T. A survey of augmented reality. Presence Teleoperators Virtual Environ.; 1997; 6, pp. 355-385. [DOI: https://dx.doi.org/10.1162/pres.1997.6.4.355]

6. Koh, R.K.C.; Duh, H.B.-L.; Gu, J. An integrated design flow in user interface and interaction for enhancing mobile AR gaming experiences. Proceedings of the 2010 IEEE International Symposium on Mixed and Augmented Reality-Arts, Media, and Humanities; Seoul, Korea, 13–16 October 2010.

7. Johnson, L.; Levine, A.; Smith, R.; Stone, S. The 2010 Horizon Report; ERIC: 2010. The 2010 Horizon Report New Media Consortium: Austin, TX, USA, 2010.

8. Fan, Y.; Feng, Z.; Mannan, A.; Khan, T.U.; Shen, C.; Saeed, S. Estimating tree position, diameter at breast height, and tree height in real-time using a mobile phone with RGB-D SLAM. Remote Sens.; 2018; 10, 1845. [DOI: https://dx.doi.org/10.3390/rs10111845]

9. Boonbrahm, S.; Kaewrat, C.; Boonbrahm, P. Using augmented reality technology in assisting english learning for primary school students. International Conference on Learning and Collaboration Technologies; Springer: Berlin/Heidelberg, Germany, 2015.

10. Chang, G.; Morreale, P.; Medicherla, P. Applications of augmented reality systems in education. Proceedings of the Society for Information Technology & Teacher Education International Conference; San Diego, CA, USA, 29 March–2 April 2010.

11. Brookes, F.; Zazryn, T. Learning: Primary and Secondary School Years. Raising Children Network. Available online: https://raisingchildren.net.au/school-age/school-learning/learning-ideas/learning-school-years (accessed on 24 June 2019).

12. Divjak, B.; Tomić, D. The impact of game-based learning on the achievement of learning goals and motivation for learning mathematics-literature review. J. Inf. Organ. Sci.; 2011; 35, pp. 15-30.

13. De Freitas, S.; Rebolledo-Mendez, G.; Liarokapis, F.; Magoulas, G.; Poulovassilis, A. Learning as immersive experiences: Using the four-dimensional framework for designing and evaluating immersive learning experiences in a virtual world. Br. J. Educ. Technol.; 2010; 41, pp. 69-85. [DOI: https://dx.doi.org/10.1111/j.1467-8535.2009.01024.x]

14. Chao, P.-Y.; Chen, G.-D. Augmenting paper-based learning with mobile phones. Interact. Comput.; 2009; 21, pp. 173-185. [DOI: https://dx.doi.org/10.1016/j.intcom.2009.01.001]

15. Huang, H.-W.; Wu, C.-W.; Chen, N.-S. The effectiveness of using procedural scaffoldings in a paper-plus-smartphone collaborative learning context. Comput. Educ.; 2012; 59, pp. 250-259. [DOI: https://dx.doi.org/10.1016/j.compedu.2012.01.015]

16. Taketa, N.; Hayashi, K.; Kato, H.; Noshida, S. Virtual pop-up book based on augmented reality. Symposium on Human Interface and the Management of Information; Springer: Berlin/Heidelberg, Germany, 2007.

17. Hantono, B.S.; Nugroho, L.E.; Santosa, P.I. Meta-review of augmented reality in education. Proceedings of the 2018 10th International Conference on Information Technology and Electrical Engineering (ICITEE); Bali, Indonesia, 24–26 July 2018.

18. Nincarean, D.; Alia, M.B.; Halim, N.D.A.; Rahman, M.H.A. Mobile Augmented Reality: The potential for education. Procedia Soc. Behav. Sci.; 2013; 103, pp. 657-664. [DOI: https://dx.doi.org/10.1016/j.sbspro.2013.10.385]

19. Barreira, J.; Bessa, M.; Pereira, L.C.; Adão, T.; Peres, E.; Magalhães, L. MOW: Augmented Reality game to learn words in different languages: Case study: Learning English names of animals in elementary school. Proceedings of the 7th Iberian Conference on Information Systems and Technologies (CISTI 2012); Madrid, Spain, 20–23 June 2012.

20. Radu, I. Why should my students use AR? A comparative review of the educational impacts of augmented-reality. Proceedings of the 2012 IEEE International Symposium on Mixed and Augmented Reality (ISMAR); Atlanta, GA, USA, 5–8 November 2012.

21. Tonnis, M.; Lochhead, I.; Madden, M.; Christophe, S.; Devaux, A.; Pettit, C.; Hedley, N. Experimental evaluation of an augmented reality visualization for directing a car driver’s attention. Proceedings of the Fourth IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR’05); Vienna, Austria, 5–8 October 2005.

22. Çöltekin, A.; Lochhead, I.; Madden, M.; Christophe, S.; Devaux, A.; Pettit, C.; Hedley, N. Extended reality in spatial sciences: A review of research challenges and future directions. ISPRS Int. J. Geo-Inf.; 2020; 9, 439. [DOI: https://dx.doi.org/10.3390/ijgi9070439]

23. Keil, J.; Edler, D.; Dickmann, F. Preparing the HoloLens for user studies: An augmented reality interface for the spatial adjustment of holographic objects in 3D indoor environments. KN-J. Cartogr. Geogr. Inf.; 2019; 69, pp. 205-215. [DOI: https://dx.doi.org/10.1007/s42489-019-00025-z]

24. Tomi, A.B.; Rambli, D.R.A. An interactive mobile augmented reality magical playbook: Learning number with the thirsty crow. Procedia Comput. Sci.; 2013; 25, pp. 123-130. [DOI: https://dx.doi.org/10.1016/j.procs.2013.11.015]

25. Midak, L.Y.; Kravets, I.V.; Kuzyshyn, O.V.; Berladyniuk, K.V.; Buzhdyhan, K.V.; Baziuk, L.V.; Uchitel, A.D. Augmented reality in process of studying astronomic concepts in primary school. Proceedings of the 3rd International Workshop on Augmented Reality in Education (AREdu 2020); Kryvyi Rih, Ukraine, 13 May 2020.

26. Zarzuela, M.M.; Pernas, F.J.D.; Martínez, L.B.; Ortega, D.G.; Rodríguez, M.A. Mobile serious game using augmented reality for supporting children’s learning about animals. Procedia Comput. Sci.; 2013; 25, pp. 375-381. [DOI: https://dx.doi.org/10.1016/j.procs.2013.11.046]

27. Billinghurst, M. Augmented reality in education. New Horiz. Learn.; 2002; 12, pp. 1-5.

28. Radu, I. Augmented reality in education: A meta-review and cross-media analysis. Pers. Ubiquitous Comput.; 2014; 18, pp. 1533-1543. [DOI: https://dx.doi.org/10.1007/s00779-013-0747-y]

29. Shoaib, H.; Jaffry, S.W. A survey of augmented reality. Proceedings of the International Conference on Virtual and Augmented Reality (ICV AR 2015); Brisbane, Australia, 25–27 March 2015.

30. Sahin, D.; Yilmaz, R.M. The effect of Augmented Reality Technology on middle school students’ achievements and attitudes towards science education. Comput. Educ.; 2020; 144, 103710. [DOI: https://dx.doi.org/10.1016/j.compedu.2019.103710]

31. De Lisi, R.; Wolford, J.L. Improving children’s mental rotation accuracy with computer game playing. J. Genet. Psychol.; 2002; 163, pp. 272-282. [DOI: https://dx.doi.org/10.1080/00221320209598683]

32. Merians, A.S.; Jack, D.; Boian, R.; Tremaine, M.; Burdea, G.C.; Adamovich, S.V.; Poizner, H. Virtual reality–augmented rehabilitation for patients following stroke. Phys. Ther.; 2002; 82, pp. 898-915. [DOI: https://dx.doi.org/10.1093/ptj/82.9.898]

33. Tang, A.; Owen, C.; Biocca, F.; Mou, W. Comparative effectiveness of augmented reality in object assembly. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems; Lauderdale, FL, USA, 5–10 March 2003.

34. Lin, C.-Y.; Chai, H.C.; Wang, J.Y.; Chen, C.J.; Liu, Y.H.; Chen, C.W.; Huang, Y.M. Augmented reality in educational activities for children with disabilities. Displays; 2016; 42, pp. 51-54. [DOI: https://dx.doi.org/10.1016/j.displa.2015.02.004]

35. Chang, Y.-J.; Chang, C.S.; Lin, C.Y.; Wang, Y.Z. Investigating students’ perceived satisfaction, behavioral intention, and effectiveness of English learning using augmented reality. Proceedings of the 2011 IEEE International Conference on Multimedia and Expo; Barcelona, Spain, 11–15 July 2011.

36. Safar, A.H.; Al-Jafar, A.A.; Al-Yousefi, Z.H. The effectiveness of using augmented reality apps in teaching the English alphabet to kindergarten children: A case study in the State of Kuwait. EURASIA J. Math. Sci. Technol. Educ.; 2016; 13, pp. 417-440. [DOI: https://dx.doi.org/10.12973/eurasia.2017.00624a]

37. Rambli, D.R.A.; Matcha, W.; Sulaiman, S. Fun learning with AR alphabet book for preschool children. Procedia Comput. Sci.; 2013; 25, pp. 211-219. [DOI: https://dx.doi.org/10.1016/j.procs.2013.11.026]

38. Cieza, E.; Lujan, D. Educational mobile application of augmented reality based on markers to improve the learning of vowel usage and numbers for children of a kindergarten in Trujillo. Procedia Comput. Sci.; 2018; 130, pp. 352-358. [DOI: https://dx.doi.org/10.1016/j.procs.2018.04.051]

39. Nainggolan, E.R.; Asymar, H.H.; Nalendra, A.R.A.; Sulaeman, F.; Radiyah, U. The implementation of augmented reality as learning media in introducing animals for early childhood education. Proceedings of the 2018 6th International Conference on Cyber and IT Service Management (CITSM); Parapat, Indonesia, 7–9 August 2018.

40. Schnürer, R.; Dind, C.; Schalcher, S.; Tschudi, P.; Hurni, L. Augmenting Printed School Atlases with Thematic 3D Maps. Multimodal Technol. Interact.; 2020; 4, 23. [DOI: https://dx.doi.org/10.3390/mti4020023]

41. Wiehr, F.; Daiber, F.; Kosmalla, F.; Krüger, A. ARTopos: Augmented reality terrain map visualization for collaborative route planning. Proceedings of the 2017 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2017 ACM International Symposium on Wearable Computers; Maui, HI, USA, 11–15 September 2017.

42. Lindgren, R.; Moshell, J.M. Supporting children’s learning with body-based metaphors in a mixed reality environment. Proceedings of the 10th International Conference on Interaction Design and Children; Ann Arbor, MI, USA, 20–23 June 2011.

43. Sin, A.K.; Zaman, H.B. Live Solar System (LSS): Evaluation of an Augmented Reality book-based educational tool. Proceedings of the 2010 International Symposium on Information Technology; Kuala Lumpur, Malaysia, 15–17 June 2010.

44. Nischelwitzer, A.; Lenz, F.J.; Searle, G.; Holzinger, A. Some aspects of the development of low-cost augmented reality learning environments as examples for future interfaces in technology enhanced learning. Proceedings of the International Conference on Universal Access in Human-Computer Interaction; Beijing, China, 22–27 July 2007.

45. Quarles, J.; Lampotang, S.; Fischler, I.; Fishwick, P.; Lok, B. A mixed reality approach for merging abstract and concrete knowledge. Proceedings of the 2008 IEEE Virtual Reality Conference; Reno, NV, USA, 8–12 March 2008.

46. Freitas, R.; Campos, P. SMART: A SysteM of Augmented Reality for Teaching 2nd grade students. People Comput. XXII Cult. Creat. Interact.; 2008; 22, pp. 27-30.

47. Morrison, A.; Oulasvirta, A.; Peltonen, P.; Lemmela, S.; Jacucci, G.; Reitmayr, G.; Juustila, A. Like bees around the hive: A comparative study of a mobile augmented reality map. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems; Boston, MA, USA, 4–9 April 2009.

48. Matcha, W.; Rambli, D.R.A. Preliminary investigation on the use of augmented reality in collaborative learning. Proceedings of the International Conference on Informatics Engineering and Information Science; Kuala Lumpur, Malaysia, 14–16 November 2011.

49. Pathomaree, N.; Charoenseang, S. Augmented reality for skill transfer in assembly task. Proceedings of the IEEE International Workshop on Robot and Human Interactive Communication 2005 (ROMAN 2005); Nashville, TN, USA, 13–15 August 2005.

50. Lin, T.-J.; Duh, H.B.L.; Li, N.; Wang, H.Y.; Tsai, C.C. An investigation of learners’ collaborative knowledge construction performances and behavior patterns in an augmented reality simulation system. Comput. Educ.; 2013; 68, pp. 314-321. [DOI: https://dx.doi.org/10.1016/j.compedu.2013.05.011]

51. Yoon, S.A.; Elinich, K.; Wang, J.; Steinmeier, C.; Tucker, S. Using augmented reality and knowledge-building scaffolds to improve learning in a science museum. Int. J. Comput.-Supported Collab. Learn.; 2012; 7, pp. 519-541. [DOI: https://dx.doi.org/10.1007/s11412-012-9156-x]

52. Kaufmann, H.; Dünser, A. Summary of usability evaluations of an educational augmented reality application. Proceedings of the International Conference on Virtual Reality; Beijing, China, 22–27 July 2007.

53. Juan, C.M.; Toffetti, G.; Abad, F.; Cano, J. Tangible cubes used as the user interface in an augmented reality game for edutainment. Proceedings of the 2010 10th IEEE International Conference on Advanced Learning Technologies; Sousse, Tunisia, 5–7 July 2010.

54. Di Serio, Á.; Ibáñez, M.B.; Kloos, C.D. Impact of an augmented reality system on students’ motivation for a visual art course. Comput. Educ.; 2013; 68, pp. 586-596. [DOI: https://dx.doi.org/10.1016/j.compedu.2012.03.002]

55. Chiang, T.H.; Yang, S.J.; Hwang, G.-J. An augmented reality-based mobile learning system to improve students’ learning achievements and motivations in natural science inquiry activities. J. Educ. Technol. Soc.; 2014; 17, pp. 352-365.

56. Vincenzi, D.A.; Valimont, B.; Macchiarella, N.; Opalenik, C.; Gangadharan, S.N.; Majoros, A.E. The effectiveness of cognitive elaboration using augmented reality as a training and learning paradigm. Proceedings of the Human Factors and Ergonomics Society Annual Meeting; Denver, CO, USA, 13–17 October 2003.

57. Herpich, F.; Guarese, R.L.M.; Tarouco, L.M.R. A comparative analysis of augmented reality frameworks aimed at the development of educational applications. Creat. Educ.; 2017; 8, 1433. [DOI: https://dx.doi.org/10.4236/ce.2017.89101]

58. Agustina, W.; Sumarto, S.; Trisno, B. Augmented reality based on stem for supporting science literacy in vocational education. Proceedings of the Journal of Physics: Conference Series; Crete, Greece, 9–15 June 2019.

59. Simonetti Ibañez, A.; Figueras, J.P. Vuforia v1. 5 SDK: Analysis and Evaluation of Capabilities. Ph.D. Thesis; Universitat Politècnica de Catalunya: Barcelona, Spain, 2013.

60. Kim, S.L.; Suk, H.J.; Kang, J.H.; Jung, J.M.; Laine, T.H.; Westlin, J. Using Unity 3D to facilitate mobile augmented reality game development. Proceedings of the 2014 IEEE World Forum on Internet of Things (WF-IoT); Seoul, Korea, 6–8 March 2014.

61. Khan, T.; Johnston, K.; Ophoff, J. The impact of an augmented reality application on learning motivation of students. Adv. Hum.-Comput. Interact.; 2019; 2019, 7208494. [DOI: https://dx.doi.org/10.1155/2019/7208494]

62. Lu, S.-J.; Liu, Y.-C. Integrating augmented reality technology to enhance children’s learning in marine education. Environ. Educ. Res.; 2015; 21, pp. 525-541. [DOI: https://dx.doi.org/10.1080/13504622.2014.911247]

63. Redondo, B.; Cózar-Gutiérrez, R.; González-Calero, J.A.; Ruiz, R.S. Integration of augmented reality in the teaching of English as a foreign language in early childhood education. Early Child. Educ. J.; 2020; 48, pp. 147-155. [DOI: https://dx.doi.org/10.1007/s10643-019-00999-5]

64. Chen, C.-H.; Chou, Y.-Y.; Huang, C.-Y. An augmented-reality-based concept map to support mobile learning for science. Asia-Pac. Educ. Res.; 2016; 25, pp. 567-578. [DOI: https://dx.doi.org/10.1007/s40299-016-0284-3]

65. McMillan, J.H.; Schumacher, S. Research in Education: Evidence-Based Inquiry, My Education Lab Series; Pearson: London, UK, 2010.

66. Küçük, S.; Yilmaz, R.; Baydas, Ö.; Göktas, Y. Augmented reality applications attitude scale in secondary schools: Validity and reliability study. Egitim ve Bilim; 2014; 39, pp. 383-392. [DOI: https://dx.doi.org/10.15390/EB.2014.3590]

67. Park, H.M. Comparing Group Means: T-Tests and One-Way ANOVA Using Stata, SAS, R, and SPSS; The University Information Techology Services (UITS) Center for Statistical and Mathematical Computing, Indiana University: Bloomington, IN, USA, 2009.

68. Keller, J.M. Motivational Design for Learning and Performance: The ARCS Model Approach; Springer: Berlin/Heidelberg, Germany, 2009.