1. Introduction
The digital transformation of the construction industry is progressing at an ever-increasing pace, with the development significantly driven by the use of Building Information Modelling (BIM), a construction analogue to the information age. BIM enables a central administration of digital geometric and alphanumeric information [1,2] that correlates with the real environment. This development is accompanied by further new technologies for the construction industry emerging due to the unprecedented amount of available information. Apart from BIM, systems using augmented reality (AR), virtual reality (VR), robotics, 3D printing, artificial intelligence (AI) and drones [3,4,5] can be found throughout the industry, establishing the term Construction 4.0 [6,7] based on Industry 4.0 [8]. Construction 4.0 focuses primarily on the connectivity and interaction between all the previously mentioned technologies [9], with the BIM model representing their central focus point.
The BIM model itself has great potential for all life cycle phases of a building, e.g., planning, construction, operation [10,11,12,13]. It cannot only be used for simulation, e.g., simulations for scheduling (4D planning) [14], but also for the timely detection of potential problems in the planning phase [15], cost management, construction management, project management, as well as facility management. Furthermore, the data included in the BIM model can be implemented directly within other technologies, e.g., AR.
1.1. AR in the AEC Industry
AR extends the real environment by an additional virtual content [16]. This can be achieved either with portable devices (mAR; mobile augmented reality) or with the help of head mounted displays (HMD). In contrast to VR, visual or auditory elements are projected in AR into the real environment. This extended reality is characterised by the immersion and interaction of the users.
AR shows great potential as a functional extension of the BIM model in the different life cycle phases of a building, especially in the construction phase. The benefits for the AEC industry of both can be found in various studies [17,18,19,20,21,22,23,24,25]. As Wang et al. [26] state: “AR will be the site-extension of BIM concept and approach. AR will maximise the potential of BIM on the construction site, and eventually become a digital toolkit for construction workers.” In addition of using the contained geometric information within the model, the incorporated alphanumeric information can be utilised for, e.g., a virtual 4D simulation (construction time simulation) projected into an existing building. Schranz et al. [24] deal with the use cases of AR in AEC, emphasising the significant added value in many processes. A valuable area of application is the inspection of built structures during and after the construction phase [27,28] for quality control, e.g., for the inspection of concrete and masonry embedment [29] or reinforcement [30]. The effectiveness of the inspection tools with 3D annotations are clearly depicted by Marino et al. [31].
1.2. AR in Education
The previously listed application scenarios lead to the assumption that AR will become an integral part of the AEC industry in the future. It is therefore important to incorporate it, in combination with all the scenarios, into the AEC education. Cacón [32], for example, presents the integration of Construction 4.0 content into a new university degree of Technologies in Civil Engineering. With regard to the application of AR, the implementation as a new technology with the corresponding use cases or as a technology to support education should be envisaged. In many cases, both of these applications go hand in hand, with AR offering many advantages as an assistive technology in both classrooms and on site. According to Dohse et al. [33], AR can significantly improve spatial awareness with its visualisation and animation. It should be noted that it does not aim to replace conventional materials but rather enhance them.
One of the most essential parameters for successful learning is the students’ motivation [34], a process which can be increased, according to Khan et al. [35], by the use of AR. The technology further improves the ability to experience learning situations [36] or understand complex issues through the projection of virtual models into the real environment [37]. Jan et al. [38] developed the application “mARble” for training students in forensic medicine. The comparison of the learning effects using “mARble” and conventional textbooks showed a favourable outcome for the new technology [39]. Further studies [40,41] confirm the improved learning results when utilising AR.
Since the beginning of the COVID-19 pandemic, classroom teaching in higher education has been discontinued or limited in most countries. The absence of face-to-face communication between students and teachers often leads to learning problems, especially for first-year students. In classroom teaching misunderstandings of spatial problems can be directly discussed and solved by the participants [42]. If one considers the current training of future engineers, the spatial interpretation of 2D images is an important aspect [43,44] and is necessary when looking at building details and plans. However, the spatial explanation is omitted in distance learning, attributing an even higher importance to the comprehensible presentation of teaching content.
The unlimited possibilities of this disruptive technology applied within education has also been recognised by companies. The formwork manufacturer Doka, for example, developed its AR app for Doka formwork systems [45]. In addition to the improved spatial imagination, the app reduces learning barriers using central data management, e.g., allowing for a quick adjustment of the languages.
Vasilevski et al. [46] conclude that mixed reality (VR, AR), especially in combination with BIM, increases the quality of the learning environment and thus the learning success in construction engineering training. Although the positive effects of the usage of AR and VR on learning success is proven, Niedermeier et al. [36] note that the usage of this technology is low, especially in higher education. Yuen et al. [41] criticise that education is often used as a business model and is therefore not freely accessible. This often poses problems for teaching staff, as access to financial resources at educational institutions is limited.
Currently, various software products enabling the easy creation of simple AR models can be found on the market. Turkan et al. [47] present a mobile AR app for teaching structural analysis, with ARTutor [48] and ARki by Darf Design [49] being further examples. Even though elaborate AR teaching scenes for AEC students are not included, ARTutor allows for the implementation of general AR content in textbooks for distance learning. The fee-based ARki software, on the other hand is designed explicitly for architectural models and allows some animation with the import limited to FBX (Filmbox) files, resulting in the vital import option of Industry Foundation Classes (IFC) files from BIM models missing. When considering the current state of the art the creation of more elaborate AR teaching scenes in AEC education without programming skills is thus tricky or costly.
This publication describes the development, testing, and evaluation of AR in civil engineering based on specific use cases. The investigation is based on the free AR platform “AR-supported Teaching” [50] explicitly developed at the TU Wien for the purpose of bringing AR to the AEC community. “AR-supported Teaching” consists of an AR editor, a web application, and an AR viewer. On the one hand, the AR editor can be used in teaching, allowing lecturers to create elaborate AR teaching scenes without programming knowledge. On the other hand, it finds its application within the AEC industry for generating instructions for the construction site or for construction meetings. Furthermore, the presented AR platform is created to bridge the gap between AR and openBIM, allowing for the design of models with any BIM authoring software. “AR-supported Teaching” is applicable both as a teaching support tool and for teaching the productive use of AR tools. The main target group for the development is students from universities with a major in civil engineering or architecture since basic knowledge in BIM is necessary for the creation of the models.
2. Background of the Software Development
In this section, the background and framework conditions for developing the AR platform “AR-supported Teaching” for the construction industry are explained. “AR-supported Teaching” is the continuation and extension of the software “IFC Desktop Editor” and “IFC AR App” software developed by Höbart [51].
The use of the AR platform in the context of AEC and education defines the requirements thereof. When applied in education several challenges are faced according to Davila Delgado et al. [52]. Due to the fact that only a limited number of people create AR models, a small amount of free content of existing AR applications in teaching is available. This problem is further exacerbated by the current lack of evaluation studies in the area of the learning impact of AR on students as well as of quality standards.
The basis of every AR scene is a 3D model. Most software solutions for creating models in the construction industry already support the openBIM standard IFC, allowing for an exchange of model information across software. Through the increased use of openBIM in the sector, the amount of created BIM (model) content rises, creating a library of information that could further be used for other applications. Therefore, the basis for the AR platform should also be IFC. Models can be created in any specialised software by experts and imported as IFC models into the AR platform for further use. This application of already existing modelling software would reduce the training costs for using the AR platform and increase the expert’s motivation of using it within their projects.
High usability accompanied by time unique possibilities within the creation of AR scenes further increase the willingness to generate AR content. Various software development kits (SDK) [53,54,55,56] for the creation of individual free AR teaching solutions exist; however, users need programming skills to apply SDK in applications useful for teaching. This poses a significant challenge for widespread use in education and reduces usability. According to a study on digitalisation in schools [57], only 21% of the teaching staff surveyed have programming skills. Furthermore, 62% of them see their lack of IT or digital skills as a hurdle to using digital media in the classroom. As a result, there is a strong call for a free, simple platform that can be used without programming skills.
The amount of AR content can also be increased by using financial means. The users pay for the activation of the AR content protected by a paywall in apps. Educational institutions, in particular, often have a tight budget and cannot afford these kinds of surcharges, calling for a new approach to content creation. With software that is easy to use without programming knowledge, both teachers and students can create AR content, making a peer-to-peer system possible. In this system, students create teaching scenes for other students, which can be subsequently expanded and improved. The advantage over the expert method is the significantly higher number of students and, thus, the higher number of models generated free of charge. Due to the temporal proximity of the students’ knowledge acquisition, they gain better understanding of the problems and uncertainties of the presented topic area (= scaffolding for users). Furthermore, the students consolidate their knowledge through repetition and recapitulation of the learning material. The disadvantage, however, is that even greater importance must be attached to quality assurance. This can be achieved through an evaluation system: The users of the AR teaching scenes should be able to rate the AR model on a Likert scale and provide feedback.
3. Materials and Methods
The research project “AR-supported Teaching” aims to solve the difficulties mentioned above and thus create an AR teaching platform tailored to construction engineering education. Therefore, the authors defined the software’s goals, requirement, and structure based on the aforementioned framework. The usability of the developed platform was tested for various use cases. Thus, the authors used a test setup with questionnaires. The feedback of the test persons helps to evaluate whether the goal was achieved and to improve the platform.
This section explains the platform development. Then, the test setup and the evaluation method for the case study, which was carried out using this AR platform, as well as the use cases are described.
The requirements for the AR platform are:
openBIM: Use of the openBIM standard IFC as the data exchange format.
Simple usability: Creating an easy-to-use development environment that enables people without in-depth programming knowledge to create AR content (e.g., work steps, comments).
openContent: Development of a web application for project management. Users can manage AR content on the web application (versioning), share it and let other users comment and develop it further. Following GitHub [58] projects, this is intended to promote the emergence of openContent for AR in the construction industry.
AR viewer: Development of a platform-independent AR viewer for smartphones or tablets and subsequently for HMD (see [59]).
3.1. AR-Platform Development
The AR platform consists of three components to achieve the intended goals: AR editor, AR viewer, and a web application. The structure and interfaces between the components are shown in Figure 1. The web application is used for project and role management, where users are given different permissions based on a hierarchical role matrix. A distinction is made between user (only views projects in the AR viewer), editor (creates projects) and group admin (manages the group and assigns users/roles). Furthermore, it is possible to organise groups in a tree structure and assign users to groups in a system-controlled way, ensuring that users can only access teaching scenes that have been released for them.
With the help of the AR editor, users can create their AR teaching scene in the role of the editor. The model, a 3D (IFC) model created in a native software, is expanded for the AR display with further functions (animations, annotations, fade-in/fade-out via buttons) and content (texts, images, videos). The created AR project can then be transferred to a database and becomes available to the AR viewer. The web service is the link between AR editor and AR viewer, allowing for the teaching scenes to be made available to the viewers on their mobile devices. App users can download created projects to their mobile AR-enabled device and start directly with user-controlled viewing. ARCore [53] is used to position the AR models in the environment. This framework enables the integration of the virtual content in the real environment and uses motion tracking for this purpose. In addition to ARCore, the AR Foundation package is used [51]. The AR viewer is primarily optimised for tablets and smartphones, not HMD, which are currently barely used due to the high costs and software reliability [60].
Live interaction between teachers (AR editor) and students (AR viewer) is made possible by an integrated text and voice chat function (= live session). Furthermore, teachers can place additional markers (addition of annotations) during a live session and link them to images and videos.
3.2. AR Editor and AR Viewer
The AR editor forms the basis for displaying the AR model in the AR viewer and enables people without in-depth experience in programming to create AR models for different tasks. The AR editor was created with a game engine and features an IFC import, several tools to manipulate or extend the model, and an export to upload the IFC files alongside a JSON file (JavaScript Object Notation) containing additional information and optional media files. Additionally, the editor can open existing projects from the server, allowing users to update them or build upon them to create a new project. The import of an IFC file is split into two pipelines, as shown in Figure 2a: importing the geometric information (Trilib loader) and importing the non-geometric information (IfcOpenShell) [51].
In the AR editor, the IFC model for the AR display is extended as previously stated by additional functions and content. IFC elements, annotations, and animations can be assigned to the buttons, which are shown or hidden when pressed (Figure 3). The AR teaching scenes are transferred to the AR viewer and displayed in AR on the mobile device.
The AR editor offers the possibility of assigning different buttons (reinforcement layer 1, reinforcement layer 2) to reinforcement layers, in order to display them step by step in the AR viewer (see Figure 4). Users can easily add user-defined animations to the elements without any programming knowledge. The animations are triggered by a button, presenting for example an animated display of the correct placement of rebars in the AR viewer. Construction processes can thus be explained and visualised step by step. Buttons and animations allow users to explore an AR model in an exploratory and self-directed manner, without predefined steps set by the creator of the model, and therefore ensure that cognitive overload is prevented by excessive information being presented at once.
After finishing all edits, a user can export the IFC project to the server (Figure 2b) where it is then available to the users. The export is executed by packaging all imported information into a folder and creating a JSON export file containing lists for IFC elements, transformation, animation, annotations, and buttons (the information which elements are shown by each button).
The JSON export and IFC files are retrieved and imported by the AR viewer via the server. The AR viewer is an application for smartphones and tablets and makes the previously created AR scene available to the user. After downloading a project, the user is prompted to position the model in the AR view through a set of steps. Once finished, different building parts can be selected, the scene can be explored, and the attached IFC data plus image, video, animation, annotations, and buttons (see Figure 4), which were previously created in the AR editor, viewed.
3.3. Tests
To evaluate the AR platform “AR-supported Teaching”, several users tested both the AR editor and the AR viewer. The Technology Acceptance Model (TAM) [61], widely applied according to King and He [62], enabled the evaluation by analysing the attitude of the users towards using the product depending on two factors: (a) “Perceived Ease of Use” (EOU) and (b) “Perceived Usefulness” (USEF). USEF is directly dependent on EOU yet can vary according to design features. Granić and Marangunić [63] conducted a literature review on the use of the TAM in education. The authors concluded that the two core parameters (EOU and USEF) can also be used in many cases to determine the acceptability of learning with the technology. The evaluation is carried in two steps, in which the user-friendliness (usability) and the utility are determined separately. In this context, utility represents the increase in learning efficiency. The usability of the applications (AR editor and AR viewer) is recorded in order to continuously improve and adapt the user interface. The second step, assessing the increase in learning efficiency, was not part of this study and will be conducted in future tests.
For a reliable analysis, a relatively small number of respondents is needed as declared by Tullis and Stetson [64], who show that 12–14 people are sufficient to obtain the same results as a more extensive survey in at least 90% of the cases. Twelve people tested the AR editor and 15 the AR viewer while participating in two lectures (“Reinforced Concrete” and “Building Constructions”) at the TU Wien. All interviewed students attend the Bachelor’s degree programme “Civil Engineering”. Figure 5 shows a student using the AR viewer on his smartphone during a lecture.
For each application, an own online questionnaire was created to evaluate the usability. The two questionnaires use standardized statements for calculating the scores of standardized usability scales presented in the next sections. In addition, the test persons were able to make suggestions for improvements and extensions, both for usability and the function range.
3.3.1. AR Editor
The SUS (System Usability Scale) developed by John Brooke in 1986 [65] was used to analyse the user-friendliness of the AR editor. It provides a high level of information with a low time commitment for test persons. The assessment was based on 10 statements, rated by the respondent on a 5-digit Likert scale (1 = “Strongly agree” to 5 = “Strongly disagree”, with the optimum being 1 for positive and 5 for negative statements). To avoid bias in the responses, the SUS statements alternated between positive and negative. The formula to calculate the SUS is described in [65]. We used an additional value—the distance to optimum (DtO). The DtO is the absolute value of the difference of the rating to either 1 (for positive statements) or to 5 (for negative statements).
After evaluating the scale, the results were rated in a range between 0 (very poor usability) and 100 (very good usability). If the value exceeded 70, the usability of the system can be described as acceptable [66,67]. The SUS was used during the evaluation because of the short time needed to answer which, however, does not massively diminish the reliably value. In addition, the SUS is not hard to use for the interviewed persons as well as for the questioners. Another benefit is the point scale which makes it possible to compare the tested software between competing products or other versions. The use of SUS in an iterative testing use case is described in [66].
3.3.2. AR Viewer
The HARUS (Handheld Augmented Reality Usability Scale) developed by Santos et al. [68], designed primarily for portable mobile devices, was used to measure the usability of the mobile AR viewer and depicts its manipulability and comprehensibility. Like the SUS, the HARUS enables an assessment of the system based on a rating range between 0 (very poor usability) and 100 (very good usability), with a value over 70 representing acceptable usability. In contrast to the SUS, the assessment was based on 16 standardised statements rated by the respondent on a 7-digit Likert scale (1 = “Strongly agree” to 7 = “Strongly disagree”, with the optimum being 1 for positive and 7 for negative statements). The DtO for the HARUS is the absolute value of the difference of the rating to either 1 (for positive statements) or to 7 (for negative statements). HARUS was used for the same reasons as the SUS. Added to this, however, is the fact that a comparison of the usability of both systems is possible due to the same scoring-range.
The interviewed students also assessed the use and effectiveness of AR in future teaching using a 5-digit Likert scale.
3.4. Use Cases of the Pilot Study
The developed AR platform “AR-supported Teaching” was used and therefore tested for various use cases, with the following section addressing the cases with the most valuable outcomes. In the two teaching use cases presented, the usability of the teaching platform was subjected to an initial test in order to draw conclusions on the improvement and further development of the platform.
The “AR-supported Teaching” platform can be used in various AEC areas. To show the versatility the authors considered four use cases in the presented study. The first two represent an application in AEC practice: (1) a guide through the construction of a building using an overall model including explanations of the work-flow (e.g., to show the sequence of work) and (2) a detailed description of a building detail in case of problems during construction. The other two use cases are intended for education: (3) transferring the two practical models into AEC education and therefore allowing students to practice using AR for practical purposes in a course and (4) supporting education through the use of AR.
3.4.1. AR Model of an Entire Building
The use of “AR-supported Teaching” makes it possible to display entire BIM models and therefore view various building designs three-dimensionally at an early stage without creating a real architectural model. As a first step, the user loads the IFC file of the entire building into the AR editor and adds buttons and animations to the digital designs. The buttons trigger animations or present explanations and videos. In addition, users can display or hide entire sub-models (e.g., structural model, building services model) separately using buttons. This additional post-processing of the BIM model enables visual comparison of variants on a spatial level. The people involved in the planning processes obtain a clearer picture of the overall situation, as the AR model can also be displayed in its original size directly on the construction site.
As a further option, the animations allow for the display of construction time simulations and therefore a visual presentation of the construction process steps. The interrelationship of the components is clearly displayed for the individual professions resulting in more structured construction meetings. Figure 6 shows the AR model of a newly built institute building at a school in Austria, which was planned and built as an openBIM pilot project. Figure 6a displays the whole institute building, Figure 6b shows the animation of the construction steps of the roof allowing for the discussion of possible roof variants in the planning phase.
Another possible application of “AR-supported Teaching” is the digital building submission based on openBIM as described by Schranz et al. [69]. With the openBIM building submission, the BIM model supports the plan checking process and can be used to enable public participation. The clear visualisation using AR can help avoid misunderstandings at an early stage and allow neighbours to view the planned new building directly on the site.
3.4.2. AR Model of a Single Component—Detailed Explanation for Construction Site with Remote Support
Problems often arise on construction sites when complicated components are not delivered pre-manufactured but need to be cast or built on site. Even with designs and plans produced based on BIM, the level of detail of the original BIM model is often insufficient. The AR platform “AR-supported Teaching” can bring the virtual information to the construction site to support the workers and clearly show the work steps towards finishing an intricate component.
In the first step, the building component is separated from the rest of the BIM model in the BIM authoring software (IFC model of the component) and loaded into the AR editor. Although the AR editor also offers the possibility to separate partial models from a larger IFC model, the BIM authoring software is better suited. Within the editor (see Figure 7), the user revises the detail, adds explanations, and creates animations, buttons and annotations. Animations can be used in conjunction with buttons to show the construction of intricate details step by step. Annotations allow textual or graphical explanations of various components. After completion, the AR scene is saved on a file server and can be accessed on the construction site without delay using the AR viewer (see Figure 8).
The AR platform has an additional live session function which allows for a remote expert to help or answer questions if any additional explanations are needed. The expert can chat with the worker on-site over the platform, insert live annotations into the AR model (green and orange pins in the AR editor in Figure 7 and the AR viewer in Figure 8, respectively), and transmit images and audio messages.
A clear advantage of the presented platform is the speed at which explanations are created in the AR model and delivered to the asking party. Compared to pure 3D model viewers, the detail can be placed directly in the intended construction site environment. Furthermore, an extension of the language range is possible, leading to an increase in comprehensibility on the construction site when using the AR viewer.
3.4.3. Teaching Setup 1—Practical Application in Courses
In the future, the application of Construction 4.0 technologies will be taught directly to the students in the classroom. For this reason, it is important to already consider the integration of the new content into the lessons [32]. “AR-supported Teaching” enables the teaching of practical relevant AR applications in AEC lessons. Using the software, the students have the possibility to either create content (BIM models, graphics, etc.) or view and assess AR models in the AR viewer. A freely selected BIM authoring software is used to create the content before the project is further processed in the AR editor. After that the teacher either adapts the AR scene for the course or uses it otherwise for peer-to-peer interaction. Both use cases, one being the creation, the other the interaction as a viewer, are examples of possible implementations in the classroom. The two teaching scenarios were used in lecturers with students within the framework of the research project.
It is planned to extend the functions of the platform to enable students to conduct a fictitious construction meeting, for example, when handing in projects.
3.4.4. Teaching Setup 2—AR-Supported Teaching
The use case “supported teaching”, which gave the AR platform its name, enables supported teaching through visual presentation, additional animations, and the live interaction with teachers, therefore being well suited for distance teaching and learning.
As with any project, the first step consists of creating new content (BIM models, graphics, etc.) promoting the creator’s understanding of BIM, as it usually involves a smaller model and can thus prevent overtaxing and repetition. Once content is available, animations, buttons, and annotations, which are later visible in the AR viewer, can be added in the AR editor. The creator of the annotations (e.g., teacher or student) differs depending on the chosen type of use. While the AR models created by teaching staff are often of a higher quality due to in-depth subject knowledge and pedagogic training [70], the projects with a significant higher rate of new and broad content are created by students. The models of the students can then be continuously improved by incorporating the receiving feedback from teachers or other students, allowing the peer-to-peer method to achieve a similarly high quality as the AR models only created by the teacher. In addition, the learning impact and knowledge acquisition that cumulates throughout the entire development process is invaluable for not only one student but for all parties involved.
The finished models with all additional content can be viewed in the following step with the help of the AR viewer. This can occur either collectively, with simultaneous auditory explanation by the teaching staff, or alone in self-study. In lockdown situations that prevent face-to-face teaching, the live session function creates a shared remote environment. Despite spatial separation, a learning environment is created and all explanations by the teachers can take place on the virtual model, enabling student-centred learning and promoting discovery and exploratory learning.
Within the framework of the research project civil engineering students used the application in different courses throughout their studies, e.g., “Reinforced Concrete” or “Building Constructions”. Figure 9 depicts a reinforcement model of a sleeve foundation for the course “Reinforced Concrete”, showing the AR viewer with numbered annotations and additional explanations activated. These annotations identify the individual reinforcement types and have attached pictures of the reinforcement bending forms. Animations lead step-by-step to the correct assembly of the reinforcement cage.
4. Results of Usability Tests of the Pilot Study
In the last two use cases presented, students tested the usability of the two applications (AR editor and AR viewer) in the AEC teaching context.
4.1. AR Editor
A total of 12 students assessed the usability of the AR editor using SUS. The application received an average of 78.33 points and can thus be described as “good” according to [67]. Table 1 lists the ten statements surveyed including the results in form of a mean calculation according to a rating on a 5-digit Likert scale. Furthermore, the standard deviation (SD) and distance to optimum (DtO) of the mean can be found in Table 1. The results show that students find the system easy and quick to understand (DtO7 0.67), feel very confident in using the AR-editor (DtO9 0.50) and no additional knowledge is needed to use the system (DtO10 0.25). Room for improvement was left in regard to the support of a technical person to use the system (DtO4 1.42), the integration of functions (DtO5 1.25), and inconsistency in the system (DtO6 1.08).
In addition to the rating, comments on the usability were submitted requesting changes and further developments. Some participants, for example, wished for short descriptions of the functions when hovering over the corresponding button. With no explanatory videos or manual available during testing, explanatory videos were added on request. Further discussions with the students showed that an improvement in clarity was necessary which could be achieved by, e.g., tables during the creation and handling of annotations and animations. Users with small notebook displays experienced problems with displaying all buttons and functions. Other feature suggestions included, for example, the extension of annotations to include PDF files, rotation of flags and the subsequent adjustment of component colours.
4.2. AR Viewer
A total of 15 students assessed the usability of the AR viewer using HARUS. The application received an average of 71.88 points and can thus be described as “okay” according to [67]. As in Table 1 for the AR editor, Table 2 lists the statements, mean results, SD, and DtO of the AR viewer assessment. For the HARUS, a 7-digit Likert scale was employed.
Using the DtO a comparison of the two categories “manipulability” (statements 1–8) and “comprehensibility” (statements 9–16) is made possible. While the category “manipulability” achieved an average DtO of 1.93, “comprehensibility” obtained a DtO of 1.44, thus, showing better results.
The rating is generally good in the category “manipulability“ but some areas show room for improvement. It is not the physical effort that is a problem (DtO1 0.47), but rather the handling of the used device (DtO8 2.00). Two hands are needed to hold the device steadily and securely making it difficult to operate the application simultaneously (DtO3 2.07) and resulting in uncomfortable holding positions (DtO2 2.80). Apart from the handling the students also assessed that the input of information could be enhanced (DtO4 2.33). Höbart [51] encountered this problem with earlier versions of the viewer and concluded that a hardware-based solution is preferable to a software-based one. Furthermore, respondents who viewed the teaching scenes with a smartphone had difficulties with the size and readability of information and the selection of annotations. Here, a revision of the GUI (Graphical User Interface) would be conceivable for a better font size adjustment.
The interviewed students also assessed the use and effectiveness of AR in future teaching using a 5-digit Likert scale. The statement whether AR can increase the learning effect achieved a DtO of 0.67. The students also claimed that AR improves teaching (DtO = 0.56) and that they would like to use AR in teaching in the future (DtO = 0.38).
5. Discussion
This publication describes the development, testing, and evaluation of the usability of the newly developed AR platform “AR-supported Teaching”. The platform includes an AR editor and an AR viewer for mobile devices. The authors began their research by determining and evaluating requirements for an AR platform that can be used in an AEC context. A pilot study with AEC students followed the development to evaluate the usability. Due to the lockdowns of the COVID-19 pandemic, the pilot study was conducted with a smaller group of people than intended. Nevertheless, the authors received good feedback and a promising outlook for the further development of the AR platform.
Although the improvement of learning through AR has been proven in several studies and AR as a Construction 4.0 technology has begun to gain importance in the AEC industry, a minimal usage of the technology can currently be observed in educational facilities. This can be traced back, for example, to the lack of available AR content which must be either purchased or created by people with programming skills. Based on this fact, the goal for the developed AR platform was set to create an easy-to-use software for people without programming skills. The users can quickly create their own AR models which they then store and manage on a server—the core of the AR platform “AR-supported Teaching”. The use of the platform is free for educational institutions and the creators of the models can set access rights accordingly. The evaluation of the AR editor in the usability tests shows that the set goal was achieved. The students particularly emphasised the AR editor’s ease of use and learnability. Some students wished for explanatory videos and an improved integration of functions, which will be all implemented in the next development phase.
Although several AR viewers are available on the market, the AR platform includes its own AR viewer currently available for Android mobile devices, with an extension to iOS mobile devices planned for the near future. It is adapted to the requirements of the AEC education and industry and allows authors to easily integrate additional functions. AR viewer displays AR content and enables ongoing interaction with the AR model, while simultaneously offering the possibility to communicate with a teacher, a remote expert or the planning engineer by live session while obtaining a direct connection with the AR editor. The students rated the usability of the AR viewer positively yet lower than the AR editor due to problems during handling, especially in the case of continuous interaction with the model. Test persons who viewed the teaching scenes with a smartphone had difficulties with the size and readability of information and the selection of annotations, while those with tablets had problems handling the device due to its size. Since only slight improvement is possible with the existing hardware, possible developments are investigated in addition to improving the GUI.
The authors currently view HMD as a solution to significantly increase the usability of AR in the assessed field. With HMD, the hands are free and virtual elements appear freely in space enabling a more natural and dynamic AR interaction (better immersion), while increasing learning motivation and efficiency. Unfortunately, HMDs are still very expensive and not practical enough for widespread use in education. As soon as big companies bring affordable HMD to market, it will be possible to investigate their use for the general public. To ensure that the developed AR editor also works for HMDs it is essential to ensure that the created AR scenes are platform-independent. The used AR Foundation SDK, for example, broadly supports integration in the HMD Microsoft HoloLens 2. Above all, improving the availability of HMD and the ability to implement software within can become an important development step.
The test results show that the AR editor can be used in teaching and allows teachers and students to create elaborate AR teaching scenes without programming knowledge. Although the teaching aspect was the original use case for the presented AR platform, it quickly became clear that many practical use cases in the AEC industry and AEC education are also possible, with many already being technically feasible. This publication gives an insight into some of the possible use cases, e.g., instructions for planning meetings, training of personnel in the AEC industry or instructions for the construction site.
The research and assessment showed that teachers and students can already use the AR platform in practical courses. In addition to the classes already mentioned, further lectures focusing on quality inspection of built parts (e.g., HVAC systems) using AR are being planned. The students will inspect an existing HVAC system and point out any errors identified in the AR viewer. Teaching staff have additionally pointed out the AR platform could be used to hold exams.
The further development of the AR editor will focus on converting it to a web application. The development of the AR editor is currently based on the Unity engine requiring an installation on a local hardware. In educational institutions, public authorities, and companies with hardware infrastructure and cyber security concepts, distribution and installation posed a significant challenge in the trial period. Therefore, developing the AR editor as a web application facilitates software adaption, as it works without installation, is independent of the operating system and allows for quick updates. This also has the advantage of better implementing the AR Editor in the existing web application for project management. One disadvantage of a web application, however, is that internet access is required during use.
The usability of the applications (AR editor and AR viewer) was assessed in the carried-out tests and surveys enabling ongoing improvements and adjustments to the user interface. The evaluation of the learning efficiency increase, the learning method and teaching effect was not subject of this study and will be assessed in future tests, where the Perceived Usefulness (USEF) will be determined. The results will then be compared to results of conventional training methods.
6. Concluding Remarks and Future Work
This publication presents the newly developed AR platform “AR-supported Teaching”, including an AR editor and an AR viewer. The platform was developed for use in AEC education, yet is also suited for productive use in the AEC industry as well as the creation of AR content based on BIM models. Although the use case is broad by using open standards, some disciplines (e.g., social science, economic science) are not able to use the developed method. One reason for this is the low use and resulting low demand for labelled, animated 3D models in these areas. After creating a model in a BIM authoring software, users can load the IFC file into the AR editor and extend the model by annotations, animation, and additional information. Currently the platform is limited to IFC files. Although this is widely used in civil engineering and architecture, at present only disciplines that work with IFC files can use the platform. However, the authors are working on an extension of the platform to use general 3D formats as well.
In a pilot study, students tested the usability of the AR editor and the AR viewer showing promising results. The SUS (System Usability Score) was used for the evaluation of the AR editor while the HARUS (Handheld Augmented Reality Usability Score) was used for the AR viewer. The students rated the usability of both applications well (SUS = 78.33/100 points and HARUS = 71.88/100 points). Some suggestions for improvement were identified and incorporated into the further development of the two applications.
The authors are very optimistic about the students’ general opinion on the use of AR in teaching. In their assessment, the students believe that AR improves education and increases the learning effect and output. They would like to use AR more in teaching as well as in their future work. In a further study, the impact on learning efficiency will be determined. A cooperation with other AEC educational institutions is being sought.
Further development steps for both the editor and the viewer are planned for the future. The AR editor will be converted into a web application allowing for a better integration into the platform and the use independent of the operating system. The AR viewer is to be extended to HMDs and iOS mobile devices. In cooperation with engineering offices and construction companies, the authors would like to further identify the needs of the AEC industry.
AR in education will prevail above all when there is enough AR content. The AR scenes will be continuously improved through constant development and use by a sufficiently large community. This should lead to an improvement in the quality of training both in educational institutions and in companies. The authors, therefore, recommend the promotion of free AR platforms, which ideally can exchange AR scenes with each other using open standards (e.g., JSON, IFC).
Conceptualization, H.U. and C.S.; methodology, H.U., G.P. and C.S.; validation, H.U., G.P. and C.S.; formal analysis, G.P.; investigation, G.P.; resources, C.S.; data curation, G.P.; writing—original draft preparation, H.U., G.P. and C.S.; writing—review and editing, H.U., G.P. and C.S.; visualization, H.U. and G.P.; supervision, H.U. and C.S.; project administration, H.U. and C.S.; funding acquisition, C.S. All authors have read and agreed to the published version of the manuscript.
Open Access Funding by TU Wien, grant number dcl032 (dCall digitalisation initiative).
Not applicable.
Not applicable.
The data presented in this study are available on request from the corresponding author. The models are not publicly available due to copyright issues.
The authors gratefully acknowledge the TU Wien Bibliothek for financial support through its Open Access Funding Program. The authors would like to thank Konstantin Höbart for the programming of the software and the always quick implementation of change requests.
The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Enlarge this image.Figure 4. AR viewer: buttons on the left side and description of the selected object on the right side (blue arrow).
Enlarge this image.Figure 6. AR model of a new institute building: (a) AR model of whole building, (b) AR model with animations of the roof.
Enlarge this image.Figure 7. AR editor with a live session: AR model of an attic (with IFC information in the box on the right side) and live session on the left side.
Enlarge this image.Figure 8. AR viewer with a live session: AR model of an attic with live session on the right side.
Enlarge this image.Figure 9. Reinforcement of a sleeve foundation: AR viewer with numbered buttons for the animations and a plan as an annotation.
Results of the SUS-statements (n = 12) (Questions according to [
| Nr. | Questions | Mean | SD | DtO |
|---|---|---|---|---|
| 1 | I think that I would like to use this system frequently. | 2.00 | 0.85 | 1.00 |
| 2 | I found the system unnecessarily complex. | 4.00 | 0.60 | 1.00 |
| 3 | I thought the system was easy to use. | 1.83 | 0.58 | 0.83 |
| 4 | I think that I would need the support of a technical person to be able to use this system. | 3.58 | 1.00 | 1.42 |
| 5 | I found the various functions in this system were well integrated. | 2.25 | 0.62 | 1.25 |
| 6 | I thought there was too much inconsistency in this system. | 3.92 | 1.16 | 1.08 |
| 7 | I would imagine that most people would learn to use this system very quickly. | 1.67 | 1.15 | 0.67 |
| 8 | I found the system very cumbersome to use. | 4.33 | 0.89 | 0.67 |
| 9 | I felt very confident using the system. | 1.50 | 0.52 | 0.50 |
| 10 | I needed to learn a lot of things before I could get going with this system. | 4.75 | 0.45 | 0.25 |
Results of the HARUS-statements (n = 15) (Questions according to [
| Nr. | Questions | Mean | SD | DtO | |
|---|---|---|---|---|---|
| manipulability | 1 | I think that interacting with this application requires a lot of body muscle effort. | 6.53 | 0.52 | 0.47 |
| 2 | I felt that using the application was comfortable for my arms and hands. | 3.80 | 1.26 | 2.80 | |
| 3 | I found the device difficult to hold while operating the application. | 4.93 | 1.75 | 2.07 | |
| 4 | I found it easy to input information through the application. | 3.33 | 1.50 | 2.33 | |
| 5 | I felt that my arm or hand became tired after using the application. | 5.07 | 1.83 | 1.93 | |
| 6 | I think the application is easy to control. | 2.93 | 1.16 | 1.93 | |
| 7 | I felt that I was losing grip and dropping the device at some point. | 5.07 | 2.28 | 1.93 | |
| 8 | I think the operation of this application is simple and uncomplicated. | 3.00 | 1.31 | 2.00 | |
| comprehensibility | 9 | I think that interacting with this application requires a lot of mental effort. | 6.60 | 0.91 | 0.40 |
| 10 | I thought the amount of information displayed on screen was appropriate. | 2.93 | 1.28 | 1.93 | |
| 11 | I thought that the information displayed on screen was easy to read. * | 2.67 | 0.98 | 1.67 | |
| 12 | I felt that the information display was responding fast enough. | 2.53 | 1.41 | 1.53 | |
| 13 | I thought that the information displayed on screen was confusing. | 5.20 | 1.86 | 1.80 | |
| 14 | I thought the words and symbols on screen were easy to read. | 2.07 | 1.16 | 1.07 | |
| 15 | I felt that the display was flickering too much. | 5.73 | 1.44 | 1.27 | |
| 16 | I thought that the information displayed on screen was consistent. | 2.87 | 1.25 | 1.87 |
* The original question was changed for the German translation.
References
1. Borrmann, A.; König, M.; Koch, C.; Beetz, J. Building Information Modeling: Technology Foundations and Industry Practice; Springer: Berlin/Heidelberg, Germany, 2018; ISBN 978-3-319-92861-6
2. Sacks, R.; Eastman, C.M.; Lee, G.; Teicholz, P. BIM Handbook: A Guide to Building Information Modeling for Owners, Managers, Designers, Engineers and Contractors; 3rd ed. John Wiley & Sons: Hoboken, NJ, USA, 2018; ISBN 978-1-119-28755-1
3. El Jazzar, M.; Schranz, C.; Urban, H.; Nassereddine, H. Integrating Construction 4.0 Technologies: A Four-Layer Implementation Plan. Front. Built Environ.; 2021; 7, 671408. [DOI: https://dx.doi.org/10.3389/fbuil.2021.671408]
4. Urban, H.; Schranz, C.; Krischmann, T.; Asmera, H.; Pinter, B. Einsatz von openBIM und KI im Bewilligungsverfahren der Stadt Wien. ÖIAZ; 2021; 166, pp. 1-9.
5. Reichenbach, S.; Kromoser, B. State of Practice of Automation in Precast Concrete Production. J. Build. Eng.; 2021; 43, 102527. [DOI: https://dx.doi.org/10.1016/j.jobe.2021.102527]
6. Sawhney, A.; Riley, M.; Irizarry, J. Construction 4.0: An Innovation Platform for the Built Environment; Routledge: London, UK, 2020; ISBN 978-0-429-39810-0
7. Klinc, R.; Turk, Ž. Construction 4.0—Digital Transformation of One of the Oldest Industries. Econ. Bus. Rev.; 2019; 21, pp. 393-410. [DOI: https://dx.doi.org/10.15458/ebr.92]
8. Lasi, H.; Fettke, P.; Kemper, H.-G.; Feld, T.; Hoffmann, M. Industry 4.0. Bus. Inf. Syst. Eng.; 2014; 6, pp. 239-242. [DOI: https://dx.doi.org/10.1007/s12599-014-0334-4]
9. Rüßmann, M.; Lorenz, M.; Gerbert, P.; Waldner, M.; Justus, J.; Engel, P.; Harnisch, M. Industry 4.0: The Future of Productivity and Growth in Manufacturing Industries; The Boston Consulting Group: Munich, Germany, 2015; 20p.
10. Sanchez, A.; Hampson, K.; Vaux, S. Delivering Value with BIM: A Whole-of-Life Approach; Routledge: London, UK, 2016; ISBN 978-1-315-65247-4
11. Gerbert, P.; Castagnino, S.; Rothballer, C.; Renz, A.; Filitz, R. Digital in Engineering and Construction: The Transformative Power of Building Information Modeling; The Boston Consulting Group: Munich, Germany, 2016; 22p.
12. The Construction Productivity Imperative. Available online: https://www.mckinsey.com/business-functions/operations/our-insights/the-construction-productivity-imperative (accessed on 15 April 2020).
13. Krischmann, T.; Urban, H.; Schranz, C. Entwicklung eines openBIM-Bewilligungsverfahrens. Bauingenieur; 2020; 95, pp. 335-344. [DOI: https://dx.doi.org/10.37544/0005-6650-2020-09-61]
14. Wang, X.; Yung, P.; Luo, H.; Truijens, M. An Innovative Method for Project Control in LNG Project through 5D CAD: A Case Study. Autom. Constr.; 2014; 45, pp. 126-135. [DOI: https://dx.doi.org/10.1016/j.autcon.2014.05.011]
15. Dallasega, P.; Rauch, E.; Linder, C. Industry 4.0 as an Enabler of Proximity for Construction Supply Chains: A Systematic Literature Review. Comput. Ind.; 2018; 99, pp. 205-225. [DOI: https://dx.doi.org/10.1016/j.compind.2018.03.039]
16. Azuma, R.; Baillot, Y.; Behringer, R.; Feiner, S.; Julier, S.; MacIntyre, B. Recent Advances in Augmented Reality. IEEE Comput. Grap. Appl.; 2001; 21, pp. 34-47. [DOI: https://dx.doi.org/10.1109/38.963459]
17. Shin, D.H.; Dunston, P.S. Identification of Application Areas for Augmented Reality in Industrial Construction Based on Technology Suitability. Autom. Constr.; 2008; 17, pp. 882-894. [DOI: https://dx.doi.org/10.1016/j.autcon.2008.02.012]
18. Wang, X.; Love, P.E.D. BIM + AR: Onsite Information Sharing and Communication via Advanced Visualization. Proceedings of the 2012 IEEE 16th International Conference on Computer Supported Work in Design; Wuhan, China, 23–25 May 2012; IEEE: Piscataway, NJ, USA, 2012; pp. 850-855.
19. Chi, H.-L.; Kang, S.-C.; Wang, X. Research Trends and Opportunities of Augmented Reality Applications in Architecture, Engineering, and Construction. Autom. Constr.; 2013; 33, pp. 116-122. [DOI: https://dx.doi.org/10.1016/j.autcon.2012.12.017]
20. Rankohi, S.; Waugh, L. Review and Analysis of Augmented Reality Literature for Construction Industry. Vis. Eng.; 2013; 1, 9. [DOI: https://dx.doi.org/10.1186/2213-7459-1-9]
21. Li, W.; Nee, A.; Ong, S. A State-of-the-Art Review of Augmented Reality in Engineering Analysis and Simulation. Multimodal Technol. Interact.; 2017; 1, 17. [DOI: https://dx.doi.org/10.3390/mti1030017]
22. Urban, H.; Schranz, C.; Gerger, A. BIM auf Baustellen mit Augmented Reality. Bauaktuell; 2019; 10, pp. 192-196.
23. Fenais, A.S.; Ariaratnam, S.T.; Ayer, S.K.; Smilovsky, N. A Review of Augmented Reality Applied to Underground Construction. J. Inf. Technol. Constr.; 2020; 25, pp. 308-324. [DOI: https://dx.doi.org/10.36680/j.itcon.2020.018]
24. Schranz, C.; Gerger, A.; Urban, H. Augmented Reality im Bauwesen: Teil 1—Anwendungs- und Anforderungsanalyse. Bauingenieur; 2020; 95, pp. 379-388. [DOI: https://dx.doi.org/10.37544/0005-6650-2020-10-49]
25. Nassereddine, H.; Schranz, C.; Bou Hatoum, M.; Urban, H. Mapping the capabilities and benefits of AR construction use-cases: A comprehensive map. Organ. Technol. Manag. Constr.; 2022; 14, pp. 2571-2582. [DOI: https://dx.doi.org/10.2478/otmcj-2022-0003]
26. Wang, X.; Kim, M.J.; Kang, S.-C. Augmented Reality in Built Environment: Classification and Implications for Future Research. Autom. Constr.; 2013; 32, pp. 1-13. [DOI: https://dx.doi.org/10.1016/j.autcon.2012.11.021]
27. Shin, D.H.; Dunston, P.S. Technology Development Needs for Advancing Augmented Reality-Based Inspection. Autom. Constr.; 2010; 19, pp. 169-182. [DOI: https://dx.doi.org/10.1016/j.autcon.2009.11.001]
28. Shin, D.H.; Dunston, P.S. Evaluation of Augmented Reality in Steel Column Inspection. Autom. Constr.; 2009; 18, pp. 118-129. [DOI: https://dx.doi.org/10.1016/j.autcon.2008.05.007]
29. Olsen, D.; Kim, J.; Taylor, J.M. Using Augmented Reality for Masonry and Concrete Embed Coordination. Proceedings of the Creative Construction Conference 2019; Budapest, Hungary, 29 June–2 July 2019; Skibniewski, M.J.; Hajdu, M. Budapest University of Technology and Economics: Budapest, Hungary, 2019; pp. 906-913.
30. Abbas, A.; Seo, J.; Kim, M. Impact of Mobile Augmented Reality System on Cognitive Behavior and Performance during Rebar Inspection Tasks. J. Comput. Civ. Eng.; 2020; 34, 04020050. [DOI: https://dx.doi.org/10.1061/(ASCE)CP.1943-5487.0000931]
31. Marino, E.; Barbieri, L.; Colacino, B.; Fleri, A.K.; Bruno, F. An Augmented Reality Inspection Tool to Support Workers in Industry 4.0 Environments. Comput. Ind.; 2021; 127, 103412. [DOI: https://dx.doi.org/10.1016/j.compind.2021.103412]
32. Chacón, R. Designing Construction 4.0 Activities for AEC Classrooms. Buildings; 2021; 11, 511. [DOI: https://dx.doi.org/10.3390/buildings11110511]
33. Dohse, F.; Nicolaisen, V.; Wetzel, S.; Bertel, S. Potential of AR for the Analysis and Training of Spatial Skills: A Case Study. Proceedings of the MuC’19: Mensch-und-Computer 2019; Hamburg, Germany, 8–11 September 2019; Alt, F.; Bulling, A.; Döring, T. ACM: New York, NY, USA, 2019; pp. 537-541.
34. Thoms, L.-J. Lehr-Lernpsychologische Grundlagen. Spektrometrie im Fernlabor; Springer Spektrum: Wiesbaden, Germany, 2019; pp. 33-69. ISBN 978-3-658-25707-1
35. Khan, T.; Johnston, K.; Ophoff, J. The Impact of an Augmented Reality Application on Learning Motivation of Students. Adv. Hum. Comput. Interact.; 2019; 2019, 7208494. [DOI: https://dx.doi.org/10.1155/2019/7208494]
36. Niedermeier, S.; Müller-Kreiner, C. VR/AR in der Lehre!? Eine Übersichtsstudie zu Zukunftsvisionen des digitalen Lernens aus der Sicht von Studierenden; Hochschule für angewandte Wissenschaften Kempten: Kempten, Germany, 2019; 13.
37. Joan, D.R.R. Enhancing Education through Mobile Augmented Reality. J. Educ. Technol.; 2015; 11, pp. 8-14. [DOI: https://dx.doi.org/10.26634/jet.11.4.3147]
38. von Jan, U.; Noll, C.; Behrends, M.; Albrecht, U.-V. MARble—Augmented Reality in Medical Education. BMT; 2012; 57, pp. 67-70. [DOI: https://dx.doi.org/10.1515/bmt-2012-4252]
39. Albrecht, U.-V.; Folta-Schoofs, K.; Behrends, M.; von Jan, U. Effects of Mobile Augmented Reality Learning Compared to Textbook Learning on Medical Students: Randomized Controlled Pilot Study. J. Med. Internet Res.; 2013; 15, e182. [DOI: https://dx.doi.org/10.2196/jmir.2497]
40. Pathania, M.; Mantri, A.; Kaur, D.P.; Singh, C.P.; Sharma, B. A Chronological Literature Review of Different Augmented Reality Approaches in Education. Tech. Know. Learn.; 2021; pp. 1-18. [DOI: https://dx.doi.org/10.1007/s10758-021-09558-7]
41. Yuen, S.C.-Y.; Yaoyuneyong, G.; Johnson, E. Augmented Reality: An Overview and Five Directions for AR in Education. JETDE; 2011; 4, pp. 119-140. [DOI: https://dx.doi.org/10.18785/jetde.0401.10]
42. Chen, H.; Feng, K.; Mo, C.; Cheng, S.; Guo, Z.; Huang, Y. Application of Augmented Reality in Engineering Graphics Education. Proceedings of the 2011 IEEE International Symposium on IT in Medicine and Education; Guangzhou, China, 9–11 December 2011; IEEE: Piscataway Township, NJ, USA, 2011; pp. 362-365.
43. Wai, J.; Lubinski, D.; Benbow, C.P. Spatial Ability for STEM Domains: Aligning over 50 Years of Cumulative Psychological Knowledge Solidifies Its Importance. J. Educ. Psychol.; 2009; 101, pp. 817-835. [DOI: https://dx.doi.org/10.1037/a0016127]
44. Alias, M.; Black, T.R.; Gray, D.E. Effect of Instructions on Spatial Visualisation Ability in Civil Engineering Students. Int. Educ. Stud.; 2002; 3, pp. 1-12.
45. Doka Augmented Reality App. Available online: https://www.doka.com/uk/home/apps/augmented-reality-app (accessed on 28 December 2021).
46. Vasilevski, N.; Birt, J. Analysing Construction Student Experiences of Mobile Mixed Reality Enhanced Learning in Virtual and Augmented Reality Environments. Res. Learn. Technol.; 2020; 28, 2329. [DOI: https://dx.doi.org/10.25304/rlt.v28.2329]
47. Turkan, Y.; Radkowski, R.; Karabulut-Ilgu, A.; Behzadan, A.H.; Chen, A. Mobile Augmented Reality for Teaching Structural Analysis. Adv. Eng. Inform.; 2017; 34, pp. 90-100. [DOI: https://dx.doi.org/10.1016/j.aei.2017.09.005]
48. Lytridis, C.; Tsinakos, A.; Kazanidis, I. ARTutor—An Augmented Reality Platform for Interactive Distance Learning. Educ. Sci.; 2018; 8, 6. [DOI: https://dx.doi.org/10.3390/educsci8010006]
49. Augmented Reality Platform for Architecture, Engineering and Construction—DARF DESIGN, London, United Kingdom. Available online: https://www.darfdesign.com/arki.html (accessed on 28 December 2021).
50. AR-Supported Teaching. Available online: https://arserver.bilab.tuwien.ac.at/ (accessed on 29 December 2021).
51. Höbart, K. AR-Schulungs-Anwendung und-Editor für 3D-BIM-Visualisierung im Bauingenieurwesen. Master’s Thesis; TU Wien: Vienna, Austria, 2020.
52. Delgado, J.M.D.; Oyedele, L.; Demian, P.; Beach, T. A Research Agenda for Augmented and Virtual Reality in Architecture, Engineering and Construction. Adv. Eng. Inform.; 2020; 45, 101122. [DOI: https://dx.doi.org/10.1016/j.aei.2020.101122]
53. ARCore—Build the Future. Available online: https://developers.google.com/ar (accessed on 15 November 2021).
54. ARKit—Augmented Reality. Available online: https://developer.apple.com/augmented-reality/arkit/ (accessed on 28 December 2021).
55. Mixed Reality Documentation—Mixed Reality. Available online: https://docs.microsoft.com/en-us/windows/mixed-reality/ (accessed on 28 December 2021).
56. AR Foundation. Available online: https://unity.com/de/unity/features/arfoundation (accessed on 28 December 2021).
57. Initiative D21 e.V. Sonderstudie “Schule Digital”: Lehrwelt, Lernwelt, Lebenswelt: Digitale Bildung im Dreieck SchülerInnen-Eltern-Lehrkräfte; Initiative D21 e.V: Berlin, Germany, 2016; ISBN 978-3-9818331-2-6
58. GitHub: Where the World Builds Software. Available online: https://github.com/ (accessed on 28 December 2021).
59. Microsoft HoloLens|Mixed Reality Technology for Business. Available online: https://www.microsoft.com/en-us/hololens/ (accessed on 28 December 2021).
60. Urban, H.; Irschik, T.; Schranz, C.; Schönauer, C. Augmented Reality im Bauwesen: Teil 2-Baustellentaugliches Trackingsystem. Bauingenieur; 2020; 95, pp. 501-508. [DOI: https://dx.doi.org/10.37544/0005-6650-2020-12-67]
61. Davis, F.D. A Technology Acceptance Model for Empirically Testing New End-User Information Systems: Theory and Results. Ph.D. Thesis; Massachusetts Institute of Technology: Cambridge, MA, USA, 1985.
62. King, W.R.; He, J. A Meta-Analysis of the Technology Acceptance Model. Inf. Manag.; 2006; 43, pp. 740-755. [DOI: https://dx.doi.org/10.1016/j.im.2006.05.003]
63. Granić, A.; Marangunić, N. Technology Acceptance Model in Educational Context: A Systematic Literature Review. Brit. J. Educ. Technol.; 2019; 50, pp. 2572-2593. [DOI: https://dx.doi.org/10.1111/bjet.12864]
64. Tullis, T.S.; Stetson, J.N. A Comparison of Questionnaires for Assessing Website Usability. Proceedings of the UPA 2004: 13th Annual UPA Conference; Minneapolis, MN, USA, 7 June 2004; Usability Professionals Association: Minneapolis, MN, USA, 2004.
65. Brooke, J. SUS—A Quick and Dirty Usability Scale. Usability Evaluation in Industry; Jordan, P.W.; Thomas, B.; McClelland, I.L.; Weerdmeester, B. CRC Press: Boca Raton, FL, USA, 1996; Volume 1, pp. 189-194. ISBN 978-0-429-15701-1
66. Bangor, A.; Kortum, P.T.; Miller, J.T. An Empirical Evaluation of the System Usability Scale. Int. J. Hum. Comput. Interact.; 2008; 24, pp. 574-594. [DOI: https://dx.doi.org/10.1080/10447310802205776]
67. Bangor, A.; Kortum, P.; Miller, J. Determining What Individual SUS Scores Mean: Adding an Adjective Rating Scale. J. Usability Stud.; 2009; 4, pp. 114-123.
68. Santos, M.E.C.; Taketomi, T.; Sandor, C.; Polvi, J.; Yamamoto, G.; Kato, H. A Usability Scale for Handheld Augmented Reality. Proceedings of the 20th ACM Symposium on Virtual Reality Software and Technology; Edinburgh, UK, 11–13 November 2014; Lau, R.; Manocha, D.; Komura, T.; Majumder, A.; Xu, W. ACM: New York, NY, USA, pp. 167-176.
69. Schranz, C.; Urban, H.; Gerger, A. Potentials of Augmented Reality in a BIM Based Building Submission Process. ITcon; 2021; 26, pp. 441-457. [DOI: https://dx.doi.org/10.36680/j.itcon.2021.024]
70. Wojciechowski, R.; Cellary, W. Evaluation of Learners’ Attitude toward Learning in ARIES Augmented Reality Environments. Comput. Educ.; 2013; 68, pp. 570-585. [DOI: https://dx.doi.org/10.1016/j.compedu.2013.02.014]
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
Details
