Introducing Immersive Virtual Reality in the Initial Phases of the Design Process—Case Study: Freshmen Designing Ephemeral Architecture

Abstract

Immersive Virtual Reality (IVR) has proven to be an important tool for the exploration and communication of architectural projects prior to their real construction; however, there have been few scientific advances of its use in the understanding, exploration, and definition of architectural space by architecture students in their initial design processes. The purpose of this research is to determine how the use of IVR incorporated in the initial phases of the architectural design process improves, among students, the achievement of three specifics design competencies, and to know the evaluation that professors make of the advantages and disadvantages of the use of this tool in the design process. A mixed methodology was applied, considering participatory observations and surveys of students and teachers concerning the initial architecture workshop on architectural careers. It was found that the three analyzed competencies are better achieved with the use of IVR due to its high utility in the perception of space on a real scale and in its interior experimentation, both referred to as important advantages by students and teachers. It is concluded that the application of the interactive and immersive VR is a pedagogical tool that allows students to get feedback from their own spatial experience to correct and improve their designs, while teachers find the tool useful in the initial phases of architectural design.

1. Introduction

One of the main activities in the training of an architect is design [1]. As an activity, it involves decision-making based on solving spatial problems [2]. Sullivan points out that decision-making consists of several methods that are used to solve a problem or generate a new opportunity [3]. In this sense, among the several activities that are carried out in an architecture workshop there are drawing, presentation, evaluation, and sketches grouping, among others [4]. Moreover, Kaur, Mantri, and Horan mention that an architecture student must have some skills in the problem-solving process, as well as fair knowledge of design [5]. However, students often experience difficulties in externally representing their ideas in 2D drawings or 3D models. Apparently, the causes of these difficulties focus on the non-understanding of design problems, the application of instructional materials, or the external representation of design ideas using traditional tools [5]. Regarding this, in recent decades, technological progress has been providing tools that allow the designed space to be expressed and experienced faster, on a real scale, and in a more realistic way. A powerful tool that current technology offers is Immersive Virtual Reality (IVR), which allows the architect to explore and communicate more ideas in the design process compared to conventional methods. This tool also allows the architect to experience the feeling of “being in the place” rather than “looking at the place”. In this way, experiencing a full-scale replicated environment can improve ideas in the design process [6]. As the student’s intentions and imagination overlap in the virtual model space, IVR becomes a tool to better understand the design problem and deepen the materialization and experience of the space. From this perspective, the current advances presented by IVR lead to a reassessment of the teaching methods of architectural design in architecture workshops and of the teacher’s role in proactively facilitating these changes through the study, application, and communication of this technology.

1.1. Immersive Virtual Reality

The beginning of virtual reality occurred in various fields of computer science during the 1950s and 1960s, especially in the field of interactive 3D computer graphics and navigation simulation in vehicles and flights [7]. The term “Virtual Reality” was coined in 1992 by Jaron Lanier [8], and it generally means a three-dimensional visual or sensory artificial environment, modeled and simulated by a computer with the help of interactive devices that send and receive information and with which a person can interact [7]. Depending on the level of interaction with the artificial environment, two extremes of Virtual Reality can be recognized, one immersive and one non-immersive, with a space in between for Augmented Reality and Mixed Reality (Figure 1).

Non-immersive Virtual Reality is one in which the content is displayed through a computer screen, TV, or mobile phone and no additional equipment is required as in Web-based virtual environments and video games [9,10], or, on the other hand, IVR, where users have to wear a head-mounted display or other devices and are completely surrounded by the virtual environment.

According to Schroeder’s definition [11], two of the most important characteristics of virtual reality are immersion and interaction; immersion is understood as the perception of being physically present in a non-physical environment that is created with images, sounds, or other stimuli that provide a totally absorbing environment and interaction, understood as the natural action that occurs between the user and the virtual environment. These characteristics position IVR as more beneficial than the non-immersive Virtual Reality.

1.2. IVR in Education

The idea of using IVR in the field of education has always been present since its inception because it allows for the possibility of learning and training in a safe environment, where authentic real-world situations can be recreated or even create new situations and virtual experiences [12] and where students face challenging situations with the possibility of repeatedly practicing new skills in an environment that allows for correction and failure without danger [9]. This important advantage of IVR coupled with the greater accessibility of the required software and hardware has made it even more attractive for learning and for the field of education in general, so much so that it has been described as a technology of great help to 21st century learning [13]. Another study [14] mentions that students retain more information and are better able to apply what they have learned after IVR training. It is therefore understandable that researchers, organizations, and educators are seriously considering this technology and looking for ways to include it in the classroom to improve the teaching and learning processes. The increasing academic attention to VR technologies has led to comprehensive reviews of VR applications for education [9,15]. For example, Merchant et al. [15] focus on desktop VR in education, while Jensen and Konradsen emphasize the application of HMD (more immersive and interactive) technologies.

On the other hand, it is important to recognize that the first practical applications of VR-based training appeared in the 2000s, in the defense, aerospace, and automotive industries [16]. VR is increasingly used in the military for methods, strategic analysis, and training in conflict situations [17]. The US military has used real-time visual simulation to support carrier landing training, emergency flight training, and air-to-air combat [18]. Another area where VR has been applied intensively is in medical science training, where VR is used, for example, for surgical or diagnostic training [19]. The use of VR technology opens new areas in the teaching and practice of medicine and biology by allowing visualizations to be manipulated with an intuitive immediacy similar to that of real objects. In a virtual environment, the user can view the scene from any point of view, objects can be dynamic, and the scene and objects can be interacted with by engaging other senses such as touch and hearing (or even smell) to enrich the visualization [20]. Numerous studies have used virtual environments to simulate situations that are too dangerous to practice in real life, for example, to teach fire safety techniques [21]. Other applications of training based on these environments can be found in engineering education in universities and industrial companies, e.g., training machine tool operators, assembly processes, or handling heavy equipment [19].

However, unlike the other three fields in which IVR is introduced such as research and development, healthcare, and industry, it is education, the field in which the tendency in the future is to decrease [22]. This prediction may be due to the fact that in most cases when IVR is used in education, it has been strictly based on the use of technology itself as a simple tool with few concepts or structural educational models [8] that support the educational process, in many cases becoming the immersive experience, itself a distractor from the learning task. Despite this, in cases where other approaches such as game-based attempts have been merged with IVR environments, a promising future has been glimpsed because they can improve learning and training methodologies because users engage in those learning environments as active participants, allowing for the development of exploration-based learning paradigms [23]. In addition, Jensen and Konradsen [9] have shown that IVR is useful for the acquisition of cognitive skills related to the recall and understanding of information and spatial and visual knowledge; psychomotor skills related to head movements, such as visual exploration or observation skills; and affective skills related to the control of the emotional response to stressful or difficult situations.

1.3. IVR in Architecture

IVR has spread widely in the field of architecture by allowing users to immerse themselves in a digital universe or in the simulation of an unbuilt project [24]. The specific subfields in which IVR has developed the most are Architecture, Engineering, and Construction (AEC). At the same time, great progress has been made in health and safety in construction, in the formation of teams and operational tasks [25], in the management of facilities, in the review and inspection of buildings [26], and in the evaluation of preliminary projects, among others, due to multisensory approach [26]. For architecture, the visualization of the proposals and solutions of the projects is very important for the understanding of the different stages in the design process [24,27,28]. Therefore, there is a constant search to increasingly incorporate the use of immersive technologies [29]. While it is true that virtual environments were adopted originally and mainly as a technology for the visualization and presentation of architectural designs, by the time, research has made important contributions to the development of practices such as the discovery of non-accessible spaces, the evaluation of the quality of the space before its construction, simulations of user behavior [30], to the acquisition of a better understanding of proposals and designs of solutions, the interaction of participants with the projected environment, and the improvement of the perception of spatial compositions [24]. Finally, it has also facilitated communication in the process of a design project and efficiently supported participatory design in decision-making [31]. The importance of the teaching of architectural design is explained in the following section.

1.4. IVR in the Teaching of Architectural Design

The use of IVR in the teaching of design, urbanism, construction, and other academic areas of architect training has not been much studied in the light of new learning theories [32], nor has the issue of learning experience been specifically addressed, but the importance of evaluating its usability to improve the effectiveness of applications is emphasized [33]. Among the experiences of the use of IVR in the teaching of architectural design, two types stand out: virtual tours in heritage contexts, and the development of projects in virtual spaces [34], both types take advantage of the important characteristic of this technology, which is the realistic and real-scale visualization of architecture. This enables new ways of studying and interpreting old constructions and construction projects, and even new paradigms of virtual architecture, free from the physical restrictions existing in traditional architecture [35]. However, most research does not take advantage of the characteristics of free creation of forms that give rise to architectural spaces.

It is necessary to design pedagogical processes and more research to consolidate the importance and influence of IVR in the teaching of architecture, and that they close the gap between professional use, education, and research [30]. In the literature review, there is an apparent resistance to deepening the use of IVR in the academic field [36] either because educational institutions are not yet able to adapt to this rapid development of tools to support creativity [37] or for other reasons. Despite this, there is an important advance as reported by Ummihusna and Zairul [38] that confirms that in the teaching of architecture, IVR has a positive influence on students in: (i) affective development, facilitating the performance of learning in terms of commitment, expectations, motivation, immersion, confidence, and level of satisfaction; (ii) cognitive development, improving the ability to apply knowledge and understand spatial qualities, facilitating imagination and the enrichment of creativity, and improving self-assessment and critical thinking. To all this, Jensen and Konradsen [9] add that psychomotor development, related to the movement of the head, such as visual exploration or observation skills, is also a positive result.

In addition, much of the research shows that IVR use has important advantages in the teaching of design, including (i) exploring and expressing ideas in the design process in comparison to traditional methods [39]; (ii) significantly influencing the creativity of the design process [37]; (iii) allowing for immediate feedback to be given to designers, which is not possible within other three-dimensional models [39]; (iv) simulating the feeling of presence in a virtual space similar to a real physical space [32,40,41]; and (v) reflecting and understanding the architectural sensory experiences of the designed space, leading to significant results [42]. From this perspective, its application also allows for exploratory experiences suitable for research tasks [23] oriented to a reflection of the relationships and spatial experience of complex designs, stimulating design in a holistic way through the human experience itself [43].

However, in order for IVR to become a tool that achieves the aforementioned, it must be achieved that (i) students, rather than being passive observers, must become active users by developing exploration-based learning paradigms [23] and (ii) students must be able to create their own content rather than just using IVR as a simulation tool [9]. If this is not achieved, it can be the case that, as has been shown in several research studies [44,45], non-immersive virtual learning scenarios can achieve greater learning than IVR scenarios. Therefore, it is recommended for optimal learning with IVR to avoid virtual interactions irrelevant to the learning task and to train students not only in basic concepts but also in the use of the virtual interaction tools themselves [46].

It is clear that research is demonstrating the importance of IVR in the architectural design process; however, the findings reported previously have concentrated almost entirely on realistic virtual tours on a real scale [35], and those who experimented with the development of projects in virtual spaces [34] have done so in the intermediate and final phases of the design process, i.e., in the preliminary project or the project design development, which are the initial phases of the design process known as the conceptual design and schematic design, the phases that do not thoroughly explore the use, application, and benefits of this technology. It is in these phases that architectural designs are actually created; the concept begins to become space through the creation of space-configuring elements and their manipulation to qualify the space. Here, a gap in knowledge may be found, which at the same time poses a challenge, since teaching and learning this according to a new paradigm of virtual education implies a change of mentality for IVR as a means and not an end in itself. To do so, a new culture needs to be created to make sense of where virtual representation media are valued as practical taxonomies to achieve an enriched and updated dialogue to “visually express” what is being created in architecture [34].

1.5. Justification

In the classical teaching–learning of architectural space that takes place in university design workshops, drawings, models, and three-dimensional digital models are used to understand the space, to configure it, and to perceive it; however, these forms of learning are allocentric, i.e., they are given in the third person and do not allow the space to be experienced as such, i.e., as in the case when a person apprehends a space in the first person and in an egocentric way by living it, walking through it, and experiencing it [47]. Therefore, the teaching of the design of architectural space is limited by this lack of direct perception of the spaces that are being designed, which mainly allows the new student to understand the space he or she is designing in order to configure it in a better way.

From the philosophy and theory of architecture, architectural space is conceived of as a communicative act intrinsically mediated through perception [48]; thus, the teaching–learning process of architectural space must have the purpose of promoting an integral reading of the codes of this language. Here, it is important to emphasize that human beings do not perceive the physical world but the psychological construct that depends on their own relationship with the environment [49], i.e., their previous experiences. So, since perception identifies the participation of context, personal experience, and expectations [50] as determining categories in the phenomenological construction of architectural space [49], the experience of living and experiencing it is recognized as another way in which the student learns architecture and precisely resorts to that experience to analyze a space or design it [51]. If the models and three-dimensional models explained above are added to the deficiency of drawings, along with situations that make it impossible to experience built spaces directly, or the students’ previous personal experiences are very limited, then the traditional way of teaching the design of architectural space will show serious shortcomings in its purpose.

In summary, IVR is a significant tool in the teaching of architectural design because it also allows one to experience the sensation of “being in the place” rather than “looking at the place” and thus to experience a full-scale replicated environment to enhance ideas in the design process [6]. As the student’s intentions and imagination are superimposed on the virtual model space, IVR becomes a tool to better understand the design problem and deepen the materialization and experience of the space [52]. However, even more important is the possibility to create each configuring element of the space in order to understand what is being designed. This is of utmost importance, especially in students who start in the career of architecture because it allows them to engage in an egocentric experience of space in its real dimension, which opens them up to a more holistic expression and imagination of space [53,54]. From this perspective, the current advances presented by IVR lead to a rethinking of the methods of teaching architectural design in architectural workshops and of the task of teachers to proactively facilitate these changes through the study, application, and communication of this technology and to interact at the time of providing feedback, which helps them to discover possible hidden errors [25].

1.6. The Purpose of the Research

The purpose of this research is to determine how the use of Immersive Virtual Reality incorporated in the initial phases of the architectural design process can help students to achieve the competences of configuring a space, qualifying a space with the use of light, color, degree of enclosure, and materials and to design a volumetric composition with aesthetic principles. A second objective is for the professors in charge of teaching the first-year architectural design subject to carry out an evaluation of the advantages and disadvantages of using this tool in the design process.

2. Methodology

This study was carried out as part of the Architectural Design Workshop subject in the first year of the architecture degree. The aim was to introduce students to the knowledge of space, thus helping them understand its configuration, its qualification, and its role as a fundamental element of architecture. To this end, their training is based on learning through discovering and constructing spatial ideas, which allows them to gain knowledge based on their own experiences of sensibility and tangible objects that are materialized through graphic expression and models. From this perspective, the exercises are designed to develop their ability to perceive and interpret the principles and structures that organize a space, culminating in a simple spatial solution. The subject assumes themes that involve theoretical foundations on composition, form, configuration, and spatial perception, as well as considering the theme of context in the synthesis exercise of the subject. The methodology involves the following stages: (i) research on the subject and context; (ii) analysis and interpretation on the information obtained; (iii) a concept approach, after an understanding of the design problem; (iv) decision making based on critical reflection; and (v) materialization of the object. An important aspect that has been observed in this process is the difficulty that students have in beginning to materialize their concept through diagrams, through graphics, and in the model, and to understand and perceive what they are beginning to create.

In the specific case of the research, the assignment was to design an ephemeral architecture of a recreational and playful nature to be installed in the courtyard of a colonial house with a strong historical connotation and whose current use is the dissemination of art and culture. The competencies to be achieved in the experimentation were:

• Being able to configure a space through vertical elements, ceilings, and floors: we want to know how students understand and perceive the elements that they are building and that configure their space, i.e., the vertical elements such as walls and columns, and the horizontal elements such as ceilings, roofs, and floors, which together configure their final space and that, being immersed in the virtual space and on a real scale, they can immediately reformulate.

• Being able to optimize the architectural space using light, color, degree of enclosure, and materials: we want to know how the students understand and perceive that the entry of light into the space, the greater or smaller enclosure of the space, and the use of colors and materials give a different character to the space, which they understand and perceive better when they are inside the it, experiencing it.

• Being able to design a volumetric composition with unity and aesthetic experience: we want to know how the students understand and perceive if the final form of their space maintains principles of aesthetic composition and how the egocentric perception of that form, i.e., the perception of the space on a real scale by final users walking through, touring, and appreciating the space as pedestrians, changes with respect to the initial perception obtained from the models and drawings.

Therefore, the design process considered the first moment of research on the issue (ephemeral architecture). A second moment considered the analysis and interpretation of the premises indicated and the place of installation. With all of this, the students elaborated on their first conceptual idea, which was expressed in drawings, texts, and models. It is from this moment that the use of IVR was considered as a tool to create architecture, make improvements, and get feedback in real time of what was being designed and constructed virtually.

2.1. The Participants

For the experimentation of the pilot study, 12 students in their first year of architectural studies at San Agustín National University in Peru were recruited (they participated voluntarily). In the recruitment of students, an archetype was defined with the following characteristics: having passed the first two courses of design and architectural drawing; being highly familiarized with computer technologies; having an interest in video games; not suffering from color blindness or epilepsy; and not having an easy tendency to dizziness, nausea, or migraine. Out of the 76 first-year students recruited, only 12 of them fit the profile. The students participating in the pilot study were six women and six men, and their average age was 19.5 years old.

Furthermore, for the evaluation of the results, the 14 specialized architectural teachers in the subject of basic architectural design (of the first year) of the same university were invited, of which only 12 participated.

2.2. Measuring Instruments

An ad-hoc questionnaire was designed to be applied after the experience of designing with IVR in order to know the opinions of the students participating in the pilot study. The questionnaire included three questions. The first one was a closed question. The students had to sort by relevance the six aspects that had most influenced their design using IVR. The aspects were: (a) perception of a building on a real scale, (b) experiencing the interior space, (c) interaction with the existing environment, (d) experimenting with 3D shapes, (e) perception of light and shadows, and (f) experimenting with different materials. The second and third questions were open-ended. The students had to mention two advantages and two disadvantages of using IVR in their design process.

For the evaluation of the usefulness of IVR in helping students to achieve the competences of configuring a space, qualifying a space with the use of light, color, degree of enclosure and materials, and designing a volumetric composition with aesthetic principles, a grade sheet was created based on the rubrics that teachers often use to evaluate in the architectural design course. This grade sheet had five evaluation criteria: (i) the idea or concept, (ii) the configuration of the space, (iii) the qualification of the space, (iv) the composition of the form, and (v) the graphic representation in two or three dimensions. However, it was only criteria 2, 3 and 4 that would directly assess the contribution of IVR. The evaluation had a 20-point-based scale (see Table 1).

Finally, for the evaluation of the advantages and disadvantages of the use of this tool in the architectural design process, another ad-hoc questionnaire was designed with seven questions to know the opinions of the teachers evaluating the pilot study.

The first four questions used a five-level Likert Scale to determine the usefulness of using IVR in the four most important stages of architectural design, which are: (a) conceptual design, (b) schematic design, (c) preliminary project, and (d) project.

The fifth question was the same question that appears in both the student and the teacher questionnaire, where they must sort by relevance the six aspects that had most influenced the design of the students using IVR. The aspects were: (a) perception of building on a real scale, (b) experiencing the interior space, (c) interaction with the existing environment, (d) experimenting with 3D shapes, (e) perception of light and shadows, and (f) experimenting with different materials.

Finally, the last two questions were open-ended questions in which teachers were asked to indicate two advantages and two disadvantages of using IVR in the student design process.

2.3. Experimental Design

The experimental design was classified into four phases. In a preliminary session, the 12 participants were informed about the objectives and intentions of the experiment, and the schedule of activities, and their informed consent to participate in the study was requested. Figure 2 shows the sequence of the four phases:

The first phase required students to propose a conceptual design of an ephemeral architecture for recreational purposes to be installed in the patio of the colonial-style house called “Chávez de la Rosa” in the Historic Center of the city of Arequipa. For this, two architect professors explained for 45 min the problem of the design, the place, the program, the user, and the definition and examples of ephemeral architecture and its usefulness in historic centers. After that, for 120 min, the students worked on their conceptual design in the traditional way, that is, with handmade sketches and explanatory texts. Each of them presented their proposal and then received feedback from both professors, who were experts in basic architectural design. Figure 3 shows the assignments carried out by each of the 12 students.

In the second phase, the students were required to devise the schematic design of the ephemeral space that they had proposed and begun in the first phase. Based on the feedback of phase one and their own reflections, the 12 participants drew and created a model with cardboard (see Figure 4). In the session with the same two teachers, they re-presented their proposals and received feedback again. Students were asked not to make any more changes to the paper sheets or models, but they could make changes in the third phase using IVR.

The purpose of the third phase was for the students to continue the schematic design of phase two but this time in a virtual environment and with the help of the Google Tilt Brush application. Before starting the design using IVR, the students were given brief training for fifteen minutes to learn about the basic tools of the software and to adapt to the virtual environment. After a brief three-minute pause, each participant was asked if they felt any discomfort related to the presence of cyber disease. No student reported discomfort, so we continued with the experimental phase of continuing the schematic design using IVR for a period of 60 min, with an intermediate pause of five minutes to ask again of any symptoms or discomfort related to the use of HMDs. Before starting the design with IVR, the images of the designs made in phase two and the built environment of the colonial house where the new ephemeral architecture was to be created were uploaded to the Tilt Brush platform (Figure 5). The task was to use the tools of freehand strokes, straight strokes, and other tools for the creation of various shapes, as well as the color palette and the scaling tool. At the end of the session, the students were administered questionnaire 1 to know their opinions of the use of the tool on the schematic design they had made.

Finally, the fourth phase consisted of the presentation and explanation of each student of the results obtained in the three phases. The protocol for this phase was as follows: each student presented their conceptual and schematic design using traditional methods (phase 1 and 2) for five minutes and answered questions for two minutes, and then the 12 evaluating teachers established a grade based on the grade sheet. Immediately afterwards, the same student presented this same design for three minutes but this time helped by the three-dimensional model hosted on the Sketchfab platform. The student turned the model, toured it, entered the space, and flew over it to better show the final result created in IVR. After the explanation, the same 12 teachers re-graded the design using the same grade sheet. In the end, the 12 students and teachers completed questionnaire 2 to record their appreciation and opinions about the use of the tool in schematic design.

3. Results

The results obtained in the first questionnaire applied to the students are shown in Figure 6. The students expressed, valuing the questions on a Likert scale of 5 levels, that the perception of architecture at full scale and the experimentation of the interior space are the most relevant aspects of the use of IVR when they are designed, while the effects of light and shadow and the perception of the constructive materials were considered the least relevant aspects.

Students’ views on the use of HMDs and IVR for design are shown in Figure 7. Among the advantages that are considered the most important over the use of IVR in architectural design is the possibility of perceiving the space on a real or natural scale; this is the most highly rated advantage. The other advantages have fewer mentions but are equally important.

Furthermore, regarding the question of mentioning the most important disadvantages found in the design experience with IVR, they mentioned as the most relevant the lack of mastery and management of the Tilt Brush application used with haptic controls, followed by the discomfort of prolonged use of HMDs and three other disadvantages, which can be seen in Figure 8.

The second questionnaire applied only to teachers; it was established in the first instance that most specialist architects consider that the use of IVR in architectural design is more useful in the final stages of the design process, that is, in the preliminary and final project, and less useful in the conceptual stage, as shown in Figure 9.

Figure 10 and Figure 11 show, according to the teachers’ point of view, the advantages and disadvantages of using IVR in schematic design. For them, the greatest advantages are the experimentation of the interior space and the possibility of touring the building; the major disadvantages are the short training time with the Tilt Brush tool and the short amount of time available to access this type of technological tool.

Finally, Figure 12 shows that professors have observed a higher rate of achievement of competences 2, 3, and 4, which means configuring a space through different elements; qualifying a space with the use of light, color, degree of enclosure, and materials; and designing a volumetric composition with aesthetic principles when students use IVR as a design tool, compared to results using traditional methods such as two-dimensional drawings and models.

The Shapiro–Wilk normality test determined that the items used in the rubric do not have a normal distribution, so for comparison, the Wilcoxon test was used. On the other hand, the overall scores did show normal distribution (p > 0.05), so the Student’s t-test was used for paired samples.

Table 2 shows that there are statistically significant differences in the qualification of item 3 (space qualification, Z = −2.724; p = 0.006; and r_bp = −0.952) with a large effect size and in that of item 5 (graphical representation, Z = −2.070; p = 0.038; r_bp = −0.421) with a moderate effect size. In both items, the scores after having used IVR are significantly higher.

Moreover, Table 3 shows us that in terms of the overall rating, there are statistically significant differences between before and after the use of IVR (t = −3.517; p = 0.005; and d = −1.015) with a large effect size, indicating that there are higher ratings after the experimental sessions.

The individual rating of each project before and after the use of IVRs can be seen in Figure 13. In all cases except for the second and seventh students, grades remained the same or improved.

4. Discussion

The students’ experience of designing a small architectural space in a virtual environment allowed them to assess six architectural features proposed by the research team. The two best-valued characteristics that have a mutual relationship were the perception of building on a real scale and experiencing the interior space. As mentioned in the introduction, IVR technology allows one to improve aspects of architectural design by the ability to show the elements on a real scale [28,55,56] and by allowing the user to perceive of the space in an experiential way [32], which allows one to incorporate the human experience in the design [43].

Moreover, after having designed the architectural space using IVR, and before an open question, the students agreed that the most relevant advantage was the possibility of perceiving the space designed on a real scale, and secondly, they established the possibility of visualizing the real environment in the virtual space, while it could interact in such an environment, which in this case with the walls and floor of the colonial courtyard, as well as the blue sky. Again, two characteristics of IVR are highlighted: one was that it allowed experimental learning, by allowing first-person experiences, and the other was that it improved the ability to learn from the environment, not only by observing it but also by interacting with it [32].

The other advantages that were less mentioned are important to comment on since they refer to the fact that in virtual environments you can experiment with new shapes, and greater geometric freedom, which can be executed easily and quickly, in addition to being able to be effortlessly corrected. These three advantages mentioned by the students lead us to recognize that two-dimensional drawing (which in many cases is a limitation for some students), when using IVR, becomes a three-dimensional and natural construction of the object or space; therefore, the drawing becomes design, and the design becomes the construction of the architectural space.

At the end of the students’ survey, they established that the most important disadvantage of using IVR in their design process was the short time of use and therefore the lack of mastery of the software used for design in IVR (Tilt Brush). We believe that this disadvantage can become a strength after doing a few more hours of training. The second and fifth disadvantages most named by the students—the discomfort from long-term use, and dizziness and eyestrain—are negative effects of the use of virtual reality that, along with other symptoms, are known as simulator disease or cyber disease. In this case, it did not happen beyond the mentioned references, and this must be taken into account because if they appear, they could skew the results of the other dependent variables [57]. The other two disadvantages were the deficiency to achieve greater accuracy of the strokes and the deficiency to represent materials more realistically. Both disadvantages mentioned by the students are not aspects that should be evaluated in this experimentation since it has been intended to achieve a schematic design where precision and realism are not important; on the contrary, a less realistic and less precise representation should be the archetype of the schematic design.

From the systematic review carried out by Ummihusna and Zairul [38], it is clear that most of the research that used IVR in architecture education as a means of exploration was applied from the third year for the architectural design process in the phases of the preliminary and final project [33]. That same trend was collected from the 12 master architects when they mostly opined that IVR can be more useful in the final two phases of architectural design. However, after having evaluated this pilot study, there were 9 of 12 architects who have opined that this technology can also be extremely useful for schematic design reasons for this pilot study.

The two advantages that have been most mentioned by the evaluating architects have been the experimentation of the interior space and the possibility of walking in and through the space; both are characteristics that only IVR can provide and that from the pedagogical point of view are of great importance. Teachers agree with students that the main disadvantage is the little time spent on training and on using the app in experimentation. Teachers recognize that greater mastery of the application would allow for better responses in the same design process and final results. The second-most-considered disadvantage was the lack of accessibility to this technology. In Latin America, it is very difficult to find a virtual reality laboratory that allows more experiments with larger populations.

Regarding the quantitative evaluation carried out by the specialist teachers, it has been found that the item with the highest rating before and after the use of IVR was the graphic representation. The graphic representation in phases 1 and 2 refers to the two-dimensional drawing that students already master after having completed their first year of studies and having taken subjects such as architectural drawing and descriptive geometry. In phase 3, i.e., after designing using IVR, graphical representation refers to the presentation of the design in three dimensions. In immersive environments, this ideal conception of drawing as an experience of architectural creation happens naturally, where drawing means projecting at the same time [58] and where drawing three-dimensionally freehand has proven to be suitable and have great potential for drawing the preliminary ideas of schematic design [59]. The drawing, in turn, represents design, and this is in turn represents the “virtual” construction of space.

As for the architectural design itself, after evaluation with the same grade sheet, the results showed that the qualification of the space using light, color, degree of enclosure, and materials displayed a significant improvement of large effect in the score obtained after using IVR, compared to that shown with two-dimensional drawings and the models of phases 1 and 2. This aspect is very important in architecture because the internal space cannot be fully represented in any form, neither learned nor lived, but through students’ own experiences [60] and virtual reality it has brought them very close to this, so that they have learned this competence in a better way.

Statistically, the other items evaluated have not represented a significant improvement. However, the evaluating teachers established that the three-dimensional model resulting from the design and freehand drawing in 3D using IVR allows for a better perception of the volumetric composition and its aesthetic experience (item 4), as well as a better configuration of the space through vertical elements, and horizontal elements such as ceiling and floor (item 2).

5. Conclusions

The introduction of IVR in the schematic phase of the architectural design process of a small-scale ephemeral architecture by students in the first year of architecture studies has important academic and pedagogical advantages over the traditional method using drawings and models. The main advantages found by the teachers are the possibility of experimentation of the space in the first person, and the possibility of touring the building internally, which is closely related to the main advantage expressed by the students, which was the perception of space on a real scale. This advantage provided by IVR, thanks to the immersion and presence, is of high pedagogical value because it allows students to better understand what they are designing, contrast their proposals with the built environment where it will be located, and rethink it immediately and easily based on the direct feedback they receive from the virtual three-dimensional design. The most significant disadvantage encountered by both teachers and students is the limited training and use that was offered to operate the software used to virtually design, but this turned out to be more of an opportunity for improvement.

The quantitative rating using a grade sheet made by the teachers of the design using IVR was superior and statistically significant compared to the evaluation made on the traditional design. This finding allows us to conclude that the advantages expressed by teachers and students are significant and objectively influenced the design. This means that of the three competencies studied, the competency of the qualification of the space using light, color, degree of enclosure, and materials further strengthens this important advantage of IVR of allowing the experimentation and perception of the space designed on a real scale. Another item that has shown statistical significance has been the graphic representation of the design, which allows us to conclude that drawing or graphically representing a small architectural design using IVR implies building it in three dimensions, which allows for better and more efficient communication of the design, which is the main function of all graphic representation. It so happens that in IVR, the process of drawing in three dimensions is, in turn, a design process, and this, in turn, is a process of building space.

Finally, we can conclude that teachers have found a useful tool in the use of IVR during the conceptual design phase that is almost comparable to the usefulness that has been appreciated through research in the phases of the preliminary and final project.

We consider as future work the incorporation of IVR into the conceptual phase of design and the comparison of ephemeral architectural proposals in public spaces in Historical Centres in Peru, Mexico, and Spain. We also consider as future work the comparison of the results of the perception of first year students with those students in their final years who have had more training in design.

Author Contributions

Conceptualization, H.C.G.-T. and M.A.C.; data curation, L.V.S. and J.M.-G.; formal analysis, H.C.G.-T., L.V.S. and J.M.-G.; funding acquisition, H.C.G.-T.; investigation, H.C.G.-T., M.A.C. and J.M.-G.; methodology, H.C.G.-T. and M.A.C.; project administration, H.C.G.-T.; resources, H.C.G.-T.; supervision, H.C.G.-T., M.A.C. and J.M.-G.; validation, J.M.-G.; visualization, L.V.S.; writing—original draft, H.C.G.-T., M.A.C. and L.V.S.; writing—review and editing, H.C.G.-T. and J.M.-G. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by Universidad Nacional de San Agustín de Arequipa, grant number IBA-IB-01-2019-UNSA.

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

The data presented in this study are available on request from the corresponding author.

Acknowledgments

Thanks to the students of the Universidad Nacional de San Agustín de Arequipa (Peru) for their generous participation in this study.

Conflicts of Interest

The authors declare no conflict of interest.

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Figures and Tables

Figure 1. Immersive and non-immersive Virtual Reality.

Figure 2. The four phases of experimental design.

Figure 3. The first phase: a conceptual design by the 12 students, using the traditional method.

Figure 4. The second phase: a schematic design by the 12 students, using the traditional method.

Figure 5. The third phase: a schematic design using IVR by the 12 students, on the Sketchfab platform. (https://sketchfab.com/johnbel/collections/rv4 (accessed on 18 February 2022)).

Figure 6. The IVR items that most influenced the design, according to students; on a Likert scale of 0–5.

Figure 7. The most highly rated advantages of using IVR in design, according to students.

Figure 8. The most highly rated disadvantages of using IVR in design, according to students.

Figure 9. The most highly rated usefulness of IVR through the various phases of architectural design, according to teachers.

Figure 10. The most highly rated advantages of using IVR in design, according to teachers.

Figure 11. The most highly rated disadvantages of using IVR in design, according to teachers.

Figure 12. The qualifications of the five items before and after the use of IVR in design.

Figure 13. Individual qualifications before and after using IVR.

Table 1

The grade sheet for a before and after design using IVR.

N°	Items	Max. Points
1	Communicates a conceptual idea linked to the ludic and recreational theme	5
2	Configures a space through vertical elements, the ceiling, and the floor	4
3	Qualifies the space using light, color, degree of enclosure, and materials	5
4	Designs a volumetric composition with unity and aesthetic expertise	4
5	Represents graphically the design in 2D or 3D, in a comprehensive way	2
Total		20

Table 2

A comparison of the rating of each grade sheet item before and after using IVR.

n = 12		Mean	DE	Z	p	rbp
Item 1	Pre test	3.375	0.5691	−0.632	0.527	-
	Post test	3.458	0.6201
Item 2	Pre test	3.167	0.3892	−1.633	0.102	-
	Post test	3.333	0.4438
Item 3	Pre test	3.375	0.6784	−2.724	0.006	−0.952
	Post test	3.958	0.5418
Item 4	Pre test	3.083	0.2887	−1.933	0.053	-
	Post test	3.375	0.3108
Item 5	Pre test	1.708	0.3965	−2.070	0.038	−0.421
	Post test	2.000	0.0000

Table 3

A comparison of the final rating before and after using IVR.

n = 12	Mean	DE	t	gl	p	d
Pre test	14.708	1.7117	−3.517	11	0.005	−1.015
Post test	16.125	1.5393

References

1. Harputlugil, T.; Gültekin, A.T.; Prins, M.; Topçu, Y.İ. Architectural Design Quality Assessment Based on Analytic Hierarchy Process: A Case Study. METU J. Fac. Archit.; 2014; 31, pp. 139-161. [DOI: https://dx.doi.org/10.4305/METU.JFA.2014.2.8]

2. Davis, D. Design as a Process the Project Development Process. Proceedings of the 1997 Annual Conference; Milwaukee, Wisconsin, 15–18 June 1997; pp. 2.130.1-2.130.7.

3. Sullivan, B. The Design Studio Method: Creative Problem Solving with UX Sketching; Routledge: London, UK, 2015; ISBN 1-315-77702-9

4. Demirkan, H.; Hasirci, D. Hidden Dimensions of Creativity Elements in Design Process. Creat. Res. J.; 2009; 21, pp. 294-301. [DOI: https://dx.doi.org/10.1080/10400410902861711]

5. Prit Kaur, D.; Mantri, A.; Horan, B. Design Implications for Adaptive Augmented Reality Based Interactive Learning Environment for Improved Concept Comprehension in Engineering Paradigms. Interact. Learn. Environ.; 2019; 30, pp. 1-19. [DOI: https://dx.doi.org/10.1080/10494820.2019.1674885]

6. Zaini, A.I.; Embi, M.R. Virtual Reality for Architectural or Territorial Representations: Usability Perceptions. Int. J. Built Environ. Sustain.; 2017; 4, pp. 131-138. [DOI: https://dx.doi.org/10.11113/ijbes.v4.n2.185]

7. Lowood, H.E. Virtual Reality. Available online: https://wwsw.britannica.com/technology/virtual-reality (accessed on 14 August 2021).

8. Lanier, J. Virtual Reality: The Promise of the Future. Interact. Learn. Int.; 1992; 8, pp. 275-279.

9. Jensen, L.; Konradsen, F. A Review of the Use of Virtual Reality Head-Mounted Displays in Education and Training. Educ. Inf. Technol.; 2018; 23, pp. 1515-1529. [DOI: https://dx.doi.org/10.1007/s10639-017-9676-0]

10. Suh, A.; Prophet, J. The State of Immersive Technology Research: A Literature Analysis. Comput. Hum. Behav.; 2018; 86, pp. 77-90. [DOI: https://dx.doi.org/10.1016/j.chb.2018.04.019]

11. Schroeder, R. Possible Worlds: The Social Dynamic of Virtual Reality Technology; Westview Press, Inc.: Boulder, CO, USA, 1996; ISBN 0-8133-2955-8

12. Makhkamova, A.; Exner, J.-P.; Greff, T.; Werth, D. Towards a Taxonomy of Virtual Reality Usage in Education: A Systematic Review. Augmented Reality and Virtual Reality; Springer: New York, NY, USA, 2020; pp. 283-296.

13. Rogers, S. Virtual Reality: The Learning Aid of the 21st Century. Secondary Virtual Reality: The Learning Aid of the 21st Century. 2019; Available online: https://www.forbes.com/sites/solrogers/2019/03/15/virtual-reality-the-learning-aid-of-the-21st-century/?sh=35df6e26139b (accessed on 13 March 2021).

14. Krokos, E.; Plaisant, C.; Varshney, A. Virtual Memory Palaces: Immersion Aids Recall. Virtual Real.; 2019; 23, pp. 1-15. [DOI: https://dx.doi.org/10.1007/s10055-018-0346-3]

15. Merchant, Z.; Goetz, E.T.; Cifuentes, L.; Keeney-Kennicutt, W.; Davis, T.J. Effectiveness of Virtual Reality-Based Instruction on Students’ Learning Outcomes in K-12 and Higher Education: A Meta-Analysis. Comput. Educ.; 2014; 70, pp. 29-40. [DOI: https://dx.doi.org/10.1016/j.compedu.2013.07.033]

16. Ma, M.; Jain, L.C.; Anderson, P. Virtual, Augmented Reality and Serious Games for Healthcare 1; Springer: New York, NY, USA, 2014; Volume 1, ISBN 3-642-54815-6

17. ter Haar, R. Virtual Reality in the Military: Present and Future. Proceedings of the 3rd Twente Student Conference IT; Enschede, The Netherlands, 20 June 2005; Citeseer: Forest Grove, OR, USA, 2005.

18. Moshell, M. Three Views of Virtual Reality: Virtual Environments in the US Military. Computer; 1993; 26, pp. 81-82. [DOI: https://dx.doi.org/10.1109/2.192003]

19. Wittstock, V.; Lorenz, M.; Pürzel, F.; Riedel, T. Immersive Presentations: Enabling Engaging Virtual Reality Based Training and Teaching by Merging Slide-Based and vr-Based Elements. Enabling Manufacturing Competitiveness and Economic Sustainability; Springer: New York, NY, USA, 2014; pp. 125-130.

20. Robb, R.A. Virtual Reality in Medicine and Biology. Information Technologies in Medicine, Medical Simulation and Education; Wiley-IEEE Press: New York, NY, USA, 2001.

21. Smith, S.; Ericson, E. Using Immersive Game-Based Virtual Reality to Teach Fire-Safety Skills to Children. Virtual Real.; 2009; 13, pp. 87-99. [DOI: https://dx.doi.org/10.1007/s10055-009-0113-6]

22. Muñoz-Saavedra, L.; Miró-Amarante, L.; Domínguez-Morales, M. Augmented and Virtual Reality Evolution and Future Tendency. Appl. Sci.; 2020; 10, 322. [DOI: https://dx.doi.org/10.3390/app10010322]

23. Checa, D.; Bustillo, A. A Review of Immersive Virtual Reality Serious Games to Enhance Learning and Training. Multimed. Tools Appl.; 2020; 79, pp. 5501-5527. [DOI: https://dx.doi.org/10.1007/s11042-019-08348-9]

24. Brandão, G.V.L.; do Amaral, W.D.H.; de Almeida, C.A.R.; Castañon, J.A.B. Virtual Reality as a Tool for Teaching Architecture. Proceedings of the International Conference of Design, User Experience, and Usability; Las Vegas, NV, USA, 15–20 July 2018; Springer: New York, NY, USA, 2018; pp. 73-82.

25. Wang, P.; Wu, P.; Wang, J.; Chi, H.-L.; Wang, X. A Critical Review of the Use of Virtual Reality in Construction Engineering Education and Training. Int. J. Environ. Res. Public Health; 2018; 15, 1204. [DOI: https://dx.doi.org/10.3390/ijerph15061204]

26. Moloney, J.; Globa, A.; Wang, R.; Khoo, C. Principles for the Application of Mixed Reality as Pre-Occupancy Evaluation Tools (P-OET) at the Early Design Stages. Archit. Sci. Rev.; 2019; 63, pp. 441-450. [DOI: https://dx.doi.org/10.1080/00038628.2019.1675138]

27. Galán Serrano, J.; García-García, C.; Felip Miralles, F. Experiencias Inmersivas Durante La Fase Conceptual Del Proyecto Arquitectónico: La Realidad Virtual Como Herramienta Para La Participación Del Usuario En El Proceso de Co-Creación. EGE Rev. Expresión Gráfica Edif.; 2018; 10, pp. 50-57. [DOI: https://dx.doi.org/10.4995/ege.2018.12447]

28. Liu, Y.; Castronovo, F.; Messner, J.; Leicht, R. Evaluating the Impact of Virtual Reality on Design Review Meetings. J. Comput. Civ. Eng.; 2019; 34, 04019045. [DOI: https://dx.doi.org/10.1061/(ASCE)CP.1943-5487.0000856]

29. Noghabaei, M.; Heydarian, A.; Balali, V.; Han, K. Trend Analysis on Adoption of Virtual and Augmented Reality in the Architecture, Engineering, and Construction Industry. Data; 2020; 5, 26. [DOI: https://dx.doi.org/10.3390/data5010026]

30. Portman, M.E.; Natapov, A.; Fisher-Gewirtzman, D. To Go Where No Man Has Gone before: Virtual Reality in Architecture, Landscape Architecture and Environmental Planning. Comput. Environ. Urban Syst.; 2015; 54, pp. 376-384. [DOI: https://dx.doi.org/10.1016/j.compenvurbsys.2015.05.001]

31. Zhang, C.; Chen, B. Enhancing Learning and Teaching for Architectural Engineering Students Using Virtual Building Design and Construction. High. Educ. Stud.; 2019; 9, pp. 45-56. [DOI: https://dx.doi.org/10.5539/hes.v9n2p45]

32. Maghool, S.A.H.; Moeini, S.H.I.; Arefazar, Y. An Educational Application Based on Virtual Reality Technology for Learning Architectural Details: Challenges and Benefits. ArchNet-IJAR Int. J. Archit. Res.; 2018; 12, 246. [DOI: https://dx.doi.org/10.26687/archnet-ijar.v12i3.1719]

33. Keenaghan, G.; Horváth, I. State of the Art of Using Virtual Reality Technologies in Built Environment Education. Proceedings of the Tenth International Symposium on Tools and Methods of Competitive Engineering, TMCE; Budapest, Hungary, 19–23 May 2014; pp. 19-23.

34. Wagemann, E.; Martínez, J. Realidad Virtual (RV) Inmersiva Para El Aprendizaje En Arquitectura. EGA Expr. Gráfica Arquit.; 2022; 27, pp. 110-123. [DOI: https://dx.doi.org/10.4995/ega.2022.15581]

35. Hernández, L.; Taibo, J.; Seoane, A.; Jaspe, A. La Percepción Del Espacio En La Visualización de Arquitectura Mediante Realidad Virtual Inmersiva. EGA Rev. Expr. Gráfica Arquit.; 2011; 16, pp. 252-261. [DOI: https://dx.doi.org/10.4995/ega.2011.1110]

36. Fonseca, D.; Cavalcanti, J.; Peña, E.; Valls, V.; Sanchez-Sepúlveda, M.; Moreira, F.; Navarro, I.; Redondo, E. Mixed Assessment of Virtual Serious Games Applied in Architectural and Urban Design Education. Sensors; 2021; 21, 3102. [DOI: https://dx.doi.org/10.3390/s21093102]

37. Obeid, S.; Demirkan, H. The Influence of Virtual Reality on Design Process Creativity in Basic Design Studios. Interact. Learn. Environ.; 2020; pp. 1-19. [DOI: https://dx.doi.org/10.1080/10494820.2020.1858116]

38. Ummihusna, A.; Zairul, M. Investigating Immersive Learning Technology Intervention in Architecture Education: A Systematic Literature Review. J. Appl. Res. High. Educ.; 2021; 14, pp. 264-281. [DOI: https://dx.doi.org/10.1108/JARHE-08-2020-0279]

39. Schnabel, M.A. The Immersive Virtual Environment Design Studio. Collaborative Design in Virtual Environments; Springer: New York, NY, USA, 2011; pp. 177-191.

40. Kuliga, S.F.; Thrash, T.; Dalton, R.C.; Hölscher, C. Virtual Reality as an Empirical Research Tool—Exploring User Experience in a Real Building and a Corresponding Virtual Model. Comput. Environ. Urban Syst.; 2015; 54, pp. 363-375. [DOI: https://dx.doi.org/10.1016/j.compenvurbsys.2015.09.006]

41. Yeom, D.; Choi, J.-H.; Kang, S.-H. Investigation of the Physiological Differences in the Immersive Virtual Reality Environment and Real Indoor Environment: Focused on Skin Temperature and Thermal Sensation. Build. Environ.; 2019; 154, pp. 44-54. [DOI: https://dx.doi.org/10.1016/j.buildenv.2019.03.013]

42. Angulo, A. On the Design of Architectural Spatial Experiences Using Immersive Simulation. Proceedings of the EAEA 11 Conference Proceedings, Envisioning Architecture: Design, Evaluation, Communication; Italy, Milan, 25–28 September 2013; pp. 151-158.

43. Alatta, R.A.; Freewan, A. Investigating the Effect of Employing Immersive Virtual Environment on Enhancing Spatial Perception within Design Process. ArchNet-IJAR Int. J. Archit. Res.; 2017; 11, 219. [DOI: https://dx.doi.org/10.26687/archnet-ijar.v11i2.1258]

44. Richards, D.; Taylor, M. A Comparison of Learning Gains When Using a 2D Simulation Tool versus a 3D Virtual World: An Experiment to Find the Right Representation Involving the Marginal Value Theorem. Comput. Educ.; 2015; 86, pp. 157-171. [DOI: https://dx.doi.org/10.1016/j.compedu.2015.03.009]

45. Makransky, G.; Terkildsen, T.S.; Mayer, R.E. Adding Immersive Virtual Reality to a Science Lab Simulation Causes More Presence but Less Learning. Learn. Instr.; 2019; 60, pp. 225-236. [DOI: https://dx.doi.org/10.1016/j.learninstruc.2017.12.007]

46. Mulders, M.; Buchner, J.; Kerres, M. A Framework for the Use of Immersive Virtual Reality in Learning Environments. Int. J. Emerg. Technol. Learn.; 2020; 15, 208. [DOI: https://dx.doi.org/10.3991/ijet.v15i24.16615]

47. Gomez-Tone, H.C.; Martin-Gutierrez, J.; Bustamante-Escapa, J.; Bustamante-Escapa, P.; Valencia-Anci, B.K. Perceived Sensations in Architectural Spaces through Immersive Virtual Reality. VITRUVIO-Int. J. Archit. Technol. Sustain.; 2021; 6, pp. 70-81. [DOI: https://dx.doi.org/10.4995/vitruvio-ijats.2021.16253]

48. Sánchez, O.; Hessman, D. El Aprendizaje de la Percepción del Espacio Arquitectónico: Una Aproximación a su Comprensión Desde la Experiencia en el Taller de Diseño Uno; Universidad Nacional de Colombia: Manizales, Colombia, 2018.

49. Almagro Holgado, M. Límites de la Noción de’affordance y de la Concepción de Lo Mental en El Marco de la Psicología Ecológica. Teorema Rev. Int. Filos.; 2020; 39, pp. 135-149.

50. Keenan, M. Perception. Encyclopedia of Health; Salem Press: Pasadena, CA, USA, 2020.

51. Gomes, R. Design, Drawing and the Teaching of Architecture through Corporeality. Arquiteturarevista; 2016; 12, 36.

52. Secretariat, A.-E. EAAE Prize 2003–2005-Writings in Architectural Education. Proceedings of the International Workshop in Copenhagen; Copenhagen, Denmark, 27–29 May 2004.

53. Espinosa, C.A.R.; Cuadrado, E.M.T.; González, J.R.A.; Álvarez, D.E.R.; Penso, M.M.; Cruz, O.A.N. de la Aplicativo de realidad virtual inmersiva para el aprendizaje de la composición volumétrica en el diseño arquitectónico. ACE Archit. City Environ.; 2021; 16, 46. [DOI: https://dx.doi.org/10.5821/ace.16.46.9633]

54. Nisha, B. The Pedagogic Value of Learning Design with Virtual Reality. Educ. Psychol.; 2019; 39, pp. 1233-1254. [DOI: https://dx.doi.org/10.1080/01443410.2019.1661356]

55. Tsiutsiura, S.; Bebeshko, B.; Khorolska, K. Vr-Technology as a Modern Architecture Tool. Manag. Dev. Complex Syst.; 2020; 42, pp. 69-74. [DOI: https://dx.doi.org/10.32347/2412-9933.2020.42.69-74]

56. Kreutzberg, A. New Virtual Reality for Architectural Investigations. Proceedings of the Fusion-Proceedings of the 32nd eCAADe Conference; Newcastle upon Tyne, UK, 10–12 September 2014; Volume 2, pp. 253-260.

57. Bimberg, P.; Weissker, T.; Kulik, A. On the Usage of the Simulator Sickness Questionnaire for Virtual Reality Research. Proceedings of the 2020 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW); Atlanta, GA, USA, 22–26 March 2020; pp. 464-467.

58. Mezo, P.M.B. La pedagogía de la iniciación en la creación arquitectónica: La inmersión y la emersión imaginarias, el espacio matriz y la propuesta incipiente. Aproximaciones a Una Pedagogía Poiética. EGA Rev. Expr. Gráf. Arquit.; 2010; 15, pp. 138-147. [DOI: https://dx.doi.org/10.4995/ega.2010.1001]

59. Gómez-Tone, H.C.; Bustamante Escapa, J.; Bustamante Escapa, P.; Martin-Gutierrez, J. The Drawing and Perception of Architectural Spaces through Immersive Virtual Reality. Sustainability; 2021; 13, 6223. [DOI: https://dx.doi.org/10.3390/su13116223]

60. Zevi, B. Saber Ver la Arquitectura; Editorial Apóstrofe—Poseidón: Buenos Aires, Argentina, 2019; ISBN 84-455-0080-5