Qualitative Analyses of e-Learning Implementation and Hybrid Teaching during the COVID-19 Pandemic at Spanish Universities

Abstract

During the COVID-19 pandemic over the academic year 2020–2021, many universities and faculties had to deal with hybrid teaching by combining face-to-face and virtual teaching approaches. The main objective herein considered was to analyze the perceptions of students and teachers from Spanish universities regarding how e-learning has actually been adopted before, during, and after the COVID-19 lockdown. We also wished to know their opinions about the usefulness and applicability of the e-learning and hybrid teaching methodologies regarding their impacts on the teaching–learning process in the university context. A thematic analysis was performed using three discussion groups (two made up of four teachers each, and another comprising five teachers with university management posts). Seventy-nine open questionnaires completed by students were also analyzed. The participants were from eight different Spanish universities: six public and two private. The obtained results revealed a preference for face-to-face teaching over virtual teaching, and the advantages offered by closer interpersonal relationships were stressed. However, the participants also indicated the potential of the e-learning and hybrid teaching methodologies, which they believed complemented one another and reinforced learning personalization. Thus, a specific need for training in the e-learning methodology and hybrid teaching format was apparent.

1. Introduction

The situation that arose from the COVID-19 lockdown gave rise to a change in university teaching during the last trimester of the academic year 2019–2020, which involved using the e-learning methodology and a completely virtual format [1,2]. Synchronous or asynchronous virtual teaching became the main teaching format employed to continue the teaching–learning (T-L) process [3]. Despite acknowledging the e-learning-related benefits, having to forcedly make these changes led to certain difficulties for university teachers [4,5]. To name only a few, these issues included a lack of resources adapted to the new online format, a lack of training and experience in this online teaching model, and a lack of student participation. All this led to a generalized perception of improvisation and uncertainty, which in turn caused feelings of work overload, stress, and anxiety [6,7,8]. The students indicated that the necessary skills for the design and management of virtual teaching were lacking, and there were also technical difficulties, no real face-to-face relationships between teachers and classmates, longer computer work times, and in agreement with the teachers’ opinions perceived difficulties in disconnecting from lessons [9,10,11,12]. The teachers could use information and communication technologies (ICT) to adapt their activities and content to individual student performance and to also promote the students’ leading role in control their own learning process given their interests [13]. These personalization learning processes involve different pedagogical and psychological aspects of student motivation, engagement, and progress in learning. In this T-L environment, students could control and co-design their learning, engage in the planning and performance of activities, acquire significant learning experiences with a specific time and space to be analyzed both individually and collectively, use resources and opportunities from the web for different activities, and receive teacher support and monitoring [14]. Personalized learning and hybrid education environments can converge and reinforce one another to generate learning experiences that help students to build learning that is truly meaningful to them [15]. The academic and organizational measures subsequently taken for the academic year 2020–2021 due to the COVID-19 pandemic meant that most Spanish universities and faculties implemented a mixed synchronous face-to-face and online or virtual teaching model [16,17,18]. This model was called hybrid teaching. Hybrid teaching could not only be taken as a new and novel teaching method, but could also be understood as a teaching format rather than a method. Nevertheless, the way in which other studies have described and used them can be confusing, as could the terminological and conceptual ambiguities for being conceived and considered to be indistinct from blended learning (b-learning) [19]. Engel and Coll [15] defined hybrid learning as the combination of physical learning spaces in which the participants are present together in the same place with online or virtual learning spaces, or where the participants are located in different places and also in online learning spaces, with them sharing the same physical place.

A hybrid environment removes the barriers between face-to-face and virtual training, and permits not only learning personalization strategies to be applied, but also for their scope and efficiency to be considerably extended [15]. However, the possibilities that ICT offer to support personalization strategies depend on how the ICT are used to design and perform the T-L activities [20]. Therefore, when the intention is to hybridize the T-L process, we must bear in mind that ICT must serve to facilitate, transform, or empower the personalization strategies, and not to simply reproduce them. Therefore, when we propose hybridizing a T-L environment, we first have to wonder about the added value of ICT to facilitate and promote personalized learning [15]. We need to determine the adequacy of implementing the e-learning methodology per se to really know what possibilities are offered in a new university context. The quick and efficient occurrence of emergency remote teaching, which involved a temporary shift to an alternate virtual instructional mode due to crisis circumstances [21,22,23], allowed learning organizations to examine what was needed to more effectively work nurture future possibilities in online and remote teaching and learning contexts [24]. In this sense, the previous Spanish higher education-related literature has focused on quantitative and qualitative methodological approaches to analyze students’ learning experiences and their expectations of the changes made in education during the lockdown in the academic year 2019–2020 [25,26]. The research has also centered on students’ identification of the advantages and disadvantages of the adopted online education mode compared to face-to-face teaching, and on aspects such as equal opportunities, well-being and satisfaction, student empowerment, resources, assessments, and teaching quality [11,27,28,29,30,31]. Regarding hybrid teaching during the academic year 2020–2021, Lorenzo-Lledó et al. [32] and Sáiz-Manzanares et al. [33] found out students’ preferences for the university education mode and identified the difficulties and experiences associated with the received teaching. Another study analyzed the degree of student and teacher satisfaction levels with and their perceptions of establishing bubble groups and pairs, and of using audiovisual platforms to provide theoretical and practical university teaching [34]. However, very few have also considered other university community members’ perceptions. Tejedor et al. [12] analyzed students and teachers’ perceptions of online teaching during lockdown. Boté-Vericad [35] assessed teachers’ perceived barriers to online teaching and preparing education content during lockdown. Díez-Gutiérrez and Gajardo [36] considered families’ perceptions during and after lockdown. Espinosa-Navarro et al. [37] studied teachers’ perspectives regarding the inclusion and use of ICT before and after lockdown. Therefore, it is necessary to keep investigating all of the characteristic elements that contribute to e-learning as a teaching method, and not only before and during lockdown but also during the hybrid teaching period, by considering the perspectives and experiences of those playing leading roles such as students and teachers.

In light of this, and in accordance with Torrecillas [38], the experience lived during lockdown, when a shift came about from face-to-face teaching to online teaching and then later to hybrid teaching, is shaping a major change in university teaching and in the way to go about teaching; hence, there is a need to continue to look in depth at evaluating these online T-L experiences. Accordingly, the present study objectives are to analyze the perceptions of students and teachers from Spanish universities by means of a thematic analysis to see: (1) how e-learning has actually been adopted before, during, and after the COVID-19 lockdown; (2) the usefulness and applicability of the e-learning methodology and hybrid teaching, and their impacts on the T-L process in the university context. We intended to look closely at the psycho-educational implications associated with technical (resources used, planning and performing T-L activities, etc.), personal (personal situation, roles, etc.), and social (contact and support, accompaniment, etc.) aspects involved in implementing the e-learning methodology to hybridize the T-L environment. This is a matter of generating empirical evidence for the hybrid teaching model, and one that can allow us to learn its scope.

2. Materials and Methods

2.1. Participants

Given the sampling variables of interest, maximum variation sampling was conducted to purposefully pick different university students and teachers in different settings and conditions to maximize the diversity that was relevant for the study purpose. The sampling variables of interest were: GENDER (female/male); TYPE OF UNIVERSITY (public/private); TRAINING (early childhood education teacher training/primary education training); COURSE (second-course/third-course/fourth-course). No first-course students were considered because they would be unable to provide their view of what happened before confinement took place in March 2020. The UNIVERSITY MAGEMENT POST variable was also contemplated for teachers (teachers with and without university management posts).

In this study, the initial sample included 108 participating students (84 women and 24 men) whose average age was 22.98 ± 5.02 years; 29 of them were excluded for answering different open-ended questions in a reduced form (YES/NO). Thus, 79 university students (64 women and 15 men) formed the final sample, whose mean age was 23.01 ± 4.96 years. They were registered for the different courses of the teacher training degrees taught in eight distinct public and private faculties and university centers. According to their training, 39 were early childhood education students, 38 studied primary education, and two studied both degrees at the same time. Five second-course, 12 third-course, and 62 fourth-course students participated.

This study also included 13 teachers (8 women and 5 men), who also came from different public and private university faculties and centers. Their mean age was 43.15 ± 9.85 years and their mean teaching experience was 3.08 ± 6.38 years. Five of them also occupied university management posts, including as vice-deans or degree coordinators (Table 1).

There were no statistically significant differences in the distribution of the criteria of the sampling variables of interest of TRAINING in the students group, and GENDER and UNIVERSITY MAGEMENT POST in the teachers group (p > 0.05). Significant differences were observed in the variables GENDER (χ² (1) = 28.69, p > 0.05), TYPE OF UNIVERSITY (χ² (1) = 36.48, p > 0.05), and COURSE (χ² (2) = 69.84, p > 0.05) in the students group. There were no significant differences between groups for the variable AGE given the course (p > 0.05). In the teachers group, significant differences appeared for the variable TYPE OF UNIVERSITY (χ² (1) = 6.23, p < 0.05).

2.2. Ethics

This study was appraised favorably by the Clinical Research Ethics Committee of Aragón (Spain) in its Minutes N.: 10/2021. Participation in the study was voluntary and based on informed consent.

2.3. Data Collection Procedure

Data collection was carried out in May and October 2021. The data collection process was carried out in the same way and by the same two trained interviewers (researcher 2 and researcher 4). An invitation was sent to 12 different university faculties and centers from different geographical areas and communities in Spain by email. When they had confirmed their interest in participating, a first questionnaire with socio-demographic aspects was forwarded to the teachers. The data collection strategy was designed to achieve mixed triangulation (tools and participants) to understand the phenomenon through the different involved actors. Three discussion groups were set up: two with four teachers each (TE) and a third with five teachers who also held university management posts (TM). The discussion groups lasted 1 h and were carried out using the Google Meet application. A link to a Google Form that included informed consent was sent to the students. The time range estimated to complete was 10–20 min. The same wording was used in the questions for all participating groups (Supplementary Materials S1). The 14 questionnaire items were defined according to the previously consulted literature [4,39,40,41] and were grouped into two dimensions:

• Previous teaching with ICT and Internet support. This included, in turn, two different reflection times, with on the one hand an evaluation of employing ICT resources in the usual teaching that ended in March 2020, with questions such as: “Were ICT normally used to support teaching? If so, in what way? With what type of technological and digital resources?”. On the other hand, an evaluation of the way in which the university teaching was carried out during the last trimester of this course, which coincided with lockdown. For instance: “How was the teaching follow-up favored? We refer to aspects such as theoretical sessions, evaluation activities, planning time, tutoring, etc.”

• Evaluation of e-learning in hybrid teaching. This centered on the academic year 2020–2021 and was characterized by the face-to-face and online teaching combination. It included questions such as: “Do you think that students and centers have enough means (technological resources, devices, and infrastructures) to give hybrid teaching?” or “How do you perceive the role that students play in online teaching? Is it similar to the role they may play during face-to-face teaching?”

2.4. Analysis

Similarly to our previous studies, a thematic analysis was conducted [42,43]. A category system was deductively devised [44]. This allowed the scripts to be used for the data collection tools to be designed. The definitions of the different information units appear in the Supplementary Materials (SM2). One discussion group and 17 questionnaires were randomly selected to run an interobserver concordance analysis, which gave index k = 0.85. No emerging categories appeared in this phase. When the thematic analysis with the whole sample was finished by only one researcher, the value of the new intrapersonal concordance analysis was k = 0.91. The thematic analysis of the questionnaires from the students and the teacher discussion groups was performed using QSR Nvivo (version 12 Plus, https://www.qsrinternational.com/nvivo/home). All of the participants remained anonymous.

3. Results

Quantitative descriptive analyses were previously performed with the obtained data (see Table 2). As with our previous studies [42,43], this information was used as an indicator to direct the thematic analysis.

This first analysis indicated the difference between the two teacher focus groups and the students group. The teachers especially emphasized the reflections made on the first dimension (about 65% vs. 35% for the second one), while the students looked more at the indicators and categories related to the dimension about the post-confinement course (54%).

We now go on to present the thematic analysis by comparing the reflections made by the students, the teachers, and the teachers with university management posts. This came with some fragments of text that were obtained directly from the open questionnaires and the discussion groups. The code employed for this purpose included two elements: a descriptor (“US” for the university students and “TE” for the teachers; “TM” was used for those teachers who occupy university management posts), followed by a number (001–108).

3.1. Previous Teaching Experiences Supported by ICT and the Internet

3.1.1. Employing ICT in Usual Teaching

Only a few data points were collected from the different participants for Indicators 1.1.1. (Frequency of Use and Tasks) and 1.1.2. (Resources Used). Almost all of them were used to number the different employed resources and tools, and the most highlighted ones were PowerPoint presentations, videos, emails, and the Moodle platform. Some of the indicated stressed functions included accompaniment for master classes and being able to share documents.

This perception generated some controversy for the participants with Indicator 1.1.3. (their opinion about the suitability of their use and function). On the one hand, the US barely evaluated this perceived use. On the other hand, the TE and TM recognized that teaching supported by ICT and the Internet use might certainly seem trivial, and both need to be implemented more into the usual teaching. The following reflection is an example of this:

We’re still a long way from a real virtual campus. This is because things arise, let’s say, in a physical or traditional campus, where encounters take place, conversations are held, and students even meet there… I’m not sure. All kinds of academic matters take place there, regardless of them being curricular or extracurricular. But a virtual campus is still simply a repository of contents, it’s not a campus. We can call it a repository. Like, well, I don’t know, like Drive or Dropbox, etc. (TE010)

This concern came over very clearly in the reflections on the perceptions of specific training in this methodology and the involved technological tools (Indicator 1.1.4.). A lack of experience in using these technologies for learning and the fact that they favored implementing the e-learning methodology were indicated. The perceived convenience of using a certain (and limited) number of resources is summarized in TM001 as follows:

Or we’re likely to think that they are because we use certain tools in our comfort zone. But when they present you with new tools, then you say you only know a bit. And that’s what happened because, when we were confined in April, the educational resources service that we had, which manages the Moodle platform, began by doing online training for the teachers and students; from saving a session in Teams and sharing the link, to creating a lesson in Moodle.

Some opinions about the wide training offer range available in the university context were also collected and could explain this lack of specific training in using certain methodologies, as indicated by the words of TE012: “Information about courses arrives, which could be e-learning or other similar ones. As they’re similar, or perhaps you need to choose from among several, you select others that have more to do with your teaching or research contents”. Along these lines, the students detected a certain training insufficiency in their early training. This is shown by US012 when reflecting on the importance of its use in teaching: “In your career, they don’t teach you to use technologies. You must just know how to use them for your teaching future”. Many US also pointed out that more training plans must be included because educational resources and technologies constantly change and evolve. For example, US020 indicated the following about their training in the 4 study years: “I think it’s not enough because technologies and resources are constantly evolving. So it’s necessary to receive training for us all to access a wide range of options to use them”.

In accordance with the previous statement, the TE thought that the US are more skilled for learning how to use educational technologies, regardless of their novelties: “So even though they don’t have specific training, when you present them with technological applications or tools, I think they adapt them quite quickly by taking them in their stride” (TE013). From this reflection, the motivating nature generated in the US when using ICT can be highlighted. This idea is also shared by US029: “I think we must develop more digital competence because it’s important to confidently create active learning experiences that arouse motivation and participation in class”.

3.1.2. Adapting e-Learning during Confinement

The last trimester of the academic year 2019–2020 was marked by the general population being confined. As a result, all of the university TE had to change to an online methodology. The different resources were referenced in Indicator 1.2.1. The technical and technological resources coincided with those mentioned in the previous point (videos, email, Moodle), and a few sporadic examples of more interactive tools were added (e.g., Genially). To hold theoretical sessions (1.2.2. Teaching Action: Theoretical Sessions), a special mention was made to the different tools used to broadcast in streaming (Google Meet, Skype, Zoom).

The more active use of education platforms was particularly stressed. However, despite certain interest being shown in the enriching didactic material (e.g., with videoconferences or inviting external stakeholders to increase motivation), all of the participating groups generally coincided in that passive tools were mostly used; for instance, more documents to act as supplementary readings for theoretical contents or slide presentations with audio to reinforce explanations, but with no possibility of synchronous interaction.

The planned evaluation activities reflected how this way of teaching was applied for this part of the academic year. This indicator (1.2.3. Activities and Evaluation Tasks) was one of those that was more differently perceived by both the TE and US because the former indicated showing more interest and concern in their reflections. Briefly, the TE and TM assumed that the evaluation process was subject to a certain degree of improvisation, which came about by the social situation that the general population found itself in at the time. Therefore, the most widespread option was that of modifying the evaluation and marking criteria by attaching more importance to autonomous US work, as TM001 remembers: “Well exams… well, let’s call it the evaluation, it was complicated. It really had to be made flexible. And perhaps evaluating all the work not included in the exams that the students had done”.

In line with this, the US reported heterogeneous performance depending on the different TE and subjects. An increase in the number of expected evaluation tasks was generally perceived as a means to substitute theoretical sessions in those subjects in which they could not be performed. However, according to what US022 points out, the evaluation activities did not often accomplish the intended learning: “Others continued with their principles with theoretical sessions as if we were in face-to-face classes. Naturally many evaluation activities and sessions didn’t work as they were expected to”. The reasons for this, as verified later, were basically related to connection problems and to difficulties in immediately interacting with both the TE and classmates.

In Indicator 1.2.4. Teaching Follow-Up and Tutoring, the US came over as being quite understanding in a generalized way: “Following up teaching was generally suitable because the blame for any problems that cropped up could not be placed on anyone (connection problems, the Internet, etc.) and they were solved pretty quickly” (TE009). The vast majority pointed out that clearing up the doubts that arose in the various subjects by email or videoconference was generally achieved individually. In this point, the TE reflected on the effort that tutoring tasks involved for both the US and TE themselves. On the one hand, TE002 indicated that the workload was too heavy, especially for the US: “They were so overwhelmed, they spent 8 h looking at screens, and there were also lots of tasks and activities linked with the videoconference sessions that had taken place”. On the other hand, the TE also perceived a feeling of complete availability and no disconnection. TM001 summarizes this as follows: “And what’s more, there was no schedule. As you were shut indoors and couldn’t leave your home, you didn’t mind if someone asked to connect with you at 10 pm”.

For Indicator 1.2.5. Support Stakeholders, once again differences appeared for the importance that the different participating stakeholders attached, with a much bigger difference for the TM group than for the US group. The TE considered that teaching and contact (or lack of contact) by means of technologies were elements that negatively affected personal relationships. This came over in the reflection made by TE008: “I cite a colleague’s sentence that I think is great “what is affective is effective”. But nothing else because a computer isn’t affective”. By this they believed that the US were the most affected by the situation due to not only the social situation, but also to the uncertain academic situation, as this teacher states: “If it was difficult for us, who are more adult, we can’t imagine what it was like for the students who, during a subject, said that we’re going to do such and such, and then said that we don’t know anything at all during another subject” (TE003). This feeling was also shared by the US, who sometimes pointed out that they felt abandoned:

The teachers were overwhelmed because they didn’t know very well which guidelines to follow, or how to do things; but if there had been more information with the students in general, and not only with delegates, students would not have felt abandoned by the university, which was the way the majority of the students felt (TE001).

The generalized feeling for the TE was that they supported one another, and not only by the encouraging facet, but also in making the attempt to get teaching done in the best possible way by covering the aforementioned feasible lack of specific training: “I think that colleagues’ support here was influential, which we have taken advantage of and we’ve done whatever we’ve wanted” (TM011).

To end the first dimension of the study object, Indicator 1.2.6. Evaluating the Usefulness of this Methodology, offered some reflections that allowed us to view how both the UE and TE perceived the use of a purely online or virtual methodology. From the US’ point of view, and despite the questions about feeling they had too much work or sometimes feeling uncertainty, a lack of knowledge, or being abandoned in some subjects, they believed that the methodology was the right one because it at least allowed them to continue with the course. As pointed out by US007, they saw that the academic community showed an interest: “I didn’t notice more active methodologies being followed, but I did note the intention of feeling empathy for the students and teachers trying to adapt to the students”.

The TE’s interest lies not so much in the end result, but in the developed process. So the first set of problems involved the possible influence that the digital divide had on the students. Regarding this, TE010 indicated: “I wouldn’t take, let’s call it, the “one size fits all” approach, or accepting that everyone had equal access to the Internet and technological means, because it’s impossible”. This situation could affect not only the way that the teaching is followed because people could not participate in videoconference sessions, but also the evaluation of people having problems to freely access platforms to finish the tasks that they had been assigned.

Another main object of concern appeared when they had to continue with the online methodology. Regarding this, TM004 suggested that the marks that the US obtained in the last trimester of the academic year 2019–2020 were not very accurate: “As you mentioned, obviously their marks were marvelous; marvelous. But they didn’t match their actual acquired learning”. This reflection was shared by other participating teachers, who believed that the main point to improve in the following courses would be the evaluation. TE002 summed this up as follows: “The main problems in the evaluation appeared when the face-to-face evaluation systems were transferred. I mean, when the aim was to reduce the evaluation to a final test”. Once again, a real need for specific training in using this teaching methodology in the university context came across.

3.2. Evaluating e-Learning in Mixed Face-to-Face and Virtual (Hybrid) Teaching during the Post-Lockdown Course

The fact that the socio-healthcare situation caused by COVID-19 persisted meant that most universities and faculties set up a hybrid teaching system during the academic year 2020–2021. Given this situation, the various education stakeholders (institution, administration and services personnel, TE, US) had to make the effort to improve the way in which to give the virtual teaching taught in the previous course. They also had to reinforce those aspects that were perceived as strong points or beneficial aspects for the T-L process. This second dimension provided the results about the e-learning methodology evaluation as part of this hybrid teaching approach.

3.2.1. Resources Availability

Based on the experiences lived between March and May 2020, for Indicator 2.1.1. Personal Resource Availability, all participants stressed normalizing the use of mobile devices (especially tablets and laptops) to follow up from teaching. Thus, the majority of comments and reflections centered on providing equipment in classrooms to be able to synchronously hold sessions (Indicator 2.1.2. Adapting the Resources Available in Classrooms and the Center): “All the classrooms have a PC, a projector, a screen, and also some cameras” (US017). However, some reflections were made about the possible digital divide being one of the main limitations to successfully following the e-learning methodology, as US019 pointed out: “It also depends on the level of income at home. It’s not much use having a well-adapted classroom if a student can’t access from home”.

In particular, the TE positively evaluated this change made by the institution, but coinciding with what was referred to in the first dimension of the study object, it required specific training to properly implement this methodology: “The problem is that we’ve improved classrooms, audio, they can listen to us, we have videos. But these aren’t the tools to be implemented, we need learning tools” (TM011).

3.2.2. Giving Hybrid Teaching by Using the e-Learning Methodology

In this category, barely any references were found to the first two indicators, namely Indicators 2.2.1. The e-Learning Concept and its Application to Teaching and 2.2.2. Applying Other Methodologies. Nonetheless, the reflections coincided with the critical position about the way in which e-learning teaching was done. The TE believed that a synchronous teaching attempt was actually made with one part of the US as a face-to-face teaching format and with the other part of the US as an e-learning format at home. This could not be taken as an actual e-learning methodology because it was characterized by other matters. TE002 indicated the following:

A 100% e-learning model was not used because adaptations had to be made, and the possibilities that we all had in knowledge terms were applied. This was because they’re completely different areas. The methods, approaches and times are different.

On this matter, the US were also critical about the way in which the e-learning methodology was presented. They had more didactic resources, but perceived that they did not have the desired effect:

Telling me to watch videos in YouTube or providing me links to 20 articles doesn’t necessarily mean more activity in my learning because it might involve me watching a boring video of no interest at all or a documentary on Channel 2 that means practically nothing or nothing at all to me because I’m a (very) passive consumer of these things (US002).

In the same category that focused on applying e-learning, Indicator 2.2.3. Measures to Facilitate Teaching Follow-Up, included the different proposals that the TE put forward to carry out hybrid teaching, which particularly referred to the virtual part. Nevertheless, the obtained results showed that there were difficulties with virtually attending to the US. The main stressed aspect was that they perceived learning more in a face-to-face teaching situation for some reasons, such as immediacy or closeness to clear up different doubts and activities. US006 hinted at a lack of suitably performing synchronous hybrid teaching being a possible reason, which was also the generalized view of the US: “Teaching was certainly not similar because teachers were more involved with the students in face-to-face learning, and they’re not qualified to give a class simultaneously online and in person”.

This negative perception was felt by the TE, and especially the TM, who channeled the complaints and negative impressions provided by all the education stakeholders. This was summarized by TE009:

Even in the hybrid format when we had 1 week of face-to-face teaching and 1 week of online teaching, the students mentioned that they understood things much better with face-to-face teaching, and the difference was huge. That’s what the students said. They were also grateful because they said things like “This week I understood everything because I had face-to-face teaching, which doesn’t happen when I’m at home”.

TE008 voiced self-critical concern by doubting the actual scope of the performed work, and despite all the efforts made: “Our participation made us alert all the time so that the students could actively participate. Sometimes you achieved this, but not every time”.

Regarding the role played by those leading the T-L process was perceived (Indicator 2.2.4. Playing the Main Roles: Students and Teachers), the US group gave more references. Nonetheless, the perceptions came in two lines of thought. First of all, the US thought that this type of teaching, especially virtual teaching, involved a heavier workload compared to traditional teaching:

During the online teaching, we worked much more compared to traditional teaching because teachers thought we had more time to perform activities, but we had the same time in fact. This led to a heavier workload (US018).

The arguments about this referred to making the effort to synchronously follow theoretical sessions, and sometimes practical ones, when technical problems (audio, image, connection) sometimes involved concentrating more. At times the evaluation tasks and activities to be performed during the same session were considered, and within a limited timeframe, apart from the supplementary activities set for outside the teaching schedule. All this led to a feeling of no disconnection, plus a second source of concern, leading to perceptions that their doubts had not been cleared up, which involved a much more active role: “When following up teaching, it’s true that they had a lot of work and things to do, but sometimes we felt as though we had no-one to ask” (US022).

Moreover, the TE believed that their role in the virtual format had also changed, and their facet as stakeholders who facilitate information was promoted, but they had to very carefully consider the ways in which to perform their actions and to make their comments. As one main reason for this, TE009 indicated the possible fear of being misinterpreted by the US in the virtual format because this lacked close or direct contact:

You feel you can make a joke or comment on a personal matter, but I didn’t feel so happy about doing these things with the online format. And I even prioritized being more communicative with information in the online format.

3.2.3. Evaluating e-Learning

The advantages and limitations associated with implementing the e-learning methodology are the first two indicators that define this category. Of the three groups of participants, the US presented more codings for both. For Indicator 2.3.1. Advantages and Facilitating Elements, the possibility of adapting to each student’s learning pace (US039) stood out, which allowed them to not only consult the available resources with no set schedule, but they could also access them as many times as they needed to. In relation to this aspect, the TE pointed out the huge advance for all of the involved stakeholders that learning the digital competence represented (TM009), because this enabled didactical resources to be generated and the different proposed evaluation activities to be performed.

Our study also highlights certain disadvantages and limiting elements (Indicator 2.3.2.) due to technical aspects (connection, quality of images or sound, etc.), how the didactic resources worked, or the way the teaching methodology was implemented. However, they all agreed on the US’ lack of attention and engagement during synchronous teaching. US007 provided an example of this: “I think that the biggest disadvantage was the dispersion that using technologies and the Internet involved because it made realizations and giving a value to the contents difficult”. This reflection was very recurrently made by the participants in the TE discussion groups: “This was also one of the peculiarities that we noticed. This is because you don’t really know if a student’s still there, if students have merely connected, and goodness knows what’s really going on” (TE009).

Regarding Indicator 2.3.3. Valuing Perceived Learning and Academic Performance, it is worth pointing out that the three types of participants coincided in the percentages of codings. Moreover, their e-learning perceptions were also similar if we bear in mind that their academic performance during academic year 2020–2021 had been quite badly affected. Doubts arose about the values of the obtained marks. Along these lines, TE002 considered that many US “noted that their efforts had not been rewarded because, in the end, they had obtained the same high marks that everyone else did”. This revealed the flexibility applied to the evaluation process.

Likewise, understanding the academic performance concept as a more complex process, and contemplating not only marks, but also acquired learning, evidently revealed the negative effect that e-learning could have had on more practical subjects, such as “workshops, laboratories, motricity rooms, music rooms, art education rooms, etc.” (TM004). The US recognized their potential (e.g., US103 indicated that “it might be very beneficial to properly use it; if not, taking a sedentary attitude might be easier, but means that none of the parts progresses”), and advocated a supplementary use as part of face-to-face teaching: “For me, ICT are necessary, but this experience has shown me that they are necessary in face-to-face teaching. There is still a long way to go to achieve good effective online teaching” (US104).

Perceiving face-to-face as being more useful than virtual teaching also appeared in Indicator 2.3.4. Psychological Perceptions (emotional and cognitive). Here, the TE pointed out that a lack of connection among US was one of the possible consequences of interaction on screens. TE009 considered that “No group cohesion is produced. Having students who you can’t see, who don’t know one another well, involves certain problems: they don’t integrate, not feeling part of the group, etc.” The US perceived this similarly and criticized the depersonalization in the T-L process, particularly in those university degrees that center on teacher training:

I think that contact has very negative results as I believe that the teacher-student link or relationship is lost because, in the end, “it’s all on the web”. I also think that its use makes us more dependent, and even the link among classmates can be lost (US009).

To supplement this, the TE also referred to feeling that they suffered from work overload when preparing and following-up sessions. Sometimes this did not have the desired effect because they did not receive the US’ responses. In the words of TE003:

My emotional management, so to speak, also came into play. I was giving everything, I had prepared a class wholeheartedly with much enthusiasm and took a really good attitude. Sadly, none of this came over due to different circumstances.

They also missed being personally in contact with the US except purely for the teaching activity: “When a class ends, you talk with them about their concerns. If they like your subject, they ask for more. I miss this contact after my classes” (TM001).

3.2.4. Applying and Proposing Improvements

In the last proposed category, the thematic analysis similarly reported a few codings for the three groups of participants about the positive aspects detected when implementing the e-learning methodology (Indicator 2.4.1. Positive Aspects, Successes, or Strong Points), and when perceiving the encountered difficulties or limitations (Indicator 2.4.2. Negative Aspects, Mistakes, or Weak Points). The expressed ideas coincided with what was expressed in a previous category for evaluating e-learning, insofar as noting the spatiotemporal break as something positive, but which was also subject to technical limitations, particularly the methodological application.

This line of thought was that which highlighted the different improvement proposals found in the last indicator (Indicator 2.4.3. Improvement Proposals), where the TM towered above the other participants. They coincided in pointing out that the e-learning methodology offered excellent benefits for teaching, but had to be employed as a supplement by eminently advocating face-to-face teaching because of its many benefits:

I expect we’ll obtain what is good from what is virtual; we’ll be trained in ICT, we’ll use it when it’s necessary, we’ll handle technologies well, but all this doesn’t remain as a hybrid format, and certainly not as a 100% virtual format (TM001).

The TE once again suggested improving the possibility of obtaining more specific training to suitably implement this methodology, and they also indicated its educational potential. TE010 summarized a change in the training offer that the university institution should take on: “We don’t need more computer training courses. We need courses that are more to do with the digital competence that combine didactics, pedagogy, and ICT. And in all senses, in specific didactics, etc.”.

The US also perceived this need, and considered that the e-learning methodology should encourage more participation, but should never involve the aforementioned feeling of work overload. This comes across in these reflections: “For online teaching, classes with more interaction are more practical because those students who go online feel more integrated into the classroom by participating from their respective homes” (US006). “It’s still very hard sitting in front of a PC for hours. We should continue to investigate motivating and enriching proposals and alternatives for students” (US013).

4. Discussion

Knowing the perspective of both the US and TE about the e-learning methodology and hybrid teaching during the COVID-19 pandemic allowed us to emphasize the real possibilities of this type of methodological approach in university training. Hybrid teaching is a teaching format that can become a leading higher education feature if it is understood as a supplement of face-to-face training. However, a lack of skills and orientation to set it up can entail problems for T-L processes [19]. Indeed, as the above-cited authors also consider, it is important to support the TE by offering them hybrid teaching training.

Until lockdown in March 2020, the participants pointed out that using ICT was not customary in normal teaching, and this situation could be related to a lack of specific training. This is in agreement with other previous studies [45,46,47]. The TEs’ age and experience were also established as determining variables that favored a lack of ICT use. It would indeed seem that the forced shift to online teaching that occurred during the months that confinement lasted was a less complex matter for the younger TE [37]. The participants also stressed that the US often lacked such knowledge or specific training, or were not used to handling some platforms and tools in the university context. Hence, the possibility of previous training to allow efficient e-learning methodology use was also requested [48].

The main advantage associated with the various groups of participants with applying the e-learning methodology was the break from the spatial dimension because it could lead to classes being followed-up, even when face-to-face attendance was not feasible. This economic investment in incorporating technological equipment into the different classes was very positively evaluated because different sessions could be broadcast live. Sometimes the prospect of saving classes was offered to respond to possible incompatible schedule problems [49].

The fact that the participants’ evaluations focused on the confinement period allowed a series of limitations related to using the e-learning methodology in the university context to be detected. They included the digital divide, which contemplated not only a lack of devices, but also difficulties with Internet access [16,50]. Moreover, the TE and TM groups particularly stressed the need to have more suitable evaluation means. This was a matter of having evaluation procedures to ensure real or adapted measurements of the US’ achieved learning as an ethical procedure, and one that would not allow the US to partake in deceitful or dishonest conduct [51]. Some difficulties related to a lack of contact or no direct US–TE interaction were also pointed out. All our participants indicated that teaching face-to-face, contact, and social relationships among the different stakeholders (US–US and US–TE) provided a series of benefits that were lost in virtual teaching, particularly when it was asynchronously carried out. Thus, e-learning was perceived as another alternative for teaching, but certainly not as the only model [52,53]. In fact, some authors talked about the possibility of adopting other methodologies such as blended learning [54].

Another result obtained in this study was a reflection on the difficulties that some participants had with their emotional management during confinement. The US often mentioned feeling abandoned by the academic institution. This view was also shared by the TE toward their colleagues. The TE and TM indicated a situation of feeling uncertainly in all areas as their main source of stress. Confinement or having to stay at home, which led to situations of social isolation, and sitting in front of a screen for so long were determining factors for mental health and affected everyone differently [55,56,57,58]. Şahin and Şahin [48] suggest taking into account the possible effects of using technologies on mental health when implementing the e-learning methodology to guarantee their success in the training context.

The main difficulty reported by both the US and TE was a lack of the participating group’s implication in the virtual format. This lack of participation has often been related to the way in which virtual teaching has been given, which does not frequently differ much from traditional face-to-face teaching. In our study, and in line with what the different authors [59,60,61] proposed, the US reported having a generalized feeling of finding it hard to pay attention to and follow-up on screens. They also perceived that they spent too long using screens to implement this methodology, although it could be combined with other types of methodologies such as the flipped classroom method [17,58,62]. Authors such as Malik and Javed [57] and Boca [62] also referred to the perception of the US doing more work in this type of teaching, a point that also came across in the present study. The TE also indicated perceiving work overload and their knowledge-building process not suitably reaching their US. All participants reported that the training process was negatively affected during confinement. This was particularly true for practical subjects, such as workshops, laboratories, and practical sessions. Indeed, previous evidence has demonstrated that the US’ satisfaction with this kind of methodology during confinement could have been influenced by the subject type [33]. Flores et al. [8] highlighted the effect that the methodology had on the academic performance in relation to the practical subjects. Our results showed that this kind of methodology could be better for the theoretical aspects of subjects. The learning inherent to professional action, as achieved by means of interpersonal relationships, could be more negatively affected.

The literature reports conflicting evaluations for academic outcomes. On the one hand, Rossettini et al. [63] pointed out the high degree of the surveyed people’s satisfaction with using online learning and reported similar results to those for 100% face-to-face teaching. According to Rizun et al. [64], this positive evaluation can be related to using suitable and intuitive materials, means, and resources. On the other hand, Sáiz-Manzanares et al. [33] showed that the US felt more satisfied with the teaching process when the e-learning methodology was applied during confinement. However, they obtained worse results compared to the next course when e-learning was partially applied. According to these authors, these differences were due to collaborative work, which could have been conducted face-to-face at that time of the pandemic. Previous research studies, such as Chen et al. [65], indicated improved academic outcomes when combining asynchronous learning activities over the Internet and conventional learning activities in the classroom. Nevertheless, their studies referred to a teaching format that tends to be understood more as b-learning than hybrid learning per se. Hybrid teaching in our study was not perceived as a teaching mode related to improvements in academic performance. Indeed, the results obtained in practical subjects could be conditioned.

Performing hybrid teaching allowed the e-learning to be differently conceived, and its virtues were especially stressed in synchronous face-to-face teaching by it becoming an ideal supplement to encourage student motivation, influencing the interactive component [52]. Khan [66] added the possibility of creating learning communities by stressing the need to more actively engage students in their own training process. Nevertheless, some studies such as those by Lin et al. [56] indicated some sources of student dissatisfaction that lead to isolation, emotional instability, and disconnection. These problems were detected in our study, particularly by the TE group. A lack of interpersonal contact, as indicated by Hauer [2], invites reflecting on the suitability of adopting this teaching methodology and format according to the type of subject in question, because e-learning might end up becoming a demotivating element if it does not entail previous habits in its use and handling, particularly in relation to the schedule flexibility and learning pace. In our study, the TE stressed the US’ autonomy and maturity as essential characteristics to optimize the e-learning setup.

The findings that appeared while reflecting on the way in which the hybrid teaching was delivered during the academic year 2020–2021 revealed the efforts made by education institutions to improve the installations as a result of the bad experiences lived during confinement (the last trimester of the academic year 2019–2020). In general terms, the availability of more personal devices and improved Internet connections at both academic centers and home was reported. All this favored using the e-learning methodology and obtaining a more positive evaluation in our study than that achieved in previous months. Nonetheless, like previous studies [4,33], a marked feeling of spending time to implement the learning process was once again highlighted, which was often due to the need to learn how the resources and tools worked.

This study is not without its limitations. There may have been some problems with sampling bias due to distribution differences in variables such as gender and the type of university in the student group, as well as the type of university in the teacher group, which could have affected the feasibility of the conclusions. However, in the Spanish university context, there is a higher percentage of women than men in this type of training, and more public universities than private universities [67]. We could consider that our sample represents the analyzed reality. Otherwise, the adequacy of a sample and its size are related to the ability of the data to provide a rich approach to the studied phenomenon and the study objectives, and cannot be detached from the study characteristics that influence the saturation [68]. As in previous studies [69,70], by using a form of code frequency counts to assess the saturation, which involved counting codes in a set of transcripts until no more codes were identified, and by randomizing the order of data to assess the influence of the sequential bias on the saturation, we determined that we needed eight open-ended questionnaires and two focus groups to complete the themes. Thus, we configured a sample that responded to the methodological demands related to rigor. Another limitation was in forming our study sample, which centered on the TE and US of PETT and ECETT. Future research lines could extend this study to other samples, particularly regarding university degrees in other knowledge macro-areas, so as to establish similarities in the obtained results and any possible differences and reasons that could appear. The perceptions of students with special education needs in an e-learning methodology being implemented could also be evaluated to know whether both the employed technological tools and teaching strategies can adapt to their characteristics and learning paces. Likewise, this study did not bear in mind the administration and services staff members. It might be interesting for future studies to include their perceptions, especially those of the people who are more directly linked with the audiovisual technicians’ posts, who help to maintain classroom equipment. Finally, and in line with the demand for specific training for both the US and TE, it is important to bear in mind that this training could address the way that the e-learning methodology could be effectively incorporated into classrooms and the specific uses of certain technological tools. In this way, the objective of related studies should be to inquire about guidelines to implement the methodology and resources to ensure a positive effect on the academic performance, understood as not only the obtained marks, but also as the learning and competences that the US acquire.

5. Conclusions

As the related research is lacking, our study attempted to extend the evidence in the Spanish higher education system for all characteristics elements that comprise e-learning as a teaching method, not only before and during lockdown, but also during the hybrid teaching period, by simultaneously considering the perceptions of those playing leading roles such as students and teachers. We have provided empirical evidence from a systematic research process that supports the daily experiences at university, by obtaining useful and certainly comparable conclusions. The present study coincides with previous ones that point out the need for specific training in using this methodology and hybrid teaching [63,71,72]. It is necessary to consider technical, personal, and social aspects such as the hybrid environment, through e-learning implementations that must adapt to each context and reality, and to present appealing formulae for the US. In turn, the suitable TE support and communication between students and teachers should be further developed to avoid negative impacts on learning and academic performance. Moreover, the selection of the employed tools and resources, and the materials generated with them, have to be perceived by the US as useful, and must promote motivation [54] and improve the participation of those students who remain at home during sessions [9]. It is necessary to increase both the participation and engagement in the tasks and activities that can be performed online, regardless of them being synchronous or asynchronous [1,73]. Moreover, and as both the TE and TM have emphasized, the employed tools and resources, particularly those used to perform evaluation tests, must ensure the academic honesty, privacy, and confidentiality [74]. Finally, it is mandatory to suitably select and plan the times required to implement e-learning [72] because the balance between the face-to-face and virtual teaching modes may not be the same for different subjects and degrees.

Hence, the participants agreed in prioritizing face-to-face teaching over virtual teaching by stressing the inherent benefits of interpersonal contact. They also understood that the e-learning methodology and the hybrid teaching format can offer complementary ways of dealing with subjects in the university context. After our analysis, we concluded that hybrid teaching can be used in higher education as a synchronous and simultaneous form of teaching in the classroom and with students online if it is understood as an option that helps to facilitate learning personalization, depending on the academic and training circumstances, demands, and needs [15,19].

Author Contributions

Conceptualization, C.P.-L. and J.C.B.; methodology, C.P.-L., M.L.-G., M.S.-B. and J.C.B.; formal analysis, C.P.-L. and M.S.-B.; investigation, J.C.B., M.S.-B. and M.L.-G.; resources, C.P.-L. and M.L.-G.; writing—original draft preparation, C.P.-L. and M.S.-B.; writing—review and editing, J.C.B. and M.L.-G.; supervision, J.C.B.; project administration, C.P.-L. and J.C.B.; funding acquisition, C.P.-L. All authors have read and agreed to the published version of the manuscript.

Institutional Review Board Statement

The study was conducted according to the guidelines of the Declaration of Helsinki, and approved by the Clinical Research Ethics Committee of Aragón (Spain) (protocol code PI20-221, 19 May 2021).

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Acknowledgments

The authors wish to thank all the representative students and teachers for their involvement in this study.

Conflicts of Interest

The authors declare no conflict of interest.

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Tables

Table 1

Participants.

Participants	Public University		Private University		Total
	F	M	F	M
ECETT Students	27	0	10	2	39
PETT students	25	13	0	0	38
ECETT + PETT students	1	0	1	0	2
Teachers	3	4	0	1	8
Teachers with university management posts	4	0	1	0	5

NB: F = Female; M = Male; ECETT = Early Childhood Education Teacher Training; PETT = Primary Education Teacher Training.

Table 2

Descriptive data analysis.

Category System	Code TM		Code TE		Code US
	N	%	N	%	N	%
1. Previous teaching experiences supported by ICT and the Internet	210	64.81	258	65.82	926	45.91
1.1. Employing ICT in usual teaching	57	17.59	95	24.23	303	15.02
1.1.1. Frequency of use and tasks	13	4.01	16	4.08	45	2.23
1.1.2. Resources used	11	3.40	15	3.83	121	6.00
1.1.3. Opinion about the suitability of their use and function	11	3.40	26	6.63	24	1.19
1.1.4. Perception of specific training in this methodology and the involved technological tools	22	6.79	38	9.69	113	5.60
1.2. Adapting e-learning during confinement	153	47.22	163	41.58	623	30.89
1.2.1. Technical and technological resources	15	4.63	15	3.83	79	3.92
1.2.2. Teaching action: Theoretical sessions	11	3.40	4	1.02	67	3.32
1.2.3. Activities and evaluation tasks	24	7.41	37	9.44	47	2.33
1.2.4. Teaching follow-up and tutoring	42	12.96	36	9.18	236	11.70
1.2.5. Support agents	35	10.80	26	6.63	85	4.21
1.2.6. Evaluating the usefulness of this methodology	26	8.02	45	11.48	109	5.40
2. Evaluating e-learning in mixed face-to-face and virtual (hybrid) teaching during the post-lockdown course	114	35.19	134	34.18	1091	54.09
2.1. Resources availability	7	2.16	6	1.53	97	4.81
2.1.1. Personal resources availability	1	0.31	4	1.02	36	1.78
2.1.2. Adapting the resources available in classrooms and the center	6	1.85	2	0.51	61	3.02
2.2. Giving hybrid teaching by using the e-learning methodology	32	9.88	38	9.69	331	16.41
2.2.1. The e-learning concept and its application to teaching	7	2.16	13	3.32	74	3.67
2.2.2. Applying other methodologies	2	0.62	6	1.53	4	0.20
2.2.3. Measures to facilitate teaching follow-up	16	4.94	10	2.55	111	5.50
2.2.4. Playing the main roles: students and teachers	7	2.16	9	2.30	142	7.04
2.3. Evaluating e-learning	42	12.96	67	17.09	514	25.48
2.3.1. Advantages and facilitating elements	8	2.47	10	2.55	109	5.40
2.3.2. Disadvantages and limiting elements	4	1.23	10	2.55	129	6.40
2.3.3. Valuing perceived learning and academic performance	24	7.41	29	7.40	148	7.34
2.3.4. Psychological perception (emotional and cognitive)	6	1.85	18	4.59	128	6.35
2.4. Applying and proposing improvements	33	10.19	23	5.87	149	7.39
2.4.1. Positive aspects, successes, or strong points	4	1.23	7	1.79	29	1.44
2.4.2. Negative aspects, mistakes, or weak points	8	2.47	7	1.79	49	2.43
2.4.3. Improvement proposals	21	6.48	9	2.30	71	3.52
Total	324	100	392	100	2017	100

BM: TM = teachers with university management posts; TE = teachers; US = university students.

Supplementary Materials

The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/su141912003/s1, Supplementary Material S1: Open-ended questionnaire for students and the teachers focus groups; Supplementary Material S2: Category System definitions; Supplementary Material S3: Consolidated criteria for reporting qualitative studies (COREQ) 32-item checklist.

References

1. Cicha, K.; Rizun, M.; Rutecka, P.; Strzelecki, A. COVID-19 and Higher Education: First-Year Students’ Expectations toward Distance Learning. Sustainability; 2021; 13, 1889. [DOI: https://dx.doi.org/10.3390/su13041889]

2. Hauer, I. Distance Learning, a Case Study in Higher Education in Timisoara, Romania. Ann. Dunarea Jos Univ. Galati. Fascicle I Econ. Appl. Inform.; 2020; 26, pp. 31-37. [DOI: https://dx.doi.org/10.35219/eai15840409133]

3. Moorhouse, B.L.; Wong, K.M. Blending asynchronous and synchronous digital technologies and instructional approaches to facilitate remote learning. J. Comput. Educ.; 2022; 9, pp. 51-70. [DOI: https://dx.doi.org/10.1007/s40692-021-00195-8]

4. Ejdys, J.; Kozłowska, J. Acceptance of e-learning at University level during the COVID-19 pandemic situation-teachers’ and students’ perspective. Pol. J. Manag. Stud.; 2021; 23, pp. 106-129. [DOI: https://dx.doi.org/10.17512/pjms.2021.23.2.07]

5. Iqbal, S.A.; Ashiq, M.; Rehman, S.U.; Rashid, S.; Tayyab, N. Students’ perceptions and experiences of online education in Pakistani Universities and Higher Education Institutes during COVID-19. Educ. Sci.; 2022; 12, 166. [DOI: https://dx.doi.org/10.3390/educsci12030166]

6. Dorfsman, M.; Horenczyk, G. Pedagogical change in university teaching undergoing COVID-19. Rev. Educ. A Distancia; 2021; 21, 27. [DOI: https://dx.doi.org/10.6018/RED.475151]

7. Fernández-Jiménez, M.A.; Mena Rodríguez, E.; Jiménez-Perona, M.I. Transformación de la universidad pública como consecuencia del COVID’19. Perspectiva del profesorado a través del método delphi. Rev. Complut. Educ.; 2021; 32, pp. 439-449. [DOI: https://dx.doi.org/10.5209/rced.70477]

8. Flores, E.; Maurera, F.; Hadweh, M.; Alonso, S.; Silva, Á.; Peña, S.; Castillo, F.; González, P.; Pauvif, F.; Bahamondes, V. et al. Nivel de satisfacción de las clases online por parte de los estudiantes de Educación Física de Chile en tiempos de pandemia. Retos; 2021; 41, pp. 123-130. [DOI: https://dx.doi.org/10.47197/retos.v0i41.82907]

9. Almaiah, M.A.; Al-Lozi, E.M.; Al-Khasawneh, A.; Shishakly, R.; Nachouki, M. Factors affecting students’ acceptance of mobile learning application in Higher Education during COVID-19 using Ann-Sem Modelling technique. Electronics; 2021; 10, 3121. [DOI: https://dx.doi.org/10.3390/electronics10243121]

10. Ferrer-Serrano, M.; Lozano-Blasco, R.; Latorre-Martínez, M.P.; Quílez-Robres, A. E-learning durante la pandemia COVID-19. Tecnologías Educativas y Estrategias Didácticas; Sánchez, E.; Colomo, E.; Ruiz, J.; Sánchez, J.; Coords,. Servicio de Publicaciones de la Universidad: Málaga, Spain, 2020; pp. 36-45.

11. Pérez-López, E.; Vázquez-Atochero, A.; Cambero Rivero, S. Distance education in COVID-19’s period: An analysis from the perspective of university students. RIED-Rev. Iberoam. Educ. A Distancia; 2021; 24, pp. 331-350. [DOI: https://dx.doi.org/10.5944/ried.24.1.27855]

12. Tejedor, S.; Cervi, L.; Tusa, F.; Parola, A. Educación en tiempos de pandemia: Reflexiones de alumnos y profesores sobre la enseñanza virtual universitaria en España, Italia y Ecuador. Rev. Lat.; 2020; 78, pp. 1-21. [DOI: https://dx.doi.org/10.4185/RLCS-2020-1466]

13. Cuban, L. The Flight of a Butterfly or the Path of a Bullet? Using Technology to Transform Teaching and Learning; Harvard Education Press: Harvard, MA, USA, 2018.

14. Bray, B.; McClaskey, K. Make Learning Personal: The What, Who, WOW, Where, and Why; Corwin Press: Thousand Oaks, CA, USA, 2014.

15. Engel, A.; Coll, C. Entornos híbridos de enseñanza y aprendizaje para promover la personalización del aprendizaje. RIED Rev. Iberoam. Educ. A Distancia; 2022; 25, pp. 225-242. [DOI: https://dx.doi.org/10.5944/RIED.25.1.31489]

16. Alkhalil, S.M.; Manasrah, A.A.; Dabbour, L.M.; Bashayreh, E.A.; Abdelhafez, E.A.; Rababa, E.G. COVID-19 pandemic and the e-learning in Higher Institutions of Education: Faculty of Engineering and Technology at Al-Zaytoonah University of Jordan as a case study. J. Hum. Behav. Soc. Environ.; 2021; 31, pp. 464-475. [DOI: https://dx.doi.org/10.1080/10911359.2020.1829243]

17. Ananga, P. Pedagogical considerations of e-learning in education for development in the face of COVID-19. Int. J. Technol. Educ. Sci.; 2020; 4, pp. 310-321. [DOI: https://dx.doi.org/10.46328/ijtes.v4i4.123]

18. Muñoz, I.; Roque, S. Nuevos modelos de docencia, desde la Declaración de Bolonia a la era de aa COVID. Miscelánea Comillas. Rev. Cienc. Hum. Y Soc.; 2021; 79, pp. 225-253. [DOI: https://dx.doi.org/10.14422/MIS.V79.I154.Y2021.007]

19. Ulla, M.B.; Perales, W.F. Hybrid teaching: Conceptualization through practice for the post COVID19 pandemic education. Front. Educ.; 2022; 7, 924594. [DOI: https://dx.doi.org/10.3389/feduc.2022.924594]

20. Coll, C.; Mauri, T.; Onrubia, J. La utilización de las Tecnologías de la Información y la Comunicación en la educación: Del diseño tecno-pedagógico a las prácticas de uso. Psicología de la Educación Virtual. Enseñar y Aprender con las Tecnologías de la Información y la Comunicación; Coll, C.; Monereo, C. Morata: Madrid, Spain, 2008; pp. 74-103.

21. Hodges, C.; Moore, S.; Lockee, B.; Trust, T.; Bond, A. The Difference between Emergency Remote Teaching and Online Learning. Available online: https://er.educause.edu/articles/2020/3/the-difference-between-emergency-remote-teaching-and-online-learning (accessed on 9 August 2022).

22. Lucas-Barcia, E.; Alonso-de-Mena, E.; Martínez-Álvarez, I.; García-Barrera, A. Formación del profesorado universitario sobre evaluación educativa en enseñanza a distancia. Rev. Tecnol. Cienc. Y Educ.; 2022; 22, pp. 39-66. [DOI: https://dx.doi.org/10.51302/tce.2022.640]

23. Stewart, W.H. A global crash-course in teaching and learning online: A thematic review of empirical Emergency Remote Teaching (ERT) studies in Higher Education during Year 1 of COVID-19. Open Prax.; 2021; 13, pp. 89-102. [DOI: https://dx.doi.org/10.5944/openpraxis.13.1.1177]

24. Erlam, G.D.; Garrett, N.; Gasteiger, N.; Lau, K.; Hoare, K.; Agarwal, S.; Haxell, A. What really matters: Experiences of emergency remote teaching in University teaching and learning during the COVID-19 Pandemic. Front. Educ.; 2021; 6, 639842. [DOI: https://dx.doi.org/10.3389/feduc.2021.639842]

25. Baltà-Salvador, R.; Olmedo-Torre, N.; Peña, M.; Renta-Davids, A.I. Academic and emotional effects of online learning during the COVID-19 pandemic on Engineering students. Educ. Inf. Technol.; 2021; 26, pp. 7407-7434. [DOI: https://dx.doi.org/10.1007/s10639-021-10593-1]

26. Ramos-Morcillo, A.J.; Leal-Costa, C.; Moral-García, J.E.; Ruzafa-Martínez, M. Experiences of Nursing students during the abrupt change from face-to-face to e-learning education during the first month of confinement due to COVID-19 in Spain. Int. J. Environ. Res. Public Health; 2020; 17, 5519. [DOI: https://dx.doi.org/10.3390/ijerph17155519]

27. Costado Dios, M.T.; Piñero Charlo, J.C. Face-to-face vs. e-learning models in the COVID-19 era: Survey research in a Spanish University. Educ. Sci.; 2021; 11, 293. [DOI: https://dx.doi.org/10.3390/educsci11060293]

28. Díez-Gutiérrez, E.; Gajardo Espinoza, K. Online Assessment in Higher Education during Spanish confinement by COVID-19: The view of students. J. Univ. Teach. Learn. Pract.; 2021; 18, 13. [DOI: https://dx.doi.org/10.53761/1.18.5.13]

29. Villa, F.G.; Urchaga Litago, J.D.; Sánchez-Fernández, A. Percepciones y expectativas en el alumnado universitario a partir de aa adaptación a la enseñanza no presencial motivada por la pandemia de COVID-19. Rev. Lat. Comun. Soc.; 2020; 78, pp. 65-85. [DOI: https://dx.doi.org/10.4185/RLCS-2020-1470]

30. Hervás-Gómez, C.; Díaz-Noguera, M.D.; De la Calle-Cabrera, A.M.; Guijarro-Cordobés, O. Perceptions of university students towards digital transformation during the pandemic. Educ. Sci.; 2021; 11, 738. [DOI: https://dx.doi.org/10.3390/educsci11110738]

31. Revilla-Cuesta, V.; Skaf, M.; Varona, J.M.; Ortega-López, V. The outbreak of the COVID-19 pandemic and its social impact on Education: Were Engineering Teachers ready to teach online?. Int. J. Environ. Res. Public Health; 2021; 18, 2127. [DOI: https://dx.doi.org/10.3390/ijerph18042127]

32. Lorenzo-Lledó, A.; Lledó, A.; Gilabert-Cerdá, A.; Lorenzo, G. The pedagogical model of hybrid teaching: Difficulties of university students in the context of COVID-19. Eur. J. Investig. Health Psychol. Educ.; 2021; 11, pp. 1320-1332. [DOI: https://dx.doi.org/10.3390/ejihpe11040096]

33. Sáiz-Manzanares, M.C.; Casanova, J.R.; Lencastre, J.A.; Almeida, L.; Martín-Antón, L.J. Satisfacción de los estudiantes con la docencia online en tiempos de COVID-19. Comunicar; 2022; 30, pp. 35-45. [DOI: https://dx.doi.org/10.3916/C70-2022-03]

34. Rodríguez-Nogueira, Ó.; Leirós-Rodríguez, R.; Quiroga-Sánchez, E.; Álvarez-Álvarez, J.M.; Álvarez-Barrio, L. Perceptions and degree of satisfaction with the Health Sciences University educational community regarding the measures adopted for the prevention of COVID-19 in the academic year 2020/2021. Eur. J. Investig. Health Psychol. Educ.; 2021; 11, pp. 1022-1029. [DOI: https://dx.doi.org/10.3390/ejihpe11030076]

35. Boté-Vericad, J.J. Perceived barriers for distance teaching in Higher Education during the COVID-19 crisis: “I never did a video before". Educ. Inf.; 2021; 37, pp. 377-397. [DOI: https://dx.doi.org/10.3233/EFI-200418]

36. Díez-Gutiérrez, E.J.; Gajardo Espinoza, K. Education online in lockdown: Limits and possibilities. The vision of families in Spain. Equal. Divers. Incl.; 2022; 41, pp. 34-48. [DOI: https://dx.doi.org/10.1108/EDI-07-2020-0194]

37. Navarro-Espinosa, J.A.; Vaquero-Abellán, M.; Perea-Moreno, A.J.; Pedrós-Pérez, G.; Aparicio-Martínez, P.; Martínez-Jiménez, P. The Higher Education sustainability before and during the COVID-19 pandemic: A Spanish and Ecuadorian case. Sustainability; 2021; 13, 6363. [DOI: https://dx.doi.org/10.3390/su13116363]

38. Torrecillas, C. El reto de la docencia online para las universidades públicas españolas ante la pandemia del COVID-19. ICEI Pap. COVID-19; 2020; 16, 4.

39. Çevik, H.; Duman, T. Analyzing the effects of e-learning on Science Education. Int. J. Instr.; 2018; 11, pp. 345-362. [DOI: https://dx.doi.org/10.12973/iji.2018.11124a]

40. Margottini, M.; Rossi, F. Self-regulation processes and feedback in online learning. J. Educ. Cult. Psychol. Stud.; 2020; 21, pp. 193-209. [DOI: https://dx.doi.org/10.7358/ecps-2020-021-marg]

41. Simamora, R.M.; De Fretes, D.; Purba, E.D.; Pasaribu, D. Practices, challenges, and prospects of online learning during COVID-19 pandemic in Higher Education: Lecturer perspectives. Stud. Learn. Teach.; 2020; 1, pp. 185-208. [DOI: https://dx.doi.org/10.46627/silet.v1i3.45]

42. Quintas-Hijós, A.; Peñarrubia-Lozano, C.; Bustamante, J.C. Analysis of the applicability and utility of a gamified didactics with exergames at Primary Schools: Qualitative findings from a natural experiment. PLoS ONE; 2020; 15, e0231269. [DOI: https://dx.doi.org/10.1371/journal.pone.0231269]

43. Peñarrubia-Lozano, C.; Segura-Berges, M.; Lizalde-Gil, M.; Bustamante, J.C. A qualitative analysis of implementing e-learning during the COVID-19 lockdown. Sustainability; 2021; 13, 3317. [DOI: https://dx.doi.org/10.3390/su13063317]

44. Strauss, A.; Corbin, C. Basics of Qualitative Research. Techniques and Procedures for Developing Grounded Theory; SAGE Publications: London, UK, 1998.

45. Zarei, S.; Mohammadi, S. Challenges of Higher Education related to e-learning in developing countries during COVID-19 spread: A review of the perspectives of students, instructors, policymakers, and ICT experts. Environ. Sci. Pollut. Res.; 2021; ahead of print [DOI: https://dx.doi.org/10.1007/s11356-021-14647-2]

46. García-Sampedro, M.; Peña-Suárez, E.; Rodríguez-Olay, L. Online education during the COVID 19 lockdown and school closures in Spain. Teachers’ perceptions. Aloma; 2021; 39, pp. 43-51. [DOI: https://dx.doi.org/10.51698/ALOMA.2021.39.2.43-51]

47. Llerena-Izquierdo, J.; Ayala-Carabajo, R. University teacher training during the COVID-19 emergency: The role of online teaching-learning tools. Information Technology and Systems. ICITS 2021. Advances in Intelligent Systems and Computing; Rocha, Á.; Ferrás, C.; López-López, P.C.; Guarda, T. Springer: New York, NY, USA, 2021; Volume 1331, pp. 90-99. [DOI: https://dx.doi.org/10.1007/978-3-030-68418-1_10]

48. Şahin, F.; Şahin, Y.L. Examining the acceptance of e-learning systems during the pandemic: The role of compatibility, enjoyment and anxiety. Int. Technol. Educ. J.; 2021; 5, pp. 1-10.

49. Halitoglu, V. Attitudes of student teachers towards distance education within the Context of COVID-19 pandemic. Int. J. Curric. Instr.; 2021; 13, pp. 816-838.

50. Mateus, J.C.; Andrada, P.; González-Cabrera, C.; Ugalde, C.; Novomisky, S. Teachers’ perspectives for a critical agenda in Media Education Post COVID-19. A comparative study in Latin America. Comunicar; 2022; 30, pp. 9-18. [DOI: https://dx.doi.org/10.3916/C70-2022-01]

51. Meccawy, Z.; Meccawy, M.; Alsobhi, A. Assessment in ‘Survival Mode’: Student and faculty perceptions of online assessment practices in HE during COVID-19 pandemic. Int. J. Educ. Integr.; 2021; 17, pp. 1-24. [DOI: https://dx.doi.org/10.1007/s40979-021-00083-9]

52. Koliasa, O.; Lelet, I.; Serebriakova, V.; Yukhymets, S. The use of hypermedia technologies in Higher Education institutions during COVID lockdown. Arab World Engl. J.; 2021; 1, pp. 68-79. [DOI: https://dx.doi.org/10.24093/awej/covid.5]

53. Mocanu, G.D.; Murariu, G.; Iordan, D.A.; Sandu, I.; Munteanu, M.O.A. The perception of the online teaching process during the COVID-19 pandemic for the students of the Physical Education and Sports Domain. Appl. Sci.; 2021; 11, 5558. [DOI: https://dx.doi.org/10.3390/app11125558]

54. Ożadowicz, A. Modified blended learning in Engineering Higher Education during the COVID-19 lockdown-building automation courses case study. Educ. Sci.; 2020; 10, 292. [DOI: https://dx.doi.org/10.3390/educsci10100292]

55. Leal-Filho, W.; Wall, T.; Rayman-Bacchus, L.; Mifsud, M.; Pritchard, D.J.; Orlovic Lovren, V.; Farinha, C.; Petrovic, D.S.; Balogun, A.-L. Impacts of COVID-19 and socialiIsolation on academic staff and students at Universities: A cross-sectional study. BMC Public Health; 2021; 21, 1213. [DOI: https://dx.doi.org/10.1186/s12889-021-11040-z]

56. Lin, Y.; Nguyen, H. International students’ perspectives on e-learning during Covid-19 in Higher Education in Australia: A study of an Asian student. Electron. J. e-Learn.; 2021; 19, pp. 241-251. [DOI: https://dx.doi.org/10.34190/ejel.19.4.2349]

57. Malik, M.; Javed, S. Perceived stress among university students in Oman during COVID-19-induced e-Learning. Middle East Curr. Psychiatry; 2021; 28, 49. [DOI: https://dx.doi.org/10.1186/s43045-021-00131-7]

58. Singh, M.; Adebayo, S.O.; Saini, M.; Singh, J. Indian Government e-learning initiatives in response to COVID-19 crisis: A case study on online learning in Indian Higher Education System. Educ. Inf. Technol.; 2021; 26, pp. 7569-7607. [DOI: https://dx.doi.org/10.1007/s10639-021-10585-1]

59. Keržič, D.; Alex, J.K.; Alvarado, R.P.B.; da Silva Bezerra, D.; Cheraghi, M.; Dobrowolska, B.; Fagbamigbe, A.F.; Faris, M.A.I.E.; França, T.; González-Fernández, B. et al. Academic student satisfaction and perceived performance in the e-learning environment during the COVID-19 pandemic: Evidence across ten countries. PLoS ONE; 2021; 16, e0258807. [DOI: https://dx.doi.org/10.1371/journal.pone.0258807]

60. Belda-Medina, J. Enhancing multimodal interaction and communicative competence through Task-Based Language Teaching (TBLT) in synchronous Computer-Mediated Communication (SCMC). Educ. Sci.; 2021; 11, 723. [DOI: https://dx.doi.org/10.3390/educsci11110723]

61. Díaz-Noguera, M.D.; Hervás-Gómez, C.; De la Calle-Cabrera, A.M.; López-Meneses, E. Autonomy, motivation, and digital pedagogy are key factors in the perceptions of Spanish Higher-Education students toward online learning during the COVID-19 pandemic. Int. J. Environ. Res. Public Health; 2022; 19, 654. [DOI: https://dx.doi.org/10.3390/ijerph19020654]

62. Boca, G.D. Factors influencing students’ behavior and attitude towards online education during COVID-19. Sustainability; 2021; 13, 7469. [DOI: https://dx.doi.org/10.3390/su13137469]

63. Rossettini, G.; Geri, T.; Turolla, A.; Viceconti, A.; Scumà, C.; Mirandola, M.; Dell’Isola, A.; Gianola, S.; Maselli, F.; Palese, A. Online teaching in Physiotherapy Education during COVID-19 pandemic in Italy: A retrospective case-control study on students’ satisfaction and performance. BMC Med. Educ.; 2021; 21, 456. [DOI: https://dx.doi.org/10.1186/s12909-021-02896-1] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/34455979]

64. Rizun, M.; Strzelecki, A. Students’ acceptance of the COVID-19 impact on shifting Higher Education to distance learning in Poland. Int. J. Environ. Res. Public Health; 2020; 17, 6468. [DOI: https://dx.doi.org/10.3390/ijerph17186468]

65. Chen, B.H.; Chiou, H.H. Learning style, sense of community and learning effectiveness in hybrid learning environment. Interact. Learn. Environ.; 2014; 22, pp. 485-496. [DOI: https://dx.doi.org/10.1080/10494820.2012.680971]

66. Khan, M.A. COVID-19’s impact on Higher Education: A rapid review of early reactive literature. Educ. Sci.; 2021; 11, 421. [DOI: https://dx.doi.org/10.3390/educsci11080421]

67. Hernández Armenteros, J.; Péres García, J.A. Crue Universidades Españolas—La Universidad Española en Cifras. 2018; Available online: https://www.crue.org/wp-content/uploads/2020/02/UEC-1718_FINAL_DIGITAL.pdf (accessed on 3 April 2022).

68. Hennink, M.; Kaiser, B.N. Sample sizes for saturation in qualitative research: A systematic review of empirical tests. Soc. Sci. Med.; 2022; 292, 114523. [DOI: https://dx.doi.org/10.1016/j.socscimed.2021.114523] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/34785096]

69. Constantinou, C.S.; Georgiou, M.; Perdikogianni, M. A comparative method for themes saturation (CoMeTS) in qualitative interviews. Qual. Res.; 2017; 17, pp. 571-588. [DOI: https://dx.doi.org/10.1177/1468794116686650]

70. Young, D.S.; Casey, E.A. An examination of the sufficiency of small qualitative samples. Soc. Work Res.; 2019; 43, pp. 53-58. [DOI: https://dx.doi.org/10.1093/swr/svy026]

71. Bakhronova, D. COVID-19 y Educación Superior. La mundialización de la educación en línea. Reli. Rev. Cienc. Soc. Y Humanid.; 2021; 6, pp. 62-69. [DOI: https://dx.doi.org/10.46652/rgn.v6i28.809]

72. Kara, A. COVID-19 pandemic and possible trends into the future of Higher Education: A review. J. Educ. Educ. Dev.; 2021; 8, pp. 9-26. [DOI: https://dx.doi.org/10.22555/joeed.v8i1.183]

73. Bao, W. COVID-19 and online teaching in Higher Education: A case study of Peking University. Hum. Behav. Emerg. Technol.; 2020; 2, pp. 113-115. [DOI: https://dx.doi.org/10.1002/hbe2.191] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/32510042]

74. Turnbull, D.; Chugh, R.; Luck, J. Transitioning to E-Learning during the COVID-19 Pandemic: How have Higher Education Institutions responded to the challenge?. Educ. Inf. Technol.; 2021; 26, pp. 6401-6419. [DOI: https://dx.doi.org/10.1007/s10639-021-10633-w] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/34177349]