1. Introduction
The onset of the COVID-19 global pandemic has imposed a negative impact on the Sustainable Development Goals (SDGs) of Education 2030 (SDG 4) declared by the United Nations in 2015 [1,2]. The goal of SDG 4 is to equip adult learners with the knowledge and skills necessary for the benefit of global sustainable development by providing flexible education pathways in higher education [3]. Higher education institutes (HEIs) have to equip adult learners with the knowledge and skills of sustainable learning in education (SLE) to cope with challenging and complicated circumstances [4,5]. Thus, HEIs are responsible for assisting adult learners in overcoming barriers and interruptions by providing sustainable quality education, which is essential for the country’s SDGs [6,7]. HEIs were compelled to rapidly redesign teaching approaches and classroom arrangements to offer flexible and sustainable learning pathways for SLE in the times of COVID-19 lockdown [8,9].
Instructors in China innovatively planned their teaching by providing learners with pre-class self-study materials during in-campus classroom lessons due to the uncertainty of intermittent lockdowns [10]. Advancements in information communication technology have made such learning arrangements that combine pre-class and in-classroom learning activities feasible. These arrangements are critical for the sustainability of education programmes under enforced home confinements and campus lockdowns during the pandemic [11,12]. Most importantly, these new pedagogical approaches can alleviate the threats of the pandemic on the sustainability of the country’s education system [13] and promote SLE amid COVID-19 [4].
Learners with flipped classroom approaches were provided with instructional materials for self-study at home before participating in practical learning activities in the classroom, reducing the reliance on face-to-face teaching instead of a traditional classroom arrangement that is totally reliant on it [14]. Learner achievement and engagement are two important indicators of the success of flipped classrooms in HEIs [15,16]. Research also indicates that the success of flipped classrooms depends on sustained learner engagement in learning activities [17,18]. According to self-determination theory (SDT), gamification has the potential to promote learner achievement and motivate learner engagement in the learning activities of flipped classrooms [19]. SDT is an empirically derived theory in social contexts that differentiates human motivation in terms of autonomous and controlled, in which autonomously motivated learners thrive in educational settings [20,21]. Gamification is the use of game-design elements in non-gaming contexts and has the potential to motivate learning [22,23]. Based on SDT, gamification is hypothesised to foster intrinsic motivation and satisfy the psychological needs for autonomy, competence, and social relatedness in flipped classrooms [19].
Flipped classroom approaches have also been adopted in HEI programmes during the COVID-19 pandemic for sustainable learning in higher education [24]. Instructors provide prerecorded instructional videos to enable learners to study on their own during home confinement periods. In addition to the flexibility, the flipped classroom approach also allows more in-class time for promoting SLE. The key components of SLE are (1) active learning; (2) independent learning; (3) collaborative learning; (4) renewing and relearning; (5) knowledge and skills transferability [4]. The key learning activities in flipped classrooms are pre-class self-study (i.e., independent learning), knowledge application (i.e., transferability), problem-solving (i.e., renewing and relearning), and peer-assisted learning (i.e., collaborative and active learning) [8,19,25,26]; these are especially advantageous in promoting SLE. HEIs in China flexibly adopted flipped classroom approaches in response to the government’s policy of “suspension of classes without interrupting learning” during the COVID-19 pandemic [27,28]. However, additional research is required to investigate the impact of the abrupt pedagogical change caused by flipped classrooms in the long history of Chinese education, which is dominated by traditional didactic teaching [29].
This study investigated the impact of the flipped classroom and gamification approaches on learner achievement and engagement in postgraduate business programmes for SLE in China during the COVID-19 pandemic. Three classes with different instructional interventions were applied: gamified flipped classroom (GFC), non-gamified flipped classroom (NFC) and gamified traditional classroom (GTC). Furthermore, the study was guided by the following two research questions:
RQ1. How does the adoption of the flipped classroom approach influence (a) learner achievement and (b) learner engagement compared to non-flipped approaches?
RQ2. How does the adoption of gamification influence (a) learner achievement and (b) learner engagement compared to non-gamified approaches?
2. Literature Review
We have the following three sections of the literature review. First, we discuss the SLE and flipped classroom approach during the COVID-19 pandemic. Second, we review how the SDT framework supports and motivates SLE and flipped classrooms. Third, we explore the integration of gamification with flipped and traditional classrooms for SLE.
2.1. SLE and Flipped Classroom during COVID-19 Pandemic
SLE is a philosophy of learning and teaching, which is not limited to education for or about sustainability [30]. SLE supports SDGs in education for adult learners, whose past knowledge and skills are viewed as flexible and receptive to modification [31]. SLE is an emerging and timely concept designed to enable learners to keep pace with technological and social changes [8]. HEIs with SLE initiatives provide learners with individual and group learning [32]. SLE promotes learners’ willingness to participate and contribute to the learning process, reflected in learners’ behavioural, emotional, and cognitive engagement in learning activities [33,34]. Ben-Eliyahu [4] summarised the key components of SLE as:
Active learning: refers to seeking information actively and intentionally;
Independent learning: refers to self-sustained learning, which involves learning by oneself, being autodidactic, identifying and finding sources for what needs to be learned;
Collaborative learning is conducted in groups with peers of different levels of knowledge and is a process of acquiring and creating knowledge;
Renewing and relearning is the self-reflection and awareness of our knowledge that might have been forgotten or need to be relearned because of advancements in a field;
Knowledge and skills transferability: refers to using knowledge and skills learned in one setting (e.g., in the classroom) to another setting (e.g., at work).
SLE could be integrated into education settings with information communication technology (ICT), online courses (e.g., MOOC), and pre-class and in-class learning [8]. Such flexible pedagogical approaches are especially beneficial to ensure learning sustainability when classroom teaching is suspended during uncertain and complex situations [35].
The COVID-19 pandemic impedes learning sustainability by imposing academic uncertainty on HEIs. Learners reported negative emotions and decreased satisfaction due to interruptions to their sustainable learning [36,37]. HEIs in China needed to rapidly adjust their instructional practices to anticipate the changes in the country’s confinement policies [38]. In sync with this development, educators in China quickly adopted the flipped classroom approach with the aid of information communication technologies [28,29]. The flipped classroom approach is a technology-enhanced pedagogy that free ups class time by providing pre-class instructional videos [39]. Flipped classrooms emphasise learner-centric and problem-solving activities inside the classroom [40], which enable independent and collaborative learning [41]. Peer-to-peer and peer-to-teacher interactions promote active learning, and problem-solving activities encourage knowledge renewing and relearning of SLE [4,29]. In addition, problem-solving activities for knowledge transferability also enhance perceived learning and learner achievement [42,43].
Chinese learners are very concerned about the delays in their academic progress [44]. Therefore, they favoured flipped classroom approaches as they could benefit from the self-paced independent learning enabled by pre-class instructional materials, such as pre-recorded videos, even during confinement and closures of campus. They could then look forward to increased engagement in collaborative learning once classroom teaching was allowed [17]. However, the flipped classroom approach was not common in HEIs worldwide or in China before the onset of the COVID-19 pandemic [45]. HEIs in China have long adopted teacher-centric didactic pedagogy in which programmes are primarily conducted face-to-face with scheduled timetables [46]. The COVID-19 pandemic precipitated an abrupt transition in traditional classroom teaching approaches. Thus, additional research with a solid theoretical foundation is required to investigate the impact of this abrupt pedagogical change on learning, mainly learner achievement and engagement [24,47]. This is especially important during the interruptions and frequent transitions between pre-class and in-class modes of learning caused by the COVID-19 pandemic [48,49,50].
2.2. Self-Determination Theory
Motivation is the main catalyst for sustainable learning behaviour [51]. SDT posits that humans inherently possess the propensity to be curious and interested in learning and developing [52,53]. The desire for sustainable development of business executives and entrepreneurs helps to attain the SDGs of the country [54]. SDT states that when the three basic psychological needs of learners (i.e., autonomy, competence, and relatedness) are fulfilled, they are motivated and are more likely to engage in education contexts [55,56]. Learner engagement is characterised by learners’ investment in learning, desire to exceed standard requirements, and preference for challenges of varying difficulty levels [57]. Learner engagement includes behavioural, emotional, and cognitive dimensions [58]. Studies by Abeysekera and Dawson [40] and Sergis et al. [59] indicate that SDT could be drawn upon to support learning in flipped classroom environments. As less class time is consumed by lectures, learners have more opportunities for self-regulated, independent, peer-to-peer collaborative learning and hands-on problem-solving activities for knowledge transfer [59]. Hence, flipped classrooms facilitate autonomy, and supportive feedback from teachers and peers promotes competence and relatedness [60].
A recent study of HEIs during the COVID-19 pandemic found that learning persistence and sustainability were also directly impacted by learners’ needs for autonomy, relatedness, and competence [61]. Therefore, higher engagement resulting from motivation, as explained by SDT, is a prerequisite for the sustainability of education programmes during the COVID-19 pandemic [62]. Supportive feedback from teachers is especially crucial for engaging learners and can be conveyed via game-design elements such as points, badges, and leaderboards. Learner engagement is positively related to perceived learning and sustainable learning [5,63,64]. Thus, flipped and gamified classrooms based on the SDT framework may promote learning through knowledge transferability in problem-solving activities and collaborative and active learning [19,65]. Furthermore, the adoption of flipped classrooms and game-design elements based on SDT has the potential to promote SLE in HEIs during the COVID-19 pandemic [4,66].
2.3. Gamification
Gamification is the use of game-design elements in a non-game environment [67], which can be a sustainable method to achieve the United Nations’ Sustainable Development Goals for quality education (SDG 4) [68]. In the context of education, both gamification and SDT aim to promote learner achievement and engagement [69]. Gamification is typically used in flipped classrooms to provide feedback to encourage learners to take on task challenges with progressively increasing difficulty levels and to motivate independent and collaborative learning [70]. Feedback and challenges are important to promote learning achievement and engagement [71]. A progressive increase in the difficulty level intrinsically motivates learners to complete more challenging tasks, especially those tasks that are relevant to their personal goals [54,72]. One major personal goal for learners in higher education business programmes is to translate knowledge to practice (i.e., knowledge transferability), as it enhances their employability [73]. Gamification promotes active learning and engagement in problem-solving activities that bridge the knowledge-to-practice gap [74,75]. Game-design elements engage learners by helping them to build new knowledge (i.e., renewing and relearning) and support learning achievement [70]. Therefore, pedagogies with gamification promote SLE (i.e., renewing and relearning, collaborative and active learning, and knowledge transferability) [4,76].
The most common game-design elements used to motivate learning in higher education are points, badges, and leaderboards [22]. These game-design elements serve the following specific purposes [77] (Table 1):
Points capture granular feedback directly related to learners’ specific actions, e.g., participation in in-class learning exercises and pursuing optional and challenging tasks.
Badges capture sustained feedback to recognise learners’ progress and contributions to the activity group tasks.
Leaderboards capture cumulative feedback on a series of actions performed by the learner and their contributions to completing tasks by displaying the number of badges earned in activity groups during the entire course.
Gamification can be adopted in flipped classrooms to motivate learners to increase their efforts and participation during the COVID-19 pandemic [78]. Recent studies found that gamification [79] and flipped classrooms [70] support positive learning achievement and engagement. Further, SDT provides the theoretical foundation on which gamification and flipped classroom pedagogies can be grounded [59,80]. Nevertheless, there is a paucity of research on gamification applied in flipped classrooms for business education among adults [81].
3. Research Methods
We adopted an explanatory sequential design with a mixed-methods approach in which both quantitative and qualitative research methods were used. When the quantitative phase is followed by the qualitative phase, it helps to explain the mechanism behind the quantitative results [82]. This approach can also be adopted to provide an insightful comparison between flipped classrooms with gamification to traditional classrooms in the existing study [60]. In this section, we first introduce participants, followed by the research design, data collection, and analysis.
3.1. Participants
Participants were adult business executives (aged 25–42 years) studying postgraduate business programmes of HEI in eastern China. Their participation was voluntary, and participants could withdraw from the study without negative consequences at any time. They were assigned to one of the three different instructional approaches (GFC, n = 25; NFC, n = 24; GTC, n = 19) for the second module in weeks 6–10. For flipped classrooms (i.e., GFC and NFC), the learners were provided with pre-class instructional videos recorded by the teacher via the institution’s learning management system (Moodle) four weeks before the scheduled in-class lessons. The videos were intended to impart basic knowledge before the in-class lessons. After watching the videos, the learners could complete self-evaluation multiple-choice quizzes during their free time. Owing to the high degree of trust and coordination between the government and Chinese higher education institutes [14], universities reopened soon after local COVID-19 cases were contained. Rigorous hygiene measures enabled the resumption of regular face-to-face lessons in classrooms. Teachers focused more on advanced topics during the in-class lessons once the institute was allowed to reopen. In this study, the learners in the flipped classrooms spent more in-class time on learning activities such as knowledge application for solving real and simulated business problems and assignment discussions.
The learners in the gamified classrooms (i.e., GFC and GTC) were given an account and access code to Qitoupiao. Guidelines and descriptions of expectations throughout the pre-class (GFC only) and in-class learning stages (GFC and GTC) were provided. These learners were not graded or provided marks for the points or badges they gained in the learning activities to prevent them from aiming at high scores instead of being motivated by the game-design elements. A summary of the guidelines and necessary information was made available on Moodle, as shown in Figure 1.
Participants in the three classrooms had to submit a post-class assignment in an essay of around 2000 words after completing classroom lessons at the end of each module. Assignment questions mainly focused on knowledge application for solving real business problems the learners are facing at work. Furthermore, all assignment questions were evaluated and approved by the programme academic team and external examiners.
3.2. Research Design
The study was conducted in the context of adult postgraduate business education programmes in China during the COVID-19 pandemic from November 2021 to January 2022. The programmes consisted of two modules. Each module lasted for five weeks and consisted of 16 h of in-class lessons (eight hours each day for Day-1 and Day-2). Three kinds of intervention, namely the gamified flipped classroom (GFC, experimental group-1), the non-gamified flipped classroom (NFC, experimental group-2), and the gamified traditional classroom (GTC, experimental group-3), were introduced in the second module, which started at the sixth week. To evaluate the influence of flipped classrooms on learner achievement and engagement (i.e., RQ 1), we compared the GFC and GTC experimental groups, and to evaluate the influence of gamification on learner achievement and learner engagement (i.e., RQ 2), we compared the GFC and NFC experimental groups. The experimental design is summarised in Table 2.
3.2.1. Class Rundown
All in-class lessons were conducted face-to-face after city lockdowns were lifted. In the first module (weeks 1–5), the students in all three experimental groups learned under the traditional classroom approach (i.e., teacher-centric didactic approach with no pre-class videos and gamification). The assignment marks from the first module were recorded as pre-intervention references. The three instructional interventions were adopted for the respective experimental groups in the second module during weeks 6–10 (Figure 2). For the flipped classrooms (GFC and NFC), eight sessions of pre-recorded instructional videos with a duration of 30 min each were recorded by the module teacher and provided via the institution’s learning management system (Moodle). Self-evaluation multiple-choice quizzes with only ten questions were offered to avoid heavy cognitive load in the pre-class stage of learning [40,83].
3.2.2. Lesson Schedule
The face-to-face lesson schedules were the same across the three experimental groups for the first four morning hours of lecture lessons. This was followed by two hours of case study lessons in the early afternoon for all three classrooms (GFC, NFC, and GTC). The case study lessons served to expand the scope of learning and knowledge through peer interactive and collaborative learning. For the flipped classrooms (GFC and NFC), there were two hours of knowledge application and assignment discussion lessons in the late afternoon of Day-1 and Day-2, respectively. In contrast, for the non-flipped classroom (GTC), there were lecture lessons in the late afternoon of both Day-1 and Day-2. Figure 3 provides details of the lesson schedule.
The case study lessons format was the same for all three classrooms, which started with a business case relevant to the morning lectures. Learners were grouped into activity groups of five to seven peer members to discuss the case according to the guiding questions. For example, the learners discussed the case of emotional pricing and its implications for marketing strategy in Figure 4. Each activity group could then choose one of three levels of difficulty to present their results: a summary report of the discussion (Easy level), real application with examples (Medium level), or real application with a business plan (Hard level). The learners’ choices were recorded in class observation reports by teaching assistants.
For the two flipped classrooms (GFC and NFC), there were two hours of knowledge application (Figure 5) and two hours of assignment discussion (Figure 6) during the late afternoon of Day-1 and Day-2, respectively. In the knowledge application lessons, the teachers introduced a topic or questions that focused on applying the knowledge learned. For example, as shown in Figure 3, the learners learned different pricing strategies in the morning lessons and then attempted to apply them in real or simulated business scenarios. These lessons were intended to strengthen their ability to apply their knowledge for solving real or simulated business problems, which was knowledge transferability and most relevant to their jobs and employment. Lastly, the assignment discussion lessons on Day-2 allowed the learners to discuss the individual assignment questions and learn with their peers collaboratively. Peer-to-peer interactive and collaborative learning could inspire further thoughts, ideas, and solutions to the business problems listed in the assignment questions. Learners could renew and relearn in the process. Because the learners in the flipped classrooms (GFC and NFC) were aided by the knowledge application and assignment discussion lessons, they subsequently required less time to complete the same assignment writing required for all three experimental groups after the classroom lessons. By contrast, those in GTC had to work on their assignment writing from the beginning and required more time to complete it by themselves after classroom lessons.
Learner surveys, interviews, class observation reports, and written feedback were collected from all three classes at the end of Day-2 after the completion of all the in-class lessons of the second module (i.e., in week 10).
3.2.3. Application of Game-Design Elements
During the case study lessons, the learners were divided into activity groups of 5–7 peers per group. Game-design elements were applied for the gamified classrooms (GFC and GTC). To ensure that gamification was effective, points, badges, and leaderboards were displayed through the Chinese classroom application Qitoupiao (‘Voting Together’). These game-design elements and their use in the two gamified classrooms are described below and in Table 3.
-
Points: Learners received points on completing tasks in the in-class learning activities. Points serve as feedback to encourage learners to complete subsequent learning activities and achieve more advanced challenges together in a group [84,85]. One point was allotted to activity groups that complete their discussion and presented the results as a summary report, which was the lowest level of difficulty (Easy) to complete their task. Two and three points were allotted to activity groups that chose the intermediate (Medium) or the most challenging (Hard) levels of difficulty to present their discussion results in a real application with examples and a business plan, respectively.
-
Badges: Learners received badges when they provided innovative ideas and solutions to problems and questions during in-class learning activities. Badges serve as recognitions of a learner’s contributions and encourage participation during learning activities [60]. Badges also promote social validation as they provide opportunities for learners to show their conformity and progress towards the expected learning behaviour with their peers [86]. Learners in an activity group obtained one badge when any member of the group presented a good idea, new knowledge, or an innovative solution to a problem or question. A ‘good idea’ refers to a new way to apply the knowledge learned in the class, ‘new knowledge’ refers to a point that was not mentioned or taught in the class, and an ‘innovative solution’ is a solution to a problem-solving task that was not mentioned or taught in the class.
-
Leaderboards: Teamwork was encouraged within each activity group and the learners were also encouraged to compare their performance with those of other activity groups in the same classroom [87]. Thus, the numbers of accumulated badges of each activity group were displayed on the class leaderboard. The leaderboard was intended to inspire intragroup peer collaborative learning and a healthy intergroup competition amongst the groups to achieve a prominent position on the leaderboard by contributing to learning activities for more badges [88,89].
3.3. Data Collection
Both quantitative and qualitative data were collected during the in-class and post-class stages (Table 4). Class observation reports were collected by teaching assistants during the in-class stages. Data were from the post-class stage and were collected based on three sources: learner survey, learner interview, and assignment marks. The survey used a 5-point Likert scale that ranged from ‘Strongly Agree’ (5) through ‘Neutral’ (3) to ‘Strongly Disagree’ (1). The learner survey and interview focused on the learners’ perceptions and their suggestions for pedagogical improvement.
The effects of classroom approaches (i.e., GFC, NFC, and GTC) on learner achievement were evaluated based on the assignment marks. The teachers graded the assignment strictly according to the marking scheme and rubrics provided by the institute. All in-class learning activities, topics and assignment questions were approved by the school appointed external examiner and the academic programme committee. Thirty per cent of the assignment marks were countermarked and sent to the external examiner for final review to ensure accurate assessment. Discrepancies in marks were discussed and were resolved in the board of examiners’ meetings.
The learners were invited to complete a survey at the end of classroom lessons in the second module. The survey consisted of questions on four themes: (i) perceived learning (Items 1–3), (ii) behavioural engagement (Items 4–8), (iii) emotional engagement (Items 9–13), and (iv) cognitive engagement (Items 14–17). It also contained one open-ended question (Item 18). The learners were free to respond, and their responses were tagged using anonymised labels, e.g., GFC-Learner 1, NFC-Learner 2, and GTC-Learner 3. Sample items for each theme from the survey are presented in Table 5.
Qualitative data sources included Item 18 of the learner survey, class observation reports, and learner interviews. Class observation was adopted for evaluating learner engagement in in-class activities in accordance with the recommendation of a prior study [92]. Studies have found learners’ behavioural and cognitive engagement to be reflected in their participation in learning activities [60,93]. Another study found that the willingness and effort to exceed minimum requirements and adopt more challenging presentation methods indicated a higher learning engagement [94]. Thus, teaching assistants recorded class observation reports which focused on two aspects of the learners’ engagement: (1) the level of difficulty that they chose for their case study results presentation and (2) their learning participation. Learning participation ranges from passive receiving, active manipulating, and constructive generating to the most engaging interactive dialoguing [92]. Learner engagement and their participation also reflected the different components of SLE [4]. Teaching assistants recorded the learners’ participation in in-class activities by checking one box each for the level of difficulty and level of participation, respectively, that most closely matched their observations of classes during the in-class case study lessons (Figure 7).
Interviews help to understand learners’ behaviour, feelings, and interpretation of the learning environment [95]. Therefore, we conducted learner interviews after classroom lessons in the second module (Week 10). The interview adopted a semi-structured approach with a protocol (Table 6). The topics covered in the interview were based on the engagement framework of Fredricks et al. [96] and McCallum et al. [97]. Topics for exploring learner engagement included (1) behavioural engagement (e.g., task participation and completion), (2) emotional engagement (e.g., enjoyment or boredom), and (3) cognitive engagement (e.g., investing effort in learning). All interviews were conducted and recorded in Chinese by the researcher. Some transcripts were translated into English for reporting purposes. The interviewees were invited to check all original transcriptions, and discrepancies were corrected to ensure accuracy. Learners’ participation in the survey and interview was voluntary, and no incentive was offered. After the completion of the second module lessons, we received responses for 49 surveys (GFC n = 20, NFC n = 17, GTC n = 12) and 26 learner interviews (GFC n = 7, NFC n = 11, GTC n = 8).
3.4. Data Analysis
3.4.1. Quantitative Data Analysis
To determine the most appropriate statistical test for quantitative data analysis, quantitative data were first tested for normality [98]. A Kolmogorov–Smirnov test showed a significant deviation from normality for learner achievement (according to assignment marks) across the three classes (first module p < 0.001; second module p = 0.017). Therefore, non-parametric tests were used to analyse the quantitative data on learner achievement [98].
The Kruskal–Wallis non-parametric test for multiple group comparisons was performed on the assignment marks from the first module, which was taught in traditional classrooms, to check the initial equivalence of the three experimental groups (i.e., GFC, NFC, and GTC). The results showed that the three experimental groups were statistically equivalent (H = 0.256, p = 0.880) for the first module. The assignment marks from the second module of the three experimental groups were then tested with the Kruskal–Wallis test at a significance level of 0.05. Multiple Mann–Whitney tests were conducted for post hoc pairwise comparisons when significant differences were found [98]. Bonferroni correction was applied to avoid Type I errors [99]. Therefore, the post hoc analyses reported effects at a significance level of 0.05/3 = 0.0167. The effect size (r) was then calculated using the following formula [100]:
where z is the z-score, and N′ is the number of participants in the two experimental groups in each pairwise comparison.3.4.2. Qualitative Data Analysis
The qualitative data were transcribed in Chinese and analysed thematically by organising the data into categories. Direct quotations from participant interviews were used to ensure data validity [101]. The qualitative analysis procedures followed the suggestion of Creswell et al. [82]. Coding started with the shortest interview transcripts using some exact wordings of the participants and concepts from the literature. All assigned codes were reviewed and grouped with redundant codes to produce a preliminary list of codes, which were then used to analyse the rest of the qualitative data. Exemplary quotes that clearly illustrated new emerging themes were identified and added to the list of codes, and similar codes were organised into subthemes.
Double-coding was adopted when data were descriptively and inferentially meaningful but not in neat or isolated units [102,103]. Thirty percent of the qualitative data were double-coded by an independent coder, and the intercoder agreement rate was checked. Disagreements between the coders were resolved through discussion. The data were validated by member checking to avoid any misunderstanding or misinterpretation [104]. Qualitative data were translated into English for reporting in this study.
Qualitative data from the written comments, feedback from learner surveys (Item 18), teaching assistant interviews, and class observation reports and teacher interviews were analysed with triangulation for comprehensive understanding [105] and to better explain the quantitative results [82].
4. Results
4.1. Learner Achievement
The Kruskal–Wallis test indicated a significant difference between the three classrooms in terms of the assignment marks obtained in the second module (H = 7.550, p = 0.023). Figure 8 shows a boxplot of the results.
Mann–Whitney tests for pairwise comparisons showed no statistically significant differences between GFC and GTC (p = 0.199) or between GFC and NFC (p = 0.117). However, the learners in GTC scored significantly higher assignment marks than those in NFC (U = 112.00, z = −2.875, p = 0.004) with an effect size r = 0.410. Table 7 shows the results for the pairwise comparison of assignment marks between NFC and GTC.
Although the qualitative data from the interviews revealed several benefits of gamification for learning achievement, the learners trusted knowledge delivered by teachers more than knowledge received from their peers, regardless of whether their peers had more firsthand experience in specific business scenarios. Interviews with both learners and teaching assistants indicated that the knowledge learned from teachers was reflected more frequently in the learners’ submissions of their assignments, which required them to apply the knowledge learned in the module to provide a solution to a real business problem in around 2000 words. Table 8 shows the major benefits of GTC identified from the learner interviews. These benefits included the generation of excitement and curiosity and the promotion of interactions and discussions in the classroom. In contrast, the learners in NFC did not experience a sense of ‘belonging’ (NFC-Learner 8) to the class. Keywords that occurred multiple times in the interviews of learners from NFC were ‘boring/dry’ (NFC-Learners 1 and 7) and ‘no interaction makes me sleepy/passive’ (NFC-Learners 8 and 10).
4.2. Learner Engagement
The Kolmogorov–Smirnov test showed that the data from the learner surveys significantly deviated from normality (p < 0.001); therefore, nonparametric Kruskal–Wallis tests were performed. The difference between GFC and NFC was significant (p = 0.014) for perceived learning. Post hoc pairwise Mann–Whitney tests showed that the learners in GFC scored significantly higher than those in NFC (U = 102, z = −2.448, p = 0.0140), with effect size r = −0.402. These results are shown in Table 9.
Class observation reports reflected that the learners in GFC engaged at the interactive dialoguing level during learning activities [93]. This shows that GFC was the most engaging of the three classrooms and performed all five SLE components. Those in NFC engaged half their time at the active manipulation level and the other half at the constructive generating level. This reveals that NFC was the least engaging among the three classrooms. Gamification elevated the participation level of learners in GTC to constructive generating, which showed three out of the five SLE components, indicating that GTC was more engaging than NFC (Table 10).
Keywords that occurred multiple times in learner interviews from the gamified classrooms were ‘active participation’ (GFC-Learners 2 and 8), ‘focused and engaged’ (GFC-Learner 7, GTC-Learner 3), and ‘the class was interesting, inspired my thoughts and stimulated deep learning’ (GFC-Learner 5). One point of contrast in the feedback between GFC and GTC related to peer-dependent vs. teacher-dependent learning:
GFC-Learner 2: ‘The presence of such exceptional classmates made me more actively engaged (in learning)’.
GTC-Learner 5: ‘In the classroom, I followed the rhythm and pace of the teacher, who enabled me to focus during the interactive learning lesson’.
The feedback from the interviews with those in NFC, in contrast, reflected that the learners faced certain obstacles to active participation:
NFC-Learner 1: ‘I wish the teacher could control classmates that engaged too much in irrelevant discussions.’
NFC-Learner 5: ‘I am not very familiar (close) with the classmates, and the (learning) interactions and bonding between us were not strong’.
In summary, there was a significant difference in learner achievement between GTC and NFC (p = 0.004). The learners in GTC scored significantly higher than those in NFC for learner achievement (U = 112.00, z = −2.875, p = 0.004), with effect size r = 0.410. Regarding learner engagement, the results of class observation reports and learner surveys indicated that the learners in GFC had the highest level of participation and perceived learning, and performed all five components of SLE.
5. Discussion
In this study, we compared the influence of flipped classrooms and gamification on learner achievement and engagement for SLE in the context of postgraduate business education during the COVID-19 pandemic. The findings are discussed in the following three subsections. First is the role of flipped classrooms and gamification during the COVID-19 pandemic. Second, teacher-dependency and learning interactions. Lastly, lessons learned for the further improvement of flipped classrooms and gamification pedagogical design for SLE in HEIs.
5.1. Role of Flipped Classroom and Gamification during COVID-19 Pandemic
Flipped classrooms were welcomed by learners [106] and helped to maintain the sustainability of learning programmes in higher education during COVID-19 lockdowns [24]. However, learner motivation declines in the absence of learning interactions [48,107]. Gamification promotes learning interactions and social networking [108]. The qualitative results of class observation reports from the teaching assistants on learner engagement concur with those of Lo and Hew [60] that learners in gamified classrooms (GFC and GTC) exhibit a stronger sense of engagement, as evidenced by their level of participation during in-class learning activities. In the context of adult postgraduate business education, the quantitative results of the post-class assignment in our study showed that GTC offers advantages for learner achievement and GFC for learner engagement, respectively. GFC improved learners’ perceived learning and engagement, but there was no significant impact on learner achievement. Several other studies have found similar results, suggesting the need for further investigation [109,110,111]. Our results confirm the observation of Bredow et al. [81] that the benefit of a simple flipped classroom (i.e., NFC) may not be apparent. According to McLean et al. [110], one reason for this might be that flipped classrooms challenge the learners’ perception of the teacher’s role as a knowledge provider to playing the combined role of knowledge application and synthesis facilitators.
5.2. Teacher-Dependency and Learning Interactions
A prior study found that, despite the autonomy that the flipped classroom pedagogy affords to learners, in-class interactive learning was still preferred over pre-class self-regulated study [39]. Peer-to-teacher interaction and learner preference for teachers to play the role of authority for knowledge delivery are dominant themes in the Asian context [112]. In flipped classrooms, teacher roles change from being like those of sages to those of facilitators, thereby conflicting with existing expectations [113]. Like previously published findings [114], our results found better learning achievement in a traditional classroom approach supplemented by gamification (i.e., GTC), as teacher recognition and feedback in the form of game-design elements were valued more than responses from peers [115]. Such teacher dependency was also observed in previous studies [116,117,118]. Teachers should introduce the importance and value of lesson content at the beginning of each lesson [119], offer a brief review of pre-class materials [120], and facilitate peer-to-teacher and peer-to-peer interactions [25].
In-class learning interactions played an important role in flipped classrooms [121], and there is a need to explore this in further detail [60,122]. Learning interactions can be either peer-to-peer or peer-to-teacher interactions. Our interviews with the teaching assistants indicated that the learners were used to a teacher-dependent style of learning, which may impede peer-to-peer learning. Learners needed the teacher to motivate the discussion when the learners’ participation waned. The following is a transcript from an interview with our experienced teaching assistant (JG):
‘Although the flipped classroom and gamified classroom may be the trends in future education, our learners are relatively older adults, and the education model that they experienced during their school ages was traditional. Many people are still inclined towards the traditional methods of education. We have to use different teaching modes according to the learners and their preferences and rely on teachers to facilitate changes’.
Therefore, to facilitate SLE in flipped classrooms, the cultural background and learning styles of the learners, especially the type of schooling that the learners experienced, must be considered. This is imperative if more instructional content is going to be imparted using the flipped classroom and gamification approaches during the COVID-19 pandemic [24].
5.3. Lessons Learned
The flipped classroom and gamification designs adopted in this study were theoretically grounded in SDT [56]. The three pedagogical interventions (GFC, NFC, and GTC) provided valuable insight into the practical application of the theories in the context of adult business education during the COVID-19 pandemic. Based on our findings, we provide three observations and recommendations for the design of flipped classrooms and gamification for SLE.
First, gamification plays an important role in motivating adult learners for learning participation and engagement, which in turn promotes SLE [123]. From the perspective of SDT, timely and evaluative feedback from teachers is vital to promoting learners’ sense of engagement [55]. Second, teachers should facilitate learning interactions at the earliest stage of flipped classrooms, such as at the beginning of the pre-class learning stage. Similar inferences have also been made in other studies [112,124]. Knowledge learned from pre-class materials must be recollected to set as the foundation of more advanced in-class learning [125]. This is critical, especially in a highly teacher-dependent learning culture. Teachers’ early participation and facilitation in both pre-class and in-class lessons would trigger learners’ curiosity and interest and stimulate initial discussions, leading to more learning participation [83]. Last, although we used the same game-design elements in both GFC and GTC, learner achievement in the flipped GFC was lower than that of the traditional GTC. Our results resonate with the findings of Jensen et al. [124] that the adoption of flipped classrooms to create more time for in-class learning may not be effective even with gamification. Teacher-dependency and learning culture must be considered. Adult learners in our study benefitted more from teacher-led lectures (i.e., from the traditional GTC) than the two flipped classroom approaches (GFC and NFC). This mirrors the study of Magana et al. [126].
Nonetheless, the flipped classroom is an effective pedagogical approach to ensuring the sustainability of education programmes in HEIs during COVID-19 lockdowns [24]. Pre-class self-study videos and materials ameliorate the anxiety of learners caused by academic uncertainty and interruptions in learning [127]. For example, one learner from GTC remarked, ‘if the school can provide pre-class videos, I can prepare in advance, integrate my previous knowledge and develop clarity on the areas in which I need to learn more (in the coming in-class lessons)’ (GTC-Learner 3). Therefore, a flipped classroom with gamification is a valuable pedagogical approach for the sustainability of higher business education, especially under the dynamic COVID-Zero strategy in China [128]. However, teachers cannot rely solely on flipped classrooms or gamification pedagogies. Indeed, the early presence of teachers along the entire learning journey, including in the pre-class stage, to facilitate peer-to-teacher and peer-to-peer facilitations is deemed necessary.
6. Conclusions and Recommendations for Future Study
This study compared learner achievement and engagement across three pedagogical approaches (i.e., GFC, NFC, and GTC). The learners in GTC performed the best in terms of learner achievement, whereas those in GFC reported the highest perceived learning and engagement for SLE. Qualitative findings from surveys and class observation reports reflected that gamification made the lessons more interesting and exciting, and promoted participation in in-class learning activities. Learners in GTC and GFC showed higher levels of participation and performed more key components of SLE, which were the reasons for higher learner achievement and engagement in comparison to NFC.
We provided three recommendations based on the study. First, flipped classrooms can be leveraged to maintain the continuity and sustainability of education programmes, especially under the uncertainty of lockdowns during the COVID-19 pandemic. Second, gamification plays a key role in improving learner achievement, and should therefore be integrated into flipped classrooms with the consideration of the learning culture and styles of the learners. Third, teachers must facilitate and be involved in all stages of flipped classrooms, especially in highly teacher-dependent learning cultures.
Notwithstanding the aforementioned contributions, there are certain limitations in the study. First, the sample sizes of the three classrooms were small, which reduces the generalisability of the results. Second, the study was conducted in a postgraduate adult business education setting and may therefore not be equally applicable to other educational contexts. Third, the duration of the study was limited to 10 weeks owing to restrictions caused by the COVID-19 pandemic. Therefore, the findings must not be interpreted as establishing that any one of the pedagogical approaches is better than the others. Further studies are necessary to explore the differences between flipped classrooms with and without gamification in larger samples and across a longer duration.
Formal analysis, investigation, writing—original draft preparation, writing—review and editing, project administration, and visualization, L.-K.N.; conceptualization, methodology, validation, data curation, and supervision, C.-K.L. All authors have read and agreed to the published version of the manuscript.
The study was conducted according to the guidelines of the Declaration of Helsinki and approved by the Human Research Ethics Committee of the Education University of Hong Kong (protocol code: 2020-2021-0436; date of approval: 28 October 2021).
Informed consent was obtained from all subjects involved in the study.
The data samples and detailed coding procedures can be accessed by contacting the author.
Thank you for the valuable and constructive feedback from the editor and professional reviewers. This study is in partial fulfilment of the Research Project/Thesis Requirements of the Education University of Hong Kong.
The authors declare no conflict of interest.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Enlarge this image.Figure 1. Summary of guidelines. (a) Guidelines for pre-class learning activities; (b) Guidelines for in-class learning activities.
Enlarge this image.Figure 4. Example of an in-class case study learning activity with three difficulty levels for presenting the results.
Game-design elements and feedback.
| Game-Design Element | Picture | Feedback | Description |
|---|---|---|---|
| Points | [Image omitted. Please see PDF.] | Granular feedback |
|
| Badges | [Image omitted. Please see PDF.] | Sustained feedback |
|
| Leaderboards | [Image omitted. Please see PDF.] | Cumulative feedback |
|
Experimental design.
| Experimental Groups and Approaches | Experimental Group 1: |
Experimental Group 2: |
Experimental Group 3: |
Research Question and Group Comparisons |
|---|---|---|---|---|
| Flipped classroom | Yes | Yes | No | RQ1: GFC and GTC |
| Gamified classroom | Yes | No | Yes | RQ2: GFC and NFC |
Application of game-design elements for gamified classrooms (GFC and GTC).
| Game-Design Element | Purpose | How They Were Awarded |
|---|---|---|
| Points |
Feedback to encourage task completion and progress towards more advanced and challenging learning activities. | Point(s) for in-class learning and presentation activities based on level of difficulty:
|
| Badges |
Recognition of learning efforts, participation, and contributions; promoting social recognition. | Badges were given to learners who presented good ideas, new knowledge or innovative solutions to a problem. |
| Leaderboards |
Inspire intragroup peer learning and intergroup competition for learning advancement. | The number of accumulated badges obtained in each activity group. |
Data sources and collection.
| Stage | Data | Purpose | RQs Addressed |
|---|---|---|---|
| In-class | Class observation report for levels of difficulty and participation | To evaluate learner engagement under the three instructional approaches (GFC, NFC, and GTC). | Learner engagement (RQ1 and RQ2) |
| Post-class | Learner survey | To evaluate learners’ perceived achievement and engagement under the three instructional approaches. | Learner perceived achievement and engagement |
| Learner interview | |||
| Assignment marks | To evaluate the learner’s overall achievement under the three instructional approaches | Learner achievement (RQ1 and RQ2) |
Sample items of the learner survey.
| Survey Items | Supporting Citation |
|---|---|
| Perceived learning (Items 1–3) |
Lo and Hew [ |
| Behavioural engagement (Items 4–8) |
Skinner et al. [ |
| Emotional engagement (Items 9–13) |
Skinner et al. [ |
| Cognitive Engagement (Items 14–17) |
Rotgans et al. [ |
Sample questions of the semi-structured learner interview protocol.
| Dependent |
Question | Citation Reference |
|---|---|---|
| Behavioural |
How did the classroom approach change the way you prepared for studying this module differently from other or previous classes you have attended? | Fredricks et al. [ |
| Emotional |
What did you find most/least interesting in your studying of this module? | MaCullum et al. [ |
| Cognitive |
Did you do anything extra that helped your learning when studying this module |
Pairwise comparison of assignment marks between NFC and GTC for the second module.
| Assignment Marks | n | Mean | SD | Mean Rank | Pairwise Comparison |
|---|---|---|---|---|---|
| NFC | 24 | 75.44 | 3.73 | 17.17 | GTC > NFC * |
| GTC | 19 | 78.79 | 3.05 | 28.11 |
* p < 0.0167 (Bonferroni correction).
Benefits of GTC.
| Theme | Example Quote |
|---|---|
| Excitement/curiosity | ‘Very excited, and engaged’ (GTC-Learner 3) |
| Peer learning | ‘Classmates encouraged each other to choose the difficult learning option’ (GTC-Learner 8) |
| Exchanges/interactions | ‘More feedback from the teacher, enabled by the rhythm of theteachingprocess, made me more attentive’ (GTC-Learner 5) |
Comparison between GFC and NFC for learner survey Item 2.
| Item | Survey Question | Class | n | Mean | SD | Mean Rank | Pairwise |
|---|---|---|---|---|---|---|---|
| 2 | I learned more because of the classroom format. | GFC | 20 | 4.80 | 0.41 | 22.40 | GFC > NFC * |
| NFC | 17 | 4.35 | 0.61 | 15.00 |
* p < 0.0167 (Bonferroni correction).
Level of difficulty and participation in in-class learning activities from class observation reports with SLE components. The order of the classrooms in terms of learner engagement and the number of SLE components that were performed was GFC > GTC > NFC.
| Classroom | Level of Difficulty | Level of |
SLE Components |
|---|---|---|---|
| GFC | Medium | Interactive |
Active learning |
| NFC | Medium | Active manipulating (50% *) | Active learning |
| Constructive |
Active learning |
||
| GTC | Medium | Constructive |
Active learning |
* % of the time learners engaged at the level of participation.
References
1. Chen, F.H.; Tsai, C.C.; Chung, P.Y.; Lo, W.S. Sustainability Learning in Education for Sustainable Development for 2030: An Observational Study Regarding Environmental Psychology and Responsible Behavior through Rural Community Travel. Sustainability; 2022; 14, 2779. [DOI: https://dx.doi.org/10.3390/su14052779]
2. Leicht, A.; Heiss, J.; Byun, W.J. Issues and Trends in Education for Sustainable Development; UNESCO Publishing: Paris, France, 2018; Volume 5.
3. Martin, M.; Godonoga, A. SDG 4-Policies for Flexible Learning Pathways in Higher Education: Taking Stock of Good Practices Internationally; IIEP-UNESCO: Paris, France, 2020.
4. Ben-Eliyahu, A. Sustainable learning in education. Sustainability; 2021; 13, 4250. [DOI: https://dx.doi.org/10.3390/su13084250]
5. Pilotti, M.A.; Al Ghazo, R. Sustainable education starts in the classroom. Sustainability; 2020; 12, 9573. [DOI: https://dx.doi.org/10.3390/su12229573]
6. Rovio-Johansson, A. Students’ Knowledge Progression: Sustainable Learning in Higher Education. Int. J. Learn. High.; 2016; 28, pp. 427-439.
7. Yassin, A.A.; Razak, N.A.; Qasem, Y.A.; Mohammed, M.A.S. Intercultural learning challenges affecting international students’ sustainable learning in Malaysian higher education institutions. Sustainability; 2020; 12, 7490. [DOI: https://dx.doi.org/10.3390/su12187490]
8. Hays, J.; Reinders, H. Sustainable learning and education: A curriculum for the future. Int. Rev. Educ.; 2020; 66, pp. 29-52. [DOI: https://dx.doi.org/10.1007/s11159-020-09820-7]
9. Pfeiffer, A.; Bezzina, S.; Denk, N.; Kriglstein, S.; Wimmer, S.; Thielen, C. Life-World Oriented Education in Times of the COVID-19 Lockdown. Twitch.tv as Pedagogical Tool. ICERI2020 Proc.; 2020; 9, 10.
10. Yu, K.; Wu, L.; Zhou, L. Research on the Mixed Education Mode for the Safety Engineering Major during the Coronavirus (COVID-19) Epidemic. Int. J. Environ. Res. Public Health; 2022; 19, 1967. [DOI: https://dx.doi.org/10.3390/ijerph19041967]
11. Pérez-Jorge, D.; Rodríguez-Jiménez, M.C.; Ariño-Mateo, E.; Barragán-Medero, F. The effect of COVID-19 in university tutoring models. Sustainability; 2020; 12, 8631. [DOI: https://dx.doi.org/10.3390/su12208631]
12. Murphy, M.P.A. COVID-19 and emergency eLearning: Consequences of the securitization of higher education for post-pandemic pedagogy. Contemp. Secur. Policy; 2020; 41, pp. 492-505. [DOI: https://dx.doi.org/10.1080/13523260.2020.1761749]
13. Le, K. Pre-Recorded Lectures, Live Online Lectures, and Student Academic Achievement. Sustainability; 2022; 14, 2910. [DOI: https://dx.doi.org/10.3390/su14052910]
14. Yang, R. China’s higher education during the COVID-19 pandemic: Some preliminary observations. High. Educ. Res. Dev.; 2020; 39, pp. 1317-1321. [DOI: https://dx.doi.org/10.1080/07294360.2020.1824212]
15. Ardhaoui, K.; Lemos, M.S.; Silva, S. Effects of new teaching approaches on motivation and achievement in higher education applied chemistry courses: A case study in Tunisia. Educ. Chem. Eng.; 2021; 36, pp. 160-170. [DOI: https://dx.doi.org/10.1016/j.ece.2021.05.004]
16. Bowden, J.L.H.; Tickle, L.; Naumann, K. The four pillars of tertiary student engagement and success: A holistic measurement approach. Stud. High. Educ.; 2021; 46, pp. 1207-1224. [DOI: https://dx.doi.org/10.1080/03075079.2019.1672647]
17. Jang, H.Y.; Kim, H.J. A meta-analysis of the cognitive, affective, and interpersonal outcomes of flipped classrooms in higher education. Educ. Sci.; 2020; 10, 115. [DOI: https://dx.doi.org/10.3390/educsci10040115]
18. Huang, B.; Hew, K.F.; Lo, C.K. Investigating the effects of gamification-enhanced flipped learning on undergraduate students’ behavioral and cognitive engagement. Interact. Learn. Environ.; 2019; 27, pp. 1106-1126. [DOI: https://dx.doi.org/10.1080/10494820.2018.1495653]
19. Sailer, M.; Sailer, M. Gamification of in-class activities in flipped classroom lectures. Br. J. Educ. Technol.; 2021; 52, pp. 75-90. [DOI: https://dx.doi.org/10.1111/bjet.12948]
20. Deci, E.L.; Ryan, R.M. Self-determination theory. Handbook of Theories of Social Psychology; Van Lange, P.A.M.; Kruglanski, A.W.; Higgins, E.T. Sage Publications Ltd.: New York, NY, USA, 2012; pp. 416-436. [DOI: https://dx.doi.org/10.4135/9781446249215.n21]
21. Reeve, J. Self-determination theory applied to educational settings. Handbook of Self-Determination Research: Theoretical and Applied Issues; Deci, E.L.; Ryan, R.M. University of Rochester Press: Rochester, NY, USA, 2002; Volume 2, pp. 183-204.
22. Deterding, S.; Dixon, D.; Khaled, R.; Nacke, L. From game design elements to gamefulness: Defining “gamification”. Proceedings of the 15th International Academic MindTrek Conference: Envisioning Future Media Environments; Tampere, Finland, 28–30 September 2011.
23. Dicheva, D.; Dichev, C.; Agre, G.; Angelova, G. Gamification in education: A systematic mapping study. J. Educ. Technol. Soc.; 2015; 18, pp. 75-88.
24. Collado-Valero, J.; Rodríguez-Infante, G.; Romero-González, M.; Gamboa-Ternero, S.; Navarro-Soria, I.; Lavigne-Cerván, R. Flipped classroom: Active methodology for sustainable learning in higher education during social distancing due to COVID-19. Sustainability; 2021; 13, 5336. [DOI: https://dx.doi.org/10.3390/su13105336]
25. Akram, A.; Sattar, K.; Ahmad, T.; Abdulghani, H.M.; John, J.; Yusoff, M.S.B. Flipping the “Learning Skills” Course during COVID-19: A Mixed-Modality Study. Educ. Med. J.; 2021; 13, pp. 31-41. [DOI: https://dx.doi.org/10.21315/eimj2021.13.3.4]
26. French, H.; Arias-Shah, A.; Gisondo, C.; Gray, M.M. Perspectives: The flipped classroom in graduate medical education. NeoReviews; 2020; 21, pp. e150-e156. [DOI: https://dx.doi.org/10.1542/neo.21-3-e150] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/32123119]
27. Peters, M.A.; Wang, H.; Ogunniran, M.O.; Huang, Y.; Green, B.; Chunga, J.O.; Quianoo, E.A.; Ren, Z.; Hollings, S.; Hayes, S. et al. China’s internationalized higher education during COVID-19: Collective student autoethnography. Postdigit. Sci. Educ.; 2020; 2, pp. 968-988. [DOI: https://dx.doi.org/10.1007/s42438-020-00128-1]
28. Wu, Z. How a Top Chinese University Is Responding to Coronavirus. 2020; Available online: https://www.weforum.org/agenda/2020/03/coronavirus-china-the-challenges-of-online-learning-for-universities/ (accessed on 28 March 2022).
29. Li, S.; Liao, X.; Burdick, W.; Tong, K. The Effectiveness of Flipped Classroom in Health Professions Education in China: A Systematic Review. J. Med. Educ. Curric.; 2020; 7, 2382120520962838. [DOI: https://dx.doi.org/10.1177/2382120520962838] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/33110941]
30. Filho, W.L.; Raath, S.; Lazzarini, B.; Vargas, V.R.; de Souza, L.; Anholon, R.; Quelhas, O.L.G.; Haddad, R.; Klavins, M.; Orlovic, V.L. The role of transformation in learning and education for sustainability. J. Clean. Prod; 2018; 199, pp. 286-295. [DOI: https://dx.doi.org/10.1016/j.jclepro.2018.07.017]
31. Willats, J.; Erlandsson, L.; Molthan-Hill, P.; Dharmasasmita, A.; Simmons, E. A university wide approach to embedding the sustainable development goals in the curriculum—A case study from the Nottingham Trent University’s Green Academy. Implementing Sustainability in the Curriculum of Universities; Springer: Cham, Switzerland, 2018; pp. 63-78.
32. Hays, J. Chaos to Capability: Educating Professionals for the 21st Century; Unitec ePress Monograph Series (1); Unitec Institute of Technology: Auckland, New Zealand, 2015; Available online: https://hdl.handle.net/10652/3053 (accessed on 28 March 2022).
33. Azevedo, R. Defining and measuring engagement and learning in science: Conceptual, theoretical, methodological, and analytical issues. Educ. Psychol.; 2015; 50, pp. 84-94. [DOI: https://dx.doi.org/10.1080/00461520.2015.1004069]
34. Ben-Eliyahu, A.; Linnenbrink-Garcia, L. Integrating the regulation of affect, behavior, and cognition into self-regulated learning paradigms among secondary and post-secondary students. Metacogn. Learn.; 2015; 10, pp. 15-42. [DOI: https://dx.doi.org/10.1007/s11409-014-9129-8]
35. Hettiarachchi, S.; Damayanthi, B.W.R.; Heenkenda, S.; Dissanayake, D.M.S.L.B.; Ranagalage, M.; Ananda, L. Student Satisfaction with Online Learning during the COVID-19 Pandemic: A Study at State Universities in Sri Lanka. Sustainability; 2021; 13, 11749. [DOI: https://dx.doi.org/10.3390/su132111749]
36. Li, S.; Wang, Y.; Xue, J.; Zhao, N.; Zhu, T. The impact of COVID-10 epidemic declaration on psychological consequences: A study on active weibo users. Int. J. Environ. Res. Public Health; 2020; 17, 2032. [DOI: https://dx.doi.org/10.3390/ijerph17062032]
37. Šakan, D.; Žuljević, D.; Rokvić, N. The role of basic psychological needs in well-being during the COVID-19 outbreak: A self-determination theory perspective. Front. Public Health; 2020; 8, 713. [DOI: https://dx.doi.org/10.3389/fpubh.2020.583181]
38. Carrillo, C.; Flores, M.A. COVID-19 and teacher education: A literature review of online teaching and learning practices. Eur. J. Teach. Educ.; 2020; 43, pp. 466-487. [DOI: https://dx.doi.org/10.1080/02619768.2020.1821184]
39. Bishop, J.; Verleger, M.A. The flipped classroom: A survey of the research. Proceedings of the 2013 ASEE Annual Conference & Exposition; Atlanta, GA, USA, 23–26 June 2013; pp. 23-1200.
40. Abeysekera, L.; Dawson, P. Motivation and cognitive load in the flipped classroom: Definition, rationale and a call for research. High. Educ. Res. Dev.; 2015; 34, pp. 1-14. [DOI: https://dx.doi.org/10.1080/07294360.2014.934336]
41. Mahmud, S.N.D.; Husnin, H.; Tuan Soh, T.M. Teaching presence in online gamified education for sustainability learning. Sustainability; 2020; 12, 3801. [DOI: https://dx.doi.org/10.3390/su12093801]
42. Galindo-Dominguez, H. Flipped Classroom in the Educational System. J. Educ. Technol. Soc.; 2021; 24, pp. 44-60.
43. Hsia, L.H.; Lin, Y.N.; Hwang, G.J. A creative problem solving-based flipped learning strategy for promoting students’ performing creativity, skills and tendencies of creative thinking and collaboration. Br. J. Educ. Technol.; 2021; 52, pp. 1771-1787. [DOI: https://dx.doi.org/10.1111/bjet.13073]
44. Cao, W.; Fang, Z.; Hou, G.; Han, M.; Xu, X.; Dong, J.; Zheng, J. The psychological impact of the COVID-19 epidemic on college students in China. Psychiatr. Res.; 2020; 287, 112934. [DOI: https://dx.doi.org/10.1016/j.psychres.2020.112934] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/32229390]
45. Hernández, D.J.; Ortiz, J.J.G.; Abellán, M.T. Metodologías Activas en la Universidad y su relación con los Enfoques de Enseñanza. Profr. Rev. De Currículum Y Form. Del Profr.; 2020; 24, pp. 76-94. [DOI: https://dx.doi.org/10.30827/profesorado.v24i1.8173]
46. Xiong, J.; Yan, J.; Fu, K.; Wang, K.; He, Y. Innovation in an authoritarian society: China during the pandemic crisis. J. Bus. Strategy; 2021; 43, pp. 79-86. [DOI: https://dx.doi.org/10.1108/JBS-10-2020-0223]
47. Zainuddin, Z. Students’ learning performance and perceived motivation in gamified flipped-class instruction. Comput. Educ.; 2018; 126, pp. 75-88. [DOI: https://dx.doi.org/10.1016/j.compedu.2018.07.003]
48. Ahshan, R. A framework of implementing strategies for active student engagement in remote/online teaching and learning during the COVID-19 pandemic. Educ. Sci.; 2021; 11, 483. [DOI: https://dx.doi.org/10.3390/educsci11090483]
49. Heo, H.; Bonk, C.J.; Doo, M.Y. Enhancing learning engagement during COVID-19 pandemic: Self-efficacy in time management, technology use, and online learning environments. J. Comput. Assist. Learn.; 2021; 37, pp. 1640-1652. [DOI: https://dx.doi.org/10.1111/jcal.12603]
50. Kuhfeld, M.; Soland, J.; Tarasawa, B.; Johnson, A.; Ruzek, E.; Liu, J. Projecting the potential impact of COVID-19 school closures on academic achievement. Educ. Res.; 2020; 49, pp. 549-565. [DOI: https://dx.doi.org/10.3102/0013189X20965918]
51. Luria, E.; Shalom, M.; Levy, D.A. Cognitive Neuroscience Perspectives on Motivation and Learning: Revisiting Self-Determination Theory. Mind Brain Educ.; 2021; 15, pp. 5-17. [DOI: https://dx.doi.org/10.1111/mbe.12275]
52. Ryan, R.M. Psychological needs and the facilitation of integrative processes. J. Pers.; 1995; 63, pp. 397-427. [DOI: https://dx.doi.org/10.1111/j.1467-6494.1995.tb00501.x] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/7562360]
53. Silvia, P.J. Interest—The curious emotion. Curr. Dir. Psychol. Sci.; 2008; 17, pp. 57-60. [DOI: https://dx.doi.org/10.1111/j.1467-8721.2008.00548.x]
54. Ashari, H.; Abbas, I.; Abdul-Talib, A.N.; Zamani, S.N.M. Entrepreneurship and Sustainable Development Goals: A Multigroup Analysis of the Moderating Effects of Entrepreneurship Education on Entrepreneurial Intention. Sustainability; 2021; 14, 431. [DOI: https://dx.doi.org/10.3390/su14010431]
55. Niemiec, C.P.; Ryan, R.M. Autonomy, competence, and relatedness in the classroom: Applying self-determination theory to educational practice. Theory Res. Educ.; 2009; 7, pp. 133-144. [DOI: https://dx.doi.org/10.1177/1477878509104318]
56. Ryan, R.M.; Deci, E.L. Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. Am. Psychol.; 2000; 55, pp. 68-78. [DOI: https://dx.doi.org/10.1037/0003-066X.55.1.68]
57. Connell, J.P.; Wellborn, J.G. Competence, autonomy, and relatedness. A Motivational Analysis of Self-System Processes; Gunnar, M.R.; Sroufe, L.A. Lawrence Erlbaum Associates, Inc.: Hillsdale, MI, USA, 1991; pp. 43-77.
58. Newmann, F.M. Student Engagement and Achievement in American Secondary Schools; Teachers College Press: New York, NY, USA, 1992.
59. Sergis, S.; Sampson, D.G.; Pelliccione, L. Investigating the impact of Flipped Classroom on students’ learning experiences: A Self-Determination Theory approach. Comput. Hum. Behav.; 2018; 78, pp. 368-378. [DOI: https://dx.doi.org/10.1016/j.chb.2017.08.011]
60. Lo, C.K.; Hew, K.F. A comparison of flipped learning with gamification, traditional learning, and online independent study: The effects on students’ mathematics achievement and cognitive engagement. Interact. Learn. Environ.; 2020; 28, pp. 464-481. [DOI: https://dx.doi.org/10.1080/10494820.2018.1541910]
61. Pelikan, E.R.; Korlat, S.; Reiter, J.; Holzer, J.; Mayerhofer, M.; Schober, B.; Spiel, C.; Hamzallari, O.; Uka, A.; Chen, J. et al. Distance learning in higher education during COVID-19: The role of basic psychological needs and intrinsic motivation for persistence and procrastination—A multi-country study. PLoS ONE; 2021; 16, e0257346. [DOI: https://dx.doi.org/10.1371/journal.pone.0257346]
62. Chiu, T.K. Applying the self-determination theory (SDT) to explain student engagement in online learning during the COVID-19 pandemic. J. Res. Technol. Educ.; 2021; 54, (Suppl. 1), pp. S14-S30. [DOI: https://dx.doi.org/10.1080/15391523.2021.1891998]
63. Bayoumy, H.M.M.; Alsayed, S. Investigating relationship of perceived learning engagement, motivation, and academic performance among nursing students: A multisite study. Adv. Med. Educ. Pract.; 2021; 12, 351. [DOI: https://dx.doi.org/10.2147/AMEP.S272745] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/33907486]
64. Panigrahi, R.; Srivastava, P.R.; Panigrahi, P.K. Effectiveness of e-learning: The mediating role of student engagement on perceived learning effectiveness. Inf. Technol. People; 2020; 34, pp. 1840-1862. [DOI: https://dx.doi.org/10.1108/ITP-07-2019-0380]
65. Lozano-Jiménez, J.E.; Huéscar, E.; Moreno-Murcia, J.A. Effects of an autonomy support intervention on the involvement of higher education students. Sustainability; 2021; 13, 5006. [DOI: https://dx.doi.org/10.3390/su13095006]
66. Orazbayeva, B.; van der Sijde, P.; Baaken, T. Autonomy, competence and relatedness–the facilitators of academic engagement in education-driven university-business cooperation. Stud. High. Educ.; 2021; 46, pp. 1406-1420. [DOI: https://dx.doi.org/10.1080/03075079.2019.1679764]
67. Deterding, S.; Sicart, M.; Nacke, L.; O’Hara, K.; Dixon, D. Gamification. Using game-design elements in non-gaming contexts. CHI’11 Extended Abstracts on Human Factors in Computing Systems; Association for Computing Machinery: New York, NY, USA, 2011; pp. 2425-2428.
68. Park, S.; Kim, S. Is sustainable online learning possible with gamification?—The effect of gamified online learning on student learning. Sustainability; 2021; 13, 4267. [DOI: https://dx.doi.org/10.3390/su13084267]
69. Ekici, M. A systematic review of the use of gamification in flipped learning. Educ. Inf. Technol.; 2021; 26, pp. 3327-3346. [DOI: https://dx.doi.org/10.1007/s10639-020-10394-y]
70. Hammill, J.; Nguyen, T.; Henderson, F. Encouraging the flip with a gamified process. Int. J. Educ. Res.; 2021; 2, 100085. [DOI: https://dx.doi.org/10.1016/j.ijedro.2021.100085]
71. Langendahl, P.A.; Cook, M.; Mark-Herbert, C. Gamification in Higher Education: Toward a Pedagogy to Engage and Motivate; Department of Economics, Swedish University of Agricultural Sciences: Uppsala, Sweden, 2016.
72. Deci, E.L.; Ryan, R.M. Optimizing students’ motivation in the era of testing and pressure: A self-determination theory perspective. Building Autonomous Learners; Springer: Singapore, 2016; pp. 9-29.
73. Rosenbaum, M.S.; Russell-Bennett, R.; Contreras-Ramírez, G. Business education in profound disruption. J. Serv. Mark.; 2021; 35, pp. 553-558. [DOI: https://dx.doi.org/10.1108/JSM-08-2021-528]
74. Chang, D.; Hwang, G.J.; Chang, S.C.; Wang, S.Y. Promoting students’ cross-disciplinary performance and higher order thinking: A peer assessment-facilitated STEM approach in a mathematics course. Educ. Technol. Res. Dev.; 2021; 69, pp. 3281-3306. [DOI: https://dx.doi.org/10.1007/s11423-021-10062-z]
75. Kressler, B.; Kressler, J. Diverse student perceptions of active learning in a large enrollment STEM course. J. Scholarsh. Teach. Learn.; 2020; 20, pp. 40-64. [DOI: https://dx.doi.org/10.14434/josotl.v20i1.24688]
76. Jusas, V.; Barisas, D.; Jančiukas, M. Game Elements towards More Sustainable Learning in Object-Oriented Programming Course. Sustainability; 2022; 14, 2325. [DOI: https://dx.doi.org/10.3390/su14042325]
77. Sailer, M.; Hense, J.U.; Mayr, S.K.; Mandl, H. How gamification motivates: An experimental study of the effects of specific game design elements on psychological need satisfaction. Comput. Hum. Behav.; 2017; 69, pp. 371-380. [DOI: https://dx.doi.org/10.1016/j.chb.2016.12.033]
78. Rincon-Flores, E.G.; Santos-Guevara, B.N. Gamification during COVID-19: Promoting active learning and motivation in higher education. Australas. J. Educ. Technol.; 2021; 37, pp. 43-60. [DOI: https://dx.doi.org/10.14742/ajet.7157]
79. Santos-Villalba, M.J.; Olivencia, J.J.L.; Navas-Parejo, M.R.; Benítez-Márquez, M.D. Higher Education Students’ Assessments towards Gamification and Sustainability: A Case Study. Sustainability; 2020; 12, 8513. [DOI: https://dx.doi.org/10.3390/su12208513]
80. Ahmed, H.D.; Asiksoy, G. The Effects of Gamified Flipped Learning Method on Student’s Innovation Skills, Self-Efficacy towards Virtual Physics Lab Course and Perceptions. Sustainability; 2021; 13, 10163. [DOI: https://dx.doi.org/10.3390/su131810163]
81. Bredow, C.A.; Roehling, P.V.; Knorp, A.J.; Sweet, A.M. To flip or not to flip? A meta-analysis of the efficacy of flipped learning in higher education. Rev. Educ. Res.; 2021; 91, pp. 878-918. [DOI: https://dx.doi.org/10.3102/00346543211019122]
82. Creswell, J.W.; Klassen, A.C.; Clark, V.L.P.; Smith, K.C. Best practices for mixed methods research in the health sciences. Bethesda Natl. Inst. Health; 2013; 12, pp. 541-545.
83. Ho, C.M.; Yeh, C.C.; Wang, J.Y.; Hu, R.H.; Lee, P.H. Pre-class online video learning and class style expectation: Patterns, association, and precision medical education. Ann. Med.; 2021; 53, pp. 1390-1401. [DOI: https://dx.doi.org/10.1080/07853890.2021.1967441]
84. Furdu, I.; Tomozei, C.; Kose, U. Pros and Cons Gamification and Gaming in Classroom.: Discovery Service for University of South Carolina. BRAIN Broad Res. Artif. Intell. Neurosci.; 2017; 8, pp. 56-62.
85. Mazarakis, A. Using Gamification for Technology Enhanced Learning: The Case of Feedback Mechanisms. Bull. Tech. Comm. Learn. Technol.; 2015; 17, pp. 6-9.
86. Hamari, J. Do badges increase user activity? A field experiment on the effects of gamification. Comput. Hum. Behav.; 2017; 71, pp. 469-478. [DOI: https://dx.doi.org/10.1016/j.chb.2015.03.036]
87. Burguillo, J.C. Using game theory and competition-based learning to stimulate student motivation and performance. Comput. Educ.; 2010; 55, pp. 566-575. [DOI: https://dx.doi.org/10.1016/j.compedu.2010.02.018]
88. Okura, M.; Carfì, D. Coopetition and game theory. J. Appl. Econ. Sci.; 2014; 9, 29.
89. Seaborn, K.; Fels, D.I. Gamification in theory and action: A survey. Int. J. Hum. Comput. Stud.; 2015; 74, pp. 14-31. [DOI: https://dx.doi.org/10.1016/j.ijhcs.2014.09.006]
90. Skinner, E.; Furrer, C.; Marchand, G.; Kindermann, T. Engagement and disaffection in the classroom: Part of a larger motivational dynamic?. J. Educ. Psychol.; 2008; 100, 765. [DOI: https://dx.doi.org/10.1037/a0012840]
91. Rotgans, J.I.; Schmidt, H.G. Cognitive engagement in the problem-based learning classroom. Adv. Health Sci. Educ. Theory Pract.; 2011; 16, pp. 465-479. [DOI: https://dx.doi.org/10.1007/s10459-011-9272-9]
92. Al-Zahrani, A.M. From passive to active: The impact of the flipped classroom through social learning platforms on higher education students’ creative thinking. Br. J. Educ. Technol.; 2015; 46, pp. 1133-1148. [DOI: https://dx.doi.org/10.1111/bjet.12353]
93. Chi, M.T.; Wylie, R. The ICAP framework: Linking cognitive engagement to active learning outcomes. Educ. Psychol.; 2014; 49, pp. 219-243. [DOI: https://dx.doi.org/10.1080/00461520.2014.965823]
94. Hew, K.F.; Huang, B.; Chu, K.W.S.; Chiu, D.K. Engaging Asian students through game mechanics: Findings from two experiment studies. Comput. Educ.; 2016; 92, pp. 221-236. [DOI: https://dx.doi.org/10.1016/j.compedu.2015.10.010]
95. Merriam, S.B.; Grenier, R.S. Qualitative Research in Practice: Examples for Discussion and Analysis; John Wiley & Sons: San Francisco, CA, USA, 2019.
96. Fredricks, J.A.; Blumenfeld, P.C.; Paris, A.H. School engagement: Potential of the concept, state of the evidence. Rev. Educ. Res.; 2004; 74, pp. 59-109. [DOI: https://dx.doi.org/10.3102/00346543074001059]
97. McCallum, S.; Schultz, J.; Sellke, K.; Spartz, J. An examination of the flipped classroom approach on college student academic involvement. Int. J. Learn. High.; 2015; 27, pp. 42-55.
98. Field, A. Discovering Statistics Using SPSS; 3rd ed. Sage Publications Ltd.: London, UK, 2009.
99. Weisstein, E.W. Bonferroni Correction. 2004; Available online: https://mathworld.wolfram.com/BonferroniCorrection.html (accessed on 28 March 2022).
100. Tomczak, M.; Tomczak, E. The need to report effect size estimates revisited. An overview of some recommended measures of effect size. Trends Sport Sci.; 2014; 1, pp. 19-25.
101. Johnson, R.B. Examining the validity structure of qualitative research. Education; 1997; 118, pp. 282-292.
102. Glesne, C. Becoming Qualitative Researchers: An Introduction; Pearson: Upper Saddle River, NJ, USA, 2016.
103. Saldana, J. Fundamentals of Qualitative Research; Oxford University Press: Oxford, UK, 2011.
104. Maxwell, J.A. Qualitative Research Design: An Interactive Approach; Sage Publications: Newbury Park, CA, USA, 2012.
105. Flick, U. Designing Qualitative Research; Sage Publications: Newbury Park, CA, USA, 2018.
106. Li, B.Z.; Cao, N.W.; Ren, C.X.; Chu, X.J.; Zhou, H.Y.; Guo, B. Flipped classroom improves nursing students’ theoretical learning in China: A meta-analysis. PLoS ONE; 2020; 15, e0237926. [DOI: https://dx.doi.org/10.1371/journal.pone.0237926] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/32853214]
107. Kormos, J.; Csizer, K. The interaction of motivation, self-regulatory strategies, and autonomous learning behavior in different learner groups. Tesol Q.; 2014; 48, pp. 275-299. [DOI: https://dx.doi.org/10.1002/tesq.129]
108. Hajarian, M.; Diaz, P. Effective Gamification: A Guideline for Gamification Workshop of WEEF-GEDC 2021 Madrid Conference. Proceedings of the 2021 World Engineering Education Forum/Global Engineering Deans Council (WEEF/GEDC); Madrid, Spain, 15–18 November 2021; pp. 506-510.
109. Adams, A.E.; Garcia, J.; Traustadóttir, T. A quasi experiment to determine the effectiveness of a “partially flipped” versus “fully flipped” undergraduate class in genetics and evolution. CBE Life Sci. Educ.; 2016; 15, 11. [DOI: https://dx.doi.org/10.1187/cbe.15-07-0157]
110. McLean, S.; Attardi, S.M.; Faden, L.; Goldszmidt, M. Flipped classrooms and student learning: Not just surface gains. Adv. Physiol. Educ.; 2016; 40, pp. 47-55. [DOI: https://dx.doi.org/10.1152/advan.00098.2015]
111. Smallhorn, M. The flipped classroom: A learning model to increase student engagement not academic achievement. Stud. Success; 2017; 8, pp. 43-53. [DOI: https://dx.doi.org/10.5204/ssj.v8i2.381]
112. Loh, C.Y.R.; Teo, T.C. Understanding Asian students learning styles, cultural influence and learning strategies. J. Educ. Soc. Pol.; 2017; 7, pp. 194-210.
113. Van Sickle, J.R. Discrepancies between student perception and achievement of learning outcomes in a flipped classroom. J. Scholarsh. Teach. Learn.; 2016; 16, pp. 29-38. [DOI: https://dx.doi.org/10.14434/josotl.v16i2.19216]
114. Zheng, L.; Bhagat, K.K.; Zhen, Y.; Zhang, X. The effectiveness of the flipped classroom on students’ learning achievement and learning motivation. J. Educ. Technol. Soc.; 2020; 23, pp. 1-15.
115. Zhao, H. Investigating learners’ use and understanding of peer and teacher feedback on writing: A comparative study in a Chinese English writing classroom. Assess. Writ.; 2010; 15, pp. 3-17. [DOI: https://dx.doi.org/10.1016/j.asw.2010.01.002]
116. Hew, K.F. Student perceptions of peer versus instructor facilitation of asynchronous online discussions: Further findings from three cases. Instr. Sci.; 2015; 43, 1938. [DOI: https://dx.doi.org/10.1007/s11251-014-9329-2]
117. Roorda, D.L.; Zee, M.; Koomen, H.M. Don’t forget student-teacher dependency! A Meta-analysis on associations with students’ school adjustment and the moderating role of student and teacher characteristics. Attach Hum. Dev.; 2021; 23, pp. 490-503. [DOI: https://dx.doi.org/10.1080/14616734.2020.1751987]
118. Vaziri, S.; Vaziri, B.; Novoa, L.J.; Torabi, E. Academic Motivation in Introductory Business Analytics Courses: A Bayesian Approach. INFORMS Trans. Educ.; 2022; 22, pp. 121-129. [DOI: https://dx.doi.org/10.1287/ited.2021.0247]
119. Long, T.; Logan, J.; Waugh, M. Students’ perceptions of the value of using videos as a pre-class learning experience in the flipped classroom. TechTrends; 2016; 60, pp. 245-252. [DOI: https://dx.doi.org/10.1007/s11528-016-0045-4]
120. Lo, C.K.; Hew, K.F. The impact of flipped classrooms on student achievement in engineering education: A meta-analysis of 10 years of research. J. Eng. Educ.; 2019; 108, pp. 523-546. [DOI: https://dx.doi.org/10.1002/jee.20293]
121. Lestari, I.W. Flipped classroom in Indonesian higher education: A mixed-method study on students’ attitudes and experiences. Stud. Engl. Lang. Educ.; 2021; 8, pp. 243-257. [DOI: https://dx.doi.org/10.24815/siele.v8i1.17636]
122. Maciejewski, W. Flipping the calculus classroom: An evaluative study. Teach. Math. Appl.; 2016; 35, pp. 187-201. [DOI: https://dx.doi.org/10.1093/teamat/hrv019]
123. Karra, S.; Karampa, V.; Paraskeva, F. Gamification design framework based on self determination theory for adult motivation. Proceedings of International Workshop on Learning Technology for Education in Cloud; Springer: Cham, Switzerland, 2019; pp. 67-78.
124. Jensen, J.L.; Holt, E.A.; Sowards, J.B.; Ogden, T.H.; West, R.E. Investigating strategies for pre-class content learning in a flipped classroom. J. Sci. Educ. Technol.; 2018; 27, pp. 523-535. [DOI: https://dx.doi.org/10.1007/s10956-018-9740-6]
125. Yorganci, S. Implementing flipped learning approach based on ‘first principles of instruction’ in mathematics courses. J. Comput. Assist. Learn.; 2020; 36, pp. 763-779. [DOI: https://dx.doi.org/10.1111/jcal.12448]
126. Magana, A.J.; Vieira, C.; Boutin, M. Characterizing engineering learners’ preferences for active and passive learning methods. IEEE Trans. Educ.; 2017; 61, pp. 46-54. [DOI: https://dx.doi.org/10.1109/TE.2017.2740203]
127. Arribathi, A.H.; Suwarto,; Rosyad, A.M.; Budiarto, M.; Supriyanti, D.; Mulyati.,. An analysis of student learning anxiety during the COVID-19 Pandemic: A Study in Higher Education. J. Contin. High. Educ.; 2021; 69, pp. 192-205. [DOI: https://dx.doi.org/10.1080/07377363.2020.1847971]
128. Liu, J.; Liu, M.; Liang, W. The Dynamic COVID-Zero Strategy in China. China CDC Wkly.; 2022; 4, pp. 74-75. [DOI: https://dx.doi.org/10.46234/ccdcw2022.015]
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
Details
