Content area
Multiple studies have investigated how serious games can foster learning in the context of education. However, findings on the impact of developing such serious games as an instructional approach are still limited, especially in education for sustainable development. Thus, this paper presents the Serious Game Development as an Instructional Approach (SGDIA) framework based on the ADDIE concept. Subsequently, a proof-of-concept study is applied to evaluate the impact of the course concept on learning. In an experimental pre-post study design at two German universities (n = 48), the effects on socio-emotional (values, self-efficacy) and cognitive (understanding, critical reflection) learning objectives relevant to education for sustainable development have been tested. The findings show that self-efficacy and pro-environmental values increased significantly after students participated in the serious game design course over three months. Additionally, in 23 out of the 48 participants, students who sought understanding during the three months of the teaching setting were more likely to critically reflect. As low-level methods of creating game content, such as simple game engines using visual programming, are becoming more widespread, the paper demonstrates how creating serious games can be a valid instructional method. Based on the findings, the potential of developing serious games as an instructional approach in higher education is discussed.
Introduction
In the current era of technology-enhanced learning and teaching, teachers are asked to use novel approaches to engage learners in an instructional setting that can facilitate affective and cognitive learning objectives. Prior research findings have already indicated that integrating educational technology into teaching settings can positively affect learning outcomes compared to teaching settings without such media (e.g., Hillmayr et al., 2020). In this context, digital serious games can be employed as a practical demonstration, as they are recognized for their ability to positively affect cognitive, affective, and behavioral learning objectives among a wide variety of disciplines (Wouters & van Oostendorp, 2017; Wouters et al., 2013). As one of their main affordances, serious games can foster a specific learning objective in addition to providing entertainment value. For instance, in the field of education for sustainable development, serious games can effectively address learning objectives in the context of climate change issues (Janakiraman et al., 2018; Katsaliaki & Mustafee, 2015; Medema et al., 2019; Wu & Lee, 2015), such as environmentally conscious behavior (Morganti et al., 2017), environmental awareness (Nunes et al., 2016), or climate change-related knowledge (Khoury et al., 2018; Spangenberger et al., 2021).
Despite the affordances of serious games for learning, there is still a low number of empirical findings that examine game development itself as an instructional approach to engage students in a learning process (Cravero et al., 2021; Kayali et al., 2015; Qian & Clark, 2016; Shute et al., 2012). Moreover, the theoretical alignment of such approaches is still limited. Overlaps with serious game development as an instructional approach can be found in the concept of problem-based learning and in the learners-as-designers approach. Problem-based learning refers to a collaborative learning setting in which students work with each other to solve a complex problem or answer a question (Crespí et al., 2022). This enables students to engage actively in their learning process, which can foster interpersonal skills and competencies such as teamwork or communication (Crespi et al., 2022). However, serious game development differs from problem-based learning in the way that serious game design focuses on the use of game-design principles to enhance learning for other learners, which is also pointed out in the learners-as-designers approach by Jonassen and Reeves (Jonassen & Reeves, 1996). By highlighting the possibility of perspective-taking within this approach, it has been shown to foster interactive learning behavior (Damnik et al., 2017). As game development-based learning focuses on decision-making with respect to specific game mechanics (Wu & Wang, 2012), the extent to which the development of a digital serious game by students within higher education is an effective method for enabling affective or cognitive learning processes has hardly been investigated so far (Cravero et al., 2021), and its theoretical alignment is still limited.
Therefore, in this paper, a concept of serious game development as an instructional approach (SGDIA) will be outlined, and results from a proof-of-concept study in the field will be presented. The SGDIA framework emphasizes game development-based learning, which is examined as a distinct pedagogical method whose theoretical foundations still require further investigation (Qian & Clark, 2016; Wu & Wang, 2012). The ADDIE concept by Branch (2009) is used as a starting point for the instructional concept. The use of this concept will be investigated in the field of higher education, and will involve students in the development of a serious game targeting a sustainable development goal. Based on our results, we will contribute to the question of how creating serious games can be a valid instructional method.
Serious game development-based learning
Wu and Wang (2012) define game development-based learning (GDBL) as a learning approach in which students are encouraged to make design decisions in order to develop their own or modify existing games based on a chosen game development framework. The core idea behind game development-based learning is to leverage the inherent motivation and engagement that game design offers to enhance learning outcomes. Wu and Wang further state that the fundamental concept of game development-based learning can be explained by constructionism learning theory (see also Vos et al., 2011). Mental construction and the creation of meaningful artifacts are claimed as the main learning activities in game development-based learning. Prior studies have demonstrated the positive effects of designing entertainment games in a teaching setting for learning outcomes in the subjects of mathematics, history, democracy education, biodiversity, or inclusiveness (e.g., Dishon & Kafai, 2020; Hacıoğlu & Dönmez Usta, 2020; Ke, 2014; Spieler & Slany, 2018; Vos et al., 2011; Wake et al., 2018). However, it has also been stressed that the theoretical foundation of game development-based learning and its empirical evidence still requires further investigation (Qian & Clark, 2016; Wu & Wang, 2012).
While game development-based learning refers mainly to entertainment games, serious game development refers to games that combine both game elements and learning content to reach a particular learning objective in education. Hence, serious game development is a very complex process because several competencies are needed to consider both the instructional content and game characteristics at the same time. As pointed out by Plass et al. (2015), “…the design process of games for learning involves balancing the need to cover the subject matter with the desire to prioritize game play” (p. 259). Therefore, the main difference when designing a serious game compared to an entertainment game is the focus on a specific learning objective to which the entire game development process should be aligned. Moreover, a meaningful embedding of serious games in teaching settings, such as a debriefing, is necessary to successfully promote the learning objectives over the long term (Garris et al., 2002).
Applied as an instructional method, in such a serious game development process students can collaboratively create an educational game for fellow learners by taking on the role of designers, also in line with the learners-as-designers approach put forth by Jonassen and Reeves (1996). This perspective-taking approach has been shown to foster interactive learning behavior (Damnik et al., 2017). Learners designing a serious game for fellow learners as an instructional approach has already gained attention in the context of education for sustainable development (ESD) in higher education (e.g., Cravero et al., 2021). During a university challenge, students were asked to develop serious games around a sustainability-related theme. Besides likability and teamwork competency, the authors also assessed self-reported knowledge about sustainability. Findings revealed that developing a game was positively associated with increased sustainability-related knowledge (Cravero et al., 2021). However, further empirical findings on the extent to which serious game development is an effective tool to foster both socio-emotional and cognitive learning processes are still scarce, and the theoretical alignment of such approaches is still missing.
Instructional concept of serious game development
Besides building on game development-based learning (Wu & Wang, 2012) and the learners-as-designers approach by Jonassen and Reeves (1996), taking serious game development as an instructional approach into a classroom setting requires guidance in the form of a systems approach, which can be found in the ADDIE concept from Branch (2009). In the context of instructional design, the ADDIE concept has been established to provide a product-oriented concept of performance-oriented learning. In a five-step cycle, Branch describes five phases that should be “…applied to instructional design in order to generate episodes of intentional learning” (Branch, 2009, p. 17). His approach is based on Gagne et al.'s (2005) principles of instructional design, which refer to internal information processing. The ADDIE concept provides educators with a lesson plan in terms of “planned activities [that] focus on guiding the students” (p. 3) to foster their construction of knowledge and skills. The letters A, D, D, I, and E each stand for a phase: Analyze, Design, Develop, Implement, and Evaluate. Phase 1 (Analyze) aims at identifying the learners’ performance gap. Phase 2 (Design) aims at completing objectives and selecting testing methods. In Phase 3 (Develop) the selection and/or development of instructional media takes place. Phase 4 (Implement) consists of planning and preparing the facilitation of the instructional strategies. Phase 5 (Evaluate) refers to the evaluation of learners’ perception, learning and transfer, and the selection of measurements, respectively. The ADDIE concept enables guided, self-regulated learning and provides a process to generate intentional learning by establishing planned learning activities (Branch, 2009). Branch (2009) emphasizes that the “goal-oriented nature of intentional learning promotes self-regulation [sic] in learners” with “an opportunity for a student to be reflective” (p.7). The author also encourages renaming each phase of the ADDIE sequence to create one's own ADDIE processes for learning.
Subsequently, in the introduced Serious Game Development Instructional Approach (SGDIA), the ADDIE concept by Branch (2009) is applied as a structure that enables learners to develop a serious game during planned learning activities. In guided but self-organized collaborative group work, the learners have to solve the question of how other learners can learn by playing a serious game. To master this overall goal, self-regulation, reflection, and perspective-taking are initiated. These aspects align with education for sustainable development, where self-regulated learning settings, critical reflection, and perspective-taking are the key cognitive learning objectives discussed to enable a shift in mindset (Bianchi et al., 2022; Kitchenham, 2008; Taylor, 2007).
Besides delivering the foundations for a systems approach that focuses on the principles of instructional design, the ADDIE concept is a product-oriented concept that can be applied to the task of serious game development, because the phases have overlaps with serious game development work packages. However, compared to the ADDIE concept, the five phases of the SGDIA serve as guidelines for learners’ performance in self-organized group work rather than as guidelines exclusively for the educators. Instead, the instructional approach to serious game development adapts the five steps of the ADDIE concept for creating serious games as part of a course concept as follows: (1) Target Audience, (2) Design Instructional Content, (3) Prototype, (4) Test, (5) Document (see Fig. 1). The SGDIA framework integrates game development-based learning and learners-as-designers as a pedagogical activity which can be grounded in constructionism learning theory (Bower, 2017; Wu & Wang, 2012): “The important issue is that the design and implementation of products are meaningful to those creating them and that learning becomes active and self-directed through the construction of artifacts” (Wu & Wang, 2012 page 7; see also Kafai, 1995). Lecturers must not only possess the content knowledge, but also create an appropriate learning environment that enables this goal to be achieved (Bower, 2017), as also stated in the so-called TPACK model by Mishra and Koehler (2006). Technological (TK), pedagogical (PK), and content (CK) knowledge have to be integrated to enable effective technology-enhanced learning (Bower, 2017).
[See PDF for image]
Fig. 1
A framework for Serious Game Development as an Instructional Approach (SGDIA), based on Branch (2009)
The instructional concept of the SGDIA results in five main tasks assigned to students, which they have to solve in a self-organized group setting. The lecturer takes on the role of a facilitator and introduces each phase with lead questions and tasks that have to be performed by the students (see Table 1), which are not exclusive, but serve as an initial starting point and can be enriched. The phases and student tasks described in Table 1 below are informed by the structure of the ADDIE concept (Branch, 2009), but have been adapted to a learner-centered approach to serious game development as an instructional method. In this study, we present a proof-of-concept where we elaborate how serious game development fosters learning objectives in the context of ESD.
Table 1. Lead questions and task performance of the serious game development phases, adapted from Branch (2009)
Phase | Lead questions | Task performance |
|---|---|---|
Analyze the target group | Who should learn what? What do you know about the target group? | Analyze the needs of the target group Analyze prior knowledge (content, technology) |
Decide on instructional design | What is to be learned with the serious game (behavior, knowledge, attitudes)? Which learning content has to be considered? What is the game about? What does the player have to achieve in the game? Which game mechanics are chosen (quests, narrative, interactions) in line with learning objectives? | Define learning objectives, learning content, and learning outcomes Align competencies and tasks with the curriculum, if necessary Decide on game mechanics (difficulty level, interactions, objects, tasks, sound, feedback, etc.) Align game mechanics with learning objectives |
Prototype | What do you want to prototype (decision-making, narrative, tasks, usability, graphics)? Which free game design software suits your serious game idea? | Technical prototyping Technical adjustments |
Test | How will you embed and align the serious game into a teaching unit? How will you set up an initial test of your game? Which design steps can be tested? | Instructional embedding of the serious game Presentation in front of peers or in the target group Test for usability, flow, or learning success |
Document | What were the main steps of the serious game development? What should you document? Which references will you use? How will you adapt the design steps in line with perceived feedback from peers or target group? | Documentation of the development process in form of a poster or presentation Further subjective or psychological assessments of the target group |
In the first phase (target audience), students are expected to form project groups and choose their target group, and the lecturer provides the theoretical background on the overall learning content, learning psychology, and game design techniques to be considered. Based on this information, the students have to choose and analyze the target group, for example, pertaining to related needs, habits, or social background, as well as gather information on their prior knowledge of the content and the technology used. During this phase, the lecturer supports the learner as an expert, responding to questions that arise.
In the second phase (design instructional content), students are required to build on this knowledge and start brainstorming their serious game concept. They decide on the learning objectives and learning content, learning tasks, and related learning theory. They also develop their first game idea and decide on a game goal, select features, and game mechanics such as quests, plot, and narrative. The lecturer again serves as an expert, in the sense of facilitating the design process, providing feedback, and answering questions. Students are required to shift their perspective to that of their target group and describe the behavior, characteristics, and typical aspects of their game’s target group to align their game concept with the target group’s needs.
In the third phase (prototype), students create the initial quest of their serious game concept using free digital game design software, such as Twinery, Construct, or GDevelop. The lecturer introduces the different game design software that students can use. The students can choose which part of the game they prefer to prototype and which software they are going to use. This decision will also be informed by their prior knowledge regarding developer software, game development, programming, graphic design, or similar areas. The objective of the prototyping activity can be understood primarily as an active learning exercise (Camburn et al., 2017), aiming at testing specific features of the students’ serious game concepts. The resulting prototype should therefore be regarded as the initial step in an iterative design process. Any subsequent refinement or further development beyond the seminar remains at the discretion of the students and depends on their individual motivation and commitment.
In the fourth phase (test), students test and present their game concept to their peers, or within the chosen target group. Testing can be voluntary for the first four steps, but is mandatory for testing the prototype. The final peer feedback session is planned and moderated by the lecturer. Additional professional feedback is also provided by the lecturer.
The fifth phase (document) is the result of all phases of the serious game development process. Students should evaluate the results of the prototype testing in order to incorporate the feedback into their final design concept in the form of a poster or presentation that will be submitted as a course assignment. The documentation contains the decision-making process as well as a description of the serious game and its learning objective.
Proof-of-concept study
To test and validate the instructional approach of serious game development in higher education, a proof-of-concept study was conducted. Proof-of-concept research refers to research that evaluates a concept, product or theory using a smaller sample size to test, validate or explore its feasibility in practice (Jobin et al., 2020; Kendig, 2016). Hence, the instructional concept of the SGDIA was applied in an authentic learning setting in education for sustainable development (ESD). Students were asked to develop a serious game about one of the 17 sustainable development goals (SDGs; United Nations, 2015) following each step of the SGDIA framework. The learning objectives of the course were chosen in line with this specific field of education. In research on the relevant competencies of ESD, it has been discussed that learners should be enabled to reflect on their role in society within project-oriented learning settings (Rieckmann, 2018; Singer-Brodowski et al., 2022). In such learning settings, learners gain practical experiences (e.g., project work), reflect on their experiences, and apply them to their beliefs or lifestyles. Further competencies such as problem-solving, critical thinking, and reflective thinking are addressed as being essential for mastering the social, economic, political, and ecological changes within the transition processes (Rieckmann, 2018; Rodríguez Aboytes & Barth, 2020; Singer-Brodowski et al., 2022). In this debate, it has also been emphasized that cognitive processes alone are not sufficient to initiate action. As Brundiers et al. (2021) point out, most of the key competencies discussed in the context of ESD are “thinking” competencies (p.24). Taking up this demand, the Green Competency Framework contains affective and cognitive competencies which should be addressed in the context of learning for environmental sustainability (Bianchi et al., 2022). At the same time, there is limited evidence on effective instructional methods that foster such competencies, as the operationalization and measurement of learning objectives are still in the beginning stages in ESD-related research (Fischer et al., 2022).
Consequently, the SGDIA framework is a project-oriented learning setting because it introduces a serious game development-based learning activity aimed at socio-emotional (self-efficacy, values) and cognitive learning processes (comprehension, critical reflection) as relevant in ESD.
The concept of self-efficacy has mainly been influenced by Bandura (1977). In his understanding, self-efficacy refers to the idea that individuals feel confident in mastering a specific task or situation. Promoting sustainability transitions requires individuals to believe in their ability to contribute to sustainable action (Bianchi et al., 2022). In conclusion, self-efficacy regarding one's belief in one´s abilities to implement knowledge about sustainable development into everyday actions serves as an affective construct to assess the learning outcome in the context of education for sustainable development. Thus, it was assumed that participating in a teaching setting focusing on serious game development about a sustainable development goal can lead to changes in self-efficacy over time, which led to our first research question:
Research question #1: Does developing serious games in higher education foster changes in self-efficacy?
Prior studies have observed that proactive climate action positively correlates with personal values (Dias et al., 2020). A re-evaluation of one’s beliefs and values can also trigger socio-emotional processes, eventually inducing changes in beliefs or values (Pang et al., 2023; Wiley et al., 2021). Furthermore, as pointed out by De Groot and Steg (2009), people with so-called altruistic or biospheric values are more likely to act pro-environmentally. Hence, changes in pro-environmental values can serve as an additional approach to evaluate the learning outcomes in alignment with education for sustainable development (Bianchi et al., 2022). In this light, participating in the serious game development course might initiate a change in students’ values over time, a conclusion that prompted the second research question.
Research question #2: Does developing serious games in higher education foster changes in pro-environmental values?
Furthermore, the combination of cognitive and socio-emotional processes is needed to induce a change in mindset and eventually pro-environmental behavior (Wiek et al., 2011; Williamson et al., 2018). In the context of education for sustainability, critical reflection is considered one of the most important cognitive learning objectives to enable a shift in mindset and is associated with behavioral changes (Bianchi et al., 2022; Kitchenham, 2008; Taylor, 2007). Reflection often occurs when individuals question their belief patterns while exploring a matter of concern. To initiate such reflection, one can engage learners in an experiment focused on a behavioral change, such as a reduction of meat consumption or plastic use (Leichenko et al., 2022). To measure critical reflection, Kember et al. (2000) developed the Critical Reflection Questionnaire. The questionnaire assesses to what extent students engage in reflective thinking within a classroom setting and is based on reflective thinking theory as well as the idea of transformative learning by Mezirow (Melacarne, 2019; Mezirow, 1997). Kember et al. (2000) identified four different levels of reflective thinking: understanding, habits, reflection, and critical reflection. The authors observed that understanding in terms of comprehending learning content is positively linked to critical reflection. Especially “unusual cases” provide the opportunity for critical reflection. Moreover, providing authentic experiences in education has been identified as a crucial factor in learning to reflect (Boyd & Fales, 1983; Kember et al., 2000). Hence, we stress that critical reflection and understanding of learning content can serve as appropriate cognitive constructs to evaluate learning processes in line with education for sustainable development. Students engage in a project-oriented learning setting that requires perspective-taking as a designer. By making a serious game with a sustainability goal for fellow learners, students have to understand the values and behavior of their game’s target group and empathize with them. Based on the learners-as-designers approach, this process fosters a more profound understanding of the learning content (Damnik et al., 2017). Furthermore, Kember et al. (2000) observed that understanding and critical reflection are positively linked with one another, a finding that informed the third, fourth, fifth, and sixth research questions:
Research question #3: Does developing serious games in higher education facilitate critical reflection?
Research question #4: Does developing serious games in higher education facilitate understanding?
Research question #5: Are critical reflection and understanding associated with changes in self-efficacy?
Research question #6: Are critical reflection and understanding associated with changes in pro-environmental values?
Methods
As described above, the study was conducted to assess the impact of a serious game development-based course on students’ socio-emotional constructs and cognitive learning processes. The experiment focused on the socio-emotional domain, investigating whether self-efficacy and pro-environmental values were influenced by the course. An assessment of cognitive learning processes (understanding and critical reflection) was also included.
Materials
The course concept was based on the SGDIA framework applied to a teaching course in the field of higher education in Germany (Spangenberger et al., 2022). The overall goal of the teaching setting implemented in the course was that students could develop a concept for a serious game addressing a sustainable development goal, create a prototype of their idea, test it within their peer group, and document it in a scientific poster. An example was conducted as follows: One group decided to develop a serious game called Metropolis for kids aged 8 to 10. In the game, players have to rescue a village called Metropolis that is threatened by drought. In line with SDG 6 (Clean Water and Sanitation), players should learn about efficient water utilization. The main game objective is reached when 100 villagers have been successfully supplied with water. A pedagogical agent in the form of a waterdrop called Hydropi helps the players master the tasks. As a free online tool to prototype the task, the group used Twinery (Twine, 2024). The results were documented in a scientific poster and/or a prototype, describing each step of the development and images of the prototype (for examples, see Spangenberger et al., 2022). In another group, the students developed a serious game concept targeting SDG 5 (see Fig. 2). In this game, the topic of domestic violence was addressed. Based on Walker’s cycle of violence (2009), the players have to identify violent relationships and learn about strategies for dealing with them.
[See PDF for image]
Fig. 2
Students’ poster example, with the kind permission of the students (Bellina L., Lozar, M., Kurteva, N.). A serious game concept aimed at learning objectives based on SDG 5
Sample
Our sample consisted of 48 students from two different German universities. The course was conducted over three semesters from 2021 to 2022 with three different groups. Two groups from the same university participated in the course during two different semesters (Group 1 and Group 2, n = 26), while another group was from a different university (Group 3, n = 22). The courses were part of the elective courses compulsory for all students, irrespective of their discipline. Students’ disciplines in Groups 1 and 2 included Geography, Information Sciences, Theater Sciences, Psychology, and History. Students in Group 3 studied Media Design or Information Technology. All students were undergraduates. The students were between 18 and 46 years old (M = 23.00; SD = 5.09; 53.99% female). The two groups that were from the same university were combined into one group to attain groups of comparable size for the subsequent analyses. There were no significant differences between the two groups that were combined.
Procedure
The proof-of-concept was implemented in a pre-post design. Students participating in the course completed a pre-questionnaire in the first session of the course and a post-questionnaire in the last session of the course three months later. The pre-questionnaire included questions related to self-efficacy, pro-environmental values, and socio-demographic items (age, gender). The post-questionnaire included questions related to self-efficacy, pro-environmental values, critical reflection, and understanding. The study was conducted in line with the ethical principles established in the Declaration of Helsinki (2013). Participants were informed about anonymity, the voluntary nature of the study, and that a refusal to participate in the study would have no negative consequences for them.
Measurements
To evaluate self-efficacy in terms of students’ self-reported confidence regarding the application of their knowledge about the sustainable development goals, three self-developed items were included in the pre- and post-questionnaire (e.g., “I am very familiar with the Sustainable Development Goals (SDGs),” “I don´t know what the Sustainable Development Goals are,” and “I can apply my knowledge about sustainability in my everyday life”). Participants rated their agreement with these statements on a Likert scale from 1 (I totally disagree) to 5 (I totally agree). Scales were implemented in German. For the scale reliability, item 2 was reversed and McDonald’s omega was computed, showing a good level of internal consistency reliability in the pre-test and a moderate internal reliability in the post-test (ωpre = .691; ωpost = .474).
To measure pro-environmental values, the Green Scale on pro-environmental consumer values by Haws et al. (2014) was used and assessed before and after the course. The scale originally consisted of six items to be rated with a Likert scale from 1 (I totally disagree) to 5 (I totally agree). It was reduced to three items, because the course did not directly inform students about sustainable consumption (e.g., “It is important to me that the products I use do not harm the environment,” “My purchase habits are affected by my concern for our environment,” and “I am willing to be inconvenienced in order to take actions that are more environmentally friendly”). Instead, items on more general values related to environmental concerns were assessed (e.g., “I consider the potential environmental impact of my actions when making many of my decisions,” “I am concerned about wasting the resources of our planet,” and “I would describe myself as environmentally responsible”). The scale was translated into German using an online translation software (DeepL), including the back-translation, and additionally checked by a native-speaking proofreader. Internal consistency reliability of the scale was assessed pre and post with McDonald’s omega (ωpre = .652; ωpost = .799).
To measure levels of understanding and critical reflection, items on understanding and critical reflection from the reflective thinking questionnaire developed by Kember et al. (2000) were used and included in the post-questionnaire. The reflective thinking questionnaire by Kember et al. (2000) originally contains 16 items that measure four different levels of reflective thinking: habitual action (e.g., “When I am working on some activities, I can do them without thinking about what I am doing”), understanding (e.g., “To pass this course I need to understand the content”), reflection (e.g., “I sometimes question the way others do something and try to think of a better way”), and critical reflection (e.g., “This course has challenged some of my firmly held ideas”). Items on understanding and critical reflection directly relate to the course content, whereas habits and reflection inquire about the general frequency of engagement in these activities, and were excluded for not being the focus of our study. The scale was translated into German using an online translation software (DeepL), including the back-translation, and additionally checked by a native-speaking proofreader. The scale's internal consistency reliability was measured using McDonald’s omega (ωunderstanding = .888 to ωcriticalreflection = .907).
A full list of all items used can be found in Appendix A.
Data and statistical analysis
To analyze changes between pre- and post-test scores of self-efficacy and pro-environmental values (RQ1 and RQ2) as well as between the two groups of participants, a mixed analysis of variance (ANOVA) was conducted. The data were not normally distributed in our sample (n = 48). However, the mixed ANOVA has been shown to be robust against violations of the normal distribution assumption (e.g., Schmider et al., 2010). To analyze whether game designing fosters critical reflection (RQ3) and understanding (RQ4), descriptive results were reported. To examine the association of critical reflection and understanding with changes in self-efficacy (RQ5) and pro-environmental values (RQ6), linear regression analysis was conducted because the change scores for self-efficacy and pro-environmental values were found to be normally distributed. To compute the different analyses, the software R Version 4.3.1, RStudio 2024.09.1, and R package afex and Jamovi (The Jamovi Project, 2022) were used.
Results
Self-efficacy (RQ1): The mixed ANOVA showed a significant main effect for the measurement occasion (F(1, 46) = 183.56, p < 0.001, η2 = 0.636). The post-hoc tests with Holm correction were significant for all comparisons. Group 1 showed significantly higher levels of self-efficacy at post-test as compared to the pre-test (Mdiff = -1.36, SE = 0.16, t(46) = -8.56, p < .001). A similar result was found for Group 3 (Mdiff = -1.82, SE = 0.17, t(46) = -10.53, p < .001). A main effect for the group also emerged (F(1, 46) = 18.08, p < 0.001, η2 = 0.181). There was a significant difference between the two subject groups at pre-test and at post-test. Students in Groups 1 and 2 (combined into one group) whose subjects included Geography, Information Sciences, Theater Sciences, Psychology, and History showed higher levels of self-efficacy before (Mdiff = 0.34, SE = 0.22, t(46) = 3.56, p = 0.002) and after the course (Mdiff = 0.79, SE = 0.11, t(46) = 2.95, p = 0.005) than students in Group 3, who studied Media Design or Information Technology. There was no significant interaction effect (F(1, 46) = 3.83, p = 0.056, η2 = 0.035). The respective mean scores can be found in Table 2. Figure 3 illustrates the various differences.
Table 2. Descriptive statistics for the main variables and results (n = 48)
Group 1 (n = 26) | Group 3 (n = 22) | |||||
|---|---|---|---|---|---|---|
M (SD) pre | M (SD) post | M (SD) pre | M (SD) post | M (SD) pre | M (SD) post | |
Self-efficacy | 2.76 (0.86) | 4.33 (0.38) | 3.13 (0.72) | 4.49 (0.39) | 2.33 (0.83) | 4.15 (0.39) |
Pro-environmental values | 3.81 (0.61) | 4.17 (0.62) | 3.82 (0.67) | 4.26 (0.64) | 3.79 (0.61) | 4.08 (0.60) |
M = Mean, SD = Standard Deviation, Group 1 comprises Subject Groups 1 and 2
[See PDF for image]
Fig. 3
Results of the mixed ANOVA for self-efficacy (SelfIndex) from pre-test (X1) to post-test (X2) and between the two subject groups (Groups 1 and 3)
Pro-environmental values (RQ2): The mixed ANOVA for this research question yielded a significant main effect for the measurement occasion (F(1, 46) = 22.25, p < 0.001, η2 = 0.078). The post-hoc tests showed a significant increase of pro-environmental values from pre-test to post-test for Group 1 (Mdiff = -0.44, SE = 0.10, t(46) = -4.20, p < .001). The difference between pre- and post-test for Group 3 marginally missed significance (Mdiff = -0.29, SE = 0.11, t(46) = -2.55, p = .071). The descriptive statistics can be found in Table 2 and the change across time is visualized in Fig. 4. The analysis did not find a significant main effect for the group (F(1, 46) = 0.41, p = 0.525, η2 = 0.007) nor a significant interaction effect (F(1, 46) = 0.93, p = 0.340, η2 = 0.004).
[See PDF for image]
Fig. 4
Results of the mixed ANOVA for pro-environmental values (GreenIndex) from pre-test (X1) to post-test (X2) and between the two subject groups (Groups 1 and 3)
Understanding and critical reflection (RQ3 and 4): Due to a technical error, only data from 23 students from one university were assessed (Group 3). On average, after the seminar 23 students reported that understanding the course content was highly relevant to passing the course (M = 3.8, SD = 0.79, n = 23). The level of critical reflection exhibited neither a high nor a low magnitude (M = 2.77, SD = 1.11, n = 23). There was a positive correlation between the level of seeking to understand the content during the seminar and critical reflection on the seminar (p = .021; Pearson’s r = 0.478), indicating that students who sought understanding were more likely to reflect critically.
Association of critical reflection and understanding with changes in self-efficacy (RQ 5): A multiple linear regression was conducted and the age of the participants was included as a control variable. The overall model was statistically significant, F(3, 18) = 10.29, p = .0004, explaining 63.2% (R2 = .63) of the variance in the change scores for self-efficacy. Critical reflection was a significant positive predictor (β = 0.59, t(18) = 4.72, p < .001), indicating that higher levels of critical reflection were associated with more positive change in self-efficacy. Age was a significant negative predictor (β = − 0.10, t(18) = − 2.85, p = .011), suggesting that older participants reported less change. However, understanding was not a significant predictor (β = − 0.07, t(18) = − 0.38, p = .712).
Association of critical reflection and understanding with changes in pro-environmental values (RQ 6): A multiple linear regression was conducted and the age of the participants was included as a control variable. The overall model was not statistically significant, F(3, 18) = 0.13, p = .940, explaining only 2.2% (R2 = .02) of the variance in the change scores for pro-environmental values. None of the predictors were statistically significant. These results suggest that none of the variables included in the model significantly predict changes in pro-environmental values for the group of Media Design and Information Technology students.
Discussion and limitations
As a primary observation of the proof-of-concept study conducted, the results indicate that serious game development as a teaching concept could foster changes in self-efficacy. After participating in the course, the students believed more strongly in their abilities to implement knowledge about sustainable development into everyday actions (RQ1), and reported a higher level of confidence in applying their knowledge to their everyday lives. Besides fostering cognitive learning (Cravero et al., 2021), our results suggest that serious game development–based learning could also be applied to foster socio-emotional learning around a sustainability-related theme. This is in line with the findings of Crespi et al. (2022), indicating that problem-based learning in a collaborative learning setting can actively engage students in the learning process and foster the development of interpersonal skills. This result was still valid even though one group differed from the other group in their prior self-reported self-efficacy: While two groups already had a higher level of self-confidence in their ability with regard to sustainability before the seminar compared to the other group, an increase was observed in both groups. As the two courses of Group 1 were part of a sustainability module and one course (Group 3) was not, this could explain the difference in perceived self-efficacy. However, regardless of this distinction, self-efficacy increased in both groups. This indicates that the SGDIA may contribute to key learning outcomes of education for sustainable development, identified by Rodríguez Aboytes and Barth (2020): an increase in self-awareness, agency, and empowerment in terms of an increase in self-confidence regarding the ability to apply knowledge in everyday life.
The debate on further constructs representing socio-emotional processes in education for sustainable development led to the assessment of pro-environmental values. In the experiment, pro-environmental values were higher after students participated in the course (RQ2). As stated above, participation in a teaching setting that integrates serious game development for sustainability-related learning content may have a significant impact on socio-emotional processes. Especially the three-month duration required a longer engagement with the learning content compared to a short-term intervention. However, long-term outcomes still need to be investigated, and future studies should evaluate whether changes remain over time.
The results also indicate that integrating serious game development into a teaching setting may be a promising approach to foster critical reflection (RQ3), being one relevant learning process in the context of ESD (Kitchenham, 2008; Bianchi et al., 2022). Following the SGDIA framework seems to open up the opportunity for students to be reflective. Moreover, as already observed by Kember et al. (2000), understanding and critical reflection were positively linked with one another. Students from the experiment who sought to understand the learning content were more likely to critically reflect (RQ4). This result is in line with the learners-as-designers approach, where learners design learning materials for their peers, a process that has been shown to promote a more profound understanding of the learning content (Damnik et al., 2017). Although our results are not generalizable due to the small number of participants, they provide an initial indication that understanding and critical reflection can potentially be addressed using the SGDIA framework, where learners develop a serious game concept for other learners around a sustainability related-theme. In our data, critical reflection was a significant positive predictor for changes in self-efficacy (RQ5). As also stressed by Pang et al. (2023) and Wiley et al. (2021), a re-evaluation of one’s beliefs can trigger socio-emotional processes, eventually inducing a change in values. This leads to the assumption that critical reflection may be a relevant influential factor for enhancing self-efficacy in the context of ESD, which should be further investigated. Age was also a positive predictor for changes in self-efficacy in our study. Younger students seemed to be more strongly influenced by the seminar compared to their older mates. However, due to the small variance in age, these results should be interpreted rather carefully. Larger studies on age differences might reveal more insight regarding the relationship between age and changes in self-efficacy. When looking for the influence of critical reflection or understanding on pro-environmental attitudes, our observations did not indicate any differences (RQ6). In sum, applying the SGDIA framework in the field of higher education may have an impact on cognitive and socio-emotional learning processes: values, self-efficacy (socio-emotional dimension), understanding, and critical reflection (cognitive dimension). To assess these constructs, we opted for valid and reliable instruments such as the Questionnaire of Reflective Thinking (Kember et al., 2000) and the GREEN scale (Haws et al., 2014), as well as the self-developed items to assess changes in self-efficacy.
In the future, further experimental studies should be applied to different disciplines to validate these findings. Moreover, a teaching setting employing serious game development might be more motivating and more acceptable for students compared to more conventional forms of teaching in higher education. This needs to be examined in future research, for instance, by addressing academic-related motivation.
As a limitation, we did not control for personal events in students’ lives or habits that might have influenced environmental concerns within or outside of the course, since the time period of the courses was about three months. Although creating game content was the core element of the course structure, other pedagogical course components, such as working in groups or receiving input on topics of sustainability, might have influenced the measures. Moreover, the conclusion that the course as a whole was effective can only be tentative, as there was no separate control group for comparison. Further long-term research is necessary to confirm the findings and to demonstrate that the development process itself explains the increase in the measured constructs. Using self-report measures was adequate to assess constructs related to internal processes. However, the influence of potentially confounding variables such as social desirability and individual-level “greenwashing” was not controlled for.
To conclude, it can be stressed that engaging students in a collaborative serious game development activity can serve as an instructional concept: involving students in a direct expressive experience that helps them to reflect upon this experience. Designing and developing a serious game is a very specific task that requires a learner-centered approach, mental construction, and creating meaningful artifacts which can be linked to constructionism learning theory (e.g., Qian & Clark, 2016; Wu & Wang, 2012). In order to develop a serious game with a sustainable development goal for fellow learners, students have to understand and define the learning objectives that are to be achieved by the game’s target group. To achieve this goal, they have to analyze the needs, values, and habits of the target group and apply this knowledge when designing quests or choosing other game features (e.g., narrative, genre, characters). As a result, students will have developed a prototype for a game that aims at a change in knowledge, emotions or behavior in fellow learners. By providing such a learning environment in which students develop tools, models or experiences on their own, quantifiable learning outcomes can be promoted such as cognitive and affective constructs. Moreover, in addition to mere factual knowledge gains, the changes in self-efficacy or beliefs around applying the newly gained insights might offer one building block in the evolution of education, preparing students for substantial challenges of the future, such as climate change.
Implications
The study has theoretical implications, as it presents a theory-informed concept of serious game development as an instructional approach. Adapted from the ADDIE concept by Branch (2009), and based on the learners-as-designers approach by Jonassen and Reeves (1996), the SGDIA approach provides systematic guidance when using a creative game-based instructional method to engage learners in a learning setting. Each phase of the SGDIA approach contains self-organized lead questions to perform specific tasks relevant for serious game development. In these phases, the principles of game development-based learning and the foundations of game-based learning (Plass et al., 2015) have been considered. Our results indicate that the ADDIE concept may serve as a beneficial foundation for creating learning phases that can be transparently provided to students. In doing so, students may be able to reflect on each learning phase to understand its purpose and actively engage with the content. Future research can further investigate its impact on learning processes in different disciplines such as teacher education, or in educational stages such as secondary or primary education. Moreover, by addressing learning objectives relevant for ESD, we have also helped respond to the call for more research on the operationalization and measurement of learning objectives in the context of ESD, which is still in its beginning stages (see Fischer et al., 2022).
As for practical implications, the approach developed here indicates how learners’ socio-emotional and cognitive learning processes may be facilitated by conducting a proof-of-concept study in the field of higher education. This initial study provides a first indication that the concept could possibly be effectively applied in practice and could guide students through serious game development. The SGDIA approach can be transferred to various serious game technologies, such as Virtual Reality or Augmented Reality.
Conclusion
In this paper, the instructional approach of serious game design was outlined within the SGDIA framework and tested in a small proof-of-concept study. In the experiment, an instructional course concept involving serious game development by students was applied in the field of higher education. By creating a serious game about a sustainable development goal for fellow learners, students were engaged in a project-oriented teaching setting. As the main results of the present study, significant effects on changes in the socio-emotional domain (self-efficacy, values) and the cognitive domain (understanding, reflection) were observed. In addition, Understanding the learning content about sustainable development goals was positively linked to critical reflection, pro-environmental values, and self-reported abilities and increased over the three-month duration of the seminar.
To conclude, the proof-of-concept study indicates that the creation of serious games may be a valid instructional method in the context of ESD and beyond (Qian & Clark, 2016). This is not self-evident, as low-level methods of creating game content, such as simple game engines using visual programming (e.g., Scratch), are becoming more widespread. Thus, games can catch up with other types of media (e.g., text or video) where not only reception but also production is a common educational practice.
Acknowledgements
We would like to thank the students (Bellina L., Lozar M., Kurteva N.) for generously making their poster available to us.
Author contributions
P.S. designed the theoretical framework, and wrote the first draft of the manuscript. P.S. and L.K. carried out the experiment. P.S., G.M.S., and A.S.-K. performed the statistical analysis. A.S.-K., L.K., V.L. and L.K.P. contributed to the writing of the manuscript. S.N. substantially aided in revising the original draft of the manuscript. All authors read and approved the final manuscript.
Funding
This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.
Data availability
The datasets generated and analyzed during the current study are not publicly available due to strict data safety and confidentiality protocols mandated by the participating universities, but are available from the corresponding author on reasonable request.
Declarations
Competing interests
The authors declare no competing interests or personal relationships that could have appeared to influence the work reported in this paper.
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
References
Bandura, A. Self-efficacy: Toward a unifying theory of behavioral change. Psychological Review; 1977; 84,
Bianchi, G., Pisiotis, U., & Cabrera, M. (2022). In Y. Punie, & M. Bacigalupo (Eds.), GreenComp: The European sustainability competence framework. Publications Office of the European Union. https://doi.org/10.2760/13286
Boyd, EM; Fales, AW. Reflective learning: Key to learning from experience. Journal of Humanistic Psychology; 1983; 23,
Branch, RM. Instructional design: The ADDIE approach; 2009; Springer: [DOI: https://dx.doi.org/10.1007/978-0-387-09506-6]
Bower, M. Design of technology-enhanced learning: Integrating research and practice; 2017; Emerald Publishing Limited: [DOI: https://dx.doi.org/10.1108/9781787141827]
Brundiers, K; Barth, M; Cebrián, G; Cohen, M; Diaz, L; Doucette-Remington, S; Dripps, W; Habron, G; Harré, N; Jarchow, M; Losch, K; Michel, J; Mochizuki, Y; Rieckmann, M; Parnell, R; Walker, P; Zint, M. Key competencies in sustainability in higher education—Toward an agreed-upon reference framework. Sustainability Science; 2021; 16,
Camburn, B; Viswanathan, V; Linsey, J; Anderson, D; Jensen, D; Crawford, R; Otto, K; Wood, K. Design prototyping methods: State of the art in strategies, techniques, and guidelines. Design Science; 2017; 3, e13. [DOI: https://dx.doi.org/10.1017/dsj.2017.10]
Cravero, S., Strada, F., Lami, I., & Bottino, A. (2021). Learning sustainability by making games. The experience of a challenge as a novel approach for Education for Sustainable Development. In 7th International Conference on Higher Education Advances (HEAd’21). https://doi.org/10.4995/HEAd21.2021.13192
Crespí, P; García-Ramos, JM; Queiruga-Dios, M. Project-based learning (PBL) and its impact on the development of interpersonal competences in higher education. Journal of New Approaches in Educational Research; 2022; 11,
Damnik, G; Proske, A; Körndle, H. Designing a constructive learning activity with interactive elements: The effects of perspective-shifting and the quality of source material. Interactive Learning Environments; 2017; 25,
De Groot, JIM; Steg, L. Mean or green: Which values can promote stable pro-environmental behavior?. Conservation Letters; 2009; 2,
Dias, NMOC; Vidal, DG; Sousa, HFPe; Dinis, MAP; Leite, Â. Exploring associations between attitudes towards climate change and motivational human values. Climate; 2020; 8,
Dishon, G; Kafai, YB. Making more of games: Cultivating perspective-taking through game design. Computers & Education; 2020; 148, 103810. [DOI: https://dx.doi.org/10.1016/j.compedu.2020.103810]
Fischer, D; King, J; Rieckmann, M; Barth, M; Büssing, A; Hemmer, I; Lindau-Bank, D. Teacher education for sustainable development: A review of an emerging research field. Journal of Teacher Education; 2022; 73,
Gagne, RM; Wager, WW; Golas, KC; Keller, JM; Russell, JD. Principles of instructional design, 5th edition. Performance Improvement; 2005; 44,
Garris, R; Ahlers, R; Driskell, JE. Games, motivation, and learning: A research and practice model. Simulation & Gaming; 2002; 33,
Hacıoğlu, Y; Dönmez Usta, N. Digital game design-based STEM activity: Biodiversity example. Science Activities; 2020; 57,
Haws, KL; Winterich, KP; Naylor, RW. Seeing the world through GREEN-tinted glasses: Green consumption values and responses to environmentally friendly products. Journal of Consumer Psychology; 2014; 24,
Hillmayr, D; Ziernwald, L; Reinhold, F; Hofer, SI; Reiss, KM. The potential of digital tools to enhance mathematics and science learning in secondary schools: A context-specific meta-analysis. Computers & Education; 2020; 153, 103897. [DOI: https://dx.doi.org/10.1016/j.compedu.2020.103897]
Janakiraman, S; Watson, SL; Watson, WR. Using game-based learning to facilitate attitude change for environmental sustainability. Journal of Education for Sustainable Development; 2018; 12,
Jobin, C., Hooge, S., & Le Masson, P. (2020). What does the proof-of-concept (POC) really prove? A historical perspective and a cross-domain analytical study. In XXIXème Conférence de l’Association Internationale de Management Stratégique (AIMS). https://hal.science/hal-02570321v2
Jonassen, DH; Reeves, TC. Jonassen, DH. Learning with technology: Using computers a cognitive tools. Handbook of research for educational communication and technology; 1996; Simon and Schuster Macmillan: pp. 693-719.
Kafai, Yasmin B. Minds in play; 1995; Routledge: [DOI: https://dx.doi.org/10.4324/9780203052914]
Katsaliaki, K; Mustafee, N. Edutainment for sustainable development. Simulation & Gaming; 2015; 46,
Kayali, F; Peters, K; Kuczwara, J; Reithofer, A; Martinek, D; Wölfle, R; Mateus-Berr, R; Lehner, Z; Silbernagl, M; Sprung, M. Participatory game design for the INTERACCT serious game for health; 2015; Springer: pp. 13-25.
Ke, F. An implementation of design-based learning through creating educational computer games: A case study on mathematics learning during design and computing. Computers & Education; 2014; 73, pp. 26-39. [DOI: https://dx.doi.org/10.1016/j.compedu.2013.12.010]
Kember, D; Leung, DYP; Jones, A; Loke, AY; McKay, J; Sinclair, K; Tse, H; Webb, C; Yuet Wong, FK; Wong, M; Yeung, E. Development of a questionnaire to measure the level of reflective thinking. Assessment & Evaluation in Higher Education; 2000; 25,
Kendig, CE. What is proof of concept research and how does it generate epistemic and ethical categories for future scientific practice?. Science and Engineering Ethics; 2016; 22,
Khoury, M; Gibson, MJ; Savic, D; Chen, AS; Vamvakeridou-Lyroudia, L; Langford, H; Wigley, S. A serious game designed to explore and understand the complexities of flood mitigation options in urban-rural catchments. Water; 2018; 10,
Kitchenham, A. The evolution of John Mezirow’s transformative learning theory. Journal of Transformative Education; 2008; 6,
Leichenko, R; Gram-Hanssen, I; O’Brien, K. Teaching the “how” of transformation. Sustainability Science; 2022; 17,
Medema, W; Mayer, I; Adamowski, J; Wals, AEJ; Chew, C. The potential of serious games to solve water problems: Editorial to the special issue on game-based approaches to sustainable water governance. Water; 2019; 11,
Melacarne, C. The theory and practice of evaluating transformative learning processes. European perspectives on transformation theory; 2019; Palgrave Macmillan: pp. 193-206. [DOI: https://dx.doi.org/10.1007/978-3-030-19159-7_13]
Mezirow, J. Transformative learning: Theory to practice. New Directions for Adult and Continuing Education; 1997; 1997,
Mishra, P; Koehler, MJ. Technological pedagogical content knowledge: A framework for teacher knowledge. Teachers College Record; 2006; 108,
Morganti, L; Pallavicini, F; Cadel, E; Candelieri, A; Archetti, F; Mantovani, F. Gaming for earth: Serious games and gamification to engage consumers in pro-environmental behaviours for energy efficiency. Energy Research & Social Science; 2017; 29, pp. 95-102. [DOI: https://dx.doi.org/10.1016/j.erss.2017.05.001]
Nunes, E. P., Luz, A. R., Lemos, E. M., Maciel, C., dos Anjos, A. M., Borges, L. C., & Nunes, C. (2016). Mobile serious game proposal for environmental awareness of children. In 2016 IEEE Frontiers in Education Conference. The crossroads of engineering and business (pp. 1–8). IEEE. https://doi.org/10.1109/FIE.2016.7757353
Pang, G; Cox, RC; Acheson, K. Assessing transformative learning in international education: Critiques and new directions based on a systematic review of literature. New Directions for Adult and Continuing Education; 2023; 2023,
Plass, JL; Homer, BD; Kinzer, CK. Foundations of game-based learning. Educational Psychologist; 2015; 50,
Qian, M; Clark, KR. Game-based learning and 21st century skills: A review of recent research. Computers in Human Behavior; 2016; 63, pp. 50-58. [DOI: https://dx.doi.org/10.1016/j.chb.2016.05.023]
Rieckmann, M. Leicht, A; Heiss, J; Byun, WJ. Learning to transform the world: Key competencies in ESD. Issues and trends in education for sustainable development; 2018; United Nations Educational: pp. 39-59.
Rodríguez Aboytes, JG; Barth, M. Transformative learning in the field of sustainability: A systematic literature review (1999–2019). International Journal of Sustainability in Higher Education; 2020; 21,
Schmider, E; Ziegler, M; Danay, E; Beyer, L; Bühner, M. Is it really robust?. Methodology; 2010; 6,
Shute, V; Ventura, M; Torres, R. Formative evaluation of students at Quest to Learn. International Journal of Learning and Media; 2012; 4,
Singer-Brodowski, M; Förster, R; Eschenbacher, S; Biberhofer, P; Getzin, S. Facing crises of unsustainability: Creating and holding safe enough spaces for transformative learning in higher education for sustainable development. Frontiers in Education; 2022; [DOI: https://dx.doi.org/10.3389/feduc.2022.787490]
Spangenberger, P; Matthes, N; Kruse, L; Draeger, I; Narciss, S; Kapp, F. Experiences with a serious game introducing basic knowledge about renewable energy technologies: A practical implementation in a German Secondary School. Journal of Education for Sustainable Development; 2021; 14,
Spangenberger, Pia; Kruse, Linda; Singer-Brodowski, Mandy. Weselek, Johanna; Kohler, Florian; Siegmund, Alexander. Transformatives Lernen mit digitalen Spielen. Entwicklung eines Serious Game durch Studierende als didaktisches Konzept für eine BNE?. Digitale Bildung für nachhaltige Entwicklung; 2022; Berlin, Heidelberg, Springer: pp. 99-110. [DOI: https://dx.doi.org/10.1007/978-3-662-65122-3_8]
Spieler, B., & Slany, W. (2018). Game development-based learning experience: gender differences in game design. In M. Ciussi (Ed.), Proceedings of the 12th European Conference on Games Based Learning (ECGBL 2017) (pp. 616–625). Academic Conference and Publishing International Limited.
Taylor, EW. An update of transformative learning theory: A critical review of the empirical research (1999–2005). International Journal of Lifelong Education; 2007; 26,
The Jamovi Project. (2022). Jamovi (Version 2.3) [Computer software]. https://www.jamovi.org
Twine. (2024). [Computer software]. Interactive Fiction Technology Foundation. https://twinery.org
United Nations. (2015). The 17 Goals | Sustainable Development. In Sustainable Development. https://sdgs.un.org/goals
Vos, N; Van Der Meijden, H; Denessen, E. Effects of constructing versus playing an educational game on student motivation and deep learning strategy use. Computers & Education; 2011; 56,
Wake, JD; Guribye, F; Wasson, B. Learning through collaborative design of location-based games. International Journal of Computer-Supported Collaborative Learning; 2018; 13,
Walker, LEA. The battered woman syndrome; 2009; 3 Springer:
Wiek, A; Withycombe, L; Redman, CL. Key competencies in sustainability: A reference framework for academic program development. Sustainability Science; 2011; 6,
Wiley, JL; Wiley, KR; Intolubbe-Chmil, L; Bhuyan, D; Acheson, K. A new, depth-based quantitative approach to assessing transformative learning. Journal of Transformative Education; 2021; 19,
Williamson, K., Satre-Meloy, A., Velasco, K., & Green, K. (2018). Climate Change Needs Behavior Change: Making the Case for Behavioral Solutions to Reduce Global Warming.
Wouters, P; van Nimwegen, C; van Oostendorp, H; van der Spek, ED. A meta-analysis of the cognitive and motivational effects of serious games. Journal of Educational Psychology; 2013; 105,
Wouters, P; van Oostendorp, H. Techniques to improve the effectiveness of serious games; 2017; Springer:
Wu, B; Wang, AI. A guideline for game development-based learning: A literature review. International Journal of Computer Games Technology; 2012; 2012, pp. 1-20. [DOI: https://dx.doi.org/10.1155/2012/103710]
Wu, JS; Lee, JJ. Climate change games as tools for education and engagement. Nature Climate Change; 2015; 5,
© The Author(s) 2025. This work is published under http://creativecommons.org/licenses/by/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.