Enhancing Student Motivation and Engagement through a Gamified Learning Environment

Abstract

Gamification is a viable strategy used to enhance motivation and engagement in programming classes among students. However, automated evaluation capabilities, which are crucial for giving students fast and correct feedback, are frequently lacking in currently available gamification technologies. This study proposes a novel web-based application that combines automated programming assessment features with gamification concepts; the aim is to provide students taking a programming course with an engaging learning environment. A survey conducted with 215 undergraduate students assessed how the tool affected the motivation and engagement of students. The findings show that the tool had a beneficial impact on students’ willingness to participate in class, study, increase their self-confidence, engage in healthy competition with peers, and learn from their mistakes. The qualitative feedback that students offered regarding the features of the tool that they liked best is also covered in the study. This paper contributes to the field of programming education by presenting a detailed gamified tool, incorporating automated evaluation and gamification in a web-based platform.

1. Introduction

Games can serve as very powerful tools for improving learning processes, from three distinct yet complementary perspectives: as tools for teaching content or skills, as an object of the learning project itself, and as a philosophy to be taken into account when designing the training process [1]. These tools, also known as gamification, have been instrumental in promoting the interests and motivations of students. In addition, student performances in the course have improved, and engagement with online learning platforms has increased. Gamification in educational systems has consistently generated profound interest among educators and researchers. It has been demonstrated that gamification has an extremely positive effect on learning [2]. Other studies have shown that the incentives and rewards offered by gamification have a theoretical basis in stimulating self-efficacy, which can either enhance or impede motivation [3].

The state of the art of web-based gamification tools has been extensively studied over the past few years. Various approaches, such as game mechanics, rewards, and leaderboards, have been used to successfully implement gamification. In addition, these tools have been found to be beneficial in educational and health-related contexts [4].

This related work focuses on the effectiveness of these tools in promoting engagement and motivation in important use cases, such as programming courses or other engineering subjects, as well as their ability to support learning.

One notable example is a study conducted by Jusas et al. [2], which examined the effects of web-based gamification on students in an object-oriented programming course. The study found that the use of gamification led to significant improvements in motivation and engagement and increased student satisfaction with the course. Another study by Marín et al. [5] found that students using a gamified platform for learning C programming obtained better marks than those using a non-gamified compiler. In [6], the authors found that adopting a gamified approach to teaching programming resulted in enhanced attendance, increased downloading of course materials, and improved final grades. Buckley et al. [7] concluded that gamified learning interventions have a positive impact on student learning and that the effects of gamification on student participation vary, depending on whether the student is motivated intrinsically or extrinsically.

To further enhance the understanding of gamified learning environments, recent research has explored different aspects of this approach. Gamification has been applied to other domains and disciplines, such as English vocabulary training [8], sustainability education [9], discovery learning [10], and palliative care awareness [11]. These studies have shown that gamification can enhance students’ interest, involvement, and performance across different learning scenarios and topics. For example, ref. [12] conducted a cluster micro-randomized trial on medical interns to evaluate the effectiveness of gamified team competition as a mHealth intervention. The study showed that team competition significantly increased daily physical activity, suggesting its potential as a mobile health intervention tool.

Moreover, some studies have explored the use of adaptive gamification, which tailors the game elements to the preferences and needs of each learner. For instance, ref. [13] investigated the effects of adaptive gamification on students’ motivation in science education. They found that adaptive gamification increased students’ intrinsic motivation and self-efficacy more than non-adaptive gamification. Durmaz et al. [14] examined the influence of gamification elements on the explicit motive dispositions of learners. They found that different types of rewards had different effects on learners’ achievement, power, and affiliation motives.

Finally, some studies have focused on designing and evaluating gamification tools for massive open online courses (MOOCs), which pose specific challenges and opportunities for engaging and motivating learners. For example, Ortega-Arranz et al. [15] presented GamiTool, a web-based tool that supports instructors in the gamification of MOOCs. Their evaluation with MOOC instructors and gamification designers showed GamiTool’s high design expressiveness, usability, and potential for adoption. This system can be used by instructors to improve student engagement, while researchers can employ it to understand the effects of gamification in MOOC settings. Khaldi et al. [16] conducted a systematic literature review about gamification in e-learning in higher education. They found that points, badges, and leaderboards (PBL elements) were the most commonly used game elements for gamifying e-learning systems. The impact of gamification in MOOCs was also reviewed by [17], revealing positive outcomes with increased participation and retention in gamified MOOCs.

1.1. Gamification

The concept of gamification has gained popularity in recent years as a way to engage and motivate people to achieve their goals. Gamification involves the integration of game design elements, such as points, badges, and leaderboards, into non-gaming contexts, to create a sense of competition and accomplishment. This technique has been applied in fields such as education, healthcare, and marketing, with the aim of improving learning outcomes, promoting healthy behaviors, and increasing engagement. The purpose is to have a methodology that can motivate the user to use it. It is important to correctly apply gamification elements to maintain user motivation. In this way, users are engaged on these platforms while acquiring knowledge. In the context of education, gamification has been the subject of study and debate among researchers over the years. In general, the implementation of gamification in the educational system has provided two crucial benefits for students: they are motivated to continue learning, and it results in improved performance compared to a more traditional educational context. It is noteworthy that better scores in practical assignments and overall scores were obtained by students who completed the gamified experience [18].

1.2. Game Design Elements

Game design elements are commonly used in gamification to create engaging and rewarding experiences for students. Some of the elements presented here were evaluated by 19 experts on gamification and education [19].

• Points: A key component in many video games. In gamification, they are used as rewards for successfully completing certain tasks within the environment. The user’s progress is also represented in a numerical way. Different types of points exist, such as experience points, redeemable points, and reputation points. Points allow the user to receive instant feedback on their progress within the platform, motivating them to continue earning more points, which will unlock additional rewards.

• Achievements/badges: In a gamified system, users are granted virtual rewards in the form of achievements and badges after they have completed certain tasks. These rewards should be consistent with the gamified task, as the aim is to maintain user motivation. Sometimes, these achievements or badges are divided into smaller rewards to make them more frequent and provide more motivation to users. Achievements and badges are represented by visual images, and they are available for users to view whenever they want.

• Leaderboards: The points obtained by each user, along with their usernames and avatars, are displayed in a list of participants. These tables create a competitive environment among students and challenges for those who are not in the top ten positions. Additionally, a personal list is maintained to show users who their competitors are if they are not among the top ten positions.

• Store: This is another important section in gamification, as rewards can be obtained by students. These achievements provide redeemable points, which can be used to purchase various additional items available in the store. These items can include “Change avatar” and “Change background”, allowing students to customize their user profiles. Other items include the possibility of purchasing extra points toward the final grade, a feature that typically ranks among the most popular and motivating for students.

1.3. Hypothesis

According to the state of the art, the implementation of a web-based gamified tool has been widely recognized as beneficial for students; it significantly enhances their learning experience and motivation in programming courses. Furthermore, the integration of an automated assessment system further contributes to the effectiveness and efficiency of the learning process. By combining these two powerful components, a novel approach to create a comprehensive tool that integrates gamification principles with automated programming assessment capabilities is proposed in this paper.

The primary objective of this research is to design, develop, and evaluate a web-based tool that leverages the advantages of gamification and automated assessment in a programming course context. The tool aims to provide students with a stimulating and engaging learning environment.

The central hypothesis of this research is that the implementation of this gamified tool will significantly increase students’ motivation and engagement in programming activities. By incorporating game-like elements, such as achievements, leaderboards, and a reward system, an immersive learning environment is created that fosters a sense of competition, collaboration, and accomplishment. It is anticipated that this motivational aspect, combined with the automated assessment system, will enhance students’ learning outcomes.

We aim to address the following research questions by implementing this tool in a university programming course:

• How does the integration of a gamification system impact students’ motivation and engagement in a programming course?

• What functionalities of the web-based gamified tool influence the students’ motivation?

The remaining sections of this paper are organized as follows. Section 2 provides an overview of the methodology applied in this study, including the web-based gamification tool used. In Section 3, the results obtained from the study are presented in detail. Finally, Section 4 presents the discussion and conclusions drawn from the research and outlines potential avenues for future work.

2. Materials and Methods

This section presents the details of the methodology carried out and the descriptions of the main functionalities included in the web-based tool.

2.1. Methodology

To answer the research questions presented in Section 1.3, a comprehensive evaluation methodology was employed, including quantitative and qualitative data collection techniques. The study consisted of a sample of students enrolled in the programming course, who interacted with the tool throughout the duration of the course. Their performance, engagement levels, and feedback were analyzed to assess the effects of the tool on their learning experiences.

To evaluate the effectiveness of the proposed tool, a comprehensive assessment was conducted. This evaluation included the quantitative analysis of students’ answers through a survey. The results of this survey will provide valuable insights into the effects of gamification and automated assessment on students’ motivation and learning outcomes in the context of programming courses.

Importantly, it should be noted that the tool was universally accessible and available to all students enrolled in the course without any restrictions. This unrestricted access was designed to ensure equal opportunities for engagement and learning experiences.

Nevertheless, the study did not employ a traditional experimental design with distinct control and experimental groups. Instead, it focused on a single cohort of students who interacted with the gamified tool, assessing their experiences and perceptions without the comparative framework of a control group. This design decision emerged from the intention to capture authentic interactions and feedback during the tool’s inaugural implementation. Recognizing the intrinsic value of comparative studies, featuring both control and experimental groups, this limitation of the current study is acknowledged. Future research will endeavor to incorporate both a control group and an experimental group to yield a more robust assessment of the effects of gamification on student learning and motivation.

According to the literature, conducting a survey and quantitatively measuring data are suggested as valid research methodologies; however, there are some limitations and considerations to keep in mind. Boeren [20] notes that survey research can gather both qualitative and quantitative data while also investigating the effects of survey methodologies on the format of questions. Barabas [21] compares survey experiments with natural experiments and suggests caution when extrapolating from survey experiments. Mills [22] investigated the extent to which researchers accessed quantitative methodology publications and found that researchers do not frequently access quantitative methodology literature to determine the best way to analyze their data. Finally, Nardi [23] provides a guide to quantitative methods, including designing a questionnaire, sampling, and analyzing data. These papers suggest that conducting a survey and quantitatively measuring data are valid research methodologies; however, researchers should be aware of the limitations and considerations of this approach.

This study was conducted at a public university in Spain, involving a total of 215 students from three distinct computer engineering degree programs: computer engineering, computer science engineering, and engineering in information systems. These programs represent specialized areas within the broader field of computer engineering, with each encompassing different educational focuses and potential career trajectories. The sample was selected through stratified random sampling, dividing the students into three strata corresponding to the different computer engineering degrees. Within each stratum, students were randomly selected using a computer-generated randomization process to ensure an unbiased selection. This method was chosen to provide a comprehensive perspective on the effects of the gamified learning environment across these diverse educational paths and to ensure that the sample was representative of the broader population of students within the computer engineering field at the university. Further demographic information, such as the year of study, gender distribution, and prior experience with gamified learning, was also collected to provide additional context and nuance to the analysis. This more detailed description of the participating student groups helps to ensure that the study’s findings are relevant and applicable to a broad spectrum of students within the field. The participants were undergraduate students enrolled in an object-oriented programming course. The course was gamified using web-based software that included features such as points, badges, and leaderboards. Student performances were assessed using programming assignments, and their motivation and engagement were measured using a survey. From March 2023 to May 2023, an online survey platform [24] was employed to administer a questionnaire to the students. Student participation was entirely voluntary, and students had the option to decide whether or not to complete the questionnaire. It is important to emphasize that the questionnaire ensured anonymity and confidentiality. This voluntary nature of participation led to a response rate of 52.5%, which accounted for the difference between the number of students taking the course and the number of students evaluating the software. Subsequently, the online survey platform provided an Excel file containing the students’ responses, facilitating the analysis of the collected data. The data analysis for this study was performed using R 4.1.1 software, supported by the R Core Team and the R Foundation for Statistical Computing. The analyses were conducted on a Windows 10 ×86-64 machine with an Intel(R) Core(TM) i7-7700 CPU operating at 3.60 GHz with 16 GB of physical memory.

The gamification process implemented in this study was thoughtfully designed with the aim of enhancing students’ engagement and motivation in the programming course. The process, which was rooted in the principles of game-based learning, leveraged elements such as points, badges, and leaderboards to create a competitive and engaging learning environment. The development began with an analysis of the learning objectives and the identification of key motivational factors that could be influenced by gamification. A series of iterative prototyping and testing cycles ensured that the gamification features were effectively integrated and resonated with the interests and needs of students. The final implementation was seamlessly embedded into the existing course structure, allowing students to participate in gamified activities alongside traditional learning.

To examine the effects of the gamification platform on students’ motivation in the programming course, the questionnaire focused on assessing the factors that influenced their motivation and engagement with the platform. Building upon existing research in the field [25,26,27,28], the survey was derived from established sets of questions that explored the various elements affecting motivation. Using a 5-point rating scale (5: completely agree; 4: agree; 3: neutral; 2: disagree; 1: completely disagree), the students were asked to rate their level of motivation when using the gamification platform for the programming course. The following reasons were evaluated by the students:

• The use of the tool has motivated me to attend class.

• I was more motivated to study and therefore obtained more points.

• Achieving points improved my self-confidence.

• The competitive environment of the leaderboard enhanced my motivation.

• When my code did not pass the tests, I felt motivated to look for errors and fix them.

In addition to rating their motivation, students were provided with an open question, “Which part did you like the most?”. This question aimed to gather qualitative feedback and opinions from the students.

In order to gain a comprehensive understanding of the results, the survey aimed to provide students with the opportunity to express their opinions regarding the research questions. By addressing these research questions, this study seeks to contribute to the growing body of knowledge on the effective integration of gamification and automated assessment in web-based learning environments. The findings will not only inform the design and development of future educational tools but also provide valuable guidance to educators and instructional designers, aiming to create engaging and effective learning experiences in the field of programming.

Additionally, to supplement the student-centered evaluations, informal interviews were conducted with the professors of the course, who are also co-authors of this paper. Experts in the field of gamification were invited to use the tool and offer their academic opinions. Furthermore, participating students were also informally interviewed to gather firsthand feedback on their experiences with the gamified learning environment. These interviews serve to triangulate the data, adding depth and context to the findings obtained through the student surveys.

2.2. Web-Based Tool

The presented tool is a user-friendly and accessible web-based platform designed to enhance the learning experience and increase student motivation in a programming course. The tool offers a seamless and intuitive interface, making it easy for students to navigate and interact with the platform. Through the implementation of a reward system, including achievements, badges, and points, students are motivated to actively engage in their learning journey. These rewards not only recognize their progress and achievements but also provide a sense of accomplishment and satisfaction.

Additionally, a ranking system is incorporated into the tool, allowing students to compare their performance with their peers. This healthy competition fosters a sense of challenge and encourages students to strive for excellence in their programming skills. By integrating gamification elements into the learning process, the goal is to create an enjoyable and motivating environment for students, instilling feelings of excitement, curiosity, and a desire to continually improve their abilities.

The tool is available online at URL https://gamificacion.cc.uah.es/, accessed on 9 August 2023, which is the Spanish version that has been used in the present research work.

Furthermore, the design stages of the web-based tool have been meticulously planned to ensure its effectiveness and user engagement. These stages encompass research and analysis, objective definition, user experience design, gamification elements design, education content development, technical development, testing, and finally, launch and deployment. A comprehensive visualization of these design stages can be found in Figure 1.

During the “Educational Content Development” stage, the course content was carefully integrated into the platform. It is pertinent to note that this content has remained consistent and has been used across multiple academic years. Given the stability and time-tested nature of the content, there were no immediate requirements to update the software post-deployment.

In addition, the “Testing” phase was not solely confined to internal evaluations. Rigorous testing was undertaken, involving not just the development team but also students from other courses. This approach was adopted to garner a diverse range of feedback and to ensure that the platform was free from errors and provided an optimal user experience. Any discrepancies or issues identified during this phase were promptly addressed, ensuring that the software was both robust and reliable upon its final deployment.

In summary, the platform integrates several key features to further enhance the user experience. It boasts an intuitive user interface, a robust reward system, progressive challenges and levels, immediate feedback mechanisms, comprehensive evaluation systems, and personalized content tailored to individual user progress. A notable aspect is the competitive edge introduced by allowing students to compare their scores with peers through a ranking system. Subsequent subsections will delve deeper into some of these features, providing a more detailed exploration of their implementation.

2.2.1. Points

Points serve as the backbone of the gamification aspect of the system, providing a tangible measure of achievement and progress. The system employs two types of points: experience points and redeemable points. Each type serves a distinct purpose and is earned through different mechanisms, fostering a sense of accomplishment and incentivizing active participation.

Experience points are designed to track and reward users’ progress in the system. Users accumulate these points as tasks are successfully completed, tests are passed, and programming skills are demonstrated. By earning experience points, users are gradually leveled up, new challenges are unlocked, and access to higher-level tasks is granted. The leveling system aims to provide a structured learning path that allows users to develop their skills incrementally.

On the other hand, redeemable points offer users the opportunity to personalize their learning journey and customize their experience within the system. These points can be earned by achieving specific milestones, reaching certain levels, or obtaining exceptional results. Redeemable points can be accumulated by users and then used to purchase items from the store.

2.2.2. Achievements

Achievements are rewards that users receive for different actions performed, similar to badges within the platform. An example of these achievements, including a welcome achievement and a completion achievement for a task group called familiarization, is shown in Figure 2.

Upon unlocking an achievement, users not only receive recognition but also earn redeemable points. These points can be accumulated and used in the store section of the platform, where users can exchange them for a wide range of items and rewards. This incentivizes users to actively engage with the learning process, striving to unlock achievements and accrue points for further exploration and personal growth within the system.

2.2.3. Badges

Badges are rewards given to users based on their actions while using the tool. An example of two badges received by a student for completing tasks on the platform is shown in Figure 3. These badges serve as incentives to encourage users to perform certain actions. For instance, when students complete all the tasks in a task group on the gamification platform, they may be awarded a badge. Each badge has a distinct design, making it easy to identify.

By earning badges, users can showcase their skills and knowledge through a visually organized collection. This collection serves as a testament to their progress and achievements within the platform, motivating them to further explore and excel in their programming journey.

2.2.4. Leaderboards

In this web-based tool, leaderboards play a significant role in enhancing student motivation and engagement. It has been expressed by students that the leaderboards have been one of the most motivating gamification elements as they compete against other students from different computer science-related degrees.

A global leaderboard is available, showcasing the top ten users, as depicted in Figure 4. This leaderboard displays the ranking position, username, university degree of the student, current level, and experience points. The positions are determined based on the accumulated experience points, with the leaderboard sorted in descending order.

To prevent students from feeling demotivated by not being among the top ten positions in the global leaderboard, another leaderboard has been implemented. This personal leaderboard, shown in Figure 5, allows students to view their individual position, even if it is lower. It enables them to identify the students directly above and below their position, fostering a sense of competition and encouraging them to strive for improvement.

The implementation of leaderboards in the tool has proven to be highly effective in motivating students to actively participate and progress in the programming course. It provides a visible representation of their achievements and progress, fostering a healthy competitive spirit among students.

Additionally, the leaderboards promote a sense of accountability and self-improvement. Students are aware that their performances are being tracked and compared to others, which instills a desire to surpass their own previous achievements. This intrinsic motivation drives students to continuously challenge themselves and aim for higher levels and experience points.

2.2.5. Store

The store page is another important element in a gamification tool. In this web-based gamification tool, this feature provides students with an exciting opportunity to personalize their experience and make meaningful choices with their redeemable points.

The store offers a range of items that can be purchased by students to enhance their profile and overall learning journey. With the redeemable points, different items such as “Change Avatar”, “Upload Animated Avatar”, “Change Profile Background”, and “+0.2 in the Final Grade” can be purchased. An example of one of these items is shown in Figure 6. When a purchase is made, the redeemable points are deducted from the student’s current points. This section of the web tool aims to offer students a clear objective for completing tasks, allowing them to earn redeemable points through achievements and subsequently spend those points in the store.

2.2.6. Tests

When a task is accessed by the student, three elements are presented on the same page. Firstly, there is the exercise statement, which can be an example like the one shown in Figure 7. Secondly, there is the code editor, which initially displays a skeleton code for the student to work on, as seen in Figure 8. Lastly, there are unit tests that the code must pass, as exemplified in Figure 9. These tests display both the input and the expected output. The output must match the expected output for the test to be considered valid. On the other hand, hidden tests serve the same purpose, but the input and expected output are unknown. In this way, the student must write the correct code and cannot cheat. Information is also provided to the user in case of execution and compilation errors.

In Figure 9, various buttons with different functionalities can also be observed. The “Compile” button serves to encode the user’s code into base64 format and send it to the automated programming assessment system solely for the purpose of compilation. In case of compilation errors, a notification is provided to the user. The “Test” button, on the other hand, sends both the code and the unit tests for evaluation. Feedback regarding the success or failure of each test is conveyed to the user. The “Send” button becomes available only when all tests have been successfully passed, thus allowing the exercise to be submitted as successfully completed; this status is subsequently recorded in the database.

Notably, the feedback mechanism plays a vital role in motivating students to actively engage with the tests. When their code fails a test, students are encouraged to review the error message and identify the specific issue in their implementation. This iterative process of debugging and refinement stimulates the students’ problem-solving skills and enhances their understanding of programming concepts.

In summary, the suggestions that the software can offer to students to correct their errors are as follows:

• Visibility of unit tests: These tests clearly display both the input and the expected output. This allows students to have a concrete understanding of what their code is supposed to achieve. If their code does not produce the expected output, they know where the issue lies.

• Hidden tests: To ensure genuine understanding and to avoid the potential for students to simply game the system, there are also hidden tests. These tests do not reveal the input or expected output. As a result, students must rely on the correctness of their code rather than attempt to fit their solution to a known output.

• Feedback mechanism: In the event of a failed test, the software provides an error message, guiding the student to the specific issue in their code. This immediate feedback mechanism encourages the student to iterative debug and refine their code, reinforcing their problem-solving skills and deepening their grasp of programming concepts.

Thus, the software does not simply ask students to “start all over again” without guidance. Instead, it offers valuable feedback that facilitates their learning process. This continuous testing, combined with real-time feedback, is central to the software’s pedagogical approach. It offers students a more engaging, dynamic learning environment and has the added benefit of boosting their motivation to actively participate and perform well in the course.

By integrating testing as an integral part of the gamified tool, students are empowered to track their progress, identify areas for improvement, and develop a growth mindset. The combination of immediate feedback, continuous testing, and iterative problem-solving creates a dynamic learning environment that nurtures students’ programming skills and fuels their intrinsic motivation to excel in the course.

3. Results

This section presents both the quantitative and qualitative results obtained through the students’ questionnaires. A subsection is also included, containing a discussion of these results.

3.1. Questionnaire Results

This subsection presents the results obtained from the survey responses regarding the effects of the gamification tool on the motivation of students in the programming course. The questionnaire aimed to assess various factors related to motivation and their influence on students’ learning experiences. Figure 10, Figure 11, Figure 12, Figure 13 and Figure 14 display the numerical outcomes obtained from the survey responses, providing an overview of the motivation levels attributed to the gamification platform.

The use of the gamification platform has positively influenced students’ motivation to attend class, as can be seen in Figure 10. A significant percentage of participants (52.21%) expressed agreement or complete agreement in this regard. Conversely, a small fraction (12.39%) disagreed or completely disagreed with the statement, suggesting that the platform may not have been as effective in motivating their class attendance.

According to the results presented in Figure 11, it was found that the participants’ motivation to study was increased by the gamification platform, resulting in a higher accumulation of points. A significant majority (86.72%) either agreed or completely agreed with this statement, highlighting the crucial role played by the platform in enhancing their motivation and academic achievements.

The acquisition of points through the platform has positively impacted students’ self-confidence, as illustrated in Figure 12. The survey results revealed that 75.22% of participants agreed or completely agreed that earning points enhanced their self-assurance, indicating that the reward system embedded in the platform played a significant role in boosting their confidence.

The competitive environment created by the platform’s leaderboard has served as a catalyst for student motivation, as shown in Figure 13. A considerable proportion of participants (38.05%) completely agreed and an additional percentage (15.04%) agreed that the competitive aspect of the leaderboard heightened their motivation. Furthermore, a notable proportion of participants (31.86%) remained neutral, indicating a range of responses to the competitive element. A smaller percentage (12.39%) disagreed, while a minority (2.66%) strongly disagreed.

Students expressed that encountering errors in their code and failing tests motivated them to actively seek and rectify these errors. The majority of participants (72.56%) agreed or completely agreed with this statement, as depicted in Figure 14, indicating that the platform effectively stimulated students’ motivation to engage with and learn from their mistakes.

The survey results provide valuable insights into the effects of the gamification platform on student motivation in the programming course. While the majority of students reported positive experiences and benefits from using the platform, it is important to acknowledge that a minority of participants did not find it as motivating. These findings highlight the need for further investigation and refinement of the gamification elements to cater to the diverse motivational needs of students (Table 1).

Inferential Statistical Analysis

One-sample T-tests were used to analyze quantitative data. The objective was to reach a conclusion about whether the use of the gamification tool has had a significant impact on student motivation to study the subject. The following null hypotheses were posed:

• The use of the tool does not motivate students to attend class ($\mu \le 3$).

• The students are not more motivated to study and to obtain more points ($\mu \le 3$).

• Achieving points does not improve the students’ self-confidence ($\mu \le 3$).

• The competitive environment of the leaderboard does not enhance the students’ motivation ($\mu \le 3$).

• The students do not feel motivated to look for errors and fix them when their code does not pass the tests ($\mu \le 3$).

Table 2 summarizes the results of the hypothesis tests. According to the p-values obtained, every null hypothesis is rejected. Therefore, it can be concluded that the population mean is greater than three for the five tests. This means that the use of the gamification tool has a significant effect on the students’ motivation. Specifically, the students are more motivated to attend class and to study the subject when they achieve points and when they advance in the leaderboard. This results in an improvement in their self-confidence. Also, they feel motivated to look for errors and fix them when their code does not pass the tests.

3.2. Open Question: Motivation Dimension

The open question “Which part did you like the most?” was included in the survey to gather insights into the factors that have influenced student motivation. The students provided responses that encompassed multiple topics, indicating a diverse range of interests and focus areas. The results presented in Figure 15 reveal the following topics and their corresponding influence percentages: 13.7% for tests/hidden tests, 12.8% for points, 12% for the final grade, 11.1% for the practical part, 9.4% for rewards/store, 8.5% for other factors, 7.7% for the general experience, 6.8% for ranking, 5.1% for learning, 4.3% for simplicity, 3.4% for fun, 2.6% for code evaluation, and 2.6% for nothing.

Tests/hidden tests, points, final grades, and the practical parts were the criteria that had the greatest influences; collectively, they accounted for almost 50% of the responses. These findings imply that students were driven by the challenge and importance of the course material, as well as by extrinsic and intrinsic rewards. Code evaluation, nothing, fun, and simplicity, which together accounted for less than 13% of the responses, were the least significant factors. These findings suggest that other characteristics of the gamified system were more highly valued by students than the feedback mechanism, entertainment value, or convenience of use. The remaining factors, which ranged from 5.1% to 9.4% of the replies, included rewards/store, other factors, general experience, ranking, and learning, which moderately impacted student motivation. These findings highlight the variety of student preferences and the necessity to create gamified lessons that accommodate various learning objectives and styles.

In addition to the quantitative analysis, it is important to consider the qualitative analysis of comments provided by students. Several students provided interesting answers regarding their preferences and the factors that motivated them within the gamification platform.

The following are some of the relevant statements made by students in response to the question:

• “The hidden tests that ensure consideration of all possible cases.”

• “The progression system: points, achievements, etc.”

• “Being motivated by the reward system.”

• “The leaderboard made us want to outperform other university degrees and have more classmates in the top 10.”

These statements highlight the positive impact of various gamification elements on student motivation and engagement. The presence of hidden tests ensures complete coverage of all possible cases, promoting a sense of thoroughness and mastery. The progress system, with its points and achievements, serves as a strong motivator for students to work harder and excel in their performance. The reward system further enhances motivation by providing tangible incentives for their efforts. The leaderboard, in particular, generates a competitive spirit among students, driving them to surpass their peers and attain a coveted position in the top 10. However, it is essential to acknowledge that the leaderboard can also have negative consequences, as it may demotivate students at the lower end of the rankings. To address this, an additional personal ranking system was implemented, as explained in Section 2.2.4.

Overall, the students’ feedback confirms the importance and effectiveness of gamification techniques in the programming course. These techniques provide additional motivation, foster a desire for improvement, and create a more engaging learning environment. It is crucial to strike a balance, ensuring that the gamification elements promote healthy competition while also promoting collaboration and a supportive learning community.

3.3. Interviews

This subsection aims to elaborate on the qualitative insights gleaned from informal interviews conducted with course instructors, academic experts in gamification, and participating students.

Firstly, the instructors of the course, who are also the co-authors of this paper, conveyed a highly favorable view of the gamified learning tool. They observed heightened levels of student motivation and engagement during the course. The consensus among the educators was that the tool has potential for broader application across various subjects.

Secondly, academic experts specialized in gamification, having utilized the tool over a period, vouch for its suitability in the context of this study. In their opinion, based on years of experience in the field, the results generated by this tool have the potential for generalization across educational contexts in Spain.

Lastly, student participants expressed enthusiastic approval of the gamified learning environment. Specific attention was drawn to the competitive aspect engendered by the leaderboard, which pitted students from different degrees against each other. This feature was noted as particularly motivating and engaging.

4. Discussion and Conclusions

In this section, the implications of the results will be discussed in relation to the research questions, and the study’s limitations and the verification of the hypotheses will be addressed. Additionally, a comparison with previous studies will be conducted to assess the alignment of the findings.

Regarding the first research question, “How does the integration of a gamification system impact students’ motivation and engagement in a programming course?”, the results from the survey indicate that the integration of the gamification system has had a positive impact on students’ motivation and engagement in the programming course. A significant percentage of students expressed that the use of the tool had motivated them to attend class, with 52.21% either completely agreeing or agreeing with this statement. This finding suggests that the gamification elements implemented in the platform have successfully incentivized students to actively participate in the course.

Furthermore, a majority of the students (82.72%) reported being more motivated to study and obtain more points. This high level of agreement demonstrates the influence of the gamification system in fostering students’ intrinsic motivation to excel in their studies. The correlation between increased motivation and the accumulation of points suggests that the point-based system effectively incentivizes students to strive for higher achievement.

These findings are in line with previous studies that have explored the effects of gamification on student motivation and engagement in educational settings. For instance, Campillo-Ferrer et al. [29] found that using Kahoot! quizzes improved students’ perception of certain concepts, increased their active participation, and motivated them toward learning in a more interactive and stimulating environment. Similarly, Ndlovu and Mhlongo [30] found that gamification has the potential to trigger and maintain students’ situational interest, which serves as a foundation for an engaged and motivated student. The consistent findings across these studies and the gamification tool presented in this paper support the notion that gamification can be an effective strategy to enhance student motivation and engagement in programming courses.

Regarding the second research question, “Which functionalities of the web-based gamified tool influence in the students’ motivation?”, the survey results shed light on the functionalities of the web-based gamified tool that have a significant influence on students’ motivation. The competitive environment of the leaderboard was identified as a strong motivator, with 53.09% of students either completely agreeing or agreeing that it enhances their motivation. This suggests that the leaderboard effectively simulates a sense of competition among students, driving them to outperform their peers and achieve higher ranks.

The acquisition of points was also found to have a positive impact on students’ motivation, as 75.72% of students either completely agreed or agreed that achieving points improved their self-confidence. This indicates that the point system is an effective extrinsic motivator, boosting students’ confidence and providing a tangible measure of their progress.

Additionally, when students’ code did not pass the tests, a majority (72.56%) expressed feeling motivated to search for errors and rectify them. This finding highlights the intrinsic motivation that arises from the challenge of overcoming coding obstacles, indicating that the gamified tool effectively promoted problem-solving skills and perseverance.

The survey also included an open question asking students about the specific parts of the web-based gamified tool they liked the most. The results provide further insights into the functionalities of the tool that influence students’ motivation. The high percentage of students expressing a liking for tests/hidden tests, points, and the final grade suggests that these elements contribute significantly to their motivation. The practical part, rewards/store, and other factors also play a notable role in influencing student motivation. The rankings and the overall gamified experience are mentioned as influential factors as well.

These findings complement the previous results regarding the effects of the leaderboard and point system on motivation. The positive influence of tests/hidden tests, practical components, and other factors align with the gamification principles of challenges, feedback, and variety, which are known to enhance motivation in educational contexts.

The functionalities identified in this study are consistent with prior research, highlighting the motivational impact of leaderboards, points, and challenges in gamified learning environments. For example, in [31], the authors found that points and leaderboards stimulated structural and trait competitiveness, which increased engagement. Additionally, ref. [32] concluded that leaderboards were successful in motivating participants to performance levels similar to that of difficult and impossible goal-setting. Thus, the findings align with previous research, indicating that these functionalities play a crucial role in driving student motivation in programming courses.

The study investigated the effects of a web-based gamified tool on students’ motivation and engagement in a programming course. Results showed that the gamification platform positively influenced students’ motivation to attend class and study. It also boosted their self-confidence and fostered a competitive spirit through the leaderboard. Encountering errors motivated students to learn from their mistakes.

Despite the positive implications of the research, certain limitations must be taken into account. First, the study was conducted solely within one educational institution, potentially narrowing the generalization of the findings. Second, the absence of a control group for comparison restricts the ability to definitively attribute the observed effects solely to the gamified tool. Both limitations present avenues for future work.

Also, it can be concluded that the research hypotheses have been empirically substantiated. The integration of a gamification system has demonstrably enhanced student motivation and engagement in a programming course. While certain limitations are acknowledged, the study contributes valuable insights to the burgeoning body of knowledge on gamification in educational settings.

In conclusion, the gamified tool significantly increased students’ motivation and engagement in the programming course, combining automated assessment with gamification principles. Future work could involve measuring student performances by using this gamification tool and assessing whether the implementation of an automated programming assessment system in the gamification platform contributes to reducing the workload of teachers. In addition to the mentioned future work, another important aspect to consider is the application of this gamification tool in various universities, to gather a diverse range of examples and feedback. By implementing the tool in different educational institutions, it will be possible to assess its effectiveness across various academic settings and disciplines.

Author Contributions

Conceptualization, C.J.H. and F.C.-E.; methodology, J.G. and A.T.; software, C.J.H. and F.C.-E.; validation, C.J.H. and F.C.-E.; resources, A.V.; writing—original draft preparation, C.J.H.; writing—review and editing, J.G., A.T. and S.O.-T.; supervision, A.T.; funding acquisition, J.G. All authors have read and agreed to the published version of the manuscript.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Not applicable.

Conflicts of Interest

The authors declare no conflict of interest.

Footnotes

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Figures and Tables

Figure 1. Illustrative overview of the design stages for the creation of the web-based gamified educational tool.

Figure 2. The user profile displays the student’s achievements. In this case, the welcome and completed familiarization task group achievements are shown.

Figure 3. The user profile displays the student’s badges. In this case, these two badges demonstrate that the student has completed two task groups.

Figure 4. The leaderboards are displayed on a separate web page. The image shows the top four, but by scrolling down, the remaining six can be seen.

Figure 5. The personal ranking is also displayed on the same web page as the other leaderboard, but below it.

Figure 6. A store item. Purchasing this item allows the user to customize their profile by changing their avatar.

Figure 7. One of the initial exercise statements that will be seen by the student. It is displayed on the left side of the task page, and the text is encoded with Markdown.

Figure 8. The code editor used to perform the task. It is located on the task page, to the right of the exercise statement.

Figure 9. The tests necessary to pass the task. In this case, a regular test can be seen that expects an output, but there are also hidden tests for which neither the input nor the expected output is known. The compile, run tests, and submit buttons are also situated here. This section is located just below the exercise statement and the code editor.

Figure 10. Motivation to attend class among students.

Figure 11. Motivation to study among students, based on their responses.

Figure 12. Impact of earning points on self-confidence among students, as indicated by their feedback.

Figure 13. Influence of the leaderboard on motivation among students, based on their perspectives.

Figure 14. Motivation derived from encountering errors and failing tests among students, according to their responses.

Figure 15. Most influential motivational topics, as perceived by students.

Table 1

Statistical summary of the survey evaluating the gamification platform.

Item	Mean	Median	Mode	SD
The use of the tool has motivated me to attend class.	3.58	4.0	3	1.04
I was more motivated to study and therefore obtain more points.	4.33	5.0	5	0.84
Achieving points improved my self-confidence.	4.12	4.0	5	0.86
The competitive environment of the leaderboard enhances my motivation.	3.73	4.0	5	1.17
When my code does not pass the tests, I feel motivated to look for errors and fix them	3.87	4.0	4	0.91

Table 2

Summary of hypothesis tests on student motivation.

Item	p-Value
The use of the tool does not motivate students to attend class.	1.475415 × 10${}^{-8}$
The students are not more motivated to study and to obtain more points.	1.026765 × 10${}^{-32}$
Achieving points does not improve the students’ self-confidence.	1.592054 × 10${}^{-26}$
The competitive environment of the leaderboard does not enhance the students’ motivation.	5.371024 × 10${}^{-10}$
The students do not feel motivated to look for errors and fix them when their code does not pass the tests.	8.823050 × 10${}^{-18}$

References

1. Molina-Carmona, R.; Llorens-Largo, F. Gamification and Advanced Technology to Enhance Motivation in Education. Informatics; 2020; 7, 20. [DOI: https://dx.doi.org/10.3390/informatics7020020]

2. Jusas, V.; Barisas, D.; Jančiukas, M. Game Elements towards More Sustainable Learning in Object-Oriented Programming Course. Sustainability; 2022; 14, 2325. [DOI: https://dx.doi.org/10.3390/su14042325]

3. Zahedi, L.; Batten, J.; Ross, M.; Potvin, G.; Damas, S.; Clarke, P.; Davis, D. Gamification in education: A mixed-methods study of gender on computer science students’ academic performance and identity development. J. Comput. High. Educ.; 2021; 33, pp. 441-474. [DOI: https://dx.doi.org/10.1007/s12528-021-09271-5]

4. Van Gaalen, A.; Brouwer, J.; Schönrock-Adema, J.; Bouwkamp-Timmer, T.; Jaarsma, D.; Georgiadis, J. Gamification of health professions education: A systematic review. Adv. Health Sci. Educ.; 2021; 26, pp. 683-711. [DOI: https://dx.doi.org/10.1007/s10459-020-10000-3] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/33128662]

5. Marín, B.; Frez, J.; Cruz-Lemus, J.A.; Genero, M. An Empirical Investigation on the Benefits of Gamification in Programming Courses. ACM Trans. Comput. Educ. (TOCE); 2018; 19, 4. [DOI: https://dx.doi.org/10.1145/3231709]

6. Fotaris, P.; Mastoras, T.; Leinfellner, R.; Rosunally, Y. Climbing up the leaderboard: An empirical study of applying Gamification techniques to a computer programming class. Electron. J. E-Learn.; 2016; 14, pp. 94-110.

7. Buckley, P.; Doyle, E. Gamification and student motivation. Interact. Learn. Environ.; 2016; 24, pp. 1162-1175. [DOI: https://dx.doi.org/10.1080/10494820.2014.964263]

8. Panmei, B.; Waluyo, B. The Pedagogical Use of Gamification in English Vocabulary Training and Learning in Higher Education. Educ. Sci.; 2023; 13, 24. [DOI: https://dx.doi.org/10.3390/educsci13010024]

9. Alzahrani, F.K.; Alhalafawy, W.S. Gamification for Learning Sustainability in the Blackboard System: Motivators and Obstacles from Faculty Members & Perspectives. Sustainability; 2023; 15, 4613. [DOI: https://dx.doi.org/10.3390/su15054613]

10. Aldalur, I.; Perez, A. Gamification and discovery learning: Motivating and involving students in the learning process. Heliyon; 2023; 9, e13135. [DOI: https://dx.doi.org/10.1016/j.heliyon.2023.e13135]

11. Reigada, C.; Sandgren, A.; Rivas, S.; Carvajal, A.; Hermida-Romero, S.; Benítez, E.; Ripoll, G.; Olza, I.; Centeno, C.; Gómez, B. Palliative care stay room—Designing, testing and evaluating a gamified social intervention to enhance palliative care awareness. BMC Palliat. Care; 2023; 22, 46. [DOI: https://dx.doi.org/10.1186/s12904-023-01166-9] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/37081434]

12. Wang, J.; Fang, Y.; Frank, E.; Walton, M.A.; Burmeister, M.; Tewari, A.; Dempsey, W.; NeCamp, T.; Sen, S.; Wu, Z. Effectiveness of gamified team competition as mHealth intervention for medical interns: A cluster micro-randomized trial. npj Digit. Med.; 2023; 6, 4. [DOI: https://dx.doi.org/10.1038/s41746-022-00746-y] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/36631665]

13. Zourmpakis, A.I.; Kalogiannakis, M.; Papadakis, S. Adaptive Gamification in Science Education: An Analysis of the Impact of implementation and Adapted game Elements on Students & Motivation. Computers; 2023; 12, 143. [DOI: https://dx.doi.org/10.3390/computers12070143]

14. Durmaz, T.B.; Fuertes, J.L.; Imbert, R. Influence of Gamification Elements on Explicit Motive Dispositions. IEEE Access; 2022; 10, pp. 118058-118071. [DOI: https://dx.doi.org/10.1109/ACCESS.2022.3220254]

15. Ortega-Arranz, A.; Asensio-Pérez, J.I.; Martínez-Monés, A.; Bote-Lorenzo, M.L.; Ortega-Arranz, H.; Kalz, M. GamiTool: Supporting Instructors in the Gamification of MOOCs. IEEE Access; 2022; 10, pp. 131965-131979. [DOI: https://dx.doi.org/10.1109/ACCESS.2022.3228762]

16. Khaldi, A.; Bouzidi, R.; Nader, F. Gamification of e-learning in higher education: A systematic literature review. Smart Learn. Environ.; 2023; 10, 10. [DOI: https://dx.doi.org/10.1186/s40561-023-00227-z]

17. De Freitas, M.J.; da Silva, M.M. Systematic literature review about gamification in MOOCs. Open Learn. J. Open Distance E-Learn.; 2023; 38, pp. 73-95. [DOI: https://dx.doi.org/10.1080/02680513.2020.1798221]

18. Domínguez, A.; de Navarrete, J.S.; de Marcos, L.; Fernández-Sanz, L.; Pagés, C.; Martínez-Herráiz, J.J. Gamifying learning experiences: Practical implications and outcomes. Comput. Educ.; 2013; 63, pp. 380-392. [DOI: https://dx.doi.org/10.1016/j.compedu.2012.12.020]

19. Toda, A.M.; Oliveira, W.; Klock, A.C.; Palomino, P.T.; Pimenta, M.; Gasparini, I.; Shi, L.; Bittencourt, I.; Isotani, S.; Cristea, A.I. A Taxonomy of Game Elements for Gamification in Educational Contexts: Proposal and Evaluation. Proceedings of the 2019 IEEE 19th International Conference on Advanced Learning Technologies (ICALT); Maceio, Brazil, 15–18 July 2019; Volume 2161-377X, pp. 84-88. [DOI: https://dx.doi.org/10.1109/ICALT.2019.00028]

20. Boeren, E. Surveys as tools to measure qualitative and quantitative data. Handbook of Research on Scholarly Publishing and Research Methods; IGI Global: Hershey, PA, USA, 2015.

21. Barabas, J.; Jerit, J. Are Survey Experiments Externally Valid?. Am. Political Sci. Rev.; 2010; 104, pp. 226-242. [DOI: https://dx.doi.org/10.1017/S0003055410000092]

22. Mills, L.; Abdulla, E.; Cribbie, R.A. Quantitative Methodology Research: Is it on Psychologists’ Reading Lists?. Tutor. Quant. Methods Psychol.; 2010; 6, pp. 52-60. [DOI: https://dx.doi.org/10.20982/tqmp.06.2.p052]

23. Nardi, P.M. Doing Survey Research: A Guide to Quantitative Methods; Routledge: London, UK, 2005.

24. EUSurvey Platform. Available online: https://ec.europa.eu/eusurvey/home/welcome (accessed on 18 May 2023).

25. Savage, N.; Birch, R.; Noussi, E. Motivation of engineering students in higher education. Eng. Educ.; 2011; 6, pp. 39-46. [DOI: https://dx.doi.org/10.11120/ened.2011.06020039]

26. Rojas-López, A.; Rincón-Flores, E.G.; Mena, J.; García-Peñalvo, F.J.; Ramírez-Montoya, M.S. Engagement in the course of programming in higher education through the use of gamification. Univers. Access Inf. Soc.; 2019; 18, pp. 583-597. [DOI: https://dx.doi.org/10.1007/s10209-019-00680-z]

27. Figueiredo, J.; García-Peñalvo, F.J. Increasing student motivation in computer programming with gamification. Proceedings of the 2020 IEEE Global Engineering Education Conference (EDUCON); Porto, Portugal, 27–30 April 2020; pp. 997-1000. [DOI: https://dx.doi.org/10.1109/EDUCON45650.2020.9125283]

28. Fotaris, P.; Mastoras, T.; Leinfellner, R.; Rosunally, Y. From Hiscore to High Marks: Empirical Study of Teaching Programming Through Gamification. European Conference on Games Based Learning; Academic Conferences and Publishing Limited: Manchester, UK, 2015.

29. Campillo-Ferrer, J.M.; Miralles-Martínez, P.; Sánchez-Ibáñez, R. Gamification in Higher Education: Impact on Student Motivation and the Acquisition of Social and Civic Key Competencies. Sustainability; 2020; 12, 4822. [DOI: https://dx.doi.org/10.3390/su12124822]

30. Ndlovu, T.N.; Mhlongo, S. An investigation into the effects of gamification on students’ situational interest in a learning environment. Proceedings of the 2020 IEEE Global Engineering Education Conference (EDUCON); Porto, Portugal, 27–30 April 2020; pp. 1187-1192.

31. Amo, L.C.; Liao, R.; Kishore, R.; Rao, H.R. Effects of structural and trait competitiveness stimulated by points and leaderboards on user engagement and performance growth: A natural experiment with gamification in an informal learning environment. Eur. J. Inf. Syst.; 2020; 29, pp. 704-730. [DOI: https://dx.doi.org/10.1080/0960085X.2020.1808540]

32. Landers, R.N.; Bauer, K.N.; Callan, R.C. Gamification of task performance with leaderboards: A goal setting experiment. Comput. Hum. Behav.; 2017; 71, pp. 508-515. [DOI: https://dx.doi.org/10.1016/j.chb.2015.08.008]