Enhancing Assessment Practices with Moodle: A Comprehensive Study on Teachers’ Use of Different Question Types and the Impact of FastTest PlugIn

Abstract

Featured Application

This work provides a comprehensive overview of current learning management system trends and Moodle’s evolving role in educational technology. It emphasises the impact of technological advancements, such as FTP, on assessment practices and aims to offer actionable insights for enhancing online assessments. FTP minimises the labour-intensive task of generating XML-formatted questions, allowing educators to focus on more pedagogical strategies rather than technical issues. FTP not only enhances efficiency but also supports educators in adapting to evolving educational demands, especially in disciplines requiring a high number of students.

Abstract

This study investigates the evolution of online assessment practices through Moodle, a leading Learning Management System (LMS). It explores Moodle’s array of question types alongside the transformative impact of FastTest PlugIn (FTP) on educators’ assessment methods during and after the COVID-19 pandemic. This research examines the efficacy of various question types and import formats for large-scale question elaboration. Surveys have been conducted before and after developing courses to train educators on FTP. By analysing trends before and after adopting FTP, the study aims to gain insight into how educators from different universities and disciplines have adapted their assessment strategies among technological advancements. The introduction of FTP has significantly simplified question creation and integration within Moodle. The study also highlights differences in assessment practices between STEM and non-STEM disciplines. Surveys indicate that, following the FTP training course, the use of all question types has increased: Matching (35%), Missing Word (39%), and Cloze (22%). This work provides a comprehensive overview of current LMS trends and Moodle’s evolving role in educational technology. It emphasises the impact of technological advancements, such as FTP, on assessment practices and aims to offer actionable insights for enhancing online assessments.

1. Introduction

1.1. Current Trends in Teaching and Learning Evaluation Methods

Before the COVID-19 pandemic, traditional evaluation methods, which involved in-person assessments, were predominantly followed. These included written exams, quizzes, presentations, and practical demonstrations [1]. These activities were carried out in physical classrooms, enabling educators to observe directly and receive immediate feedback. Standardised tests were also a common tool for assessing student performance at various educational levels. This teaching type was carried out both in Science, Technology, Engineering, and Mathematics (STEM) and non-STEM courses. During COVID-19, a rapid shift to remote learning environments was needed, leading to significant changes in learning and evaluation methods [1]. Schools and universities had to adopt digital tools and rethink their assessment and teaching strategies to adapt to remote education [2].

The rapid evolution towards online learning, accelerated by the COVID-19 pandemic, has underscored the pivotal role of Learning Management Systems (LMSs) in modern education [3]. LMS platforms are robust frameworks for delivering course content, facilitating student engagement and, especially, conducting assessments remotely [4,5].

Among the innumerable LMS platforms available, Moodle (v.4.4) stands out as it is well-known for its versatility and extensive tools for online assessments [5,6]. Furthermore, Moodle provides educators with a diverse array of question types, ranging from fundamental Multiple-Choice and True/False formats to more sophisticated structures like Cloze and Numerical questions [7]. These question types ensure educators can vary learning styles and assessment methods according to their needs [8]. The number of users of the most popular LMS platforms worldwide is shown in [9], with Moodle standing out significantly above the rest.

1.2. FastTest PlugIn to Enhance Moodle’s User Experience

The use of online quizzes for both formative and evaluation assessments has gained popularity [10], providing students with unlimited testing opportunities and multimedia features, as stated in [11,12,13]. Online quizzes for these purposes have been widely used at university levels. In [14], True/False questionaries were used to evaluate students. This work concluded that a closed-question format should at least provide explanations why students’ answers are incorrect. In [15], the results indicated that students perceived online assessments as restrictive for their science courses, which led to decreasing grades. This decay was likely attributed to an increasing reliance on multiple-choice questions. Consequently, it is stated that further research should explore the limitations of Moodle to enhance the LMS user experience [10,16].

A potential improvement could be the ability to quickly generate extensive question banks for students. This could encourage instructors to engage more deeply and dedicate additional time to developing high-quality material rather than spending time inputting questions into Moodle [17]. This approach could be optimised with a flipped classroom model, where class time is used for students to complete online tasks in the instructor’s presence [18]. Consequently, educators can prevent cheating and encourage more meaningful engagement with the material if, in addition, we include gamification through rewards for doing the activities faster and adequately [19].

In [20], the official plugins recognized by Moodle are listed. These plugins have been developed by several authors and their efficiency and applicability have been studied in some cases. This is further described in the comparative study of Moodle Plugins to facilitate computer-based assessments presented in [6]. This study outlined that parameterised questions can generate a large number of different questions, which is very useful for large classes and avoids fraudulent behaviour.

In this context, integrating tools such as FTP has further enhanced Moodle’s utility in assessment practices, as demonstrated in [21]. This recent innovation was designed to rationalise the creation and integration of diverse question types into Moodle assessment and training courses. FTP simplifies this creation and integration of complex question types into Moodle, which is usually hard work, as stated in numerous works [2,22]. As a result, FTP minimises the labour-intensive task of generating XML-formatted questions, allowing educators to focus on more pedagogical strategies rather than technical issues. FTP not only enhances efficiency but also supports educators in adapting to evolving educational demands, especially in disciplines requiring a high number of students [21].

On the one hand, FTP allows the implementation of gamification in university-level courses. On the other hand, studies indicate that feedback is crucial in guiding students’ learning in the right direction, paving the way for them to acquire knowledge independently [23]. This is particularly important, as XML file types allow for highly detailed feedback, including formulas that cannot be incorporated into other types of files. Feedback is an excellent tool to guide students on what concepts they are acquiring and what they have yet to learn. This study will focus on the Moodle platform, as it is one of the world’s most widely used LMS platforms for conducting online assessments. To do so, the following sections present an analysis of the question types (Section 1.3) and import formats of large question banks (Section 1.4) available in the platform to understand the possibilities towards the implementation of FTP.

1.3. Standard Question Types for Online Assessment in Moodle

Research detailing specifically the different question types within Moodle’s online assessment framework is scarce. Hence, the following data are approximate and may vary based on the specific educational context and the expertise level of Moodle users. Generally, Multiple-Choice and True/False questions dominate due to their efficiency and ease of use for both content creators and students. As shown in [24] (Moodle’s menu to add a question type), by default, Moodle prioritises commonly used question types in its question bank interface.

Table 1 shows these question types, with a brief description, as well as whether they can be configured using FTP. Seven of the sixteen question types listed cannot be configured directly using FTP. However, it should be noted that the numerical (5) and calculated (7, 8, and 9) question types can be “simulated” using Cloze questions (13). In addition, they can be used with many more functions if configured with FTP. While this question type is limited to the functions that Moodle has available [25], FTP is developed in Excel and, therefore, has all the functions provided by this program.

On the other hand, the “Random Short-Answer Matching” (14) are not questions as such but are question types that are generated using the previously created “short answer” questions. So, the only ones that cannot be created using FTP are 10 and 11 because they are questions that must be marked on an image, something complicated to program, concluding that only two question types cannot be configured (and/or simulated) using FTP.

1.4. Import Formats for Question Banks in Moodle

Moodle supports various file formats for importing question banks, each accommodating different question types, as indicated in [26]. The process of importing files is the same, regardless of the type of file to be imported. What is different is the way the questions are configured within each file type. Table 2 presents the question types most commonly used in Moodle and the format type with which they can be imported. Of all these import formats in Moodle, GIFT and AIKEN are the most used because they are the easiest to set up. While not as set-up-friendly, the Moodle XML format is the most versatile as it supports all question types. Their features, advantages, and limitations are described below.

1.4.1. GIFT Format

The GIFT format is valued for its versatility, accommodating diverse question types, and making it convenient for rapid question creation. Its simplicity is reflected in its straightforward syntax, enabling editing in plain text without requiring advanced programming skills, as seen in Figure 1. Additionally, GIFT files are lightweight and easily transferable between different Moodle systems. One of the primary advantages of the GIFT format is its ease of creation and editing, as it can be manipulated using any basic text editor. Moreover, it facilitates portability by sharing entire quizzes as a single file. Despite its utility, GIFT does have limitations. It does not support all question types available in Moodle, restricting its use primarily to Multiple-Choice and other simpler question formats.

1.4.2. AIKEN Format

The AIKEN format employs a straightforward structure where each Multiple-Choice question begins with a question statement followed by answer options, as shown in Figure 2. This simplicity makes it ideal for the rapid creation of Multiple-Choice questions. AIKEN stands out for its ease of use, offering a simple format that facilitates the quick and efficient creation of Multiple-Choice questions. It is compatible with basic text editors, allowing educators to generate and modify quizzes easily. However, AIKEN is limited to creating only Multiple-Choice questions and does not support additional metadata beyond the question statement and answer options. The AIKEN format is an excellent choice for those looking for a quick and efficient way to import Multiple-Choice questions into Moodle, known for its simplicity and user-friendly nature.

1.4.3. Moodle XML Format

The Moodle XML format is widely preferred for its robustness and ability to import a diverse range of question types. It allows for detailed creation and editing of questions, incorporating additional metadata, such as feedback, scores, and categories. This capability enhances the precision and depth of assessments conducted through Moodle. Moodle XML supports nearly all question types that Moodle can handle, demonstrating its exceptional versatility in accommodating various assessment needs and formats. Several tools are available to rationalise the creation of XML files compatible with Moodle, including specialised editors and plugins designed to simplify the question elaboration process [6].

Moodle XML Format also enables a detailed question description, including text and multimedia formatting options. It facilitates the creation of complex and well-structured questions. It also supports the inclusion of feedback tailored to correct and incorrect responses and includes images and videos. Despite its advantages, Moodle XML Format requires basic knowledge of XML for manual creation and editing of questions. While powerful, manual creation and editing of XML files can be tedious without specific tools or XML editors. Figure 3 shows the same Multiple-Choice question as in the previous formats, in XML format. The GIFT and XML formats include feedback for all responses. The XML encoding varies for each question type.

1.4.4. Comparison of Ease of Use and Opinions

When comparing different formats for importing questions into Moodle, each offers distinct advantages depending on user preferences and requirements.

• Moodle XML: Known for its comprehensiveness and versatility, Moodle XML supports a wide range of question types and detailed configurations. However, its structured nature and requirement for basic XML knowledge can challenge some users.

• GIFT: Valued for its simplicity and speed, GIFT is favoured by educators needing a quick solution for creating and editing quizzes. Its compatibility with basic text editors allows for straightforward management of various question types, making it an efficient choice for content creators.

• AIKEN: The simplest of the formats, AIKEN specialises in Multiple-Choice questions but does not extend to other question types. It is particularly suitable for users seeking a straightforward method to import and manage Multiple-Choice quizzes.

As mentioned earlier, Figure 1 (question 1) and Figure 2 and Figure 3 show the same question with the configuration for each format type. It is shown how the configuration varies depending on the format type, with the simplest being the AIKEN format and the most complex (and also most comprehensive) being the XML format. Educators often prefer GIFT for its ease of use and compatibility with text editors, enabling rapid quiz creation. In contrast, administrators and developers prefer Moodle XML for its ability to handle complex imports, including diverse question types and specific configurations necessary for comprehensive assessments. In summary, the choice between Moodle XML, GIFT, and AIKEN depends largely on the user’s technical proficiency, the complexity of the assessment needs, and the desired ease of use in creating and managing quizzes within the Moodle learning environment. Considering the results of this comparison, in this work, it is opted to use Moodle XML as the import format in Moodle as it supports a wider range of question format. The required knowledge on XML will be addressed by the development of a digital tool that will provide the XML coding.

1.5. Scope of This Work and Research Questions

This study investigates the pivotal role of Moodle in online assessments, particularly focusing on how the platform’s capabilities have evolved in response to the demands of the COVID-19 pandemic. This work seeks to emphasise the necessity of continuous professional development for educators and the importance of integrating advanced technological tools in teaching methods. Moreover, as stated in [2], this work aims at prioritising student well-being and academic success in digital environments.

To do so, first, this work introduced a method to automatically evaluate and help students’ learning by developing FTP for the Moodle framework. Second, this method was presented to and tested by several educators from different universities and disciplines. Surveys have been carried out before and after the attendance to the courses to train educators on FTP. The research examines the efficacy of various question formats and import methods—such as GIFT, AIKEN, and Moodle XML—for large-scale question imports. Also, by analysing trends before and after adopting FTP, the study aims to illuminate how educators have adapted their assessment strategies amidst technological advancements. Given so, the key aspects of this research include the following:

• Evolution of assessment practices: the study explores how the COVID-19 pandemic has reshaped educators’ assessment methods, emphasising the transition to online platforms and the adoption of new technologies like FTP.

• Question formats and import methods: examining different question formats and import methods provides insights into their effectiveness and ease of use for educators. This includes a comparative analysis of GIFT, AIKEN, and Moodle XML formats.

• Impact of FTP: the research focuses on the influence of FTP on educators’ ability to create and utilise diverse question types, as well as its effect on the frequency of multimedia use in assessments. FTP also makes it easy to use images, which can be attractive to students.

• STEM vs. Non-STEM Practices: the study also assesses whether there are differences in assessment question types used by educators from STEM and non-STEM disciplines.

This approach provides a comprehensive overview of current LMS trends and Moodle’s evolving role in educational technology. It emphasises the impact of technological advancements, such as FTP, on assessment practices and aims to offer actionable insights for enhancing online assessments. Considering the key points, the work has addressed the following research questions (RQs):

• RQ1: How have educators’ assessment methods changed before and after COVID-19?

• RQ2: Are there differences in the assessment question types used by STEM and non-STEM educators?

• RQ3: Have the assessment question types used by educators changed before and after adopting FTP?

• RQ4: Do educators use images more frequently in different question types after implementing FTP?

2. Methods

The method followed in this work is shown in Figure 4. First, FTP is developed for the Moodle framework. Its main features are introduced in Section 2.1. Next, the case study sample to test the method is presented in Section 2.2. In Section 2.3, the characteristics of the surveys conducted before the training to validate the sample are described. In Section 2.4, the characteristic of the delivery of the courses/workshops are presented. It should be noted that the method has been presented in training courses and workshops at conferences to educators from different areas of knowledge. Next, the surveys conducted after the training/explanation of FTP are presented in Section 2.5. Finally, in Section 2.6 the evaluation of the responses given by educators is presented.

A brief description of the methodology followed to design the survey on the use of questions in Moodle is presented in this section.

• Definition of Objectives: Initially, the objectives were established by considering what information was needed and its intended purpose.

• Population Identification: Initially, this work aimed to include educators from all educational cycles. However, only university instructors participated, as very few secondary school teachers attended the courses/workshops. Consequently, this portion of the population was excluded due to their negligible influence on the sample.

• Question Type Selection: The next step involved selecting the types of questions to be included in the survey. Originally, it was planned to focus solely on the question types that can be configured using the FTP application. However, additional question types within the quiz creation category were incorporated to assess their usage by educators. This inclusion led to significant improvements in the FTP application.

• Question Design: With the gathered information, the questions were designed with the following considerations: they needed to be clear and concise; each question had to directly relate to the survey objectives; questions should be neutral to avoid biasing responses; the questionnaire should combine various question types for a more comprehensive view; and measurement scales, such as Likert scales and/or numeric scales, should be used to evaluate attitudes and opinions.

• Survey Structure: Once the questions were designed, the survey was structured to explain its purpose and ensure participant anonymity. The questions were organized logically, starting from general inquiries, and progressing to more specific ones. It concluded with an expression of gratitude for participation and space for additional comments.

• Pilot-Test: Before publishing the survey, a pilot test was conducted with a small group to identify potential comprehension issues or ambiguities. Adjustments were made to both the questions and survey structure based on feedback from this test.

• Survey Application: Two similar surveys were designed: one to be carried out before the FTP course/workshop and another at the end, to enable the comparison of results. Once the surveys were finalized, they were implemented using Google Forms [27]. The first survey was distributed to the instructors registered for the course/workshop prior to the first class through the Moodle platform. The second survey was sent via the platform after the course concluded.

• Data Analysis: Data analysis included both quantitative and qualitative methods.

• Additional Considerations: A concise survey design was employed to maintain participant interest, and incentives were offered to enhance response rates. Additionally, all measures were taken to respect participant privacy throughout the process.

The surveys aimed at the continuous improvement of the FTP application, with objectives defined. First, information about the user profile of the application was sought by inquiring about their teaching discipline and age range. This would help identify the type of instruction and the kinds of questions/problems the educators would pose. Furthermore, given the varying complexity of question types that can be created in Moodle, it was important to understand the types of questions each user typically utilized before mastering the FTP application. This would allow for an evaluation of whether any differences existed in the types of questions used pre- and post-exposure to the application, either due to their relevance to the subjects taught or simply due to ease of configuration. Lastly, respondents were asked to provide suggestions for improvement and their impressions of the application.

2.1. Development of FTP for the Moodle Framework

FTP for Moodle is a tool that assists educators in creating questionaries to evaluate students. Hence, the main task of educators is to focus on formulating the questions they wish to ask. Within this plugin, a screen is presented for each type of question, displaying all necessary fields to be filled out. This enables the rapid and efficient creation of a substantial number of questions of the most common types in Moodle. Thus, with FTP, it is possible to save significant time in the development of the students’ evaluation.

Once the educator has composed the questions, FTP generates an XML file containing all the created questions or a specified range. This entire process occurs within the application, without the need for an internet connection or access to the Moodle platform. The only instance requiring a connection is when uploading the question bank to Moodle, which can be accomplished by importing the XML file into the platform. The plugin allows for the simultaneous creation of up to 100 identical questions with varying data.

The primary advantages of using FTP include (i) time savings, as it significantly reduces the time spent on question creation; (ii) consistency, as it ensures greater uniformity in the formulation of questions and in the review of their content (particularly for questions involving mathematical formulas); and (iii) efficiency, enabling the rapid generation of large quantities of questions. A screenshot of a problem developed for Engineering courses in shown in Figure 5. The view of the FTP application with the XML code of the problem generated to be imported in Moodle can be seen in Figure 6. A screenshot of the problem presented in the Moodle platform is presented in Figure 7. Readers are referred to [6] for further information regarding the functionality of the plugin.

2.2. Case Study

The training courses for the FTP application were conducted with university professors from various universities and all disciplines. The courses have been delivered to educators at Spanish universities. Spain is a country that extensively uses the Moodle platform. This is evidenced by Moodle’s statistics, which indicate that Spain has the highest number of registered sites worldwide [28]. Given its size, this results in a significantly high percentage of users compared to other countries.

Ten courses were conducted at different universities, and three were offered as online workshops at teaching innovation conferences. Of the university courses, the first was conducted in March 2021, and the last was in November 2023. A total of 352 responses were obtained to the pre-course surveys and 150 responses to the post-course surveys. For the first course (March 2021), there were so many applications that three consecutive sessions had to be held. For the second, two consecutive sessions were conducted. The others have been one per academic year/university.

To try to make the sample as extensive as possible, the use made by university professors (from all disciplines) of the different question types that can be used in Moodle to evaluate students online was analysed. When the course/workshop on FTP was offered, users were not informed that they would need Excel and Windows to use FTP. One of the objectives was to have a more comprehensive sample of teachers. The other objective was for them to become familiar with FTP; some mentioned that they would ask colleagues for a computer with Windows and Excel to use it.

For this research, surveys were conducted before and after the course with all the teachers who received the training. Table 3 shows the different areas of knowledge to which the surveyed teachers belonged. The first column indicates whether or not the area is considered STEM. The following columns show the number of teachers who completed the survey before and after the course along with the percentage of each. In the last column, to see if the sample of both surveys is valid, the difference between the two surveys is portrayed. Regarding the field of knowledge, when comparing both surveys, the most remarkable difference is in Legal Sciences, with 3.1%, which does not indicate a significant difference.

Table 4 shows the different age ranges of the surveyed teachers. The first two columns display the total number of teachers who completed the survey before the course and the percentage of each age group. The following columns show the number of teachers who completed the survey after the course and, in the last column, the difference between both surveys. Regarding the age range comparison, the maximum difference occurs in Legal Sciences, with a 2.5% difference, indicating no significant variation. Therefore, based on this data, it can be considered that both surveys are comparable.

2.3. Pre-Course Surveys

Prior to the FTP workshop or course, a survey was conducted among the instructors enrolled to understand the question types they used. Later, after the course, educators were asked the same questions to compare the sample and to assess their preferences. Ultimately, the goal is to analyse how the pandemic has influenced the use of LMS e-learning platforms for teaching. The following questions were asked at the beginning of the training as follows:

• The following two questions were asked to verify the validity of the sample, ensuring that not only Windows and Excel users (necessary for using FTP) enrolled in the course:

  • ○. What operating system do you normally use on your computer?

  • ○. What software do you normally use to create spreadsheets?

• The following question was to determine if there is a difference between STEM and non-STEM educators:

  • ○. What is your teaching area (if you teach in more than one, indicate the area with the most teaching hours)?

• The following questions were to determine if teachers were Moodle questionnaire users before and after COVID-19 and if Moodle was easy to use for them. Also, it was intended to understand what question types they used or why they did not use Moodle questionnaires:

  • ○. Before the COVID-19 lockdown, how often did you use the Moodle platform to create quizzes for your students?

  • ○. Indicate the question types you used (70% of respondents)/Indicate why you did not use Moodle quizzes (30% of respondents).

• The following questions were only for those who used Moodle quizzes (70%):

  • ○. Entering questions on the Moodle platform is easy and quick (respond according to your agreement or disagreement with this statement).

  • ○. Indicate the question types you used before COVID-19 (Multiple Choice, True/False, Matching, Short Answer, etc.).

• The following questions were for all the educators. Only 9 out of 352 did not use questions after COVID-19:

  • ○. Indicate the question types you used after COVID-19 (Multiple Choice, True/False, Matching, Short Answer, etc.).

• Using images in Moodle often results in losing the link to the images when a course or question bank is cloned. Therefore, the following was asked:

  • ○. When creating Moodle questions, have you ever used images and/or videos for the question stem or possible answers?

2.4. Delivery of the Courses/Workshops

The training courses on the FTP application were conducted with university professors from various universities and all disciplines. Nevertheless, they were conducted at universities that use Moodle. They were very well received, as many lecturers had only used Moodle’s question types when the pandemic arrived. Ten courses were conducted at different universities, and three were offered as online workshops at teaching innovation conferences.

In the training courses, the work environment of the application was shown, as well as the fields to fill out for different question types (including CLOZE type without having to know how to code them), exporting the questions to use them with a game, and tricks for generating many identical questions but with different data.

Additionally, some Excel tricks were taught, as the tool is based on this program. Many questions can be generated quickly and easily, including images, having complete control over them. Furthermore, these questions can also be exported in text format, so if you also need to create written tests with this type of content, you can use the same question bank for written exams.

Finally, by using this application, teachers will be able to keep their question banks organised both in the Virtual Campus and in their computer files, as the application greatly facilitates organisation.

2.5. Post-Course Surveys

These surveys were conducted to understand and to check if the lecturers had changed their views on quizzes after learning about FTP. The goal was to understand whether they find it challenging to use the Moodle platform to create questions and what question types they plan to use once they are familiar with the application. Although fewer teachers responded to the post-course survey compared to the pre-course one, the following sections demonstrate that this sample can be considered valid. The following questions were asked after the training as follows:

• To check if the educators responding to the second survey were not just those who could use FTP (Windows and Excel) as well as the question types they used before, the following questions were asked again:

  • ○. What operating system do you normally use on your computer?

  • ○. What software do you normally use to create spreadsheets?

  • ○. What is your teaching area (if you teach in more than one, indicate the area with the most teaching hours)?

  • ○. Before knowing the FastTest PlugIn application, how often did you use the Moodle platform to create quizzes for your students?

  • ○. Indicate the question types you used before knowing FTP (Multiple Choice, True/False, Matching, Short Answer, etc.).

• The following questions were asked to check the question types they plan to use after the course and if they find it easy to use images with FTP:

  • ○. Entering questions on the Moodle platform using FastTest PlugIn is easy and quick.

  • ○. Indicate the question types you plan to use now that you have the FastTest PlugIn application (Multiple Choice, True/False, Matching, Short Answer, etc.).

  • ○. When creating Moodle questions, have you ever used images and/or videos for the question stem or possible answers?

  • ○. Do you think it might be interesting to use images in Moodle quizzes using FastTest PlugIn?

• To try to improve the application, the final question was

  • ○. Here, you can write what you think of the FastTest PlugIn application, if you would recommend it, and if it meets your expectations.

2.6. Analysis of the Responses

At the end of each course, the questions were analysed, especially the last one, to introduce improvements in the application and the course training. Both statistical and qualitative analyses were carried out. A statistical analysis was conducted to evaluate changes in the use of various question types before and after exposure to the FTP training. The McNemar test [29] was employed to compare the proportions of responses before and after the training, as the data involved paired observations (the same participants responded before and after the intervention). Using Equation (1), the McNemar test specifically compares the proportion of teachers who switched from using a question type to not using it (b) with those who switched from not using it to using it (c).

χ² = (b − c)²/(b + c)(1)

where

b refers to the number of teachers who used the type of question before but not after the training (i.e., they stopped using this type of question after the training).

c refers to the number of teachers who did not use the type of question before but used it after the training (i.e., they started using this type of question after the training).

In this case, the null hypothesis (H₀) was defined as follows: The training course has no impact on the variation in the types of questions used by the educators. The contingency coefficient (C) was calculated as a measure of effect size. C indicates the strength of the association between variables, ranging from 0 (no association) to 1 (perfect association). It is calculated using Equation (2),

C = √(χ²/(N + χ²))(2)

where

χ² is a measure of the difference between the observed and expected frequencies of the variables.

N corresponds to sample size.

The responses and their corresponding analysis are presented in the results section of this work. Some of the improvements introduced are

• Locking cells to prevent invalid data entry for Moodle, according to the question type.

• Warning messages if data is missing to configure the question properly.

• Inclusion of special codes to allow bold text, coloured text, and white spaces without HTML knowledge.

• Inclusion of Essay and Cloze question types.

• Addition of a form to generate Cloze questions without needing to know how they are coded.

• Inclusion of unprotected cells to create multiple questions with random data using the “RANDBETWEEN (bottom, top)” function.

• Inclusion of the possibility to enter equations in HTML format.

• Translation capability into any language using Google Translate.

• Ability to export Multiple-Choice questions to a game.

• Inclusion of drag-and-drop onto text question types.

3. Results

This section analyses the responses obtained from the questionnaires conducted before and after the FTP application course. As the surveys were voluntary, not all teachers who took the course responded to the questionnaires; a total of 352 responses were received for the initial and 150 responses for the final questionnaires. Hence, to verify the validity of the results obtained, a validation of the sample is first carried out. Later, results obtained before and after the training received by educators are presented and analysed.

3.1. Validation of the Sample

First, as indicated in the methodology, it is necessary to verify the validity of the sample. To this end, the initial questions in the pre-course and post-course surveys have been compared. This allows us to determine if the responses from the samples are comparable. It should be noted that the courses have been delivered to educators at Spanish universities.

Regarding the percentage of STEM and non-STEM teachers: in the initial survey, 129 out of 352 educators were non-STEM teachers, which equals 37% of the sample. In the final survey, 55 out of 150 educators were non-STEM teachers, which also interestingly equates to 37% of the sample. Thus, concerning the type of teaching provided by the educators enrolled in the courses, it is checked that the comparison between both questionnaires can be considered valid.

Figure 8 presents the percentage of operating systems software used among instructors. It can be seen that 79% of educators use Windows, while 21% use another operating system. This indicates that the questionnaires were answered by educators who may be unable to use FTP, as it only functions on Windows. When differentiating between STEM and non-STEM instructors, this data vary slightly (Windows usage increases to 83%) but not significantly.

In the questionnaire conducted after the FTP course, the percentage of users of other operating systems is lower than in the pre-course survey (from 21% down to 13%). Likely, those who cannot use FTP on other systems did not respond. However, it is confirmed that the response rate is still adequate, as there are instructors who are not Windows users and yet completed the post-course survey.

In [30], the market share of desktop operating systems worldwide from April 2019 to April 2020 is shown. The data on the use of PC operating systems in Spain (approx. 80%) are also presented.

Comparing the survey data with these figures, the results are rather similar, confirming the validity of the sample.

In Figure 9, the percentage of type of spreadsheet software used is presented. A similar trend is observed in this case compared to the previous analysis. The percentage of educators who did not use Excel as their spreadsheet software decreased from 13% to 9%. This reduction indicates that there are instructors who completed the post-course survey despite not being Excel users. Therefore, the sample also includes users who do not employ the necessary tools to use FTP.

Next, to analyse the need for an application that helps the instructor enter questions into the Moodle platform, instructors were asked about their usage of question types before and after COVID-19. Concerning the question, Before the COVID-19 lockdown, how often did you use the Moodle platform to create quizzes for students? It can be seen in Figure 10 that 48% of the instructors indicated they used it frequently or relatively frequently. A total of 21% responded that they used it infrequently, and 31% did not use it at all (110 out of 352). No significant difference was detected between STEM and non-STEM teachers.

Of the teachers who responded that they did not use Moodle quizzes at all, to the question Indicate the reason why you did not use Moodle quizzes, 50% replied Because I did not know about them or did not know how to configure them/Because preparing questions on the Moodle platform is too laborious. 21% responded Because I was not teaching. 22% said Because it is better to do them in person to avoid fraud/Because I did not see their usefulness for my subjects/students. The remaining 7% cited Other reasons.

For teachers who responded that they did use Moodle quizzes (regardless of how frequently), they were asked if Entering questions on the Moodle platform is easy and quick, ranging from 1 for “strongly disagree” to 10 for “strongly agree”. As shown in Figure 11, 65% rated it between 1 and 5, while 35% rated it above 5. More specifically, 36% rated it between 1 and 4, 39% between 5 and 6, and only 25% between 7 and 10. This demonstrates the value of having tools that assist in entering questions into Moodle.

Additionally, of these 110 teachers, when asked to Indicate the question types you use after COVID-19, only 16 teachers (15%) responded None. This clearly indicates that Moodle’s use of quizzes has increased significantly after the pandemic. For these reasons, it is necessary to provide training on this topic and use applications that facilitate the input of questions into the Moodle platform.

Once the data have been analysed to verify the validity of the sample, the following sections analyse the rest of the responses regarding Moodle quizzes before and after the training.

3.2. Results Prior to Training

After the sample validation is confirmed and the data from the two questionnaires are proved comparable, the results of the questions regarding Moodle quizzes are presented below. Figure 12 shows the percentages of question types used before COVID-19. It can be observed that the most commonly used question types are Multiple Choice, True/False, Matching, and Short Answer. In addition, no differences in the results are obtained between STEM and non-STEM instructors.

Figure 13 shows the percentages of question types used after COVID-19. It is observed that the most used types are the same, but all of them have seen a substantial increase in usage percentage.

Figure 14 illustrates the increase for each type, with a 25% increase for Multiple Choice questions, 14% for True/False, 16% for Short Answer, and 9% for Matching questions.

Finally, users were asked, Have you changed your way of using quizzes in Moodle?, to which 39% indicated No change, while 61% responded Yes. Based on everything analysed so far, it can be affirmed that COVID-19 has changed the way of working with Moodle question banks.

3.3. Results Post Training

This section presents the results of the questionaries carried out after the training on using Moodle question types, that is, before and after becoming familiar with the FTP application. This question was repeated after the course to validate the sample and to see whether the lecturers who responded to the survey after the course were using the same question types as they did before the course. Figure 15 shows the percentages of question types used before familiarising themselves with FTP.

The results presented in Figure 15 are similar to those obtained previously and presented in Figure 13. Figure 16 compares the differences in the use of question types before the training received on FTP. Comparing both surveys regarding the use of question types can be valid, and the sample can be considered valid.

Figure 17 shows the percentage of question types instructors plan to use after familiarising themselves with FTP. It can be observed that all types have seen an increase in their usage. This increase is more apparent when the differences in usage before and after FPT are shown (Figure 18). It is observed that all question types have increased in usage percentage except for Essay questions. This might be due to the fact that this is the only type that is not automatically evaluated. The increased use of the Cloze (22%) question type (which requires some coding knowledge if FTP is not used) is particularly noteworthy. Before the training, only 11% of the educators used it, which is a very high increase in the percentage compared to the typical usage of this question type.

Additionally, in this case, there is a difference in usage between STEM and non-STEM educators. A total of 22% of teachers indicated they would use this question type after learning about FTP. When considering STEM instructors specifically, who can create more complex problem-solving questions, the percentage rises to 26%. This is likely due to the nature of these questions, which are highly useful for technical problems where multiple questions can be asked about different data within a single problem with variable inputs. This is less common in non-STEM studies. There is also a noticeable increase in the usage of Missing Word questions. Configuring these questions in Moodle is complex, and FTP significantly facilitates their creation. This is likely the reason for their increased usage.

Instructors were also asked the following question: Entering questions into the Moodle platform using FastTest PlugIn is easy and fast. Figure 19 shows the percentages of responses, with 1 indicating strongly disagree and 10 indicating strongly agree. Most instructors believe that using FTP helps generate questions in Moodle. Only 14% responded between 1 and 5. When differentiating between STEM and non-STEM instructors, the percentage of STEM educators rated lower is inferior, dropping to 11%.

Additionally, most instructors who rated 4 or lower attended the training course during its initial edition when the application was still experiencing minor issues. These problems have been resolved in subsequent versions. Nevertheless, the training received by instructors on FTP enabled them to learn how to create different question types. They were encouraged to use different question types, especially Cloze, which typically causes more apprehension due to coding. The use of this question type may have increased mainly because FTP automatically generates the coding, eliminating the need to understand how to input codes.

In regard to working with images in Moodle, issues can arise when using the same questions in different courses. If the design is not appropriately configured, image links can potentially be broken. For this reason, users were asked When creating Moodle questions, have you ever used images and/or videos for the statement or possible answers? In this case, as shown in Figure 20, 50% of the instructors were already using images in Moodle before knowing FTP, but this percentage varies between STEM and non-STEM fields. STEM instructors use images more frequently in Moodle (57%), whereas non-STEM teachers use them less (38%).

The FTP application aims to simplify the use of images in quizzes (among other functions) to prevent link issues. To verify if this objective was achieved, users were asked Do you find it easy to insert images and/or videos using the FastTest PlugIn application? As shown in Figure 21, 63% of the instructors answered Yes. When focusing on STEM educators, this percentage rises to 68%. Only 15% responded No, and 22% indicated they had not yet tried it.

The use of images by teachers has increased significantly. Furthermore, the increase among STEM educators is much greater when distinguishing between STEM and non-STEM instructors. This is likely because generating images and managing hyperlinks may be more challenging for non-STEM teachers. Lastly, teachers were asked Do you think it could be interesting to use images in Moodle quizzes using FastTest PlugIn? As shown in Figure 22, 94% of teachers responded Yes.

4. Analysis of the Results

4.1. Statistical Analysis of the Results

The analysis has revealed statistically significant differences in the use of several question types, including Multiple Choice, True/False, Matching, Missing Word, Cloze, and Drag and Drop. This suggests that the training on the use of FTP has had a noticeable impact on the shift towards these types of questions. In Table 5, the results from the McNemar test—χ²—statistical analyses are listed. The contingency coefficient indicates that the most pronounced changes can be observed in Matching and Missing Word questions, while changes in Short Answer and Essay questions were less significant. The results suggest that the training intervention had a positive impact on the shift towards using certain types of questions, particularly Matching and Missing Word. This indicates that FTP facilitated the adoption of these types of questions.

The data have been further analysed by separating the responses into two groups: STEM and non-STEM teachers. This separation allowed for an assessment of whether differences existed in the way teachers from each group responded to the training. Table 6 lists the McNemar test results on the question types use by disciplines. Significant changes were observed in the use of almost all question types in the STEM group. Matching and Missing Word questions exhibited the strongest associations with the training, suggesting that STEM teachers were particularly responsive to the new tools and strategies. However, no significant changes were observed in the use of any question type, indicating that the training has had a limited impact on non-STEM teachers.

4.2. Qualitative Insights of the Results

Of the 352 teachers who responded to the pre-course survey, 174 answered the question “Here you can write down what you expect from the course”. Most participants expect that the FTP course will help them improve and optimize the creation of quizzes in Moodle, especially in terms of efficiency, agility, and ease of managing large question banks. Many highlighted the importance of automating the process of creating and varying questions, which would save time when preparing quizzes for multiple groups or exams. Others focused on gaining more control over the questions, particularly in how to organize, store, reuse, and modify question banks, as well as generating questions with automatic variants.

Of the 150 teachers who completed the post-course survey, 65 responded to the question “Here you can write down what you think of the FTP application, whether you would recommend it, and whether it was what you expected”. Most participants provided positive feedback on FTP, emphasizing its usefulness in simplifying the creation and management of quizzes in Moodle. The course has generally exceeded expectations, with many participants stating they would recommend the tool to colleagues. The application’s intuitive nature, ability to streamline quiz creation, and the capacity for organizing and managing large question banks were frequently highlighted. However, a few users mentioned challenges, such as initial learning curves, bugs, or limitations with specific operating systems, particularly MacOS.

In Appendix A, the most representative comments from the pre- and post-training surveys are grouped.

4.3. Discussion on the Research Questions

This section presents the responses to the research questions raised at the beginning of this research, after the analysis of the results was carried out.

• RQ1: The transition in assessment methods due to COVID-19 has been substantial, as educators adapted to unfamiliar approaches to meet new challenges in evaluating the students’ academic performance.

• RQ2: There is no significant difference in the question types of assessment used between STEM and non-STEM instructors. It could be noted that the only differences observed are in the use of Cloze quizzes and images.

• RQ3: The introduction of FTP has significantly transformed the use of questions. Prior to the training and implementation of FTP, the GIFT format predominated, restricting question types to Multiple-Choice, True/False, Matching, Short Answer, Numerical and Essay. With FTP, the ease of generating questions via XML dramatically expanded the available options in Moodle, motivating educators to explore and use a broader range of standard question types.

• RQ4: After using FTP, educators use images in their quizzes more frequently. FTP facilitates the insertion of images, which previously may have been perceived as more complex, and many educators had not considered using images before.

These insights illustrate how FTP has positively impacted educators’ assessment practices, expanding their capabilities and options within Moodle. With FTP, feedback can be easily given to the student, which can help them improve their learning. This can help the teacher configure and adapt the assessment questions to each student. In addition, using images makes the assessment questions more attractive to students, and therefore, they will show more interest.

5. Conclusions and Future Works

5.1. Particularities of This Research

This study focused on the evolution of online assessment practices through Moodle, exploring Moodle’s question types and import formats. In addition, the impact of FTP on educators’ assessment methods during and after the COVID-19 pandemic was studied. It has been concluded that the pandemic has marked a shift in assessment methods. Before the pandemic, 47% of educators used Moodle quizzes frequently, while 31% responded that they had never used it. After the pandemic, 86% of the instructors started using Moodle question types for their teaching.

The validation of the sample was carried out by checking the proportion of STEM and non-STEM educators in each of the surveys carried out, as well as the age range. Regarding field of knowledge, there is not a significant difference. The maximum difference (field of knowledge is 3.1% and age range is 2.5%) indicates no significant variation.

FTP notably impacted the question types used in Moodle. Before the training was received, the GIFT format was predominantly used for question imports, which had a limited range of question types. After the training on FTP usage, there was a significant change as educators started exploring a more comprehensive range of question formats. In this case, 15% of educators who initially did not use Moodle quizzes started using them after becoming familiar with FTP. The introduction of FTP has also facilitated the integration of images and videos into Moodle quizzes. Prior to the training, 50% of educators had used images in their quizzes, with STEM ones using them more frequently at 57%. After FTP training, 63% started inserting multimedia elements, with 68% of STEM educators reporting being very satisfied. Educators have also been observed to use a broader range of question types after the training, including Cloze questions, which increased in usage. In this case, only 11% of educators used Cloze questions before the training. This percentage increased to 22% among STEM educators after the training, pointing out that FTP had a positive impact on expanding the use of question format options.

5.2. Main Findings and Implications for Practice

This work demonstrates that FTP has significantly enhanced Moodle’s assessment capabilities by simplifying the creation and integration of diverse question types and multimedia elements. The positive response from educators, 94% of them finding the use of images in quizzes via FTP interesting, underscores the tool’s effectiveness in improving Moodle-based assessments.

Concerning the statistical analysis of the results, it can be concluded that the training has been highly effective for STEM teachers in promoting the use of specific types of questions (especially Matching and Missing Word). However, it has had little impact on non-STEM teachers. Future research should focus on understanding why these differences occurred and develop tailored interventions to support non-STEM educators. Additionally, the larger sample size of STEM teachers should be considered when interpreting these results. Also, the results suggest that the training intervention has had a positive impact on the shift towards using certain types of questions, particularly Matching and Missing Word. This indicates that FTP facilitated the adoption of these types of questions.

As a result of this research, the primary recommendation for Moodle quiz users, based on the findings of this study, is that while the use of Cloze-type questions (nested responses) has significantly increased, it still represents a very low percentage of overall usage relative to its capabilities and versatility. All Moodle quiz users (both STEM and non-STEM) are encouraged to experiment with creating this type of question using FTP, as no coding knowledge is required.

In conclusion, FTP has been well-received by participants, who appreciate its ability to simplify quiz creation, automate processes, and organize question banks effectively. While a few participants noted technical issues, the majority found the tool highly valuable and expressed their intention to recommend and continue using it in their teaching practices.

The FTP course for Moodle has as its main expectations the improvement of efficiency, speed, and ease in the creation and management of quizzes. Participants are looking for a tool that allows them to automate the generation of questions and question banks in order to optimize their time and effort in developing exams and continuous assessments while minimizing errors in Moodle and improving their teaching experience.

Additionally, since FTP facilitates the incorporation of images into questions—an aspect that is particularly appealing to today’s students—it is recommended to make greater use of images in quiz questions.

5.3. Limitations and Future Works

The courses were delivered only to educators at Spanish universities. The authors would like to indicate that emails from users from other countries have been received with inquiries about the application. However, they are not part of this study, as they did not complete the survey. During workshops on the application at international conferences, the survey was made available, but fewer than five participants responded, which is why their data have not been included in this research.

It was found out that the effectiveness of FTP might have varied depending on the timing and quality of the training received. The study notes that some instructors who rated FTP less favourably attended earlier training sessions when the application had unresolved issues. Variability in training quality and the evolution of the FTP application could influence educators’ experiences and perceptions. The study provides a snapshot of the immediate effects of FTP on Moodle assessment practices. It does not address the long-term sustainability of these changes or whether the observed improvements in question types and multimedia integration will persist over time. These limitations suggest areas for future research and highlight the need for a more comprehensive understanding of how technological tools like FTP influence educational practices across different contexts and over more extended periods.

As future research work, it could be interesting to dive into the application of artificial intelligence (AI) in engineering courses, as stated in [31,32]. In the context of FTP, with the development of AI, specifically ChatGPT, future work will analyse how AI influences question generation. It could also be interesting to explore whether ChatGPT can replace FTP. It has also been checked that both images and Cloze questions can be very handy for non-STEM disciplines. Therefore, future training courses will include more examples of Cloze questions for these disciplines, incorporating images (“a picture is worth a thousand words”).

Author Contributions

M.H.-G.-M.: Conceptualisation, Methodology, Validation, Investigation, Data curation, Writing—original draft preparation, Funding acquisition. M.-V.R.-G.-C.: Writing—review and editing, Visualisation, Funding acquisition. All authors have read and agreed to the published version of the manuscript.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

The original contributions presented in the study are included in the article, further inquiries can be directed to the corresponding author.

Conflicts of Interest

The authors declare no conflict of interest.

Footnotes

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Figures and Tables

Figure 1. Example of a Moodle question in GIFT format: (1) Multiple Choice; (2) True/False; (3) Short Answer; (4) Numerical; and (5) Matching. Elaborated by the authors.

Figure 2. Example of a Moodle Multiple-Choice question in AIKEN format. Elaborated by the authors.

Figure 3. Example of Moodle’s Multiple-Choice question in XML format. Elaborated by the authors.

Figure 4. Flowchart of the methodology followed. Elaborated by the authors.

Figure 5. Example of a problem prepared in MS Excel for an Engineering course. Elaborated by the authors.

Figure 6. View of the FTP application with the XML code of the problem generated to be imported in Moodle. Elaborated by the authors.

Figure 7. Statement of the problem in Moodle (students’ visualization). Elaborated by the authors.

Figure 8. Percentage of operating systems used. Elaborated by the authors.

Figure 9. Percentage of spreadsheet software used. Elaborated by the authors.

Figure 10. Responses to the question before the COVID-19 lockdown, how often did you use the Moodle platform to create quizzes for students? Elaborated by the authors.

Figure 11. Responses to the question Entering questions on the Moodle platform is easy and quick. Elaborated by the authors.

Figure 12. Percentages of question types used before COVID-19. Elaborated by the authors.

Figure 13. Percentages of question types used after COVID-19. Elaborated by the authors.

Figure 14. Increase in use of question types before and after COVID-19. Elaborated by the authors.

Figure 15. Percentages of question types used before FTP. Elaborated by the authors.

Figure 16. Percentage differences in the use of question types between questionaries before the training received on FTP. Elaborated by the authors.

Figure 17. Percentage of usage of question types that instructors plan to use after familiarising themselves with FTP. Elaborated by the authors.

Figure 18. Differences in usage of question types before and after FTP. Elaborated by the authors.

Figure 19. Responses to the question Entering questions on the Moodle platform, using FTP, is easy and quick. Elaborated by the authors.

Figure 20. Responses to the question When creating Moodle questions, have you ever used images and/or videos for the statement or possible answers? Before knowing FTP. Elaborated by the authors.

Figure 21. Responses to the question When creating Moodle questions, have you ever used images and/or videos for the statement or possible answers? After knowing FTP. Elaborated by the authors.

Figure 22. Responses to the question Do you think it could be interesting to use images in Moodle quizzes using FastTest PlugIn? Elaborated by the authors.

Table 1

Question types available in Moodle and their description.

	Question Type	Description of Question Type	Can Be Config.Using FTP?
1	Multiple-Choice	with single or multiple correct answers	Yes
2	True/False	with true or false statements	Yes
3	Matching	where students match items from two columns	Yes
4	Short Answer	requiring brief text responses	Yes
5	Numerical	expecting numerical answers	Not exactly
6	Essay	requiring longer, manually graded responses	Yes
7	Calculated	involving calculations	Not exactly
8	Calculated Multiple Choice	with Calculated Multiple-Choice	Not exactly
9	Calculated simple	with simple calculated	Not exactly
10	Drag and drop into the text	requiring students to drag items into a text block	Yes
11	Drag and drop markers	where students drag markers onto an image	No
12	Drag and drop onto the image	where students drag items onto an image	No
13	Embedded answers (Cloze)	allowing multiple types of embedded answers	Yes
14	Random Short-Answer Matching	generated from existing Short Answer Questions	Not necessary
15	Select missing words	where students select missing words in a sentence	Yes
16	Description	used for providing descriptions within a quiz	Yes

Table 2

Question types supported by each format type.

Format Type	Aiken Format	Blackboard	Embedded Answers	Exam View	GIFT Format	Missing Word	Moodle XML	WebCT
Multiple Choice	X	X		X	X		X	X
True/False		X		X	X		X	X
Matching		X		X	X		X
Short Answer		X		X	X		X	X
Numerical				X	X		X
Essay		X		X	X		X	X
Calculated							X	X
Drag and Drop							X
Cloze		X	X				X
Missing Words						X	X
TOTAL	1	6	1	6	6	1	10	5

Table 3

Teachers (No. and %) who responded to the questionnaires by knowledge area.

Knowledge Area	STEM	Nº Teach.Before	% Teach.Before	Nº Teach.After	% Teach. After	Diff.
Experimental Sciences	Yes	59	16.8%	29	19.3%	2.6%
Life Sciences	Yes	15	4.3%	8	5.3%	1.1%
Medical and Health Sciences	Yes	30	8.5%	10	6.7%	1.9%
Engineering and Architecture	Yes	120	34.1%	48	32.0%	2.1%
Social Sciences	No	47	13.4%	16	10.7%	2.7%
Legal Sciences	No	22	6.3%	14	9.3%	3.1%
Economic and Business Sciences	No	22	6.3%	10	6.7%	0.4%
Humanities	No	19	5.4%	7	4.7%	0.7%
Teacher Training	No	7	2.0%	4	2.7%	0.7%
Languages	No	4	1.1%	1	0.7%	0.5%
Services	No	1	0.3%	1	0.7%	0.4%
Others	No	6	1.7%	2	1.3%	0.4%
TOTAL	-	352	-	150	-	-

Table 4

Teachers (No. and %) who responded to the questionnaires by age range.

Age Range	Nº Teach.Before	% Teach.Before	Nº Teach. After	% Teach.After	Diff.
<30	78	6.8%	37	6.0%	0.8%
30–39	91	22.2%	38	24.7%	2.5%
40–49	133	25.9%	56	25.3%	0.5%
50–59	26	37.8%	10	37.3%	0.5%
>59	24	7.4%	9	6.7%	0.7%
TOTAL	352	-	150	-	-

Table 5

McNemar test results on the use of question types. p-value refers to the area under the density curve of the χ² distribution.

Question Type	χ2	p-Value	Conclusion	C Coefficient
Multiple Choice	10.67	<0.05	Reject H0	√(10.67/(150 + 10.67)) ≈ 0.26
True/False	4.18	<0.05	Reject H0	√(4.18/(150 + 4.18)) ≈ 0.16
Matching	13.92	<0.05	Reject H0	√(13.92/(150 + 13.92)) ≈ 0.29
Short Answer	1.44	>0.05	Fail to Reject H0	√(1.44/(150 + 1.44)) ≈ 0.09
Missing Word	12.96	<0.05	Reject H0	√(12.96/(150 + 12.96)) ≈ 0.29
Cloze	8.41	<0.05	Reject H0	√(8.41/(150 + 8.41)) ≈ 0.23
Essay	5.76	>0.05	Fail to Reject H0	√(5.76/(150 + 5.76)) ≈ 0.19
Drag and Drop	12.96	<0.05	Reject H0	√(12.96/(150 + 12.96)) ≈ 0.28

Table 6

McNemar test results on the use of question types. p-value refers to the area under the density curve of the χ² distribution.

Group	Question Type	χ2	p-Value	Conclusion	C Coefficient
STEM	Multiple Choice	11.67	<0.05	Reject H0	0.26
STEM	True/False	5.45	<0.05	Reject H0	0.16
STEM	Matching	13.92	<0.05	Reject H0	0.29
STEM	Short Answer	1.44	>0.05	Fail to Reject H0	0.09
STEM	Missing Word	12.96	<0.05	Reject H0	0.29
STEM	Cloze	8.41	<0.05	Reject H0	0.23
STEM	Essay	5.76	>0.05	Fail to Reject H0	0.19
STEM	Drag and Drop	12.96	<0.05	Reject H0	0.28
NO STEM	Multiple Choice	2.25	>0.05	Fail to Reject H0	0.15
NO STEM	True/False	1.43	>0.05	Fail to Reject H0	0.12
NO STEM	Matching	1.69	>0.05	Fail to Reject H0	0.13
NO STEM	Short Answer	0.25	>0.05	Fail to Reject H0	0.05
NO STEM	Missing Word	1.44	>0.05	Fail to Reject H0	0.12
NO STEM	Cloze	3.36	>0.05	Fail to Reject H0	0.18
NO STEM	Essay	3.13	>0.05	Fail to Reject H0	0.17
NO STEM	Drag and Drop	0.25	>0.05	Fail to Reject H0	0.05

Appendix A

In this section, the most representative comments from the pre- and post-training surveys are grouped.

Table A1

Pre-training survey comments grouped by main areas of interest.

Area	Description	Comments
Optimization and speed in quiz creation	Most comments focus on the need to speed up and simplify quiz creation in Moodle, particularly through automating the question creation process and using question banks with automatic variations.	‘Increase operational efficiency in managing quizzes.’ ‘Reduce the time required to create quizzes in Moodle.’ ‘Facilitate the creation of question banks.’
Automation and variation of questions	Several participants highlight their interest in automating the creation of questions, especially by including variables to generate different versions of the same question automatically.	‘Automation in the import of questions.’ ‘Easily create questions that can generate random variations.’ ‘Generate multiple questions quickly.’
Efficient use of Moodle	Many participants want to improve their knowledge of Moodle to make better use of the quizzes and options available on the platform.	‘Learn new techniques to optimize my work with Moodle.’ ‘Know how to properly manage quizzes.’ ‘Deepen the use of Moodle.’
Reuse and management of question banks	Another group of comments refers to the need to reuse questions and manage question banks more efficiently, including the ability to organize and update questions more easily.	‘Being able to reuse questions from previous courses.’ ‘Improve the handling of question banks.’ ‘Being able to associate each question with information such as how many times I’ve used it.’
Ease of use and reduction of errors	Some comments focus on the need for tools that are easy to use, with fewer errors, and more intuitive so they can be integrated into their daily workflow.	‘A simpler way to create quizzes.’ ‘Learn how to use quizzes more easily.’ ‘Make the creation of quizzes easier.’

Table A2

Post-training survey comments grouped by main areas of interest.

Area	Description	Comments
Positive Feedback and Ease of Use	Many participants found FTP useful, intuitive, and helpful in simplifying quiz creation and managing large question banks. Several commented on the application’s ability to save time and effort when creating and organizing questions.	‘It’s a very useful and intuitive tool.’ ‘It simplifies the creation of all types of quizzes for Moodle.’ ‘The application exceeded expectations. Highly recommendable.’
Recommendations and Future Use	A significant number of participants mentioned they would recommend FTP to their colleagues. Many expressed plans to use it regularly in their teaching practice.	‘Yes, I would recommend it. Thanks for sharing your work.’ ‘I will use and recommend it.’ ‘I’ve already recommended it to colleagues.’
Surpassing Expectations	Several users indicated that the application had exceeded their expectations in terms of functionality and ease of use.	‘It was more than I expected.’ ‘It has exceeded my expectations and more.’ ‘A very pleasant surprise!’
Technical Issues and Suggestions for Improvement	Some participants mentioned technical difficulties, such as bugs or slow performance. Others pointed out areas for improvement, like better compatibility with MacOS or reducing the number of confirmation windows.	‘Unfortunately, it had many errors in its application.’ ‘The application is still in beta and needs improvement.’ ‘On MacOS, exporting content was problematic.’
Initial Learning Curve	While the tool was generally well-received, a few participants found the initial learning process challenging but recognized that the time investment was worthwhile once they became more familiar with the tool.	‘It takes some time to learn, but the results are immediate.’ ‘The app seemed complicated at first, but once you start working with it, it gets easier.’

References

1. Draskovic, D.; Misic, M.; Stanisavljevic, Z. Transition from Traditional to LMS Supported Examining: A Case Study in Computer Engineering. Comput. Appl. Eng. Educ.; 2016; 24, pp. 775-786. [DOI: https://dx.doi.org/10.1002/cae.21750]

2. Sato, S.N.; Condes Moreno, E.; Rubio-Zarapuz, A.; Dalamitros, A.A.; Yañez-Sepulveda, R.; Tornero-Aguilera, J.F.; Clemente-Suárez, V.J. Navigating the New Normal: Adapting Online and Distance Learning in the Post-Pandemic Era. Educ. Sci.; 2023; 14, 19. [DOI: https://dx.doi.org/10.3390/educsci14010019]

3. Keržič, D.; Alex, J.K.; Alvarado, R.P.B.; Bezerra, D.d.S.; Cheraghi, M.; Dobrowolska, B.; Fagbamigbe, A.F.; Faris, M.E.; França, T.; González-Fernández, B. et al. Academic Student Satisfaction and Perceived Performance in the E-Learning Environment during the COVID-19 Pandemic: Evidence across Ten Countries. PLoS ONE; 2021; 16, e0258807. [DOI: https://dx.doi.org/10.1371/journal.pone.0258807] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/34669757]

4. Gamage, S.H.P.W.; Ayres, J.R.; Behrend, M.B. A Systematic Review on Trends in Using Moodle for Teaching and Learning. Int. J. STEM Educ.; 2022; 9, pp. 1-24. [DOI: https://dx.doi.org/10.1186/s40594-021-00323-x] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/35096513]

5. Kumar, S.; Gankotiya, A.K.; Dutta, K. A Comparative Study of Moodle with Other E-Learning Systems. Proceedings of the 2011 3rd International Conference on Electronics Computer Technology; Kanyakumari, India, 8–10 April 2011; pp. 414-418.

6. Huerta, M.; Caballero-Hernández, J.A.; Fernández-Ruiz, M.A. Comparative Study of Moodle Plugins to Facilitate the Adoption of Computer-Based Assessments. Appl. Sci.; 2022; 12, 8996. [DOI: https://dx.doi.org/10.3390/app12188996]

7. Aydin, C.C.; Tirkes, G. Open Source Learning Management Systems in E-Learning and Moodle. Proceedings of the IEEE EDUCON 2010 Conference; Madrid, Spain, 14–16 April 2010; pp. 593-600.

8. Mutawa, A.M.; Al Muttawa, J.A.K.; Sruthi, S. The Effectiveness of Using H5P for Undergraduate Students in the Asynchronous Distance Learning Environment. Appl. Sci.; 2023; 13, 4983. [DOI: https://dx.doi.org/10.3390/app13084983]

9. Bit4learn Popular LMS Platforms in the World [Plataformas LMS Más Populares del Mundo]. Available online: https://bit4learn.com/es/lms/plataformas-lms-mas-populares/ (accessed on 19 July 2024).

10. McDaniel, M.A.; Little, J.L. Multiple-Choice and Short-Answer Quizzing on Equal Footing in the Classroom. The Cambridge Handbook of Cognition and Education; Cambridge University Press: Cambridge, UK, 2019; pp. 480-499.

11. Cigdem, H.; Korkusuz, M.E.; Karaçaltı, C. Gamified Learning: Assessing the Influence of Leaderboards on Online Formative Quizzes in a Computer Programming Course. Comput. Appl. Eng. Educ.; 2024; 32, e22697. [DOI: https://dx.doi.org/10.1002/cae.22697]

12. Jurgelaitis, M.; Čeponienė, L.; Čeponis, J.; Drungilas, V. Implementing Gamification in a University-level UML Modeling Course: A Case Study. Comput. Appl. Eng. Educ.; 2019; 27, pp. 332-343. [DOI: https://dx.doi.org/10.1002/cae.22077]

13. Gamarra, M.; Dominguez, A.; Velazquez, J.; Páez, H. A Gamification Strategy in Engineering Education—A Case Study on Motivation and Engagement. Comput. Appl. Eng. Educ.; 2022; 30, pp. 472-482. [DOI: https://dx.doi.org/10.1002/cae.22466]

14. Enders, N.; Gaschler, R.; Kubik, V. Online Quizzes with Closed Questions in Formal Assessment: How Elaborate Feedback Can Promote Learning. Psychol. Learn. Teach.; 2021; 20, pp. 91-106. [DOI: https://dx.doi.org/10.1177/1475725720971205]

15. Khan, S.; Khan, R.A. Online Assessments: Exploring Perspectives of University Students. Educ. Inf. Technol.; 2019; 24, pp. 661-677. [DOI: https://dx.doi.org/10.1007/s10639-018-9797-0]

16. Althunibat, A.; Alzyadat, W.; Almarashdeh, I.; Alsmadi, M.; Al Shawabkeh, A.O.; Abuhamdah, A.; Alzaqebah, M. Learning Experience of Students Using the Learning Management System: User’s Perspective on the Use of Moodle in the University of Jordan. Adv. Hum.-Comput. Interact.; 2023; 2023, pp. 1-11. [DOI: https://dx.doi.org/10.1155/2023/6659245]

17. Paturusi, S.; Chisaki, Y.; Usagawa, T. Development and Evaluation of Online Quizzes to Enhance Learning Performance: A Survey of Student Assessment through MOODLE in Indonesian National University. Proceedings of the International Conference on Information, Communication Technology and System (ICTS) 2014; Surabaya, Indonesia, 24 September 2014; pp. 211-216.

18. Campillo-Ferrer, J.M.; Miralles-Martínez, P. Effectiveness of the Flipped Classroom Model on Students’ Self-Reported Motivation and Learning during the COVID-19 Pandemic. Humanit. Soc. Sci. Commun.; 2021; 8, 176. [DOI: https://dx.doi.org/10.1057/s41599-021-00860-4]

19. Boettcher, K.E.R.; Fischer, M.; Hellmich, J. Case Study: Flipped Classroom with Gamification in a Hybrid Fluid Mechanics Course. Chem. Ing. Tech.; 2024; 96, pp. 1509-1515. [DOI: https://dx.doi.org/10.1002/cite.202300237]

20. Moodle TM Question Types-Moodle. Available online: https://moodle.org/plugins/browse.php?list=category&id=29 (accessed on 4 October 2024).

21. Huerta-Gomez-Merodio, M.; Fernández-Ruiz, M.A.; Requena-Garcia-Cruz, M.V. Using FastTest PlugIn for the Design of Remote and Hybrid Learning Environments to Improve the Engineering Skills of University Students. Eur. J. Educ.; 2024; 59, e12654. [DOI: https://dx.doi.org/10.1111/ejed.12654]

22. Smith, A.; McConnell, L.; Iyer, P.; Allman-Farinelli, M.; Chen, J. Co-Designing Assessment Tasks with Students in Tertiary Education: A Scoping Review of the Literature. Assess. Eval. High. Educ.; 2024; pp. 1-20. [DOI: https://dx.doi.org/10.1080/02602938.2024.2376648]

23. Sousa-Vieira, M.E.; López-Ardao, J.C.; Fernández-Veiga, M.; Rodríguez-Pérez, M.; Herrería-Alonso, S. An Open-source Platform for Using Gamification and Social Learning Methodologies in Engineering Education: Design and Experience. Comput. Appl. Eng. Educ.; 2016; 24, pp. 813-826. [DOI: https://dx.doi.org/10.1002/cae.21746]

24. Moodle TM Question Types-MoodleDocs. Available online: https://docs.moodle.org/404/en/Question_types (accessed on 18 July 2024).

25. Moodle TM Calculated Question Type. Available online: https://docs.moodle.org/404/en/Calculated_question_type#Available_functions (accessed on 29 July 2024).

26. Moodle TM Import Questions-Moodle. Available online: https://docs.moodle.org/4x/sv/Import_questions (accessed on 18 July 2024).

27. Google Inc. Google Forms. Available online: https://www.google.es/intl/es/forms/about/ (accessed on 5 May 2021).

28. Moodle TM Stats-Moodle. Available online: https://stats.moodle.org/ (accessed on 9 October 2024).

29. McNemar, Q. Note on the Sampling Error of the Difference between Correlated Proportions or Percentages. Psychometrika; 1947; 12, pp. 153-157. [DOI: https://dx.doi.org/10.1007/BF02295996] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/20254758]

30. Statcounter-GlobalStats Desktop Operating System Market Share Worldwide. Available online: https://gs.statcounter.com/os-market-share/desktop/worldwide/#monthly-201904-202004 (accessed on 19 July 2024).

31. Menekse, M. Envisioning the Future of Learning and Teaching Engineering in the Artificial Intelligence Era: Opportunities and Challenges. J. Eng. Educ.; 2023; 112, pp. 578-582. [DOI: https://dx.doi.org/10.1002/jee.20539]

32. Johri, A.; Katz, A.S.; Qadir, J.; Hingle, A. Generative Artificial Intelligence and Engineering Education. J. Eng. Educ.; 2023; 112, pp. 572-577. [DOI: https://dx.doi.org/10.1002/jee.20537]