THE USEFULNESS AND USABILITY OF MOODLE LMS AS EMPLOYED BY ZARQA UNIVERSITY IN JORDAN

Abstract

This research investigates the usefulness and usability of Moodle LMS, as employed by a case study university (Zarqa University), based on students' experience with Moodle LMS on desktop/laptop and mobile/tablet interfaces. A total of 320 students participated in this research in two stages. The results of this research showed that the students at Zarqa University used frequently only two features on Moodle on both its interfaces. The results of this research also showed that a large number of the students required the provision of 15 features on the local instance of Moodle which are not currently provided. Furthermore, the results showed that a large number of the students identified nine usability problems on the interfaces of Moodle, and a large number suggested nine improvements to the design of Moodle to make it more usable. Finally, this research suggests recommendations to improve the usefulness and usability of Moodle LMS as used by the case study university.

Headnote

ABSTRACT

This research investigates the usefulness and usability of Moodle LMS, as employed by a case study university (Zarqa University), based on students' experience with Moodle LMS on desktop/laptop and mobile/tablet interfaces. A total of 320 students participated in this research in two stages. The results of this research showed that the students at Zarqa University used frequently only two features on Moodle on both its interfaces. The results of this research also showed that a large number of the students required the provision of 15 features on the local instance of Moodle which are not currently provided. Furthermore, the results showed that a large number of the students identified nine usability problems on the interfaces of Moodle, and a large number suggested nine improvements to the design of Moodle to make it more usable. Finally, this research suggests recommendations to improve the usefulness and usability of Moodle LMS as used by the case study university.

Keywords: Moodle, Learning Management Systems, LMS, usability, students' perception, Zarqa University.

INTRODUCTION

Nowadays, Information Technology (IT) plays an important part in our lives and education is one of the main sectors which has been affected by IT. As a result, e-learning has become one of the most common approaches which is increasingly employed among academic institutions and universities. E-learning can be defined as an approach which uses computer and communication technologies to support and improve learning. E-learning is usually based on Learning Management Systems (LMSs), which are also called either course management systems or virtual learning environments (Kakasevski et al, 2008; Minović et al., 2008; Chung et al, 2013). LMSs are software systems which are designed to support the management of educational courses for students, either the traditional face-to-face classroom courses or distance education (Simonson, 2007; Mtebe. and Kissaka, 2015). LMSs improve both the effectiveness and efficiency of the education process in higher education institutions (Onacan M. and Erturk, 2016).

LMSs are either commercial software (such as Blackboard and WebCT) or free, open source software (such as Moodle). Commercial software systems are very powerful but they carry high licence fees (Melton, 2006), which not affordable by most academic institutions and universities. Alternatively, the free open source software does not come at such a high cost. An example of a popular open source LMS is Moodle, which is used widely in academic institutions and universities. Moodle is considered a strong competitor to the commercial LMSs and it is usually the first LMS to be chosen when a robust, secure and integrated e-learning system is needed without any licensing fees (Ivanc et al., 2012; Moodle, 2018).

Moodle is considered to be the most widely used learning management system in academic and enterprise institutions in the world. Specifically, it has 144,332,474 users in 228 countries (Moodle, 2018). Moodle has a simple interface and it is continually reviewed and improved to suit the needs of its users. Also, Moodle has been translated into more than 120 languages and it can be customised to meet its users' needs (Moodle, 2018). Despite the increasing use of Moodle LMS, research has shown that Moodle has usability issues from the point view of its various users (Kakasevski et al, 2008; Melton, 2006; Minović et al., 2008; Rosato et al., 2004; Senol et al., 2014; Ssekakubo et al., 2013; Thuseethan et al., 2014). Usability is "a quality attribute that assesses how easy user interfaces are to use" (Nielsen, 2012). It is one of the most important requirements of LMSs; it improves the effectiveness of students' learning and the overall learning experience (Althobait and Mayhew, 2016; Katsanos et al, 2012; Onacan and Erturkm, 2016; Orfanou et al, 2015). Various methods can be used to evaluate the usability of interfaces. These methods can be either user-based evaluation methods or evaluator-based methods. The user-based evaluation methods include methods which involve users; they aim to collect data on users' satisfaction with an interface (e.g. via questionnaires) or users' performance while interacting with the interface (e.g. user testing) (Hasan et al., 2012). Alternatively, evaluator-based methods include methods which involve evaluators in the process of evaluating and identifying usability problems on an interface (e.g. the heuristic evaluation method, pluralistic walkthroughs) (Hasan et al., 2012).

Despite the increasing use of Moodle LMS, there is a lack of research which investigates the usefulness and usability of Moodle LMS as used by a case study university from the viewpoint of students. Specifically, there is a lack of research which uncovers whether Moodle LMS supports the requirements of students and whether the features which are required are actually provided by the local instance of Moodle as used by the case study university. Also, there is a lack of research which investigates whether the desktop and mobile interfaces of Moodle LMS are usable from the viewpoint of students, or research which offers details concerning usability problems on both interfaces.

AIMS AND OBJECTIVES

The aim of this research is to investigate the usefulness and usability of Moodle LMS, as used by Zarqa University in Jordan, based on students' experience with Moodle LMS on desktop/laptop and mobile/tablet interfaces.

The specific objectives of the research are:

1. To ask 80 students, using a questionnaire, to identify the following: those features of Moodle LMS which are used and those which are required; usability problems and suggestions to improve the usability of Moodle LMS;

2. To analyse the results obtained from the questionnaire developed in Objective 1;

3. To develop a questionnaire based on the students' answers obtained from Objective 2;

4. To ask 240 students to rate the identified items in the questionnaire developed in Objective 3;

5. To suggest recommendations to create more useful and usable LMSs in Moodle LMS based on Objective 4.

LITERATURE REVIEW

Research has indicated that user-based evaluation methods are mostly used in testing the usability of learning systems (Ivanc et al., 2012). For example, Senol et al., (2014), Thuseethan et al. (2014), Al-Sarrayrih et al. (2010), and Rosato et al. (2004) used questionnaires to evaluate the usability of Moodle which was used in various universities from the perceptions of students. The results of these studies provided examples of usability problems with the interface of Moodle from the viewpoint of students. These included the following: not easy to use for the first time (Senol et al., 2014); inappropriate choice of colours (Senol et al., 2014); slow downloading of Moodle's pages (Senol et al., 2014); inconsistency problems (e.g. font size and colours) (Thuseethan et al., 2014); the complexity of using the system to log in (Thuseethan et al., 2014); the complexity of submitting assignments (Thuseethan et al., 2014, Rosato et al., 2004; Melton, 2006; Kakasevski et al., 2008); the lack of help included in the system (Thuseethan et al., 2014); the lack of error prevention and recovery (Thuseethan et al., 2014); faults in the internal search function (Thuseethan et al., 2014); not easy to recover when they made mistakes (Al-Sarrayrih et al., 2010); posting a discussion message (Rosato et al., 2004 ). However, the results obtained from Al-Sarrayrih et al. 's study (2010) showed that most of the students (73%) agreed that Moodle had all the functions and capabilities that they expected it to have.

Alternatively, Baytiyeh (2013) and Ivanović et al. (2013) also employed questionnaires but to investigate both students' and teachers' perceptions and use of Moodle in various universities. The results of these studies showed that the students and teachers had positive experiences with Moodle. For example, the results obtained from Baytiyeh's study (2013) showed that the majority of the users believed that Moodle was easy to use and it improved the communication between students and teachers. The results of the study conducted by Ivanović et al. (2013) showed that, despite the students being satisfied with Moodle, they provided suggestions to improve the quality of the teaching materials on Moodle including: presenting additional exercises with different difficulty levels; presenting examples of previous exams; and adding more tests and assignments for students. Also, the students suggested using their own local language for Moodle's interface instead of the English language interface

Melton (2006), however, employed the user testing method using Japanese graduate students to evaluate the usability of registering for Moodle and uploading an assignment in Moodle. The English language user interface of Moodle was used during the test. The results showed that the users did not face usability problems while registering for Moodle. However, the results showed that half of the students faced difficulties while submitting an assignment to Moodle. Also, it was difficult for the students to use the English language interface instead of a Japanese interface. This corresponds with Ivanović et al.'s (2013) research which stressed the importance of changing the interface of Moodle to be in the students' own language rather than using the English language interface.

Rather than employing user-based evaluation methods, Martin et al. (2008) employed the heuristic evaluation method, using experts to evaluate the usability of three main learning management systems, including Moodle. The results showed that the experts had positive experiences with Moodle. For example, they indicated that Moodle has the ability to help users recognise and recover from errors. The results, however, showed that Moodle had weaknesses regarding flexibility and efficiency of use.

Alternatively, Kakasevski et al.'s study (2008) employed both the heuristic evaluation method and user testing methods to evaluate the usability of Moodle from the perspectives of both students and teachers. The results showed that the students and teachers were more familiar with using the user interface in their own local language (Macedonian) instead of the English language interface. This is similar to the two studies presented above (Melton, 2006; Ivanović et al., 2013). Also, the results showed that Moodle had usability problems regarding the assignment and online chat features.

All the above studies evaluated the usability of the Moodle learning management system from either students', or both teachers' and students', point of view when using a desktop interface. Few studies were found in the literature, however, which considered the usability of Moodle on both desktop and mobile interfaces. Minović et al. 's study (2008), for example, evaluated the usability of Moodle on both desktop and mobile interfaces using the think aloud and questionnaire methods. The results showed that the students faced difficulties in performing the easiest tasks on Moodle using both desktop and mobile interfaces. However, the number of errors using the mobile interface was higher compared to those on a desktop interface.

Alternatively, Ssekakubo et al. (2013), using questionnaires, investigated students' experience with various interfaces (desktops, laptops, tablets, mobiles) used for accessing two learning managements systems, including Moodle. The results showed that the following LMS services were most desired and used by the students: assignments, announcements, resources, course outlines and the chat room. The results showed that the mobile interface for Moodle had inadequate design (e.g. it was slow to open some pages). The authors indicated that the mobile interface for Moodle had usability and compatibility problems but no examples of usability problems were presented in either of the studies of Minović et al. (2008) or Ssekakubo et al. (2013).

The literature showed that Moodle LMS is used widely in various universities but it still has usability problems which need to be considered to improve users' experience of it. However, there is a lack of research which investigates users' experience with Moodle in terms of explaining features that are required by the students and whether Moodle supports them or not. Also, there is a lack of research which investigates detailed usability problems on various interfaces of Moodle, including desktop and mobile interfaces. Finally, there is a lack of research which suggests guidelines for developing a more useful and usable Moodle LMS from the perspective of users.

METHODOLOGY

Data were collected in two stages using two groups of students in order specifically to achieve objectives one and four of this research, respectively.

Stage one:

A total of 80 undergraduate students who were registered on two courses related to human computer interaction and software testing at the Faculty of Information Technology at Zarqa University in Jordan were asked to participate. The students who participated in this study were given extra marks. However, their participation was voluntary.

A questionnaire was designed which aimed to gather data from the students regarding their experience of Moodle while using both desktop/laptop devices and mobile/tablet devices. It comprised two sections: Section 1 involved closed-questions which were designed to gather students' background information, Internet experience and Moodle experience while Section 2 included the following four open-ended requests:

1. List the features which you are using in Moodle.

2. List the features which you would like to use in Moodle but which are not supported by it.

3. List the weak design features in Moodle which prevent you from interacting with it successfully.

4. Suggest improvements to the design of Moodle to make it more usable.

The questionnaire was uploaded to Moodle as homework for the students who had registered for the two courses. The students were given two weeks as a deadline and they were asked to submit their answers to the questionnaire to Moodle. The students who used only desktop/laptop devices to access Moodle were asked to answer the questions in Section 2 (four open-ended questions) once, while the students who used both desktop/laptop devices and mobile/tablet devices to access Moodle were asked to answer them twice to gather data regarding their experience with Moodle using the various interfaces

Data obtained from the questionnaires were translated into English from Arabic. The data were analysed to uncover students' experience with the features and usability of Moodle. Descriptive analysis was used for Section 1 of the questionnaire to describe the characteristics of the students and their experience regarding the Internet and Moodle. The students' characteristics are presented in the Results Section. The analysis of the answers to each of the four open-ended questions in Section 2 followed the same procedure; the answers for each question were examined and categorised. They generated common sub-themes (sub-categories) which suggested corresponding themes (categories). The list of the identified themes and sub-themes that was generated from the analysis of each question is presented in the Results Section.

Stage two:

A total of 240 undergraduate students from four faculties: Faculty of Information Technology, Faculty of Business, Faculty of Nursing, and Faculty of Engineering at Zarqa University were asked to participate. Their participation was voluntary.

A questionnaire was developed based on the students' answers to the questionnaire that was developed in Stage one to confirm the data which were collected in that stage. It consisted of two sections: Section 1 involved closed-questions which were designed to gather students' background information, Internet experience and Moodle experience. However, Section 2 involved four parts corresponding to the four open questions which were given to the students in Stage one. Each section consisted of questions which were the sub-themes generated from the analysis of the students' answers to each of the four open-ended questions collected in Stage one. The students were asked to rate the identified sub-themes based on the degree of conformance to each statement based on a seven-point rating scale (Likert scale). In order to gather data from the students regarding their experience with Moodle in terms of the type of interface they used, the questionnaire was designed to include Section 1 and two copies of Section 2. The students who used only desktop/laptop devices to access Moodle were asked to answer Section 1 and Section 2 of the questionnaire, while the students who used both desktop/ laptop devices and mobile/tablet devices to access Moodle were asked to answer Section 1 and two copies of Section 2 to gather data regarding their experience with Moodle using the two interfaces. It is worth mentioning that the questionnaire was pilot tested using five undergraduate students from the same university where the study was conducted. The pilot study identified some ambiguity in the questionnaire so the results from the pilot test were taken into consideration and changes were made. Data were then collected over a three-week period.

Descriptive analysis was used for Section 1 of the questionnaire to describe the characteristics of the students and their experience regarding the Internet and Moodle. The students' characteristics are presented in the Results Section. Likert scores were calculated for each statement in Section 2 to describe students' confirmation to the sub-themes identified in stage one of the research. It is worth mentioning that, for the purpose of the analysis that used the Likert scale in this research, a Likert score of 1-3 was regarded as a negative response, 5-7 a positive response and 4 a neutral one. The Likert scores for the statements (sub-themes) and their explanations are presented in the Results Section. The version of Moodle LMS used by the university was 2.9. However, several plug-ins updates were installed by an administrator in the university to update it.

RESULTS

Students' Characteristics

A total of 71 students out of the 80 (88.75%) participated in Stage one of this study and uploaded answers to the questionnaire; 35 of them answered Section 2 of the questionnaire twice and expressed their experience with the Moodle while using both desktop/laptop devices and mobile/tablet devices. The majority of the students were in the age range of 18 to 22 years. Most of the students (70%) were male while females represented 30% of the participants. Half of the students were from the Software Engineering Department (65%). Students from other departments also participated, specifically from: Computer Science (16%); Computer Information Systems (12%) and Internet Technology (7%). The students who participated were also in their second (18%) third (52%) and fourth (30%) years of study. The majority (92%) had more than three years' experience of using the Internet, and all the students used the Internet daily. Concerning the students' experience with Moodle, the majority (82%) had more than three semesters' experience using Moodle, and the majority (72%) used Moodle daily.

Additionally, a total of 225 students out of the 240 (90.75%) participated in Stage two of this study. The majority of them were in the age range of 18 to 22 years. Males represented (60%) of the students while females represented 40%. Regarding their Faculty, 40% of the students were from the Faculty of Business; 30% of the students were from the Faculty of Engineering; 15% were from the Faculty of Nursing and the rest (15%) were from the Faculty of Information Technology. The students who participated were also in their second (12%) third (33%) and fourth (55%) years of study. The majority (96%) had more than three years' experience of using the Internet, and all the students used the Internet daily. Concerning the students' experience with Moodle, the majority (95%) had more than three semesters' experience using Moodle, and the majority (83%) used Moodle daily.

Students' Experience in Moodle

This section presents the results obtained from analysing the data that were collected from the two groups of students concerning their experience of the features and usability of Moodle on desktop/ laptop devices and mobile/tablet interfaces. It includes four subsections related to: features supported by Moodle; features not supported by the local instance of Moodle; usability problems in Moodle; and suggested improvements to the design of Moodle. Each subsection presents the results obtained from Group one of the students followed by the results obtained from Group two.

Features Supported by Moodle

Group one of the students identified six common uses of Moodle LMS on either desktop/laptop devices or mobile/tablet devices, as shown in Table 1. These represent six features that are supported by Moodle. However, the Likert scores for these six features obtained from Group two of the students showed that two out of the six features were the most frequently used on both desktop/laptop devices and mobile/tablet devices. These related to:

1. Downloading materials and learning resources, which included: course outlines, presentations, books and assignments.

2. Uploading assignments.

Features not Supported by the Local Instance of Moodle

The results obtained from Group one of the students showed that the students required a large number of features (18) in Moodle which were not actually provided by the local instance of Moodle they used. The 18 identified features consisted of 15 features that were identified commonly on the desktop/laptop and mobile/tablet interfaces of Moodle, and three features that were identified uniquely on its desktop/laptop interface. The 18 identified features suggested three major categories which related to communication, presentation and interaction. Table 2 shows the 18 common required features after they were organised into the three identified categories. The following table presents the three identified categories, and the Likert scores for their features obtained from Group two of the students:

Communication: The Likert scores for Moodle's features that were required by the students and related to the Communication category obtained from Group two of the students showed that only one out of the three features was required by a large number of students on both desktop/laptop devices and mobile/tablet devices. This related to "Communication among students":i.e. allowing them to send messages to each other.

Presentation: The Likert scores related to the Presentation category for Moodle's features that were required by the students, and which were obtained from Group two of the students, showed that eight of the nine identified features were required by a large number of students on the two interfaces of Moodle:

Displaying the material and learning resources (course outlines, presentations, books) of the courses before registration.

* Displaying previous exam questions or information about the nature of the course exams.

* Displaying recorded videos of lectures given by the course teachers.

* Displaying the dates for the first, second and final exams.

* Displaying the marks for the first, second and final exams.

* Displaying the study plans (outlines) for all the courses.

* Displaying a monthly evaluation for students in each course and providing comments for them regarding their progress on the course.

* Displaying department advertisements, news and university news.

Interaction: The Likert scores related to the Interaction category for Moodle's features that were required by the students, and which were obtained from Group two of the students, showed that all six identified features were required by a large number of students on the two interfaces of Moodle:

* Providing the user with feedback after carrying out any action during his/her interaction with the system.

* Allowing the students to provide suggestions or comments or feedback.

* Allowing the students to make changes to uploaded assignments, such as update/ delete.

* Sending an alert (e.g. a message to the students' mobiles or emails) when a teacher uploads new material or an assignment.

* Sending a reminder to students regarding submitting a required assignment or material before their deadline.

* Supplying support or help to students.

Usability Problems in Moodle

The students identified 17 usability problems on the interface of Moodle which prevented them from interacting with the interface successfully. The 17 identified usability problems consisted of: 11 usability problems that were commonly identified on the desktop/laptop and mobile/tablet interfaces of Moodle; four usability problems that were identified uniquely on the desktop/laptop interface of Moodle; and two usability problems that were identified uniquely on the mobile/tablet interface of Moodle. However, 16 out of the 17 identified usability problems were in fact usability problems specific to the interfaces of the local instance of Moodle used by Zarqa University. The 17 identified usability problems suggested four major categories which related to: design; links; content; and ease of use, speed and the internal search function. Table 3 shows the 17 usability problems that were identified by Group one of the students, and the Likert score for each usability problem obtained from Group two of the students. The identified usability problems with a Likert score rating 5 to 7 confirmed the seriousness of the usability problem from the students' point of view. The following table presents the identified usability problems and the Likert scores for them obtained from Group two of the students:

Design: The students identified five common usability problems related to the design of the two interfaces of Moodle. However, the Likert scores for these problems obtained from Group two of the students showed that only one of these problems was confirmed by the students. This related to inconsistency in the language of the interface. Part of the content of the pages is displayed using the Arabic language and the other part is displayed using English.

Links: The results showed that Group one of the students identified three common usability problems related to the Links category on the two interfaces of Moodle. It is worth mentioning that these problems were specific to the interfaces of the local instance of Moodle LMS that was used by Zarqa University. The Likert scores obtained from Group two of the students confirmed two of these usability problems. These related to:

The Moodle link on the home page of the University website was not obvious; it was located at the bottom of the page in a small font size, as shown in Figure 1.

The location of the login links on the home page of Moodle was not obvious: one was located at the top of the home page of Moodle but it was written using a small font size; the other was located in the middle of the page instead of at the top.

Content: The results showed that Group one of the students identified five common usability problems related to the content of Moodle's pages on the two interfaces of Moodle. All these problems were specific to the interfaces of the local instance of Moodle LMS that was used by Zarqa University. The Likert scores obtained from Group two of the students confirmed four out of the five content usability problems. These related to:

* Inappropriate content: the home page of Moodle displayed unnecessary content (e.g. information about e-learning applications).

* Moodle pages do not have links to the registration page or to the home page of the university.

* On the navigation menu, the courses are displayed by their numbers and not by their names which is confusing and not clear. This is shown in Figure 2.

* The student can view the titles of all courses in all faculties but he/she cannot access any of them.

Ease of use, speed and internal search function: The results showed that Group one of the students identified four common usability problems related to this category; three of them were identified on the two interfaces of Moodle while one of them was identified only on the mobile/tablet interface of Moodle, as shown in Table 3. Only one of the four identified usability problems related to this area is a general problem in Moodle LMS. This is that it is not easy to submit an assignment. The other three usability problems related to this area are specific to the interfaces of the local instance of Moodle LMS that was used by the university where the study took place. The Likert scores obtained from Group two of the students confirmed two out of the four identified usability problems. These related to:

* Not easy to submit an assignment.

* The location of the internal search function is inappropriate; it is located at the bottom of the page and not at the top as expected, as shown in Figure 3.

Suggested Improvements to the Design of Moodle

This section presents ten common improvements to the design of Moodle suggested by Group one of the students to make it more usable. The ten suggested improvements comprised: nine common improvements that were suggested concerning the two interfaces of Moodle (desktop/laptop and mobile/tablet), and one unique improvement that was suggested for the mobile/tablet interface of Moodle. The ten suggested improvements fell into two major categories: design; and links, contents and ease of use. Table 4 shows the ten suggested improvements, presented in the two major categories that were identified, together with the Likert score for each improvement obtained from Group two of the students. The following sections explain the suggested improvements and the Likert scores for them obtained from Group two of the students:

Design: The results showed that Group one of the students identified six common improvements to the design of Moodle's interfaces: five of them concerned the two interfaces of Moodle (desktop/ laptop and mobile/tablet) while one of them was suggested for the mobile/tablet interface of Moodle. The Likert scores obtained from Group two of the students confirmed the importance of five out of the six identified improvements, as shown in Table 4. The Likert scores showed that the students confirmed the importance of:

* Changing the two interfaces of Moodle (desktop/laptop and mobile/tablet) to support the Arabic language.

* Improving the design of Moodle's pages on the two interfaces.

* Changing the colours of the design of the two interfaces of Moodle.

* Changing the font size used for the two interfaces of Moodle (increasing the font size).

* Changing the design of the mobile/tablet interface of Moodle so that the size of the pages would be adjusted automatically according to the size of the mobile screen (free screen size).

Links, content and ease of use: The results showed that Group one of the students identified four common improvements to the two interfaces of Moodle (desktop/laptop and mobile/tablet) related to this category. The Likert scores obtained from Group two of the students confirmed the importance of the four identified improvements, as shown in Table 4. The four improvements related to:

* Making the link to open Moodle more visible: e.g. place the link at the top of the home page and use a larger font size.

* Improving the speed of downloading Moodle's pages.

* Adding a link to the library system to the desktop/laptop interface of Moodle.

* Making Moodle easier to use.

DISCUSSION

This research uniquely investigates the usefulness and usability of Moodle LMS utilised by a case study university (Zarqa University in Jordan) based on students' experience with Moodle LMS on both desktop and mobile interfaces. It also presents results which could be useful, not only for the case study university, but for all other universities. This research sheds the light on the importance of considering and involving students when employing Moodle LMS in order to provide/enable features based on students' needs and preferences. Also, this research reveals, in detail, usability problems which should be considered while developing Moodle LMS in order to improve students' experience of using Moodle LMS and to make it more enjoyable.

The results of this research identified six commonly used features on both of Moodle's interfaces and showed that only two out of the six features were most frequently used. Also, the results of this research showed that the students identified 18 features they required on both interfaces of Moodle LMS, as well as revealing that 15 out of the 17 features were required by a larger number of students. The fact that these features are actually features that are supported by Moodle LMS but not provided by the local instance of Moodle used by Zarqa University raises an important issue which needs to be considered seriously. The lack of communication between the administrator of Moodle LMS at Zarqa University and the students regarding providing features in Moodle LMS required by the students reduces the value of Moodle LMS and hinders its usefulness, showing that Moodle LMS does not address the students' needs when it is capable of doing so. The findings of this research suggest that Zarqa University specifically, and other universities which employ Moodle LMS, should involve students in deciding which features to provide or enable in Moodle LMS in order to attain the greatest benefits from the system.

Furthermore, the results of this research identified 17 usability problems on the interfaces of Moodle and showed that nine of the 17 problems were considered serious problems from the viewpoint of the students. Only one of the identified problems was also identified by earlier research which evaluated the usability of Moodle LMS (Thuseethan et al., 2014; Rosato et al., 2004; Melton, 2006; Kakasevski et al, 2008). This problem related to: Not easy to submit an assignment. This problem is a general one which could be considered by Moodle LMS. The other eight usability problems which were identified by a large number of students, as explained in Section 5.2.3, were specific to the local instance of Moodle used by Zarqa University. The identified usability problems should be addressed by the administrator of Moodle LMS at Zarqa University to improve the system's overall usability from the viewpoint of the students and to provide an enjoyable experience for the students while they are using it.

This research also presented ten suggested improvements to the design of Moodle to make it more usable from the viewpoint of the students with nine of the suggested improvements attracting high ratings. The improvements related to both interfaces (desktop and mobile) except for one which was specific to the mobile interface. This related to changing the design so that the size of the pages would be adjusted automatically according to the size of the mobile screen (free screen size). It is worth mentioning that one of the suggested improvements in this research was also suggested in earlier research (Ivanović et al, 2013; Melton, 2006; Kakasevski et al, 2008) which related to the use of a local language for Moodle LMS interfaces instead of the English language.

Based on the results obtained from the students, this research uniquely suggests recommendations to be considered by the case study university to improve the usefulness and usability of Moodle LMS on both desktop and mobile interfaces. The suggested recommendations could also be useful to other universities which have not considered students' preferences when employing Moodle LMS. The recommendations consist of two parts: Part One includes recommendations regarding features that could be supported by Moodle LMS, while Part Two consists of recommendations of design characteristics that could be considered to obtain more usable interfaces for Moodle LMS.

Part One: Recommended Features

The following presents recommendations regarding features that could be supported by an LMS to make it more useful from the viewpoint of students:

1. Downloading materials and learning resources to include course outlines, presentations, books and assignments.

2. Uploading assignments.

3. Permitting communication among students: e.g. allowing them to send messages to each other.

4. Displaying the material and learning resources (course outlines, presentations, books) of the courses before registration.

5. Displaying previous exam questions or information about the nature of the course exams.

6. Displaying recorded videos of lectures given by the course teachers.

7. Displaying the dates for the first, second and final exams.

8. Displaying the marks for the first, second and final exams.

9. Displaying the study plans (outlines) for all the courses.

10. Displaying a monthly evaluation for students in each course and providing comments for them regarding their progress on the course.

11. Displaying department advertisements, news and university news.

12. Providing the user with feedback after carrying out any action during his/her interaction with the system.

13. Allowing the students to provide suggestions, comments or feedback.

14. Allowing the students to make changes to an uploaded assignment such as update/ delete.

15. Sending an alert (e.g. a message to the students' mobiles or emails) when a teacher uploads new material or an assignment.

16. Sending a reminder to students regarding submitting a required assignment or material before the deadline.

17. Supplying support and help to students.

Part Two: Recommended Design Characteristics

The following presents recommendations for design characteristics to be considered while developing an LMS to make it more usable from the viewpoint of students:

1. Making the language of the interfaces consistent.

2. Making the Moodle link on the home page of the university obvious.

3. Making the location of the login links on the home page of Moodle obvious.

4. Making the content of the home page appropriate.

5. Adding links on Moodle pages to the registration page or to the home page of the university.

6. Displaying, on the navigation menu, the courses by their names and not by their numbers.

7. Not displaying the titles of all courses in all faculties; a student should be able to display his/her registered courses only.

8. Making it easier to submit an assignment.

9. Making the location of the internal search function appropriate.

10. Changing the interface to support the Arabic language.

11. Improving the design of Moodle's pages.

12. Changing the colours of the design, such as using the blue colour.

13. Changing the font size used for the website (increasing the font size).

14. Changing the design so that the size of the pages would be adjusted automatically according to the size of the mobile screen (free screen size).

15. Making the link to open Moodle more visible: e.g. placing the link at the top of the home page and using a larger font size.

16. Improving the speed of downloading Moodle's pages.

17. Adding a link to the library system.

18. Making Moodle easier to use.

CONCLUSION

This research investigated the usefulness and usability of Moodle LMS employed by a case study university from the viewpoint of 320 students, taking into consideration two interfaces of Moodle: desktop/laptop and mobile/tablet. The results of this research uncovered two features which were used frequently by all the students on Moodle LMS, and another 15 features which were not currently provided by the local instance of Moodle used by the case study university but which were required by a large number of students. Also, the results of this research showed that a large number of students identified nine usability problems on Moodle interfaces and a large number also identified and suggested nine improvements to the design of Moodle interfaces from their point of view. Based on the results, this research then presented recommendations to create a more useful and usable LMS.

The results are particularly useful for developers, evaluators and managers of Moodle LMSs at Zarqa University, and could be useful also for other universities and/or academic institutions which use Moodle. This research recommends taking advantage of the various features supported by Moodle in order to obtain a useful and usable LMS. The recommendations presented in this research provide guidance regarding the features and usability issues which should be taken into consideration when designing and/or evaluating an LMS to achieve a more useful and usable system which satisfies the needs of the students.

This research, however, has two limitations. The first relates to the fact that it reflects the viewpoints only of students regarding aspects of Moodle; other users, including teachers and administrative staff, were not considered. The second relates to the fact that it employed only open-ended questionnaires to identify the usability problems on Moodle's interfaces. This method is considered to be an indirect usability method because it does not study Moodle's interfaces directly. Other direct usability methods, such as user testing which can be used to observe and record actual users' interactions with the interface, were not employed.

Sidebar

Manuscript first received: 2019/02/06. Manuscript accepted: 2019/10/22

Address for correspondence:

Layla Hasan, Software Engineering Department, Zarqa University, Zarqa, Jordan. Email: [email protected]

References

REFERENCES

Al-Sarrayrih H. S., Knipping L., & Zorn E. (2010). "Evaluation of a Moodle based learning management system applied at Berlin institute of technology based on Iso-9126", in the Proceedings of ICL 2010 Conference Hasselt Belgium, 880-887.

Baytiyeh H. (2013). "Perceptions of engineering professors and students regarding the acceptance and use of Moodle", in the Proceedings of the 120th ASEE Annual Conference & Ex^mtkm.

Braccini A. M, Silvestri C, Za S., & D'Atri A. (2009). "Users' perception of open source e-learning platform quality: the case of Moodle", in the Proceedings of VIIIIASTED International Conference on Web-based Education, Uskov.

Chung C., Pasquini L., and Koh C. (2013). "Web-Based learning management system considerations for higher education", Learning and Performance Quarterly, 1(4), 24-37.

Hasan L., Morris A., and Probets S. (2012). "A comparison of usability evaluation methods for evaluating e-commerce websites", Behaviour & Information Technology, 31(7), 707-737.

ISO 9241-11, (1998). International standard first edition. ergonomic requirements for office work with visual display terminals (vdts), Part11: guidance on usability, , <http://www.idemployee.id.tue.nl/ g.w.m.rauterberg/lecturenotes/ISO9241part11.pdf>, [accessed 03.04.16].

Ivanc D., Vasiu R., & Onita M. (2012). "Usability evaluation of a LMS mobile web interface", in the Proceedings of the 18th International Conference, ICIST 2012, Kaunas, Lithuania, September 13-14, 2012, 348-361.

Ivanović Mi., Putnik Z., Komlenov Ž, Welzer T., Hölbl M, & Schweighofer T. (2013). "Usability and privacy aspects of moodle: students' and teachers' perspective", Informatica, 37, 221-230.

Kakasevski G., Mihajlov M., Arsenovski S., & Chungurski S. (2008). "Evaluating usability in learning management system Moodle", in the Proceedings of the ITI2008 30th Int. Conf on Information Technology Interfaces, June 23-26, Cavtat, Croatia.

Martin L., Martínez D. R., Revilla O., José M., Santos O. C., & Boticario, J. G. (2008). "Usability in e-Learning platforms: heuristics comparison between Moodle Sakai and dotLRN", in the Proceedings of International Conference and Workshops on Community based environment.

Melton, J. (2006). "The LMS Moodle: a usability evaluation, Languages Issue, 11/12(1), 1-24.

Minović M., Štavljanin V., Milovanović M., & Starcević D. (2008). "Usability issues of e-Leaming systems: case-study for Moodle learning management system", in R. Meersman, Z. Tari, and P. Herrero (Eds.): OTM 2008 Workshops, LNCS 5333, 561-570, 2008. Springer-Verlag Berlin Heidelberg.

Moodle, <https://moodle.org/>, [accessed 10.01.2018].

Mtebe J. & Kissaka M. (2015). "Heuristics for evaluating usability of learning management systems in Africa", in the Proceedings of the IST-Africa 2015 Conference.

Nielsen J. (2012). Usability 101: Introduction to usability. Nielsen Norman Group.com,. <https://www. nngroup.com/articles/usability-101-introduction-to-usability/ >, [accessed 27.12.2018].

Onacan M. & Erturk A. (2016). "Usability evaluation of learning management system in a higher education institution: a scale development study", Journal of Global Strategic Management, 10(2).

Rosato J., Dodds C., & Laughlin, S. (2004). "Usability of course management systems by students", Department of Computer Information Systems/ Computer Science, College of Scholastica, Duluth.

Ssekakubo G., Suleman H., & Marsden G. (2013). "Designing mobile LMS interfaces: Learners' expectations and experiences", Interactive Technology and Smart Education, 10(2), 147-167. Available at: http://www.emeraldinsight.com/doi/abs/10.1108/ITSE-12-2012-0031 [Accessed January 6, 2015].

Senol l., Gecili H. & Durdu P. (2014). "Usability evaluation of a Moodle based learning management system", in the Proceedings of EdMedia 201, Tampere, Finland, June 23-26.

Simonson M. (2007). "Course management systems", Quarterly Review of Distance Education, S(1), vii-ix.

Thuseethan S., Achchuthan S., & Kuhanesan S. (2014). "Usability evaluation of learning management systems in Sri Lankan universities", arXiv preprint arXiv:1412.0197.