Evaluation Of Rapid Development System Using Eye Tracker

Abstract

This paper presents the results of the evaluation of Linear Axis Rapid Development System10 (RDS), which is under development as part of a NSF funded project. The Linear Axis RDS is used in teaching control design/insertion in the Mechanical Engineering curriculum at a mid- sized midwestern university in the United States. The Linear Axis RDS has a graphical user interface with three main modes: simulate, emulate, and implement. The objective of this evaluation was to test the overall effectiveness of the Linear Axis RDS. A combination of qualitative and quantitative research methods were applied in the evaluation of thirty-four participants from the Mechanical and Aerospace Engineering Department at Missouri University of Science and Tech. The Technology Acceptance Model, a model that has been used extensively to study acceptance of technology was used to guide the study. Learning Styles and Learning Outcomes were added to study the learning effects of the system. Eye tracking was used in two of the tasks to provide both qualitative and quantitative data. Eye tracking is an innovative method that is increasingly being used in the field of human-computer interaction for usability studies, as it can provide useful insight into the cognitive aspect of the users. Based on the data analysis, a significant improvement was noticed in users interest after using RDS. Statistical analysis showed significant increase in career interest in science followed closely by enjoyment. Results from the analysis on learning outcomes suggest the RDS was perceived to have high real world applicability. Results also showed an increase in knowledge gained after using the system. The Technology Acceptance Model (TAM) constructs such as perceived ease of use (PEOU), perceived usefulness (PU), attitude (ATT) and intention to use (INT) were found to influence the learning outcome. Eye tracking results validated the results from the survey analysis. The gaze plots and heat maps indicated that the participants were able to identify important areas of the interface, such as tip box and help button, which were newly developed. Overall the results suggest that RDS is well received by the participants and is an effective learning tool

I. Introduction

Engineering education has benefitted tremendously from the use of information technology. One of the key aspects of engineering education is hands-on experiments, either using the main equipments or by using tools that simulate the equipments.2The use of IT tools to simulate the equipment enhances the quality of education as well as accommodates students with various learning styles.2

An important aspect of such educational software is that it should be easy to use, learn and understand. In essence it should be usable. Usability is defined as “the capability of the software product to be understood, learned, used and attractive to the user, when used under specified conditions.”1 Usability testing plays an important role in the development of interactive educational software and user-centered design is quintessential for promoting its usage. “Usability problems of educational software can be one source of disturbance within the learning process by distracting attention from the learning task and consequently increasing the extraneous cognitive load” 4