Promoting student engagement with GPTutor: An intelligent tutoring system powered by generative AI

Abstract

Intelligent Tutoring Systems (ITS) are computer systems that mimic human tutoring behavior while providing immediate feedback. With the rise of Generative Artificial Intelligence (GenAI), numerous ITS integrated with GenAI have been developed. Student engagement is critical for improving learning processes and outcomes. Therefore, it is important to examine the effectiveness of ITS integrated with GenAI in promoting student engagement in educational practice. This paper presents an explanatory mixed-method case study involving 880 undergraduate students who used GPTutor, an ITS powered by GenAI. First, a survey research was conducted to investigate the relationship between students’ actual interaction with GPTutor and their self-reported student engagement from three dimensions: Behavioral Engagement, Cognitive Engagement, and Emotional Engagement. Next, focus groups were conducted with a subsample of survey participants to better understand how and under what circumstances GPTutor improved student engagement. The focus groups also explored potential design improvements for GPTutor and other ITS powered by GenAI. The results of the survey research revealed a complex relationship between feature usage and student engagement. Specifically, engagement with the chatbot is significantly and positively associated with behavioral and emotional engagement, but not cognitive engagement. The exercise generator feature had no significant associations with any of the three dimensions of student engagement. The results of the focus groups shed some light on these relationships, revealing how GPTutor was used only when it was perceived as useful, and this perceived usefulness was shaped by the students’ perception of the difficulty of the course and whether their support system could adequately address questions they may have. Its usefulness was found to increase as the course progressed, particularly as examinations approached. As the examinations approached, it was increasingly clear that the exercise generator was preferred over the chatbot. The participants also made this clear by expressing how GPTutor could be improved, notably by increasing the capabilities of the chatbot to include multimodal media, like video recordings of lectures. In general, leveraging survey data, interview data, and back-end trace data from GenAI, this research makes an original contribution to AI-supported effective learning environments and design strategies to optimize the educational experiences of higher education students.