Engagement with analytics feedback and its relationship to self-regulated learning competence and course performance

Abstract

Incorporating feedback into the learning process is crucial for learners’ success. Recent advancements in Learning Analytics (LA) and Artificial Intelligence (AI) have introduced a range of feedback generation and delivery opportunities to improve teaching and learning processes, yet questions remain about how learners engage with such tools and their impact. In this study, we investigated postgraduate students’ engagement with analytics feedback in relation to their level of self-regulated learning (SRL) competence and performance in a semester-long (ten-week) course. We specifically focused on the Interactive-Constructive-Active–Passive (ICAP) framework of cognitive engagement. Initially, students were asked to participate in an established SRL questionnaire, based on Zimmerman’s theory of self-regulation to evaluate their SRL competence (N = 39). Throughout the semester, their online behaviour data from Moodle and Google Docs was collected and analyzed to form personalised analytics feedback for each student. We examined how students with different SRL competencies engage with analytics feedback and the impact of this engagement on students’ course performance. Results indicated that students with high SRL competence actively engage with analytics feedback more than students with low SRL competence. However, students' analytics feedback engagement did not significantly affect their course performance. Additionally, we analyzed students’ reflections on the feedback provided to investigate how they perceived it in relation to their learning experiences and performance. Students argued in their reflections that analytics feedback was beneficial in identifying and regulating their online behaviours and providing motivation through objective insights. They also noted limitations in accurately reflecting their behaviours and learning quality, the need for more personalised recommendations and timely feedback, and suggested design improvements to ensure clarity, foster interaction and incorporate tailored, in-depth insights. We conclude the paper with a discussion on future design and research suggestions for ways of monitoring and supporting students’ cognitive engagement with analytics feedback interventions.