Identification of Action Units Related to Affective States in a Tutoring System for Mathematics

Abstract

Affect is an important element of the learning process both in the classroom and with educational technology. This paper presents analyses in relation to the identification of Action Units (AUs) related to affective states and their impact on learning with a tutoring system. To assess affect, a tool was devised to identify AUs on pictures of human faces. Action Units are combinations of individual facial muscles or groups of muscles that create facial expressions in association with affect. Pictures from a population of students were taken while using an intelligent tutoring system for mathematics in a secondary school in a suburban school in Veracruz, Mexico. The students were asked to interact with the tutoring system for 40 minutes and they were photographed with the tool at a rate of 1 picture every 5 seconds acquiring a dataset consisting of 16,800 photos. To achieve identification, the software analyzes individual pictures using Principal Component Analyses (PCA) and Euclidian distance. The tool developed to classify affective states shows 88.88% accuracy in the identification of AUs when matching the recognized AU to the Cohn-Kanade AU-Coded facial expression database. The analyses also elicited the most common AUs for the population and their association with learning with the intelligent tutoring system. These preliminary results shed light on the issues of affect in relation to learning mathematics with tutoring systems and pave the way for the implementation of coping strategies based on the automatic recognition of facial expressions.