Modeling Cognitive Process From Visual Attention Geometry in AR/VR: A Machine Learning Approach

Abstract

The ecological theory of perception maintains that visual optic arrays contain geometric patterns that afford information to a perceiver, influencing interactions with one's environment. In this thesis, the author develops methods of visual field decomposition in 2D optical arrays and 3D spaces to characterize visual attention patterns and cognitive states from data previously collected in a Virtual Reality (VR) simulator. A novel algorithm was developed to isolate markers of human visual attention dynamically as gaze vectors adapt continuously with movement through space.

Supervised and unsupervised machine learning models were constructed and trained on a combination of visual attention features to understand the contributions of distinct markers of attention to psychophysical behaviors. Hypothesis-driven studies and respective results aided the selection and validation of meaningful classification categories and boundaries in the areas of: attention shift, response time, and decision-making. This thesis yielded machine learning models that successfully classified cognitive states from patterns in 2D and 3D visual attention. Predictive analytics and visualization methods developed in this work show the potential for AI-enhanced eye tracking technology to augment decision-making during visual search and navigation applications.