Evaluating Multi-Modality Mobile Behavioral Biometric Fusion Using Public Datasets

Abstract

Behavioral biometric-based continuous user authentication is promising for securing mobile phones while complementing traditional security mechanisms. The state of the art so far lacks in presenting a generalized performance range for such an authentication system which is believed to have practical applicability. Such performance estimation can be utilized during commercially deploying such security measures in real life. Therefore, the goal of this thesis is to establish an evaluation framework for mobile behavioral biometrics and conduct a comprehensive evaluation of different algorithms and datasets. Toward this end, this thesis has made four major contributions. First, we conduct a comprehensive survey on the broad spectrum of stationary mobile behavioral biometrics and propose a conceptual framework that can classify and group the user behaviors of the state of the art. Second, we collect our novel CU Mobile datasets [13] where users sit and type to fill out an Android registration form and evaluate it utilizing our continuous authentication framework. Third, we further fine-tune our authentication framework and utilize it to evaluate two public datasets (sitting and typing data) using machine learning algorithms. In this process, we also explore several feature sets. Lastly, we expand our dataset evaluation using deep learning algorithms over the same two datasets. To elaborate further on this, we utilize multiple behavioral biometric modalities from our collected CU mobile datasets and report the authentication performances. We focus deeply on the motion events from this dataset by evaluating a set of score level fusion techniques using the acceleration and gyroscope (angular velocity) data as it is essential for continuous user authentication. We also conduct experiments on the dataset’s touch events (swipes and strokes). Following this we evaluate the performance of user authentication based on acceleration, gyroscope, and swipe modalities from two other public mobile datasets, HMOG [1] and BB-MAS [2] extracted with different feature sets (Frank et al.’s, 2012 [12] Touchalytics features on the swipe and median, HMOG (Sitov´a et al., 2015) [10], and Shen (Shen et al., 2017) [11] on the motion events) to observe the variation in authentication performance. We evaluate the performances of both individual modalities and their fusion. We evaluate the fusion of multiple modalities using Nandakumar’s likelihood ratio-based score fusion (Nandakumar et al., 2007) [3]) by utilizing machine learning algorithms (both one-class and binary SVMs). We further expand our evaluation by applying deep learning algorithms (namely, Multi-Layer Perceptron and Convolutional Neural Network) on the same public datasets to observe the change in the performance range of mobile behavioral biometrics. Through this evaluation of the two public datasets, we observe that in the case of the machine learning-based experiments BB-MAS [2] consistently performs better than HMOG [1] due to the presence of longer swipe trajectory and absence of the effect of concept drift. However, while evaluating the datasets using deep learning we observe that the effect of concept drift on HMOG [1] is overcome as this class of algorithms can better handle the larger state space. Furthermore, the difference between the performances of HMOG [1] and BB-MAS [2] is less pronounced when evaluated with deep learning algorithms.