SRED: Secure and Robust Emotion Detection for Advanced Driver Assistance Systems

Abstract

Facial emotion detection is among the most accessible and non-intrusive techniques for assessing driver states in Advanced Driver Assistance Systems (ADAS). Facial landmarks—such as eyes, nose, and mouth—are central to emotion detection models, providing essential cues for inference. However, deep learning models employed for this task are highly susceptible to adversarial attacks, where imperceptible perturbations can lead to significant performance degradation. Gradient-based adversarial attacks, such as FGSM, BIM, PGD, are highly effective in changing the decision boundaries of the models and causing misclassifications. This thesis investigates the oftenoverlooked vulnerability of facial landmark regions and introduces SRED (Secure and Robust Emotion Detection), a novel framework tailored for real-time deployment in ADAS environments. Three state-of-the-art deep learning models—ResNet18, MobileNetV2, and EfficientNetB0—are trained on widely used datasets (KMU-FED, KDEF, and FER2013), incorporating both standard and attention-guided training paradigms. Robustness is evaluated against adversarial attacks including FGSM, BIM, and PGD, with a focus on resilience under cross-perturbation scenarios. Notably, the transferability analysis reveals that adversarial examples generated using attention-masked models are particularly damaging when applied to similarly trained models—for example, MobileNetV2’s accuracy on the KDEF dataset plummeted from 74.11% to just 7.77% under such conditions. To enhance interpretability, SRED also integrates saliency map analysis, offering insights into the critical facial regions influencing the model’s decisions.