Abstract

Artificial intelligence has been successfully applied in various fields, one of which is computer vision. In this study, a deep neural network (DNN) was adopted for Facial emotion recognition (FER). One of the objectives in this study is to identify the critical facial features on which the DNN model focuses for FER. In particular, we utilized a convolutional neural network (CNN), the combination of squeeze-and-excitation network and the residual neural network, for the task of FER. We utilized AffectNet and the Real-World Affective Faces Database (RAF-DB) as the facial expression databases that provide learning samples for the CNN. The feature maps were extracted from the residual blocks for further analysis. Our analysis shows that the features around the nose and mouth are critical facial landmarks for the neural networks. Cross-database validations were conducted between the databases. The network model trained on AffectNet achieved 77.37% accuracy when validated on the RAF-DB, while the network model pretrained on AffectNet and then transfer learned on the RAF-DB results in validation accuracy of 83.37%. The outcomes of this study would improve the understanding of neural networks and assist with improving computer vision accuracy.

Details

Title
A study on computer vision for facial emotion recognition
Author
Huang, Zi-Yu 1 ; Chiang, Chia-Chin 1 ; Chen, Jian-Hao 2 ; Chen, Yi-Chian 3 ; Chung, Hsin-Lung 1 ; Cai, Yu-Ping 4 ; Hsu, Hsiu-Chuan 5 

 National Kaohsiung University of Science and Technology, Department of Mechanical Engineering, Kaohsiung, Taiwan (GRID:grid.412071.1) (ISNI:0000 0004 0639 0070) 
 National Chengchi University, Graduate Institute of Applied Physics, Taipei, Taiwan (GRID:grid.412042.1) (ISNI:0000 0001 2106 6277) 
 Fooyin University, Department of Occupational Safety and Hygiene, Kaohsiung, Taiwan (GRID:grid.411396.8) (ISNI:0000 0000 9230 8977) 
 Hsin Sheng Junior College of Medical Care and Management, Department of Nursing, Taoyuan, Taiwan (GRID:grid.412071.1) 
 National Chengchi University, Graduate Institute of Applied Physics, Taipei, Taiwan (GRID:grid.412042.1) (ISNI:0000 0001 2106 6277); National Chengchi University, Department of Computer Science, Taipei, Taiwan (GRID:grid.412042.1) (ISNI:0000 0001 2106 6277) 
Pages
8425
Publication year
2023
Publication date
2023
Publisher
Nature Publishing Group
e-ISSN
20452322
Source type
Scholarly Journal
Language of publication
English
ProQuest document ID
2818596585
Copyright
© The Author(s) 2023. This work is published under http://creativecommons.org/licenses/by/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.