Content area
Full text
Received May 31, 2017; Revised Jul 24, 2017; Accepted Aug 14, 2017
This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
1. Introduction
Emotion plays a powerful role in social influence: not only does it include psychological responses to external stimuli or one’s own stimuli but it is also accompanied by physiological responses to psychological reactions in individuals’ daily lives. Emotional influences are manifested across a variety of levels and modalities [1]. On the one hand, peripheral signals are related to the somatic nervous system and show physiological changes in emotion states. For instance, there are physical signals that emerge: facial expressions, verbal speech, or body language. On the other hand, there are also influences on cognitive processes, including coping behaviors such as wishful thinking, resignation, or blame-shifting. The goal of our research is to perform a multimodal fusion between EEGs and peripheral physiological signals for emotion recognition.
Previous studies have investigated the use of peripheral and brain signals separately, but little attention has been paid thus far to a fusion between brain and peripheral signals. In one study, Ekman and Friesen made a pioneering contribution to modern facial expression recognition [2]. They defined the six basic expressions of human beings, that is, pleasure, anger, surprise, fear, disgust, and sadness, and identified the categories of objects to be investigated. Mase made use of optical flow to determine the main direction of movement of the muscles and then constructed the Face Recognition System [3]. Picard and Daily at MIT Media Laboratory developed pattern recognition algorithms that attained 78.4% classification accuracy for three categories of emotion states using the peripheral signals of galvanic skin resistance, blood pressure, respiration, and skin temperature [4].
Compared to periphery physiological signals, EEG signals have been proven to provide greater insights into emotional processes and responses. Furthermore, because EEG has been widely used in BCIs, the study of EEG-based emotion detection may provide great value for improving the user experience and...