Computer Science ›› 2022, Vol. 49 ›› Issue (4): 30-36.doi: 10.11896/jsjkx.210900200

• Special Issue of Social Computing Based Interdisciplinary Integration • Previous Articles     Next Articles

EEG Emotion Recognition Based on Spatiotemporal Self-Adaptive Graph ConvolutionalNeural Network

GAO Yue, FU Xiang-ling, OUYANG Tian-xiong, CHEN Song-ling, YAN Chen-wei   

  1. School of Computer Science (National Pilot Software Engineering School), Beijing University of Posts and Telecommunications, Beijing 100876, China; Key Laboratory of Trustworthy Distributed Computing and Service (BUPT), Ministry of Education, Beijing 100876, China
  • Received:2021-09-24 Revised:2021-12-28 Published:2022-04-01
  • About author:GAO Yue,born in 1998,Ph.D candidate.His main research interests include natural language processing,me-dical informatics and recommender system.FU Xiang-ling,born in 1975,Ph.D,associate professor.Her main research interests include medical informatics,deep learning and text mining.
  • Supported by:
    This work was supported by the Beijing Natural Science Foundation(M22012,L192026) and National Natural Science Foundation of China(82071171).

Abstract: With the rapid development of human-computer interaction in computer aided field, EEG has become the main means of emotion recognition.Meanwhile, graph network has attracted wide attention due to its excellent ability to represent topological data.To further improve the representation performance of graph network on multi-channel EEG signals, in this paper, conside-ring the sparsity and infrequency of EEG signals, a self-adaptive brain graph convolutional network with spatiotemporal attention (SABGCN-ST) is proposed.The method solves the sparsity of emotion via the spatiotemporal attention mechanism and explores the functional connections between different electrode channels via the self-adaptive brain network topological adjacent matrix.Finally, the feature learning of graph structure is operated via graph convolution, and the emotion is predicted.Extensive experiments conduct on two benchmark datasets DEAP and SEED prove that SABGCN-ST has a significant advantage in accuracy compared with baseline models, and the average accuracy of SABGCN-ST reaches 84.91%.

Key words: Deep learning, Electroencephalogram, Emotion recognition, Graph convolutional neural network, Self-adaptive adjacent matrix, Spatio-temporal attention mechanism

CLC Number: 

  • TP183
[1] COWIE R,DOUGLAS C E,TSAPATSOULIS N,et al.Emo-tion recognition in human-computer interaction[J].IEEE Signal Processing Magazine,2001,18(1):32-80.
[2] ANDERSON K,MCOWAN P.A real-time automated systemfor the recognition of human facial expressions[J].IEEE Tran-sactions on Systems,Man,and Cybernetics,Part B (Cyberne-tics),2006,36(1):96-105.
[3] HUANG X,ZHAO G,HONG X,et al.Spontaneous facial micro-expression analysis using spatiotemporal completed local quantized patterns[J].Neurocomputing,2016,175:564-578.
[4] HUANG X,WANG S,ZHAO G,et al.Facial micro-expression recognition using spatiotemporal local binary pattern with integral projection[C]// IEEE International Conference on Computer Vision Workshop.2015:1-9.
[5] HUANG X,WANG S,LIU X,et al.Discriminative spatiotemporal local binary pattern with revisited integral projection for spontaneous facial micro-expression recognition[J].IEEE Transactions on Affective Computing,2017,10(1):32-47.
[6] LIU Y,ZHANG J,YAN W,et al.A main directional mean optical flow feature for spontaneous micro-expression recognition[J].IEEE Transactions on Affective Computing,2016,7(4):299-310.
[7] YAN J,ZHENG W,XIN M,et al.Integrating facial expression and body gesture in videos for emotion recognition[J].IEICE Transactions on Information and Systems,2014,E97-D(3):610-613.
[8] ANG J,DHILLON R,KRUPSKI A,et al.Prosody-based automatic detection of annoyance and frustration in human-computer dialog[C]//International Conference on Spoken Language Processing.2002:2037-2040.
[9] ZHENG W.Multichannel EEG-based emotion recognition viagroup sparse canonical correlation analysis[J].IEEE Transactions on Cognitive and Developmental Systems,2017,9(3):281-290.
[10] CHENG B,LIU G.Emotion recognition from surface EMG signal using wavelet transform and neural network[C]// 2nd International Conference on Bioinformatics and Biomedical Engineering.2008:1363-1366.
[11] AGRAFIOTI F,HATZINAKOS D,ANDERSON A.ECG pattern analysis for emotion detection[J].IEEE Transactions on Affective Computing,2012,3(1):102-115.
[12] NOBLE W S.What is a support vector machine?[J].Nature Biotechnology,2006,24(12):1565-1567.
[13] MERT A,AKAN A.Emotion recognition from EEG signals by using multivariate empirical mode decomposition[J].PatternAnalysis and Applications,2018,21(1):81-89.
[14] ZHENG W L,LU B.Investigating critical frequency bands and channels for eeg-based emotion recognition with deep neural networks[J].IEEE Transactions on Autonomous Mental Development,2015,7(3):162-175.
[15] LI Y,ZHENG W,ZONG Y,et al.A bi-hemisphere domain adversarial neural network model for EEG emotion recognition[J].IEEE Transactions on Affective Computing,2021,12(2):494-504.
[16] LI Y,ZHENG W,WANG L,et al.From regional to globalbrain:a novel hierarchical spatial-temporal neural network mo-del for EEG emotion recognition[J/OL].IEEE Transactions on Affective Computing.https://doi.ieeecomputersociety.org/10.1109/TAFFC.2019.2922912.
[17] SONG T,ZHENG W,SONG P,et al.EEG emotion recognition using dynamical graph convolutional neural networks[J].IEEE Transactions on Affective Computing,2018,11(3):532-541.
[18] LI P,LIU H,SI Y,et al.EEG based emotion recognition bycombining functional connectivity network and local activations[J].IEEE Transactions on Biomedical Engineering,2019,66(10):2869-2881.
[1] XU Yong-xin, ZHAO Jun-feng, WANG Ya-sha, XIE Bing, YANG Kai. Temporal Knowledge Graph Representation Learning [J]. Computer Science, 2022, 49(9): 162-171.
[2] RAO Zhi-shuang, JIA Zhen, ZHANG Fan, LI Tian-rui. Key-Value Relational Memory Networks for Question Answering over Knowledge Graph [J]. Computer Science, 2022, 49(9): 202-207.
[3] TANG Ling-tao, WANG Di, ZHANG Lu-fei, LIU Sheng-yun. Federated Learning Scheme Based on Secure Multi-party Computation and Differential Privacy [J]. Computer Science, 2022, 49(9): 297-305.
[4] WANG Jian, PENG Yu-qi, ZHAO Yu-fei, YANG Jian. Survey of Social Network Public Opinion Information Extraction Based on Deep Learning [J]. Computer Science, 2022, 49(8): 279-293.
[5] HAO Zhi-rong, CHEN Long, HUANG Jia-cheng. Class Discriminative Universal Adversarial Attack for Text Classification [J]. Computer Science, 2022, 49(8): 323-329.
[6] JIANG Meng-han, LI Shao-mei, ZHENG Hong-hao, ZHANG Jian-peng. Rumor Detection Model Based on Improved Position Embedding [J]. Computer Science, 2022, 49(8): 330-335.
[7] SUN Qi, JI Gen-lin, ZHANG Jie. Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection [J]. Computer Science, 2022, 49(8): 172-177.
[8] HOU Yu-tao, ABULIZI Abudukelimu, ABUDUKELIMU Halidanmu. Advances in Chinese Pre-training Models [J]. Computer Science, 2022, 49(7): 148-163.
[9] ZHOU Hui, SHI Hao-chen, TU Yao-feng, HUANG Sheng-jun. Robust Deep Neural Network Learning Based on Active Sampling [J]. Computer Science, 2022, 49(7): 164-169.
[10] SU Dan-ning, CAO Gui-tao, WANG Yan-nan, WANG Hong, REN He. Survey of Deep Learning for Radar Emitter Identification Based on Small Sample [J]. Computer Science, 2022, 49(7): 226-235.
[11] HU Yan-yu, ZHAO Long, DONG Xiang-jun. Two-stage Deep Feature Selection Extraction Algorithm for Cancer Classification [J]. Computer Science, 2022, 49(7): 73-78.
[12] CHENG Cheng, JIANG Ai-lian. Real-time Semantic Segmentation Method Based on Multi-path Feature Extraction [J]. Computer Science, 2022, 49(7): 120-126.
[13] XU Ming-ke, ZHANG Fan. Head Fusion:A Method to Improve Accuracy and Robustness of Speech Emotion Recognition [J]. Computer Science, 2022, 49(7): 132-141.
[14] LIU Wei-ye, LU Hui-min, LI Yu-peng, MA Ning. Survey on Finger Vein Recognition Research [J]. Computer Science, 2022, 49(6A): 1-11.
[15] SUN Fu-quan, CUI Zhi-qing, ZOU Peng, ZHANG Kun. Brain Tumor Segmentation Algorithm Based on Multi-scale Features [J]. Computer Science, 2022, 49(6A): 12-16.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!