计算机科学 ›› 2021, Vol. 48 ›› Issue (12): 312-318.doi: 10.11896/jsjkx.201000141

• 人工智能 • 上一篇    下一篇

融合频率和通道卷积注意的脑电(EEG)情感识别

柴冰1,2, 李冬冬1,2, 王喆1, 高大启1   

  1. 1 华东理工大学信息科学与工程学院 上海200237
    2 苏州大学计算机信息处理技术重点实验室 江苏 苏州215006
  • 收稿日期:2020-10-24 修回日期:2021-03-20 出版日期:2021-12-15 发布日期:2021-11-26
  • 通讯作者: 高大启(gaodaqi@ecust.edu.cn)
  • 作者简介:2271987116@qq.com
  • 基金资助:
    国家自然科学基金(61806078,62076094,61976091);上海市教育发展基金会和上海市教育委员会“曙光计划”(61725301);国家重大新药开发科技专项(2019ZX09201004);上海市科技计划项目(20511100600)

EEG Emotion Recognition Based on Frequency and Channel Convolutional Attention

CHAI Bing1,2, LI Dong-dong1,2, WANG Zhe1, GAO Da-qi1   

  1. 1 School of Information Science and Engineering,East China University of Science and Technology,Shanghai 200237,China
    2 Provincial Key Laboratory of Computer Information Processing Technology,Soochow University,Suzhou,Jiangsu 215006,China
  • Received:2020-10-24 Revised:2021-03-20 Online:2021-12-15 Published:2021-11-26
  • About author:CHAI Bing,born in 1996,postgraduate,is a member of China Computer Federation.Her main research interests include deep learning and emotion recognition.
    GAO Da-qi,born in 1957,Ph.D,professor.His main research interests include machine learning and pattern recognition.
  • Supported by:
    National Natural Science Foundation of China(61806078,62076094,61976091),“Shuguang Program” Supported by Shanghai Education Development Foundation and Shanghai Municipal Education Commission(61725301),National Major Scientific and Technological Special Project(2019ZX09201004) and Shanghai Science and Technology Program(20511100600).

摘要: 现有的脑电(EEG)情感识别研究普遍采用神经网络和单一注意机制来学习情感特征,具有相对单一的特征表示。而神经科学研究表明,不同频率和电极通道的脑电信号对情感有不同的响应程度,因此文中提出了一种融合频率和电极通道卷积注意的方法,用于脑电情感识别。具体来说,首先将EEG信号分解到不同的频带上并提取相应的帧级特征,然后用预激活残差网络来学习深层次的脑电情感相关特征,同时在残差网络的每个预激活残差单元中都融入频率和电极通道卷积注意模块,以建模脑电信号的频率和电极通道信息,并生成脑电特征的最终注意表示。在DEAP和DREAMER数据集上的独立于受试者场景下的实验结果表明,所提出的卷积注意方法相比单一注意机制更有助于增强EEG信号中情感显著信息的导入,并且能产生更好的情感识别结果。

关键词: 残差网络, 脑电情感识别, 频率和电极通道卷积注意, 特征表示, 预激活残差单元

Abstract: The existing emotion recognition researches generally use neural network and attention mechanism to learn emotional features,which have relatively single feature representation.Moreover,neuroscience studies have shown that EEG signals of different frequencies and channels have different responses to emotion.Therefore,this paper proposes a method of fusing frequency and electrode channel convolutional attention for EEG emotion recognition.Specifically,EEG signals are firstly decomposed into different frequency bands and the corresponding frame-level features are extracted.Then the pre-activated residual network is employed to learn deep emotion-relevant features.At the same time,the frequency and electrode channel convolutional attention module is integrated into each pre-activated residual unit of residual network to model the frequency and channel information of EEG signals,thus generating final representation of EEG features.Experiments on DEAP and DREAMER datasets show that the proposed method helps to enhance the importing of emotion-salient information in EEG signals when compared with single-layer attention mechanism,and generates better recognition performance.

Key words: EEG emotion recognition, Feature representation, Frequency and electrode channel convolutional attention, Pre-activated residual unit, Residual network

中图分类号: 

  • TP301
[1]JAIN D,SHAMSOLMOALI P,SEHDEV P.Extended deep neural network for facial emotion recognition[J].Pattern Recognition Letters,2019,120:69-74.
[2]LIU Z,WU M,CAO W,et al.Speech emotion recognition based on feature selection and extreme learning machine decision tree[J].Neurocomputing,2018,273:271-280.
[3]GU Y,CHEN S,MARSIC I.Deep multimodal learning for emotion recognition in spoken language[C]//IEEE International Conference on Acoustics,Speech and Signal Processing.Calgary:IEEE Press,2018:5079-5083.
[4]ALARCAO S,FONSECA M.Emotions recognition using EEG signals:A survey[J].IEEE Transactions on Affective Computing,2019,10(3):374-393.
[5]HAMADA M,ZAIDAN B,ZAIDAN A.A systematic review for human EEG brain signals based emotion classification,feature extraction,brain condition,group comparison[J].Journal of Medical Systems,2018,42(9):1-25.
[6]KNYAZEV G G.EEG delta oscillations as a correlate of basic homeostatic and motivational processes[J].Neuroscience & Biobehavioral Reviews,2012,36(1):677-695.
[7]SANDE M C,KNUT E,PETER H,et al.EEG theta power is an early marker of cognitive decline in dementia due to Alzheimer's disease[J].Journal of Alzheimer's Disease,2018,64(4):1359-1371.
[8]FINK A,ROMINGER C,BENEDEK M,et al.EEG alpha activity during imagining creative moves in soccer decision-making situations[J].Neuropsychologia,2018,114:118-124.
[9]GOLA M,MAGNUSKI M,SZUMSKA I,et al.EEG beta band activity is related to attention and attentional deficits in the visualperformance of elderly subjects[J].International Journal of Psychophysiology,2013,89(3):334-341.
[10]MAFFEI A,SPIRONELLI C,ANGRILLI A.Affective and cor- tical EEG gamma responses to emotional movies in women with high vs low traits of empathy[J].Neuropsychologia,2019,133:107175.
[11]HADJIDIMITRIOU S,HADJILEONTIADIS L.EEG-based classification of music appraisal responses using time-frequency analysis and familiarity ratings[J].IEEE Transactions on Affective Computing,2013,4(2):161-172.
[12]MOHAMMADI Z,FROUNCHI J,AMIRI M.Wavelet-based emotion recognition system using EEG signal[J].Neural Computing and Applications,2017,28(8):1985-1990.
[13]PIHO L,TJAHJADI T.A mutual information based adaptive windowing of informative EEG for emotion recognition[J].IEEE Transactions on Affective Computing,2020,11(4):722-735.
[14]TRIPATHI S,ACHARYA S,SHARMA R,et al.Using deep and convolutional neural networks for accurate emotion classification on DEAP dataset[C]//Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence.IEEE Press,2017:4746-4752.
[15]LI Y,ZHENG W,ZONG Y,et al.A bi-hemisphere domain adversarial neural network model for EEG emotion recognition[J].IEEE Transactions on Affective Computing,2021,12(2):494-504.
[16]SONG T,ZHENG W,SONG P,et al.EEG emotion recognition using dynamical graph convolutional neural networks[J].IEEE Transactions on Affective Computing,2020,11(3):532-541.
[17]PANDEY P,SEEJA K.Subject independent emotion recognition from EEG using VMD and deep learning[J].Journal of King Saud University-Computer and Information Sciences,2019,https://doi.org/10.1016/j.jksuci.2019.11.003.
[18]SHARMA R,PACHORI R,SIRCAR P.Automated emotion recognition based on higher order statistics and deep learning algorithm[J].Biomedical Signal Processing and Control,2020,58:101867.
[19]ZHANG D,YAO L,CHEN K,et al.A convolutional recurrent attention model for subject-independent EEG signal analysis[J].IEEE Signal Processing Letters,2019,26(5):715-719.
[20]CHEN J,JIANG D,ZHANG Y.A hierarchical bidirectional GRU model with attention for EEG-based emotion classification[J].IEEE Access,2019,7:118530-118540.
[21]QIU J,LI X,HU K.Correlated attention networks for multimodal emotion recognition[C]//IEEE International Conference on Bioinformatics and Biomedicine.IEEE Press,2018:2656-2660.
[22]DUAN R,ZHU J,LU B.Differential entropy feature for EEG-based emotion classification[C]//2013 6th International IEEE/EMBS Conference on Neural Engineering(NER).IEEE Press,2013:81-84.
[23]WOO S,PARK J,LEE J.CBAM:Convolutional block attention module[C]//Proceedings of the European Conference on Computer Vision(ECCV).2018.
[24]HE K,ZHANG X,REN S,et al.Identity mappings in deep residual networks[C]//European Conference on Computer Vision.Cham:Springer,2016:630-645.
[25]KOELSTRA S,MUHL C,SOLEYMANI M,et al.DEAP:A database for emotion analysis using physiological signals[J].IEEE Transactions on Affective Computing,2011,3(1):18-31.
[26]KATSIGIANNIS S,RAMZAN N.Dreamer:A database for emotion recognition through EEG and ECG signals from wire-less low-cost off-the-shelf devices[J].IEEE Journal of Biome-dical and Health Informatics,2017,22(1):98-107.
[27]ROZGIC V,VITALADEVUNI S,PRASAD R.Robust EEG emotion classification using segment leveldecision fusion[C]//IEEE International Conference on Acoustics,Speech and Signal Processing.Vancouver:IEEE Press,2013:1286-1290.
[1] 王馨彤, 王璇, 孙知信.
基于多尺度记忆残差网络的网络流量异常检测模型
Network Traffic Anomaly Detection Method Based on Multi-scale Memory Residual Network
计算机科学, 2022, 49(8): 314-322. https://doi.org/10.11896/jsjkx.220200011
[2] 高荣华, 白强, 王荣, 吴华瑞, 孙想.
改进注意力机制的多叉树网络多作物早期病害识别方法
Multi-tree Network Multi-crop Early Disease Recognition Method Based on Improved Attention Mechanism
计算机科学, 2022, 49(6A): 363-369. https://doi.org/10.11896/jsjkx.210500044
[3] 赵人行, 徐频捷, 刘瑶.
基于深度卷积残差网络的心电单导联房颤检测方法
ECG-based Atrial Fibrillation Detection Based on Deep Convolutional Residual Neural Network
计算机科学, 2022, 49(5): 186-193. https://doi.org/10.11896/jsjkx.220200002
[4] 韩红旗, 冉亚鑫, 张运良, 桂婕, 高雄, 易梦琳.
基于共同子空间分类学习的跨媒体检索研究
Study on Cross-media Information Retrieval Based on Common Subspace Classification Learning
计算机科学, 2022, 49(5): 33-42. https://doi.org/10.11896/jsjkx.210200157
[5] 高心悦, 田汉民.
基于改进U-Net网络的液滴分割方法
Droplet Segmentation Method Based on Improved U-Net Network
计算机科学, 2022, 49(4): 227-232. https://doi.org/10.11896/jsjkx.210300193
[6] 张红民, 李萍萍, 房晓冰, 刘宏.
改进YOLOv3网络模型的人体异常行为检测方法
Human Abnormal Behavior Detection Method Based on Improved YOLOv3 Network Model
计算机科学, 2022, 49(4): 233-238. https://doi.org/10.11896/jsjkx.210300251
[7] 韩洁, 陈俊芬, 李艳, 湛泽聪.
基于自注意力的自监督深度聚类算法
Self-supervised Deep Clustering Algorithm Based on Self-attention
计算机科学, 2022, 49(3): 134-143. https://doi.org/10.11896/jsjkx.210100001
[8] 瞿中, 陈雯.
基于空洞卷积和多特征融合的混凝土路面裂缝检测
Concrete Pavement Crack Detection Based on Dilated Convolution and Multi-features Fusion
计算机科学, 2022, 49(3): 192-196. https://doi.org/10.11896/jsjkx.210100164
[9] 郭琳, 李晨, 陈晨, 赵睿, 范仕霖, 徐星雨.
基于通道注意递归残差网络的图像超分辨率重建
Image Super-resolution Reconstruction Using Recursive ResidualNetwork Based on ChannelAttention
计算机科学, 2021, 48(8): 139-144. https://doi.org/10.11896/jsjkx.200500150
[10] 许华杰, 张晨强, 苏国韶.
基于深层卷积残差网络的航拍图建筑物精确分割方法
Accurate Segmentation Method of Aerial Photography Buildings Based on Deep Convolutional Residual Network
计算机科学, 2021, 48(8): 169-174. https://doi.org/10.11896/jsjkx.200500096
[11] 暴雨轩, 芦天亮, 杜彦辉, 石达.
基于i_ResNet34模型和数据增强的深度伪造视频检测方法
Deepfake Videos Detection Method Based on i_ResNet34 Model and Data Augmentation
计算机科学, 2021, 48(7): 77-85. https://doi.org/10.11896/jsjkx.210300258
[12] 王建明, 黎向锋, 叶磊, 左敦稳, 张丽萍.
基于信道注意结构的生成对抗网络医学图像去模糊
Medical Image Deblur Using Generative Adversarial Networks with Channel Attention
计算机科学, 2021, 48(6A): 101-106. https://doi.org/10.11896/jsjkx.200600144
[13] 牛康力, 谌雨章, 张龚平, 谭前程, 王绎冲, 罗美琪.
基于深度学习的无人机航拍车流量监测
Vehicle Flow Measuring of UVA Based on Deep Learning
计算机科学, 2021, 48(6A): 275-280. https://doi.org/10.11896/jsjkx.200900149
[14] 龚航, 刘培顺.
夜间行驶车辆远光灯检测方法
Detection Method of High Beam in Night Driving Vehicle
计算机科学, 2021, 48(12): 256-263. https://doi.org/10.11896/jsjkx.200700026
[15] 杨月麟, 毕宗泽.
基于深度学习的网络流量异常检测
Network Anomaly Detection Based on Deep Learning
计算机科学, 2021, 48(11A): 540-546. https://doi.org/10.11896/jsjkx.201200077
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!