计算机科学

计算机科学 ›› 2023, Vol. 50 ›› Issue (6A): 220300262-9.doi: 10.11896/jsjkx.220300262

• 人工智能 • 上一篇    下一篇

基于双流结构缩放和多重注意力机制的轻量级脑电情感识别方法

雷颖1,3, 刘峰1,2,3,4   

  1. 1 华东师范大学计算机科学与技术学院 上海 200062;
    2 上海对外经贸大学人工智能与变革管理研究院 上海 201620;
    3 华东师范大学上海智能教育研究院 上海 200062;
    4 华东师范大学心理与认知科学学院上海市心理健康与危机干预重点实验室 上海 200062
  • 出版日期:2023-06-10 发布日期:2023-06-12
  • 通讯作者: 刘峰(lsttoy@163.com)
  • 作者简介:(lycherishcs@163.com)
  • 基金资助:
    上海市科技计划项目;中央高校基本科研业务费专项资金(20DZ2260300)

LDM-EEG:A Lightweight EEG Emotion Recognition Method Based on Dual-stream Structure Scaling and Multiple Attention Mechanisms

LEI Ying1,3, LIU Feng1,2,3,4   

  1. 1 School of Computer Science and Technology,East China Normal University,Shanghai 200062,China;
    2 Institute of Artificial Intelligence and Change Management,Shanghai University of International Business and Economics,Shanghai 201620,China;
    3 Institute of AI for Education,East China Normal University,Shanghai 200062,China;
    4 Shanghai Key Laboratory of Mental Health and Psychological Crisis Intervention,School of Psychology and Cognitive Science,East China Normal University,Shanghai 200062,China
  • Online:2023-06-10 Published:2023-06-12
  • About author:LEI Ying,born in 2000,is a student member of China Computer Federation.Her main research interests include affective computing,EEG emotion recognition and deep learning. LIU Feng,born in 1988.Ph.D candidate,engineer,is a senior member of China Computer Federation.His main research interests include deep lear-ning,computational affection and blockchain technology.
  • Supported by:
    Research Project of Shanghai Science and Technology Commission and Fundamental Research Funds for the Central Universities of Ministry of Education of China(20DZ2260300).

PDF (PC)

324

摘要： 脑电情感识别是一个复杂程度高、信息密度大、海量数据的多通道时序信号分类问题。为在保持现有分类精度的情况下减少计算参数量,实现脑电情感识别的精度与性能最优,提出了一种基于双流结构缩放和多重注意力机制的轻量级网络(LDM-EEG) 。该网络以基于脑电信号的微分熵特征构造的时域-空域图谱和频域-空域图谱作为输入,采用对称的双流结构对上述两种特征分别处理,通过节约参数的新型残差模块和网络缩放机制来实现轻量化,并利用新型的通道-时/频-空多重注意力机制和后注意力机制提升模型特征聚合能力。实验结果表明,在参数量明显减小的情况下,模型在SEED数据集上实现了95.18%的准确率,达到了领域的最优结果。进一步地,在略低于现有模型准确率的基础上,其将参数量缩减了98％。

关键词: 脑电情感识别, 时频双流, 多重注意力, 轻量级, 结构缩放, 可计算情感

Abstract: EEG emotion recognition is a multi-channel time-series signal classification problem with high complexity,high information density and massive data.In order to achieve optimal accuracy and performance of EEG emotion recognition with fewer computational parameters while maintaining the existing classification accuracy,this paper proposes a lightweight network(LDM-EEG) based on dual-stream structural scaling and multiple attention mechanisms.The network takes the time-space and frequency-space maps constructed based on the differential entropy features of EEG signals as the input,processes the two features separately using a symmetric dual-stream structure,achieves lightweighting through a novel parameter-saving residual module and a network scaling mechanism,and enhances the model feature aggregation capability using a novel channel-time/frequency-space multiple attention mechanism and a post-attention mechanism.Experimental results show that the accuracy of the model is 95.18% with significantly reduced number of parameters,which achieves the optimal result in the domain.Further,about 98% reduction in the number of parameters has been achieved with slightly lower accuracy than the existing models.

Key words: EEG emotion recognition, Time-frequency dual streaming, Multiple attention, Lightweight, Structural scaling, Computational affection

中图分类号: 

  • TP183
[1]PICARD R W.Affective Computing for HCI[C]//Proceedings of the HCI International 1999-Proceedings of the 8th International Conference on Human-Computer Interaction.Munich,Germany,1999.
[2]ZHANG J,LIU F,ZHOU A.Off-TANet:a Lightweight Neural Micro-Expression Recognizer with Optical Flow Features and Integrated Attention Mechanism[C]//Pacific Rim International Conference on Artificial Intelligence.Cham:Springer,2021:266-279.
[3]LIU F,WANG H Y,SHEN S Y,et al.OPO-FCM:A Computational Affection Based OCC-PAD-OCEAN Federation Cognitive Modeling Approach[J].IEEE Transactions on Computational Social Systems,2022.
[4]FU Z,LIU F,ZHANG J,et al.SAGN:Semantic Adaptive Graph Network for Skeleton-Based Human Action Recognition[C]//Proceedings of the 2021 International Conference on Multimedia Retrieval.2021:110-117.
[5]ZHENG W L,LU B L.Investigating critical frequency bands and channels for EEG-based emotion recognition with deep neural networks[J].IEEE Transactions on Autonomous Mental Development,2015,7(3):162-175.
[6]KRIZHEVSKY A, SUTSKEVER I, HINTON G E.Imagenet classification with deep convolutional neural networks[J].Advances in Neural Information Processing Systems,2012,25.
[7]SONG T, ZHENG W, SONG P,et al.EEG emotion recognition using dynamical graph convolutional neural networks[J].IEEE Transactions on Affective Computing,2018,11(3):532-541.
[8]ZHONG P,WANG D,MIAO C.EEG-based emotion recognition using regularized graph neural networks[J].arXiv:1907.07835,2019.
[9]JIA Z,LIN Y,CAI X,et al.Sst-emotionnet:Spatial-spectral-temporal based attention 3d dense network for eeg emotion recognition[C]//Proceedings of the 28th ACM International Conference on Multimedia.2020:2909-2917.
[10]FRANTZIDIS C A,BRATSAS C,PAPADELIS C L,et al.Toward emotion aware computing:an integrated approach using multichannel neurophysiological recordings and affective visual stimuli[J].IEEE Transactions on Information Technology in Biomedicine,2010,14(3):589-597.
[11]HJORTH B.EEG analysis based on time domain properties[J].Electroencephalography and Clinical Neurophysiology,1970,29(3):306-310.
[12]KROUPI E,YAZDANI A,EBRAHIMI T.EEG correlates of different emotional states elicited during watching music videos[C]//International Conference on Affective Computing and Intelligent Interaction.Berlin:Springer,2011:457-466.
[13]WANG X W,NIE D,LU B L.EEG-based emotion recognition using frequency domain features and support vector machines[C]//International Conference on Neural Information Proces-sing.Berlin:Springer,2011:734-743.
[14]HOSSEINI S A,KHALILZADEH M A,NAGHIBI-SISTANIM B,et al.Higher order spectra analysis of EEG signals in emotional stress states[C]//2010 Second International Conference on Information Technology and Computer Science.IEEE,2010:60-63.
[15]DUAN R N,ZHU J Y,LU B L.Differential entropy feature for EEG-based emotion classification[C]//2013 6th International IEEE/EMBS Conference on Neural Engineering(NER).IEEE,2013:81-84.
[16]ZHENG W L,ZHU J Y,PENG Y,et al.EEG-based emotionclassification using deep belief networks[C]//2014 IEEE International Conference on Multimedia and EXPO(ICME).IEEE,2014:1-6.
[17]SIDDHARTH S,JUNG T P,SEJNOWSKI T J.Utilizing deep learning towards multi-modal bio-sensing and vision-based affective computing[J].arXiv:1905.07039,2019.
[18]BAHARI F,JANGHORBANI A.Eeg-based emotion recogni-tion using recurrence plot analysis and k nearest neighbor classifier[C]//2013 20th Iranian Conference on Biomedical Enginee-ring(ICBME).IEEE,2013:228-233.
[19]WANG X W,NIE D,LU B L.EEG-based emotion recognition using frequency domain features and support vector machines[C]//International Conference on Neural Information Proces-sing.Berlin:Springer,2011:734-743.
[20]ZEILER M D,FERGUS R.Visualizing and understanding con-volutional networks[C]//European Conference on Computer Vision.Cham:Springer,2014:818-833.
[21]WU Z,PAN S,CHEN F,et al.A comprehensive survey ongraph neural networks[J].IEEE Transactions on Neural Networks and Learning Systems,2020,32(1):4-24.
[22]GRAVES A,MOHAMED A R,HINTON G.Speech recognition with deep recurrent neural networks[C]//1988 International Conference on Acoustics,Speech,and Signal Processing(ICASSP-88).IEEE,2013.
[23]LI Y,ZHENG W,WANG L,et al.From regional to globalbrain:A novel hierarchical spatial-temporal neural network model for EEG emotion recognition[J].IEEE Transactions on Affective Computing,2022,13(2):568-578.
[24]SOOD E, TANNERT S, MÜLLER P,et al.Improving natural language processing tasks with human gaze-guided neural attention[J].Advances in Neural Information Processing Systems,2020,33:6327-6341.
[25]SUN Y,FISHER R.Object-based visual attention for computer vision[J].Artificial intelligence,2003,146(1):77-123.
[26]ZHOU P,YANG W,CHEN W,et al.Modality attention forend-to-end audio-visual speech recognition[C]//2019 IEEE International Conference on Acoustics,Speech and Signal Proces-sing(ICASSP 2019).IEEE,2019:6565-6569.
[27]PANG Y,XIE J,KHAN M H,et al.Mask-guided attention network for occluded pedestrian detection[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2019:4967-4975.
[28]HU J,SHEN L,SUN G.Squeeze-and-excitation networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:7132-7141.
[29]WOO S,PARK J,LEE J Y,et al.Cbam:Convolutional block attention module[C]//Proceedings of the European Conference on Computer Vision(ECCV).2018:3-19.
[30]LI X,WANG W,HU X,et al.Selective kernel networks[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2019:510-519.
[31]LIN Z,ZHANG Z,CHEN L Z,et al.Interactive image segmentation with first click attention[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2020:13339-13348.
[32]CUI C,GAO T,WEI S,et al.PP-LCNet:A Lightweight CPU Convolutional Neural Network[J].arXiv:2109.15099,2021.
[33]SRINIVAS A,LIN T Y,PARMAR N,et al.Bottleneck transformers for visual recognition[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2021:16519-16529.
[34]TAO W,LI C,SONG R,et al.EEG-based emotion recognition via channel-wise attention and self attention[J].IEEE Transactions on Affective Computing,2023,14(1):382-293.
[35]CHAI B,LI D D,WANG Z,et al.EEG Emotion Recognition Based on Frequency and Convolutional Attention[J].Computer Science,2021,48(12):312-318.
[36]ZHU M,GUPTA S.To prune,or not to prune:exploring the efficacy of pruning for model compression[J].arXiv:1710.01878,2017.
[37]HINTON G,VINYALS O,DEAN J.Distilling the knowledge in a neural network[J].arXiv:1503.02531,2015.
[38]HOWARD A G,ZHU M,CHEN B,et al.Mobilenets:Efficient convolutional neural networks for mobile vision applications[J].arXiv:1704.04861,2017.
[39]SANDLER M,HOWARD A,ZHU M,et al.Mobilenetv2:Inverted residuals and linear bottlenecks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:4510-4520.
[40]HOWARD A,SANDLER M,CHU G,et al.Searching for mobilenetv3[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2019:1314-1324.
[41]ZHANG X,ZHOU X,LIN M,et al.Shufflenet:An extremely efficient convolutional neural network for mobile devices[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:6848-6856.
[42]MA N, ZHANG X, ZHENG H T,et al.Shufflenet v2:Practical guidelines for efficient cnn architecture design[C]//Proceedings of the European Conference on Computer Vision(ECCV).2018:116-131.
[43]ELSKEN T,METZEN J H,HUTTER F.Neural architecture search:A survey[J].The Journal of Machine Learning Research,2019,20(1):1997-2017.
[44]CAI H,CHEN T,ZHANG W,et al.Efficient architecturesearch by network transformation[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2018.
[45]LIU H,SIMONYAN K,YANG Y.Darts:Differentiable architecture search[J].arXiv:1806.09055,2018.
[46]TAN M,LE Q.Efficientnet:Rethinking model scaling for convolutional neural networks[C]//International Conference on Machine Learning.PMLR,2019:6105-6114.
[47]XIE S,GIRSHICK R,DOLLÁR P,et al.Aggregated residual transformations for deep neural networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017:1492-1500.
[48]KNYAZEV G G.EEG delta oscillations as a correlate of basic homeostatic and motivational processes[J].Neuroscience & Biobehavioral Reviews,2012,36(1):677-695.
[49]MUSAEUS C S,ENGEDAL K,HØGH P,et al.EEG thetapower is an early marker of cognitive decline in dementia due to Alzheimer’s disease[J].Journal of Alzheimer’s Disease,2018,64(4):1359-1371.
[50]FINK A,ROMINGER C,BENEDEK M,et al.EEG alpha activity during imagining creative moves in soccer decision-making situations[J].Neuropsychologia,2018,114:118-124.
[51]GOLA M,MAGNUSKI M,SZUMSKA I,et al.EEG beta band activity is related to attention and attentional deficits in the visualperformance of elderly subjects[J].International Journal of Psychophysiology,2013,89(3):334-341.
[52]MAFFEI A,SPIRONELLI C,ANGRILLI A.Affective and cortical EEG gamma responses to emotional movies in women with high vs low traits of empathy[J].Neuropsychologia,2019,133:107175.
[53]HE K,ZHANG X,REN S,et al.Deep residual learning forimage recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:770-778.
[54]YE R,LIU F,ZHANG L.3D depthwise convolution:reducing model parameters in 3D vision tasks[C]//Canadian Conference on Artificial Intelligence.Cham:Springer,2019:186-199.
[55]SUYKENS J A K,VANDEWALLE J.Least squares support vector machine classifiers[J].Neural Processing Letters,1999,9(3):293-300.
[56]ZHENG W.Multichannel EEG-based emotion recognition viagroup sparse canonical correlation analysis[J].IEEE Transactions on Cognitive and Developmental Systems,2016,9(3):281-290.
[57]LI Y,ZHENG W,CUI Z,et al.A Novel Neural Network Model based on Cerebral Hemispheric Asymmetry for EEG Emotion Recognition[C]//IJCAI.2018:1561-1567.
[58]LI Y,WANG L,ZHENG W,et al.A novel bi-hemispheric discrepancy model for eeg emotion recognition[J].IEEE Transactions on Cognitive and Developmental Systems,2020,13(2):354-367.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市渝北区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发