基于双流结构缩放和多重注意力机制的轻量级脑电情感识别方法

doi:10.11896/jsjkx.220300262

Abstract

Abstract: EEG emotion recognition is a multi-channel time-series signal classification problem with high complexity,high information density and massive data.In order to achieve optimal accuracy and performance of EEG emotion recognition with fewer computational parameters while maintaining the existing classification accuracy,this paper proposes a lightweight network(LDM-EEG) based on dual-stream structural scaling and multiple attention mechanisms.The network takes the time-space and frequency-space maps constructed based on the differential entropy features of EEG signals as the input,processes the two features separately using a symmetric dual-stream structure,achieves lightweighting through a novel parameter-saving residual module and a network scaling mechanism,and enhances the model feature aggregation capability using a novel channel-time/frequency-space multiple attention mechanism and a post-attention mechanism.Experimental results show that the accuracy of the model is 95.18% with significantly reduced number of parameters,which achieves the optimal result in the domain.Further,about 98% reduction in the number of parameters has been achieved with slightly lower accuracy than the existing models.

Key words: EEG emotion recognition, Time-frequency dual streaming, Multiple attention, Lightweight, Structural scaling, Computational affection

CLC Number:

TP183

LEI Ying, LIU Feng. LDM-EEG:A Lightweight EEG Emotion Recognition Method Based on Dual-stream Structure Scaling and Multiple Attention Mechanisms[J].Computer Science, 2023, 50(6A): 220300262-9.

References

[1]PICARD R W.Affective Computing for HCI[C]//Proceedings of the HCI International 1999-Proceedings of the 8th International Conference on Human-Computer Interaction.Munich,Germany,1999.
[2]ZHANG J,LIU F,ZHOU A.Off-TANet:a Lightweight Neural Micro-Expression Recognizer with Optical Flow Features and Integrated Attention Mechanism[C]//Pacific Rim International Conference on Artificial Intelligence.Cham:Springer,2021:266-279.
[3]LIU F,WANG H Y,SHEN S Y,et al.OPO-FCM:A Computational Affection Based OCC-PAD-OCEAN Federation Cognitive Modeling Approach[J].IEEE Transactions on Computational Social Systems,2022.
[4]FU Z,LIU F,ZHANG J,et al.SAGN:Semantic Adaptive Graph Network for Skeleton-Based Human Action Recognition[C]//Proceedings of the 2021 International Conference on Multimedia Retrieval.2021:110-117.
[5]ZHENG W L,LU B L.Investigating critical frequency bands and channels for EEG-based emotion recognition with deep neural networks[J].IEEE Transactions on Autonomous Mental Development,2015,7(3):162-175.
[6]KRIZHEVSKY A, SUTSKEVER I, HINTON G E.Imagenet classification with deep convolutional neural networks[J].Advances in Neural Information Processing Systems,2012,25.
[7]SONG T, ZHENG W, SONG P,et al.EEG emotion recognition using dynamical graph convolutional neural networks[J].IEEE Transactions on Affective Computing,2018,11(3):532-541.
[8]ZHONG P,WANG D,MIAO C.EEG-based emotion recognition using regularized graph neural networks[J].arXiv:1907.07835,2019.
[9]JIA Z,LIN Y,CAI X,et al.Sst-emotionnet:Spatial-spectral-temporal based attention 3d dense network for eeg emotion recognition[C]//Proceedings of the 28th ACM International Conference on Multimedia.2020:2909-2917.
[10]FRANTZIDIS C A,BRATSAS C,PAPADELIS C L,et al.Toward emotion aware computing:an integrated approach using multichannel neurophysiological recordings and affective visual stimuli[J].IEEE Transactions on Information Technology in Biomedicine,2010,14(3):589-597.
[11]HJORTH B.EEG analysis based on time domain properties[J].Electroencephalography and Clinical Neurophysiology,1970,29(3):306-310.
[12]KROUPI E,YAZDANI A,EBRAHIMI T.EEG correlates of different emotional states elicited during watching music videos[C]//International Conference on Affective Computing and Intelligent Interaction.Berlin:Springer,2011:457-466.
[13]WANG X W,NIE D,LU B L.EEG-based emotion recognition using frequency domain features and support vector machines[C]//International Conference on Neural Information Proces-sing.Berlin:Springer,2011:734-743.
[14]HOSSEINI S A,KHALILZADEH M A,NAGHIBI-SISTANIM B,et al.Higher order spectra analysis of EEG signals in emotional stress states[C]//2010 Second International Conference on Information Technology and Computer Science.IEEE,2010:60-63.
[15]DUAN R N,ZHU J Y,LU B L.Differential entropy feature for EEG-based emotion classification[C]//2013 6th International IEEE/EMBS Conference on Neural Engineering(NER).IEEE,2013:81-84.
[16]ZHENG W L,ZHU J Y,PENG Y,et al.EEG-based emotionclassification using deep belief networks[C]//2014 IEEE International Conference on Multimedia and EXPO(ICME).IEEE,2014:1-6.
[17]SIDDHARTH S,JUNG T P,SEJNOWSKI T J.Utilizing deep learning towards multi-modal bio-sensing and vision-based affective computing[J].arXiv:1905.07039,2019.
[18]BAHARI F,JANGHORBANI A.Eeg-based emotion recogni-tion using recurrence plot analysis and k nearest neighbor classifier[C]//2013 20th Iranian Conference on Biomedical Enginee-ring(ICBME).IEEE,2013:228-233.
[19]WANG X W,NIE D,LU B L.EEG-based emotion recognition using frequency domain features and support vector machines[C]//International Conference on Neural Information Proces-sing.Berlin:Springer,2011:734-743.
[20]ZEILER M D,FERGUS R.Visualizing and understanding con-volutional networks[C]//European Conference on Computer Vision.Cham:Springer,2014:818-833.
[21]WU Z,PAN S,CHEN F,et al.A comprehensive survey ongraph neural networks[J].IEEE Transactions on Neural Networks and Learning Systems,2020,32(1):4-24.
[22]GRAVES A,MOHAMED A R,HINTON G.Speech recognition with deep recurrent neural networks[C]//1988 International Conference on Acoustics,Speech,and Signal Processing(ICASSP-88).IEEE,2013.
[23]LI Y,ZHENG W,WANG L,et al.From regional to globalbrain:A novel hierarchical spatial-temporal neural network model for EEG emotion recognition[J].IEEE Transactions on Affective Computing,2022,13(2):568-578.
[24]SOOD E, TANNERT S, MÜLLER P,et al.Improving natural language processing tasks with human gaze-guided neural attention[J].Advances in Neural Information Processing Systems,2020,33:6327-6341.
[25]SUN Y,FISHER R.Object-based visual attention for computer vision[J].Artificial intelligence,2003,146(1):77-123.
[26]ZHOU P,YANG W,CHEN W,et al.Modality attention forend-to-end audio-visual speech recognition[C]//2019 IEEE International Conference on Acoustics,Speech and Signal Proces-sing(ICASSP 2019).IEEE,2019:6565-6569.
[27]PANG Y,XIE J,KHAN M H,et al.Mask-guided attention network for occluded pedestrian detection[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2019:4967-4975.
[28]HU J,SHEN L,SUN G.Squeeze-and-excitation networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:7132-7141.
[29]WOO S,PARK J,LEE J Y,et al.Cbam:Convolutional block attention module[C]//Proceedings of the European Conference on Computer Vision(ECCV).2018:3-19.
[30]LI X,WANG W,HU X,et al.Selective kernel networks[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2019:510-519.
[31]LIN Z,ZHANG Z,CHEN L Z,et al.Interactive image segmentation with first click attention[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2020:13339-13348.
[32]CUI C,GAO T,WEI S,et al.PP-LCNet:A Lightweight CPU Convolutional Neural Network[J].arXiv:2109.15099,2021.
[33]SRINIVAS A,LIN T Y,PARMAR N,et al.Bottleneck transformers for visual recognition[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2021:16519-16529.
[34]TAO W,LI C,SONG R,et al.EEG-based emotion recognition via channel-wise attention and self attention[J].IEEE Transactions on Affective Computing,2023,14(1):382-293.
[35]CHAI B,LI D D,WANG Z,et al.EEG Emotion Recognition Based on Frequency and Convolutional Attention[J].Computer Science,2021,48(12):312-318.
[36]ZHU M,GUPTA S.To prune,or not to prune:exploring the efficacy of pruning for model compression[J].arXiv:1710.01878,2017.
[37]HINTON G,VINYALS O,DEAN J.Distilling the knowledge in a neural network[J].arXiv:1503.02531,2015.
[38]HOWARD A G,ZHU M,CHEN B,et al.Mobilenets:Efficient convolutional neural networks for mobile vision applications[J].arXiv:1704.04861,2017.
[39]SANDLER M,HOWARD A,ZHU M,et al.Mobilenetv2:Inverted residuals and linear bottlenecks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:4510-4520.
[40]HOWARD A,SANDLER M,CHU G,et al.Searching for mobilenetv3[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2019:1314-1324.
[41]ZHANG X,ZHOU X,LIN M,et al.Shufflenet:An extremely efficient convolutional neural network for mobile devices[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:6848-6856.
[42]MA N, ZHANG X, ZHENG H T,et al.Shufflenet v2:Practical guidelines for efficient cnn architecture design[C]//Proceedings of the European Conference on Computer Vision(ECCV).2018:116-131.
[43]ELSKEN T,METZEN J H,HUTTER F.Neural architecture search:A survey[J].The Journal of Machine Learning Research,2019,20(1):1997-2017.
[44]CAI H,CHEN T,ZHANG W,et al.Efficient architecturesearch by network transformation[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2018.
[45]LIU H,SIMONYAN K,YANG Y.Darts:Differentiable architecture search[J].arXiv:1806.09055,2018.
[46]TAN M,LE Q.Efficientnet:Rethinking model scaling for convolutional neural networks[C]//International Conference on Machine Learning.PMLR,2019:6105-6114.
[47]XIE S,GIRSHICK R,DOLLÁR P,et al.Aggregated residual transformations for deep neural networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017:1492-1500.
[48]KNYAZEV G G.EEG delta oscillations as a correlate of basic homeostatic and motivational processes[J].Neuroscience & Biobehavioral Reviews,2012,36(1):677-695.
[49]MUSAEUS C S,ENGEDAL K,HØGH P,et al.EEG thetapower is an early marker of cognitive decline in dementia due to Alzheimer’s disease[J].Journal of Alzheimer’s Disease,2018,64(4):1359-1371.
[50]FINK A,ROMINGER C,BENEDEK M,et al.EEG alpha activity during imagining creative moves in soccer decision-making situations[J].Neuropsychologia,2018,114:118-124.
[51]GOLA M,MAGNUSKI M,SZUMSKA I,et al.EEG beta band activity is related to attention and attentional deficits in the visualperformance of elderly subjects[J].International Journal of Psychophysiology,2013,89(3):334-341.
[52]MAFFEI A,SPIRONELLI C,ANGRILLI A.Affective and cortical EEG gamma responses to emotional movies in women with high vs low traits of empathy[J].Neuropsychologia,2019,133:107175.
[53]HE K,ZHANG X,REN S,et al.Deep residual learning forimage recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:770-778.
[54]YE R,LIU F,ZHANG L.3D depthwise convolution:reducing model parameters in 3D vision tasks[C]//Canadian Conference on Artificial Intelligence.Cham:Springer,2019:186-199.
[55]SUYKENS J A K,VANDEWALLE J.Least squares support vector machine classifiers[J].Neural Processing Letters,1999,9(3):293-300.
[56]ZHENG W.Multichannel EEG-based emotion recognition viagroup sparse canonical correlation analysis[J].IEEE Transactions on Cognitive and Developmental Systems,2016,9(3):281-290.
[57]LI Y,ZHENG W,CUI Z,et al.A Novel Neural Network Model based on Cerebral Hemispheric Asymmetry for EEG Emotion Recognition[C]//IJCAI.2018:1561-1567.
[58]LI Y,WANG L,ZHENG W,et al.A novel bi-hemispheric discrepancy model for eeg emotion recognition[J].IEEE Transactions on Cognitive and Developmental Systems,2020,13(2):354-367.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

Comments

Recommended 0

No Suggested Reading articles found!

LDM-EEG:A Lightweight EEG Emotion Recognition Method Based on Dual-stream Structure Scaling and Multiple Attention Mechanisms

PDF (PC)

Abstract

Cite this article

share this article

References

Related Articles 15

Metrics

Comments

Recommended 0

[1]	ZHANG Xiao, DONG Hongbin. Lightweight Multi-view Stereo Integrating Coarse Cost Volume and Bilateral Grid [J]. Computer Science, 2023, 50(8): 125-132.
[2]	DOU Zhi, HU Chenguang, LIANG Jingyi, ZHENG Liming, LIU Guoqi. Lightweight Target Detection Algorithm Based on Improved Yolov4-tiny [J]. Computer Science, 2023, 50(6A): 220700006-7.
[3]	LIANG Weiliang, LI Yue, WANG Pengfei. Lightweight Face Generation Method Based on TransEditor and Its Application Specification [J]. Computer Science, 2023, 50(2): 221-230.
[4]	HAO Qiang, LI Jie, ZHANG Man, WANG Lu. Spatial Non-cooperative Target Components Recognition Algorithm Based on Improved YOLOv3 [J]. Computer Science, 2022, 49(6A): 358-362.
[5]	ZHANG Jia-hao, LIU Feng, QI Jia-yin. Lightweight Micro-expression Recognition Architecture Based on Bottleneck Transformer [J]. Computer Science, 2022, 49(6A): 370-377.
[6]	SUN Chang-di, PAN Zhi-song, ZHANG Yan-yan. Re-lightweight Method of MobileNet Based on Low-cost Deformable Convolution [J]. Computer Science, 2022, 49(12): 312-318.
[7]	HE Peng-hao, YU Ying, XU Chao-yue. Image Super-resolution Reconstruction Network Based on Dynamic Pyramid and Subspace Attention [J]. Computer Science, 2022, 49(11A): 210900202-8.
[8]	Abudukelimu ABULIZI, ZHANG Yu-ning, Alimujiang YASEN, GUO Wen-qiang, Abudukelimu HALIDANMU. Survey of Research on Extended Models of Pre-trained Language Models [J]. Computer Science, 2022, 49(11A): 210800125-12.
[9]	GONG Hao-tian, ZHANG Meng. Lightweight Anchor-free Object Detection Algorithm Based on Keypoint Detection [J]. Computer Science, 2021, 48(8): 106-110.
[10]	LI Shan, XU Xin-zheng. Parallel Pruning from Two Aspects for VGG16 Optimization [J]. Computer Science, 2021, 48(6): 227-233.
[11]	PAN Ming-yuan, SONG Hui-hui, ZHANG Kai-hua, LIU Qing-shan. Learning Global Guided Progressive Feature Aggregation Lightweight Network for Salient Object Detection [J]. Computer Science, 2021, 48(6): 103-109.
[12]	CHAI Bing, LI Dong-dong, WANG Zhe, GAO Da-qi. EEG Emotion Recognition Based on Frequency and Channel Convolutional Attention [J]. Computer Science, 2021, 48(12): 312-318.
[13]	LIU Yan, QIN Pin-le, ZENG Jian-chao. Multi-object Tracking Algorithm Based on YOLOv3 and Hierarchical Data Association [J]. Computer Science, 2021, 48(11A): 370-375.
[14]	CHEN Hao-nan, LEI Yin-jie, WANG Hao. Lightweight Lane Detection Model Based on Row-column Decoupled Sampling [J]. Computer Science, 2021, 48(11A): 416-419.
[15]	WANG Rui-jin, TANG Yu-cheng, PEI Xi-kai, GUO Shang-tong, ZHANG Feng-li. Block-chain Privacy Protection Scheme Based on Lightweight Homomorphic Encryption and Zero-knowledge Proof [J]. Computer Science, 2021, 48(11A): 547-551.